--Task-- name: DSPrimeTask_ds_idrepo_0 enabled: True class_name: DSSearchRateTask source_name: overseer-0 source_namespace: >default< target_name: ds-idrepo-0 target_namespace: >default< start: 0.0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: [] wait_for: [] preceding_task: None options: {'_options_': "--noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 1000000 --argument 'inc(0, 1000000)'"} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [2024-04-25 17:39:23] DSPrimeTask_ds_idrepo_0 pre : N/A ________________________________________________________________________________ [2024-04-25 17:39:23] - INFO: target component (ds-idrepo-0) is a DS pod [2024-04-25 17:39:23] - INFO: Timeout for this Task has changed to 59s (-1 seconds) [2024-04-25 17:39:23] - INFO: timeout was set to unlimited, because it was unset Pod log file : /results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt Pod log : /results/orders/DSPrimeTask_ds_idrepo_0 ________________________________________________________________________________ [2024-04-25 17:39:23] DSPrimeTask_ds_idrepo_0 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/clients/ds/bin/searchrate --maxDuration 1 --bindPassword "Te3hvGVKgyQ3Y4m0fbG16hKkzkr9FMrw" --bindDn "uid=admin" --trustAll --useStartTls --port 1389 --statInterval 5 --noPropertiesFile --hostname 'ds-idrepo-0.ds-idrepo' --noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 1000000 --argument 'inc(0, 1000000)' '(uid=user.{})' ________________________________________________________________________________ [2024-04-25 17:39:23] DSPrimeTask_ds_idrepo_0 create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file: /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/order.json { "name": "DSPrimeTask_ds_idrepo_0", "command": "/lodestar/clients/ds/bin/searchrate --maxDuration 1 --bindPassword **** --bindDn \"uid=admin\" --trustAll --useStartTls --port 1389 --statInterval 5 --noPropertiesFile --hostname 'ds-idrepo-0.ds-idrepo' --noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 1000000 --argument 'inc(0, 1000000)' '(uid=user.{})'", "out-dir": "/results/orders/DSPrimeTask_ds_idrepo_0" } ________________________________________________________________________________ [2024-04-25 17:39:23] DSPrimeTask_ds_idrepo_0 loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2024-04-25 17:39:23,798 INFO 2024-04-25 17:39:23,798 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2024-04-25 17:39:23,798 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2024-04-25 17:39:24,093 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-25 17:39:24,094 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:24,094 DEBUG --- stdout --- 2024-04-25 17:39:24,094 DEBUG WAITING 2024-04-25 17:39:24,094 DEBUG --- stderr --- 2024-04-25 17:39:24,094 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 27 0 --:--:-- --:--:-- --:--:-- 27 ________________________________________________________________________________ [2024-04-25 17:39:24] DSPrimeTask_ds_idrepo_0 start_order : Send the order to the overseer pod ________________________________________________________________________________ 2024-04-25 17:39:24,094 INFO 2024-04-25 17:39:24,094 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:24,095 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:24,399 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:24,399 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:24,399 DEBUG --- stdout --- 2024-04-25 17:39:24,399 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:24,399 DEBUG --- stderr --- 2024-04-25 17:39:24,399 DEBUG 2024-04-25 17:39:24,400 INFO 2024-04-25 17:39:24,400 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/order.json overseer-0-56c5c978d9-v8wm9:/results/orders/order.json 2024-04-25 17:39:24,400 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-25 17:39:25,791 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:25,791 DEBUG --- stdout --- 2024-04-25 17:39:25,791 DEBUG 2024-04-25 17:39:25,791 DEBUG --- stderr --- 2024-04-25 17:39:25,791 DEBUG Order has been successfully forwarded to overseer ________________________________________________________________________________ [2024-04-25 17:39:25] DSPrimeTask_ds_idrepo_0 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 59s Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 5s, retry Order is running since 10s, retry PASS : Order stopped by itself after 15s and before timeout (59s) ________________________________________________________________________________ [2024-04-25 17:39:41] DSPrimeTask_ds_idrepo_0 stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2024-04-25 17:39:41,471 INFO 2024-04-25 17:39:41,471 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:41,471 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:41,798 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:41,798 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:41,798 DEBUG --- stdout --- 2024-04-25 17:39:41,798 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:41,798 DEBUG --- stderr --- 2024-04-25 17:39:41,798 DEBUG 2024-04-25 17:39:41,798 INFO 2024-04-25 17:39:41,798 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-56c5c978d9-v8wm9 -- rm -f /results/orders/order.json 2024-04-25 17:39:41,798 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-25 17:39:42,603 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:42,603 DEBUG --- stdout --- 2024-04-25 17:39:42,603 DEBUG 2024-04-25 17:39:42,603 DEBUG --- stderr --- 2024-04-25 17:39:42,603 DEBUG Order ran during 16s ________________________________________________________________________________ [2024-04-25 17:39:42] DSPrimeTask_ds_idrepo_0 download_logs : Download individual order files ________________________________________________________________________________ [2024-04-25 17:39:42] - PASS: file already downloaded /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/order.json 2024-04-25 17:39:42,604 INFO 2024-04-25 17:39:42,604 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:42,604 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:42,907 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:42,908 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:42,908 DEBUG --- stdout --- 2024-04-25 17:39:42,908 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:42,908 DEBUG --- stderr --- 2024-04-25 17:39:42,908 DEBUG Checking pod: overseer-0-56c5c978d9-v8wm9 for file exists, if not wait. 2024-04-25 17:39:42,908 INFO 2024-04-25 17:39:42,908 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:42,908 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:43,235 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:43,236 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:43,236 DEBUG --- stdout --- 2024-04-25 17:39:43,236 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:43,236 DEBUG --- stderr --- 2024-04-25 17:39:43,236 DEBUG 2024-04-25 17:39:43,237 INFO 2024-04-25 17:39:43,237 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-56c5c978d9-v8wm9 --stdin -- bash -c 'if [ -f "/results/orders/DSPrimeTask_ds_idrepo_0/rc.txt" ]; then echo "/results/orders/DSPrimeTask_ds_idrepo_0/rc.txt file found"; exit 0; else echo "/results/orders/DSPrimeTask_ds_idrepo_0/rc.txt file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_ds_idrepo_0/rc.txt file found" 2024-04-25 17:39:43,237 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-25 17:39:44,095 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-25 17:39:44,095 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:44,095 DEBUG --- stdout --- 2024-04-25 17:39:44,095 DEBUG /results/orders/DSPrimeTask_ds_idrepo_0/rc.txt file found 2024-04-25 17:39:44,095 DEBUG --- stderr --- 2024-04-25 17:39:44,095 DEBUG 2024-04-25 17:39:44,095 INFO 2024-04-25 17:39:44,095 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:44,095 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:44,401 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:44,402 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:44,402 DEBUG --- stdout --- 2024-04-25 17:39:44,402 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:44,402 DEBUG --- stderr --- 2024-04-25 17:39:44,402 DEBUG 2024-04-25 17:39:44,402 INFO 2024-04-25 17:39:44,402 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-56c5c978d9-v8wm9 --stdin -- du -k /results/orders/DSPrimeTask_ds_idrepo_0/rc.txt 2024-04-25 17:39:45,223 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-25 17:39:45,224 DEBUG --- stdout --- 2024-04-25 17:39:45,224 DEBUG 4 /results/orders/DSPrimeTask_ds_idrepo_0/rc.txt 2024-04-25 17:39:45,224 DEBUG --- stderr --- 2024-04-25 17:39:45,224 DEBUG 2024-04-25 17:39:45,224 INFO - Download file (size 4k) 2024-04-25 17:39:45,224 INFO 2024-04-25 17:39:45,224 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:45,224 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:45,565 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:45,566 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:45,566 DEBUG --- stdout --- 2024-04-25 17:39:45,566 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:45,566 DEBUG --- stderr --- 2024-04-25 17:39:45,566 DEBUG source : /results/orders/DSPrimeTask_ds_idrepo_0/rc.txt @ overseer-0-56c5c978d9-v8wm9 destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/rc.txt 2024-04-25 17:39:45,566 INFO 2024-04-25 17:39:45,566 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:45,566 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:45,896 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:45,896 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:45,896 DEBUG --- stdout --- 2024-04-25 17:39:45,896 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:45,896 DEBUG --- stderr --- 2024-04-25 17:39:45,896 DEBUG 2024-04-25 17:39:45,897 INFO 2024-04-25 17:39:45,897 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-56c5c978d9-v8wm9:/results/orders/DSPrimeTask_ds_idrepo_0/rc.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/rc.txt 2024-04-25 17:39:45,897 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-04-25 17:39:46,660 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:46,660 DEBUG --- stdout --- 2024-04-25 17:39:46,660 DEBUG tar: Removing leading `/' from member names 2024-04-25 17:39:46,660 DEBUG --- stderr --- 2024-04-25 17:39:46,660 DEBUG 2024-04-25 17:39:46,661 INFO 2024-04-25 17:39:46,661 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:46,661 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:46,981 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:46,982 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:46,982 DEBUG --- stdout --- 2024-04-25 17:39:46,982 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:46,982 DEBUG --- stderr --- 2024-04-25 17:39:46,982 DEBUG Checking pod: overseer-0-56c5c978d9-v8wm9 for file exists, if not wait. 2024-04-25 17:39:46,982 INFO 2024-04-25 17:39:46,982 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:46,982 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:47,300 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:47,300 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:47,300 DEBUG --- stdout --- 2024-04-25 17:39:47,300 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:47,300 DEBUG --- stderr --- 2024-04-25 17:39:47,300 DEBUG 2024-04-25 17:39:47,301 INFO 2024-04-25 17:39:47,301 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-56c5c978d9-v8wm9 --stdin -- bash -c 'if [ -f "/results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt" ]; then echo "/results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt file found"; exit 0; else echo "/results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt file found" 2024-04-25 17:39:47,301 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-25 17:39:48,126 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-25 17:39:48,126 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:48,126 DEBUG --- stdout --- 2024-04-25 17:39:48,127 DEBUG /results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt file found 2024-04-25 17:39:48,127 DEBUG --- stderr --- 2024-04-25 17:39:48,127 DEBUG 2024-04-25 17:39:48,127 INFO 2024-04-25 17:39:48,127 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:48,127 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:48,434 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:48,434 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:48,434 DEBUG --- stdout --- 2024-04-25 17:39:48,435 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:48,435 DEBUG --- stderr --- 2024-04-25 17:39:48,435 DEBUG 2024-04-25 17:39:48,435 INFO 2024-04-25 17:39:48,435 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-56c5c978d9-v8wm9 --stdin -- du -k /results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt 2024-04-25 17:39:49,266 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-25 17:39:49,266 DEBUG --- stdout --- 2024-04-25 17:39:49,266 DEBUG 0 /results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt 2024-04-25 17:39:49,266 DEBUG --- stderr --- 2024-04-25 17:39:49,266 DEBUG 2024-04-25 17:39:49,266 INFO - Download file (size 0k) 2024-04-25 17:39:49,267 INFO 2024-04-25 17:39:49,267 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:49,267 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:49,628 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:49,629 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:49,629 DEBUG --- stdout --- 2024-04-25 17:39:49,629 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:49,629 DEBUG --- stderr --- 2024-04-25 17:39:49,629 DEBUG source : /results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt @ overseer-0-56c5c978d9-v8wm9 destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/stderr.txt 2024-04-25 17:39:49,629 INFO 2024-04-25 17:39:49,629 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:49,629 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:49,947 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:49,947 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:49,947 DEBUG --- stdout --- 2024-04-25 17:39:49,947 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:49,947 DEBUG --- stderr --- 2024-04-25 17:39:49,947 DEBUG 2024-04-25 17:39:49,948 INFO 2024-04-25 17:39:49,948 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-56c5c978d9-v8wm9:/results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/stderr.txt 2024-04-25 17:39:49,948 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-04-25 17:39:50,702 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:50,703 DEBUG --- stdout --- 2024-04-25 17:39:50,703 DEBUG tar: Removing leading `/' from member names 2024-04-25 17:39:50,703 DEBUG --- stderr --- 2024-04-25 17:39:50,703 DEBUG 2024-04-25 17:39:50,703 INFO 2024-04-25 17:39:50,703 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:50,703 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:51,011 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:51,012 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:51,012 DEBUG --- stdout --- 2024-04-25 17:39:51,012 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:51,012 DEBUG --- stderr --- 2024-04-25 17:39:51,012 DEBUG Checking pod: overseer-0-56c5c978d9-v8wm9 for file exists, if not wait. 2024-04-25 17:39:51,012 INFO 2024-04-25 17:39:51,012 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:51,012 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:51,321 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:51,321 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:51,322 DEBUG --- stdout --- 2024-04-25 17:39:51,322 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:51,322 DEBUG --- stderr --- 2024-04-25 17:39:51,322 DEBUG 2024-04-25 17:39:51,322 INFO 2024-04-25 17:39:51,322 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-56c5c978d9-v8wm9 --stdin -- bash -c 'if [ -f "/results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt" ]; then echo "/results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt file found"; exit 0; else echo "/results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt file found" 2024-04-25 17:39:51,322 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-25 17:39:52,142 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-25 17:39:52,142 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:52,142 DEBUG --- stdout --- 2024-04-25 17:39:52,142 DEBUG /results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt file found 2024-04-25 17:39:52,142 DEBUG --- stderr --- 2024-04-25 17:39:52,142 DEBUG 2024-04-25 17:39:52,143 INFO 2024-04-25 17:39:52,143 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:52,143 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:52,466 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:52,466 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:52,466 DEBUG --- stdout --- 2024-04-25 17:39:52,466 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:52,466 DEBUG --- stderr --- 2024-04-25 17:39:52,466 DEBUG 2024-04-25 17:39:52,466 INFO 2024-04-25 17:39:52,466 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-56c5c978d9-v8wm9 --stdin -- du -k /results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt 2024-04-25 17:39:53,282 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-25 17:39:53,282 DEBUG --- stdout --- 2024-04-25 17:39:53,282 DEBUG 4 /results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt 2024-04-25 17:39:53,282 DEBUG --- stderr --- 2024-04-25 17:39:53,282 DEBUG 2024-04-25 17:39:53,282 INFO - Download file (size 4k) 2024-04-25 17:39:53,283 INFO 2024-04-25 17:39:53,283 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:53,283 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:53,581 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:53,582 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:53,582 DEBUG --- stdout --- 2024-04-25 17:39:53,582 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:53,582 DEBUG --- stderr --- 2024-04-25 17:39:53,582 DEBUG source : /results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt @ overseer-0-56c5c978d9-v8wm9 destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/stdout.txt 2024-04-25 17:39:53,582 INFO 2024-04-25 17:39:53,582 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:53,582 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:53,888 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:53,888 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:53,888 DEBUG --- stdout --- 2024-04-25 17:39:53,888 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:53,888 DEBUG --- stderr --- 2024-04-25 17:39:53,888 DEBUG 2024-04-25 17:39:53,888 INFO 2024-04-25 17:39:53,888 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-56c5c978d9-v8wm9:/results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/stdout.txt 2024-04-25 17:39:53,889 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-04-25 17:39:54,665 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:54,665 DEBUG --- stdout --- 2024-04-25 17:39:54,665 DEBUG tar: Removing leading `/' from member names 2024-04-25 17:39:54,665 DEBUG --- stderr --- 2024-04-25 17:39:54,665 DEBUG ________________________________________________________________________________ [2024-04-25 17:39:54] DSPrimeTask_ds_idrepo_0 show_cmd : Display order command ________________________________________________________________________________ 2024-04-25 17:39:54,666 INFO 2024-04-25 17:39:54,666 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:54,666 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:54,965 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:54,965 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:54,965 DEBUG --- stdout --- 2024-04-25 17:39:54,965 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:54,965 DEBUG --- stderr --- 2024-04-25 17:39:54,965 DEBUG Checking pod: overseer-0-56c5c978d9-v8wm9 for file exists, if not wait. 2024-04-25 17:39:54,965 INFO 2024-04-25 17:39:54,965 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:54,966 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:55,269 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:55,270 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:55,270 DEBUG --- stdout --- 2024-04-25 17:39:55,270 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:55,270 DEBUG --- stderr --- 2024-04-25 17:39:55,270 DEBUG 2024-04-25 17:39:55,270 INFO 2024-04-25 17:39:55,270 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-56c5c978d9-v8wm9 --stdin -- bash -c 'if [ -f "/results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt" ]; then echo "/results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt file found"; exit 0; else echo "/results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt file found" 2024-04-25 17:39:55,271 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-25 17:39:56,103 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-25 17:39:56,103 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:56,103 DEBUG --- stdout --- 2024-04-25 17:39:56,103 DEBUG /results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt file found 2024-04-25 17:39:56,103 DEBUG --- stderr --- 2024-04-25 17:39:56,103 DEBUG 2024-04-25 17:39:56,104 INFO 2024-04-25 17:39:56,104 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:56,104 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:56,428 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:56,428 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:56,428 DEBUG --- stdout --- 2024-04-25 17:39:56,429 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:56,429 DEBUG --- stderr --- 2024-04-25 17:39:56,429 DEBUG 2024-04-25 17:39:56,429 INFO 2024-04-25 17:39:56,429 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-56c5c978d9-v8wm9 --stdin -- du -k /results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt 2024-04-25 17:39:57,300 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-25 17:39:57,301 DEBUG --- stdout --- 2024-04-25 17:39:57,301 DEBUG 4 /results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt 2024-04-25 17:39:57,301 DEBUG --- stderr --- 2024-04-25 17:39:57,301 DEBUG 2024-04-25 17:39:57,301 INFO - Download file (size 4k) 2024-04-25 17:39:57,301 INFO 2024-04-25 17:39:57,301 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:57,301 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:57,608 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:57,609 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:57,609 DEBUG --- stdout --- 2024-04-25 17:39:57,609 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:57,609 DEBUG --- stderr --- 2024-04-25 17:39:57,609 DEBUG source : /results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt @ overseer-0-56c5c978d9-v8wm9 destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/cmd.txt 2024-04-25 17:39:57,609 INFO 2024-04-25 17:39:57,609 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-25 17:39:57,609 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-25 17:39:57,944 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-25 17:39:57,944 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:57,944 DEBUG --- stdout --- 2024-04-25 17:39:57,944 DEBUG overseer-0-56c5c978d9-v8wm9 2024-04-25 17:39:57,944 DEBUG --- stderr --- 2024-04-25 17:39:57,944 DEBUG 2024-04-25 17:39:57,945 INFO 2024-04-25 17:39:57,945 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-56c5c978d9-v8wm9:/results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/cmd.txt 2024-04-25 17:39:57,945 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-04-25 17:39:58,693 INFO [loop_until]: OK (rc = 0) 2024-04-25 17:39:58,693 DEBUG --- stdout --- 2024-04-25 17:39:58,693 DEBUG tar: Removing leading `/' from member names 2024-04-25 17:39:58,693 DEBUG --- stderr --- 2024-04-25 17:39:58,693 DEBUG 2024-04-25 17:39:58,694 DEBUG [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/cmd.txt is small (1 lines), printing all lines: 2024-04-25 17:39:58,694 DEBUG ----- output ----- 2024-04-25 17:39:58,694 DEBUG /lodestar/clients/ds/bin/searchrate --maxDuration 1 --bindPassword "Te3hvGVKgyQ3Y4m0fbG16hKkzkr9FMrw" --bindDn "uid=admin" --trustAll --useStartTls --port 1389 --statInterval 5 --noPropertiesFile --hostname 'ds-idrepo-0.ds-idrepo' --noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 1000000 --argument 'inc(0, 1000000)' '(uid=user.{})' 2024-04-25 17:39:58,694 INFO ________________________________________________________________________________ [2024-04-25 17:39:58] DSPrimeTask_ds_idrepo_0 show_rc : Display rc order ________________________________________________________________________________ 2024-04-25 17:39:58,694 DEBUG [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/rc.txt is small (1 lines), printing all lines: 2024-04-25 17:39:58,694 DEBUG ----- output ----- 2024-04-25 17:39:58,694 DEBUG CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) 2024-04-25 17:39:58,694 INFO ________________________________________________________________________________ [2024-04-25 17:39:58] DSPrimeTask_ds_idrepo_0 show_stdout : Display stdout order ________________________________________________________________________________ 2024-04-25 17:39:58,694 DEBUG [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/stdout.txt is small (7 lines), printing all lines: 2024-04-25 17:39:58,694 DEBUG ----- output ----- 2024-04-25 17:39:58,694 DEBUG Warming up for 10 seconds... -------------------------------------------------------------------------------------------- | Throughput | Response Time | Additional | | (ops/second) | (milliseconds) | Statistics | | recent average | recent average 99.9% 99.99% 99.999% | err/sec Entries/Srch | -------------------------------------------------------------------------------------------- | 0.0 0.0 | - - 10.16 11.14 11.14 | 0.0 - | 2024-04-25 17:39:58,694 INFO ________________________________________________________________________________ [2024-04-25 17:39:58] DSPrimeTask_ds_idrepo_0 show_stderr : Display stderr order ________________________________________________________________________________ 2024-04-25 17:39:58,695 DEBUG [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/06_Validation/02_DSPrimeTask_ds_idrepo_0/stderr.txt is small (0 lines), printing all lines: 2024-04-25 17:39:58,695 DEBUG ----- output ----- 2024-04-25 17:39:58,695 DEBUG 2024-04-25 17:39:58,695 INFO ________________________________________________________________________________ [2024-04-25 17:39:58] DSPrimeTask_ds_idrepo_0 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2024-04-25 17:39:58] DSPrimeTask_ds_idrepo_0 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2024-04-25 17:39:58] DSPrimeTask_ds_idrepo_0 post : Check output ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped