--Task-- name: DSPrimeTask_ds_idrepo_2 enabled: True class_name: DSSearchRateTask source_name: overseer-0 source_namespace: >default< target_name: ds-idrepo-2 target_namespace: >default< start: 0.0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: ['DSPrimeTask_ds_idrepo_1'] wait_for: [] preceding_task: None options: {'_options_': "--noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 10000000 --argument 'inc(0, 10000000)'"} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [2024-04-24 05:30:55] DSPrimeTask_ds_idrepo_2 pre : N/A ________________________________________________________________________________ [2024-04-24 05:30:55] - INFO: target component (ds-idrepo-2) is a DS pod [2024-04-24 05:30:55] - INFO: Timeout for this Task has changed to 59s (-1 seconds) [2024-04-24 05:30:55] - INFO: timeout was set to unlimited, because it was unset Pod log file : /results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt Pod log : /results/orders/DSPrimeTask_ds_idrepo_2 ________________________________________________________________________________ [2024-04-24 05:30:55] DSPrimeTask_ds_idrepo_2 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/clients/ds/bin/searchrate --maxDuration 1 --bindPassword "KOZVtiYFX4Yi5XEpffswoBMqHpOIR2lc" --bindDn "uid=admin" --trustAll --useStartTls --port 1389 --statInterval 120 --noPropertiesFile --hostname 'ds-idrepo-2.ds-idrepo' --noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 10000000 --argument 'inc(0, 10000000)' '(uid=user.{})' ________________________________________________________________________________ [2024-04-24 05:30:55] DSPrimeTask_ds_idrepo_2 create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file: /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/order.json { "name": "DSPrimeTask_ds_idrepo_2", "command": "/lodestar/clients/ds/bin/searchrate --maxDuration 1 --bindPassword **** --bindDn \"uid=admin\" --trustAll --useStartTls --port 1389 --statInterval 120 --noPropertiesFile --hostname 'ds-idrepo-2.ds-idrepo' --noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 10000000 --argument 'inc(0, 10000000)' '(uid=user.{})'", "out-dir": "/results/orders/DSPrimeTask_ds_idrepo_2" } ________________________________________________________________________________ [2024-04-24 05:30:55] DSPrimeTask_ds_idrepo_2 loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2024-04-24 05:30:55,794 INFO 2024-04-24 05:30:55,794 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2024-04-24 05:30:55,794 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2024-04-24 05:30:56,035 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-24 05:30:56,035 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:30:56,036 DEBUG --- stdout --- 2024-04-24 05:30:56,036 DEBUG WAITING 2024-04-24 05:30:56,036 DEBUG --- stderr --- 2024-04-24 05:30:56,036 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 32 0 --:--:-- --:--:-- --:--:-- 32 ________________________________________________________________________________ [2024-04-24 05:30:56] DSPrimeTask_ds_idrepo_2 start_order : Send the order to the overseer pod ________________________________________________________________________________ 2024-04-24 05:30:56,036 INFO 2024-04-24 05:30:56,036 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:30:56,036 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:30:56,356 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:30:56,356 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:30:56,356 DEBUG --- stdout --- 2024-04-24 05:30:56,357 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:30:56,357 DEBUG --- stderr --- 2024-04-24 05:30:56,357 DEBUG 2024-04-24 05:30:56,357 INFO 2024-04-24 05:30:56,357 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/order.json overseer-0-55d5975bd7-gp92m:/results/orders/order.json 2024-04-24 05:30:56,357 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-24 05:30:57,755 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:30:57,755 DEBUG --- stdout --- 2024-04-24 05:30:57,755 DEBUG 2024-04-24 05:30:57,755 DEBUG --- stderr --- 2024-04-24 05:30:57,755 DEBUG Order has been successfully forwarded to overseer ________________________________________________________________________________ [2024-04-24 05:30:57] DSPrimeTask_ds_idrepo_2 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 59s Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING PASS : Order stopped by itself after 2m and before timeout (59s) ________________________________________________________________________________ [2024-04-24 05:32:58] DSPrimeTask_ds_idrepo_2 stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2024-04-24 05:32:58,120 INFO 2024-04-24 05:32:58,120 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:32:58,121 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:32:58,460 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:32:58,460 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:32:58,461 DEBUG --- stdout --- 2024-04-24 05:32:58,461 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:32:58,461 DEBUG --- stderr --- 2024-04-24 05:32:58,461 DEBUG 2024-04-24 05:32:58,461 INFO 2024-04-24 05:32:58,461 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-55d5975bd7-gp92m -- rm -f /results/orders/order.json 2024-04-24 05:32:58,461 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-24 05:32:59,270 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:32:59,271 DEBUG --- stdout --- 2024-04-24 05:32:59,271 DEBUG 2024-04-24 05:32:59,271 DEBUG --- stderr --- 2024-04-24 05:32:59,271 DEBUG Order ran during 2m 01s ________________________________________________________________________________ [2024-04-24 05:32:59] DSPrimeTask_ds_idrepo_2 download_logs : Download individual order files ________________________________________________________________________________ [2024-04-24 05:32:59] - PASS: file already downloaded /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/order.json 2024-04-24 05:32:59,272 INFO 2024-04-24 05:32:59,272 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:32:59,272 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:32:59,610 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:32:59,610 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:32:59,610 DEBUG --- stdout --- 2024-04-24 05:32:59,610 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:32:59,610 DEBUG --- stderr --- 2024-04-24 05:32:59,610 DEBUG Checking pod: overseer-0-55d5975bd7-gp92m for file exists, if not wait. 2024-04-24 05:32:59,611 INFO 2024-04-24 05:32:59,611 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:32:59,611 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:32:59,937 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:32:59,939 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:32:59,939 DEBUG --- stdout --- 2024-04-24 05:32:59,939 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:32:59,939 DEBUG --- stderr --- 2024-04-24 05:32:59,939 DEBUG 2024-04-24 05:32:59,940 INFO 2024-04-24 05:32:59,940 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-55d5975bd7-gp92m --stdin -- bash -c 'if [ -f "/results/orders/DSPrimeTask_ds_idrepo_2/rc.txt" ]; then echo "/results/orders/DSPrimeTask_ds_idrepo_2/rc.txt file found"; exit 0; else echo "/results/orders/DSPrimeTask_ds_idrepo_2/rc.txt file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_ds_idrepo_2/rc.txt file found" 2024-04-24 05:32:59,940 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-24 05:33:00,801 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-24 05:33:00,801 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:00,801 DEBUG --- stdout --- 2024-04-24 05:33:00,801 DEBUG /results/orders/DSPrimeTask_ds_idrepo_2/rc.txt file found 2024-04-24 05:33:00,801 DEBUG --- stderr --- 2024-04-24 05:33:00,802 DEBUG 2024-04-24 05:33:00,802 INFO 2024-04-24 05:33:00,802 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:00,802 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:01,116 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:01,117 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:01,117 DEBUG --- stdout --- 2024-04-24 05:33:01,117 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:01,117 DEBUG --- stderr --- 2024-04-24 05:33:01,117 DEBUG 2024-04-24 05:33:01,117 INFO 2024-04-24 05:33:01,117 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-55d5975bd7-gp92m --stdin -- du -k /results/orders/DSPrimeTask_ds_idrepo_2/rc.txt 2024-04-24 05:33:02,012 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-24 05:33:02,013 DEBUG --- stdout --- 2024-04-24 05:33:02,013 DEBUG 4 /results/orders/DSPrimeTask_ds_idrepo_2/rc.txt 2024-04-24 05:33:02,013 DEBUG --- stderr --- 2024-04-24 05:33:02,013 DEBUG 2024-04-24 05:33:02,013 INFO - Download file (size 4k) 2024-04-24 05:33:02,013 INFO 2024-04-24 05:33:02,013 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:02,013 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:02,334 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:02,334 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:02,334 DEBUG --- stdout --- 2024-04-24 05:33:02,335 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:02,335 DEBUG --- stderr --- 2024-04-24 05:33:02,335 DEBUG source : /results/orders/DSPrimeTask_ds_idrepo_2/rc.txt @ overseer-0-55d5975bd7-gp92m destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/rc.txt 2024-04-24 05:33:02,335 INFO 2024-04-24 05:33:02,335 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:02,335 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:02,649 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:02,649 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:02,649 DEBUG --- stdout --- 2024-04-24 05:33:02,649 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:02,649 DEBUG --- stderr --- 2024-04-24 05:33:02,649 DEBUG 2024-04-24 05:33:02,650 INFO 2024-04-24 05:33:02,650 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-55d5975bd7-gp92m:/results/orders/DSPrimeTask_ds_idrepo_2/rc.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/rc.txt 2024-04-24 05:33:02,650 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-04-24 05:33:03,414 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:03,415 DEBUG --- stdout --- 2024-04-24 05:33:03,415 DEBUG tar: Removing leading `/' from member names 2024-04-24 05:33:03,415 DEBUG --- stderr --- 2024-04-24 05:33:03,415 DEBUG 2024-04-24 05:33:03,415 INFO 2024-04-24 05:33:03,415 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:03,415 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:03,734 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:03,734 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:03,734 DEBUG --- stdout --- 2024-04-24 05:33:03,734 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:03,734 DEBUG --- stderr --- 2024-04-24 05:33:03,735 DEBUG Checking pod: overseer-0-55d5975bd7-gp92m for file exists, if not wait. 2024-04-24 05:33:03,735 INFO 2024-04-24 05:33:03,735 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:03,735 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:04,102 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:04,103 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:04,103 DEBUG --- stdout --- 2024-04-24 05:33:04,103 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:04,103 DEBUG --- stderr --- 2024-04-24 05:33:04,103 DEBUG 2024-04-24 05:33:04,103 INFO 2024-04-24 05:33:04,103 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-55d5975bd7-gp92m --stdin -- bash -c 'if [ -f "/results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt" ]; then echo "/results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt file found"; exit 0; else echo "/results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt file found" 2024-04-24 05:33:04,104 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-24 05:33:04,997 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-24 05:33:04,999 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:04,999 DEBUG --- stdout --- 2024-04-24 05:33:04,999 DEBUG /results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt file found 2024-04-24 05:33:05,000 DEBUG --- stderr --- 2024-04-24 05:33:05,000 DEBUG 2024-04-24 05:33:05,000 INFO 2024-04-24 05:33:05,000 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:05,000 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:05,362 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:05,362 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:05,362 DEBUG --- stdout --- 2024-04-24 05:33:05,362 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:05,362 DEBUG --- stderr --- 2024-04-24 05:33:05,362 DEBUG 2024-04-24 05:33:05,362 INFO 2024-04-24 05:33:05,363 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-55d5975bd7-gp92m --stdin -- du -k /results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt 2024-04-24 05:33:06,242 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-24 05:33:06,242 DEBUG --- stdout --- 2024-04-24 05:33:06,242 DEBUG 0 /results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt 2024-04-24 05:33:06,242 DEBUG --- stderr --- 2024-04-24 05:33:06,242 DEBUG 2024-04-24 05:33:06,242 INFO - Download file (size 0k) 2024-04-24 05:33:06,243 INFO 2024-04-24 05:33:06,243 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:06,243 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:06,558 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:06,558 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:06,558 DEBUG --- stdout --- 2024-04-24 05:33:06,558 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:06,558 DEBUG --- stderr --- 2024-04-24 05:33:06,558 DEBUG source : /results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt @ overseer-0-55d5975bd7-gp92m destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/stderr.txt 2024-04-24 05:33:06,558 INFO 2024-04-24 05:33:06,558 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:06,559 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:06,881 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:06,881 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:06,881 DEBUG --- stdout --- 2024-04-24 05:33:06,881 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:06,881 DEBUG --- stderr --- 2024-04-24 05:33:06,881 DEBUG 2024-04-24 05:33:06,881 INFO 2024-04-24 05:33:06,882 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-55d5975bd7-gp92m:/results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/stderr.txt 2024-04-24 05:33:06,882 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-04-24 05:33:07,649 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:07,649 DEBUG --- stdout --- 2024-04-24 05:33:07,649 DEBUG tar: Removing leading `/' from member names 2024-04-24 05:33:07,649 DEBUG --- stderr --- 2024-04-24 05:33:07,649 DEBUG 2024-04-24 05:33:07,650 INFO 2024-04-24 05:33:07,650 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:07,650 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:07,961 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:07,961 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:07,961 DEBUG --- stdout --- 2024-04-24 05:33:07,961 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:07,961 DEBUG --- stderr --- 2024-04-24 05:33:07,961 DEBUG Checking pod: overseer-0-55d5975bd7-gp92m for file exists, if not wait. 2024-04-24 05:33:07,962 INFO 2024-04-24 05:33:07,962 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:07,962 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:08,281 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:08,281 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:08,281 DEBUG --- stdout --- 2024-04-24 05:33:08,281 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:08,281 DEBUG --- stderr --- 2024-04-24 05:33:08,281 DEBUG 2024-04-24 05:33:08,282 INFO 2024-04-24 05:33:08,282 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-55d5975bd7-gp92m --stdin -- bash -c 'if [ -f "/results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt" ]; then echo "/results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt file found"; exit 0; else echo "/results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt file found" 2024-04-24 05:33:08,282 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-24 05:33:09,126 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-24 05:33:09,126 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:09,126 DEBUG --- stdout --- 2024-04-24 05:33:09,126 DEBUG /results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt file found 2024-04-24 05:33:09,127 DEBUG --- stderr --- 2024-04-24 05:33:09,127 DEBUG 2024-04-24 05:33:09,127 INFO 2024-04-24 05:33:09,127 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:09,127 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:09,440 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:09,440 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:09,440 DEBUG --- stdout --- 2024-04-24 05:33:09,440 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:09,440 DEBUG --- stderr --- 2024-04-24 05:33:09,440 DEBUG 2024-04-24 05:33:09,440 INFO 2024-04-24 05:33:09,441 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-55d5975bd7-gp92m --stdin -- du -k /results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt 2024-04-24 05:33:10,275 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-24 05:33:10,275 DEBUG --- stdout --- 2024-04-24 05:33:10,276 DEBUG 4 /results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt 2024-04-24 05:33:10,277 DEBUG --- stderr --- 2024-04-24 05:33:10,277 DEBUG 2024-04-24 05:33:10,277 INFO - Download file (size 4k) 2024-04-24 05:33:10,277 INFO 2024-04-24 05:33:10,277 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:10,277 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:10,596 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:10,596 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:10,596 DEBUG --- stdout --- 2024-04-24 05:33:10,596 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:10,596 DEBUG --- stderr --- 2024-04-24 05:33:10,596 DEBUG source : /results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt @ overseer-0-55d5975bd7-gp92m destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/stdout.txt 2024-04-24 05:33:10,597 INFO 2024-04-24 05:33:10,597 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:10,597 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:10,914 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:10,914 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:10,914 DEBUG --- stdout --- 2024-04-24 05:33:10,914 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:10,914 DEBUG --- stderr --- 2024-04-24 05:33:10,914 DEBUG 2024-04-24 05:33:10,915 INFO 2024-04-24 05:33:10,915 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-55d5975bd7-gp92m:/results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/stdout.txt 2024-04-24 05:33:10,915 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-04-24 05:33:11,726 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:11,726 DEBUG --- stdout --- 2024-04-24 05:33:11,726 DEBUG tar: Removing leading `/' from member names 2024-04-24 05:33:11,726 DEBUG --- stderr --- 2024-04-24 05:33:11,726 DEBUG ________________________________________________________________________________ [2024-04-24 05:33:11] DSPrimeTask_ds_idrepo_2 show_cmd : Display order command ________________________________________________________________________________ 2024-04-24 05:33:11,727 INFO 2024-04-24 05:33:11,727 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:11,727 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:12,043 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:12,043 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:12,043 DEBUG --- stdout --- 2024-04-24 05:33:12,043 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:12,043 DEBUG --- stderr --- 2024-04-24 05:33:12,043 DEBUG Checking pod: overseer-0-55d5975bd7-gp92m for file exists, if not wait. 2024-04-24 05:33:12,044 INFO 2024-04-24 05:33:12,044 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:12,044 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:12,358 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:12,359 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:12,359 DEBUG --- stdout --- 2024-04-24 05:33:12,359 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:12,359 DEBUG --- stderr --- 2024-04-24 05:33:12,359 DEBUG 2024-04-24 05:33:12,359 INFO 2024-04-24 05:33:12,359 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-55d5975bd7-gp92m --stdin -- bash -c 'if [ -f "/results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt" ]; then echo "/results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt file found"; exit 0; else echo "/results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt file found" 2024-04-24 05:33:12,359 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-24 05:33:13,200 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-24 05:33:13,200 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:13,200 DEBUG --- stdout --- 2024-04-24 05:33:13,201 DEBUG /results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt file found 2024-04-24 05:33:13,201 DEBUG --- stderr --- 2024-04-24 05:33:13,201 DEBUG 2024-04-24 05:33:13,201 INFO 2024-04-24 05:33:13,201 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:13,201 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:13,522 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:13,522 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:13,523 DEBUG --- stdout --- 2024-04-24 05:33:13,523 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:13,523 DEBUG --- stderr --- 2024-04-24 05:33:13,523 DEBUG 2024-04-24 05:33:13,523 INFO 2024-04-24 05:33:13,523 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-55d5975bd7-gp92m --stdin -- du -k /results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt 2024-04-24 05:33:14,374 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-24 05:33:14,374 DEBUG --- stdout --- 2024-04-24 05:33:14,374 DEBUG 4 /results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt 2024-04-24 05:33:14,374 DEBUG --- stderr --- 2024-04-24 05:33:14,374 DEBUG 2024-04-24 05:33:14,374 INFO - Download file (size 4k) 2024-04-24 05:33:14,375 INFO 2024-04-24 05:33:14,375 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:14,375 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:14,691 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:14,691 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:14,691 DEBUG --- stdout --- 2024-04-24 05:33:14,691 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:14,691 DEBUG --- stderr --- 2024-04-24 05:33:14,691 DEBUG source : /results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt @ overseer-0-55d5975bd7-gp92m destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/cmd.txt 2024-04-24 05:33:14,692 INFO 2024-04-24 05:33:14,692 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-24 05:33:14,692 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-24 05:33:15,008 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-24 05:33:15,009 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:15,009 DEBUG --- stdout --- 2024-04-24 05:33:15,009 DEBUG overseer-0-55d5975bd7-gp92m 2024-04-24 05:33:15,009 DEBUG --- stderr --- 2024-04-24 05:33:15,009 DEBUG 2024-04-24 05:33:15,009 INFO 2024-04-24 05:33:15,009 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-55d5975bd7-gp92m:/results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/cmd.txt 2024-04-24 05:33:15,009 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-04-24 05:33:15,768 INFO [loop_until]: OK (rc = 0) 2024-04-24 05:33:15,768 DEBUG --- stdout --- 2024-04-24 05:33:15,768 DEBUG tar: Removing leading `/' from member names 2024-04-24 05:33:15,768 DEBUG --- stderr --- 2024-04-24 05:33:15,768 DEBUG 2024-04-24 05:33:15,768 DEBUG [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/cmd.txt is small (1 lines), printing all lines: 2024-04-24 05:33:15,768 DEBUG ----- output ----- 2024-04-24 05:33:15,769 DEBUG /lodestar/clients/ds/bin/searchrate --maxDuration 1 --bindPassword "KOZVtiYFX4Yi5XEpffswoBMqHpOIR2lc" --bindDn "uid=admin" --trustAll --useStartTls --port 1389 --statInterval 120 --noPropertiesFile --hostname 'ds-idrepo-2.ds-idrepo' --noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 10000000 --argument 'inc(0, 10000000)' '(uid=user.{})' 2024-04-24 05:33:15,769 INFO ________________________________________________________________________________ [2024-04-24 05:33:15] DSPrimeTask_ds_idrepo_2 show_rc : Display rc order ________________________________________________________________________________ 2024-04-24 05:33:15,769 DEBUG [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/rc.txt is small (1 lines), printing all lines: 2024-04-24 05:33:15,769 DEBUG ----- output ----- 2024-04-24 05:33:15,769 DEBUG CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) 2024-04-24 05:33:15,769 INFO ________________________________________________________________________________ [2024-04-24 05:33:15] DSPrimeTask_ds_idrepo_2 show_stdout : Display stdout order ________________________________________________________________________________ 2024-04-24 05:33:15,769 DEBUG [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/stdout.txt is small (7 lines), printing all lines: 2024-04-24 05:33:15,769 DEBUG ----- output ----- 2024-04-24 05:33:15,769 DEBUG Warming up for 10 seconds... -------------------------------------------------------------------------------------------- | Throughput | Response Time | Additional | | (ops/second) | (milliseconds) | Statistics | | recent average | recent average 99.9% 99.99% 99.999% | err/sec Entries/Srch | -------------------------------------------------------------------------------------------- | 0.0 0.0 | - - 10.09 11.21 11.21 | 0.0 - | 2024-04-24 05:33:15,769 INFO ________________________________________________________________________________ [2024-04-24 05:33:15] DSPrimeTask_ds_idrepo_2 show_stderr : Display stderr order ________________________________________________________________________________ 2024-04-24 05:33:15,769 DEBUG [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/06_Validation/05_DSPrimeTask_ds_idrepo_2/stderr.txt is small (0 lines), printing all lines: 2024-04-24 05:33:15,770 DEBUG ----- output ----- 2024-04-24 05:33:15,770 DEBUG 2024-04-24 05:33:15,770 INFO ________________________________________________________________________________ [2024-04-24 05:33:15] DSPrimeTask_ds_idrepo_2 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2024-04-24 05:33:15] DSPrimeTask_ds_idrepo_2 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2024-04-24 05:33:15] DSPrimeTask_ds_idrepo_2 post : Check output ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped