--Task-- name: DSPrimeTask_id_repo_0 enabled: True class_name: DSSearchRateTask source_name: overseer-0 source_namespace: >default< target_name: ds-idrepo-0 target_namespace: >default< start: 0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: [] wait_for: [] options: {'_options_': "--noRebind --warmUpDuration 10 --maxDuration 90 --numConnections 64 --numConcurrentRequests 8 -b 'ou=people,ou=identities'"} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [06/Apr/2023 23:33:06] DSPrimeTask_id_repo_0 pre : N/A ________________________________________________________________________________ [06/Apr/2023 23:33:06] - INFO: target (ds-idrepo-0) is a ds pod [06/Apr/2023 23:33:06] - INFO: Timeout for this Task has changed to 59s (-1 seconds) [06/Apr/2023 23:33:06] - INFO: timeout was set to unlimited, because it was unset Pod log file : /results/orders/DSPrimeTask_id_repo_0/stdout.txt Pod log : /results/orders/DSPrimeTask_id_repo_0 [run_command]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "fwG9YpsslkrZHDsXR1RKtRiiUPWtG36g" -b "ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor" -s base "objectclass=*" ds-mon-base-dn-entry-count [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor ds-mon-base-dn-entry-count: 1000022 --- stderr --- ________________________________________________________________________________ [06/Apr/2023 23:33:07] DSPrimeTask_id_repo_0 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ [run_command]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "fwG9YpsslkrZHDsXR1RKtRiiUPWtG36g" -b "ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor" -s base "objectclass=*" ds-mon-base-dn-entry-count [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor ds-mon-base-dn-entry-count: 1000022 --- stderr --- /lodestar/clients/ds/bin/searchrate --noPropertiesFile -h ds-idrepo-0.ds-idrepo -w "fwG9YpsslkrZHDsXR1RKtRiiUPWtG36g" -D "uid=admin" --trustAll --useStartTls -p 1389 -i 5 --noRebind --warmUpDuration 10 --maxDuration 90 --numConnections 64 --numConcurrentRequests 8 -b 'ou=people,ou=identities' -g 'rand(0,1000022)' '(uid=user.{})' ________________________________________________________________________________ [06/Apr/2023 23:33:09] DSPrimeTask_id_repo_0 create_order : Create order for overseer with the command to run ________________________________________________________________________________ [run_command]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "fwG9YpsslkrZHDsXR1RKtRiiUPWtG36g" -b "ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor" -s base "objectclass=*" ds-mon-base-dn-entry-count [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor ds-mon-base-dn-entry-count: 1000021 --- stderr --- Order file : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0/order.json { "name": "DSPrimeTask_id_repo_0", "command": "/lodestar/clients/ds/bin/searchrate --noPropertiesFile -h ds-idrepo-0.ds-idrepo -w \"fwG9YpsslkrZHDsXR1RKtRiiUPWtG36g\" -D \"uid=admin\" --trustAll --useStartTls -p 1389 -i 5 --noRebind --warmUpDuration 10 --maxDuration 90 --numConnections 64 --numConcurrentRequests 8 -b 'ou=people,ou=identities' -g 'rand(0,1000021)' '(uid=user.{})'", "out-dir": "/results/orders/DSPrimeTask_id_repo_0" } ________________________________________________________________________________ [06/Apr/2023 23:33:11] DSPrimeTask_id_repo_0 loop_until_ready : Loop until overseer pod is ready to accept order ________________________________________________________________________________ [loop_until]: curl -k https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" [loop_until]: (max_time=120, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- WAITING --- stderr --- % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 33 0 --:--:-- --:--:-- --:--:-- 33 ________________________________________________________________________________ [06/Apr/2023 23:33:11] DSPrimeTask_id_repo_0 start_order : Send the order to the overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6dc8b44f76-dp2dt --- stderr --- [loop_until]: kubectl --namespace=xlou cp /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0/order.json overseer-0-6dc8b44f76-dp2dt:/results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order has been successfully forwarded to overseer ________________________________________________________________________________ [06/Apr/2023 23:33:12] DSPrimeTask_id_repo_0 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 59s Waiting for status (curl --fail -k https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 5s, retry Order is running since 10s, retry Order is running since 15s, retry Order is running since 20s, retry Order is running since 26s, retry Order is running since 31s, retry Order is running since 36s, retry Order is running since 41s, retry Order is running since 46s, retry Order is running since 52s, retry Order is running since 57s, retry Order is running since 1m 02s, retry Order is running since 1m 07s, retry Order is running since 1m 13s, retry Order is running since 1m 18s, retry Order is running since 1m 23s, retry Order is running since 1m 28s, retry PASS : Order stopped by itself after 1m 33s and before timeout (59s) ________________________________________________________________________________ [06/Apr/2023 23:34:46] DSPrimeTask_id_repo_0 stop_order : Stop the order on overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6dc8b44f76-dp2dt --- stderr --- [loop_until]: kubectl --namespace=xlou exec overseer-0-6dc8b44f76-dp2dt -- rm -f /results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order ran during 1m 34s ________________________________________________________________________________ [06/Apr/2023 23:34:47] DSPrimeTask_id_repo_0 download_logs : Download order.tar.gz ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6dc8b44f76-dp2dt --- stderr --- Checking pod: overseer-0-6dc8b44f76-dp2dt for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6dc8b44f76-dp2dt --- stderr --- [loop_until]: kubectl --namespace=xlou exec overseer-0-6dc8b44f76-dp2dt -i -- bash -c 'if [ -f "/results/orders/DSPrimeTask_id_repo_0/order.tar.gz" ]; then echo "/results/orders/DSPrimeTask_id_repo_0/order.tar.gz file found"; exit 0; else echo "/results/orders/DSPrimeTask_id_repo_0/order.tar.gz file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_id_repo_0/order.tar.gz file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/DSPrimeTask_id_repo_0/order.tar.gz file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6dc8b44f76-dp2dt --- stderr --- [run_command]: kubectl --namespace=xlou exec overseer-0-6dc8b44f76-dp2dt -i -- du -k /results/orders/DSPrimeTask_id_repo_0/order.tar.gz [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 4 /results/orders/DSPrimeTask_id_repo_0/order.tar.gz --- stderr --- - Download file (size 4k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6dc8b44f76-dp2dt --- stderr --- source : /results/orders/DSPrimeTask_id_repo_0/order.tar.gz @ overseer-0-6dc8b44f76-dp2dt destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0/order.tar.gz [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6dc8b44f76-dp2dt --- stderr --- [loop_until]: kubectl --namespace=xlou cp overseer-0-6dc8b44f76-dp2dt:/results/orders/DSPrimeTask_id_repo_0/order.tar.gz /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0/order.tar.gz [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- - Extract archive zip file : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0/order.tar.gz destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0 ________________________________________________________________________________ [06/Apr/2023 23:34:51] DSPrimeTask_id_repo_0 show_cmd : Display order command ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0/cmd.txt is small (1 lines), printing all lines: ----- output ----- /lodestar/clients/ds/bin/searchrate --noPropertiesFile -h ds-idrepo-0.ds-idrepo -w "fwG9YpsslkrZHDsXR1RKtRiiUPWtG36g" -D "uid=admin" --trustAll --useStartTls -p 1389 -i 5 --noRebind --warmUpDuration 10 --maxDuration 90 --numConnections 64 --numConcurrentRequests 8 -b 'ou=people,ou=identities' -g 'rand(0,1000021)' '(uid=user.{})' ________________________________________________________________________________ [06/Apr/2023 23:34:51] DSPrimeTask_id_repo_0 show_rc : Display rc order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0/rc.txt is small (1 lines), printing all lines: ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [06/Apr/2023 23:34:51] DSPrimeTask_id_repo_0 show_stdout : Display stdout order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0/stdout.txt is small (22 lines), printing all lines: ----- output ----- Warming up for 10 seconds... -------------------------------------------------------------------------------------------- | Throughput | Response Time | Additional | | (ops/second) | (milliseconds) | Statistics | | recent average | recent average 99.9% 99.99% 99.999% | err/sec Entries/Srch | -------------------------------------------------------------------------------------------- | 83723.9 83723.9 | 6.126 6.126 112.20 247.46 312.48 | 0.0 1.0 | | 80463.3 82093.9 | 6.365 6.243 109.05 232.78 299.89 | 0.0 1.0 | | 83456.6 82548.1 | 6.129 6.205 109.05 204.47 293.60 | 0.0 1.0 | | 82442.8 82521.8 | 6.209 6.206 106.95 192.94 291.50 | 0.0 1.0 | | 86846.6 83386.8 | 5.893 6.141 105.38 170.92 291.50 | 0.0 1.0 | | 85425.2 83726.5 | 5.989 6.115 103.81 161.48 285.21 | 0.0 1.0 | | 83768.4 83732.5 | 6.108 6.114 102.76 149.95 283.12 | 0.0 1.0 | | 85610.4 83967.2 | 5.977 6.096 102.24 136.31 283.12 | 0.0 1.0 | | 84947.6 84076.2 | 6.010 6.087 101.19 131.07 281.02 | 0.0 1.0 | | 86576.8 84326.2 | 5.921 6.070 100.66 127.40 278.92 | 0.0 1.0 | | 84602.8 84351.4 | 6.043 6.067 99.61 125.83 276.82 | 0.0 1.0 | | 85411.8 84439.7 | 5.998 6.062 99.09 123.73 274.73 | 0.0 1.0 | | 85864.6 84549.3 | 5.942 6.052 98.57 122.16 274.73 | 0.0 1.0 | | 86119.8 84661.5 | 5.954 6.045 98.57 122.16 272.63 | 0.0 1.0 | | 84954.4 84681.0 | 6.027 6.044 98.57 120.59 272.63 | 0.0 1.0 | | 85441.7 84728.5 | 5.985 6.040 98.57 121.63 270.53 | 0.0 1.0 | ________________________________________________________________________________ [06/Apr/2023 23:34:51] DSPrimeTask_id_repo_0 show_stderr : Display stderr order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/03_DSPrimeTask_id_repo_0/stderr.txt is small (0 lines), printing all lines: ----- output ----- ________________________________________________________________________________ [06/Apr/2023 23:34:51] DSPrimeTask_id_repo_0 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [06/Apr/2023 23:34:51] DSPrimeTask_id_repo_0 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [06/Apr/2023 23:34:51] DSPrimeTask_id_repo_0 post : Check output ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped