--Task-- name: DSPrimeTask_id_repo_1 enabled: True class_name: DSSearchRateTask source_name: overseer-0 source_namespace: >default< target_name: ds-idrepo-1 target_namespace: >default< start: 0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: ['DSPrimeTask_id_repo_0'] wait_for: [] options: {'_options_': "--noRebind --warmUpDuration 10 --maxDuration 90 --numConnections 64 --numConcurrentRequests 8 -b 'ou=people,ou=identities'"} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [07/Apr/2023 20:02:20] DSPrimeTask_id_repo_1 pre : N/A ________________________________________________________________________________ [07/Apr/2023 20:02:20] - INFO: target (ds-idrepo-1) is a ds pod [07/Apr/2023 20:02:20] - INFO: Timeout for this Task has changed to 59s (-1 seconds) [07/Apr/2023 20:02:20] - INFO: timeout was set to unlimited, because it was unset Pod log file : /results/orders/DSPrimeTask_id_repo_1/stdout.txt Pod log : /results/orders/DSPrimeTask_id_repo_1 [run_command]: kubectl --namespace=xlou exec ds-idrepo-1 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "QH7lGWph4PYDPNyy25azR1uhrzDbTfCw" -b "ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor" -s base "objectclass=*" ds-mon-base-dn-entry-count [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor ds-mon-base-dn-entry-count: 1000021 --- stderr --- ________________________________________________________________________________ [07/Apr/2023 20:02:22] DSPrimeTask_id_repo_1 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ [run_command]: kubectl --namespace=xlou exec ds-idrepo-1 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "QH7lGWph4PYDPNyy25azR1uhrzDbTfCw" -b "ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor" -s base "objectclass=*" ds-mon-base-dn-entry-count [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor ds-mon-base-dn-entry-count: 1000021 --- stderr --- /lodestar/clients/ds/bin/searchrate --noPropertiesFile -h ds-idrepo-1.ds-idrepo -w "QH7lGWph4PYDPNyy25azR1uhrzDbTfCw" -D "uid=admin" --trustAll --useStartTls -p 1389 -i 5 --noRebind --warmUpDuration 10 --maxDuration 90 --numConnections 64 --numConcurrentRequests 8 -b 'ou=people,ou=identities' -g 'rand(0,1000021)' '(uid=user.{})' ________________________________________________________________________________ [07/Apr/2023 20:02:24] DSPrimeTask_id_repo_1 create_order : Create order for overseer with the command to run ________________________________________________________________________________ [run_command]: kubectl --namespace=xlou exec ds-idrepo-1 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "QH7lGWph4PYDPNyy25azR1uhrzDbTfCw" -b "ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor" -s base "objectclass=*" ds-mon-base-dn-entry-count [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ds-mon-base-dn=ou=identities,ds-cfg-backend-id=amIdentityStore,cn=backends,cn=monitor ds-mon-base-dn-entry-count: 1000021 --- stderr --- Order file : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1/order.json { "name": "DSPrimeTask_id_repo_1", "command": "/lodestar/clients/ds/bin/searchrate --noPropertiesFile -h ds-idrepo-1.ds-idrepo -w \"QH7lGWph4PYDPNyy25azR1uhrzDbTfCw\" -D \"uid=admin\" --trustAll --useStartTls -p 1389 -i 5 --noRebind --warmUpDuration 10 --maxDuration 90 --numConnections 64 --numConcurrentRequests 8 -b 'ou=people,ou=identities' -g 'rand(0,1000021)' '(uid=user.{})'", "out-dir": "/results/orders/DSPrimeTask_id_repo_1" } ________________________________________________________________________________ [07/Apr/2023 20:02:25] DSPrimeTask_id_repo_1 loop_until_ready : Loop until overseer pod is ready to accept order ________________________________________________________________________________ [loop_until]: curl -k https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" [loop_until]: (max_time=120, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- WAITING --- stderr --- % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 33 0 --:--:-- --:--:-- --:--:-- 33 ________________________________________________________________________________ [07/Apr/2023 20:02:26] DSPrimeTask_id_repo_1 start_order : Send the order to the overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-bbc6f89f5-98xnl --- stderr --- [loop_until]: kubectl --namespace=xlou cp /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1/order.json overseer-0-bbc6f89f5-98xnl:/results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order has been successfully forwarded to overseer ________________________________________________________________________________ [07/Apr/2023 20:02:27] DSPrimeTask_id_repo_1 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 59s Waiting for status (curl --fail -k https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 5s, retry Order is running since 10s, retry Order is running since 15s, retry Order is running since 20s, retry Order is running since 26s, retry Order is running since 31s, retry Order is running since 36s, retry Order is running since 41s, retry Order is running since 46s, retry Order is running since 52s, retry Order is running since 57s, retry Order is running since 1m 02s, retry Order is running since 1m 07s, retry Order is running since 1m 13s, retry Order is running since 1m 18s, retry Order is running since 1m 23s, retry Order is running since 1m 28s, retry PASS : Order stopped by itself after 1m 33s and before timeout (59s) ________________________________________________________________________________ [07/Apr/2023 20:04:01] DSPrimeTask_id_repo_1 stop_order : Stop the order on overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-bbc6f89f5-98xnl --- stderr --- [loop_until]: kubectl --namespace=xlou exec overseer-0-bbc6f89f5-98xnl -- rm -f /results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order ran during 1m 34s ________________________________________________________________________________ [07/Apr/2023 20:04:02] DSPrimeTask_id_repo_1 download_logs : Download order.tar.gz ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-bbc6f89f5-98xnl --- stderr --- Checking pod: overseer-0-bbc6f89f5-98xnl for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-bbc6f89f5-98xnl --- stderr --- [loop_until]: kubectl --namespace=xlou exec overseer-0-bbc6f89f5-98xnl -i -- bash -c 'if [ -f "/results/orders/DSPrimeTask_id_repo_1/order.tar.gz" ]; then echo "/results/orders/DSPrimeTask_id_repo_1/order.tar.gz file found"; exit 0; else echo "/results/orders/DSPrimeTask_id_repo_1/order.tar.gz file not found"; exit 1; fi' | grep "/results/orders/DSPrimeTask_id_repo_1/order.tar.gz file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/DSPrimeTask_id_repo_1/order.tar.gz file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-bbc6f89f5-98xnl --- stderr --- [run_command]: kubectl --namespace=xlou exec overseer-0-bbc6f89f5-98xnl -i -- du -k /results/orders/DSPrimeTask_id_repo_1/order.tar.gz [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 4 /results/orders/DSPrimeTask_id_repo_1/order.tar.gz --- stderr --- - Download file (size 4k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-bbc6f89f5-98xnl --- stderr --- source : /results/orders/DSPrimeTask_id_repo_1/order.tar.gz @ overseer-0-bbc6f89f5-98xnl destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1/order.tar.gz [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-bbc6f89f5-98xnl --- stderr --- [loop_until]: kubectl --namespace=xlou cp overseer-0-bbc6f89f5-98xnl:/results/orders/DSPrimeTask_id_repo_1/order.tar.gz /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1/order.tar.gz [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- - Extract archive zip file : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1/order.tar.gz destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1 ________________________________________________________________________________ [07/Apr/2023 20:04:06] DSPrimeTask_id_repo_1 show_cmd : Display order command ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1/cmd.txt is small (1 lines), printing all lines: ----- output ----- /lodestar/clients/ds/bin/searchrate --noPropertiesFile -h ds-idrepo-1.ds-idrepo -w "QH7lGWph4PYDPNyy25azR1uhrzDbTfCw" -D "uid=admin" --trustAll --useStartTls -p 1389 -i 5 --noRebind --warmUpDuration 10 --maxDuration 90 --numConnections 64 --numConcurrentRequests 8 -b 'ou=people,ou=identities' -g 'rand(0,1000021)' '(uid=user.{})' ________________________________________________________________________________ [07/Apr/2023 20:04:06] DSPrimeTask_id_repo_1 show_rc : Display rc order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1/rc.txt is small (1 lines), printing all lines: ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [07/Apr/2023 20:04:06] DSPrimeTask_id_repo_1 show_stdout : Display stdout order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1/stdout.txt is small (22 lines), printing all lines: ----- output ----- Warming up for 10 seconds... -------------------------------------------------------------------------------------------- | Throughput | Response Time | Additional | | (ops/second) | (milliseconds) | Statistics | | recent average | recent average 99.9% 99.99% 99.999% | err/sec Entries/Srch | -------------------------------------------------------------------------------------------- | 78018.6 78018.6 | 6.591 6.591 108.00 230.69 371.20 | 0.0 1.0 | | 78263.3 78140.9 | 6.529 6.560 105.38 214.96 360.71 | 0.0 1.0 | | 78687.2 78323.0 | 6.499 6.539 103.81 200.28 358.61 | 0.0 1.0 | | 77772.0 78185.3 | 6.575 6.548 102.76 186.65 354.42 | 0.0 1.0 | | 78924.6 78333.1 | 6.491 6.537 102.24 171.97 352.32 | 0.0 1.0 | | 80049.8 78619.2 | 6.402 6.514 101.71 162.53 350.22 | 0.0 1.0 | | 80329.6 78863.6 | 6.354 6.491 101.19 158.33 320.86 | 0.0 1.0 | | 80186.2 79028.9 | 6.396 6.479 100.66 153.09 306.18 | 0.0 1.0 | | 80050.0 79142.4 | 6.394 6.469 99.61 146.80 304.09 | 0.0 1.0 | | 79922.8 79220.4 | 6.392 6.461 99.61 133.17 289.41 | 0.0 1.0 | | 79356.8 79232.8 | 6.458 6.461 99.61 127.93 281.02 | 0.0 1.0 | | 76857.0 79034.8 | 6.644 6.476 99.61 123.21 278.92 | 0.0 1.0 | | 79419.2 79064.4 | 6.455 6.474 99.61 122.16 276.82 | 0.0 1.0 | | 78801.2 79045.6 | 6.492 6.476 99.61 120.59 265.29 | 0.0 1.0 | | 79241.2 79058.6 | 6.450 6.474 99.09 116.92 264.24 | 0.0 1.0 | | 80545.7 79151.4 | 6.363 6.467 99.09 116.39 263.19 | 0.0 1.0 | ________________________________________________________________________________ [07/Apr/2023 20:04:06] DSPrimeTask_id_repo_1 show_stderr : Display stderr order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/06_Validation/04_DSPrimeTask_id_repo_1/stderr.txt is small (0 lines), printing all lines: ----- output ----- ________________________________________________________________________________ [07/Apr/2023 20:04:06] DSPrimeTask_id_repo_1 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [07/Apr/2023 20:04:06] DSPrimeTask_id_repo_1 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [07/Apr/2023 20:04:06] DSPrimeTask_id_repo_1 post : Check output ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped