--Task-- name: generate-ldif enabled: True class_name: GenerateLdifTask source_name: overseer-0 source_namespace: >default< target_name: overseer-0 target_namespace: >default< start: 0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: ['preload-with-gatling'] wait_for: [] options: {'ldif_file': '/results/ldap.ldif', 'num_users': 1000, 'csv_file': '/results/idm/user_feeder.csv'} group_name: None Current dir: /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/pyrock ________________________________________________________________________________ [19/Mar/2023 22:06:47] generate-ldif pre : N/A ________________________________________________________________________________ [19/Mar/2023 22:06:47] - INFO: Timeout for this Task has changed to 8h 20m (30000 seconds) [19/Mar/2023 22:06:47] - INFO: timeout was set to GenerateLdifTask default (num_users * 3 / 100 seconds or 60 seconds whatever is higher), because it was unset Pod log file : /results/orders/generate-ldif/stdout.txt Pod log : /results/orders/generate-ldif ________________________________________________________________________________ [19/Mar/2023 22:06:47] generate-ldif print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ python3 /lodestar/pyrock/shared/scripts/ds/gen_ldif.py -f /results/ldap.ldif -n 1000 -c /results/idm/user_feeder.csv ________________________________________________________________________________ [19/Mar/2023 22:06:47] generate-ldif create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file : /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif/order.json { "name": "generate-ldif", "command": "python3 /lodestar/pyrock/shared/scripts/ds/gen_ldif.py -f /results/ldap.ldif -n 1000 -c /results/idm/user_feeder.csv", "out-dir": "/results/orders/generate-ldif" } ________________________________________________________________________________ [19/Mar/2023 22:06:47] generate-ldif loop_until_ready : Loop until overseer pod is ready to accept order ________________________________________________________________________________ [loop_until]: curl -k https://overseer-0.jnkns-pt-dbg-tsts-dbg-thr-tsts-463.iam.perf-daily.engineeringpit.com/orders/status.txt | grep "WAITING" [loop_until]: (max_time=120, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- WAITING --- stderr --- % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 12 0 --:--:-- --:--:-- --:--:-- 12 ________________________________________________________________________________ [19/Mar/2023 22:06:48] generate-ldif start_order : Send the order to the overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5bcc86bb94-8h9vq --- stderr --- [loop_until]: kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 cp /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif/order.json overseer-0-5bcc86bb94-8h9vq:/results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order has been successfully forwarded to overseer ________________________________________________________________________________ [19/Mar/2023 22:06:50] generate-ldif wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 8h 20m Waiting for status (curl --fail -k https://overseer-0.jnkns-pt-dbg-tsts-dbg-thr-tsts-463.iam.perf-daily.engineeringpit.com/orders/status.txt) == WAITING Order is running since 5s, retry Order is running since 10s, retry Order is running since 16s, retry Order is running since 21s, retry Order is running since 27s, retry Order is running since 32s, retry Order is running since 37s, retry Order is running since 43s, retry Order is running since 48s, retry Order is running since 54s, retry Order is running since 59s, retry PASS : Order stopped by itself after 1m 04s and before timeout (8h 20m) ________________________________________________________________________________ [19/Mar/2023 22:07:55] generate-ldif stop_order : Stop the order on overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5bcc86bb94-8h9vq --- stderr --- [loop_until]: kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 exec overseer-0-5bcc86bb94-8h9vq -- rm -f /results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order ran during 1m 06s ________________________________________________________________________________ [19/Mar/2023 22:07:56] generate-ldif download_logs : Download order.tar.gz ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5bcc86bb94-8h9vq --- stderr --- Checking pod: overseer-0-5bcc86bb94-8h9vq for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5bcc86bb94-8h9vq --- stderr --- [loop_until]: kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 exec overseer-0-5bcc86bb94-8h9vq -i -- bash -c 'if [ -f "/results/orders/generate-ldif/order.tar.gz" ]; then echo "/results/orders/generate-ldif/order.tar.gz file found"; exit 0; else echo "/results/orders/generate-ldif/order.tar.gz file not found"; exit 1; fi' | grep "/results/orders/generate-ldif/order.tar.gz file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 1s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/generate-ldif/order.tar.gz file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5bcc86bb94-8h9vq --- stderr --- [run_command]: kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 exec overseer-0-5bcc86bb94-8h9vq -i -- du -k /results/orders/generate-ldif/order.tar.gz [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 4 /results/orders/generate-ldif/order.tar.gz --- stderr --- - Download file (size 4k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5bcc86bb94-8h9vq --- stderr --- source : /results/orders/generate-ldif/order.tar.gz @ overseer-0-5bcc86bb94-8h9vq destination : /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif/order.tar.gz [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 get pods -l app=overseer-0 -o jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5bcc86bb94-8h9vq --- stderr --- [loop_until]: kubectl --namespace=jnkns-pt-dbg-tsts-dbg-thr-tsts-463 cp overseer-0-5bcc86bb94-8h9vq:/results/orders/generate-ldif/order.tar.gz /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif/order.tar.gz [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- - Extract archive zip file : /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif/order.tar.gz destination : /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif ________________________________________________________________________________ [19/Mar/2023 22:08:01] generate-ldif show_cmd : Display order command ________________________________________________________________________________ [print_head_tail]: Input file /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif/cmd.txt is small (1 lines), printing all lines: ----- output ----- python3 /lodestar/pyrock/shared/scripts/ds/gen_ldif.py -f /results/ldap.ldif -n 1000 -c /results/idm/user_feeder.csv ________________________________________________________________________________ [19/Mar/2023 22:08:01] generate-ldif show_rc : Display rc order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif/rc.txt is small (1 lines), printing all lines: ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [19/Mar/2023 22:08:01] generate-ldif show_stdout : Display stdout order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif/stdout.txt is small (0 lines), printing all lines: ----- output ----- ________________________________________________________________________________ [19/Mar/2023 22:08:01] generate-ldif show_stderr : Display stderr order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/scratch/workspaces/workspace/pit-debug-tests-debug-other-tests-463-1679261215469-1550440051/results/pyrock/platform_login_pta/04_Data_init/03_generate-ldif/stderr.txt is small (0 lines), printing all lines: ----- output ----- ________________________________________________________________________________ [19/Mar/2023 22:08:01] generate-ldif check_rc_and_logs : Check order's outputs ________________________________________________________________________________ PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [19/Mar/2023 22:08:01] generate-ldif print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [19/Mar/2023 22:08:01] generate-ldif post : Check output ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped