--Task-- name: preload-with-gatling enabled: True class_name: PreloadWithGatling source_name: overseer-0 source_namespace: >default< target_name: idm target_namespace: >default< start: 0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: ['prepare-workload'] wait_for: ['Pre-check Num Users'] options: {'check-before-preload': False, 'simulation': 'idm.simple_managed_users.ManagedUsersPreload', 'concurrency': 20, 'java-opts': '-Dcsvfile=/results/idm/preload.csv -Drealm=alpha'} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [2024-01-23 18:17:03] preload-with-gatling pre : N/A ________________________________________________________________________________ [2024-01-23 18:17:03] - INFO: Timeout for this Task has changed to 2h (7200 seconds) [2024-01-23 18:17:03] - INFO: timeout was set to GatlingTask default, because it was unset Pod log file : /results/orders/preload-with-gatling/stdout.txt Pod log : /results/orders/preload-with-gatling [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- ________________________________________________________________________________ [2024-01-23 18:17:04] preload-with-gatling print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- /lodestar/clients/gatling-for-pyrock.sh --run-mode local --simulation idm.simple_managed_users.ManagedUsersPreload --simulations-folder /lodestar/pyrock/shared/clients/gatling/ --run-description idm.simple_managed_users.ManagedUsersPreload@ema-perf-benchmark --results-folder /results/gatling/preload-with-gatling --extra-run-jvm-options "-Xmx4G" --no-reports ________________________________________________________________________________ [2024-01-23 18:17:04] preload-with-gatling create_order : Create order for overseer with the command to run ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- Order file: /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/order.json { "name": "preload-with-gatling", "command": " /lodestar/clients/gatling-for-pyrock.sh --run-mode local --simulation idm.simple_managed_users.ManagedUsersPreload --simulations-folder /lodestar/pyrock/shared/clients/gatling/ --run-description idm.simple_managed_users.ManagedUsersPreload@ema-perf-benchmark --results-folder /results/gatling/preload-with-gatling --extra-run-jvm-options \"-Xmx4G\" --no-reports", "out-dir": "/results/orders/preload-with-gatling", "environment": { "JAVA_OPTS": "-Dam_host=openam-ema-perf-benchmark.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@forgerock.com -Dcheck_before_preload=False -Dconcurrency=20 -Dcsvfile=/results/idm/preload.csv -Didc=true -Didm_host=openam-ema-perf-benchmark.forgeblocks.com -Dnum_users=1000000 -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -Duser_password=T35!tr0c-k123" } } ________________________________________________________________________________ [2024-01-23 18:17:04] preload-with-gatling loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" [loop_until]: (max_time=300, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- WAITING --- stderr --- % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 33 0 --:--:-- --:--:-- --:--:-- 33 ________________________________________________________________________________ [2024-01-23 18:17:04] preload-with-gatling start_order : Send the order to the overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/order.json overseer-0-6878479dbd-zw79z:/results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order has been successfully forwarded to overseer [2024-01-23 18:17:06] - INFO: Check if Gatling simulation started successfully [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z -- head /results/orders/preload-with-gatling/stdout.txt | grep "started..." [loop_until]: (max_time=300, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: started... - retry [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: started... - retry [loop_until]: Function succeeded after 12s (rc=0) - failed to find expected output: started... - retry [loop_until]: Function succeeded after 17s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- GATLING_HOME is set to /home/jenkins/lodestar/clients/gatling 18:17:09.874 [WARN ] i.g.c.ZincCompiler$ - -target is deprecated: Use -release instead to compile against the correct platform API. 18:17:19.156 [WARN ] i.g.c.ZincCompiler$ - one warning found Gatling 3.10.3 is available! (you're using 3.9.5) think time multiplier is 0x Simulation idm.simple_managed_users.ManagedUsersPreload started... 18:17:22.821 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one --- stderr --- ________________________________________________________________________________ [2024-01-23 18:17:24] preload-with-gatling wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 2h Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 5s, retry Order is running since 10s, retry Order is running since 15s, retry Order is running since 20s, retry Order is running since 26s, retry Order is running since 31s, retry Order is running since 36s, retry Order is running since 41s, retry Order is running since 46s, retry Order is running since 52s, retry Order is running since 57s, retry Order is running since 1m 02s, retry Order is running since 1m 07s, retry Order is running since 1m 13s, retry Order is running since 1m 18s, retry Order is running since 1m 23s, retry Order is running since 1m 28s, retry Order is running since 1m 33s, retry Order is running since 1m 39s, retry Order is running since 1m 44s, retry Order is running since 1m 49s, retry Order is running since 1m 54s, retry Order is running since 2m, retry Order is running since 2m 05s, retry Order is running since 2m 10s, retry Order is running since 2m 15s, retry Order is running since 2m 20s, retry Order is running since 2m 26s, retry Order is running since 2m 31s, retry Order is running since 2m 36s, retry Order is running since 2m 41s, retry Order is running since 2m 47s, retry Order is running since 2m 52s, retry Order is running since 2m 57s, retry Order is running since 3m 02s, retry Order is running since 3m 07s, retry Order is running since 3m 13s, retry Order is running since 3m 18s, retry Order is running since 3m 23s, retry Order is running since 3m 28s, retry Order is running since 3m 33s, retry Order is running since 3m 39s, retry Order is running since 3m 44s, retry Order is running since 3m 49s, retry Order is running since 3m 54s, retry Order is running since 4m, retry Order is running since 4m 05s, retry Order is running since 4m 10s, retry Order is running since 4m 15s, retry Order is running since 4m 20s, retry Order is running since 4m 26s, retry Order is running since 4m 31s, retry Order is running since 4m 36s, retry Order is running since 4m 41s, retry Order is running since 4m 47s, retry Order is running since 4m 52s, retry Order is running since 4m 57s, retry Order is running since 5m 02s, retry Order is running since 5m 07s, retry Order is running since 5m 13s, retry Order is running since 5m 18s, retry Order is running since 5m 23s, retry Order is running since 5m 28s, retry Order is running since 5m 34s, retry Order is running since 5m 39s, retry Order is running since 5m 44s, retry Order is running since 5m 49s, retry Order is running since 5m 54s, retry Order is running since 6m, retry Order is running since 6m 05s, retry Order is running since 6m 10s, retry Order is running since 6m 15s, retry Order is running since 6m 20s, retry Order is running since 6m 26s, retry Order is running since 6m 31s, retry Order is running since 6m 36s, retry Order is running since 6m 41s, retry Order is running since 6m 47s, retry Order is running since 6m 52s, retry Order is running since 6m 57s, retry Order is running since 7m 02s, retry Order is running since 7m 07s, retry Order is running since 7m 13s, retry Order is running since 7m 18s, retry Order is running since 7m 23s, retry Order is running since 7m 28s, retry Order is running since 7m 34s, retry Order is running since 7m 39s, retry Order is running since 7m 44s, retry Order is running since 7m 49s, retry Order is running since 7m 54s, retry Order is running since 8m, retry Order is running since 8m 05s, retry Order is running since 8m 10s, retry Order is running since 8m 15s, retry Order is running since 8m 21s, retry Order is running since 8m 26s, retry Order is running since 8m 31s, retry Order is running since 8m 36s, retry Order is running since 8m 41s, retry Order is running since 8m 47s, retry Order is running since 8m 52s, retry Order is running since 8m 57s, retry Order is running since 9m 02s, retry Order is running since 9m 08s, retry Order is running since 9m 13s, retry Order is running since 9m 18s, retry Order is running since 9m 23s, retry Order is running since 9m 28s, retry Order is running since 9m 34s, retry Order is running since 9m 39s, retry Order is running since 9m 44s, retry Order is running since 9m 49s, retry Order is running since 9m 54s, retry Order is running since 10m, retry Order is running since 10m 05s, retry Order is running since 10m 10s, retry Order is running since 10m 15s, retry Order is running since 10m 21s, retry Order is running since 10m 26s, retry Order is running since 10m 31s, retry Order is running since 10m 36s, retry Order is running since 10m 41s, retry Order is running since 10m 47s, retry Order is running since 10m 52s, retry Order is running since 10m 57s, retry Order is running since 11m 02s, retry Order is running since 11m 07s, retry Order is running since 11m 13s, retry Order is running since 11m 18s, retry Order is running since 11m 23s, retry Order is running since 11m 28s, retry Order is running since 11m 34s, retry Order is running since 11m 39s, retry Order is running since 11m 44s, retry Order is running since 11m 49s, retry Order is running since 11m 54s, retry Order is running since 12m, retry Order is running since 12m 05s, retry Order is running since 12m 10s, retry Order is running since 12m 15s, retry Order is running since 12m 21s, retry Order is running since 12m 26s, retry Order is running since 12m 31s, retry Order is running since 12m 36s, retry Order is running since 12m 41s, retry Order is running since 12m 47s, retry Order is running since 12m 52s, retry Order is running since 12m 57s, retry Order is running since 13m 02s, retry Order is running since 13m 08s, retry Order is running since 13m 13s, retry Order is running since 13m 18s, retry Order is running since 13m 23s, retry Order is running since 13m 28s, retry Order is running since 13m 34s, retry Order is running since 13m 39s, retry Order is running since 13m 44s, retry Order is running since 13m 49s, retry Order is running since 13m 55s, retry Order is running since 14m, retry Order is running since 14m 05s, retry Order is running since 14m 10s, retry Order is running since 14m 15s, retry Order is running since 14m 21s, retry Order is running since 14m 26s, retry Order is running since 14m 31s, retry Order is running since 14m 36s, retry Order is running since 14m 41s, retry Order is running since 14m 47s, retry Order is running since 14m 52s, retry Order is running since 14m 57s, retry Order is running since 15m 02s, retry Order is running since 15m 08s, retry Order is running since 15m 13s, retry Order is running since 15m 18s, retry Order is running since 15m 23s, retry Order is running since 15m 28s, retry Order is running since 15m 34s, retry Order is running since 15m 39s, retry Order is running since 15m 44s, retry Order is running since 15m 49s, retry Order is running since 15m 54s, retry Order is running since 16m, retry Order is running since 16m 05s, retry Order is running since 16m 10s, retry Order is running since 16m 15s, retry Order is running since 16m 21s, retry Order is running since 16m 26s, retry Order is running since 16m 31s, retry Order is running since 16m 36s, retry Order is running since 16m 41s, retry Order is running since 16m 47s, retry Order is running since 16m 52s, retry Order is running since 16m 57s, retry Order is running since 17m 02s, retry Order is running since 17m 07s, retry Order is running since 17m 13s, retry Order is running since 17m 18s, retry Order is running since 17m 23s, retry Order is running since 17m 28s, retry Order is running since 17m 34s, retry Order is running since 17m 39s, retry Order is running since 17m 44s, retry Order is running since 17m 49s, retry Order is running since 17m 54s, retry Order is running since 18m, retry Order is running since 18m 05s, retry Order is running since 18m 10s, retry Order is running since 18m 15s, retry Order is running since 18m 20s, retry Order is running since 18m 26s, retry Order is running since 18m 31s, retry Order is running since 18m 36s, retry Order is running since 18m 41s, retry Order is running since 18m 47s, retry Order is running since 18m 52s, retry Order is running since 18m 57s, retry Order is running since 19m 02s, retry Order is running since 19m 07s, retry Order is running since 19m 13s, retry Order is running since 19m 18s, retry Order is running since 19m 23s, retry Order is running since 19m 28s, retry Order is running since 19m 34s, retry Order is running since 19m 39s, retry Order is running since 19m 44s, retry Order is running since 19m 49s, retry Order is running since 19m 54s, retry Order is running since 20m, retry Order is running since 20m 05s, retry Order is running since 20m 10s, retry Order is running since 20m 15s, retry Order is running since 20m 21s, retry Order is running since 20m 26s, retry Order is running since 20m 31s, retry Order is running since 20m 36s, retry Order is running since 20m 41s, retry Order is running since 20m 47s, retry Order is running since 20m 52s, retry Order is running since 20m 57s, retry Order is running since 21m 02s, retry Order is running since 21m 07s, retry Order is running since 21m 13s, retry Order is running since 21m 18s, retry Order is running since 21m 23s, retry Order is running since 21m 28s, retry Order is running since 21m 34s, retry Order is running since 21m 39s, retry Order is running since 21m 44s, retry Order is running since 21m 49s, retry Order is running since 21m 54s, retry Order is running since 22m, retry Order is running since 22m 05s, retry Order is running since 22m 10s, retry Order is running since 22m 15s, retry Order is running since 22m 21s, retry Order is running since 22m 26s, retry Order is running since 22m 31s, retry Order is running since 22m 36s, retry Order is running since 22m 41s, retry Order is running since 22m 47s, retry Order is running since 22m 52s, retry Order is running since 22m 57s, retry Order is running since 23m 02s, retry Order is running since 23m 07s, retry Order is running since 23m 13s, retry Order is running since 23m 18s, retry Order is running since 23m 23s, retry Order is running since 23m 28s, retry Order is running since 23m 34s, retry Order is running since 23m 39s, retry Order is running since 23m 44s, retry Order is running since 23m 49s, retry Order is running since 23m 54s, retry Order is running since 24m, retry Order is running since 24m 05s, retry Order is running since 24m 10s, retry Order is running since 24m 15s, retry Order is running since 24m 21s, retry Order is running since 24m 26s, retry Order is running since 24m 31s, retry Order is running since 24m 36s, retry Order is running since 24m 41s, retry Order is running since 24m 47s, retry Order is running since 24m 52s, retry Order is running since 24m 57s, retry Order is running since 25m 02s, retry Order is running since 25m 08s, retry Order is running since 25m 13s, retry Order is running since 25m 18s, retry Order is running since 25m 23s, retry Order is running since 25m 28s, retry Order is running since 25m 34s, retry Order is running since 25m 39s, retry Order is running since 25m 44s, retry Order is running since 25m 49s, retry Order is running since 25m 54s, retry Order is running since 26m, retry Order is running since 26m 05s, retry Order is running since 26m 10s, retry Order is running since 26m 15s, retry Order is running since 26m 21s, retry Order is running since 26m 26s, retry Order is running since 26m 31s, retry Order is running since 26m 36s, retry Order is running since 26m 41s, retry Order is running since 26m 47s, retry Order is running since 26m 52s, retry Order is running since 26m 57s, retry Order is running since 27m 02s, retry Order is running since 27m 07s, retry Order is running since 27m 13s, retry Order is running since 27m 18s, retry Order is running since 27m 23s, retry Order is running since 27m 28s, retry Order is running since 27m 34s, retry Order is running since 27m 39s, retry Order is running since 27m 44s, retry Order is running since 27m 49s, retry Order is running since 27m 54s, retry Order is running since 28m, retry Order is running since 28m 05s, retry Order is running since 28m 10s, retry Order is running since 28m 15s, retry Order is running since 28m 20s, retry Order is running since 28m 26s, retry Order is running since 28m 31s, retry Order is running since 28m 36s, retry Order is running since 28m 41s, retry Order is running since 28m 47s, retry Order is running since 28m 52s, retry Order is running since 28m 57s, retry Order is running since 29m 02s, retry Order is running since 29m 07s, retry Order is running since 29m 13s, retry Order is running since 29m 18s, retry Order is running since 29m 23s, retry Order is running since 29m 28s, retry Order is running since 29m 34s, retry Order is running since 29m 39s, retry Order is running since 29m 44s, retry Order is running since 29m 49s, retry Order is running since 29m 54s, retry Order is running since 30m, retry Order is running since 30m 05s, retry Order is running since 30m 10s, retry Order is running since 30m 15s, retry Order is running since 30m 20s, retry Order is running since 30m 26s, retry Order is running since 30m 31s, retry Order is running since 30m 36s, retry Order is running since 30m 41s, retry Order is running since 30m 47s, retry Order is running since 30m 52s, retry Order is running since 30m 57s, retry Order is running since 31m 02s, retry Order is running since 31m 07s, retry Order is running since 31m 13s, retry Order is running since 31m 18s, retry Order is running since 31m 23s, retry Order is running since 31m 28s, retry Order is running since 31m 34s, retry Order is running since 31m 39s, retry Order is running since 31m 44s, retry Order is running since 31m 49s, retry Order is running since 31m 54s, retry Order is running since 32m, retry Order is running since 32m 05s, retry Order is running since 32m 10s, retry Order is running since 32m 15s, retry Order is running since 32m 20s, retry Order is running since 32m 26s, retry Order is running since 32m 31s, retry Order is running since 32m 36s, retry Order is running since 32m 41s, retry Order is running since 32m 47s, retry Order is running since 32m 52s, retry Order is running since 32m 57s, retry Order is running since 33m 02s, retry Order is running since 33m 07s, retry Order is running since 33m 13s, retry Order is running since 33m 18s, retry Order is running since 33m 23s, retry Order is running since 33m 28s, retry Order is running since 33m 34s, retry Order is running since 33m 39s, retry Order is running since 33m 44s, retry Order is running since 33m 49s, retry Order is running since 33m 54s, retry Order is running since 34m, retry Order is running since 34m 05s, retry Order is running since 34m 10s, retry Order is running since 34m 15s, retry Order is running since 34m 21s, retry Order is running since 34m 26s, retry Order is running since 34m 31s, retry Order is running since 34m 36s, retry Order is running since 34m 41s, retry Order is running since 34m 47s, retry Order is running since 34m 52s, retry Order is running since 34m 57s, retry Order is running since 35m 02s, retry Order is running since 35m 08s, retry Order is running since 35m 13s, retry Order is running since 35m 18s, retry Order is running since 35m 23s, retry Order is running since 35m 28s, retry Order is running since 35m 34s, retry Order is running since 35m 39s, retry Order is running since 35m 44s, retry Order is running since 35m 49s, retry Order is running since 35m 55s, retry Order is running since 36m, retry Order is running since 36m 05s, retry Order is running since 36m 10s, retry Order is running since 36m 15s, retry Order is running since 36m 21s, retry Order is running since 36m 26s, retry Order is running since 36m 31s, retry Order is running since 36m 36s, retry Order is running since 36m 42s, retry Order is running since 36m 47s, retry Order is running since 36m 52s, retry Order is running since 36m 57s, retry Order is running since 37m 02s, retry Order is running since 37m 08s, retry Order is running since 37m 13s, retry Order is running since 37m 18s, retry Order is running since 37m 23s, retry Order is running since 37m 29s, retry Order is running since 37m 34s, retry Order is running since 37m 39s, retry Order is running since 37m 44s, retry Order is running since 37m 49s, retry Order is running since 37m 55s, retry Order is running since 38m, retry Order is running since 38m 05s, retry Order is running since 38m 10s, retry Order is running since 38m 16s, retry Order is running since 38m 21s, retry Order is running since 38m 26s, retry Order is running since 38m 31s, retry Order is running since 38m 36s, retry Order is running since 38m 42s, retry Order is running since 38m 47s, retry Order is running since 38m 52s, retry Order is running since 38m 57s, retry Order is running since 39m 03s, retry Order is running since 39m 08s, retry Order is running since 39m 13s, retry Order is running since 39m 18s, retry Order is running since 39m 23s, retry Order is running since 39m 29s, retry Order is running since 39m 34s, retry Order is running since 39m 39s, retry Order is running since 39m 44s, retry Order is running since 39m 49s, retry Order is running since 39m 55s, retry Order is running since 40m, retry Order is running since 40m 05s, retry Order is running since 40m 10s, retry Order is running since 40m 16s, retry Order is running since 40m 21s, retry Order is running since 40m 26s, retry Order is running since 40m 31s, retry Order is running since 40m 36s, retry Order is running since 40m 42s, retry Order is running since 40m 47s, retry Order is running since 40m 52s, retry Order is running since 40m 57s, retry Order is running since 41m 03s, retry Order is running since 41m 08s, retry Order is running since 41m 13s, retry Order is running since 41m 18s, retry Order is running since 41m 23s, retry Order is running since 41m 29s, retry Order is running since 41m 34s, retry Order is running since 41m 39s, retry Order is running since 41m 44s, retry Order is running since 41m 50s, retry Order is running since 41m 55s, retry Order is running since 42m, retry Order is running since 42m 05s, retry Order is running since 42m 10s, retry Order is running since 42m 16s, retry Order is running since 42m 21s, retry Order is running since 42m 26s, retry Order is running since 42m 31s, retry Order is running since 42m 37s, retry Order is running since 42m 42s, retry Order is running since 42m 47s, retry Order is running since 42m 52s, retry Order is running since 42m 57s, retry Order is running since 43m 03s, retry Order is running since 43m 08s, retry Order is running since 43m 13s, retry Order is running since 43m 18s, retry Order is running since 43m 24s, retry Order is running since 43m 29s, retry Order is running since 43m 34s, retry Order is running since 43m 39s, retry Order is running since 43m 44s, retry Order is running since 43m 50s, retry Order is running since 43m 55s, retry Order is running since 44m, retry Order is running since 44m 05s, retry Order is running since 44m 11s, retry Order is running since 44m 16s, retry Order is running since 44m 21s, retry Order is running since 44m 26s, retry Order is running since 44m 31s, retry Order is running since 44m 37s, retry Order is running since 44m 42s, retry Order is running since 44m 47s, retry Order is running since 44m 52s, retry Order is running since 44m 58s, retry Order is running since 45m 03s, retry Order is running since 45m 08s, retry Order is running since 45m 13s, retry Order is running since 45m 18s, retry Order is running since 45m 24s, retry Order is running since 45m 29s, retry Order is running since 45m 34s, retry Order is running since 45m 39s, retry Order is running since 45m 44s, retry Order is running since 45m 50s, retry Order is running since 45m 55s, retry Order is running since 46m, retry Order is running since 46m 05s, retry Order is running since 46m 11s, retry Order is running since 46m 16s, retry Order is running since 46m 21s, retry Order is running since 46m 26s, retry Order is running since 46m 31s, retry Order is running since 46m 37s, retry Order is running since 46m 42s, retry Order is running since 46m 47s, retry Order is running since 46m 52s, retry Order is running since 46m 57s, retry Order is running since 47m 03s, retry Order is running since 47m 08s, retry Order is running since 47m 13s, retry Order is running since 47m 18s, retry Order is running since 47m 24s, retry Order is running since 47m 29s, retry Order is running since 47m 34s, retry Order is running since 47m 39s, retry Order is running since 47m 44s, retry Order is running since 47m 50s, retry Order is running since 47m 55s, retry Order is running since 48m, retry Order is running since 48m 05s, retry Order is running since 48m 11s, retry Order is running since 48m 16s, retry Order is running since 48m 21s, retry Order is running since 48m 26s, retry Order is running since 48m 31s, retry Order is running since 48m 37s, retry Order is running since 48m 42s, retry Order is running since 48m 47s, retry Order is running since 48m 52s, retry Order is running since 48m 58s, retry Order is running since 49m 03s, retry Order is running since 49m 08s, retry Order is running since 49m 13s, retry Order is running since 49m 18s, retry Order is running since 49m 24s, retry Order is running since 49m 29s, retry Order is running since 49m 34s, retry Order is running since 49m 39s, retry Order is running since 49m 45s, retry Order is running since 49m 50s, retry Order is running since 49m 55s, retry Order is running since 50m, retry Order is running since 50m 05s, retry Order is running since 50m 11s, retry Order is running since 50m 16s, retry Order is running since 50m 21s, retry Order is running since 50m 26s, retry Order is running since 50m 32s, retry Order is running since 50m 37s, retry Order is running since 50m 42s, retry Order is running since 50m 47s, retry Order is running since 50m 52s, retry Order is running since 50m 58s, retry Order is running since 51m 03s, retry Order is running since 51m 08s, retry Order is running since 51m 13s, retry Order is running since 51m 19s, retry Order is running since 51m 24s, retry Order is running since 51m 29s, retry Order is running since 51m 34s, retry Order is running since 51m 39s, retry Order is running since 51m 45s, retry Order is running since 51m 50s, retry Order is running since 51m 55s, retry Order is running since 52m, retry Order is running since 52m 05s, retry Order is running since 52m 11s, retry Order is running since 52m 16s, retry Order is running since 52m 21s, retry Order is running since 52m 26s, retry Order is running since 52m 32s, retry Order is running since 52m 37s, retry Order is running since 52m 42s, retry Order is running since 52m 47s, retry Order is running since 52m 52s, retry Order is running since 52m 58s, retry Order is running since 53m 03s, retry Order is running since 53m 08s, retry Order is running since 53m 13s, retry Order is running since 53m 19s, retry Order is running since 53m 24s, retry Order is running since 53m 29s, retry Order is running since 53m 34s, retry Order is running since 53m 39s, retry Order is running since 53m 45s, retry Order is running since 53m 50s, retry Order is running since 53m 55s, retry Order is running since 54m, retry Order is running since 54m 06s, retry Order is running since 54m 11s, retry Order is running since 54m 16s, retry Order is running since 54m 21s, retry Order is running since 54m 26s, retry Order is running since 54m 32s, retry Order is running since 54m 37s, retry Order is running since 54m 42s, retry Order is running since 54m 47s, retry Order is running since 54m 53s, retry Order is running since 54m 58s, retry Order is running since 55m 03s, retry Order is running since 55m 08s, retry Order is running since 55m 13s, retry Order is running since 55m 19s, retry Order is running since 55m 24s, retry Order is running since 55m 29s, retry Order is running since 55m 34s, retry Order is running since 55m 40s, retry Order is running since 55m 45s, retry Order is running since 55m 50s, retry Order is running since 55m 55s, retry Order is running since 56m, retry Order is running since 56m 06s, retry Order is running since 56m 11s, retry Order is running since 56m 16s, retry Order is running since 56m 21s, retry Order is running since 56m 27s, retry Order is running since 56m 32s, retry Order is running since 56m 37s, retry Order is running since 56m 42s, retry Order is running since 56m 47s, retry Order is running since 56m 53s, retry Order is running since 56m 58s, retry Order is running since 57m 03s, retry Order is running since 57m 08s, retry Order is running since 57m 14s, retry Order is running since 57m 19s, retry Order is running since 57m 24s, retry Order is running since 57m 29s, retry Order is running since 57m 34s, retry Order is running since 57m 40s, retry Order is running since 57m 45s, retry Order is running since 57m 50s, retry Order is running since 57m 55s, retry Order is running since 58m, retry Order is running since 58m 06s, retry Order is running since 58m 11s, retry Order is running since 58m 16s, retry Order is running since 58m 21s, retry Order is running since 58m 27s, retry Order is running since 58m 32s, retry Order is running since 58m 37s, retry Order is running since 58m 42s, retry Order is running since 58m 47s, retry Order is running since 58m 53s, retry Order is running since 58m 58s, retry Order is running since 59m 03s, retry Order is running since 59m 08s, retry Order is running since 59m 14s, retry Order is running since 59m 19s, retry Order is running since 59m 24s, retry Order is running since 59m 29s, retry Order is running since 59m 34s, retry Order is running since 59m 40s, retry Order is running since 59m 45s, retry Order is running since 59m 50s, retry Order is running since 59m 55s, retry PASS : Order stopped by itself after 1h and before timeout (2h) ________________________________________________________________________________ [2024-01-23 19:17:25] preload-with-gatling stop_order : Stop the order on overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z -- rm -f /results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order ran during 1h ________________________________________________________________________________ [2024-01-23 19:17:26] preload-with-gatling download_logs : Download individual order files ________________________________________________________________________________ [2024-01-23 19:17:26] - PASS: file already downloaded /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/order.json [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- Checking pod: overseer-0-6878479dbd-zw79z for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z --stdin -- bash -c 'if [ -f "/results/orders/preload-with-gatling/rc.txt" ]; then echo "/results/orders/preload-with-gatling/rc.txt file found"; exit 0; else echo "/results/orders/preload-with-gatling/rc.txt file not found"; exit 1; fi' | grep "/results/orders/preload-with-gatling/rc.txt file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/preload-with-gatling/rc.txt file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z --stdin -- du -k /results/orders/preload-with-gatling/rc.txt [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 4 /results/orders/preload-with-gatling/rc.txt --- stderr --- - Download file (size 4k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- source : /results/orders/preload-with-gatling/rc.txt @ overseer-0-6878479dbd-zw79z destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/rc.txt [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-6878479dbd-zw79z:/results/orders/preload-with-gatling/rc.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/rc.txt [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- Checking pod: overseer-0-6878479dbd-zw79z for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z --stdin -- bash -c 'if [ -f "/results/orders/preload-with-gatling/stderr.txt" ]; then echo "/results/orders/preload-with-gatling/stderr.txt file found"; exit 0; else echo "/results/orders/preload-with-gatling/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/preload-with-gatling/stderr.txt file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/preload-with-gatling/stderr.txt file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z --stdin -- du -k /results/orders/preload-with-gatling/stderr.txt [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 0 /results/orders/preload-with-gatling/stderr.txt --- stderr --- - Download file (size 0k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- source : /results/orders/preload-with-gatling/stderr.txt @ overseer-0-6878479dbd-zw79z destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/stderr.txt [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-6878479dbd-zw79z:/results/orders/preload-with-gatling/stderr.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/stderr.txt [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- Checking pod: overseer-0-6878479dbd-zw79z for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z --stdin -- bash -c 'if [ -f "/results/orders/preload-with-gatling/stdout.txt" ]; then echo "/results/orders/preload-with-gatling/stdout.txt file found"; exit 0; else echo "/results/orders/preload-with-gatling/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/preload-with-gatling/stdout.txt file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/preload-with-gatling/stdout.txt file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z --stdin -- du -k /results/orders/preload-with-gatling/stdout.txt [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 1308 /results/orders/preload-with-gatling/stdout.txt --- stderr --- - Download file (size 1308k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- source : /results/orders/preload-with-gatling/stdout.txt @ overseer-0-6878479dbd-zw79z destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/stdout.txt [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-6878479dbd-zw79z:/results/orders/preload-with-gatling/stdout.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/stdout.txt [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- ________________________________________________________________________________ [2024-01-23 19:17:38] preload-with-gatling show_cmd : Display order command ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- Checking pod: overseer-0-6878479dbd-zw79z for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z --stdin -- bash -c 'if [ -f "/results/orders/preload-with-gatling/cmd.txt" ]; then echo "/results/orders/preload-with-gatling/cmd.txt file found"; exit 0; else echo "/results/orders/preload-with-gatling/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/preload-with-gatling/cmd.txt file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/preload-with-gatling/cmd.txt file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-6878479dbd-zw79z --stdin -- du -k /results/orders/preload-with-gatling/cmd.txt [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 4 /results/orders/preload-with-gatling/cmd.txt --- stderr --- - Download file (size 4k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- source : /results/orders/preload-with-gatling/cmd.txt @ overseer-0-6878479dbd-zw79z destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/cmd.txt [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-6878479dbd-zw79z --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-6878479dbd-zw79z:/results/orders/preload-with-gatling/cmd.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/cmd.txt [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/cmd.txt is small (1 lines), printing all lines: ----- output ----- /lodestar/clients/gatling-for-pyrock.sh --run-mode local --simulation idm.simple_managed_users.ManagedUsersPreload --simulations-folder /lodestar/pyrock/shared/clients/gatling/ --run-description idm.simple_managed_users.ManagedUsersPreload@ema-perf-benchmark --results-folder /results/gatling/preload-with-gatling --extra-run-jvm-options "-Xmx4G" --no-reports ________________________________________________________________________________ [2024-01-23 19:17:42] preload-with-gatling show_rc : Display rc order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/rc.txt is small (1 lines), printing all lines: ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [2024-01-23 19:17:42] preload-with-gatling show_stdout : Display stdout order ________________________________________________________________________________ [print_head_tail]: Print head (100 lines) and tail (100 lines) of input file (/mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/stdout.txt is 16411 lines): ----- output ----- GATLING_HOME is set to /home/jenkins/lodestar/clients/gatling 18:17:09.874 [WARN ] i.g.c.ZincCompiler$ - -target is deprecated: Use -release instead to compile against the correct platform API. 18:17:19.156 [WARN ] i.g.c.ZincCompiler$ - one warning found Gatling 3.10.3 is available! (you're using 3.9.5) think time multiplier is 0x Simulation idm.simple_managed_users.ManagedUsersPreload started... 18:17:22.821 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.821 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.821 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.822 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.825 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.826 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.886 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.886 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.886 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.892 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.895 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.895 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.897 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.898 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.951 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.952 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.953 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.963 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.963 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:22.964 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.018 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.026 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.030 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.089 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.090 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.151 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.162 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.221 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.229 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.283 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.295 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.348 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.359 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.403 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.423 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.465 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.487 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.525 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.550 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.585 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one 18:17:23.609 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-2' crashed with 'j.u.NoSuchElementException: No attribute named 'id' is defined', forwarding to the next one ================================================================================ 2024-01-23 18:17:27 5s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=1266 KO=0 ) > restLoginInitiate (OK=20 KO=0 ) > restLoginUsernamePasswordCallback (OK=20 KO=0 ) > Skip 2FA (OK=20 KO=0 ) > User Authorize PKCE stage (OK=20 KO=0 ) > User AccessToken PKCE stage (OK=20 KO=0 ) > Create Managed User (OK=1166 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 20 / done: 0 ================================================================================ ================================================================================ 2024-01-23 18:17:32 10s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=2644 KO=0 ) > restLoginInitiate (OK=20 KO=0 ) > restLoginUsernamePasswordCallback (OK=20 KO=0 ) > Skip 2FA (OK=20 KO=0 ) > User Authorize PKCE stage (OK=20 KO=0 ) > User AccessToken PKCE stage (OK=20 KO=0 ) > Create Managed User (OK=2544 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 20 / done: 0 ================================================================================ ================================================================================ 2024-01-23 18:17:37 15s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=4024 KO=0 ) > restLoginInitiate (OK=20 KO=0 ) > restLoginUsernamePasswordCallback (OK=20 KO=0 ) > Skip 2FA (OK=20 KO=0 ) > User Authorize PKCE stage (OK=20 KO=0 ) > User AccessToken PKCE stage (OK=20 KO=0 ) > Create Managed User (OK=3924 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% [...] > Skip 2FA (OK=20 KO=0 ) > User Authorize PKCE stage (OK=20 KO=0 ) > User AccessToken PKCE stage (OK=20 KO=0 ) > Create Managed User (OK=933279 KO=54 ) ---- Errors -------------------------------------------------------------------- > status.find.in(201,403,401), but actually found 502 28 (51.85%) > Request timeout to openam-ema-perf-benchmark.forgeblocks.com/3 24 (44.44%) 4.49.170.241:443 after 60000 ms > status.find.in(201,403,401), but actually found 500 2 ( 3.70%) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 20 / done: 0 ================================================================================ ================================================================================ 2024-01-23 19:17:17 3595s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=934537 KO=54 ) > restLoginInitiate (OK=20 KO=0 ) > restLoginUsernamePasswordCallback (OK=20 KO=0 ) > Skip 2FA (OK=20 KO=0 ) > User Authorize PKCE stage (OK=20 KO=0 ) > User AccessToken PKCE stage (OK=20 KO=0 ) > Create Managed User (OK=934437 KO=54 ) ---- Errors -------------------------------------------------------------------- > status.find.in(201,403,401), but actually found 502 28 (51.85%) > Request timeout to openam-ema-perf-benchmark.forgeblocks.com/3 24 (44.44%) 4.49.170.241:443 after 60000 ms > status.find.in(201,403,401), but actually found 500 2 ( 3.70%) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 20 / done: 0 ================================================================================ ================================================================================ 2024-01-23 19:17:22 3600s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=935700 KO=54 ) > restLoginInitiate (OK=20 KO=0 ) > restLoginUsernamePasswordCallback (OK=20 KO=0 ) > Skip 2FA (OK=20 KO=0 ) > User Authorize PKCE stage (OK=20 KO=0 ) > User AccessToken PKCE stage (OK=20 KO=0 ) > Create Managed User (OK=935600 KO=54 ) ---- Errors -------------------------------------------------------------------- > status.find.in(201,403,401), but actually found 502 28 (51.85%) > Request timeout to openam-ema-perf-benchmark.forgeblocks.com/3 24 (44.44%) 4.49.170.241:443 after 60000 ms > status.find.in(201,403,401), but actually found 500 2 ( 3.70%) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 20 / done: 0 ================================================================================ ================================================================================ 2024-01-23 19:17:24 3602s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=935722 KO=54 ) > restLoginInitiate (OK=40 KO=0 ) > restLoginUsernamePasswordCallback (OK=20 KO=0 ) > Skip 2FA (OK=20 KO=0 ) > User Authorize PKCE stage (OK=20 KO=0 ) > User AccessToken PKCE stage (OK=20 KO=0 ) > Create Managed User (OK=935602 KO=54 ) ---- Errors -------------------------------------------------------------------- > status.find.in(201,403,401), but actually found 502 28 (51.85%) > Request timeout to openam-ema-perf-benchmark.forgeblocks.com/3 24 (44.44%) 4.49.170.241:443 after 60000 ms > status.find.in(201,403,401), but actually found 500 2 ( 3.70%) ---- Managed User Create Preload ----------------------------------------------- [##########################################################################]100% waiting: 0 / active: 0 / done: 20 ================================================================================ Simulation idm.simple_managed_users.ManagedUsersPreload completed in 3602 seconds ---------------------- [23/Jan/2024 19:17:24] Create tar file preload-with-gatling.tar.gz ---------------------- RESULTS_FOLDER=/results/gatling/ RESULT_FOLDER=preload-with-gatling SIMULATION_PATH=preload-with-gatling/manageduserspreload-20240123181721367 PWD=/results/gatling/preload-with-gatling/manageduserspreload-20240123181721367 [cmd] tar -czvf /results/gatling//preload-with-gatling.tar.gz --exclude=simulation.log * /results/gatling ---------------------- [23/Jan/2024 19:17:24] Update /results/gatling//latest ---------------------- ---------------------- [23/Jan/2024 19:17:24] Update /results/gatling//latest.tar.gz ---------------------- ________________________________________________________________________________ [2024-01-23 19:17:42] preload-with-gatling show_stderr : Display stderr order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/platform/05_Data_init/04_preload-with-gatling/stderr.txt is small (0 lines), printing all lines: ----- output ----- ________________________________________________________________________________ [2024-01-23 19:17:42] preload-with-gatling check_rc_and_logs : Check order's outputs ________________________________________________________________________________ PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2024-01-23 19:17:42] preload-with-gatling print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2024-01-23 19:17:42] preload-with-gatling post : N/A ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped