--Task-- name: preload-with-gatling enabled: True class_name: GatlingTask source_name: overseer-0 source_namespace: >default< target_name: idm target_namespace: >default< start: 0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: ['prepare-workload'] wait_for: ['Pre-check Num Users'] options: {'check-before-preload': True, 'simulation': 'idm.simple_managed_users.ManagedUsersPreload', 'concurrency': 35, 'java-opts': '-Dcsvfile=/results/idm/preload.csv -Doutput=/results/idm/user_feeder.csv -Drealm=alpha -Duser_password=Pa_ssw0rd'} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [05/Sep/2023 17:15:22] preload-with-gatling pre : N/A ________________________________________________________________________________ [05/Sep/2023 17:15:22] - INFO: Timeout for this Task has changed to 2h (7200 seconds) [05/Sep/2023 17:15:22] - INFO: timeout was set to GatlingTask default, because it was unset Pod log file : /results/orders/preload-with-gatling/stdout.txt Pod log : /results/orders/preload-with-gatling ________________________________________________________________________________ [05/Sep/2023 17:15:22] preload-with-gatling print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/clients/gatling-for-pyrock.sh --run-mode local --simulation idm.simple_managed_users.ManagedUsersPreload --simulations-folder /lodestar/pyrock/shared/clients/gatling/ --run-description idm.simple_managed_users.ManagedUsersPreload@openam-ema-perf-benchmark --results-folder /results/gatling/preload-with-gatling --extra-run-jvm-options "-Xmx4G" --no-reports ________________________________________________________________________________ [05/Sep/2023 17:15:22] preload-with-gatling create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling/order.json { "name": "preload-with-gatling", "command": " /lodestar/clients/gatling-for-pyrock.sh --run-mode local --simulation idm.simple_managed_users.ManagedUsersPreload --simulations-folder /lodestar/pyrock/shared/clients/gatling/ --run-description idm.simple_managed_users.ManagedUsersPreload@openam-ema-perf-benchmark --results-folder /results/gatling/preload-with-gatling --extra-run-jvm-options \"-Xmx4G\" --no-reports", "out-dir": "/results/orders/preload-with-gatling", "environment": { "JAVA_OPTS": "-Dam_host=openam-ema-perf-benchmark.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@forgerock.com -Dcheck_before_preload=True -Dconcurrency=35 -Dcsvfile=/results/idm/preload.csv -Didc=true -Didm_host=openam-ema-perf-benchmark.forgeblocks.com -Dnum_users=200000 -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Doutput=/results/idm/user_feeder.csv -Drealm=alpha -Duser_password=Pa_ssw0rd" } } ________________________________________________________________________________ [05/Sep/2023 17:15:22] preload-with-gatling loop_until_ready : Loop until overseer pod is ready to accept order ________________________________________________________________________________ [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" [loop_until]: (max_time=120, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- WAITING --- stderr --- % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 27 0 --:--:-- --:--:-- --:--:-- 27 ________________________________________________________________________________ [05/Sep/2023 17:15:22] preload-with-gatling start_order : Send the order to the overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5c8fd85b4c-98j79 --- stderr --- [loop_until]: kubectl --namespace=xlou cp /mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling/order.json overseer-0-5c8fd85b4c-98j79:/results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order has been successfully forwarded to overseer [05/Sep/2023 17:15:24] - INFO: Check if Gatling simulation started successfully [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5c8fd85b4c-98j79 --- stderr --- [loop_until]: kubectl --namespace=xlou exec overseer-0-5c8fd85b4c-98j79 -- head /results/orders/preload-with-gatling/stdout.txt | grep "started..." [loop_until]: (max_time=300, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: started... - retry [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: started... - retry [loop_until]: Function succeeded after 12s (rc=0) - failed to find expected output: started... - retry [loop_until]: Function succeeded after 17s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- GATLING_HOME is set to /home/jenkins/lodestar/clients/gatling 18:15:28.133 [WARN ] i.g.c.ZincCompiler$ - -target is deprecated: Use -release instead to compile against the correct platform API. 18:15:37.894 [WARN ] i.g.c.ZincCompiler$ - one warning found think time multiplier is 0x 18:15:40.459 [WARN ] i.g.c.s.e.ElCompiler$ - You're still using the deprecated ${} pattern for Gatling EL. Please use the #{} pattern instead. Simulation idm.simple_managed_users.ManagedUsersPreload started... --- stderr --- ________________________________________________________________________________ [05/Sep/2023 17:15:42] preload-with-gatling wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 2h Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 2m, retry Order is running since 4m, retry Order is running since 6m 01s, retry Order is running since 8m 01s, retry PASS : Order stopped by itself after 10m 01s and before timeout (2h) ________________________________________________________________________________ [05/Sep/2023 17:25:43] preload-with-gatling stop_order : Stop the order on overseer pod ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5c8fd85b4c-98j79 --- stderr --- [loop_until]: kubectl --namespace=xlou exec overseer-0-5c8fd85b4c-98j79 -- rm -f /results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order ran during 10m 21s ________________________________________________________________________________ [05/Sep/2023 17:25:45] preload-with-gatling download_logs : Download order.tar.gz ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5c8fd85b4c-98j79 --- stderr --- Checking pod: overseer-0-5c8fd85b4c-98j79 for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5c8fd85b4c-98j79 --- stderr --- [loop_until]: kubectl --namespace=xlou exec overseer-0-5c8fd85b4c-98j79 --stdin -- bash -c 'if [ -f "/results/orders/preload-with-gatling/order.tar.gz" ]; then echo "/results/orders/preload-with-gatling/order.tar.gz file found"; exit 0; else echo "/results/orders/preload-with-gatling/order.tar.gz file not found"; exit 1; fi' | grep "/results/orders/preload-with-gatling/order.tar.gz file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/preload-with-gatling/order.tar.gz file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5c8fd85b4c-98j79 --- stderr --- [run_command]: kubectl --namespace=xlou exec overseer-0-5c8fd85b4c-98j79 --stdin -- du -k /results/orders/preload-with-gatling/order.tar.gz [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 8 /results/orders/preload-with-gatling/order.tar.gz --- stderr --- - Download file (size 8k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5c8fd85b4c-98j79 --- stderr --- source : /results/orders/preload-with-gatling/order.tar.gz @ overseer-0-5c8fd85b4c-98j79 destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling/order.tar.gz [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5c8fd85b4c-98j79 --- stderr --- [loop_until]: kubectl --namespace=xlou cp overseer-0-5c8fd85b4c-98j79:/results/orders/preload-with-gatling/order.tar.gz /mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling/order.tar.gz [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- - Extract archive zip file : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling/order.tar.gz destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling ________________________________________________________________________________ [05/Sep/2023 17:25:48] preload-with-gatling show_cmd : Display order command ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling/cmd.txt is small (1 lines), printing all lines: ----- output ----- /lodestar/clients/gatling-for-pyrock.sh --run-mode local --simulation idm.simple_managed_users.ManagedUsersPreload --simulations-folder /lodestar/pyrock/shared/clients/gatling/ --run-description idm.simple_managed_users.ManagedUsersPreload@openam-ema-perf-benchmark --results-folder /results/gatling/preload-with-gatling --extra-run-jvm-options "-Xmx4G" --no-reports ________________________________________________________________________________ [05/Sep/2023 17:25:48] preload-with-gatling show_rc : Display rc order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling/rc.txt is small (1 lines), printing all lines: ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [05/Sep/2023 17:25:48] preload-with-gatling show_stdout : Display stdout order ________________________________________________________________________________ [print_head_tail]: Print head (100 lines) and tail (100 lines) of input file (/mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling/stdout.txt is 2048 lines): ----- output ----- GATLING_HOME is set to /home/jenkins/lodestar/clients/gatling 18:15:28.133 [WARN ] i.g.c.ZincCompiler$ - -target is deprecated: Use -release instead to compile against the correct platform API. 18:15:37.894 [WARN ] i.g.c.ZincCompiler$ - one warning found think time multiplier is 0x 18:15:40.459 [WARN ] i.g.c.s.e.ElCompiler$ - You're still using the deprecated ${} pattern for Gatling EL. Please use the #{} pattern instead. Simulation idm.simple_managed_users.ManagedUsersPreload started... ================================================================================ 2023-09-05 18:15:46 5s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=862 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=687 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 35 / done: 0 ================================================================================ ================================================================================ 2023-09-05 18:15:51 10s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=2103 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=1928 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 35 / done: 0 ================================================================================ ================================================================================ 2023-09-05 18:15:56 15s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=3172 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=2997 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 35 / done: 0 ================================================================================ ================================================================================ 2023-09-05 18:16:01 20s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=4068 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=3893 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 35 / done: 0 ================================================================================ ================================================================================ 2023-09-05 18:16:06 25s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=5617 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=5442 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 35 / done: 0 ================================================================================ ================================================================================ 2023-09-05 18:16:11 30s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=7320 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) [...] > Global (OK=194728 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=194553 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 35 / done: 0 ================================================================================ ================================================================================ 2023-09-05 18:25:21 580s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=196376 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=196201 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 35 / done: 0 ================================================================================ ================================================================================ 2023-09-05 18:25:26 585s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=198010 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=197835 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 35 / done: 0 ================================================================================ ================================================================================ 2023-09-05 18:25:31 590s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=199650 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=199475 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [--------------------------------------------------------------------------] 0% waiting: 0 / active: 35 / done: 0 ================================================================================ ================================================================================ 2023-09-05 18:25:33 591s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=200175 KO=0 ) > restLoginInitiate (OK=35 KO=0 ) > restLoginUsernamePasswordCallback (OK=35 KO=0 ) > Skip 2FA (OK=35 KO=0 ) > User Authorize PKCE stage (OK=35 KO=0 ) > User AccessToken PKCE stage (OK=35 KO=0 ) > Create Managed User (OK=200000 KO=0 ) ---- Managed User Create Preload ----------------------------------------------- [##########################################################################]100% waiting: 0 / active: 0 / done: 35 ================================================================================ Simulation idm.simple_managed_users.ManagedUsersPreload completed in 591 seconds ---------------------- [05/Sep/2023 18:25:33] Create tar file preload-with-gatling.tar.gz ---------------------- RESULTS_FOLDER=/results/gatling/ RESULT_FOLDER=preload-with-gatling SIMULATION_PATH=preload-with-gatling/manageduserspreload-20230905171540551 PWD=/results/gatling/preload-with-gatling/manageduserspreload-20230905171540551 [cmd] tar -czvf /results/gatling//preload-with-gatling.tar.gz --exclude=simulation.log * /results/gatling ---------------------- [05/Sep/2023 18:25:33] Update /results/gatling//latest ---------------------- ---------------------- [05/Sep/2023 18:25:33] Update /results/gatling//latest.tar.gz ---------------------- ________________________________________________________________________________ [05/Sep/2023 17:25:48] preload-with-gatling show_stderr : Display stderr order ________________________________________________________________________________ [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/benchmark/05_Data_init/04_preload-with-gatling/stderr.txt is small (0 lines), printing all lines: ----- output ----- ________________________________________________________________________________ [05/Sep/2023 17:25:48] preload-with-gatling check_rc_and_logs : Check order's outputs ________________________________________________________________________________ PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [05/Sep/2023 17:25:48] preload-with-gatling print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [05/Sep/2023 17:25:48] preload-with-gatling post : Post ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped