--Task-- name: benchmark-300 enabled: True class_name: GatlingTask source_name: overseer-0 source_namespace: >default< target_name: am target_namespace: >default< start: 0.0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates below loop: False interval: None dependencies: [] wait_for: ['benchmark-200'] preceding_task: benchmark-200 options: {'duration': 3600, 'java-opts': '-Drealm=alpha -Duser_password=Pa_ssw0rd -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html', 'testCase': 'idc.benchmark.Mix2024', 'testMode': 'open', 'throughput': 300} group_name: None Current dir: /mnt/disk1/xslou/workshop/lodestar-fork/pyrock 2025-06-14 00:04:47,173 INFO 2025-06-14 00:04:47,173 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 00:04:47,173 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 00:04:47,455 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 00:04:47,455 INFO [loop_until]: OK (rc = 0) 2025-06-14 00:04:47,455 DEBUG --- stdout --- 2025-06-14 00:04:47,455 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 00:04:47,455 DEBUG --- stderr --- 2025-06-14 00:04:47,455 DEBUG 2025-06-14 00:04:47,455 INFO Lodestar assumed that full stack is deployed so will set all products host URLs in Gatling system properties 2025-06-14 00:04:47 - INFO: Timeout for this Task has changed to 1h 12m (4320 seconds) 2025-06-14 00:04:47 - INFO: timeout was calculated based on simulation duration parameter 2025-06-14 00:04:47 - INFO: Interval for this Task has changed to 2m (120 seconds) 2025-06-14 00:04:47 - INFO: interval was set to Task default because it was unset, based on self.timeout value because task is allowed to stop by itself Pod log file : /results/orders/benchmark-300/stdout.txt Pod log : /results/orders/benchmark-300 ________________________________________________________________________________ [2025-06-14 00:04:47] benchmark-300 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@tof-10" --results-folder="/results/gatling/benchmark-300" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=**** ________________________________________________________________________________ [2025-06-14 00:04:47] benchmark-300 create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file: /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/order.json { "name": "benchmark-300", "command": " /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class=\"com.forgerock.pyrock.Loadtest\" --run-description=\"idc.benchmark.Mix2024@tof-10\" --results-folder=\"/results/gatling/benchmark-300\" --no-reports --simulation-jvm-args=\"-XX:+ExitOnOutOfMemoryError -Xmx4G\" --simulation-system-properties=\"-Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=Pa_ssw0rd\"", "out-dir": "/results/orders/benchmark-300" } ________________________________________________________________________________ [2025-06-14 00:04:47] benchmark-300 loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2025-06-14 00:04:47,463 INFO 2025-06-14 00:04:47,463 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2025-06-14 00:04:47,463 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-06-14 00:04:47,676 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 00:04:47,676 INFO [loop_until]: OK (rc = 0) 2025-06-14 00:04:47,676 DEBUG --- stdout --- 2025-06-14 00:04:47,676 DEBUG WAITING 2025-06-14 00:04:47,676 DEBUG --- stderr --- 2025-06-14 00:04:47,676 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 34 0 --:--:-- --:--:-- --:--:-- 34 ________________________________________________________________________________ [2025-06-14 00:04:47] benchmark-300 start_order : Send the order to the overseer pod ________________________________________________________________________________ 2025-06-14 00:04:47,676 INFO 2025-06-14 00:04:47,676 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 00:04:47,676 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 00:04:47,957 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 00:04:47,957 INFO [loop_until]: OK (rc = 0) 2025-06-14 00:04:47,957 DEBUG --- stdout --- 2025-06-14 00:04:47,957 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 00:04:47,957 DEBUG --- stderr --- 2025-06-14 00:04:47,957 DEBUG 2025-06-14 00:04:47,958 INFO 2025-06-14 00:04:47,958 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/order.json overseer-0-898c87646-bjhnj:/results/orders/order.json 2025-06-14 00:04:47,958 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 00:04:49,347 INFO [loop_until]: OK (rc = 0) 2025-06-14 00:04:49,347 DEBUG --- stdout --- 2025-06-14 00:04:49,347 DEBUG 2025-06-14 00:04:49,347 DEBUG --- stderr --- 2025-06-14 00:04:49,347 DEBUG Order has been successfully forwarded to overseer 2025-06-14 00:04:49 - INFO: Check if Gatling simulation started successfully 2025-06-14 00:04:49,348 INFO 2025-06-14 00:04:49,348 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj -- head --lines 5000 /results/orders/benchmark-300/stdout.txt | grep "Simulation com.forgerock.pyrock.Loadtest started..." 2025-06-14 00:04:49,348 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-06-14 00:04:50,094 INFO [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-06-14 00:04:55,847 INFO [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-06-14 00:05:01,617 INFO [loop_until]: Function succeeded after 12s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-06-14 00:05:07,363 INFO [loop_until]: Function succeeded after 18s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-06-14 00:05:13,112 INFO [loop_until]: Function succeeded after 23s (rc=0) - expected pattern found 2025-06-14 00:05:13,112 INFO [loop_until]: OK (rc = 0) 2025-06-14 00:05:13,112 DEBUG --- stdout --- 2025-06-14 00:05:13,112 DEBUG 2025-06-14 01:04:51,192 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmp2299mokf 2025-06-14 01:04:51,196 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-300" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=**** Gatling 3.13.5 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 2025-06-14 00:05:13,112 DEBUG --- stderr --- 2025-06-14 00:05:13,112 DEBUG ________________________________________________________________________________ [2025-06-14 00:05:13] benchmark-300 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 1h 12m. Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 2m, retry Order is running since 4m, retry Order is running since 6m, retry Order is running since 8m 01s, retry Order is running since 10m 01s, retry Order is running since 12m 01s, retry Order is running since 14m 01s, retry Order is running since 16m 02s, retry Order is running since 18m 02s, retry Order is running since 20m 02s, retry Order is running since 22m 03s, retry Order is running since 24m 03s, retry Order is running since 26m 03s, retry Order is running since 28m 04s, retry Order is running since 30m 04s, retry Order is running since 32m 04s, retry Order is running since 34m 04s, retry Order is running since 36m 05s, retry Order is running since 38m 05s, retry Order is running since 40m 05s, retry Order is running since 42m 05s, retry Order is running since 44m 06s, retry Order is running since 46m 06s, retry Order is running since 48m 06s, retry Order is running since 50m 07s, retry Order is running since 52m 07s, retry Order is running since 54m 07s, retry Order is running since 56m 07s, retry Order is running since 58m 08s, retry PASS : Order stopped by itself after 1h and before timeout (1h 12m) ________________________________________________________________________________ [2025-06-14 01:05:21] benchmark-300 stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2025-06-14 01:05:21,640 INFO 2025-06-14 01:05:21,640 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 01:05:21,640 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 01:05:21,947 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 01:05:21,947 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:21,947 DEBUG --- stdout --- 2025-06-14 01:05:21,947 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 01:05:21,947 DEBUG --- stderr --- 2025-06-14 01:05:21,947 DEBUG 2025-06-14 01:05:21,947 INFO 2025-06-14 01:05:21,947 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj -- rm -f /results/orders/order.json 2025-06-14 01:05:21,947 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 01:05:22,736 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:22,736 DEBUG --- stdout --- 2025-06-14 01:05:22,736 DEBUG 2025-06-14 01:05:22,736 DEBUG --- stderr --- 2025-06-14 01:05:22,736 DEBUG Order ran during 1h ________________________________________________________________________________ [2025-06-14 01:05:22] benchmark-300 download_logs : Download individual order files ________________________________________________________________________________ 2025-06-14 01:05:22 - PASS: file already downloaded /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/order.json 2025-06-14 01:05:22,736 INFO 2025-06-14 01:05:22,736 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 01:05:22,736 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 01:05:23,012 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 01:05:23,012 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:23,012 DEBUG --- stdout --- 2025-06-14 01:05:23,012 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 01:05:23,012 DEBUG --- stderr --- 2025-06-14 01:05:23,012 DEBUG Checking if file exists on pod 'overseer-0-898c87646-bjhnj', wait max 3 minutes if not. 2025-06-14 01:05:23,013 INFO 2025-06-14 01:05:23,013 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- bash -c 'if [ -f "/results/orders/benchmark-300/rc.txt" ]; then echo "/results/orders/benchmark-300/rc.txt file found"; exit 0; else echo "/results/orders/benchmark-300/rc.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-300/rc.txt file found" 2025-06-14 01:05:23,013 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 01:05:23,820 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 01:05:23,820 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:23,820 DEBUG --- stdout --- 2025-06-14 01:05:23,820 DEBUG /results/orders/benchmark-300/rc.txt file found 2025-06-14 01:05:23,820 DEBUG --- stderr --- 2025-06-14 01:05:23,820 DEBUG 2025-06-14 01:05:23,821 INFO 2025-06-14 01:05:23,821 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- du -k /results/orders/benchmark-300/rc.txt | grep "^\d+ .+$" 2025-06-14 01:05:23,821 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-06-14 01:05:24,626 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 01:05:24,627 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:24,627 DEBUG --- stdout --- 2025-06-14 01:05:24,627 DEBUG 4 /results/orders/benchmark-300/rc.txt 2025-06-14 01:05:24,627 DEBUG --- stderr --- 2025-06-14 01:05:24,627 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-300/rc.txt @ overseer-0-898c87646-bjhnj destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/rc.txt 2025-06-14 01:05:24,627 INFO 2025-06-14 01:05:24,627 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-bjhnj:/results/orders/benchmark-300/rc.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/rc.txt 2025-06-14 01:05:24,628 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-06-14 01:05:25,397 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:25,397 DEBUG --- stdout --- 2025-06-14 01:05:25,397 DEBUG tar: Removing leading `/' from member names 2025-06-14 01:05:25,397 DEBUG --- stderr --- 2025-06-14 01:05:25,397 DEBUG 2025-06-14 01:05:25,397 INFO 2025-06-14 01:05:25,397 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 01:05:25,397 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 01:05:25,674 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 01:05:25,674 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:25,674 DEBUG --- stdout --- 2025-06-14 01:05:25,674 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 01:05:25,674 DEBUG --- stderr --- 2025-06-14 01:05:25,674 DEBUG Checking if file exists on pod 'overseer-0-898c87646-bjhnj', wait max 3 minutes if not. 2025-06-14 01:05:25,675 INFO 2025-06-14 01:05:25,675 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- bash -c 'if [ -f "/results/orders/benchmark-300/stderr.txt" ]; then echo "/results/orders/benchmark-300/stderr.txt file found"; exit 0; else echo "/results/orders/benchmark-300/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-300/stderr.txt file found" 2025-06-14 01:05:25,675 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 01:05:26,486 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 01:05:26,486 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:26,486 DEBUG --- stdout --- 2025-06-14 01:05:26,486 DEBUG /results/orders/benchmark-300/stderr.txt file found 2025-06-14 01:05:26,486 DEBUG --- stderr --- 2025-06-14 01:05:26,486 DEBUG 2025-06-14 01:05:26,486 INFO 2025-06-14 01:05:26,486 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- du -k /results/orders/benchmark-300/stderr.txt | grep "^\d+ .+$" 2025-06-14 01:05:26,486 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-06-14 01:05:27,288 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 01:05:27,288 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:27,288 DEBUG --- stdout --- 2025-06-14 01:05:27,288 DEBUG 0 /results/orders/benchmark-300/stderr.txt 2025-06-14 01:05:27,288 DEBUG --- stderr --- 2025-06-14 01:05:27,288 DEBUG - Download file (size 0B) source : /results/orders/benchmark-300/stderr.txt @ overseer-0-898c87646-bjhnj destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stderr.txt 2025-06-14 01:05:27,289 INFO 2025-06-14 01:05:27,289 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-bjhnj:/results/orders/benchmark-300/stderr.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stderr.txt 2025-06-14 01:05:27,289 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-06-14 01:05:28,185 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:28,185 DEBUG --- stdout --- 2025-06-14 01:05:28,185 DEBUG tar: Removing leading `/' from member names 2025-06-14 01:05:28,185 DEBUG --- stderr --- 2025-06-14 01:05:28,185 DEBUG 2025-06-14 01:05:28,186 INFO 2025-06-14 01:05:28,186 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 01:05:28,186 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 01:05:28,465 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 01:05:28,465 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:28,465 DEBUG --- stdout --- 2025-06-14 01:05:28,465 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 01:05:28,465 DEBUG --- stderr --- 2025-06-14 01:05:28,465 DEBUG Checking if file exists on pod 'overseer-0-898c87646-bjhnj', wait max 3 minutes if not. 2025-06-14 01:05:28,466 INFO 2025-06-14 01:05:28,466 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- bash -c 'if [ -f "/results/orders/benchmark-300/stdout.txt" ]; then echo "/results/orders/benchmark-300/stdout.txt file found"; exit 0; else echo "/results/orders/benchmark-300/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-300/stdout.txt file found" 2025-06-14 01:05:28,466 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 01:05:29,269 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 01:05:29,269 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:29,269 DEBUG --- stdout --- 2025-06-14 01:05:29,269 DEBUG /results/orders/benchmark-300/stdout.txt file found 2025-06-14 01:05:29,269 DEBUG --- stderr --- 2025-06-14 01:05:29,269 DEBUG 2025-06-14 01:05:29,270 INFO 2025-06-14 01:05:29,270 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- du -k /results/orders/benchmark-300/stdout.txt | grep "^\d+ .+$" 2025-06-14 01:05:29,270 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-06-14 01:05:30,080 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 01:05:30,080 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:30,080 DEBUG --- stdout --- 2025-06-14 01:05:30,080 DEBUG 28728 /results/orders/benchmark-300/stdout.txt 2025-06-14 01:05:30,080 DEBUG --- stderr --- 2025-06-14 01:05:30,080 DEBUG - Download file (size 28.728MB) source : /results/orders/benchmark-300/stdout.txt @ overseer-0-898c87646-bjhnj destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stdout.txt 2025-06-14 01:05:30,081 INFO 2025-06-14 01:05:30,081 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-bjhnj:/results/orders/benchmark-300/stdout.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stdout.txt 2025-06-14 01:05:30,081 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-06-14 01:05:32,238 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:32,238 DEBUG --- stdout --- 2025-06-14 01:05:32,238 DEBUG tar: Removing leading `/' from member names 2025-06-14 01:05:32,238 DEBUG --- stderr --- 2025-06-14 01:05:32,238 DEBUG ________________________________________________________________________________ [2025-06-14 01:05:32] benchmark-300 show_cmd : Display order command ________________________________________________________________________________ 2025-06-14 01:05:32,238 INFO 2025-06-14 01:05:32,238 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 01:05:32,238 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 01:05:32,522 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 01:05:32,522 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:32,522 DEBUG --- stdout --- 2025-06-14 01:05:32,522 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 01:05:32,522 DEBUG --- stderr --- 2025-06-14 01:05:32,522 DEBUG Checking if file exists on pod 'overseer-0-898c87646-bjhnj', wait max 3 minutes if not. 2025-06-14 01:05:32,523 INFO 2025-06-14 01:05:32,523 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- bash -c 'if [ -f "/results/orders/benchmark-300/cmd.txt" ]; then echo "/results/orders/benchmark-300/cmd.txt file found"; exit 0; else echo "/results/orders/benchmark-300/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-300/cmd.txt file found" 2025-06-14 01:05:32,523 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 01:05:33,340 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 01:05:33,340 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:33,340 DEBUG --- stdout --- 2025-06-14 01:05:33,340 DEBUG /results/orders/benchmark-300/cmd.txt file found 2025-06-14 01:05:33,340 DEBUG --- stderr --- 2025-06-14 01:05:33,340 DEBUG 2025-06-14 01:05:33,341 INFO 2025-06-14 01:05:33,341 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- du -k /results/orders/benchmark-300/cmd.txt | grep "^\d+ .+$" 2025-06-14 01:05:33,341 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-06-14 01:05:34,146 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 01:05:34,146 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:34,146 DEBUG --- stdout --- 2025-06-14 01:05:34,146 DEBUG 4 /results/orders/benchmark-300/cmd.txt 2025-06-14 01:05:34,146 DEBUG --- stderr --- 2025-06-14 01:05:34,146 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-300/cmd.txt @ overseer-0-898c87646-bjhnj destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/cmd.txt 2025-06-14 01:05:34,146 INFO 2025-06-14 01:05:34,147 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-bjhnj:/results/orders/benchmark-300/cmd.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/cmd.txt 2025-06-14 01:05:34,147 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-06-14 01:05:34,875 INFO [loop_until]: OK (rc = 0) 2025-06-14 01:05:34,875 DEBUG --- stdout --- 2025-06-14 01:05:34,875 DEBUG tar: Removing leading `/' from member names 2025-06-14 01:05:34,875 DEBUG --- stderr --- 2025-06-14 01:05:34,875 DEBUG 2025-06-14 01:05:34,876 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/cmd.txt is small (1 lines), printing all lines: 2025-06-14 01:05:34,876 INFO ----- output ----- /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@tof-10" --results-folder="/results/gatling/benchmark-300" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=M@ythe4th -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-06-14 01:05:34] benchmark-300 show_rc : Display rc order ________________________________________________________________________________ 2025-06-14 01:05:34,876 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/rc.txt is small (1 lines), printing all lines: 2025-06-14 01:05:34,876 INFO ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [2025-06-14 01:05:34] benchmark-300 show_stdout : Display stdout order ________________________________________________________________________________ 2025-06-14 01:05:34,925 DEBUG [print_head_tail]: Print head (100 lines) and tail (100 lines) of input file (/mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stdout.txt is 595763 lines): 2025-06-14 01:05:35,000 INFO ----- output ----- 2025-06-14 01:04:51,192 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmp2299mokf 2025-06-14 01:04:51,196 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-300" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=**** Gatling 3.13.5 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... ================================================================================ 2025-06-14 00:05:14 GMT 4s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=893 KO=0 ) > authAsPolicyAdmin (OK=1 KO=0 ) > restLoginInitiate (OK=74 KO=0 ) > restLoginUsernamePasswordCallback (OK=71 KO=0 ) > User Authorize PKCE stage (OK=85 KO=0 ) > User AccessToken PKCE stage (OK=81 KO=0 ) > Skip 2FA (OK=20 KO=0 ) > tokenUserinfo (OK=19 KO=0 ) > userRestlogin (OK=42 KO=0 ) > EvalPolicy (OK=41 KO=0 ) > RootUserinfo (OK=31 KO=0 ) > loginTreeWorkflow_restLoginInitiate (OK=3 KO=0 ) > loginTreeWorkflow_restLoginUsernamePasswordCallback (OK=3 KO=0 ) > UI-Post-Login - stage 1 (OK=27 KO=0 ) > UI-Post-Login - stage 1 Redirect 1 (OK=26 KO=0 ) > UI AM idFromSession (OK=26 KO=0 ) > UI AM validateGoto (OK=26 KO=0 ) > UI-Post-Login - stage 2 (OK=24 KO=0 ) > UI-Post-Login - stage 2 Redirect 1 (OK=23 KO=0 ) > UI Enduser appAuthHelper2 (OK=20 KO=0 ) > UI Enduser appAuthHelper1 (OK=20 KO=0 ) > getIdToken (OK=1 KO=0 ) > UI-Post-Login - stage 3 (OK=18 KO=0 ) > getIdToken Redirect 1 (OK=1 KO=0 ) > UI-Post-Login - stage 3 Redirect 1 (OK=16 KO=0 ) > createOrganization (OK=1 KO=0 ) > UI Enduser sessioncheck (OK=15 KO=0 ) > UI IDM uiconfig (OK=15 KO=0 ) > UI IDM version (OK=15 KO=0 ) > UI IDM features (OK=14 KO=0 ) > UI IDM login (OK=15 KO=0 ) > UI IDM privilege (OK=15 KO=0 ) > queryOrganization (OK=1 KO=0 ) > UI IDM dashboard (OK=13 KO=0 ) > UI IDM user (OK=13 KO=0 ) > UI IDM managed user schema (OK=13 KO=0 ) > deleteOrganization (OK=1 KO=0 ) > UI IDM oidcToken (OK=11 KO=0 ) > userinfo (OK=10 KO=0 ) > UI-Logout (OK=9 KO=0 ) > UI-Logout Redirect 1 (OK=9 KO=0 ) > UI AM revoke1 (OK=8 KO=0 ) > UI AM revoke2 (OK=8 KO=0 ) > UI AM endsession (OK=8 KO=0 ) ---- getToken ------------------------------------------------------------------ [--------------------------------------------------------------------------] 0% waiting: 0 / active: 1 / done: 0 ---- idc.benchmark.Mix2024 ----------------------------------------------------- [- ] 0% waiting: 1084670 / active: 53 / done: 72 ================================================================================ ================================================================================ 2025-06-14 00:05:19 GMT 9s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=4582 KO=0 ) > authAsPolicyAdmin (OK=1 KO=0 ) > restLoginInitiate (OK=314 KO=0 ) > restLoginUsernamePasswordCallback (OK=305 KO=0 ) > User Authorize PKCE stage (OK=391 KO=0 ) > User AccessToken PKCE stage (OK=381 KO=0 ) > Skip 2FA (OK=86 KO=0 ) > tokenUserinfo (OK=95 KO=0 ) > userRestlogin (OK=160 KO=0 ) > EvalPolicy (OK=159 KO=0 ) > RootUserinfo (OK=209 KO=0 ) > loginTreeWorkflow_restLoginInitiate (OK=15 KO=0 ) > loginTreeWorkflow_restLoginUsernamePasswordCallback (OK=14 KO=0 ) > UI-Post-Login - stage 1 (OK=115 KO=0 ) > UI-Post-Login - stage 1 Redirect 1 (OK=114 KO=0 ) > UI AM idFromSession (OK=112 KO=0 ) > UI AM validateGoto (OK=112 KO=0 ) > UI-Post-Login - stage 2 (OK=103 KO=0 ) > UI-Post-Login - stage 2 Redirect 1 (OK=103 KO=0 ) > UI Enduser appAuthHelper2 (OK=100 KO=0 ) > UI Enduser appAuthHelper1 (OK=100 KO=0 ) > getIdToken (OK=20 KO=0 ) > UI-Post-Login - stage 3 (OK=93 KO=0 ) > getIdToken Redirect 1 (OK=19 KO=0 ) > UI-Post-Login - stage 3 Redirect 1 (OK=90 KO=0 ) > createOrganization (OK=19 KO=0 ) > UI Enduser sessioncheck (OK=87 KO=0 ) > UI IDM uiconfig (OK=87 KO=0 ) > UI IDM version (OK=87 KO=0 ) > UI IDM features (OK=87 KO=0 ) > UI IDM login (OK=87 KO=0 ) > UI IDM privilege (OK=87 KO=0 ) > queryOrganization (OK=18 KO=0 ) > UI IDM dashboard (OK=85 KO=0 ) [...] ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 02:05:09.605 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: userRestlogin: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1075730,HashMap(gatling.http.cache.baseUrl -> https://openam-tof-10.forgeblocks.com:443, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@3a8e498, username -> user.6445990, tokenIdCookieName -> amlbcookie, policyEvalTokenId -> , gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@7438b917, password -> Pa_ssw0rd),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$821/0x00007860646983d0@3d7d666e,io.netty.channel.epoll.EpollEventLoop@14f5da2c) ========================= HTTP request: POST https://openam-tof-10.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: Content-Type: apppliation/json Accept-API-Version: resource=2.0, protocol=1.0 user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 X-OpenAM-Username: user.6445990 X-OpenAM-Password: Pa_ssw0rd accept: */* host: openam-tof-10.forgeblocks.com content-length: 0 ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 02:05:09.606 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'userTokenId' is defined 02:05:09.606 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request ' UI IDM features' failed for user 1075330: j.i.IOException: Premature close 02:05:09.606 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: UI IDM features: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1075330,HashMap(access_token -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIyODdlN2UxYS04OWY4LTQ1YjktYTI5OS04NWIzMzQ1ZmM0MzAiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiZmY3MTA1ZDYtMmNiMS00ZTE3LWIyYmItZDkyY2YyMDkzMDU4LTU5ODIwMzM5Iiwic3VibmFtZSI6IjI4N2U3ZTFhLTg5ZjgtNDViOS1hMjk5LTg1YjMzNDVmYzQzMCIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXRvZi0xMC5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJRSHNkeGgtM2hKbEszTC15WjJPNTZBWGtBZGsiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NDk4NjMxMDgsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJvcGVuaWQiLCJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NDk4NjMxMDgsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzQ5ODY2NzA4LCJpYXQiOjE3NDk4NjMxMDgsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6ImdLcXBrNVVhZDY1ckFCMV9CMThhazJkXzdJWSJ9.hEVJU6EAO0hxHMpPHspSBJUEaKdcbf_qHsOnXD4zMEc, username -> user.1067521, tokenIdCookieName -> amlbcookie, policyEvalTokenId -> , access_token_2 -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIyODdlN2UxYS04OWY4LTQ1YjktYTI5OS04NWIzMzQ1ZmM0MzAiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiZmY3MTA1ZDYtMmNiMS00ZTE3LWIyYmItZDkyY2YyMDkzMDU4LTU5ODI0MDA5Iiwic3VibmFtZSI6IjI4N2U3ZTFhLTg5ZjgtNDViOS1hMjk5LTg1YjMzNDVmYzQzMCIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXRvZi0xMC5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJPRHEyRnJBR0ZNcWFvQmxVNnJBdGczbnFCajgiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NDk4NjMxMDksImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NDk4NjMxMDgsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzQ5ODY2NzA5LCJpYXQiOjE3NDk4NjMxMDksImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6InE0NTJEeVY1dThWTmx6aTRsQ0MxWDgtSWlORSJ9.s1AgqhxVMdtkTAn0zy4o18z91BTVSjKvzTC1qEq_bqo, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@7eb13ac0, gatling.http.referer -> https://openam-tof-10.forgeblocks.com/am/, authcode -> GCoqNuYZ6ksXU9Tuw9XoJYWVv30, callbacks -> {"tokenId":"96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*","successUrl":"/enduser/?realm=/alpha","realm":"/alpha"}, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-tof-10.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1749863108398), CookieKey(7573b68ed0f3690,openam-tof-10.forgeblocks.com,/) -> StoredCookie(7573b68ed0f3690=96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1749863108398))), user_uuid -> 287e7e1a-89f8-45b9-a299-85b3345fc430, password -> Pa_ssw0rd, challenge -> xWrauTjT4OM3Me7fNYbcKhcgAKxCMYz8mKFXJj3H6UY, gatling.http.cache.baseUrl -> https://openam-tof-10.forgeblocks.com:443, fake-ip -> 202.8.1.85, verifier -> LTY5Nzc5MDU0ODIzMTgxNTA2NzA, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@a66a636, gatling.http.cache.contentCache -> io.gatling.core.util.cache.Cache@7023ed08, tokenId -> 96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, fake-user-agent -> Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile/14E5239e Safari/602.1),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$821/0x00007860646983d0@3d7d666e,io.netty.channel.epoll.EpollEventLoop@14f5da2c) ========================= HTTP request: GET https://openam-tof-10.forgeblocks.com/openidm/info/features?_queryFilter=true headers: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 referer: https://openam-tof-10.forgeblocks.com/am/ Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIyODdlN2UxYS04OWY4LTQ1YjktYTI5OS04NWIzMzQ1ZmM0MzAiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiZmY3MTA1ZDYtMmNiMS00ZTE3LWIyYmItZDkyY2YyMDkzMDU4LTU5ODI0MDA5Iiwic3VibmFtZSI6IjI4N2U3ZTFhLTg5ZjgtNDViOS1hMjk5LTg1YjMzNDVmYzQzMCIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXRvZi0xMC5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJPRHEyRnJBR0ZNcWFvQmxVNnJBdGczbnFCajgiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NDk4NjMxMDksImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NDk4NjMxMDgsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzQ5ODY2NzA5LCJpYXQiOjE3NDk4NjMxMDksImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6InE0NTJEeVY1dThWTmx6aTRsQ0MxWDgtSWlORSJ9.s1AgqhxVMdtkTAn0zy4o18z91BTVSjKvzTC1qEq_bqo Accept-Api-Version: accept: */* cookie: amlbcookie=01; 7573b68ed0f3690=96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..* host: openam-tof-10.forgeblocks.com cookies: amlbcookie=01, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None 7573b68ed0f3690=96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 02:05:09.606 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'restLoginInitiate' failed for user 1075714: j.i.IOException: Premature close 02:05:09.606 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: restLoginInitiate: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1075714,HashMap(gatling.http.cache.baseUrl -> https://openam-tof-10.forgeblocks.com:443, username -> user.7921985, tokenIdCookieName -> amlbcookie, policyEvalTokenId -> , gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@24f4096f, fake-user-agent -> Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36, password -> Pa_ssw0rd, fake-ip -> 0.90.132.157, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@603ca54d),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$821/0x00007860646983d0@3d7d666e,io.netty.channel.epoll.EpollEventLoop@14f5da2c) ========================= HTTP request: POST https://openam-tof-10.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: accept: application/json content-type: application/json user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36 X-Forwarded-For: 0.90.132.157 host: openam-tof-10.forgeblocks.com content-length: 0 body:StringChunksRequestBody{charset=UTF-8, content=} ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 02:05:09.606 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-160' crashed with 'j.u.NoSuchElementException: No attribute named 'callbacks' is defined', forwarding to the next one 02:05:09.606 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request ' UI IDM version' failed for user 1075330: j.i.IOException: Premature close 02:05:09.606 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: UI IDM version: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1075330,HashMap(access_token -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIyODdlN2UxYS04OWY4LTQ1YjktYTI5OS04NWIzMzQ1ZmM0MzAiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiZmY3MTA1ZDYtMmNiMS00ZTE3LWIyYmItZDkyY2YyMDkzMDU4LTU5ODIwMzM5Iiwic3VibmFtZSI6IjI4N2U3ZTFhLTg5ZjgtNDViOS1hMjk5LTg1YjMzNDVmYzQzMCIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXRvZi0xMC5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJRSHNkeGgtM2hKbEszTC15WjJPNTZBWGtBZGsiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NDk4NjMxMDgsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJvcGVuaWQiLCJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NDk4NjMxMDgsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzQ5ODY2NzA4LCJpYXQiOjE3NDk4NjMxMDgsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6ImdLcXBrNVVhZDY1ckFCMV9CMThhazJkXzdJWSJ9.hEVJU6EAO0hxHMpPHspSBJUEaKdcbf_qHsOnXD4zMEc, username -> user.1067521, tokenIdCookieName -> amlbcookie, policyEvalTokenId -> , access_token_2 -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIyODdlN2UxYS04OWY4LTQ1YjktYTI5OS04NWIzMzQ1ZmM0MzAiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiZmY3MTA1ZDYtMmNiMS00ZTE3LWIyYmItZDkyY2YyMDkzMDU4LTU5ODI0MDA5Iiwic3VibmFtZSI6IjI4N2U3ZTFhLTg5ZjgtNDViOS1hMjk5LTg1YjMzNDVmYzQzMCIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXRvZi0xMC5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJPRHEyRnJBR0ZNcWFvQmxVNnJBdGczbnFCajgiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NDk4NjMxMDksImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NDk4NjMxMDgsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzQ5ODY2NzA5LCJpYXQiOjE3NDk4NjMxMDksImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6InE0NTJEeVY1dThWTmx6aTRsQ0MxWDgtSWlORSJ9.s1AgqhxVMdtkTAn0zy4o18z91BTVSjKvzTC1qEq_bqo, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@7eb13ac0, gatling.http.referer -> https://openam-tof-10.forgeblocks.com/am/, authcode -> GCoqNuYZ6ksXU9Tuw9XoJYWVv30, callbacks -> {"tokenId":"96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*","successUrl":"/enduser/?realm=/alpha","realm":"/alpha"}, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-tof-10.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1749863108398), CookieKey(7573b68ed0f3690,openam-tof-10.forgeblocks.com,/) -> StoredCookie(7573b68ed0f3690=96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1749863108398))), user_uuid -> 287e7e1a-89f8-45b9-a299-85b3345fc430, password -> Pa_ssw0rd, challenge -> xWrauTjT4OM3Me7fNYbcKhcgAKxCMYz8mKFXJj3H6UY, gatling.http.cache.baseUrl -> https://openam-tof-10.forgeblocks.com:443, fake-ip -> 202.8.1.85, verifier -> LTY5Nzc5MDU0ODIzMTgxNTA2NzA, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@a66a636, gatling.http.cache.contentCache -> io.gatling.core.util.cache.Cache@7023ed08, tokenId -> 96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, fake-user-agent -> Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.75 Mobile/14E5239e Safari/602.1),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$821/0x00007860646983d0@3d7d666e,io.netty.channel.epoll.EpollEventLoop@14f5da2c) ========================= HTTP request: GET https://openam-tof-10.forgeblocks.com/openidm/info/version headers: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 referer: https://openam-tof-10.forgeblocks.com/am/ Accept-Api-Version: accept: */* cookie: amlbcookie=01; 7573b68ed0f3690=96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..* host: openam-tof-10.forgeblocks.com cookies: amlbcookie=01, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None 7573b68ed0f3690=96e--rcBcPUfl0164MEDeXod808.*AAJTSQACMDIAAlNLABxlSEg2aXhoek85djVhdkE0VkVkNkorQk9sbTA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 2025-06-14 02:05:09,728 INFO Gatling simulation run ended successfully 2025-06-14 02:05:09,729 DEBUG No archive name provided. No archive will be created ________________________________________________________________________________ [2025-06-14 01:05:35] benchmark-300 show_stderr : Display stderr order ________________________________________________________________________________ 2025-06-14 01:05:35,001 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stderr.txt is small (0 lines), printing all lines: 2025-06-14 01:05:35,001 INFO ----- output ----- ________________________________________________________________________________ [2025-06-14 01:05:35] benchmark-300 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2025-06-14 01:05:35] benchmark-300 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2025-06-14 01:05:35] benchmark-300 post : Post method ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped