--Task-- name: benchmark-10 enabled: True class_name: GatlingTask source_name: overseer-0 source_namespace: >default< target_name: am target_namespace: >default< start: 0.0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates below loop: False interval: None dependencies: [] wait_for: [] preceding_task: None options: {'duration': 3600, 'java-opts': '-Drealm=alpha -Duser_password=Pa_ssw0rd -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html', 'testCase': 'idc.benchmark.Mix2024', 'testMode': 'open', 'throughput': 10} group_name: None Current dir: /mnt/disk1/xslou/workshop/lodestar-fork/pyrock 2025-09-16 05:34:41,877 INFO 2025-09-16 05:34:41,877 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 05:34:41,877 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 05:34:42,159 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 05:34:42,159 INFO [loop_until]: OK (rc = 0) 2025-09-16 05:34:42,159 DEBUG --- stdout --- 2025-09-16 05:34:42,159 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 05:34:42,159 DEBUG --- stderr --- 2025-09-16 05:34:42,159 DEBUG 2025-09-16 05:34:42,159 INFO Lodestar assumed that full stack is deployed so will set all products host URLs in Gatling system properties 2025-09-16 05:34:42 - INFO: Timeout for this Task has changed to 1h 12m (4320 seconds) 2025-09-16 05:34:42 - INFO: timeout was calculated based on simulation duration parameter 2025-09-16 05:34:42 - INFO: Interval for this Task has changed to 2m (120 seconds) 2025-09-16 05:34:42 - INFO: interval was set to Task default because it was unset, based on self.timeout value because task is allowed to stop by itself Pod log file : /results/orders/benchmark-10/stdout.txt Pod log : /results/orders/benchmark-10 2025-09-16 05:34:42 - INFO: Get overseer pod resources requests memory 2025-09-16 05:34:42,943 INFO 2025-09-16 05:34:42,943 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods overseer-0-57d79d468-khs5t --output jsonpath={.spec.containers[].resources.requests.memory} 2025-09-16 05:34:42,943 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 05:34:43,240 INFO [loop_until]: OK (rc = 0) 2025-09-16 05:34:43,240 DEBUG --- stdout --- 2025-09-16 05:34:43,240 DEBUG 4Gi 2025-09-16 05:34:43,240 DEBUG --- stderr --- 2025-09-16 05:34:43,240 DEBUG ________________________________________________________________________________ [2025-09-16 05:34:43] benchmark-10 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@perf-haproxy" --results-folder="/results/gatling/benchmark-10" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-09-16 05:34:43] benchmark-10 create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file: /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/order.json { "name": "benchmark-10", "command": " /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class=\"com.forgerock.pyrock.Loadtest\" --run-description=\"idc.benchmark.Mix2024@perf-haproxy\" --results-folder=\"/results/gatling/benchmark-10\" --no-reports --simulation-jvm-args=\"-XX:+ExitOnOutOfMemoryError -Xmx4G\" --simulation-system-properties=\"-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd\"", "out-dir": "/results/orders/benchmark-10" } ________________________________________________________________________________ [2025-09-16 05:34:43] benchmark-10 loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2025-09-16 05:34:43,248 INFO 2025-09-16 05:34:43,248 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2025-09-16 05:34:43,248 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-09-16 05:34:43,527 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 05:34:43,527 INFO [loop_until]: OK (rc = 0) 2025-09-16 05:34:43,527 DEBUG --- stdout --- 2025-09-16 05:34:43,527 DEBUG WAITING 2025-09-16 05:34:43,527 DEBUG --- stderr --- 2025-09-16 05:34:43,527 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 25 0 --:--:-- --:--:-- --:--:-- 25 ________________________________________________________________________________ [2025-09-16 05:34:43] benchmark-10 start_order : Send the order to the overseer pod ________________________________________________________________________________ 2025-09-16 05:34:43,528 INFO 2025-09-16 05:34:43,528 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 05:34:43,528 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 05:34:43,806 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 05:34:43,806 INFO [loop_until]: OK (rc = 0) 2025-09-16 05:34:43,806 DEBUG --- stdout --- 2025-09-16 05:34:43,806 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 05:34:43,806 DEBUG --- stderr --- 2025-09-16 05:34:43,806 DEBUG 2025-09-16 05:34:43,807 INFO 2025-09-16 05:34:43,807 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/order.json overseer-0-57d79d468-khs5t:/results/orders/order.json 2025-09-16 05:34:43,807 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 05:34:44,720 INFO [loop_until]: OK (rc = 0) 2025-09-16 05:34:44,720 DEBUG --- stdout --- 2025-09-16 05:34:44,720 DEBUG 2025-09-16 05:34:44,720 DEBUG --- stderr --- 2025-09-16 05:34:44,720 DEBUG Order has been successfully forwarded to overseer 2025-09-16 05:34:44 - INFO: Check if Gatling simulation started successfully 2025-09-16 05:34:44,720 INFO 2025-09-16 05:34:44,721 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- head --lines 5000 /results/orders/benchmark-10/stdout.txt | grep "Simulation com.forgerock.pyrock.Loadtest started..." 2025-09-16 05:34:44,721 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-09-16 05:34:45,269 INFO [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-16 05:34:50,816 INFO [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-16 05:34:56,352 INFO [loop_until]: Function succeeded after 11s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-16 05:35:01,907 INFO [loop_until]: Function succeeded after 17s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-16 05:35:07,597 INFO [loop_until]: Function succeeded after 22s (rc=0) - expected pattern found 2025-09-16 05:35:07,597 INFO [loop_until]: OK (rc = 0) 2025-09-16 05:35:07,597 DEBUG --- stdout --- 2025-09-16 05:35:07,597 DEBUG 2025-09-16 06:34:46,928 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmpcbp5e4r9 2025-09-16 06:34:46,932 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-10" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 2025-09-16 05:35:07,597 DEBUG --- stderr --- 2025-09-16 05:35:07,597 DEBUG ________________________________________________________________________________ [2025-09-16 05:35:07] benchmark-10 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 1h 12m. Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 2m, retry Order is running since 4m, retry Order is running since 6m, retry Order is running since 8m 01s, retry Order is running since 10m 01s, retry Order is running since 12m 01s, retry Order is running since 14m 01s, retry Order is running since 16m 02s, retry Order is running since 18m 02s, retry Order is running since 20m 02s, retry Order is running since 22m 02s, retry Order is running since 24m 03s, retry Order is running since 26m 03s, retry Order is running since 28m 03s, retry Order is running since 30m 03s, retry Order is running since 32m 04s, retry Order is running since 34m 04s, retry Order is running since 36m 04s, retry Order is running since 38m 05s, retry Order is running since 40m 05s, retry Order is running since 42m 05s, retry Order is running since 44m 05s, retry Order is running since 46m 06s, retry Order is running since 48m 06s, retry Order is running since 50m 06s, retry Order is running since 52m 06s, retry Order is running since 54m 07s, retry Order is running since 56m 07s, retry Order is running since 58m 07s, retry PASS : Order stopped by itself after 1h and before timeout (1h 12m) ________________________________________________________________________________ [2025-09-16 06:35:15] benchmark-10 stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2025-09-16 06:35:15,538 INFO 2025-09-16 06:35:15,538 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 06:35:15,538 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 06:35:15,838 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 06:35:15,838 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:15,838 DEBUG --- stdout --- 2025-09-16 06:35:15,838 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 06:35:15,838 DEBUG --- stderr --- 2025-09-16 06:35:15,838 DEBUG 2025-09-16 06:35:15,838 INFO 2025-09-16 06:35:15,838 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- rm -f /results/orders/order.json 2025-09-16 06:35:15,838 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 06:35:16,444 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:16,444 DEBUG --- stdout --- 2025-09-16 06:35:16,444 DEBUG 2025-09-16 06:35:16,444 DEBUG --- stderr --- 2025-09-16 06:35:16,444 DEBUG Order ran during 1h ________________________________________________________________________________ [2025-09-16 06:35:16] benchmark-10 download_logs : Download individual order files ________________________________________________________________________________ 2025-09-16 06:35:16 - PASS: file already downloaded /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/order.json 2025-09-16 06:35:16,445 INFO 2025-09-16 06:35:16,445 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 06:35:16,445 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 06:35:16,722 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 06:35:16,723 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:16,723 DEBUG --- stdout --- 2025-09-16 06:35:16,723 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 06:35:16,723 DEBUG --- stderr --- 2025-09-16 06:35:16,723 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-16 06:35:16,723 INFO 2025-09-16 06:35:16,723 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-10/rc.txt" ]; then echo "/results/orders/benchmark-10/rc.txt file found"; exit 0; else echo "/results/orders/benchmark-10/rc.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-10/rc.txt file found" 2025-09-16 06:35:16,724 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 06:35:17,267 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 06:35:17,267 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:17,268 DEBUG --- stdout --- 2025-09-16 06:35:17,268 DEBUG /results/orders/benchmark-10/rc.txt file found 2025-09-16 06:35:17,268 DEBUG --- stderr --- 2025-09-16 06:35:17,268 DEBUG 2025-09-16 06:35:17,268 INFO 2025-09-16 06:35:17,268 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-10/rc.txt | grep "^\d+ .+$" 2025-09-16 06:35:17,268 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-16 06:35:17,812 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 06:35:17,813 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:17,813 DEBUG --- stdout --- 2025-09-16 06:35:17,813 DEBUG 4 /results/orders/benchmark-10/rc.txt 2025-09-16 06:35:17,813 DEBUG --- stderr --- 2025-09-16 06:35:17,813 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-10/rc.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/rc.txt 2025-09-16 06:35:17,814 INFO 2025-09-16 06:35:17,814 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-10/rc.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/rc.txt 2025-09-16 06:35:17,814 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-16 06:35:18,351 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:18,351 DEBUG --- stdout --- 2025-09-16 06:35:18,351 DEBUG tar: Removing leading `/' from member names 2025-09-16 06:35:18,351 DEBUG --- stderr --- 2025-09-16 06:35:18,351 DEBUG 2025-09-16 06:35:18,351 INFO 2025-09-16 06:35:18,352 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 06:35:18,352 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 06:35:18,659 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 06:35:18,659 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:18,659 DEBUG --- stdout --- 2025-09-16 06:35:18,659 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 06:35:18,659 DEBUG --- stderr --- 2025-09-16 06:35:18,659 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-16 06:35:18,660 INFO 2025-09-16 06:35:18,660 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-10/stderr.txt" ]; then echo "/results/orders/benchmark-10/stderr.txt file found"; exit 0; else echo "/results/orders/benchmark-10/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-10/stderr.txt file found" 2025-09-16 06:35:18,660 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 06:35:19,204 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 06:35:19,204 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:19,204 DEBUG --- stdout --- 2025-09-16 06:35:19,204 DEBUG /results/orders/benchmark-10/stderr.txt file found 2025-09-16 06:35:19,204 DEBUG --- stderr --- 2025-09-16 06:35:19,204 DEBUG 2025-09-16 06:35:19,204 INFO 2025-09-16 06:35:19,204 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-10/stderr.txt | grep "^\d+ .+$" 2025-09-16 06:35:19,204 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-16 06:35:19,740 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 06:35:19,740 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:19,740 DEBUG --- stdout --- 2025-09-16 06:35:19,740 DEBUG 0 /results/orders/benchmark-10/stderr.txt 2025-09-16 06:35:19,740 DEBUG --- stderr --- 2025-09-16 06:35:19,740 DEBUG - Download file (size 0B) source : /results/orders/benchmark-10/stderr.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stderr.txt 2025-09-16 06:35:19,741 INFO 2025-09-16 06:35:19,741 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-10/stderr.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stderr.txt 2025-09-16 06:35:19,741 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-16 06:35:20,277 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:20,277 DEBUG --- stdout --- 2025-09-16 06:35:20,277 DEBUG tar: Removing leading `/' from member names 2025-09-16 06:35:20,277 DEBUG --- stderr --- 2025-09-16 06:35:20,278 DEBUG 2025-09-16 06:35:20,278 INFO 2025-09-16 06:35:20,278 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 06:35:20,278 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 06:35:20,558 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 06:35:20,558 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:20,558 DEBUG --- stdout --- 2025-09-16 06:35:20,559 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 06:35:20,559 DEBUG --- stderr --- 2025-09-16 06:35:20,559 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-16 06:35:20,559 INFO 2025-09-16 06:35:20,559 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-10/stdout.txt" ]; then echo "/results/orders/benchmark-10/stdout.txt file found"; exit 0; else echo "/results/orders/benchmark-10/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-10/stdout.txt file found" 2025-09-16 06:35:20,559 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 06:35:21,092 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 06:35:21,092 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:21,092 DEBUG --- stdout --- 2025-09-16 06:35:21,092 DEBUG /results/orders/benchmark-10/stdout.txt file found 2025-09-16 06:35:21,092 DEBUG --- stderr --- 2025-09-16 06:35:21,092 DEBUG 2025-09-16 06:35:21,092 INFO 2025-09-16 06:35:21,092 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-10/stdout.txt | grep "^\d+ .+$" 2025-09-16 06:35:21,092 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-16 06:35:21,643 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 06:35:21,643 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:21,643 DEBUG --- stdout --- 2025-09-16 06:35:21,643 DEBUG 16472 /results/orders/benchmark-10/stdout.txt 2025-09-16 06:35:21,643 DEBUG --- stderr --- 2025-09-16 06:35:21,643 DEBUG - Download file (size 16.472MB) source : /results/orders/benchmark-10/stdout.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stdout.txt 2025-09-16 06:35:21,645 INFO 2025-09-16 06:35:21,645 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-10/stdout.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stdout.txt 2025-09-16 06:35:21,645 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-16 06:35:23,195 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:23,195 DEBUG --- stdout --- 2025-09-16 06:35:23,195 DEBUG tar: Removing leading `/' from member names 2025-09-16 06:35:23,195 DEBUG --- stderr --- 2025-09-16 06:35:23,195 DEBUG ________________________________________________________________________________ [2025-09-16 06:35:23] benchmark-10 show_cmd : Display order command ________________________________________________________________________________ 2025-09-16 06:35:23,196 INFO 2025-09-16 06:35:23,196 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 06:35:23,196 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 06:35:23,475 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 06:35:23,475 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:23,475 DEBUG --- stdout --- 2025-09-16 06:35:23,475 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 06:35:23,475 DEBUG --- stderr --- 2025-09-16 06:35:23,475 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-16 06:35:23,476 INFO 2025-09-16 06:35:23,476 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-10/cmd.txt" ]; then echo "/results/orders/benchmark-10/cmd.txt file found"; exit 0; else echo "/results/orders/benchmark-10/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-10/cmd.txt file found" 2025-09-16 06:35:23,476 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 06:35:24,022 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 06:35:24,022 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:24,022 DEBUG --- stdout --- 2025-09-16 06:35:24,022 DEBUG /results/orders/benchmark-10/cmd.txt file found 2025-09-16 06:35:24,022 DEBUG --- stderr --- 2025-09-16 06:35:24,022 DEBUG 2025-09-16 06:35:24,022 INFO 2025-09-16 06:35:24,022 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-10/cmd.txt | grep "^\d+ .+$" 2025-09-16 06:35:24,022 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-16 06:35:24,551 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 06:35:24,551 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:24,551 DEBUG --- stdout --- 2025-09-16 06:35:24,551 DEBUG 4 /results/orders/benchmark-10/cmd.txt 2025-09-16 06:35:24,551 DEBUG --- stderr --- 2025-09-16 06:35:24,552 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-10/cmd.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/cmd.txt 2025-09-16 06:35:24,552 INFO 2025-09-16 06:35:24,552 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-10/cmd.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/cmd.txt 2025-09-16 06:35:24,552 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-16 06:35:25,097 INFO [loop_until]: OK (rc = 0) 2025-09-16 06:35:25,097 DEBUG --- stdout --- 2025-09-16 06:35:25,097 DEBUG tar: Removing leading `/' from member names 2025-09-16 06:35:25,097 DEBUG --- stderr --- 2025-09-16 06:35:25,097 DEBUG 2025-09-16 06:35:25,097 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/cmd.txt is small (1 lines), printing all lines: 2025-09-16 06:35:25,097 INFO ----- output ----- /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@perf-haproxy" --results-folder="/results/gatling/benchmark-10" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=M@ythe4th -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-09-16 06:35:25] benchmark-10 show_rc : Display rc order ________________________________________________________________________________ 2025-09-16 06:35:25,097 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/rc.txt is small (1 lines), printing all lines: 2025-09-16 06:35:25,097 INFO ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [2025-09-16 06:35:25] benchmark-10 show_stdout : Display stdout order ________________________________________________________________________________ 2025-09-16 06:35:25,114 DEBUG [print_head_tail]: Print head (100 lines) and tail (100 lines) of input file (/mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stdout.txt is 120503 lines): 2025-09-16 06:35:25,137 INFO ----- output ----- 2025-09-16 06:34:46,928 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmpcbp5e4r9 2025-09-16 06:34:46,932 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-10" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 06:35:08.699 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'authAsPolicyAdmin' failed for user 1: status.find.in([200, 209], 304), found 401 06:35:08.712 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: authAsPolicyAdmin: KO status.find.in([200, 209], 304), found 401 ========================= Session: Session(getToken,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@8b5e52c, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@1e23b4ce, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, 22286a7b-1bc5-412b-b78f-e49dfe808b59 -> 0),KO,List(ExitOnCompleteLoopBlock(22286a7b-1bc5-412b-b78f-e49dfe808b59)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x00007efe7c6a0000@74d73c95,io.netty.channel.epoll.EpollEventLoop@f79a760) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: X-OpenAM-Username: policyAdmin Content-Type: application/json Accept-API-Version: resource=2.0, protocol=1.0 X-OpenAM-Password: Pa_ssw0rd user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 401 Unauthorized headers: x-frame-options: SAMEORIGIN content-security-policy-report-only: frame-ancestors 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline' x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache content-type: application/json Content-Length: 62 date: Tue, 16 Sep 2025 05:35:08 GMT x-forgerock-transactionid: 885f1064-1ce3-4125-9cb4-4f43e1d27245 strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"code":401,"reason":"Unauthorized","message":"Login failure"} <<<<<<<<<<<<<<<<<<<<<<<<< 06:35:08.715 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-8' crashed with 'j.u.NoSuchElementException: No attribute named 'policyEvalTokenId' is defined', forwarding to the next one 06:35:08.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:08.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:08.996 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:08.996 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:09.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:09.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:10.492 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:10.493 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one ================================================================================ 2025-09-16 05:35:10 GMT 5s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=2 KO=1 ) > authAsPolicyAdmin (OK=0 KO=1 ) > restLoginInitiate (OK=2 KO=0 ) ---- Errors -------------------------------------------------------------------- > status.find.in([200, 209], 304), found 401 1 (100.0%) ---- getToken ------------------------------------------------------------------ [--------------------------------------------------------------------------] 0% waiting: 0 / active: 1 / done: 0 ---- idc.benchmark.Mix2024 ----------------------------------------------------- [- ] 0% waiting: 35864 / active: 3 / done: 1 ================================================================================ 06:35:10.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:10.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:11.018 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 06:35:11.492 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:11.492 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:11.800 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 06:35:11.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:11.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:12.492 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:12.492 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:12.706 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 06:35:12.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:12.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:13.181 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 06:35:13.492 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:13.492 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:13.765 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 06:35:13.991 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:13.992 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:14.189 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 06:35:14.325 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 06:35:14.325 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 06:35:14.658 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one [...] > loginTreeWorkflow_restLoginUsernamePasswordCallback (OK=1260 KO=0 ) ---- Errors -------------------------------------------------------------------- > EvalPolicy: No attribute named 'tokenIdCookieName' is defined 11534 (99.94%) > status.find.in([200, 209], 304), found 401 2 ( 0.02%) > status.find.in([200, 209], 304), found 502 2 ( 0.02%) > status.find.is(302), but actually found 502 1 ( 0.01%) > userinfo: No attribute named 'access_token' is defined 1 ( 0.01%) > UI AM revoke1: No attribute named 'access_token' is defined 1 ( 0.01%) ---- getToken ------------------------------------------------------------------ [--------------------------------------------------------------------------] 0% waiting: 0 / active: 1 / done: 0 ---- idc.benchmark.Mix2024 ----------------------------------------------------- [#########################################################################-] 99% waiting: 291 / active: 2 / done: 35575 ================================================================================ ================================================================================ 2025-09-16 06:35:05 GMT 3600s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=271533 KO=5 ) > authAsPolicyAdmin (OK=0 KO=2 ) > restLoginInitiate (OK=22782 KO=0 ) > userRestlogin (OK=11534 KO=0 ) > restLoginUsernamePasswordCallback (OK=7572 KO=0 ) > UI-Post-Login - stage 1 (OK=7572 KO=0 ) > UI-Post-Login - stage 1 Redirect 1 (OK=7572 KO=0 ) > UI AM validateGoto (OK=7572 KO=0 ) > UI AM idFromSession (OK=7572 KO=0 ) > User Authorize PKCE stage (OK=15143 KO=1 ) > User AccessToken PKCE stage (OK=15142 KO=0 ) > UI-Post-Login - stage 2 (OK=7572 KO=0 ) > UI-Post-Login - stage 2 Redirect 1 (OK=7572 KO=0 ) > UI Enduser appAuthHelper1 (OK=7572 KO=0 ) > UI Enduser appAuthHelper2 (OK=7572 KO=0 ) > UI-Post-Login - stage 3 (OK=7570 KO=1 ) > UI-Post-Login - stage 3 Redirect 1 (OK=7570 KO=0 ) > UI Enduser sessioncheck (OK=7570 KO=0 ) > UI IDM uiconfig (OK=7570 KO=0 ) > UI IDM version (OK=7570 KO=0 ) > UI IDM login (OK=7569 KO=1 ) > UI IDM features (OK=7570 KO=0 ) > UI IDM privilege (OK=7570 KO=0 ) > UI IDM user (OK=7570 KO=0 ) > UI IDM dashboard (OK=7570 KO=0 ) > UI IDM managed user schema (OK=7570 KO=0 ) > UI IDM oidcToken (OK=7571 KO=0 ) > userinfo (OK=7570 KO=0 ) > UI-Logout (OK=7571 KO=0 ) > UI-Logout Redirect 1 (OK=7571 KO=0 ) > UI AM revoke1 (OK=7570 KO=0 ) > UI AM revoke2 (OK=7571 KO=0 ) > UI AM endsession (OK=7571 KO=0 ) > loginTreeWorkflow_restLoginInitiate (OK=1260 KO=0 ) > loginTreeWorkflow_restLoginUsernamePasswordCallback (OK=1260 KO=0 ) ---- Errors -------------------------------------------------------------------- > EvalPolicy: No attribute named 'tokenIdCookieName' is defined 11534 (99.94%) > status.find.in([200, 209], 304), found 401 2 ( 0.02%) > status.find.in([200, 209], 304), found 502 2 ( 0.02%) > status.find.is(302), but actually found 502 1 ( 0.01%) > userinfo: No attribute named 'access_token' is defined 1 ( 0.01%) > UI AM revoke1: No attribute named 'access_token' is defined 1 ( 0.01%) ---- getToken ------------------------------------------------------------------ [--------------------------------------------------------------------------] 0% waiting: 0 / active: 1 / done: 0 ---- idc.benchmark.Mix2024 ----------------------------------------------------- [#########################################################################-] 99% waiting: 291 / active: 2 / done: 35575 ================================================================================ 07:35:05.990 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'userRestlogin' failed for user 35578: j.i.IOException: Premature close 07:35:05.990 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: userRestlogin: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,35578,HashMap(gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@7beaa5a3, username -> user.7024897, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@14a806ba, password -> Pa_ssw0rd),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x00007efe7c6a0000@658f4bc6,io.netty.channel.epoll.EpollEventLoop@2bef51f2) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: Content-Type: apppliation/json Accept-API-Version: resource=2.0, protocol=1.0 user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 X-OpenAM-Username: user.7024897 X-OpenAM-Password: Pa_ssw0rd accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 07:35:05.991 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 07:35:05.993 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 07:35:05.993 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 2025-09-16 07:35:06,065 INFO Gatling simulation run ended successfully 2025-09-16 07:35:06,065 DEBUG No archive name provided. No archive will be created ________________________________________________________________________________ [2025-09-16 06:35:25] benchmark-10 show_stderr : Display stderr order ________________________________________________________________________________ 2025-09-16 06:35:25,137 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stderr.txt is small (0 lines), printing all lines: 2025-09-16 06:35:25,137 INFO ----- output ----- ________________________________________________________________________________ [2025-09-16 06:35:25] benchmark-10 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2025-09-16 06:35:25] benchmark-10 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2025-09-16 06:35:25] benchmark-10 post : Post method ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped