--Task-- name: benchmark-550 enabled: True class_name: GatlingTask source_name: overseer-0 source_namespace: >default< target_name: am target_namespace: >default< start: 0.0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates below loop: False interval: None dependencies: [] wait_for: ['benchmark-500'] preceding_task: benchmark-500 options: {'duration': 3600, 'java-opts': '-Drealm=alpha -Duser_password=Pa_ssw0rd -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html', 'testCase': 'idc.benchmark.Mix2024', 'testMode': 'open', 'throughput': 550} group_name: None Current dir: /mnt/disk1/xslou/workshop/lodestar-fork/pyrock 2025-09-19 21:40:00,221 INFO 2025-09-19 21:40:00,221 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 21:40:00,221 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 21:40:00,508 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 21:40:00,508 INFO [loop_until]: OK (rc = 0) 2025-09-19 21:40:00,509 DEBUG --- stdout --- 2025-09-19 21:40:00,509 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 21:40:00,509 DEBUG --- stderr --- 2025-09-19 21:40:00,510 DEBUG 2025-09-19 21:40:00,510 INFO Lodestar assumed that full stack is deployed so will set all products host URLs in Gatling system properties 2025-09-19 21:40:00 - INFO: Timeout for this Task has changed to 1h 12m (4320 seconds) 2025-09-19 21:40:00 - INFO: timeout was calculated based on simulation duration parameter 2025-09-19 21:40:00 - INFO: Interval for this Task has changed to 2m (120 seconds) 2025-09-19 21:40:00 - INFO: interval was set to Task default because it was unset, based on self.timeout value because task is allowed to stop by itself Pod log file : /results/orders/benchmark-550/stdout.txt Pod log : /results/orders/benchmark-550 ________________________________________________________________________________ [2025-09-19 21:40:00] benchmark-550 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@perf-haproxy" --results-folder="/results/gatling/benchmark-550" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=550 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-09-19 21:40:00] benchmark-550 create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file: /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/order.json { "name": "benchmark-550", "command": " /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class=\"com.forgerock.pyrock.Loadtest\" --run-description=\"idc.benchmark.Mix2024@perf-haproxy\" --results-folder=\"/results/gatling/benchmark-550\" --no-reports --simulation-jvm-args=\"-XX:+ExitOnOutOfMemoryError -Xmx4G\" --simulation-system-properties=\"-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=550 -Duser_password=Pa_ssw0rd\"", "out-dir": "/results/orders/benchmark-550" } ________________________________________________________________________________ [2025-09-19 21:40:00] benchmark-550 loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2025-09-19 21:40:00,520 INFO 2025-09-19 21:40:00,520 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2025-09-19 21:40:00,520 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-09-19 21:40:00,731 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 21:40:00,731 INFO [loop_until]: OK (rc = 0) 2025-09-19 21:40:00,731 DEBUG --- stdout --- 2025-09-19 21:40:00,731 DEBUG WAITING 2025-09-19 21:40:00,731 DEBUG --- stderr --- 2025-09-19 21:40:00,731 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 35 0 --:--:-- --:--:-- --:--:-- 35 ________________________________________________________________________________ [2025-09-19 21:40:00] benchmark-550 start_order : Send the order to the overseer pod ________________________________________________________________________________ 2025-09-19 21:40:00,732 INFO 2025-09-19 21:40:00,732 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 21:40:00,732 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 21:40:01,017 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 21:40:01,017 INFO [loop_until]: OK (rc = 0) 2025-09-19 21:40:01,017 DEBUG --- stdout --- 2025-09-19 21:40:01,017 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 21:40:01,017 DEBUG --- stderr --- 2025-09-19 21:40:01,017 DEBUG 2025-09-19 21:40:01,017 INFO 2025-09-19 21:40:01,017 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/order.json overseer-0-57d79d468-khs5t:/results/orders/order.json 2025-09-19 21:40:01,018 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 21:40:01,933 INFO [loop_until]: OK (rc = 0) 2025-09-19 21:40:01,933 DEBUG --- stdout --- 2025-09-19 21:40:01,933 DEBUG 2025-09-19 21:40:01,933 DEBUG --- stderr --- 2025-09-19 21:40:01,933 DEBUG Order has been successfully forwarded to overseer 2025-09-19 21:40:01 - INFO: Check if Gatling simulation started successfully 2025-09-19 21:40:01,933 INFO 2025-09-19 21:40:01,933 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- head --lines 5000 /results/orders/benchmark-550/stdout.txt | grep "Simulation com.forgerock.pyrock.Loadtest started..." 2025-09-19 21:40:01,933 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-09-19 21:40:02,490 INFO [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-19 21:40:08,041 INFO [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-19 21:40:13,603 INFO [loop_until]: Function succeeded after 11s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-19 21:40:19,160 INFO [loop_until]: Function succeeded after 17s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-19 21:40:24,759 INFO [loop_until]: Function succeeded after 22s (rc=0) - expected pattern found 2025-09-19 21:40:24,759 INFO [loop_until]: OK (rc = 0) 2025-09-19 21:40:24,759 DEBUG --- stdout --- 2025-09-19 21:40:24,759 DEBUG 2025-09-19 22:40:04,314 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmp9djk_4qz 2025-09-19 22:40:04,318 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-550" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=550 -Duser_password=Pa_ssw0rd System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 2025-09-19 21:40:24,759 DEBUG --- stderr --- 2025-09-19 21:40:24,759 DEBUG ________________________________________________________________________________ [2025-09-19 21:40:24] benchmark-550 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 1h 12m. Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 2m, retry Order is running since 4m, retry Order is running since 6m, retry Order is running since 8m 01s, retry Order is running since 10m 01s, retry Order is running since 12m 01s, retry Order is running since 14m 01s, retry Order is running since 16m 02s, retry Order is running since 18m 02s, retry Order is running since 20m 02s, retry Order is running since 22m 02s, retry Order is running since 24m 03s, retry Order is running since 26m 03s, retry Order is running since 28m 03s, retry Order is running since 30m 04s, retry Order is running since 32m 04s, retry Order is running since 34m 04s, retry Order is running since 36m 04s, retry Order is running since 38m 05s, retry Order is running since 40m 05s, retry Order is running since 42m 05s, retry Order is running since 44m 06s, retry Order is running since 46m 06s, retry Order is running since 48m 06s, retry Order is running since 50m 06s, retry Order is running since 52m 07s, retry Order is running since 54m 07s, retry Order is running since 56m 07s, retry Order is running since 58m 07s, retry Order is running since 1h, retry Order is running since 1h 02m, retry Order is running since 1h 04m, retry Order is running since 1h 06m, retry Order is running since 1h 08m, retry Order is running since 1h 10m, retry Order is running since 1h 12m, retry ERROR : Order is still running after timeout 1h 12m Setting result to TIMEOUT ________________________________________________________________________________ [2025-09-19 22:52:34] benchmark-550 stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2025-09-19 22:52:34,624 INFO 2025-09-19 22:52:34,624 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 22:52:34,624 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 22:52:34,932 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 22:52:34,932 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:34,932 DEBUG --- stdout --- 2025-09-19 22:52:34,932 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 22:52:34,932 DEBUG --- stderr --- 2025-09-19 22:52:34,932 DEBUG 2025-09-19 22:52:34,933 INFO 2025-09-19 22:52:34,933 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- rm -f /results/orders/order.json 2025-09-19 22:52:34,933 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 22:52:35,535 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:35,535 DEBUG --- stdout --- 2025-09-19 22:52:35,535 DEBUG 2025-09-19 22:52:35,535 DEBUG --- stderr --- 2025-09-19 22:52:35,535 DEBUG Order ran during 1h 12m ________________________________________________________________________________ [2025-09-19 22:52:35] benchmark-550 download_logs : Download individual order files ________________________________________________________________________________ 2025-09-19 22:52:35 - PASS: file already downloaded /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/order.json 2025-09-19 22:52:35,536 INFO 2025-09-19 22:52:35,536 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 22:52:35,536 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 22:52:35,817 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 22:52:35,817 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:35,817 DEBUG --- stdout --- 2025-09-19 22:52:35,818 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 22:52:35,818 DEBUG --- stderr --- 2025-09-19 22:52:35,818 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-19 22:52:35,818 INFO 2025-09-19 22:52:35,818 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-550/rc.txt" ]; then echo "/results/orders/benchmark-550/rc.txt file found"; exit 0; else echo "/results/orders/benchmark-550/rc.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-550/rc.txt file found" 2025-09-19 22:52:35,818 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 22:52:36,375 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 22:52:36,375 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:36,375 DEBUG --- stdout --- 2025-09-19 22:52:36,375 DEBUG /results/orders/benchmark-550/rc.txt file found 2025-09-19 22:52:36,375 DEBUG --- stderr --- 2025-09-19 22:52:36,375 DEBUG 2025-09-19 22:52:36,375 INFO 2025-09-19 22:52:36,375 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-550/rc.txt | grep "^\d+ .+$" 2025-09-19 22:52:36,375 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-19 22:52:36,922 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 22:52:36,922 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:36,922 DEBUG --- stdout --- 2025-09-19 22:52:36,922 DEBUG 0 /results/orders/benchmark-550/rc.txt 2025-09-19 22:52:36,922 DEBUG --- stderr --- 2025-09-19 22:52:36,922 DEBUG - Download file (size 0B) source : /results/orders/benchmark-550/rc.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/rc.txt 2025-09-19 22:52:36,923 INFO 2025-09-19 22:52:36,923 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-550/rc.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/rc.txt 2025-09-19 22:52:36,923 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-19 22:52:37,560 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:37,560 DEBUG --- stdout --- 2025-09-19 22:52:37,560 DEBUG 2025-09-19 22:52:37,561 DEBUG --- stderr --- 2025-09-19 22:52:37,561 DEBUG 2025-09-19 22:52:37,561 INFO 2025-09-19 22:52:37,561 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 22:52:37,561 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 22:52:37,845 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 22:52:37,845 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:37,845 DEBUG --- stdout --- 2025-09-19 22:52:37,845 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 22:52:37,845 DEBUG --- stderr --- 2025-09-19 22:52:37,845 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-19 22:52:37,846 INFO 2025-09-19 22:52:37,846 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-550/stderr.txt" ]; then echo "/results/orders/benchmark-550/stderr.txt file found"; exit 0; else echo "/results/orders/benchmark-550/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-550/stderr.txt file found" 2025-09-19 22:52:37,846 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 22:52:38,390 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 22:52:38,390 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:38,390 DEBUG --- stdout --- 2025-09-19 22:52:38,390 DEBUG /results/orders/benchmark-550/stderr.txt file found 2025-09-19 22:52:38,390 DEBUG --- stderr --- 2025-09-19 22:52:38,390 DEBUG 2025-09-19 22:52:38,390 INFO 2025-09-19 22:52:38,390 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-550/stderr.txt | grep "^\d+ .+$" 2025-09-19 22:52:38,390 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-19 22:52:38,946 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 22:52:38,946 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:38,946 DEBUG --- stdout --- 2025-09-19 22:52:38,946 DEBUG 0 /results/orders/benchmark-550/stderr.txt 2025-09-19 22:52:38,946 DEBUG --- stderr --- 2025-09-19 22:52:38,947 DEBUG - Download file (size 0B) source : /results/orders/benchmark-550/stderr.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/stderr.txt 2025-09-19 22:52:38,947 INFO 2025-09-19 22:52:38,947 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-550/stderr.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/stderr.txt 2025-09-19 22:52:38,947 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-19 22:52:39,498 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:39,499 DEBUG --- stdout --- 2025-09-19 22:52:39,499 DEBUG tar: Removing leading `/' from member names 2025-09-19 22:52:39,499 DEBUG --- stderr --- 2025-09-19 22:52:39,499 DEBUG 2025-09-19 22:52:39,499 INFO 2025-09-19 22:52:39,499 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 22:52:39,499 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 22:52:39,795 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 22:52:39,795 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:39,795 DEBUG --- stdout --- 2025-09-19 22:52:39,795 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 22:52:39,795 DEBUG --- stderr --- 2025-09-19 22:52:39,795 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-19 22:52:39,796 INFO 2025-09-19 22:52:39,796 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-550/stdout.txt" ]; then echo "/results/orders/benchmark-550/stdout.txt file found"; exit 0; else echo "/results/orders/benchmark-550/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-550/stdout.txt file found" 2025-09-19 22:52:39,796 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 22:52:40,345 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 22:52:40,345 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:40,345 DEBUG --- stdout --- 2025-09-19 22:52:40,345 DEBUG /results/orders/benchmark-550/stdout.txt file found 2025-09-19 22:52:40,345 DEBUG --- stderr --- 2025-09-19 22:52:40,345 DEBUG 2025-09-19 22:52:40,345 INFO 2025-09-19 22:52:40,345 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-550/stdout.txt | grep "^\d+ .+$" 2025-09-19 22:52:40,345 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-19 22:52:40,892 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 22:52:40,892 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:40,892 DEBUG --- stdout --- 2025-09-19 22:52:40,892 DEBUG 117320 /results/orders/benchmark-550/stdout.txt 2025-09-19 22:52:40,892 DEBUG --- stderr --- 2025-09-19 22:52:40,892 DEBUG - Download file (size 117.32MB exceeds limit of 100MB, only first 1000 and last 1000 lines will be downloaded) source : /results/orders/benchmark-550/stdout.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/stdout.txt 2025-09-19 22:52:40,893 INFO 2025-09-19 22:52:40,894 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- head -n 1000 /results/orders/benchmark-550/stdout.txt 2025-09-19 22:52:41,626 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2025-09-19 22:52:41,626 DEBUG --- stdout --- 2025-09-19 22:52:41,627 DEBUG [print_head_tail]: Print head (80 lines) and tail (80 lines) of input string (1000 lines): 2025-09-19 22:52:41,627 DEBUG ----- output ----- 2025-09-19 22:40:04,314 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmp9djk_4qz 2025-09-19 22:40:04,318 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-550" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=550 -Duser_password=Pa_ssw0rd System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 22:40:24.763 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'authAsPolicyAdmin' failed for user 1: status.find.in([200, 209], 304), found 401 22:40:24.775 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: authAsPolicyAdmin: KO status.find.in([200, 209], 304), found 401 ========================= Session: Session(getToken,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@4ab74239, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@7e46e47d, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, b1262f98-1175-4afb-a909-2d54d838ba2c -> 0),KO,List(ExitOnCompleteLoopBlock(b1262f98-1175-4afb-a909-2d54d838ba2c)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x0000795d4069bb38@53c6ef70,io.netty.channel.epoll.EpollEventLoop@f79a760) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: X-OpenAM-Username: policyAdmin Content-Type: application/json Accept-API-Version: resource=2.0, protocol=1.0 X-OpenAM-Password: Pa_ssw0rd user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 401 Unauthorized headers: x-frame-options: SAMEORIGIN content-security-policy-report-only: frame-ancestors 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline' x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache content-type: application/json Content-Length: 62 date: Fri, 19 Sep 2025 21:40:24 GMT x-forgerock-transactionid: 10bb6834-1d57-4be9-8def-762f3db49e1e strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"code":401,"reason":"Unauthorized","message":"Login failure"} <<<<<<<<<<<<<<<<<<<<<<<<< 22:40:24.777 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-8' crashed with 'j.u.NoSuchElementException: No attribute named 'policyEvalTokenId' is defined', forwarding to the next one 22:40:24.882 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:24.882 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:24.993 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:24.993 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.105 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.105 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.216 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.216 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.327 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.328 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.439 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.440 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.552 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.552 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.564 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.564 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.589 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.589 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.626 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.626 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.659 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.659 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.663 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.664 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.700 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:25.701 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.701 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.738 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.738 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one [...] date: Fri, 19 Sep 2025 21:40:29 GMT x-forgerock-transactionid: c78e96d2-780e-432e-a4b5-f0320688da09 strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"authId":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ3aGl0ZWxpc3Qtc3RhdGUiOiI4OWZmYjg5NS02NTk5LTQyNjAtOGU0NS1lNmFlZGRkNGY3YjciLCJvdGsiOiI2Zjc3YTM0MmY0czZwbDF0ZDE4ZTM4aXZhMCIsInJlYWxtIjoiL2FscGhhIiwic2Vzc2lvbklkIjoiKkFBSlRTUUFDTURJQUJIUjVjR1VBQ0VwWFZGOUJWVlJJQUFKVE1RQUNNREUuKmV5SjBlWEFpT2lKS1YxUWlMQ0pqZEhraU9pSktWMVFpTENKaGJHY2lPaUpJVXpJMU5pSjkuWlhsS01HVllRV2xQYVVwTFZqRlJhVXhEU214aWJVMXBUMmxLUWsxVVNUUlJNRXBFVEZWb1ZFMXFWVEpKYVhkcFdWZDRia2xxYjJsYVIyeDVTVzR3TGk1elpFbG9kVk5LWmtWQ00yeElRbU5LTWtaTVFuaFJMbEYxUlhrd016RjFSSGRNUTNwa2JIY3RXRkJRZDJ0ZlR6RkNVbTl1WDJOaFF6WnNORFJPWlRSUWRHWTBNREl6VW5GaVlWVk5YeTF5YlZSTk1rdFNSelJKVUhKMk5VeHdUblJqWWpKRFFYRkVWRFk0WVVOVFIwaEZNaTFtU1hGYWFGOVlSRTVzUTBVNVlXZDZTVFpvZG14U2MyTm9aMGd4VTFGdGFGaFZlbUpNYzFsUVJFRnpSVGhRTFZWd05tdHVTM2RPTWpaMlFucFNUMnRHYmt4aFFUbDJkR04wY25Kbk5uWXhhRkZNYWt4U2FHczBSRE5oWld0TmFWZHpOSEJhUm10WWMybFNWVkkwVm5rdGNERnRPVE5pY2tkMGJVUkRTMGhoWDFOT1RreE5iMjVwWWtadVpFeDBVMmRpWkc1QlIzTXhiWE55WDJVNVUzWjZXbmd3UzIxdFltVnBTRkExVUhSeGRubHNhMnRCVEhkdlYwb3dWVEZpTkVWdFNXNWpRMUpuZEdaSVRWRTJkVjl2Um05eVJHZGpNak14UjBvME1XZHZSbGxOWkhKeVpWWmtOMHBzWWpWVE1ISk9jMUF3UkhadWVXWXpORWx1ZVdOM1lYQmFjV2N0Vm14TlFtTkxla3RMWVZsclF6TXdWQzFRTW5Zd05IWlBaa2xrVG1SUlREWkRhekZCTlhCSlZFcGZOVXhQT1RVeVdXWlNWVFpZWTJSVlNHNWFaWFp3Ulc5cFluUlVhWEo1WjFkUGVVUlhNM1JCWkhaQllqTkNkR015Y25aTmVHNXFUbEJrWkdka1dXWjRaUzFQUjJwb1VYWnlVR0pVTWxKNldEaGpYMVZXYURRemRUSk1lR1ZOVm5wd1NXUnpkME5PT0dsVlltaHpPRjlxWXpOaWNGUk1USFU1V2xseFYybHhhR1pLZGpnd1lUSnFiRzF1VVVGelRFVjJUVGxNWXpOWFpFRlViVUprVXpKWFJFMXJXVlJwUVRGcU1WQTBTVVo0Wms4MGEzTnZWUzFWTFZJdGRqUlNNbDlsUVZaM1RFMTViVXQ1ZWtKb01YbzRMV056U2pGMVEzcEdYMVppVUc4NWF5MVlTRkpOZG1aaU9WazNNMGhtTUVVeFpHZDNOM1ZUZEhabVFsZE5YME5yZWxwNFdHUkVSMlZ6VGs5SE5tY3lkREpLV25CU09WWnVTbkJHYzBwNVNGQkNPVE5UY2pobk1rbEtOSEJmVldkNlVIcE5iRVJNTURsbGVrNXFSekJDWjBRMmFERXlUV0ZQY1RKdlNWcE5kakZVYVRKbmJXSkJVV0ZWVDFaS09UQmxTSFZYZERoVFdVUnFVR1ZoZUV0VWJEa3hTRlY1WjJGT1ZVdDRabU5JU1ZWalpuQkhPRE5pV1VaNmVuWnJRbXRaTFhkZlNWbzNZbTlqWkVRMVNWOWxPR1JGYTBkcU0yaFNielF5UmxSNlJHdFJWazVzUkdKU1ZWSnpNMkpJUmsxM1MwTXRaWGxUWkZCbmEwNTBabE5hVmt0RFRHY3dkR3Q2VjFCemFGbDFRMEY2UVVScGFYZEJWVTlzT0VSUVprRkhPSGR6UjFoQlZrUm5RMEkzYms5NVdXbzBTSFV5ZDFKUlYwMW5XRGR6T0VWbmNWcG1ZMTlHWlVab1QyaEdXRUZuZVdGU1VHaGZaalpWWHkwd1prdGplalpXZGpGVldtYzFhMXBJV2xvMk9FWkdSeTE0U0hjNVJqZHlMVTgxYjFoZmIycExlRTR5VWtKTE5VdDBhams1YVZjNVNqQm1jVlp0YjBOdldXdzJNVXR4VEdzNFZFOWpaMnRmUVZCU2FXWXROR2hPVjBaTVVWbHVVbGd6TWtscFRXMXRWa3B4ZFdwaE5sQlFVRFJpWTNGak9ISXlVMGRmU2t0bFFVeHhkakYzTmxCa1ZrODNOMlY0UjFGT0xVcE9hbGgyUjFSdFpqRlFlbk0yVUcxVVJ6UlFiVkZvVlRBdGNtSklRVWxuVDFveloydG1WbEpNYzA5Wk5taEZXRkpvUTA5NGVVY3lRWFZ4U1ZKSk1EQllOamhTT0RsQ0xXUlZTRFppUkRCZlduVlNYeTFxTjNKMFprTnpWVFZ4ZWpKTWVXc3RiMVY0TnpjNFVIazRkbUZwVWtwZlozaDNNMHRuUkVWNVkyTnNZekJ6V20welduSjBaWGczYUZvNWVURnRNbkZYWTI1RVdHVnZkMVppZURKRVVWRjBOVUZGT0VoZk0xWnlPV2xNTlZadWFtNWxTVzgyZG1SdmFHVlBNSFpHZEhsMWNVaDZhbTFKTlZkelNFOUtYMVJLZDB4eFVtaEtOMmxKV1RSRE1qUndWa1ZmYjNabFRUTjJaR0ptTTNSeWN6UkRlbmhuWmtSS1NteG1NRUZ1WlRkM1RuQlBNWEJPYTJGNFRrMXViVWhEV1ZCM2RWUTVhM05MWkZsZk4xcEZjRlJQTWtJNFRFbzRTRzFUZVZNd2VEQmhlVmhwTlZCblh6UXdNVFpNTlhjd1NGbE5Wa0pIV1VKa1RFY3hRakJsYm5weU5tdFRhbVZFYTBWS2NVNWFWVmd3Y21odVZFSTVSRGhwZDJ4cmJIZHZNMjl3ZW5sbk1YTkRUMVJ5TkhOdVdqQjNkVFI2TVdOTmJESnJWbFp0UVdsMVRWRkNjSGwwVUd4c1pIRndUMHRGTTBKQ05ubE5TMmRCZW5KMWRYcGFlRGxpT1RaTmNHaFlaWEJOYTNkQldEUmtUblZqVGs4MmFFRXhaRVZ5YUZGbE5XWlVlbkZmUVZKek1GVk1UVmRwT1dOSGNHVm1hbTVDVjFKbU1pMURlVjlUZEdWSE9WQXliV3B6ZUVWamNreE1UMmQzTjBrd1MwOW9TMDh6WjFsSWEyRlRWR0l6YzB3NGFWOUxXakZMZGxaSU1sSTJlV00wT0dWM2JVeHVkMmhVWkRoTFNIbFRaMEZ5VEdZdGVrd3hiekJxT1ZaNmFVRm9kamszVUZoZlFYUXlOMlJtWTFkMWJ6ZFpjblZKTVhkSFdsTkxWVnAzT0ZwR1NGZzBNaTFoYW1oTll6bGZRWEZrT1ZCNU9HdzVNemw1U0cxc1VtbFJZMkp2TFRjd1RXdHdhVzF1TnpWcVNqSnhaV05hTVZsamQwMXNWRTFWWlZCTkxYRldaelZtVVVoWlFteFNZbTgxY21NeVRtOUtkMlZXVVdWRmRGbEpObGxyUW1JMlRXOTFOMUZsTW1WNWQzcHFabTFDYldsUVZ6RjZNREUxZWpscGRVNU1hbkZoWjA5RFRGb3pXazE0WkdkU1ZYRm5UM0ZTYTBKdFJERlBaRmwwT0ZKR0xUVnVOVEpaZEc1R1FWQldjRTFEZFVONmIzUlZaM050WlY5RFZHMDFja05tZGtnd2FtNXFhWEpDZVdSaVVYTnZSalJWWWpkdVpFcGlXVzFvVDB0aVp6Tm1PVmRYYUhWSVZGQnpURkJMV1hsTE1raHhVRWRuWm5kR2FXOVVkemhETFVob2RtMUROVmQzWlhRNFJ6VTBSRWhWYjJWSlZ6WTVVbTFRVjFabU5YcGhSVlpqUWtjMVJGcFNNM2x3T0VvMGIzcExNM1o2VVROSlZXcEdhR3BZYXpBeFRGTnVNVVV6WVdWWWFuaERNakJqYkZSbmRHNUlkUzFmVWxsUFNXTlJkeTVpU1RCVmVEUmtNMEpEVVhsVVVsTnhVWGRtYWxsbi44d3I3b3ZVTnowUzdVZElPM0gxWkV2S19rdHZ5MGtRVzVQTWZEZkVUN2dvIiwiZXhwIjoxNzU4MzE4MzI5LCJpYXQiOjE3NTgzMTgwMjl9.QzNaZDP5XiYfOfXo2mTBpu6aqyI2TQXZKqYam1TsjHI","callbacks":[{"type":"BooleanAttributeInputCallback","output":[{"name":"name","value":"preferences/updates"},{"name":"prompt","value":"Send me news and updates"},{"name":"required","value":false},{"name":"policies","value":{}},{"name":"failedPolicies","value":[]},{"name":"validateOnly","value":false},{"name":"value","value":false}],"input":[{"name":"IDToken1","value":false},{"name":"IDToken1validateOnly","value":false}],"_id":2},{"type":"BooleanAttributeInputCallback","output":[{"name":"name","value":"preferences/marketing"},{"name":"prompt","value":"Send me special offers and services"},{"name":"required","value":false},{"name":"policies","value":{}},{"name":"failedPolicies","value":[]},{"name":"validateOnly","value":false},{"name":"value","value":false}],"input":[{"name":"IDToken2","value":false},{"name":"IDToken2validateOnly","value":false}],"_id":3}],"header":"Please select your preferences"} <<<<<<<<<<<<<<<<<<<<<<<<< 22:40:29.998 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:30.001 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.001 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.011 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.011 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.021 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.021 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.031 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.031 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.041 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.041 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.042 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:30.050 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.050 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.051 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:30.060 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.060 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.070 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.070 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.072 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:30.080 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.080 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.090 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.090 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.100 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.100 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.102 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:30.110 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.110 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.120 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.120 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.130 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.130 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.140 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.140 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.149 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.149 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.159 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.159 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.169 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.169 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.179 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.179 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.189 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.189 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.192 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:30.199 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.199 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.209 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.209 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.219 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.219 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.229 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.229 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.237 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:30.239 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.239 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.248 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.248 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.258 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.258 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.268 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.268 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.275 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:30.278 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.278 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.288 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.288 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:30.298 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:30.298 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 2025-09-19 22:52:41,627 DEBUG --- stderr --- 2025-09-19 22:52:41,627 DEBUG 2025-09-19 22:52:41,627 INFO 2025-09-19 22:52:41,627 INFO 2025-09-19 22:52:41,628 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- tail -n 1000 /results/orders/benchmark-550/stdout.txt 2025-09-19 22:52:42,376 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2025-09-19 22:52:42,377 DEBUG --- stdout --- 2025-09-19 22:52:42,377 DEBUG [print_head_tail]: Print head (80 lines) and tail (80 lines) of input string (1000 lines): 2025-09-19 22:52:42,377 DEBUG ----- output ----- 22:48:07.518 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.520 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.520 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.520 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.520 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.520 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:07.521 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.521 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.521 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.521 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.522 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.522 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.524 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.524 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.526 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.526 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.527 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.527 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.529 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:07.531 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.531 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.533 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.533 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.533 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.533 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.534 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.534 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.539 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.539 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.540 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.540 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.540 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.540 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.542 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:07.543 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.543 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.545 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:07.546 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.546 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.549 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.549 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.549 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.549 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.552 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:07.555 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:07.556 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:07.558 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.558 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.560 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.560 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.562 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:07.562 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:07.562 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:07.570 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'userRestlogin' failed for user 247404: jsonPath($.tokenId).find.exists, found nothing 22:48:07.570 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: userRestlogin: KO jsonPath($.tokenId).find.exists, found nothing ========================= Session: Session(idc.benchmark.Mix2024,247404,HashMap(gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@530eda71, username -> user.3742713, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@1502279c, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758318487570))), password -> Pa_ssw0rd),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x0000795d4069bb38@d0a99e1,io.netty.channel.epoll.EpollEventLoop@30f5a68a) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: Content-Type: apppliation/json Accept-API-Version: resource=2.0, protocol=1.0 user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 X-OpenAM-Username: user.3742713 X-OpenAM-Password: Pa_ssw0rd accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 200 OK headers: [...] x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache set-cookie: amlbcookie=01; Path=/; Domain=openam-perf-haproxy.forgeblocks.com; Secure; HttpOnly; SameSite=none content-type: application/json Content-Length: 5283 date: Fri, 19 Sep 2025 21:48:07 GMT x-forgerock-transactionid: e47050da-d650-4677-b461-2716402a48d1 strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"authId":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ3aGl0ZWxpc3Qtc3RhdGUiOiJlYTJjZThiOS0yNDFkLTRlODQtOWRlYS1mYWI5OTFiN2ZjOWQiLCJvdGsiOiJsbWQ5bGd1MnAzOW5jNnFqdTVkY3QyaGhwaSIsInJlYWxtIjoiL2FscGhhIiwic2Vzc2lvbklkIjoiKkFBSlRTUUFDTURJQUJIUjVjR1VBQ0VwWFZGOUJWVlJJQUFKVE1RQUNNREUuKmV5SjBlWEFpT2lKS1YxUWlMQ0pqZEhraU9pSktWMVFpTENKaGJHY2lPaUpJVXpJMU5pSjkuWlhsS01HVllRV2xQYVVwTFZqRlJhVXhEU214aWJVMXBUMmxLUWsxVVNUUlJNRXBFVEZWb1ZFMXFWVEpKYVhkcFdWZDRia2xxYjJsYVIyeDVTVzR3TGk1aFZYaFZTVE0wYTJKRE5rZEpVRWxyZEVkUWJtSkJMbFJVUW1OS2EwdG9aMmRTUVMwd2MxcE5PRkI2YmtONGIzSkhlalY2UVhWVldYYzFVakF6U21sTmJreHdWVUpWZGtobExWQlNiM2N0ZFdJMk0ySmxZblZCWDJrNVpHNU1XRmRuUzNadmFVbE9lVlZuTlVFMlpGQkRZbXcyYTJ0ck1sZHpTWFpxUTJSbGFqRmFVMmhYYkRSM05DMXNSRmM1ZEZSeGJqUldSM0ZmU0hoM2FHTnZlR2hhTmpkT1FtWTJRMHh2YkRReVkxTkdOVmd5VEV4aWNtY3phbWgzY2xwTWRXdDFVV1pYVkVkZmVUUnNYMG93U0VSVWNFaE9OMWRmYWt4Rllrc3pTRE14WjJ0S1prUXdNMnhwU1d0dVNFcHhhV3BTYVY5NFNqQTJhV3M1WkVkV0xVWjFNa1k1T1hKWlRERlhXRm96UVVJdFVVbHFVM0J3VlZWTWFXcHRSamhmYjA5NlZVZFZaRmRzYUdGeFdUaGlaamMwUmxGelFXaFVlRTFuYkVFek5uUjJjR3RYYzFKVmFWVXRNVUZQZWpkU1ZFeERaMnhzZDFoR1NFbEllVmRMTWxNeGJFdFdaaTAxYjBoSWRFVktMVE14Y0c1V2VVOXpURVJxYTJGcVVuZ3dka3gyU0UxeFJWcE9jakZUUW05b2FVSkZia3d3VEZGeU9FSlNabWxzZGw4MFFqZEpXRTFOYUVad2RFMU5XVlI0V1cxRWRYTlFXVVJYYWxvd1Z6bExja3ByVTNkbmFIcHdUV1ZEU2t4dWMzQXpNekpoWDNwUFZGRmlZblY1TW5aZk5WbDFhVVozTlZRd1VXUmlPVGhMVjFBMmQwdHZTMjVUWlhGbFZuWlNObW8zY0ZOS2NrbEJaVlJVWW1sQk56SXlaMnRWVVdkMWVXUk5YekJTUkZkdk5VazBkbmxsVjFwZlNFbHhWRVkwTVc5alVIVXRNQzFQU21JdFFqVndhR0ZuZG10TGJYSlNXbFpoZFdGWU9FdEVkRGsxVWtac1JVcFRla3h2WVMwNFNUZzFkMUpRUVRsa00yNTFUMEphZDBSRlVtUmhTbm95YldsTGJUQnpNWEl3YVY5UVJuaHJWRzVTTTI0NVFXcHRZMFZyYkVOVE9EVmxNa0pTVEdsSGNHc3RjRjlKZFc5b09XSm9ORWQzVGxkNE0xQTRiSGxSTFRkUU9HWkhiVUl5V2tod05ISnVlWHBaV1MxUFZFUnhjbGhTYWxNNE4zWTBWME5TY0d0WlR6UlZUVFIwY21wcGJua3lRMGxWVlc1dlJWUjVYMWRyTlcxNmMwUjJYMWxPZDBkWmQwZFNRams1ZUdsalNWaHpTV3REWTJReVRuVlJWWGhmTWpacUxUQnhTV3R1UWpOdVpHUnNNVE42TFY5Nk9IWnBhazFxUWpSNVZXcExka0ZMYWsxeU5GbG9jMXBTWkVoVE5GcGlhRGRZVkdNeVVsUk1RVVpZV1V4amJGVkRaVEZFWm1Sak1FdDRhMmhWT1dOVlRsRTBZVEY2V0hKaVVYWmxjSFEwVGw5amNYUnFNbDloWlMxdE5sRkZZa0ZqWW5OQ1N6ZG9NMVIwTlVGek0wazBhWGh3Y0ROeE5rVlFhemhpVnpGbFdUUXRTR2sxYUVWVVVubFZhWEJyUjFOTE0yVlNSVFZQYmpBeWFEVkljV1p2WjFCS2FsSklhR2hVYTFoeFRXSkNTbmh4YzBaa1JWTlZPVE5pYm1aT05WVlBSSE13YzB4TlRqWTFNakZEZWxvd1FWZG9OekpQV25sRmRFaEhWa2RrY25oSGQxVktlbEE1ZUVwNlNuWk9jRGt3VGxONVpIVXpjbmgyT0hZeGRtdFJORFpuV0hnMGNEaFpWMXBDTVdkUFpXVklTQzAxU0RJeFZrbFdObTVUTmxkSGQwSmFTVXQwU0Mxek9XOWxYMU4zT0ZReVlUQlZVRnBJU1hWSFpuQnBXVkpNVFdWaWFqVTRZamhTZURKRFZtZ3pWREZ1T1RaVmRHbFNXVlF3U21sUFZVdGthbE4xUmtobFgySTBRazlMWDNwMmJqTjFaa2REYUdKSlRDMWtlREJmZEhOWFJGUnFlazFEWDFSQ2FsTm1SRkpIZERreVNFdFVjM3AyZUhweFltNXhRWEZzTjJkclpESjBlbkJxWTFoaWRXdHlUMVY2YVUxbk1uVk9ValJrYkVOQlIwMW5XRUZwTTJKMVIweFJWVFJ2Wmw4M1dub3hOM1ZMU1VOTU4yTXpNMUpaWTNSaGRVWm9UMjV3U1VSMVIzcGZibU5UYW5wTVEyNTVaV3hhYzNvMFJIRk1SbTV4VjJWdlJuWXpWV1ZVUW1JMmNqa3djREZvTldrd2NFMWFaMDF5YVhabVoxTllZamMxT0d0U1FUQnlha3RTU2pWbFFVd3RSa3hyVkRCeFNXeGFTMDlSZDIxTFptTnpkMFE1ZDFoNmFsTkpkWHBCY0VwU1Z6VjBSMEV6TXpaeFdteFFZMFkyYWxwMlYyMTVMVGRaZG5oalV6STRiMHhCUWpOMWJrMUNUbU15VW5oNFgwa3dSMmhPU2xGQmRYQm9RMGt4ZDNsaFMyOVZUazl5UmswMlQxUklibWxaWm1OMU9HOXFiMTlRUTNKWGVEWk1Uazh6VFU5VFN6bEVNMkl0V1ZrMU1scEVORkZ5U1ZaRFEwMTBjRXB5VUhWeGRXNHRXV0ZoTmt4d00yd3piamRhYjNkTFlXaG1SMWxQTUV4aVNUQmhaSEZPY25OMk1qZzRka3Q0Ukd0bVQyWTRiamcxTm5VeE1tOWtNREpyUkZvMFlVcEVPRFJOWlVScmFYcEhVbU5YTld0d05FSXdjVFkxZVd3dE9VcE1hRWh1ZG5VdGJsTlJjVkYyTnpselpHOTBUalJyUWw5NFlsSlVWM2w2YlRNNFpVVm1WRkJMU1c1NGVsRXRkWFZFVGxaTllYZEVSeTAyVURSV2RtcDBZemhHUlVoNU5GbEJaMjFvY0MxdlMwVm5MVEZhUTJoelkyVkVNa05LTjFkTWVtUXlMVFJ1V1RadVEwZHFUV3RIVUdNdGMyOXpabUV5TjJGZmNXZEhiMkZVUWt4SE4yTnhVMU5NUm5sQlFrMDJhamQwUlRkR01HRndNMjlEYmxwNmQzSmxiVk01UTFadk1VZDJka3BrVlcxUk0zUmlVV0l4YWtGSWJsOVlRbHByU1Y5clNqSkRVa0ZEU0ZCS0xWRjFUMGRSYVdWWU9XOTBia0l5YVRWR1h6RXpRa1ZqUTBKak1qbHpiRkJKV2kxVGRuWkZTREU1WW1zNVFrUnJXbkpJU0RsT2QwOWZhRnBTUm5ONE5FNDVVbE5FYmkxVlRtNUdNbTlXWVU0dFRURmFRM28xVjE5R01rNUZVMVJrVjJReVNtdGlla2RZTkVsUVdYSlBhMFpqWTNKMFEzaEdXbTU1U0ZKTGR5MHlRbE5PVjNGTVVUZHZUVFpHT0ROc1JERmtMVXRmYWxnM1pHVjJhR3RoUkc1eVMxbEZNa28yU1Vzd05XVmFhREZqY2twbE1tUmhWblF5UVdoWmFVTk5URUl4ZG5WWlRqTmpkSFZYYnpjd1VrWnZibWRFV1ZkdVJIWmFWVzlxZUdSc1NVMW5NV3hGY3paR2VEWkdNR1p1VVZKS1UxQmlkVEpMTjJrM2VGWk5keTVmV1RjeGNtNUNUell5ZDFOUVZtNXFhSFJrV0VGQi5IcUNMVkVnM1hjeWZ3dVRhMXJ6dm1PRzQ5TWNaSnV5Slp1LUx0eExQaEwwIiwiZXhwIjoxNzU4MzE4Nzg3LCJpYXQiOjE3NTgzMTg0ODd9.GTpfX2R-9yuBUGJ2LQ_DhnEecuC0g1zam_6cYZuHrUA","callbacks":[{"type":"BooleanAttributeInputCallback","output":[{"name":"name","value":"preferences/updates"},{"name":"prompt","value":"Send me news and updates"},{"name":"required","value":false},{"name":"policies","value":{}},{"name":"failedPolicies","value":[]},{"name":"validateOnly","value":false},{"name":"value","value":false}],"input":[{"name":"IDToken1","value":false},{"name":"IDToken1validateOnly","value":false}],"_id":2},{"type":"BooleanAttributeInputCallback","output":[{"name":"name","value":"preferences/marketing"},{"name":"prompt","value":"Send me special offers and services"},{"name":"required","value":false},{"name":"policies","value":{}},{"name":"failedPolicies","value":[]},{"name":"validateOnly","value":false},{"name":"value","value":false}],"input":[{"name":"IDToken2","value":false},{"name":"IDToken2validateOnly","value":false}],"_id":3}],"header":"Please select your preferences"} <<<<<<<<<<<<<<<<<<<<<<<<< 22:48:08.005 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.005 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.005 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.005 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.005 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.009 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.010 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.010 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.011 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.011 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.014 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.017 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.019 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.019 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.020 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.020 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.023 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.023 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.023 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.023 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.028 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.028 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.028 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.028 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.028 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.029 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.029 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.029 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.029 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.030 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.030 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.030 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.031 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.031 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.032 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.032 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.033 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.033 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.035 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.035 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.035 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.035 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.036 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.036 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.036 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.036 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.037 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.038 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.038 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.042 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.042 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.042 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.042 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.043 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.044 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.044 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.044 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.044 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.045 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.045 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.046 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.046 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found 2025-09-19 22:52:42,377 DEBUG --- stderr --- 2025-09-19 22:52:42,377 DEBUG 2025-09-19 22:52:42,377 INFO ________________________________________________________________________________ [2025-09-19 22:52:42] benchmark-550 show_cmd : Display order command ________________________________________________________________________________ 2025-09-19 22:52:42,378 INFO 2025-09-19 22:52:42,378 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 22:52:42,378 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 22:52:42,660 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 22:52:42,661 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:42,661 DEBUG --- stdout --- 2025-09-19 22:52:42,661 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 22:52:42,661 DEBUG --- stderr --- 2025-09-19 22:52:42,661 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-19 22:52:42,661 INFO 2025-09-19 22:52:42,662 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-550/cmd.txt" ]; then echo "/results/orders/benchmark-550/cmd.txt file found"; exit 0; else echo "/results/orders/benchmark-550/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-550/cmd.txt file found" 2025-09-19 22:52:42,662 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 22:52:43,203 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 22:52:43,203 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:43,203 DEBUG --- stdout --- 2025-09-19 22:52:43,203 DEBUG /results/orders/benchmark-550/cmd.txt file found 2025-09-19 22:52:43,203 DEBUG --- stderr --- 2025-09-19 22:52:43,203 DEBUG 2025-09-19 22:52:43,203 INFO 2025-09-19 22:52:43,203 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-550/cmd.txt | grep "^\d+ .+$" 2025-09-19 22:52:43,203 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-19 22:52:43,742 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 22:52:43,742 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:43,742 DEBUG --- stdout --- 2025-09-19 22:52:43,742 DEBUG 4 /results/orders/benchmark-550/cmd.txt 2025-09-19 22:52:43,742 DEBUG --- stderr --- 2025-09-19 22:52:43,742 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-550/cmd.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/cmd.txt 2025-09-19 22:52:43,742 INFO 2025-09-19 22:52:43,742 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-550/cmd.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/cmd.txt 2025-09-19 22:52:43,743 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-19 22:52:44,306 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:44,306 DEBUG --- stdout --- 2025-09-19 22:52:44,306 DEBUG tar: Removing leading `/' from member names 2025-09-19 22:52:44,306 DEBUG --- stderr --- 2025-09-19 22:52:44,306 DEBUG 2025-09-19 22:52:44,306 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/cmd.txt is small (1 lines), printing all lines: 2025-09-19 22:52:44,306 INFO ----- output ----- /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@perf-haproxy" --results-folder="/results/gatling/benchmark-550" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=M@ythe4th -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=550 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-09-19 22:52:44] benchmark-550 show_rc : Display rc order ________________________________________________________________________________ 2025-09-19 22:52:44,306 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/rc.txt is small (0 lines), printing all lines: 2025-09-19 22:52:44,306 INFO ----- output ----- ________________________________________________________________________________ [2025-09-19 22:52:44] benchmark-550 show_stdout : Display stdout order ________________________________________________________________________________ 2025-09-19 22:52:44,307 DEBUG [print_head_tail]: Print head (100 lines) and tail (100 lines) of input file (/mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/stdout.txt is 2010 lines): 2025-09-19 22:52:44,307 INFO ----- output ----- NOTE: This file was too big so it was truncated. Only first 1000 and last 1000 lines are present. ------------------------------------------------------------ 2025-09-19 22:40:04,314 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmp9djk_4qz 2025-09-19 22:40:04,318 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-550" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=550 -Duser_password=Pa_ssw0rd System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 22:40:24.763 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'authAsPolicyAdmin' failed for user 1: status.find.in([200, 209], 304), found 401 22:40:24.775 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: authAsPolicyAdmin: KO status.find.in([200, 209], 304), found 401 ========================= Session: Session(getToken,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@4ab74239, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@7e46e47d, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, b1262f98-1175-4afb-a909-2d54d838ba2c -> 0),KO,List(ExitOnCompleteLoopBlock(b1262f98-1175-4afb-a909-2d54d838ba2c)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x0000795d4069bb38@53c6ef70,io.netty.channel.epoll.EpollEventLoop@f79a760) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: X-OpenAM-Username: policyAdmin Content-Type: application/json Accept-API-Version: resource=2.0, protocol=1.0 X-OpenAM-Password: Pa_ssw0rd user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 401 Unauthorized headers: x-frame-options: SAMEORIGIN content-security-policy-report-only: frame-ancestors 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline' x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache content-type: application/json Content-Length: 62 date: Fri, 19 Sep 2025 21:40:24 GMT x-forgerock-transactionid: 10bb6834-1d57-4be9-8def-762f3db49e1e strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"code":401,"reason":"Unauthorized","message":"Login failure"} <<<<<<<<<<<<<<<<<<<<<<<<< 22:40:24.777 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-8' crashed with 'j.u.NoSuchElementException: No attribute named 'policyEvalTokenId' is defined', forwarding to the next one 22:40:24.882 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:24.882 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:24.993 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:24.993 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.105 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.105 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.216 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.216 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.327 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.328 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.439 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.440 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.552 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.552 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.564 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.564 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.589 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.589 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.626 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.626 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.659 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.659 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.663 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.664 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.700 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:25.701 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.701 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.738 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.738 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.743 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:25.770 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.771 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.774 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.775 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.812 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.812 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.849 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.849 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.886 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.886 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.889 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:40:25.923 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.923 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:40:25.960 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:40:25.960 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one [...] HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: Content-Type: apppliation/json Accept-API-Version: resource=2.0, protocol=1.0 user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 X-OpenAM-Username: user.1909517 X-OpenAM-Password: Pa_ssw0rd accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 200 OK headers: x-frame-options: SAMEORIGIN content-security-policy-report-only: frame-ancestors 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline' x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache set-cookie: amlbcookie=01; Path=/; Domain=openam-perf-haproxy.forgeblocks.com; Secure; HttpOnly; SameSite=none content-type: application/json Content-Length: 5283 date: Fri, 19 Sep 2025 21:48:07 GMT x-forgerock-transactionid: e47050da-d650-4677-b461-2716402a48d1 strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"authId":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ3aGl0ZWxpc3Qtc3RhdGUiOiJlYTJjZThiOS0yNDFkLTRlODQtOWRlYS1mYWI5OTFiN2ZjOWQiLCJvdGsiOiJsbWQ5bGd1MnAzOW5jNnFqdTVkY3QyaGhwaSIsInJlYWxtIjoiL2FscGhhIiwic2Vzc2lvbklkIjoiKkFBSlRTUUFDTURJQUJIUjVjR1VBQ0VwWFZGOUJWVlJJQUFKVE1RQUNNREUuKmV5SjBlWEFpT2lKS1YxUWlMQ0pqZEhraU9pSktWMVFpTENKaGJHY2lPaUpJVXpJMU5pSjkuWlhsS01HVllRV2xQYVVwTFZqRlJhVXhEU214aWJVMXBUMmxLUWsxVVNUUlJNRXBFVEZWb1ZFMXFWVEpKYVhkcFdWZDRia2xxYjJsYVIyeDVTVzR3TGk1aFZYaFZTVE0wYTJKRE5rZEpVRWxyZEVkUWJtSkJMbFJVUW1OS2EwdG9aMmRTUVMwd2MxcE5PRkI2YmtONGIzSkhlalY2UVhWVldYYzFVakF6U21sTmJreHdWVUpWZGtobExWQlNiM2N0ZFdJMk0ySmxZblZCWDJrNVpHNU1XRmRuUzNadmFVbE9lVlZuTlVFMlpGQkRZbXcyYTJ0ck1sZHpTWFpxUTJSbGFqRmFVMmhYYkRSM05DMXNSRmM1ZEZSeGJqUldSM0ZmU0hoM2FHTnZlR2hhTmpkT1FtWTJRMHh2YkRReVkxTkdOVmd5VEV4aWNtY3phbWgzY2xwTWRXdDFVV1pYVkVkZmVUUnNYMG93U0VSVWNFaE9OMWRmYWt4Rllrc3pTRE14WjJ0S1prUXdNMnhwU1d0dVNFcHhhV3BTYVY5NFNqQTJhV3M1WkVkV0xVWjFNa1k1T1hKWlRERlhXRm96UVVJdFVVbHFVM0J3VlZWTWFXcHRSamhmYjA5NlZVZFZaRmRzYUdGeFdUaGlaamMwUmxGelFXaFVlRTFuYkVFek5uUjJjR3RYYzFKVmFWVXRNVUZQZWpkU1ZFeERaMnhzZDFoR1NFbEllVmRMTWxNeGJFdFdaaTAxYjBoSWRFVktMVE14Y0c1V2VVOXpURVJxYTJGcVVuZ3dka3gyU0UxeFJWcE9jakZUUW05b2FVSkZia3d3VEZGeU9FSlNabWxzZGw4MFFqZEpXRTFOYUVad2RFMU5XVlI0V1cxRWRYTlFXVVJYYWxvd1Z6bExja3ByVTNkbmFIcHdUV1ZEU2t4dWMzQXpNekpoWDNwUFZGRmlZblY1TW5aZk5WbDFhVVozTlZRd1VXUmlPVGhMVjFBMmQwdHZTMjVUWlhGbFZuWlNObW8zY0ZOS2NrbEJaVlJVWW1sQk56SXlaMnRWVVdkMWVXUk5YekJTUkZkdk5VazBkbmxsVjFwZlNFbHhWRVkwTVc5alVIVXRNQzFQU21JdFFqVndhR0ZuZG10TGJYSlNXbFpoZFdGWU9FdEVkRGsxVWtac1JVcFRla3h2WVMwNFNUZzFkMUpRUVRsa00yNTFUMEphZDBSRlVtUmhTbm95YldsTGJUQnpNWEl3YVY5UVJuaHJWRzVTTTI0NVFXcHRZMFZyYkVOVE9EVmxNa0pTVEdsSGNHc3RjRjlKZFc5b09XSm9ORWQzVGxkNE0xQTRiSGxSTFRkUU9HWkhiVUl5V2tod05ISnVlWHBaV1MxUFZFUnhjbGhTYWxNNE4zWTBWME5TY0d0WlR6UlZUVFIwY21wcGJua3lRMGxWVlc1dlJWUjVYMWRyTlcxNmMwUjJYMWxPZDBkWmQwZFNRams1ZUdsalNWaHpTV3REWTJReVRuVlJWWGhmTWpacUxUQnhTV3R1UWpOdVpHUnNNVE42TFY5Nk9IWnBhazFxUWpSNVZXcExka0ZMYWsxeU5GbG9jMXBTWkVoVE5GcGlhRGRZVkdNeVVsUk1RVVpZV1V4amJGVkRaVEZFWm1Sak1FdDRhMmhWT1dOVlRsRTBZVEY2V0hKaVVYWmxjSFEwVGw5amNYUnFNbDloWlMxdE5sRkZZa0ZqWW5OQ1N6ZG9NMVIwTlVGek0wazBhWGh3Y0ROeE5rVlFhemhpVnpGbFdUUXRTR2sxYUVWVVVubFZhWEJyUjFOTE0yVlNSVFZQYmpBeWFEVkljV1p2WjFCS2FsSklhR2hVYTFoeFRXSkNTbmh4YzBaa1JWTlZPVE5pYm1aT05WVlBSSE13YzB4TlRqWTFNakZEZWxvd1FWZG9OekpQV25sRmRFaEhWa2RrY25oSGQxVktlbEE1ZUVwNlNuWk9jRGt3VGxONVpIVXpjbmgyT0hZeGRtdFJORFpuV0hnMGNEaFpWMXBDTVdkUFpXVklTQzAxU0RJeFZrbFdObTVUTmxkSGQwSmFTVXQwU0Mxek9XOWxYMU4zT0ZReVlUQlZVRnBJU1hWSFpuQnBXVkpNVFdWaWFqVTRZamhTZURKRFZtZ3pWREZ1T1RaVmRHbFNXVlF3U21sUFZVdGthbE4xUmtobFgySTBRazlMWDNwMmJqTjFaa2REYUdKSlRDMWtlREJmZEhOWFJGUnFlazFEWDFSQ2FsTm1SRkpIZERreVNFdFVjM3AyZUhweFltNXhRWEZzTjJkclpESjBlbkJxWTFoaWRXdHlUMVY2YVUxbk1uVk9ValJrYkVOQlIwMW5XRUZwTTJKMVIweFJWVFJ2Wmw4M1dub3hOM1ZMU1VOTU4yTXpNMUpaWTNSaGRVWm9UMjV3U1VSMVIzcGZibU5UYW5wTVEyNTVaV3hhYzNvMFJIRk1SbTV4VjJWdlJuWXpWV1ZVUW1JMmNqa3djREZvTldrd2NFMWFaMDF5YVhabVoxTllZamMxT0d0U1FUQnlha3RTU2pWbFFVd3RSa3hyVkRCeFNXeGFTMDlSZDIxTFptTnpkMFE1ZDFoNmFsTkpkWHBCY0VwU1Z6VjBSMEV6TXpaeFdteFFZMFkyYWxwMlYyMTVMVGRaZG5oalV6STRiMHhCUWpOMWJrMUNUbU15VW5oNFgwa3dSMmhPU2xGQmRYQm9RMGt4ZDNsaFMyOVZUazl5UmswMlQxUklibWxaWm1OMU9HOXFiMTlRUTNKWGVEWk1Uazh6VFU5VFN6bEVNMkl0V1ZrMU1scEVORkZ5U1ZaRFEwMTBjRXB5VUhWeGRXNHRXV0ZoTmt4d00yd3piamRhYjNkTFlXaG1SMWxQTUV4aVNUQmhaSEZPY25OMk1qZzRka3Q0Ukd0bVQyWTRiamcxTm5VeE1tOWtNREpyUkZvMFlVcEVPRFJOWlVScmFYcEhVbU5YTld0d05FSXdjVFkxZVd3dE9VcE1hRWh1ZG5VdGJsTlJjVkYyTnpselpHOTBUalJyUWw5NFlsSlVWM2w2YlRNNFpVVm1WRkJMU1c1NGVsRXRkWFZFVGxaTllYZEVSeTAyVURSV2RtcDBZemhHUlVoNU5GbEJaMjFvY0MxdlMwVm5MVEZhUTJoelkyVkVNa05LTjFkTWVtUXlMVFJ1V1RadVEwZHFUV3RIVUdNdGMyOXpabUV5TjJGZmNXZEhiMkZVUWt4SE4yTnhVMU5NUm5sQlFrMDJhamQwUlRkR01HRndNMjlEYmxwNmQzSmxiVk01UTFadk1VZDJka3BrVlcxUk0zUmlVV0l4YWtGSWJsOVlRbHByU1Y5clNqSkRVa0ZEU0ZCS0xWRjFUMGRSYVdWWU9XOTBia0l5YVRWR1h6RXpRa1ZqUTBKak1qbHpiRkJKV2kxVGRuWkZTREU1WW1zNVFrUnJXbkpJU0RsT2QwOWZhRnBTUm5ONE5FNDVVbE5FYmkxVlRtNUdNbTlXWVU0dFRURmFRM28xVjE5R01rNUZVMVJrVjJReVNtdGlla2RZTkVsUVdYSlBhMFpqWTNKMFEzaEdXbTU1U0ZKTGR5MHlRbE5PVjNGTVVUZHZUVFpHT0ROc1JERmtMVXRmYWxnM1pHVjJhR3RoUkc1eVMxbEZNa28yU1Vzd05XVmFhREZqY2twbE1tUmhWblF5UVdoWmFVTk5URUl4ZG5WWlRqTmpkSFZYYnpjd1VrWnZibWRFV1ZkdVJIWmFWVzlxZUdSc1NVMW5NV3hGY3paR2VEWkdNR1p1VVZKS1UxQmlkVEpMTjJrM2VGWk5keTVmV1RjeGNtNUNUell5ZDFOUVZtNXFhSFJrV0VGQi5IcUNMVkVnM1hjeWZ3dVRhMXJ6dm1PRzQ5TWNaSnV5Slp1LUx0eExQaEwwIiwiZXhwIjoxNzU4MzE4Nzg3LCJpYXQiOjE3NTgzMTg0ODd9.GTpfX2R-9yuBUGJ2LQ_DhnEecuC0g1zam_6cYZuHrUA","callbacks":[{"type":"BooleanAttributeInputCallback","output":[{"name":"name","value":"preferences/updates"},{"name":"prompt","value":"Send me news and updates"},{"name":"required","value":false},{"name":"policies","value":{}},{"name":"failedPolicies","value":[]},{"name":"validateOnly","value":false},{"name":"value","value":false}],"input":[{"name":"IDToken1","value":false},{"name":"IDToken1validateOnly","value":false}],"_id":2},{"type":"BooleanAttributeInputCallback","output":[{"name":"name","value":"preferences/marketing"},{"name":"prompt","value":"Send me special offers and services"},{"name":"required","value":false},{"name":"policies","value":{}},{"name":"failedPolicies","value":[]},{"name":"validateOnly","value":false},{"name":"value","value":false}],"input":[{"name":"IDToken2","value":false},{"name":"IDToken2validateOnly","value":false}],"_id":3}],"header":"Please select your preferences"} <<<<<<<<<<<<<<<<<<<<<<<<< 22:48:08.005 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.005 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.005 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.005 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.005 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.009 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.010 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.010 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.011 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.011 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.014 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.017 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.019 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.019 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.020 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.020 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.023 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.023 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.023 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.023 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.028 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.028 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.028 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.028 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.028 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.029 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.029 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.029 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.029 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.030 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.030 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.030 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.031 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.031 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.032 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.032 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.033 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.033 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.035 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.035 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.035 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.035 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.036 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.036 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.036 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.036 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.037 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.038 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.038 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.042 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.042 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.042 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.042 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.043 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.044 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.044 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.044 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.044 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.045 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 22:48:08.045 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 22:48:08.046 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 22:48:08.046 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found ________________________________________________________________________________ [2025-09-19 22:52:44] benchmark-550 show_stderr : Display stderr order ________________________________________________________________________________ 2025-09-19 22:52:44,308 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/07_benchmark-550/stderr.txt is small (0 lines), printing all lines: 2025-09-19 22:52:44,308 INFO ----- output ----- ________________________________________________________________________________ [2025-09-19 22:52:44] benchmark-550 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2025-09-19 22:52:44] benchmark-550 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Logs for pod: {pod} 2025-09-19 22:52:44,542 INFO 2025-09-19 22:52:44,542 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm logs overseer-0-57d79d468-khs5t 2025-09-19 22:52:44,543 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 22:52:45,220 INFO [loop_until]: OK (rc = 0) 2025-09-19 22:52:45,220 DEBUG --- stdout --- 2025-09-19 22:52:45,221 DEBUG [print_head_tail]: Print head (100 lines) and tail (100 lines) of input string (5394 lines): 2025-09-19 22:52:45,221 DEBUG ----- output ----- [2025-09-11 02:04:59] httpd web_server : Server is running at http://localhost:8080 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2025-09-11 02:04:59] overseer: Waiting for order /results/orders/order.json ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 10.106.40.147 - - [11/Sep/2025 02:05:04] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [11/Sep/2025 02:05:07] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [11/Sep/2025 05:18:42] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [11/Sep/2025 11:40:49] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [11/Sep/2025 17:13:25] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [12/Sep/2025 04:15:45] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [12/Sep/2025 07:35:02] code 404, message File not found 10.106.40.147 - - [12/Sep/2025 07:35:02] "GET /robots.txt HTTP/1.1" 404 - 10.106.40.147 - - [12/Sep/2025 07:35:03] code 404, message File not found 10.106.40.147 - - [12/Sep/2025 07:35:03] "GET /gatling/benchmark-200/ HTTP/1.1" 404 - 10.106.40.147 - - [12/Sep/2025 07:35:07] code 404, message File not found 10.106.40.147 - - [12/Sep/2025 07:35:07] "GET /gatling/benchmark-100/loadtest-20250613220330160/ HTTP/1.1" 404 - 10.106.40.147 - - [12/Sep/2025 09:38:19] code 404, message File not found 10.106.40.147 - - [12/Sep/2025 09:38:19] "GET /gatling/benchmark-100/ HTTP/1.1" 404 - 10.106.40.147 - - [12/Sep/2025 10:26:16] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [12/Sep/2025 16:02:13] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [12/Sep/2025 23:44:00] code 404, message File not found 10.106.40.147 - - [12/Sep/2025 23:44:00] "GET /gatling/ HTTP/1.1" 404 - 10.106.40.147 - - [12/Sep/2025 23:44:04] code 404, message File not found 10.106.40.147 - - [12/Sep/2025 23:44:04] "GET /gatling/benchmark-300/loadtest-20250613062913800/ HTTP/1.1" 404 - 10.106.40.147 - - [13/Sep/2025 04:39:59] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [13/Sep/2025 04:54:29] "GET /lost%2Bfound/ HTTP/1.1" 200 - 10.106.40.147 - - [13/Sep/2025 05:05:32] "GET /orders/ HTTP/1.1" 200 - 10.106.40.147 - - [13/Sep/2025 05:19:49] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [13/Sep/2025 10:44:33] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [13/Sep/2025 13:44:37] code 404, message File not found 10.106.40.147 - - [13/Sep/2025 13:44:37] "GET /robots.txt HTTP/1.1" 404 - 10.106.40.147 - - [13/Sep/2025 13:44:37] code 404, message File not found 10.106.40.147 - - [13/Sep/2025 13:44:37] "GET /gatling/benchmark-300/loadtest-20250611234042501/ HTTP/1.1" 404 - 10.106.40.147 - - [13/Sep/2025 16:01:43] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [14/Sep/2025 03:50:45] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [14/Sep/2025 03:53:57] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 03:53:57] "GET /gatling/benchmark-100/loadtest-20250613042735144/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 07:36:57] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 07:36:57] "GET /gatling/benchmark-500/loadtest-20250612014222105/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 07:37:01] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 07:37:01] "GET /gatling/benchmark-500/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 08:43:13] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 08:43:13] "GET /gatling/benchmark-300/loadtest-20250614000508954/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 08:43:17] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 08:43:17] "GET /gatling/benchmark-300/loadtest-20250611032033940/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 08:43:21] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 08:43:21] "GET /gatling/benchmark-10/loadtest-20250611001805609/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 09:59:35] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [14/Sep/2025 10:34:06] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 10:34:06] "GET /gatling/benchmark-500/loadtest-20250614020648742/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 10:34:09] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 10:34:09] "GET /gatling/benchmark-300/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 11:40:33] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 11:40:33] "GET /gatling/benchmark-400/loadtest-20250614010557750/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 15:14:45] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [14/Sep/2025 18:48:17] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 18:48:17] "GET /robots.txt HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 18:48:17] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 18:48:17] "GET /gatling/benchmark-500/loadtest-20250613083055790/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 19:46:06] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 19:46:06] "GET /gatling/benchmark-400/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 19:46:10] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 19:46:10] "GET /gatling/benchmark-600/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 19:46:13] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 19:46:13] "GET /gatling/benchmark-550/loadtest-20250610000802442/ HTTP/1.1" 404 - 10.106.40.147 - - [14/Sep/2025 22:35:26] code 404, message File not found 10.106.40.147 - - [14/Sep/2025 22:35:26] "GET /gatling/benchmark-550/loadtest-20250614030739048/ HTTP/1.1" 404 - 10.106.40.147 - - [15/Sep/2025 00:36:21] code 404, message File not found 10.106.40.147 - - [15/Sep/2025 00:36:21] "GET /gatling/preload-with-gatling/ HTTP/1.1" 404 - 10.106.40.147 - - [15/Sep/2025 00:36:25] code 404, message File not found 10.106.40.147 - - [15/Sep/2025 00:36:25] "GET /gatling/benchmark-100/loadtest-20250611213902958/ HTTP/1.1" 404 - 10.106.40.147 - - [15/Sep/2025 02:48:21] code 404, message File not found 10.106.40.147 - - [15/Sep/2025 02:48:21] "GET /gatling/benchmark-400/loadtest-20250613073004669/ HTTP/1.1" 404 - 10.106.40.147 - - [15/Sep/2025 02:48:25] code 404, message File not found 10.106.40.147 - - [15/Sep/2025 02:48:25] "GET /gatling/benchmark-300/loadtest-20250609210532612/ HTTP/1.1" 404 - 10.106.40.147 - - [15/Sep/2025 04:42:06] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [15/Sep/2025 04:43:49] code 404, message File not found 10.106.40.147 - - [15/Sep/2025 04:43:49] "GET /gatling/cleanup/loadtest-20250515231745014/ HTTP/1.1" 404 - 10.106.40.147 - - [15/Sep/2025 12:09:38] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [15/Sep/2025 17:40:23] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [15/Sep/2025 18:15:43] "GET / HTTP/1.1" 200 - 10.106.40.147 - - [15/Sep/2025 18:18:05] "GET /orders/status.txt HTTP/1.1" 200 - [2025-09-15 18:18:07] overseer: Start new order benchmark-5 (id 1) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2025-09-15 18:18:07] order benchmark-5: Start command ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2025-09-15 18:18:07] order benchmark-5: Running command : /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.CH2023@perf-haproxy" --results-folder="/results/gatling/benchmark-5" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=300 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Dprefix=self5- -Drealm=alpha -DtestCase=idc.benchmark.CH2023 -DtestMode=open -Dthroughput=5 -Duser_password=Pa_ssw0rd" [2025-09-15 18:18:07] order benchmark-5: Http access : http://localhost:8080/orders/benchmark-5 [2025-09-15 18:18:07] order benchmark-5: Full command stored in : /results/orders/benchmark-5/cmd.txt [2025-09-15 18:18:07] order benchmark-5: RC file : /results/orders/benchmark-5/rc.txt [2025-09-15 18:18:07] order benchmark-5: Stdout file : /results/orders/benchmark-5/stdout.txt [2025-09-15 18:18:07] order benchmark-5: Stderr file : /results/orders/benchmark-5/stderr.txt [2025-09-15 18:18:07] order benchmark-5: Process 90 has been successfully launched 10.106.40.147 - - [15/Sep/2025 18:18:35] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [15/Sep/2025 18:18:40] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [15/Sep/2025 18:18:45] "GET /orders/status.txt HTTP/1.1" 200 - [...] 10.106.40.147 - - [19/Sep/2025 21:55:44] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 21:57:44] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 21:59:45] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:01:45] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:03:45] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:05:45] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:07:46] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:09:46] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:11:46] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:13:46] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:15:47] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:17:47] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:19:47] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:21:47] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:23:48] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:25:48] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:27:48] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:29:49] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:31:49] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:33:49] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:35:49] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:37:50] "GET /orders/status.txt HTTP/1.1" 200 - [2025-09-19 22:39:42] order benchmark-500: CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) [2025-09-19 22:39:42] overseer: order stopped by itself benchmark-500 : /results/orders/benchmark-500 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2025-09-19 22:39:43] overseer: Waiting for order /results/orders/order.json ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2025-09-19 22:39:43] overseer: Http access : http://localhost:8080 10.106.40.147 - - [19/Sep/2025 22:39:50] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:40:00] "GET /orders/benchmark-500/rc.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:40:00] "GET /orders/status.txt HTTP/1.1" 200 - [2025-09-19 22:40:02] order benchmark-550: pod has restarted during the run or same order has been run several times, backup current /results/orders/benchmark-550 to /results/orders/benchmark-550_20250919_224002 [2025-09-19 22:40:02] overseer: Start new order benchmark-550 (id 106) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2025-09-19 22:40:02] order benchmark-550: Start command ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2025-09-19 22:40:02] order benchmark-550: Running command : /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@perf-haproxy" --results-folder="/results/gatling/benchmark-550" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=550 -Duser_password=Pa_ssw0rd" [2025-09-19 22:40:02] order benchmark-550: Http access : http://localhost:8080/orders/benchmark-550 [2025-09-19 22:40:02] order benchmark-550: Full command stored in : /results/orders/benchmark-550/cmd.txt [2025-09-19 22:40:02] order benchmark-550: RC file : /results/orders/benchmark-550/rc.txt [2025-09-19 22:40:02] order benchmark-550: Stdout file : /results/orders/benchmark-550/stdout.txt [2025-09-19 22:40:02] order benchmark-550: Stderr file : /results/orders/benchmark-550/stderr.txt [2025-09-19 22:40:02] order benchmark-550: Process 26481 has been successfully launched 10.106.40.147 - - [19/Sep/2025 22:42:24] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:44:25] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:46:25] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:48:25] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:50:26] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:52:26] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:54:26] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:56:26] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 22:58:27] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:00:27] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:02:27] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:04:27] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:06:28] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:08:28] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:10:28] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:12:29] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:14:29] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:16:29] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:18:29] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:20:30] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:22:30] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:24:30] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:26:31] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:28:31] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:30:31] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:32:31] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:34:32] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:36:32] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:38:32] "GET /orders/status.txt HTTP/1.1" 200 - [2025-09-19 23:40:30] order benchmark-550: CMD FAILURE: command returns code 1 (expected [0, -9, 143, 137, -15]) OSError: [Errno 28] No space left on device During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/lodestar/pyrock/shared/scripts/overseer/run.py", line 190, in overseer.run() File "/lodestar/pyrock/shared/scripts/overseer/run.py", line 118, in run if self.order.is_running(): ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/jenkins/lodestar/pyrock/shared/scripts/overseer/Order.py", line 146, in is_running self.write_rc_file() File "/home/jenkins/lodestar/pyrock/shared/scripts/overseer/Order.py", line 207, in write_rc_file with open(self.rc_file, "w") as fd: OSError: [Errno 28] No space left on device 10.106.40.147 - - [19/Sep/2025 23:40:32] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:42:33] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:44:33] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:46:33] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:48:34] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:50:34] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:52:34] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.40.147 - - [19/Sep/2025 23:52:44] "GET /orders/benchmark-550/rc.txt HTTP/1.1" 200 - 2025-09-19 22:52:45,221 DEBUG --- stderr --- 2025-09-19 22:52:45,221 DEBUG ________________________________________________________________________________ [2025-09-19 22:52:45] benchmark-550 post : Post method ________________________________________________________________________________ Task has been successfully stopped