--Task-- name: benchmark-300 enabled: True class_name: GatlingTask source_name: overseer-0 source_namespace: >default< target_name: am target_namespace: >default< start: 0.0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates below loop: False interval: None dependencies: [] wait_for: ['benchmark-200'] preceding_task: benchmark-200 options: {'duration': 3600, 'java-opts': '-Drealm=alpha -Duser_password=Pa_ssw0rd -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html', 'testCase': 'idc.benchmark.Mix2024', 'testMode': 'open', 'throughput': 300} group_name: None Current dir: /mnt/disk1/xslou/workshop/lodestar-fork/pyrock 2025-09-19 18:37:41,312 INFO 2025-09-19 18:37:41,312 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 18:37:41,312 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 18:37:41,600 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 18:37:41,600 INFO [loop_until]: OK (rc = 0) 2025-09-19 18:37:41,600 DEBUG --- stdout --- 2025-09-19 18:37:41,600 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 18:37:41,600 DEBUG --- stderr --- 2025-09-19 18:37:41,600 DEBUG 2025-09-19 18:37:41,600 INFO Lodestar assumed that full stack is deployed so will set all products host URLs in Gatling system properties 2025-09-19 18:37:41 - INFO: Timeout for this Task has changed to 1h 12m (4320 seconds) 2025-09-19 18:37:41 - INFO: timeout was calculated based on simulation duration parameter 2025-09-19 18:37:41 - INFO: Interval for this Task has changed to 2m (120 seconds) 2025-09-19 18:37:41 - INFO: interval was set to Task default because it was unset, based on self.timeout value because task is allowed to stop by itself Pod log file : /results/orders/benchmark-300/stdout.txt Pod log : /results/orders/benchmark-300 ________________________________________________________________________________ [2025-09-19 18:37:41] benchmark-300 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@perf-haproxy" --results-folder="/results/gatling/benchmark-300" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-09-19 18:37:41] benchmark-300 create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file: /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/order.json { "name": "benchmark-300", "command": " /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class=\"com.forgerock.pyrock.Loadtest\" --run-description=\"idc.benchmark.Mix2024@perf-haproxy\" --results-folder=\"/results/gatling/benchmark-300\" --no-reports --simulation-jvm-args=\"-XX:+ExitOnOutOfMemoryError -Xmx4G\" --simulation-system-properties=\"-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=Pa_ssw0rd\"", "out-dir": "/results/orders/benchmark-300" } ________________________________________________________________________________ [2025-09-19 18:37:41] benchmark-300 loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2025-09-19 18:37:41,608 INFO 2025-09-19 18:37:41,608 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2025-09-19 18:37:41,608 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-09-19 18:37:41,816 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 18:37:41,816 INFO [loop_until]: OK (rc = 0) 2025-09-19 18:37:41,816 DEBUG --- stdout --- 2025-09-19 18:37:41,816 DEBUG WAITING 2025-09-19 18:37:41,816 DEBUG --- stderr --- 2025-09-19 18:37:41,816 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 35 0 --:--:-- --:--:-- --:--:-- 35 ________________________________________________________________________________ [2025-09-19 18:37:41] benchmark-300 start_order : Send the order to the overseer pod ________________________________________________________________________________ 2025-09-19 18:37:41,817 INFO 2025-09-19 18:37:41,817 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 18:37:41,817 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 18:37:42,100 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 18:37:42,100 INFO [loop_until]: OK (rc = 0) 2025-09-19 18:37:42,100 DEBUG --- stdout --- 2025-09-19 18:37:42,100 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 18:37:42,100 DEBUG --- stderr --- 2025-09-19 18:37:42,100 DEBUG 2025-09-19 18:37:42,101 INFO 2025-09-19 18:37:42,101 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/order.json overseer-0-57d79d468-khs5t:/results/orders/order.json 2025-09-19 18:37:42,101 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 18:37:42,997 INFO [loop_until]: OK (rc = 0) 2025-09-19 18:37:42,997 DEBUG --- stdout --- 2025-09-19 18:37:42,997 DEBUG 2025-09-19 18:37:42,997 DEBUG --- stderr --- 2025-09-19 18:37:42,997 DEBUG Order has been successfully forwarded to overseer 2025-09-19 18:37:42 - INFO: Check if Gatling simulation started successfully 2025-09-19 18:37:42,998 INFO 2025-09-19 18:37:42,998 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- head --lines 5000 /results/orders/benchmark-300/stdout.txt | grep "Simulation com.forgerock.pyrock.Loadtest started..." 2025-09-19 18:37:42,998 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-09-19 18:37:43,551 INFO [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-19 18:37:49,112 INFO [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-19 18:37:54,673 INFO [loop_until]: Function succeeded after 11s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-19 18:38:00,228 INFO [loop_until]: Function succeeded after 17s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-19 18:38:05,781 INFO [loop_until]: Function succeeded after 22s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-19 18:38:11,463 INFO [loop_until]: Function succeeded after 28s (rc=0) - expected pattern found 2025-09-19 18:38:11,463 INFO [loop_until]: OK (rc = 0) 2025-09-19 18:38:11,463 DEBUG --- stdout --- 2025-09-19 18:38:11,463 DEBUG [print_head_tail]: Print head (100 lines) and tail (100 lines) of input string (446 lines): 2025-09-19 18:38:11,463 DEBUG ----- output ----- 2025-09-19 19:37:45,445 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmpy3frtfst 2025-09-19 19:37:45,449 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-300" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=Pa_ssw0rd Gatling 3.13.5 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 19:38:06.018 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'authAsPolicyAdmin' failed for user 1: status.find.in([200, 209], 304), found 401 19:38:06.030 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: authAsPolicyAdmin: KO status.find.in([200, 209], 304), found 401 ========================= Session: Session(getToken,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@44d81079, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@5fc81320, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, 1a73cf7f-46ee-49f4-b553-cf4f1d882f1c -> 0),KO,List(ExitOnCompleteLoopBlock(1a73cf7f-46ee-49f4-b553-cf4f1d882f1c)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@32d6f61f,io.netty.channel.epoll.EpollEventLoop@f79a760) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: X-OpenAM-Username: policyAdmin Content-Type: application/json Accept-API-Version: resource=2.0, protocol=1.0 X-OpenAM-Password: Pa_ssw0rd user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 401 Unauthorized headers: x-frame-options: SAMEORIGIN content-security-policy-report-only: frame-ancestors 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline' x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache content-type: application/json Content-Length: 62 date: Fri, 19 Sep 2025 18:38:05 GMT x-forgerock-transactionid: b52ae9f1-b7ae-4a7b-b65b-701bf590dbb1 strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"code":401,"reason":"Unauthorized","message":"Login failure"} <<<<<<<<<<<<<<<<<<<<<<<<< 19:38:06.033 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-8' crashed with 'j.u.NoSuchElementException: No attribute named 'policyEvalTokenId' is defined', forwarding to the next one 19:38:06.219 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.220 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.419 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.420 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.620 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.620 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.836 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.836 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.887 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.887 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.930 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:06.953 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.954 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.018 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.019 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.021 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.021 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.087 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.088 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.126 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.131 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.155 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.155 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.207 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.221 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.222 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.288 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.289 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.355 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.356 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.422 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.423 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.488 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.489 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.528 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.555 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.555 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.622 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.622 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.688 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.688 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.755 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.756 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.822 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.822 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.856 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one [...] > UI AM validateGoto (OK=35 KO=0 ) > User Authorize PKCE stage (OK=64 KO=0 ) > User AccessToken PKCE stage (OK=62 KO=0 ) > UI-Post-Login - stage 2 (OK=31 KO=0 ) > UI-Post-Login - stage 2 Redirect 1 (OK=31 KO=0 ) > UI Enduser appAuthHelper1 (OK=31 KO=0 ) > UI Enduser appAuthHelper2 (OK=31 KO=0 ) > userRestlogin (OK=32 KO=0 ) > UI-Post-Login - stage 3 (OK=29 KO=0 ) > UI-Post-Login - stage 3 Redirect 1 (OK=29 KO=0 ) > UI IDM uiconfig (OK=28 KO=0 ) > UI Enduser sessioncheck (OK=28 KO=0 ) > UI IDM version (OK=28 KO=0 ) > UI IDM features (OK=28 KO=0 ) > UI IDM login (OK=28 KO=0 ) > UI IDM privilege (OK=28 KO=0 ) > UI IDM managed user schema (OK=28 KO=0 ) > UI IDM dashboard (OK=28 KO=0 ) > UI IDM user (OK=27 KO=0 ) > UI IDM oidcToken (OK=26 KO=0 ) > userinfo (OK=26 KO=0 ) > UI-Logout (OK=25 KO=0 ) > UI-Logout Redirect 1 (OK=25 KO=0 ) > UI AM revoke2 (OK=25 KO=0 ) > UI AM revoke1 (OK=25 KO=0 ) > UI AM endsession (OK=25 KO=0 ) > loginTreeWorkflow_restLoginInitiate (OK=4 KO=0 ) > loginTreeWorkflow_restLoginUsernamePasswordCallback (OK=4 KO=0 ) ---- Errors -------------------------------------------------------------------- > EvalPolicy: No attribute named 'tokenIdCookieName' is defined 32 (96.97%) > status.find.in([200, 209], 304), found 401 1 ( 3.03%) ---- getToken ------------------------------------------------------------------ [--------------------------------------------------------------------------] 0% waiting: 0 / active: 1 / done: 0 ---- idc.benchmark.Mix2024 ----------------------------------------------------- [- ] 0% waiting: 1084811 / active: 13 / done: 112 ================================================================================ 19:38:10.815 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.816 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.834 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.834 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.852 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.852 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.870 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.870 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.888 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.888 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.906 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.906 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.925 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.925 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.935 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:10.943 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.943 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.961 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.961 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.979 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.979 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:10.997 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:10.997 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.016 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.016 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.026 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:11.034 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.034 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.047 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:11.052 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.052 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.070 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.070 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.088 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.088 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.090 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:11.106 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.106 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.125 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.125 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.143 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.143 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.157 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:11.161 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.161 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.172 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:11.179 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.179 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.197 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.197 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.216 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.216 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.234 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.234 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.252 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.252 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.270 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.270 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:11.288 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:11.288 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 2025-09-19 18:38:11,463 DEBUG --- stderr --- 2025-09-19 18:38:11,463 DEBUG ________________________________________________________________________________ [2025-09-19 18:38:11] benchmark-300 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 1h 12m. Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 2m, retry Order is running since 4m, retry Order is running since 6m, retry Order is running since 8m 01s, retry Order is running since 10m 01s, retry Order is running since 12m 01s, retry Order is running since 14m 01s, retry Order is running since 16m 02s, retry Order is running since 18m 02s, retry Order is running since 20m 02s, retry Order is running since 22m 02s, retry Order is running since 24m 03s, retry Order is running since 26m 03s, retry Order is running since 28m 03s, retry Order is running since 30m 04s, retry Order is running since 32m 04s, retry Order is running since 34m 04s, retry Order is running since 36m 04s, retry Order is running since 38m 05s, retry Order is running since 40m 05s, retry Order is running since 42m 05s, retry Order is running since 44m 05s, retry Order is running since 46m 06s, retry Order is running since 48m 06s, retry Order is running since 50m 06s, retry Order is running since 52m 06s, retry Order is running since 54m 07s, retry Order is running since 56m 07s, retry Order is running since 58m 07s, retry PASS : Order stopped by itself after 1h and before timeout (1h 12m) ________________________________________________________________________________ [2025-09-19 19:38:19] benchmark-300 stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2025-09-19 19:38:19,479 INFO 2025-09-19 19:38:19,479 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 19:38:19,479 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 19:38:19,828 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 19:38:19,828 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:19,828 DEBUG --- stdout --- 2025-09-19 19:38:19,828 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 19:38:19,828 DEBUG --- stderr --- 2025-09-19 19:38:19,828 DEBUG 2025-09-19 19:38:19,829 INFO 2025-09-19 19:38:19,829 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- rm -f /results/orders/order.json 2025-09-19 19:38:19,829 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 19:38:20,431 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:20,431 DEBUG --- stdout --- 2025-09-19 19:38:20,431 DEBUG 2025-09-19 19:38:20,431 DEBUG --- stderr --- 2025-09-19 19:38:20,431 DEBUG Order ran during 1h ________________________________________________________________________________ [2025-09-19 19:38:20] benchmark-300 download_logs : Download individual order files ________________________________________________________________________________ 2025-09-19 19:38:20 - PASS: file already downloaded /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/order.json 2025-09-19 19:38:20,431 INFO 2025-09-19 19:38:20,431 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 19:38:20,431 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 19:38:20,716 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 19:38:20,716 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:20,716 DEBUG --- stdout --- 2025-09-19 19:38:20,716 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 19:38:20,716 DEBUG --- stderr --- 2025-09-19 19:38:20,716 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-19 19:38:20,717 INFO 2025-09-19 19:38:20,717 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-300/rc.txt" ]; then echo "/results/orders/benchmark-300/rc.txt file found"; exit 0; else echo "/results/orders/benchmark-300/rc.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-300/rc.txt file found" 2025-09-19 19:38:20,717 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 19:38:21,271 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 19:38:21,271 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:21,271 DEBUG --- stdout --- 2025-09-19 19:38:21,271 DEBUG /results/orders/benchmark-300/rc.txt file found 2025-09-19 19:38:21,271 DEBUG --- stderr --- 2025-09-19 19:38:21,271 DEBUG 2025-09-19 19:38:21,271 INFO 2025-09-19 19:38:21,271 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-300/rc.txt | grep "^\d+ .+$" 2025-09-19 19:38:21,271 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-19 19:38:21,814 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 19:38:21,814 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:21,814 DEBUG --- stdout --- 2025-09-19 19:38:21,814 DEBUG 4 /results/orders/benchmark-300/rc.txt 2025-09-19 19:38:21,814 DEBUG --- stderr --- 2025-09-19 19:38:21,814 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-300/rc.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/rc.txt 2025-09-19 19:38:21,815 INFO 2025-09-19 19:38:21,815 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-300/rc.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/rc.txt 2025-09-19 19:38:21,815 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-19 19:38:22,504 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:22,505 DEBUG --- stdout --- 2025-09-19 19:38:22,505 DEBUG 2025-09-19 19:38:22,505 DEBUG --- stderr --- 2025-09-19 19:38:22,505 DEBUG 2025-09-19 19:38:22,505 INFO 2025-09-19 19:38:22,505 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 19:38:22,505 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 19:38:22,790 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 19:38:22,790 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:22,790 DEBUG --- stdout --- 2025-09-19 19:38:22,790 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 19:38:22,790 DEBUG --- stderr --- 2025-09-19 19:38:22,790 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-19 19:38:22,791 INFO 2025-09-19 19:38:22,791 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-300/stderr.txt" ]; then echo "/results/orders/benchmark-300/stderr.txt file found"; exit 0; else echo "/results/orders/benchmark-300/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-300/stderr.txt file found" 2025-09-19 19:38:22,792 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 19:38:23,384 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 19:38:23,384 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:23,384 DEBUG --- stdout --- 2025-09-19 19:38:23,384 DEBUG /results/orders/benchmark-300/stderr.txt file found 2025-09-19 19:38:23,384 DEBUG --- stderr --- 2025-09-19 19:38:23,384 DEBUG 2025-09-19 19:38:23,385 INFO 2025-09-19 19:38:23,385 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-300/stderr.txt | grep "^\d+ .+$" 2025-09-19 19:38:23,385 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-19 19:38:23,941 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 19:38:23,942 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:23,942 DEBUG --- stdout --- 2025-09-19 19:38:23,942 DEBUG 0 /results/orders/benchmark-300/stderr.txt 2025-09-19 19:38:23,942 DEBUG --- stderr --- 2025-09-19 19:38:23,942 DEBUG - Download file (size 0B) source : /results/orders/benchmark-300/stderr.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stderr.txt 2025-09-19 19:38:23,942 INFO 2025-09-19 19:38:23,942 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-300/stderr.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stderr.txt 2025-09-19 19:38:23,942 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-19 19:38:24,486 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:24,487 DEBUG --- stdout --- 2025-09-19 19:38:24,487 DEBUG tar: Removing leading `/' from member names 2025-09-19 19:38:24,487 DEBUG --- stderr --- 2025-09-19 19:38:24,487 DEBUG 2025-09-19 19:38:24,487 INFO 2025-09-19 19:38:24,487 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 19:38:24,487 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 19:38:24,792 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 19:38:24,792 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:24,792 DEBUG --- stdout --- 2025-09-19 19:38:24,792 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 19:38:24,792 DEBUG --- stderr --- 2025-09-19 19:38:24,792 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-19 19:38:24,793 INFO 2025-09-19 19:38:24,793 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-300/stdout.txt" ]; then echo "/results/orders/benchmark-300/stdout.txt file found"; exit 0; else echo "/results/orders/benchmark-300/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-300/stdout.txt file found" 2025-09-19 19:38:24,793 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 19:38:25,351 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 19:38:25,351 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:25,351 DEBUG --- stdout --- 2025-09-19 19:38:25,351 DEBUG /results/orders/benchmark-300/stdout.txt file found 2025-09-19 19:38:25,351 DEBUG --- stderr --- 2025-09-19 19:38:25,351 DEBUG 2025-09-19 19:38:25,352 INFO 2025-09-19 19:38:25,352 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-300/stdout.txt | grep "^\d+ .+$" 2025-09-19 19:38:25,352 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-19 19:38:25,893 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 19:38:25,893 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:25,893 DEBUG --- stdout --- 2025-09-19 19:38:25,893 DEBUG 427260 /results/orders/benchmark-300/stdout.txt 2025-09-19 19:38:25,893 DEBUG --- stderr --- 2025-09-19 19:38:25,893 DEBUG - Download file (size 427.26MB exceeds limit of 100MB, only first 1000 and last 1000 lines will be downloaded) source : /results/orders/benchmark-300/stdout.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stdout.txt 2025-09-19 19:38:25,893 INFO 2025-09-19 19:38:25,894 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- head -n 1000 /results/orders/benchmark-300/stdout.txt 2025-09-19 19:38:26,639 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2025-09-19 19:38:26,639 DEBUG --- stdout --- 2025-09-19 19:38:26,639 DEBUG [print_head_tail]: Print head (80 lines) and tail (80 lines) of input string (1000 lines): 2025-09-19 19:38:26,639 DEBUG ----- output ----- 2025-09-19 19:37:45,445 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmpy3frtfst 2025-09-19 19:37:45,449 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-300" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=Pa_ssw0rd Gatling 3.13.5 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 19:38:06.018 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'authAsPolicyAdmin' failed for user 1: status.find.in([200, 209], 304), found 401 19:38:06.030 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: authAsPolicyAdmin: KO status.find.in([200, 209], 304), found 401 ========================= Session: Session(getToken,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@44d81079, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@5fc81320, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, 1a73cf7f-46ee-49f4-b553-cf4f1d882f1c -> 0),KO,List(ExitOnCompleteLoopBlock(1a73cf7f-46ee-49f4-b553-cf4f1d882f1c)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@32d6f61f,io.netty.channel.epoll.EpollEventLoop@f79a760) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: X-OpenAM-Username: policyAdmin Content-Type: application/json Accept-API-Version: resource=2.0, protocol=1.0 X-OpenAM-Password: Pa_ssw0rd user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 401 Unauthorized headers: x-frame-options: SAMEORIGIN content-security-policy-report-only: frame-ancestors 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline' x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache content-type: application/json Content-Length: 62 date: Fri, 19 Sep 2025 18:38:05 GMT x-forgerock-transactionid: b52ae9f1-b7ae-4a7b-b65b-701bf590dbb1 strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"code":401,"reason":"Unauthorized","message":"Login failure"} <<<<<<<<<<<<<<<<<<<<<<<<< 19:38:06.033 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-8' crashed with 'j.u.NoSuchElementException: No attribute named 'policyEvalTokenId' is defined', forwarding to the next one 19:38:06.219 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.220 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.419 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.420 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.620 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.620 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.836 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.836 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.887 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.887 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.930 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:06.953 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.954 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.018 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.019 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.021 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.021 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.087 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.088 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.126 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.131 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.155 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.155 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.207 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.221 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.222 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one [...] 19:38:14.286 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.298 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.298 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.310 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.310 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.321 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.321 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.323 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.333 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.333 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.345 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.345 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.357 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.357 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.363 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.368 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.368 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.380 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.380 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.387 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.392 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.392 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.392 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.404 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.404 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.415 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.416 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.427 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.427 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.439 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.440 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.451 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.452 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.463 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.464 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.472 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.474 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.475 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.484 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.486 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.486 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.498 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.499 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.510 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.511 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.521 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.522 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.527 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.532 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.533 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.534 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.546 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.546 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.558 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.558 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.567 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.569 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.569 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.580 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.581 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.583 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.592 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.593 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.605 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.605 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.617 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.617 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.621 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.628 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.628 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.637 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.640 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.640 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.650 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.652 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.652 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.659 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:14.664 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:14.664 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:14.675 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 2025-09-19 19:38:26,639 DEBUG --- stderr --- 2025-09-19 19:38:26,639 DEBUG 2025-09-19 19:38:26,639 INFO 2025-09-19 19:38:26,640 INFO 2025-09-19 19:38:26,640 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- tail -n 1000 /results/orders/benchmark-300/stdout.txt 2025-09-19 19:38:27,383 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2025-09-19 19:38:27,383 DEBUG --- stdout --- 2025-09-19 19:38:27,383 DEBUG [print_head_tail]: Print head (80 lines) and tail (80 lines) of input string (1000 lines): 2025-09-19 19:38:27,384 DEBUG ----- output ----- 20:38:05.581 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.581 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.584 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.584 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.585 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.586 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.586 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.586 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.592 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 20:38:05.594 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.594 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.598 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.598 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.600 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 20:38:05.600 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.600 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.600 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.600 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.603 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.603 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.603 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.603 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.612 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 20:38:05.614 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.614 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.617 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.617 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.620 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.620 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.621 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.621 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.621 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.621 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.622 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.622 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.623 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.623 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.623 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.623 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.625 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.625 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.627 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.627 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.630 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.630 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.633 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.633 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.640 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 20:38:05.641 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.641 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.643 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.643 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.648 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.648 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.649 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 20:38:05.664 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.664 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.675 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.675 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.676 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.676 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.681 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.681 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.684 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 20:38:05.686 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.686 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.694 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.694 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.696 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 20:38:05.699 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.699 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.700 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 20:38:05.703 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.703 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.708 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.708 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.712 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.712 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 20:38:05.715 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 20:38:05.715 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one [...] cookies: amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None d0d18cac672abbc=oxaOzpkWl3pJDUUiI1bQ2AO_MhE.*AAJTSQACMDIAAlNLABx0RXR1SHlKKzl1Y2pFMGRpV3cwZ2VHMkxBS2c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: restLoginInitiate: KO j.n.c.ClosedChannelException ========================= Session: Session(idc.benchmark.Mix2024,1075899,HashMap(gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, username -> user.5470047, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@72f93000, fake-user-agent -> Mozilla/5.0 (Linux; Android 8.0.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Mobile Safari/537.36, password -> Pa_ssw0rd, fake-ip -> 180.107.76.191, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@39186a8c),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@170a160d,io.netty.channel.epoll.EpollEventLoop@650eab8) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/authenticate headers: accept: application/json content-type: application/json user-agent: Mozilla/5.0 (Linux; Android 8.0.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Mobile Safari/537.36 X-Forwarded-For: 180.107.76.191 host: openam-perf-haproxy.forgeblocks.com body:StringChunksRequestBody{charset=UTF-8, content=} ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 20:38:05.823 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-143' crashed with 'j.u.NoSuchElementException: No attribute named 'callbacks' is defined', forwarding to the next one 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request ' UI AM validateGoto' failed for user 1075844: j.i.IOException: Premature close 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request ' UI IDM features' failed for user 1075707: j.i.IOException: Premature close 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: UI AM validateGoto: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1075844,HashMap(gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, username -> user.8780516, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@15c76614, gatling.http.referer -> https://openam-perf-haproxy.forgeblocks.com/am/, callbacks -> {"tokenId":"Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*","successUrl":"/enduser/?realm=/alpha","realm":"/alpha"}, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685726), CookieKey(d0d18cac672abbc,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(d0d18cac672abbc=Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685726))), password -> Pa_ssw0rd, fake-ip -> 159.87.114.0, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@41bccd0, gatling.http.cache.contentCache -> io.gatling.core.util.cache.Cache@1a12727c, tokenId -> Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, fake-user-agent -> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@170a160d,io.netty.channel.epoll.EpollEventLoop@239b0f9d) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/users?_action=validateGoto headers: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 Accept-API-Version: resource=2.1, protocol=1.0 referer: https://openam-perf-haproxy.forgeblocks.com/am/ accept: */* cookie: amlbcookie=01; d0d18cac672abbc=Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..* origin: https://openam-perf-haproxy.forgeblocks.com host: openam-perf-haproxy.forgeblocks.com content-length: 0 cookies: amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None d0d18cac672abbc=Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: UI IDM features: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1075707,HashMap(access_token -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhM2Q5Mjc1My1hZDhkLTQ5ODgtYjk5Yi1mMTAxOTY1YzlhNTEiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiNWVlMjIzMWEtZjcyMS00ZjhkLWExNjMtNWI5N2NhOGViOTM0LTQwNDgwMDU3Iiwic3VibmFtZSI6ImEzZDkyNzUzLWFkOGQtNDk4OC1iOTliLWYxMDE5NjVjOWE1MSIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXBlcmYtaGFwcm94eS5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJIdl81czlqam9laUNkNnBpb1J5Q0pYT2JneU0iLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NTgzMTA2ODUsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJvcGVuaWQiLCJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NTgzMTA2ODUsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzU4MzE0Mjg1LCJpYXQiOjE3NTgzMTA2ODUsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6ImRQTWJDTDl6S0hTZm84Y2pMVzRpcV9GZ0VWcyJ9.zUfgHFRg_Hc7Yl3SNIguTN9wX_P6_T1BzMbBzMJ2G4E, username -> user.7379863, access_token_2 -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhM2Q5Mjc1My1hZDhkLTQ5ODgtYjk5Yi1mMTAxOTY1YzlhNTEiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiNWVlMjIzMWEtZjcyMS00ZjhkLWExNjMtNWI5N2NhOGViOTM0LTQwNDgxMjk0Iiwic3VibmFtZSI6ImEzZDkyNzUzLWFkOGQtNDk4OC1iOTliLWYxMDE5NjVjOWE1MSIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXBlcmYtaGFwcm94eS5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJ1VEFrbUNUcVZJbzN2LXVCOG9RSWtMWTFfXzgiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NTgzMTA2ODUsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NTgzMTA2ODUsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzU4MzE0Mjg1LCJpYXQiOjE3NTgzMTA2ODUsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6ImYyN1VoVnpHWkljN1BnRUlVUlJlWklHOWsxSSJ9.G2xGCfSOsehrU3HJXFxjPKqG8fIprtAqkpHJIYfIBeY, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@371f86bd, gatling.http.referer -> https://openam-perf-haproxy.forgeblocks.com/am/, authcode -> 8O-mCeg8mypF-9t6QZwOdu7uGNg, callbacks -> {"tokenId":"UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..*","successUrl":"/enduser/?realm=/alpha","realm":"/alpha"}, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685279), CookieKey(d0d18cac672abbc,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(d0d18cac672abbc=UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685279))), user_uuid -> a3d92753-ad8d-4988-b99b-f101965c9a51, password -> Pa_ssw0rd, challenge -> Qgj4EP1YZaQY7KfCt2Yx2mDOg-vFZCiCK0FkP4QYuGE, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, fake-ip -> 78.112.3.0, verifier -> ODIwNjg0Nzg5Nzc1ODQwNzUwNQ, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@57c4a053, gatling.http.cache.contentCache -> io.gatling.core.util.cache.Cache@e8a292c, tokenId -> UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, fake-user-agent -> Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@170a160d,io.netty.channel.epoll.EpollEventLoop@650eab8) ========================= HTTP request: GET https://openam-perf-haproxy.forgeblocks.com/openidm/info/features?_queryFilter=true headers: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 referer: https://openam-perf-haproxy.forgeblocks.com/am/ Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhM2Q5Mjc1My1hZDhkLTQ5ODgtYjk5Yi1mMTAxOTY1YzlhNTEiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiNWVlMjIzMWEtZjcyMS00ZjhkLWExNjMtNWI5N2NhOGViOTM0LTQwNDgxMjk0Iiwic3VibmFtZSI6ImEzZDkyNzUzLWFkOGQtNDk4OC1iOTliLWYxMDE5NjVjOWE1MSIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXBlcmYtaGFwcm94eS5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJ1VEFrbUNUcVZJbzN2LXVCOG9RSWtMWTFfXzgiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NTgzMTA2ODUsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NTgzMTA2ODUsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzU4MzE0Mjg1LCJpYXQiOjE3NTgzMTA2ODUsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6ImYyN1VoVnpHWkljN1BnRUlVUlJlWklHOWsxSSJ9.G2xGCfSOsehrU3HJXFxjPKqG8fIprtAqkpHJIYfIBeY Accept-Api-Version: accept: */* cookie: amlbcookie=01; d0d18cac672abbc=UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..* host: openam-perf-haproxy.forgeblocks.com cookies: amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None d0d18cac672abbc=UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 2025-09-19 20:38:05,913 INFO Gatling simulation run ended successfully 2025-09-19 20:38:05,914 DEBUG No archive name provided. No archive will be created 2025-09-19 19:38:27,384 DEBUG --- stderr --- 2025-09-19 19:38:27,384 DEBUG 2025-09-19 19:38:27,384 INFO ________________________________________________________________________________ [2025-09-19 19:38:27] benchmark-300 show_cmd : Display order command ________________________________________________________________________________ 2025-09-19 19:38:27,385 INFO 2025-09-19 19:38:27,385 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-19 19:38:27,385 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-19 19:38:27,674 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-19 19:38:27,674 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:27,674 DEBUG --- stdout --- 2025-09-19 19:38:27,674 DEBUG overseer-0-57d79d468-khs5t 2025-09-19 19:38:27,674 DEBUG --- stderr --- 2025-09-19 19:38:27,674 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-19 19:38:27,675 INFO 2025-09-19 19:38:27,675 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-300/cmd.txt" ]; then echo "/results/orders/benchmark-300/cmd.txt file found"; exit 0; else echo "/results/orders/benchmark-300/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-300/cmd.txt file found" 2025-09-19 19:38:27,675 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-19 19:38:28,235 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 19:38:28,235 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:28,235 DEBUG --- stdout --- 2025-09-19 19:38:28,235 DEBUG /results/orders/benchmark-300/cmd.txt file found 2025-09-19 19:38:28,235 DEBUG --- stderr --- 2025-09-19 19:38:28,235 DEBUG 2025-09-19 19:38:28,236 INFO 2025-09-19 19:38:28,236 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-300/cmd.txt | grep "^\d+ .+$" 2025-09-19 19:38:28,236 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-19 19:38:28,801 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-19 19:38:28,801 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:28,801 DEBUG --- stdout --- 2025-09-19 19:38:28,801 DEBUG 4 /results/orders/benchmark-300/cmd.txt 2025-09-19 19:38:28,801 DEBUG --- stderr --- 2025-09-19 19:38:28,801 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-300/cmd.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/cmd.txt 2025-09-19 19:38:28,802 INFO 2025-09-19 19:38:28,802 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-300/cmd.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/cmd.txt 2025-09-19 19:38:28,802 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-19 19:38:29,342 INFO [loop_until]: OK (rc = 0) 2025-09-19 19:38:29,342 DEBUG --- stdout --- 2025-09-19 19:38:29,342 DEBUG 2025-09-19 19:38:29,342 DEBUG --- stderr --- 2025-09-19 19:38:29,342 DEBUG 2025-09-19 19:38:29,342 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/cmd.txt is small (1 lines), printing all lines: 2025-09-19 19:38:29,342 INFO ----- output ----- /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@perf-haproxy" --results-folder="/results/gatling/benchmark-300" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=M@ythe4th -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-09-19 19:38:29] benchmark-300 show_rc : Display rc order ________________________________________________________________________________ 2025-09-19 19:38:29,342 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/rc.txt is small (1 lines), printing all lines: 2025-09-19 19:38:29,342 INFO ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [2025-09-19 19:38:29] benchmark-300 show_stdout : Display stdout order ________________________________________________________________________________ 2025-09-19 19:38:29,343 DEBUG [print_head_tail]: Print head (100 lines) and tail (100 lines) of input file (/mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stdout.txt is 2010 lines): 2025-09-19 19:38:29,343 INFO ----- output ----- NOTE: This file was too big so it was truncated. Only first 1000 and last 1000 lines are present. ------------------------------------------------------------ 2025-09-19 19:37:45,445 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmpy3frtfst 2025-09-19 19:37:45,449 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-300" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=300 -Duser_password=Pa_ssw0rd Gatling 3.13.5 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 19:38:06.018 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'authAsPolicyAdmin' failed for user 1: status.find.in([200, 209], 304), found 401 19:38:06.030 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: authAsPolicyAdmin: KO status.find.in([200, 209], 304), found 401 ========================= Session: Session(getToken,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@44d81079, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@5fc81320, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, 1a73cf7f-46ee-49f4-b553-cf4f1d882f1c -> 0),KO,List(ExitOnCompleteLoopBlock(1a73cf7f-46ee-49f4-b553-cf4f1d882f1c)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@32d6f61f,io.netty.channel.epoll.EpollEventLoop@f79a760) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: X-OpenAM-Username: policyAdmin Content-Type: application/json Accept-API-Version: resource=2.0, protocol=1.0 X-OpenAM-Password: Pa_ssw0rd user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 401 Unauthorized headers: x-frame-options: SAMEORIGIN content-security-policy-report-only: frame-ancestors 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline' x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache content-type: application/json Content-Length: 62 date: Fri, 19 Sep 2025 18:38:05 GMT x-forgerock-transactionid: b52ae9f1-b7ae-4a7b-b65b-701bf590dbb1 strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"code":401,"reason":"Unauthorized","message":"Login failure"} <<<<<<<<<<<<<<<<<<<<<<<<< 19:38:06.033 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-8' crashed with 'j.u.NoSuchElementException: No attribute named 'policyEvalTokenId' is defined', forwarding to the next one 19:38:06.219 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.220 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.419 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.420 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.620 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.620 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.820 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.836 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.836 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.887 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.887 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:06.930 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:06.953 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:06.954 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.018 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.019 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.021 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.021 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.087 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.088 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.126 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.131 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.155 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.155 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.207 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.221 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.222 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.288 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.289 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.355 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.356 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.422 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.423 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.488 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.489 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.528 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 19:38:07.555 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.555 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.622 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.622 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.688 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 19:38:07.688 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 19:38:07.755 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one [...] 20:38:05.822 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 20:38:05.822 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request ' UI IDM version' failed for user 1075700: j.i.IOException: Premature close 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'restLoginInitiate' failed for user 1075899: j.n.c.ClosedChannelException 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: UI IDM version: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1075700,HashMap(access_token -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI4Zjc1NThmMS1hMGVkLTRmN2EtYWFlYS01NThjYjdmMWY3MWQiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiNWVlMjIzMWEtZjcyMS00ZjhkLWExNjMtNWI5N2NhOGViOTM0LTQwNDgwMDE0Iiwic3VibmFtZSI6IjhmNzU1OGYxLWEwZWQtNGY3YS1hYWVhLTU1OGNiN2YxZjcxZCIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXBlcmYtaGFwcm94eS5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJ3S1R1bmJSTElkQmFQaEpnR3dNZlMwaU96TkkiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NTgzMTA2ODUsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJvcGVuaWQiLCJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NTgzMTA2ODUsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzU4MzE0Mjg1LCJpYXQiOjE3NTgzMTA2ODUsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6IjQ0X0pnWHNkbU5Zekp5WUhMQVJtNTJaVG1JdyJ9.rQbMopvmvv8UkEL8hsE2EwzdxEumoGYjti5cFm8Bl2k, username -> user.2603391, access_token_2 -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI4Zjc1NThmMS1hMGVkLTRmN2EtYWFlYS01NThjYjdmMWY3MWQiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiYzRlNmY4NmMtNWE3Zi00ZmJmLTk3ODgtYTc2MzcxOWQ0ZWI2LTQwMzc1NTY5Iiwic3VibmFtZSI6IjhmNzU1OGYxLWEwZWQtNGY3YS1hYWVhLTU1OGNiN2YxZjcxZCIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXBlcmYtaGFwcm94eS5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJzUE9NUTE4UFgwMTVzUUM3ZllJcTkxNEstaU0iLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NTgzMTA2ODUsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NTgzMTA2ODUsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzU4MzE0Mjg1LCJpYXQiOjE3NTgzMTA2ODUsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6IlVLbUdkckRwcmktZ3ZDcE9uSlJVVTBkQjI1cyJ9.RXju1w9VH1S3ONvlKABHhYGxXKyk_5aSK36uPDKketc, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@6f6ca595, gatling.http.referer -> https://openam-perf-haproxy.forgeblocks.com/am/, authcode -> EyeUVjf42VD2_g-Mz-CYnRYL1GI, callbacks -> {"tokenId":"oxaOzpkWl3pJDUUiI1bQ2AO_MhE.*AAJTSQACMDIAAlNLABx0RXR1SHlKKzl1Y2pFMGRpV3cwZ2VHMkxBS2c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*","successUrl":"/enduser/?realm=/alpha","realm":"/alpha"}, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685275), CookieKey(d0d18cac672abbc,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(d0d18cac672abbc=oxaOzpkWl3pJDUUiI1bQ2AO_MhE.*AAJTSQACMDIAAlNLABx0RXR1SHlKKzl1Y2pFMGRpV3cwZ2VHMkxBS2c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685275))), user_uuid -> 8f7558f1-a0ed-4f7a-aaea-558cb7f1f71d, password -> Pa_ssw0rd, challenge -> zNjPRdTS6YJoKAcBC7BjUinztdgVc7OUdFkZwZwblZU, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, fake-ip -> 11.220.229.198, verifier -> NjQwNDI1NzM3MDQ0MzQyNDMyOA, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@33a9405b, gatling.http.cache.contentCache -> io.gatling.core.util.cache.Cache@3ca5ad19, tokenId -> oxaOzpkWl3pJDUUiI1bQ2AO_MhE.*AAJTSQACMDIAAlNLABx0RXR1SHlKKzl1Y2pFMGRpV3cwZ2VHMkxBS2c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, fake-user-agent -> Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@170a160d,io.netty.channel.epoll.EpollEventLoop@239b0f9d) ========================= HTTP request: GET https://openam-perf-haproxy.forgeblocks.com/openidm/info/version headers: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 referer: https://openam-perf-haproxy.forgeblocks.com/am/ Accept-Api-Version: accept: */* cookie: amlbcookie=01; d0d18cac672abbc=oxaOzpkWl3pJDUUiI1bQ2AO_MhE.*AAJTSQACMDIAAlNLABx0RXR1SHlKKzl1Y2pFMGRpV3cwZ2VHMkxBS2c9AAR0eXBlAANDVFMAAlMxAAIwMQ..* host: openam-perf-haproxy.forgeblocks.com cookies: amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None d0d18cac672abbc=oxaOzpkWl3pJDUUiI1bQ2AO_MhE.*AAJTSQACMDIAAlNLABx0RXR1SHlKKzl1Y2pFMGRpV3cwZ2VHMkxBS2c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: restLoginInitiate: KO j.n.c.ClosedChannelException ========================= Session: Session(idc.benchmark.Mix2024,1075899,HashMap(gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, username -> user.5470047, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@72f93000, fake-user-agent -> Mozilla/5.0 (Linux; Android 8.0.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Mobile Safari/537.36, password -> Pa_ssw0rd, fake-ip -> 180.107.76.191, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@39186a8c),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@170a160d,io.netty.channel.epoll.EpollEventLoop@650eab8) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/authenticate headers: accept: application/json content-type: application/json user-agent: Mozilla/5.0 (Linux; Android 8.0.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Mobile Safari/537.36 X-Forwarded-For: 180.107.76.191 host: openam-perf-haproxy.forgeblocks.com body:StringChunksRequestBody{charset=UTF-8, content=} ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 20:38:05.823 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-143' crashed with 'j.u.NoSuchElementException: No attribute named 'callbacks' is defined', forwarding to the next one 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request ' UI AM validateGoto' failed for user 1075844: j.i.IOException: Premature close 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request ' UI IDM features' failed for user 1075707: j.i.IOException: Premature close 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: UI AM validateGoto: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1075844,HashMap(gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, username -> user.8780516, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@15c76614, gatling.http.referer -> https://openam-perf-haproxy.forgeblocks.com/am/, callbacks -> {"tokenId":"Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*","successUrl":"/enduser/?realm=/alpha","realm":"/alpha"}, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685726), CookieKey(d0d18cac672abbc,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(d0d18cac672abbc=Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685726))), password -> Pa_ssw0rd, fake-ip -> 159.87.114.0, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@41bccd0, gatling.http.cache.contentCache -> io.gatling.core.util.cache.Cache@1a12727c, tokenId -> Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, fake-user-agent -> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@170a160d,io.netty.channel.epoll.EpollEventLoop@239b0f9d) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/users?_action=validateGoto headers: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 Accept-API-Version: resource=2.1, protocol=1.0 referer: https://openam-perf-haproxy.forgeblocks.com/am/ accept: */* cookie: amlbcookie=01; d0d18cac672abbc=Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..* origin: https://openam-perf-haproxy.forgeblocks.com host: openam-perf-haproxy.forgeblocks.com content-length: 0 cookies: amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None d0d18cac672abbc=Yy7YJoyrDSiScSAyvv6dz4VCx6U.*AAJTSQACMDIAAlNLABxCcUpCcUs4WUVyYms0YXR1MngzT3Q2OTN0bjA9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 20:38:05.823 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: UI IDM features: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1075707,HashMap(access_token -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhM2Q5Mjc1My1hZDhkLTQ5ODgtYjk5Yi1mMTAxOTY1YzlhNTEiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiNWVlMjIzMWEtZjcyMS00ZjhkLWExNjMtNWI5N2NhOGViOTM0LTQwNDgwMDU3Iiwic3VibmFtZSI6ImEzZDkyNzUzLWFkOGQtNDk4OC1iOTliLWYxMDE5NjVjOWE1MSIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXBlcmYtaGFwcm94eS5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJIdl81czlqam9laUNkNnBpb1J5Q0pYT2JneU0iLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NTgzMTA2ODUsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJvcGVuaWQiLCJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NTgzMTA2ODUsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzU4MzE0Mjg1LCJpYXQiOjE3NTgzMTA2ODUsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6ImRQTWJDTDl6S0hTZm84Y2pMVzRpcV9GZ0VWcyJ9.zUfgHFRg_Hc7Yl3SNIguTN9wX_P6_T1BzMbBzMJ2G4E, username -> user.7379863, access_token_2 -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhM2Q5Mjc1My1hZDhkLTQ5ODgtYjk5Yi1mMTAxOTY1YzlhNTEiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiNWVlMjIzMWEtZjcyMS00ZjhkLWExNjMtNWI5N2NhOGViOTM0LTQwNDgxMjk0Iiwic3VibmFtZSI6ImEzZDkyNzUzLWFkOGQtNDk4OC1iOTliLWYxMDE5NjVjOWE1MSIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXBlcmYtaGFwcm94eS5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJ1VEFrbUNUcVZJbzN2LXVCOG9RSWtMWTFfXzgiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NTgzMTA2ODUsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NTgzMTA2ODUsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzU4MzE0Mjg1LCJpYXQiOjE3NTgzMTA2ODUsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6ImYyN1VoVnpHWkljN1BnRUlVUlJlWklHOWsxSSJ9.G2xGCfSOsehrU3HJXFxjPKqG8fIprtAqkpHJIYfIBeY, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@371f86bd, gatling.http.referer -> https://openam-perf-haproxy.forgeblocks.com/am/, authcode -> 8O-mCeg8mypF-9t6QZwOdu7uGNg, callbacks -> {"tokenId":"UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..*","successUrl":"/enduser/?realm=/alpha","realm":"/alpha"}, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685279), CookieKey(d0d18cac672abbc,openam-perf-haproxy.forgeblocks.com,/) -> StoredCookie(d0d18cac672abbc=UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1758310685279))), user_uuid -> a3d92753-ad8d-4988-b99b-f101965c9a51, password -> Pa_ssw0rd, challenge -> Qgj4EP1YZaQY7KfCt2Yx2mDOg-vFZCiCK0FkP4QYuGE, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, fake-ip -> 78.112.3.0, verifier -> ODIwNjg0Nzg5Nzc1ODQwNzUwNQ, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@57c4a053, gatling.http.cache.contentCache -> io.gatling.core.util.cache.Cache@e8a292c, tokenId -> UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, fake-user-agent -> Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x000078de186987a0@170a160d,io.netty.channel.epoll.EpollEventLoop@650eab8) ========================= HTTP request: GET https://openam-perf-haproxy.forgeblocks.com/openidm/info/features?_queryFilter=true headers: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 referer: https://openam-perf-haproxy.forgeblocks.com/am/ Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhM2Q5Mjc1My1hZDhkLTQ5ODgtYjk5Yi1mMTAxOTY1YzlhNTEiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiNWVlMjIzMWEtZjcyMS00ZjhkLWExNjMtNWI5N2NhOGViOTM0LTQwNDgxMjk0Iiwic3VibmFtZSI6ImEzZDkyNzUzLWFkOGQtNDk4OC1iOTliLWYxMDE5NjVjOWE1MSIsImlzcyI6Imh0dHBzOi8vb3BlbmFtLXBlcmYtaGFwcm94eS5mb3JnZWJsb2Nrcy5jb206NDQzL2FtL29hdXRoMi9yZWFsbXMvcm9vdC9yZWFsbXMvYWxwaGEiLCJ0b2tlbk5hbWUiOiJhY2Nlc3NfdG9rZW4iLCJ0b2tlbl90eXBlIjoiQmVhcmVyIiwiYXV0aEdyYW50SWQiOiJ1VEFrbUNUcVZJbzN2LXVCOG9RSWtMWTFfXzgiLCJub25jZSI6IjEyMzQiLCJjbGllbnRfaWQiOiJlbmRVc2VyVUlDbGllbnQiLCJhdWQiOiJlbmRVc2VyVUlDbGllbnQiLCJuYmYiOjE3NTgzMTA2ODUsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJzY29wZSI6WyJmcjppZG06KiJdLCJhdXRoX3RpbWUiOjE3NTgzMTA2ODUsInJlYWxtIjoiL2FscGhhIiwiZXhwIjoxNzU4MzE0Mjg1LCJpYXQiOjE3NTgzMTA2ODUsImV4cGlyZXNfaW4iOjM2MDAsImp0aSI6ImYyN1VoVnpHWkljN1BnRUlVUlJlWklHOWsxSSJ9.G2xGCfSOsehrU3HJXFxjPKqG8fIprtAqkpHJIYfIBeY Accept-Api-Version: accept: */* cookie: amlbcookie=01; d0d18cac672abbc=UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..* host: openam-perf-haproxy.forgeblocks.com cookies: amlbcookie=01, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None d0d18cac672abbc=UbnAXyrWyO_HAhZLA6YBiFqpa1w.*AAJTSQACMDIAAlNLABxTdWtyajNMU3E1aHIrVUV3MVdIOUFsZUZFTXM9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-perf-haproxy.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 2025-09-19 20:38:05,913 INFO Gatling simulation run ended successfully 2025-09-19 20:38:05,914 DEBUG No archive name provided. No archive will be created ________________________________________________________________________________ [2025-09-19 19:38:29] benchmark-300 show_stderr : Display stderr order ________________________________________________________________________________ 2025-09-19 19:38:29,344 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/04_benchmark-300/stderr.txt is small (0 lines), printing all lines: 2025-09-19 19:38:29,344 INFO ----- output ----- ________________________________________________________________________________ [2025-09-19 19:38:29] benchmark-300 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2025-09-19 19:38:29] benchmark-300 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2025-09-19 19:38:29] benchmark-300 post : Post method ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped