--Task-- name: benchmark-10 enabled: True class_name: GatlingTask source_name: overseer-0 source_namespace: >default< target_name: am target_namespace: >default< start: 0.0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates below loop: False interval: None dependencies: [] wait_for: [] preceding_task: None options: {'duration': 3600, 'java-opts': '-Drealm=alpha -Duser_password=Pa_ssw0rd -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html', 'testCase': 'idc.benchmark.Mix2024', 'testMode': 'open', 'throughput': 10} group_name: None Current dir: /mnt/disk1/xslou/workshop/lodestar-fork/pyrock 2025-09-16 16:14:34,337 INFO 2025-09-16 16:14:34,337 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 16:14:34,337 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 16:14:34,616 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 16:14:34,616 INFO [loop_until]: OK (rc = 0) 2025-09-16 16:14:34,616 DEBUG --- stdout --- 2025-09-16 16:14:34,616 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 16:14:34,616 DEBUG --- stderr --- 2025-09-16 16:14:34,616 DEBUG 2025-09-16 16:14:34,616 INFO Lodestar assumed that full stack is deployed so will set all products host URLs in Gatling system properties 2025-09-16 16:14:34 - INFO: Timeout for this Task has changed to 1h 12m (4320 seconds) 2025-09-16 16:14:34 - INFO: timeout was calculated based on simulation duration parameter 2025-09-16 16:14:34 - INFO: Interval for this Task has changed to 2m (120 seconds) 2025-09-16 16:14:34 - INFO: interval was set to Task default because it was unset, based on self.timeout value because task is allowed to stop by itself Pod log file : /results/orders/benchmark-10/stdout.txt Pod log : /results/orders/benchmark-10 2025-09-16 16:14:35 - INFO: Get overseer pod resources requests memory 2025-09-16 16:14:35,390 INFO 2025-09-16 16:14:35,390 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods overseer-0-57d79d468-khs5t --output jsonpath={.spec.containers[].resources.requests.memory} 2025-09-16 16:14:35,390 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 16:14:35,700 INFO [loop_until]: OK (rc = 0) 2025-09-16 16:14:35,700 DEBUG --- stdout --- 2025-09-16 16:14:35,700 DEBUG 4Gi 2025-09-16 16:14:35,700 DEBUG --- stderr --- 2025-09-16 16:14:35,700 DEBUG ________________________________________________________________________________ [2025-09-16 16:14:35] benchmark-10 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@perf-haproxy" --results-folder="/results/gatling/benchmark-10" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-09-16 16:14:35] benchmark-10 create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file: /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/order.json { "name": "benchmark-10", "command": " /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class=\"com.forgerock.pyrock.Loadtest\" --run-description=\"idc.benchmark.Mix2024@perf-haproxy\" --results-folder=\"/results/gatling/benchmark-10\" --no-reports --simulation-jvm-args=\"-XX:+ExitOnOutOfMemoryError -Xmx4G\" --simulation-system-properties=\"-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd\"", "out-dir": "/results/orders/benchmark-10" } ________________________________________________________________________________ [2025-09-16 16:14:35] benchmark-10 loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2025-09-16 16:14:35,708 INFO 2025-09-16 16:14:35,708 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2025-09-16 16:14:35,708 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-09-16 16:14:35,962 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 16:14:35,962 INFO [loop_until]: OK (rc = 0) 2025-09-16 16:14:35,962 DEBUG --- stdout --- 2025-09-16 16:14:35,962 DEBUG WAITING 2025-09-16 16:14:35,962 DEBUG --- stderr --- 2025-09-16 16:14:35,962 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 28 0 --:--:-- --:--:-- --:--:-- 28 ________________________________________________________________________________ [2025-09-16 16:14:35] benchmark-10 start_order : Send the order to the overseer pod ________________________________________________________________________________ 2025-09-16 16:14:35,962 INFO 2025-09-16 16:14:35,963 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 16:14:35,963 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 16:14:36,246 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 16:14:36,246 INFO [loop_until]: OK (rc = 0) 2025-09-16 16:14:36,246 DEBUG --- stdout --- 2025-09-16 16:14:36,246 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 16:14:36,246 DEBUG --- stderr --- 2025-09-16 16:14:36,246 DEBUG 2025-09-16 16:14:36,246 INFO 2025-09-16 16:14:36,246 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/order.json overseer-0-57d79d468-khs5t:/results/orders/order.json 2025-09-16 16:14:36,246 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 16:14:37,422 INFO [loop_until]: OK (rc = 0) 2025-09-16 16:14:37,422 DEBUG --- stdout --- 2025-09-16 16:14:37,422 DEBUG 2025-09-16 16:14:37,422 DEBUG --- stderr --- 2025-09-16 16:14:37,422 DEBUG Order has been successfully forwarded to overseer 2025-09-16 16:14:37 - INFO: Check if Gatling simulation started successfully 2025-09-16 16:14:37,422 INFO 2025-09-16 16:14:37,422 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- head --lines 5000 /results/orders/benchmark-10/stdout.txt | grep "Simulation com.forgerock.pyrock.Loadtest started..." 2025-09-16 16:14:37,422 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-09-16 16:14:37,976 INFO [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-16 16:14:43,551 INFO [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-16 16:14:49,096 INFO [loop_until]: Function succeeded after 11s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-16 16:14:54,635 INFO [loop_until]: Function succeeded after 17s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-09-16 16:15:00,167 INFO [loop_until]: Function succeeded after 22s (rc=0) - expected pattern found 2025-09-16 16:15:00,168 INFO [loop_until]: OK (rc = 0) 2025-09-16 16:15:00,168 DEBUG --- stdout --- 2025-09-16 16:15:00,168 DEBUG 2025-09-16 17:14:39,914 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmp30pz0k1s 2025-09-16 17:14:39,918 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-10" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 17:14:59.090 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'authAsPolicyAdmin' failed for user 1: status.find.in([200, 209], 304), found 401 17:14:59.102 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: authAsPolicyAdmin: KO status.find.in([200, 209], 304), found 401 ========================= Session: Session(getToken,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@42ffc2c, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@1adb60e2, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, b4b28079-eff6-4dd1-a8cf-6b5c211335f6 -> 0),KO,List(ExitOnCompleteLoopBlock(b4b28079-eff6-4dd1-a8cf-6b5c211335f6)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x0000785eb46a0000@2df2448a,io.netty.channel.epoll.EpollEventLoop@f79a760) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: X-OpenAM-Username: policyAdmin Content-Type: application/json Accept-API-Version: resource=2.0, protocol=1.0 X-OpenAM-Password: Pa_ssw0rd user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 401 Unauthorized headers: x-frame-options: SAMEORIGIN content-security-policy-report-only: frame-ancestors 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline' x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache content-type: application/json Content-Length: 62 date: Tue, 16 Sep 2025 16:14:59 GMT x-forgerock-transactionid: da183b26-e0e4-4fc4-b7bb-0b3690a53185 strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"code":401,"reason":"Unauthorized","message":"Login failure"} <<<<<<<<<<<<<<<<<<<<<<<<< 17:14:59.105 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-8' crashed with 'j.u.NoSuchElementException: No attribute named 'policyEvalTokenId' is defined', forwarding to the next one 2025-09-16 16:15:00,168 DEBUG --- stderr --- 2025-09-16 16:15:00,168 DEBUG ________________________________________________________________________________ [2025-09-16 16:15:00] benchmark-10 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 1h 12m. Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 2m, retry Order is running since 4m, retry Order is running since 6m, retry Order is running since 8m 01s, retry Order is running since 10m 01s, retry Order is running since 12m 01s, retry Order is running since 14m 01s, retry Order is running since 16m 02s, retry Order is running since 18m 02s, retry Order is running since 20m 02s, retry Order is running since 22m 02s, retry Order is running since 24m 03s, retry Order is running since 26m 03s, retry Order is running since 28m 03s, retry Order is running since 30m 04s, retry Order is running since 32m 04s, retry Order is running since 34m 04s, retry Order is running since 36m 04s, retry Order is running since 38m 05s, retry Order is running since 40m 05s, retry Order is running since 42m 05s, retry Order is running since 44m 05s, retry Order is running since 46m 06s, retry Order is running since 48m 06s, retry Order is running since 50m 06s, retry Order is running since 52m 07s, retry Order is running since 54m 07s, retry Order is running since 56m 07s, retry Order is running since 58m 07s, retry PASS : Order stopped by itself after 1h and before timeout (1h 12m) ________________________________________________________________________________ [2025-09-16 17:15:08] benchmark-10 stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2025-09-16 17:15:08,282 INFO 2025-09-16 17:15:08,283 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 17:15:08,283 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 17:15:08,605 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 17:15:08,606 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:08,606 DEBUG --- stdout --- 2025-09-16 17:15:08,606 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 17:15:08,606 DEBUG --- stderr --- 2025-09-16 17:15:08,606 DEBUG 2025-09-16 17:15:08,606 INFO 2025-09-16 17:15:08,606 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- rm -f /results/orders/order.json 2025-09-16 17:15:08,606 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 17:15:09,206 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:09,206 DEBUG --- stdout --- 2025-09-16 17:15:09,206 DEBUG 2025-09-16 17:15:09,206 DEBUG --- stderr --- 2025-09-16 17:15:09,206 DEBUG Order ran during 1h ________________________________________________________________________________ [2025-09-16 17:15:09] benchmark-10 download_logs : Download individual order files ________________________________________________________________________________ 2025-09-16 17:15:09 - PASS: file already downloaded /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/order.json 2025-09-16 17:15:09,207 INFO 2025-09-16 17:15:09,207 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 17:15:09,207 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 17:15:09,489 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 17:15:09,489 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:09,489 DEBUG --- stdout --- 2025-09-16 17:15:09,489 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 17:15:09,489 DEBUG --- stderr --- 2025-09-16 17:15:09,489 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-16 17:15:09,490 INFO 2025-09-16 17:15:09,490 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-10/rc.txt" ]; then echo "/results/orders/benchmark-10/rc.txt file found"; exit 0; else echo "/results/orders/benchmark-10/rc.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-10/rc.txt file found" 2025-09-16 17:15:09,490 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 17:15:10,053 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 17:15:10,053 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:10,054 DEBUG --- stdout --- 2025-09-16 17:15:10,054 DEBUG /results/orders/benchmark-10/rc.txt file found 2025-09-16 17:15:10,054 DEBUG --- stderr --- 2025-09-16 17:15:10,054 DEBUG 2025-09-16 17:15:10,054 INFO 2025-09-16 17:15:10,054 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-10/rc.txt | grep "^\d+ .+$" 2025-09-16 17:15:10,054 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-16 17:15:10,607 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 17:15:10,607 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:10,607 DEBUG --- stdout --- 2025-09-16 17:15:10,607 DEBUG 4 /results/orders/benchmark-10/rc.txt 2025-09-16 17:15:10,607 DEBUG --- stderr --- 2025-09-16 17:15:10,607 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-10/rc.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/rc.txt 2025-09-16 17:15:10,609 INFO 2025-09-16 17:15:10,609 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-10/rc.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/rc.txt 2025-09-16 17:15:10,609 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-16 17:15:11,174 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:11,174 DEBUG --- stdout --- 2025-09-16 17:15:11,174 DEBUG 2025-09-16 17:15:11,174 DEBUG --- stderr --- 2025-09-16 17:15:11,174 DEBUG 2025-09-16 17:15:11,174 INFO 2025-09-16 17:15:11,174 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 17:15:11,174 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 17:15:11,458 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 17:15:11,458 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:11,458 DEBUG --- stdout --- 2025-09-16 17:15:11,458 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 17:15:11,458 DEBUG --- stderr --- 2025-09-16 17:15:11,458 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-16 17:15:11,459 INFO 2025-09-16 17:15:11,459 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-10/stderr.txt" ]; then echo "/results/orders/benchmark-10/stderr.txt file found"; exit 0; else echo "/results/orders/benchmark-10/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-10/stderr.txt file found" 2025-09-16 17:15:11,459 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 17:15:11,996 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 17:15:11,996 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:11,996 DEBUG --- stdout --- 2025-09-16 17:15:11,996 DEBUG /results/orders/benchmark-10/stderr.txt file found 2025-09-16 17:15:11,996 DEBUG --- stderr --- 2025-09-16 17:15:11,996 DEBUG 2025-09-16 17:15:11,997 INFO 2025-09-16 17:15:11,997 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-10/stderr.txt | grep "^\d+ .+$" 2025-09-16 17:15:11,997 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-16 17:15:12,550 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 17:15:12,550 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:12,550 DEBUG --- stdout --- 2025-09-16 17:15:12,550 DEBUG 0 /results/orders/benchmark-10/stderr.txt 2025-09-16 17:15:12,550 DEBUG --- stderr --- 2025-09-16 17:15:12,550 DEBUG - Download file (size 0B) source : /results/orders/benchmark-10/stderr.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stderr.txt 2025-09-16 17:15:12,550 INFO 2025-09-16 17:15:12,550 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-10/stderr.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stderr.txt 2025-09-16 17:15:12,550 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-16 17:15:13,091 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:13,091 DEBUG --- stdout --- 2025-09-16 17:15:13,091 DEBUG 2025-09-16 17:15:13,091 DEBUG --- stderr --- 2025-09-16 17:15:13,091 DEBUG 2025-09-16 17:15:13,091 INFO 2025-09-16 17:15:13,092 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 17:15:13,092 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 17:15:13,378 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 17:15:13,378 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:13,378 DEBUG --- stdout --- 2025-09-16 17:15:13,378 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 17:15:13,378 DEBUG --- stderr --- 2025-09-16 17:15:13,378 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-16 17:15:13,379 INFO 2025-09-16 17:15:13,379 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-10/stdout.txt" ]; then echo "/results/orders/benchmark-10/stdout.txt file found"; exit 0; else echo "/results/orders/benchmark-10/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-10/stdout.txt file found" 2025-09-16 17:15:13,379 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 17:15:13,944 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 17:15:13,944 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:13,944 DEBUG --- stdout --- 2025-09-16 17:15:13,944 DEBUG /results/orders/benchmark-10/stdout.txt file found 2025-09-16 17:15:13,944 DEBUG --- stderr --- 2025-09-16 17:15:13,944 DEBUG 2025-09-16 17:15:13,945 INFO 2025-09-16 17:15:13,945 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-10/stdout.txt | grep "^\d+ .+$" 2025-09-16 17:15:13,945 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-16 17:15:14,486 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 17:15:14,486 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:14,486 DEBUG --- stdout --- 2025-09-16 17:15:14,486 DEBUG 21912 /results/orders/benchmark-10/stdout.txt 2025-09-16 17:15:14,486 DEBUG --- stderr --- 2025-09-16 17:15:14,486 DEBUG - Download file (size 21.912MB) source : /results/orders/benchmark-10/stdout.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stdout.txt 2025-09-16 17:15:14,487 INFO 2025-09-16 17:15:14,487 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-10/stdout.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stdout.txt 2025-09-16 17:15:14,487 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-16 17:15:15,952 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:15,952 DEBUG --- stdout --- 2025-09-16 17:15:15,952 DEBUG tar: Removing leading `/' from member names 2025-09-16 17:15:15,952 DEBUG --- stderr --- 2025-09-16 17:15:15,952 DEBUG ________________________________________________________________________________ [2025-09-16 17:15:15] benchmark-10 show_cmd : Display order command ________________________________________________________________________________ 2025-09-16 17:15:15,953 INFO 2025-09-16 17:15:15,953 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-09-16 17:15:15,953 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-09-16 17:15:16,229 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-09-16 17:15:16,229 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:16,229 DEBUG --- stdout --- 2025-09-16 17:15:16,229 DEBUG overseer-0-57d79d468-khs5t 2025-09-16 17:15:16,229 DEBUG --- stderr --- 2025-09-16 17:15:16,229 DEBUG Checking if file exists on pod 'overseer-0-57d79d468-khs5t', wait max 3 minutes if not. 2025-09-16 17:15:16,230 INFO 2025-09-16 17:15:16,230 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- bash -c 'if [ -f "/results/orders/benchmark-10/cmd.txt" ]; then echo "/results/orders/benchmark-10/cmd.txt file found"; exit 0; else echo "/results/orders/benchmark-10/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-10/cmd.txt file found" 2025-09-16 17:15:16,230 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-09-16 17:15:16,776 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 17:15:16,776 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:16,776 DEBUG --- stdout --- 2025-09-16 17:15:16,776 DEBUG /results/orders/benchmark-10/cmd.txt file found 2025-09-16 17:15:16,776 DEBUG --- stderr --- 2025-09-16 17:15:16,776 DEBUG 2025-09-16 17:15:16,776 INFO 2025-09-16 17:15:16,776 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-57d79d468-khs5t -- du -k /results/orders/benchmark-10/cmd.txt | grep "^\d+ .+$" 2025-09-16 17:15:16,776 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-09-16 17:15:17,318 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-09-16 17:15:17,319 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:17,319 DEBUG --- stdout --- 2025-09-16 17:15:17,319 DEBUG 4 /results/orders/benchmark-10/cmd.txt 2025-09-16 17:15:17,319 DEBUG --- stderr --- 2025-09-16 17:15:17,319 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-10/cmd.txt @ overseer-0-57d79d468-khs5t destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/cmd.txt 2025-09-16 17:15:17,319 INFO 2025-09-16 17:15:17,319 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-57d79d468-khs5t:/results/orders/benchmark-10/cmd.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/cmd.txt 2025-09-16 17:15:17,319 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-09-16 17:15:17,860 INFO [loop_until]: OK (rc = 0) 2025-09-16 17:15:17,860 DEBUG --- stdout --- 2025-09-16 17:15:17,860 DEBUG tar: Removing leading `/' from member names 2025-09-16 17:15:17,860 DEBUG --- stderr --- 2025-09-16 17:15:17,860 DEBUG 2025-09-16 17:15:17,860 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/cmd.txt is small (1 lines), printing all lines: 2025-09-16 17:15:17,860 INFO ----- output ----- /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@perf-haproxy" --results-folder="/results/gatling/benchmark-10" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=M@ythe4th -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-09-16 17:15:17] benchmark-10 show_rc : Display rc order ________________________________________________________________________________ 2025-09-16 17:15:17,860 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/rc.txt is small (1 lines), printing all lines: 2025-09-16 17:15:17,860 INFO ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [2025-09-16 17:15:17] benchmark-10 show_stdout : Display stdout order ________________________________________________________________________________ 2025-09-16 17:15:17,883 DEBUG [print_head_tail]: Print head (100 lines) and tail (100 lines) of input file (/mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stdout.txt is 157218 lines): 2025-09-16 17:15:17,911 INFO ----- output ----- 2025-09-16 17:14:39,914 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmp30pz0k1s 2025-09-16 17:14:39,918 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-10" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-haproxy.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-perf-haproxy.forgeblocks.com -Dlogin_cookie=d0d18cac672abbc -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=10 -Duser_password=Pa_ssw0rd System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 17:14:59.090 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'authAsPolicyAdmin' failed for user 1: status.find.in([200, 209], 304), found 401 17:14:59.102 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: authAsPolicyAdmin: KO status.find.in([200, 209], 304), found 401 ========================= Session: Session(getToken,1,Map(gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@42ffc2c, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@1adb60e2, gatling.http.cache.baseUrl -> https://openam-perf-haproxy.forgeblocks.com:443, b4b28079-eff6-4dd1-a8cf-6b5c211335f6 -> 0),KO,List(ExitOnCompleteLoopBlock(b4b28079-eff6-4dd1-a8cf-6b5c211335f6)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$822/0x0000785eb46a0000@2df2448a,io.netty.channel.epoll.EpollEventLoop@f79a760) ========================= HTTP request: POST https://openam-perf-haproxy.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: X-OpenAM-Username: policyAdmin Content-Type: application/json Accept-API-Version: resource=2.0, protocol=1.0 X-OpenAM-Password: Pa_ssw0rd user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 accept: */* host: openam-perf-haproxy.forgeblocks.com content-length: 0 ========================= HTTP response: version: HTTP/1.1 status: 401 Unauthorized headers: x-frame-options: SAMEORIGIN content-security-policy-report-only: frame-ancestors 'self'; script-src 'self' 'unsafe-eval' 'unsafe-inline' x-content-type-options: nosniff cache-control: private content-api-version: resource=2.1 expires: 0 pragma: no-cache content-type: application/json Content-Length: 62 date: Tue, 16 Sep 2025 16:14:59 GMT x-forgerock-transactionid: da183b26-e0e4-4fc4-b7bb-0b3690a53185 strict-transport-security: max-age=31536000; includeSubDomains; preload; x-robots-tag: none Via: 1.1 google Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 body: {"code":401,"reason":"Unauthorized","message":"Login failure"} <<<<<<<<<<<<<<<<<<<<<<<<< 17:14:59.105 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-8' crashed with 'j.u.NoSuchElementException: No attribute named 'policyEvalTokenId' is defined', forwarding to the next one 17:15:00.833 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:00.834 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:01.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:01.830 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:02.022 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 17:15:02.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:02.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:02.978 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 17:15:03.330 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:03.330 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one ================================================================================ 2025-09-16 16:15:03 GMT 5s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=4 KO=1 ) > authAsPolicyAdmin (OK=0 KO=1 ) > restLoginInitiate (OK=2 KO=0 ) > userRestlogin (OK=2 KO=0 ) ---- Errors -------------------------------------------------------------------- > EvalPolicy: No attribute named 'tokenIdCookieName' is defined 2 (66.67%) > status.find.in([200, 209], 304), found 401 1 (33.33%) ---- getToken ------------------------------------------------------------------ [--------------------------------------------------------------------------] 0% waiting: 0 / active: 1 / done: 0 ---- idc.benchmark.Mix2024 ----------------------------------------------------- [ ] 0% waiting: 35942 / active: 0 / done: 4 ================================================================================ 17:15:03.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:03.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:04.330 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:04.330 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:04.453 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 17:15:04.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:04.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:04.988 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 17:15:05.330 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:05.330 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:05.437 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 17:15:05.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:05.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:05.945 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'tokenIdCookieName' is defined 17:15:06.330 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:06.330 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:06.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one 17:15:06.829 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-199' crashed with 'j.u.NoSuchElementException: key not found: tokenIdCookieName', forwarding to the next one 17:15:07.163 [ERROR] i.g.c.a.b.SessionHookBuilder$$anon$1 - 'hook-200' crashed with 'j.u.NoSuchElementException: key not found: policyEvalTokenId', forwarding to the next one [...] > UI IDM managed user schema (OK=7606 KO=0 ) > UI IDM user (OK=7606 KO=0 ) > UI IDM oidcToken (OK=7605 KO=1 ) > userinfo (OK=7605 KO=0 ) > UI-Logout (OK=7606 KO=0 ) > UI-Logout Redirect 1 (OK=7606 KO=0 ) > UI AM endsession (OK=7605 KO=0 ) > UI AM revoke1 (OK=7605 KO=0 ) > UI AM revoke2 (OK=7605 KO=0 ) > loginTreeWorkflow_restLoginInitiate (OK=1235 KO=0 ) > loginTreeWorkflow_restLoginUsernamePasswordCallback (OK=1235 KO=0 ) > loginTreeWorkflow_restloginProgressiveProfileCallback (OK=72 KO=0 ) > restloginProgressiveProfileCallback (OK=436 KO=0 ) ---- Errors -------------------------------------------------------------------- > EvalPolicy: No attribute named 'tokenIdCookieName' is defined 11503 (94.32%) > jsonPath($.tokenId).find.exists, found nothing 681 ( 5.58%) > status.find.in([200, 209], 304), found 401 3 ( 0.02%) > status.find.in([200, 209], 304), found 502 3 ( 0.02%) > status.find.is(200), but actually found 502 1 ( 0.01%) > headerRegex((location,#id_token=(.*))).find.exists, found noth 1 ( 0.01%) ing > userinfo: No attribute named 'access_token' is defined 1 ( 0.01%) > UI AM revoke1: No attribute named 'access_token' is defined 1 ( 0.01%) > UI AM revoke2: No attribute named 'access_token_2' is defined 1 ( 0.01%) > UI AM endsession: No attribute named 'id_token_2' is defined 1 ( 0.01%) ---- getToken ------------------------------------------------------------------ [--------------------------------------------------------------------------] 0% waiting: 0 / active: 1 / done: 0 ---- idc.benchmark.Mix2024 ----------------------------------------------------- [#########################################################################-] 99% waiting: 281 / active: 1 / done: 35664 ================================================================================ ================================================================================ 2025-09-16 17:14:58 GMT 3600s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=272512 KO=689 ) > authAsPolicyAdmin (OK=0 KO=2 ) > restLoginInitiate (OK=22927 KO=0 ) > userRestlogin (OK=10821 KO=682 ) > restLoginUsernamePasswordCallback (OK=7606 KO=1 ) > UI-Post-Login - stage 1 (OK=7607 KO=0 ) > UI-Post-Login - stage 1 Redirect 1 (OK=7607 KO=0 ) > UI AM idFromSession (OK=7606 KO=1 ) > UI AM validateGoto (OK=7607 KO=0 ) > User Authorize PKCE stage (OK=15212 KO=0 ) > User AccessToken PKCE stage (OK=15212 KO=0 ) > UI-Post-Login - stage 2 (OK=7607 KO=0 ) > UI-Post-Login - stage 2 Redirect 1 (OK=7607 KO=0 ) > UI Enduser appAuthHelper1 (OK=7607 KO=0 ) > UI Enduser appAuthHelper2 (OK=7607 KO=0 ) > UI-Post-Login - stage 3 (OK=7606 KO=0 ) > UI-Post-Login - stage 3 Redirect 1 (OK=7606 KO=0 ) > UI IDM uiconfig (OK=7606 KO=0 ) > UI Enduser sessioncheck (OK=7606 KO=0 ) > UI IDM version (OK=7606 KO=0 ) > UI IDM login (OK=7604 KO=2 ) > UI IDM features (OK=7606 KO=0 ) > UI IDM privilege (OK=7606 KO=0 ) > UI IDM dashboard (OK=7606 KO=0 ) > UI IDM managed user schema (OK=7606 KO=0 ) > UI IDM user (OK=7606 KO=0 ) > UI IDM oidcToken (OK=7605 KO=1 ) > userinfo (OK=7605 KO=0 ) > UI-Logout (OK=7606 KO=0 ) > UI-Logout Redirect 1 (OK=7606 KO=0 ) > UI AM endsession (OK=7605 KO=0 ) > UI AM revoke1 (OK=7605 KO=0 ) > UI AM revoke2 (OK=7605 KO=0 ) > loginTreeWorkflow_restLoginInitiate (OK=1235 KO=0 ) > loginTreeWorkflow_restLoginUsernamePasswordCallback (OK=1235 KO=0 ) > loginTreeWorkflow_restloginProgressiveProfileCallback (OK=72 KO=0 ) > restloginProgressiveProfileCallback (OK=436 KO=0 ) ---- Errors -------------------------------------------------------------------- > EvalPolicy: No attribute named 'tokenIdCookieName' is defined 11503 (94.32%) > jsonPath($.tokenId).find.exists, found nothing 681 ( 5.58%) > status.find.in([200, 209], 304), found 401 3 ( 0.02%) > status.find.in([200, 209], 304), found 502 3 ( 0.02%) > status.find.is(200), but actually found 502 1 ( 0.01%) > headerRegex((location,#id_token=(.*))).find.exists, found noth 1 ( 0.01%) ing > userinfo: No attribute named 'access_token' is defined 1 ( 0.01%) > UI AM revoke1: No attribute named 'access_token' is defined 1 ( 0.01%) > UI AM revoke2: No attribute named 'access_token_2' is defined 1 ( 0.01%) > UI AM endsession: No attribute named 'id_token_2' is defined 1 ( 0.01%) ---- getToken ------------------------------------------------------------------ [--------------------------------------------------------------------------] 0% waiting: 0 / active: 1 / done: 0 ---- idc.benchmark.Mix2024 ----------------------------------------------------- [#########################################################################-] 99% waiting: 281 / active: 1 / done: 35664 ================================================================================ 2025-09-16 18:14:58,908 INFO Gatling simulation run ended successfully 2025-09-16 18:14:58,909 DEBUG No archive name provided. No archive will be created ________________________________________________________________________________ [2025-09-16 17:15:17] benchmark-10 show_stderr : Display stderr order ________________________________________________________________________________ 2025-09-16 17:15:17,912 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/01_benchmark-10/stderr.txt is small (0 lines), printing all lines: 2025-09-16 17:15:17,912 INFO ----- output ----- ________________________________________________________________________________ [2025-09-16 17:15:17] benchmark-10 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2025-09-16 17:15:18] benchmark-10 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2025-09-16 17:15:18] benchmark-10 post : Post method ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped