--Task-- name: benchmark-500 enabled: True class_name: GatlingTask source_name: overseer-0 source_namespace: >default< target_name: am target_namespace: >default< start: 0.0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates below loop: False interval: None dependencies: [] wait_for: ['benchmark-400'] preceding_task: benchmark-400 options: {'duration': 3600, 'java-opts': '-Drealm=alpha -Duser_password=Pa_ssw0rd -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html', 'testCase': 'idc.benchmark.Mix2024', 'testMode': 'open', 'throughput': 500} group_name: None Current dir: /mnt/disk1/xslou/workshop/lodestar-fork/pyrock 2025-06-14 02:06:26,466 INFO 2025-06-14 02:06:26,466 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 02:06:26,466 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 02:06:26,743 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 02:06:26,743 INFO [loop_until]: OK (rc = 0) 2025-06-14 02:06:26,743 DEBUG --- stdout --- 2025-06-14 02:06:26,743 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 02:06:26,743 DEBUG --- stderr --- 2025-06-14 02:06:26,743 DEBUG 2025-06-14 02:06:26,743 INFO Lodestar assumed that full stack is deployed so will set all products host URLs in Gatling system properties 2025-06-14 02:06:26 - INFO: Timeout for this Task has changed to 1h 12m (4320 seconds) 2025-06-14 02:06:26 - INFO: timeout was calculated based on simulation duration parameter 2025-06-14 02:06:26 - INFO: Interval for this Task has changed to 2m (120 seconds) 2025-06-14 02:06:26 - INFO: interval was set to Task default because it was unset, based on self.timeout value because task is allowed to stop by itself Pod log file : /results/orders/benchmark-500/stdout.txt Pod log : /results/orders/benchmark-500 ________________________________________________________________________________ [2025-06-14 02:06:26] benchmark-500 print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@tof-10" --results-folder="/results/gatling/benchmark-500" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=500 -Duser_password=**** ________________________________________________________________________________ [2025-06-14 02:06:26] benchmark-500 create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file: /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/order.json { "name": "benchmark-500", "command": " /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class=\"com.forgerock.pyrock.Loadtest\" --run-description=\"idc.benchmark.Mix2024@tof-10\" --results-folder=\"/results/gatling/benchmark-500\" --no-reports --simulation-jvm-args=\"-XX:+ExitOnOutOfMemoryError -Xmx4G\" --simulation-system-properties=\"-Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=500 -Duser_password=Pa_ssw0rd\"", "out-dir": "/results/orders/benchmark-500" } ________________________________________________________________________________ [2025-06-14 02:06:26] benchmark-500 loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2025-06-14 02:06:26,751 INFO 2025-06-14 02:06:26,751 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2025-06-14 02:06:26,751 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-06-14 02:06:26,963 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 02:06:26,963 INFO [loop_until]: OK (rc = 0) 2025-06-14 02:06:26,963 DEBUG --- stdout --- 2025-06-14 02:06:26,963 DEBUG WAITING 2025-06-14 02:06:26,963 DEBUG --- stderr --- 2025-06-14 02:06:26,963 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 34 0 --:--:-- --:--:-- --:--:-- 34 ________________________________________________________________________________ [2025-06-14 02:06:26] benchmark-500 start_order : Send the order to the overseer pod ________________________________________________________________________________ 2025-06-14 02:06:26,963 INFO 2025-06-14 02:06:26,963 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 02:06:26,963 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 02:06:27,238 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 02:06:27,238 INFO [loop_until]: OK (rc = 0) 2025-06-14 02:06:27,238 DEBUG --- stdout --- 2025-06-14 02:06:27,238 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 02:06:27,238 DEBUG --- stderr --- 2025-06-14 02:06:27,238 DEBUG 2025-06-14 02:06:27,239 INFO 2025-06-14 02:06:27,239 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/order.json overseer-0-898c87646-bjhnj:/results/orders/order.json 2025-06-14 02:06:27,239 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 02:06:28,625 INFO [loop_until]: OK (rc = 0) 2025-06-14 02:06:28,625 DEBUG --- stdout --- 2025-06-14 02:06:28,625 DEBUG 2025-06-14 02:06:28,625 DEBUG --- stderr --- 2025-06-14 02:06:28,625 DEBUG Order has been successfully forwarded to overseer 2025-06-14 02:06:28 - INFO: Check if Gatling simulation started successfully 2025-06-14 02:06:28,625 INFO 2025-06-14 02:06:28,625 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj -- head --lines 5000 /results/orders/benchmark-500/stdout.txt | grep "Simulation com.forgerock.pyrock.Loadtest started..." 2025-06-14 02:06:28,625 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-06-14 02:06:29,389 INFO [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-06-14 02:06:35,136 INFO [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-06-14 02:06:40,886 INFO [loop_until]: Function succeeded after 12s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-06-14 02:06:46,635 INFO [loop_until]: Function succeeded after 18s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-06-14 02:06:52,379 INFO [loop_until]: Function succeeded after 23s (rc=0) - expected pattern found 2025-06-14 02:06:52,380 INFO [loop_until]: OK (rc = 0) 2025-06-14 02:06:52,380 DEBUG --- stdout --- 2025-06-14 02:06:52,380 DEBUG 2025-06-14 03:06:30,818 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmp1i3xguau 2025-06-14 03:06:30,822 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-500" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=500 -Duser_password=**** Gatling 3.13.5 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... 2025-06-14 02:06:52,380 DEBUG --- stderr --- 2025-06-14 02:06:52,380 DEBUG ________________________________________________________________________________ [2025-06-14 02:06:52] benchmark-500 wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 1h 12m. Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING Order is running since 2m, retry Order is running since 4m, retry Order is running since 6m, retry Order is running since 8m 01s, retry Order is running since 10m 01s, retry Order is running since 12m 01s, retry Order is running since 14m 02s, retry Order is running since 16m 02s, retry Order is running since 18m 02s, retry Order is running since 20m 02s, retry Order is running since 22m 03s, retry Order is running since 24m 03s, retry Order is running since 26m 03s, retry Order is running since 28m 04s, retry Order is running since 30m 04s, retry Order is running since 32m 04s, retry Order is running since 34m 04s, retry Order is running since 36m 05s, retry Order is running since 38m 05s, retry Order is running since 40m 05s, retry Order is running since 42m 06s, retry Order is running since 44m 06s, retry Order is running since 46m 06s, retry Order is running since 48m 07s, retry Order is running since 50m 07s, retry Order is running since 52m 07s, retry Order is running since 54m 07s, retry Order is running since 56m 08s, retry Order is running since 58m 08s, retry PASS : Order stopped by itself after 1h and before timeout (1h 12m) ________________________________________________________________________________ [2025-06-14 03:07:01] benchmark-500 stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2025-06-14 03:07:01,163 INFO 2025-06-14 03:07:01,163 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 03:07:01,163 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 03:07:01,465 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 03:07:01,466 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:01,466 DEBUG --- stdout --- 2025-06-14 03:07:01,466 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 03:07:01,466 DEBUG --- stderr --- 2025-06-14 03:07:01,466 DEBUG 2025-06-14 03:07:01,466 INFO 2025-06-14 03:07:01,466 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj -- rm -f /results/orders/order.json 2025-06-14 03:07:01,466 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 03:07:02,259 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:02,260 DEBUG --- stdout --- 2025-06-14 03:07:02,260 DEBUG 2025-06-14 03:07:02,260 DEBUG --- stderr --- 2025-06-14 03:07:02,260 DEBUG Order ran during 1h ________________________________________________________________________________ [2025-06-14 03:07:02] benchmark-500 download_logs : Download individual order files ________________________________________________________________________________ 2025-06-14 03:07:02 - PASS: file already downloaded /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/order.json 2025-06-14 03:07:02,260 INFO 2025-06-14 03:07:02,261 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 03:07:02,261 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 03:07:02,542 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 03:07:02,542 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:02,542 DEBUG --- stdout --- 2025-06-14 03:07:02,542 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 03:07:02,542 DEBUG --- stderr --- 2025-06-14 03:07:02,542 DEBUG Checking if file exists on pod 'overseer-0-898c87646-bjhnj', wait max 3 minutes if not. 2025-06-14 03:07:02,543 INFO 2025-06-14 03:07:02,543 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- bash -c 'if [ -f "/results/orders/benchmark-500/rc.txt" ]; then echo "/results/orders/benchmark-500/rc.txt file found"; exit 0; else echo "/results/orders/benchmark-500/rc.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-500/rc.txt file found" 2025-06-14 03:07:02,543 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 03:07:03,379 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 03:07:03,379 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:03,379 DEBUG --- stdout --- 2025-06-14 03:07:03,379 DEBUG /results/orders/benchmark-500/rc.txt file found 2025-06-14 03:07:03,379 DEBUG --- stderr --- 2025-06-14 03:07:03,379 DEBUG 2025-06-14 03:07:03,380 INFO 2025-06-14 03:07:03,380 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- du -k /results/orders/benchmark-500/rc.txt | grep "^\d+ .+$" 2025-06-14 03:07:03,380 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-06-14 03:07:04,203 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 03:07:04,203 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:04,203 DEBUG --- stdout --- 2025-06-14 03:07:04,203 DEBUG 4 /results/orders/benchmark-500/rc.txt 2025-06-14 03:07:04,203 DEBUG --- stderr --- 2025-06-14 03:07:04,204 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-500/rc.txt @ overseer-0-898c87646-bjhnj destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/rc.txt 2025-06-14 03:07:04,204 INFO 2025-06-14 03:07:04,204 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-bjhnj:/results/orders/benchmark-500/rc.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/rc.txt 2025-06-14 03:07:04,204 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-06-14 03:07:04,941 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:04,941 DEBUG --- stdout --- 2025-06-14 03:07:04,941 DEBUG tar: Removing leading `/' from member names 2025-06-14 03:07:04,941 DEBUG --- stderr --- 2025-06-14 03:07:04,941 DEBUG 2025-06-14 03:07:04,941 INFO 2025-06-14 03:07:04,941 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 03:07:04,941 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 03:07:05,224 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 03:07:05,224 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:05,224 DEBUG --- stdout --- 2025-06-14 03:07:05,224 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 03:07:05,224 DEBUG --- stderr --- 2025-06-14 03:07:05,224 DEBUG Checking if file exists on pod 'overseer-0-898c87646-bjhnj', wait max 3 minutes if not. 2025-06-14 03:07:05,225 INFO 2025-06-14 03:07:05,225 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- bash -c 'if [ -f "/results/orders/benchmark-500/stderr.txt" ]; then echo "/results/orders/benchmark-500/stderr.txt file found"; exit 0; else echo "/results/orders/benchmark-500/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-500/stderr.txt file found" 2025-06-14 03:07:05,225 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 03:07:06,041 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 03:07:06,041 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:06,041 DEBUG --- stdout --- 2025-06-14 03:07:06,041 DEBUG /results/orders/benchmark-500/stderr.txt file found 2025-06-14 03:07:06,041 DEBUG --- stderr --- 2025-06-14 03:07:06,041 DEBUG 2025-06-14 03:07:06,041 INFO 2025-06-14 03:07:06,041 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- du -k /results/orders/benchmark-500/stderr.txt | grep "^\d+ .+$" 2025-06-14 03:07:06,041 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-06-14 03:07:06,860 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 03:07:06,860 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:06,860 DEBUG --- stdout --- 2025-06-14 03:07:06,860 DEBUG 0 /results/orders/benchmark-500/stderr.txt 2025-06-14 03:07:06,860 DEBUG --- stderr --- 2025-06-14 03:07:06,860 DEBUG - Download file (size 0B) source : /results/orders/benchmark-500/stderr.txt @ overseer-0-898c87646-bjhnj destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/stderr.txt 2025-06-14 03:07:06,861 INFO 2025-06-14 03:07:06,861 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-bjhnj:/results/orders/benchmark-500/stderr.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/stderr.txt 2025-06-14 03:07:06,861 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-06-14 03:07:07,638 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:07,638 DEBUG --- stdout --- 2025-06-14 03:07:07,638 DEBUG tar: Removing leading `/' from member names 2025-06-14 03:07:07,638 DEBUG --- stderr --- 2025-06-14 03:07:07,638 DEBUG 2025-06-14 03:07:07,639 INFO 2025-06-14 03:07:07,639 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 03:07:07,639 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 03:07:07,919 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 03:07:07,919 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:07,919 DEBUG --- stdout --- 2025-06-14 03:07:07,919 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 03:07:07,919 DEBUG --- stderr --- 2025-06-14 03:07:07,919 DEBUG Checking if file exists on pod 'overseer-0-898c87646-bjhnj', wait max 3 minutes if not. 2025-06-14 03:07:07,921 INFO 2025-06-14 03:07:07,921 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- bash -c 'if [ -f "/results/orders/benchmark-500/stdout.txt" ]; then echo "/results/orders/benchmark-500/stdout.txt file found"; exit 0; else echo "/results/orders/benchmark-500/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-500/stdout.txt file found" 2025-06-14 03:07:07,921 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 03:07:08,743 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 03:07:08,743 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:08,743 DEBUG --- stdout --- 2025-06-14 03:07:08,743 DEBUG /results/orders/benchmark-500/stdout.txt file found 2025-06-14 03:07:08,743 DEBUG --- stderr --- 2025-06-14 03:07:08,743 DEBUG 2025-06-14 03:07:08,744 INFO 2025-06-14 03:07:08,744 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- du -k /results/orders/benchmark-500/stdout.txt | grep "^\d+ .+$" 2025-06-14 03:07:08,744 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-06-14 03:07:09,569 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 03:07:09,569 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:09,569 DEBUG --- stdout --- 2025-06-14 03:07:09,569 DEBUG 47560 /results/orders/benchmark-500/stdout.txt 2025-06-14 03:07:09,569 DEBUG --- stderr --- 2025-06-14 03:07:09,569 DEBUG - Download file (size 47.56MB) source : /results/orders/benchmark-500/stdout.txt @ overseer-0-898c87646-bjhnj destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/stdout.txt 2025-06-14 03:07:09,570 INFO 2025-06-14 03:07:09,570 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-bjhnj:/results/orders/benchmark-500/stdout.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/stdout.txt 2025-06-14 03:07:09,570 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-06-14 03:07:12,015 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:12,015 DEBUG --- stdout --- 2025-06-14 03:07:12,015 DEBUG tar: Removing leading `/' from member names 2025-06-14 03:07:12,015 DEBUG --- stderr --- 2025-06-14 03:07:12,015 DEBUG ________________________________________________________________________________ [2025-06-14 03:07:12] benchmark-500 show_cmd : Display order command ________________________________________________________________________________ 2025-06-14 03:07:12,015 INFO 2025-06-14 03:07:12,015 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-06-14 03:07:12,015 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-06-14 03:07:12,296 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-06-14 03:07:12,296 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:12,296 DEBUG --- stdout --- 2025-06-14 03:07:12,297 DEBUG overseer-0-898c87646-bjhnj 2025-06-14 03:07:12,297 DEBUG --- stderr --- 2025-06-14 03:07:12,297 DEBUG Checking if file exists on pod 'overseer-0-898c87646-bjhnj', wait max 3 minutes if not. 2025-06-14 03:07:12,297 INFO 2025-06-14 03:07:12,297 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- bash -c 'if [ -f "/results/orders/benchmark-500/cmd.txt" ]; then echo "/results/orders/benchmark-500/cmd.txt file found"; exit 0; else echo "/results/orders/benchmark-500/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/benchmark-500/cmd.txt file found" 2025-06-14 03:07:12,298 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-06-14 03:07:13,175 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 03:07:13,176 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:13,176 DEBUG --- stdout --- 2025-06-14 03:07:13,176 DEBUG /results/orders/benchmark-500/cmd.txt file found 2025-06-14 03:07:13,176 DEBUG --- stderr --- 2025-06-14 03:07:13,176 DEBUG 2025-06-14 03:07:13,176 INFO 2025-06-14 03:07:13,177 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-bjhnj --stdin -- du -k /results/orders/benchmark-500/cmd.txt | grep "^\d+ .+$" 2025-06-14 03:07:13,177 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-06-14 03:07:14,078 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-06-14 03:07:14,078 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:14,079 DEBUG --- stdout --- 2025-06-14 03:07:14,079 DEBUG 4 /results/orders/benchmark-500/cmd.txt 2025-06-14 03:07:14,079 DEBUG --- stderr --- 2025-06-14 03:07:14,079 DEBUG - Download file (size 4kB) source : /results/orders/benchmark-500/cmd.txt @ overseer-0-898c87646-bjhnj destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/cmd.txt 2025-06-14 03:07:14,079 INFO 2025-06-14 03:07:14,079 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-bjhnj:/results/orders/benchmark-500/cmd.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/cmd.txt 2025-06-14 03:07:14,079 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-06-14 03:07:14,836 INFO [loop_until]: OK (rc = 0) 2025-06-14 03:07:14,836 DEBUG --- stdout --- 2025-06-14 03:07:14,836 DEBUG tar: Removing leading `/' from member names 2025-06-14 03:07:14,836 DEBUG --- stderr --- 2025-06-14 03:07:14,836 DEBUG 2025-06-14 03:07:14,836 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/cmd.txt is small (1 lines), printing all lines: 2025-06-14 03:07:14,836 INFO ----- output ----- /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idc.benchmark.Mix2024@tof-10" --results-folder="/results/gatling/benchmark-500" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=M@ythe4th -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=500 -Duser_password=Pa_ssw0rd" ________________________________________________________________________________ [2025-06-14 03:07:14] benchmark-500 show_rc : Display rc order ________________________________________________________________________________ 2025-06-14 03:07:14,837 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/rc.txt is small (1 lines), printing all lines: 2025-06-14 03:07:14,837 INFO ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [2025-06-14 03:07:14] benchmark-500 show_stdout : Display stdout order ________________________________________________________________________________ 2025-06-14 03:07:14,921 DEBUG [print_head_tail]: Print head (100 lines) and tail (100 lines) of input file (/mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/stdout.txt is 994693 lines): 2025-06-14 03:07:15,045 INFO ----- output ----- 2025-06-14 03:06:30,818 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmp1i3xguau 2025-06-14 03:06:30,822 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/benchmark-500" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-tof-10.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dduration=3600 -Didc=true -Didm_host=openam-tof-10.forgeblocks.com -Dlogin_cookie=7573b68ed0f3690 -Dnum_users=10000000 -Doauth2_client_id=endUserUIClient -Doauth2_redirect_uri=/enduser/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idc.benchmark.Mix2024 -DtestMode=open -Dthroughput=500 -Duser_password=**** Gatling 3.13.5 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Simulation com.forgerock.pyrock.Loadtest started... ================================================================================ 2025-06-14 02:06:54 GMT 5s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=1598 KO=0 ) > authAsPolicyAdmin (OK=1 KO=0 ) > restLoginInitiate (OK=146 KO=0 ) > restLoginUsernamePasswordCallback (OK=137 KO=0 ) > userRestlogin (OK=49 KO=0 ) > UI-Post-Login - stage 1 (OK=42 KO=0 ) > EvalPolicy (OK=47 KO=0 ) > UI-Post-Login - stage 1 Redirect 1 (OK=41 KO=0 ) > Skip 2FA (OK=45 KO=0 ) > UI AM idFromSession (OK=41 KO=0 ) > UI AM validateGoto (OK=40 KO=0 ) > User Authorize PKCE stage (OK=153 KO=0 ) > User AccessToken PKCE stage (OK=146 KO=0 ) > RootUserinfo (OK=57 KO=0 ) > UI-Post-Login - stage 2 (OK=35 KO=0 ) > tokenUserinfo (OK=42 KO=0 ) > UI-Post-Login - stage 2 Redirect 1 (OK=33 KO=0 ) > UI Enduser appAuthHelper1 (OK=32 KO=0 ) > UI Enduser appAuthHelper2 (OK=32 KO=0 ) > UI-Post-Login - stage 3 (OK=28 KO=0 ) > loginTreeWorkflow_restLoginInitiate (OK=5 KO=0 ) > UI-Post-Login - stage 3 Redirect 1 (OK=27 KO=0 ) > UI IDM uiconfig (OK=26 KO=0 ) > UI Enduser sessioncheck (OK=26 KO=0 ) > UI IDM version (OK=26 KO=0 ) > loginTreeWorkflow_restLoginUsernamePasswordCallback (OK=5 KO=0 ) > UI IDM login (OK=26 KO=0 ) > UI IDM features (OK=26 KO=0 ) > UI IDM privilege (OK=26 KO=0 ) > UI IDM dashboard (OK=25 KO=0 ) > UI IDM user (OK=25 KO=0 ) > UI IDM managed user schema (OK=24 KO=0 ) > UI IDM oidcToken (OK=22 KO=0 ) > userinfo (OK=20 KO=0 ) > UI-Logout (OK=19 KO=0 ) > getIdToken (OK=10 KO=0 ) > UI-Logout Redirect 1 (OK=19 KO=0 ) > getIdToken Redirect 1 (OK=10 KO=0 ) > UI AM revoke1 (OK=18 KO=0 ) > UI AM revoke2 (OK=18 KO=0 ) > UI AM endsession (OK=18 KO=0 ) > createOrganization (OK=10 KO=0 ) > queryOrganization (OK=10 KO=0 ) > deleteOrganization (OK=10 KO=0 ) ---- getToken ------------------------------------------------------------------ [--------------------------------------------------------------------------] 0% waiting: 0 / active: 1 / done: 0 ---- idc.benchmark.Mix2024 ----------------------------------------------------- [- ] 0% waiting: 1809895 / active: 86 / done: 122 ================================================================================ ================================================================================ 2025-06-14 02:06:59 GMT 10s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=7478 KO=0 ) > authAsPolicyAdmin (OK=1 KO=0 ) > restLoginInitiate (OK=539 KO=0 ) > restLoginUsernamePasswordCallback (OK=529 KO=0 ) > userRestlogin (OK=252 KO=0 ) > UI-Post-Login - stage 1 (OK=183 KO=0 ) > EvalPolicy (OK=246 KO=0 ) > UI-Post-Login - stage 1 Redirect 1 (OK=181 KO=0 ) > Skip 2FA (OK=171 KO=0 ) > UI AM idFromSession (OK=176 KO=0 ) > UI AM validateGoto (OK=174 KO=0 ) > User Authorize PKCE stage (OK=655 KO=0 ) > User AccessToken PKCE stage (OK=638 KO=0 ) > RootUserinfo (OK=348 KO=0 ) > UI-Post-Login - stage 2 (OK=160 KO=0 ) > tokenUserinfo (OK=160 KO=0 ) > UI-Post-Login - stage 2 Redirect 1 (OK=155 KO=0 ) > UI Enduser appAuthHelper1 (OK=152 KO=0 ) > UI Enduser appAuthHelper2 (OK=152 KO=0 ) > UI-Post-Login - stage 3 (OK=142 KO=0 ) > loginTreeWorkflow_restLoginInitiate (OK=21 KO=0 ) > UI-Post-Login - stage 3 Redirect 1 (OK=138 KO=0 ) > UI IDM uiconfig (OK=133 KO=0 ) > UI Enduser sessioncheck (OK=133 KO=0 ) > UI IDM version (OK=130 KO=0 ) > loginTreeWorkflow_restLoginUsernamePasswordCallback (OK=20 KO=0 ) > UI IDM login (OK=130 KO=0 ) > UI IDM features (OK=130 KO=0 ) > UI IDM privilege (OK=130 KO=0 ) > UI IDM dashboard (OK=129 KO=0 ) > UI IDM user (OK=129 KO=0 ) > UI IDM managed user schema (OK=128 KO=0 ) > UI IDM oidcToken (OK=126 KO=0 ) > userinfo (OK=123 KO=0 ) [...] >>>>>>>>>>>>>>>>>>>>>>>>>> Request: UI IDM features: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1794449,HashMap(access_token -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJmMzE4NjhjNS0wYTNkLTQ0ODAtODEzMy00MjZiYmViYzBlMDEiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiOTVkNWVkZWItYmRlZC00NGY3LWFlMjAtNGY2OWI1OTk2MmJlLTE0OTgwNzQ5MyIsInN1Ym5hbWUiOiJmMzE4NjhjNS0wYTNkLTQ0ODAtODEzMy00MjZiYmViYzBlMDEiLCJpc3MiOiJodHRwczovL29wZW5hbS10b2YtMTAuZm9yZ2VibG9ja3MuY29tOjQ0My9hbS9vYXV0aDIvcmVhbG1zL3Jvb3QvcmVhbG1zL2FscGhhIiwidG9rZW5OYW1lIjoiYWNjZXNzX3Rva2VuIiwidG9rZW5fdHlwZSI6IkJlYXJlciIsImF1dGhHcmFudElkIjoiU2NGZzNtQ2E5R1N2VWozUHQtZmVBTzFtRjRJIiwibm9uY2UiOiIxMjM0IiwiY2xpZW50X2lkIjoiZW5kVXNlclVJQ2xpZW50IiwiYXVkIjoiZW5kVXNlclVJQ2xpZW50IiwibmJmIjoxNzQ5ODcwNDA4LCJncmFudF90eXBlIjoiYXV0aG9yaXphdGlvbl9jb2RlIiwic2NvcGUiOlsib3BlbmlkIiwiZnI6aWRtOioiXSwiYXV0aF90aW1lIjoxNzQ5ODcwNDA4LCJyZWFsbSI6Ii9hbHBoYSIsImV4cCI6MTc0OTg3NDAwOCwiaWF0IjoxNzQ5ODcwNDA4LCJleHBpcmVzX2luIjozNjAwLCJqdGkiOiJhT1ZOMGxjNWdEVlJ0NjVxb1hfQUJkYU42bVUifQ.yMQvXVKi5Ay8nCkFCOHCInBuhf3X_5gB0XC-sHXuITg, username -> user.3463910, tokenIdCookieName -> amlbcookie, policyEvalTokenId -> , access_token_2 -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJmMzE4NjhjNS0wYTNkLTQ0ODAtODEzMy00MjZiYmViYzBlMDEiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiZmY3MTA1ZDYtMmNiMS00ZTE3LWIyYmItZDkyY2YyMDkzMDU4LTE0ODA1OTAxNSIsInN1Ym5hbWUiOiJmMzE4NjhjNS0wYTNkLTQ0ODAtODEzMy00MjZiYmViYzBlMDEiLCJpc3MiOiJodHRwczovL29wZW5hbS10b2YtMTAuZm9yZ2VibG9ja3MuY29tOjQ0My9hbS9vYXV0aDIvcmVhbG1zL3Jvb3QvcmVhbG1zL2FscGhhIiwidG9rZW5OYW1lIjoiYWNjZXNzX3Rva2VuIiwidG9rZW5fdHlwZSI6IkJlYXJlciIsImF1dGhHcmFudElkIjoiVUVkaFFzWjNtbW4zRFpVT2RlZ1VjbUE5NWdFIiwibm9uY2UiOiIxMjM0IiwiY2xpZW50X2lkIjoiZW5kVXNlclVJQ2xpZW50IiwiYXVkIjoiZW5kVXNlclVJQ2xpZW50IiwibmJmIjoxNzQ5ODcwNDA5LCJncmFudF90eXBlIjoiYXV0aG9yaXphdGlvbl9jb2RlIiwic2NvcGUiOlsiZnI6aWRtOioiXSwiYXV0aF90aW1lIjoxNzQ5ODcwNDA4LCJyZWFsbSI6Ii9hbHBoYSIsImV4cCI6MTc0OTg3NDAwOSwiaWF0IjoxNzQ5ODcwNDA5LCJleHBpcmVzX2luIjozNjAwLCJqdGkiOiJMV1EwZmRnZ1pvWlJ4a01ad25kTEJLR1B3cUUifQ.yQSixXPFKsMjqwspktTtnmRlBeoNv7-AeHrfEwpsTy4, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@75600a27, gatling.http.referer -> https://openam-tof-10.forgeblocks.com/am/, authcode -> tyf2l5YNy9IgJOdkPUscEVvlGls, callbacks -> {"tokenId":"XHw3YmHQ9QSEeSwgyXVZuyo4W3k.*AAJTSQACMDIAAlNLABx1L2pzMTVZWlhGRjA0b2NhMERMdUxWUnlVVG89AAR0eXBlAANDVFMAAlMxAAIwMQ..*","successUrl":"/enduser/?realm=/alpha","realm":"/alpha"}, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-tof-10.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1749870408178), CookieKey(7573b68ed0f3690,openam-tof-10.forgeblocks.com,/) -> StoredCookie(7573b68ed0f3690=XHw3YmHQ9QSEeSwgyXVZuyo4W3k.*AAJTSQACMDIAAlNLABx1L2pzMTVZWlhGRjA0b2NhMERMdUxWUnlVVG89AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1749870408178))), user_uuid -> f31868c5-0a3d-4480-8133-426bbebc0e01, password -> Pa_ssw0rd, challenge -> i8cn1R2c3FveEuzKs0vp-iZuGMSDvpZJ_V-pxrSyzjw, gatling.http.cache.baseUrl -> https://openam-tof-10.forgeblocks.com:443, fake-ip -> 122.246.39.207, verifier -> NzY4NDgzMDY0Mzk2OTkwMjk5Mw, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@2053958, gatling.http.cache.contentCache -> io.gatling.core.util.cache.Cache@7c34d304, tokenId -> XHw3YmHQ9QSEeSwgyXVZuyo4W3k.*AAJTSQACMDIAAlNLABx1L2pzMTVZWlhGRjA0b2NhMERMdUxWUnlVVG89AAR0eXBlAANDVFMAAlMxAAIwMQ..*, fake-user-agent -> Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$821/0x000078b34c6983d0@12ffd192,io.netty.channel.epoll.EpollEventLoop@f79a760) ========================= HTTP request: GET https://openam-tof-10.forgeblocks.com/openidm/info/features?_queryFilter=true headers: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 referer: https://openam-tof-10.forgeblocks.com/am/ Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJmMzE4NjhjNS0wYTNkLTQ0ODAtODEzMy00MjZiYmViYzBlMDEiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiZmY3MTA1ZDYtMmNiMS00ZTE3LWIyYmItZDkyY2YyMDkzMDU4LTE0ODA1OTAxNSIsInN1Ym5hbWUiOiJmMzE4NjhjNS0wYTNkLTQ0ODAtODEzMy00MjZiYmViYzBlMDEiLCJpc3MiOiJodHRwczovL29wZW5hbS10b2YtMTAuZm9yZ2VibG9ja3MuY29tOjQ0My9hbS9vYXV0aDIvcmVhbG1zL3Jvb3QvcmVhbG1zL2FscGhhIiwidG9rZW5OYW1lIjoiYWNjZXNzX3Rva2VuIiwidG9rZW5fdHlwZSI6IkJlYXJlciIsImF1dGhHcmFudElkIjoiVUVkaFFzWjNtbW4zRFpVT2RlZ1VjbUE5NWdFIiwibm9uY2UiOiIxMjM0IiwiY2xpZW50X2lkIjoiZW5kVXNlclVJQ2xpZW50IiwiYXVkIjoiZW5kVXNlclVJQ2xpZW50IiwibmJmIjoxNzQ5ODcwNDA5LCJncmFudF90eXBlIjoiYXV0aG9yaXphdGlvbl9jb2RlIiwic2NvcGUiOlsiZnI6aWRtOioiXSwiYXV0aF90aW1lIjoxNzQ5ODcwNDA4LCJyZWFsbSI6Ii9hbHBoYSIsImV4cCI6MTc0OTg3NDAwOSwiaWF0IjoxNzQ5ODcwNDA5LCJleHBpcmVzX2luIjozNjAwLCJqdGkiOiJMV1EwZmRnZ1pvWlJ4a01ad25kTEJLR1B3cUUifQ.yQSixXPFKsMjqwspktTtnmRlBeoNv7-AeHrfEwpsTy4 Accept-Api-Version: accept: */* cookie: amlbcookie=01; 7573b68ed0f3690=XHw3YmHQ9QSEeSwgyXVZuyo4W3k.*AAJTSQACMDIAAlNLABx1L2pzMTVZWlhGRjA0b2NhMERMdUxWUnlVVG89AAR0eXBlAANDVFMAAlMxAAIwMQ..* host: openam-tof-10.forgeblocks.com cookies: amlbcookie=01, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None 7573b68ed0f3690=XHw3YmHQ9QSEeSwgyXVZuyo4W3k.*AAJTSQACMDIAAlNLABx1L2pzMTVZWlhGRjA0b2NhMERMdUxWUnlVVG89AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 04:06:49.375 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'userRestlogin' failed for user 1795060: j.i.IOException: Premature close 04:06:49.375 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: userRestlogin: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1795060,HashMap(gatling.http.cache.baseUrl -> https://openam-tof-10.forgeblocks.com:443, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@15b3faa8, username -> user.4401422, tokenIdCookieName -> amlbcookie, policyEvalTokenId -> , gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@60a4a426, password -> Pa_ssw0rd),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$821/0x000078b34c6983d0@12ffd192,io.netty.channel.epoll.EpollEventLoop@239b0f9d) ========================= HTTP request: POST https://openam-tof-10.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: Content-Type: apppliation/json Accept-API-Version: resource=2.0, protocol=1.0 user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 X-OpenAM-Username: user.4401422 X-OpenAM-Password: Pa_ssw0rd accept: */* host: openam-tof-10.forgeblocks.com content-length: 0 ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 04:06:49.376 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'userTokenId' is defined 04:06:49.376 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request ' UI IDM privilege' failed for user 1794428: j.i.IOException: Premature close 04:06:49.376 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - Request 'userRestlogin' failed for user 1795108: j.i.IOException: Premature close 04:06:49.376 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: UI IDM privilege: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1794428,HashMap(access_token -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI4Yzk3Njk4ZC0wODEwLTQ1NjEtYTZkMy0wN2Y4ODE1MzI5NzMiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiMWM0YWQwZmEtOTlmNS00MjZiLTgzNDQtNmYxNzg5NTU2ZTUzLTE0OTA4OTMwOSIsInN1Ym5hbWUiOiI4Yzk3Njk4ZC0wODEwLTQ1NjEtYTZkMy0wN2Y4ODE1MzI5NzMiLCJpc3MiOiJodHRwczovL29wZW5hbS10b2YtMTAuZm9yZ2VibG9ja3MuY29tOjQ0My9hbS9vYXV0aDIvcmVhbG1zL3Jvb3QvcmVhbG1zL2FscGhhIiwidG9rZW5OYW1lIjoiYWNjZXNzX3Rva2VuIiwidG9rZW5fdHlwZSI6IkJlYXJlciIsImF1dGhHcmFudElkIjoiU2N6R200ZHU3NGZ4UUxtU3BOVDRidFlFMmJzIiwibm9uY2UiOiIxMjM0IiwiY2xpZW50X2lkIjoiZW5kVXNlclVJQ2xpZW50IiwiYXVkIjoiZW5kVXNlclVJQ2xpZW50IiwibmJmIjoxNzQ5ODcwNDA4LCJncmFudF90eXBlIjoiYXV0aG9yaXphdGlvbl9jb2RlIiwic2NvcGUiOlsib3BlbmlkIiwiZnI6aWRtOioiXSwiYXV0aF90aW1lIjoxNzQ5ODcwNDA4LCJyZWFsbSI6Ii9hbHBoYSIsImV4cCI6MTc0OTg3NDAwOCwiaWF0IjoxNzQ5ODcwNDA4LCJleHBpcmVzX2luIjozNjAwLCJqdGkiOiJkUVhvWmlxekxJWEhyMjVmU3ZGenVmekNPVG8ifQ.lbZw04IA8pezeawYV-WFrkspu527d3KuH7GQ9VSAtls, username -> user.2060249, tokenIdCookieName -> amlbcookie, policyEvalTokenId -> , access_token_2 -> eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI4Yzk3Njk4ZC0wODEwLTQ1NjEtYTZkMy0wN2Y4ODE1MzI5NzMiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiOTVkNWVkZWItYmRlZC00NGY3LWFlMjAtNGY2OWI1OTk2MmJlLTE0OTgxMjM5MyIsInN1Ym5hbWUiOiI4Yzk3Njk4ZC0wODEwLTQ1NjEtYTZkMy0wN2Y4ODE1MzI5NzMiLCJpc3MiOiJodHRwczovL29wZW5hbS10b2YtMTAuZm9yZ2VibG9ja3MuY29tOjQ0My9hbS9vYXV0aDIvcmVhbG1zL3Jvb3QvcmVhbG1zL2FscGhhIiwidG9rZW5OYW1lIjoiYWNjZXNzX3Rva2VuIiwidG9rZW5fdHlwZSI6IkJlYXJlciIsImF1dGhHcmFudElkIjoiY0pmQWdOeGdjQUtzSU1UUldxNHVlNnJ6dko0Iiwibm9uY2UiOiIxMjM0IiwiY2xpZW50X2lkIjoiZW5kVXNlclVJQ2xpZW50IiwiYXVkIjoiZW5kVXNlclVJQ2xpZW50IiwibmJmIjoxNzQ5ODcwNDA5LCJncmFudF90eXBlIjoiYXV0aG9yaXphdGlvbl9jb2RlIiwic2NvcGUiOlsiZnI6aWRtOioiXSwiYXV0aF90aW1lIjoxNzQ5ODcwNDA4LCJyZWFsbSI6Ii9hbHBoYSIsImV4cCI6MTc0OTg3NDAwOSwiaWF0IjoxNzQ5ODcwNDA5LCJleHBpcmVzX2luIjozNjAwLCJqdGkiOiJVYWc2d1JOQkpCdnlRSEticTZRMWxzR0tMTncifQ.sMwUk4GBYa8P1-dcsPcEAYk65_unRpHPNd5RN6tSFY4, gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@7bf3cf63, gatling.http.referer -> https://openam-tof-10.forgeblocks.com/am/, authcode -> qB8C6EhG-QjhLfa1zEwAN3vPZdg, callbacks -> {"tokenId":"w1EcjibacraEkT4HjOk2mGi8Xek.*AAJTSQACMDIAAlNLABxjQ0c0RkU0VUExb3ZrTG14aFhWOFNhclh3Rjg9AAR0eXBlAANDVFMAAlMxAAIwMQ..*","successUrl":"/enduser/?realm=/alpha","realm":"/alpha"}, gatling.http.cookies -> CookieJar(Map(CookieKey(amlbcookie,openam-tof-10.forgeblocks.com,/) -> StoredCookie(amlbcookie=01, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1749870408132), CookieKey(7573b68ed0f3690,openam-tof-10.forgeblocks.com,/) -> StoredCookie(7573b68ed0f3690=w1EcjibacraEkT4HjOk2mGi8Xek.*AAJTSQACMDIAAlNLABxjQ0c0RkU0VUExb3ZrTG14aFhWOFNhclh3Rjg9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None,false,false,1749870408132))), user_uuid -> 8c97698d-0810-4561-a6d3-07f881532973, password -> Pa_ssw0rd, challenge -> kb-NaFgOe5ihNaEHsgxPNM_Et7sBFrVrWrZhPWQ7yFY, gatling.http.cache.baseUrl -> https://openam-tof-10.forgeblocks.com:443, fake-ip -> 42.42.42.42, verifier -> MzgyMTYzNDg0Nzc2MzgwMTYyNQ, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@4e08a54f, gatling.http.cache.contentCache -> io.gatling.core.util.cache.Cache@d63a284, tokenId -> w1EcjibacraEkT4HjOk2mGi8Xek.*AAJTSQACMDIAAlNLABxjQ0c0RkU0VUExb3ZrTG14aFhWOFNhclh3Rjg9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, fake-user-agent -> Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$821/0x000078b34c6983d0@12ffd192,io.netty.channel.epoll.EpollEventLoop@30f5a68a) ========================= HTTP request: POST https://openam-tof-10.forgeblocks.com/openidm/privilege?_action=listPrivileges headers: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 referer: https://openam-tof-10.forgeblocks.com/am/ Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI4Yzk3Njk4ZC0wODEwLTQ1NjEtYTZkMy0wN2Y4ODE1MzI5NzMiLCJjdHMiOiJPQVVUSDJfU1RBVEVMRVNTX0dSQU5UIiwiYXV0aF9sZXZlbCI6MCwiYXVkaXRUcmFja2luZ0lkIjoiOTVkNWVkZWItYmRlZC00NGY3LWFlMjAtNGY2OWI1OTk2MmJlLTE0OTgxMjM5MyIsInN1Ym5hbWUiOiI4Yzk3Njk4ZC0wODEwLTQ1NjEtYTZkMy0wN2Y4ODE1MzI5NzMiLCJpc3MiOiJodHRwczovL29wZW5hbS10b2YtMTAuZm9yZ2VibG9ja3MuY29tOjQ0My9hbS9vYXV0aDIvcmVhbG1zL3Jvb3QvcmVhbG1zL2FscGhhIiwidG9rZW5OYW1lIjoiYWNjZXNzX3Rva2VuIiwidG9rZW5fdHlwZSI6IkJlYXJlciIsImF1dGhHcmFudElkIjoiY0pmQWdOeGdjQUtzSU1UUldxNHVlNnJ6dko0Iiwibm9uY2UiOiIxMjM0IiwiY2xpZW50X2lkIjoiZW5kVXNlclVJQ2xpZW50IiwiYXVkIjoiZW5kVXNlclVJQ2xpZW50IiwibmJmIjoxNzQ5ODcwNDA5LCJncmFudF90eXBlIjoiYXV0aG9yaXphdGlvbl9jb2RlIiwic2NvcGUiOlsiZnI6aWRtOioiXSwiYXV0aF90aW1lIjoxNzQ5ODcwNDA4LCJyZWFsbSI6Ii9hbHBoYSIsImV4cCI6MTc0OTg3NDAwOSwiaWF0IjoxNzQ5ODcwNDA5LCJleHBpcmVzX2luIjozNjAwLCJqdGkiOiJVYWc2d1JOQkpCdnlRSEticTZRMWxzR0tMTncifQ.sMwUk4GBYa8P1-dcsPcEAYk65_unRpHPNd5RN6tSFY4 Accept-Api-Version: accept: */* cookie: amlbcookie=01; 7573b68ed0f3690=w1EcjibacraEkT4HjOk2mGi8Xek.*AAJTSQACMDIAAlNLABxjQ0c0RkU0VUExb3ZrTG14aFhWOFNhclh3Rjg9AAR0eXBlAANDVFMAAlMxAAIwMQ..* origin: https://openam-tof-10.forgeblocks.com host: openam-tof-10.forgeblocks.com content-length: 0 cookies: amlbcookie=01, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None 7573b68ed0f3690=w1EcjibacraEkT4HjOk2mGi8Xek.*AAJTSQACMDIAAlNLABxjQ0c0RkU0VUExb3ZrTG14aFhWOFNhclh3Rjg9AAR0eXBlAANDVFMAAlMxAAIwMQ..*, domain=openam-tof-10.forgeblocks.com, path=/, secure, HTTPOnly, SameSite=None ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 04:06:49.376 [DEBUG] i.g.h.e.r.DefaultStatsProcessor - >>>>>>>>>>>>>>>>>>>>>>>>>> Request: userRestlogin: KO j.i.IOException: Premature close ========================= Session: Session(idc.benchmark.Mix2024,1795108,HashMap(gatling.http.cache.baseUrl -> https://openam-tof-10.forgeblocks.com:443, gatling.http.cache.dns -> io.gatling.http.resolver.ShufflingNameResolver@5f2e4978, username -> user.5354161, tokenIdCookieName -> amlbcookie, policyEvalTokenId -> , gatling.http.ssl.sslContexts -> io.gatling.http.util.SslContexts@10b28ca0, password -> Pa_ssw0rd),KO,List(),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$821/0x000078b34c6983d0@12ffd192,io.netty.channel.epoll.EpollEventLoop@239b0f9d) ========================= HTTP request: POST https://openam-tof-10.forgeblocks.com/am/json/realms/root/realms/alpha/authenticate headers: Content-Type: apppliation/json Accept-API-Version: resource=2.0, protocol=1.0 user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 12_5_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 X-OpenAM-Username: user.5354161 X-OpenAM-Password: Pa_ssw0rd accept: */* host: openam-tof-10.forgeblocks.com content-length: 0 ========================= HTTP response: <<<<<<<<<<<<<<<<<<<<<<<<< 04:06:49.376 [ERROR] i.g.h.a.HttpRequestAction - Failed to build request EvalPolicy: No attribute named 'userTokenId' is defined 2025-06-14 04:06:49,525 INFO Gatling simulation run ended successfully 2025-06-14 04:06:49,525 DEBUG No archive name provided. No archive will be created ________________________________________________________________________________ [2025-06-14 03:07:15] benchmark-500 show_stderr : Display stderr order ________________________________________________________________________________ 2025-06-14 03:07:15,045 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_benchmark/07_Scenario_benchmark/06_benchmark-500/stderr.txt is small (0 lines), printing all lines: 2025-06-14 03:07:15,045 INFO ----- output ----- ________________________________________________________________________________ [2025-06-14 03:07:15] benchmark-500 check_rc_and_logs : Check order's outputs ________________________________________________________________________________ CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2025-06-14 03:07:16] benchmark-500 print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2025-06-14 03:07:16] benchmark-500 post : Post method ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped