--Task-- name: cleanup enabled: True class_name: GatlingTask source_name: overseer-0 source_namespace: >default< target_name: idm target_namespace: >default< start: 0.0 stop: None timeout: 1h (3600 seconds) loop: False interval: None dependencies: ['dump-uid'] wait_for: ['Pre-check Num Users'] preceding_task: None options: {'check-before-preload': True, 'testCase': 'idm.ManagedUsersDelete', 'concurrency': 30, 'java-opts': '-Dcsvfile=/results/idm/user_feeder.csv -Drealm=alpha '} group_name: None Current dir: /mnt/disk1/xslou/workshop/lodestar-fork/pyrock 2025-05-15 23:17:27,418 INFO 2025-05-15 23:17:27,418 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-05-15 23:17:27,418 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-05-15 23:17:27,693 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-05-15 23:17:27,693 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:27,693 DEBUG --- stdout --- 2025-05-15 23:17:27,693 DEBUG overseer-0-898c87646-8747d 2025-05-15 23:17:27,693 DEBUG --- stderr --- 2025-05-15 23:17:27,693 DEBUG 2025-05-15 23:17:27,693 INFO Lodestar assumed that full stack is deployed so will set all products host URLs in Gatling system properties 2025-05-15 23:17:27 - INFO: Interval for this Task has changed to 5s (5 seconds) 2025-05-15 23:17:27 - INFO: interval was set to Task default because it was unset, based on self.timeout value because task is allowed to stop by itself Pod log file : /results/orders/cleanup/stdout.txt Pod log : /results/orders/cleanup 2025-05-15 23:17:28 - INFO: Get overseer pod resources requests memory 2025-05-15 23:17:28,304 INFO 2025-05-15 23:17:28,304 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods overseer-0-898c87646-8747d --output jsonpath={.spec.containers[].resources.requests.memory} 2025-05-15 23:17:28,304 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-05-15 23:17:28,574 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:28,574 DEBUG --- stdout --- 2025-05-15 23:17:28,574 DEBUG 4Gi 2025-05-15 23:17:28,574 DEBUG --- stderr --- 2025-05-15 23:17:28,574 DEBUG ________________________________________________________________________________ [2025-05-15 23:17:28] cleanup print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idm.ManagedUsersDelete@perf-10m" --results-folder="/results/gatling/cleanup" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties=" -Dam_host=openam-perf-10m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dcheck_before_preload=True -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-10m.forgeblocks.com -Dlogin_cookie=3398f6a1737d73a -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=**** ________________________________________________________________________________ [2025-05-15 23:17:28] cleanup create_order : Create order for overseer with the command to run ________________________________________________________________________________ Order file: /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/order.json { "name": "cleanup", "command": " /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class=\"com.forgerock.pyrock.Loadtest\" --run-description=\"idm.ManagedUsersDelete@perf-10m\" --results-folder=\"/results/gatling/cleanup\" --no-reports --simulation-jvm-args=\"-XX:+ExitOnOutOfMemoryError -Xmx4G\" --simulation-system-properties=\" -Dam_host=openam-perf-10m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dcheck_before_preload=True -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-10m.forgeblocks.com -Dlogin_cookie=3398f6a1737d73a -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=T35!tr0c-k123\"", "out-dir": "/results/orders/cleanup" } ________________________________________________________________________________ [2025-05-15 23:17:28] cleanup loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2025-05-15 23:17:28,582 INFO 2025-05-15 23:17:28,582 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2025-05-15 23:17:28,582 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-05-15 23:17:28,877 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-05-15 23:17:28,877 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:28,877 DEBUG --- stdout --- 2025-05-15 23:17:28,878 DEBUG WAITING 2025-05-15 23:17:28,878 DEBUG --- stderr --- 2025-05-15 23:17:28,878 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 24 0 --:--:-- --:--:-- --:--:-- 24 ________________________________________________________________________________ [2025-05-15 23:17:28] cleanup start_order : Send the order to the overseer pod ________________________________________________________________________________ 2025-05-15 23:17:28,878 INFO 2025-05-15 23:17:28,878 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-05-15 23:17:28,878 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-05-15 23:17:29,151 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-05-15 23:17:29,152 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:29,152 DEBUG --- stdout --- 2025-05-15 23:17:29,152 DEBUG overseer-0-898c87646-8747d 2025-05-15 23:17:29,152 DEBUG --- stderr --- 2025-05-15 23:17:29,152 DEBUG 2025-05-15 23:17:29,152 INFO 2025-05-15 23:17:29,152 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/order.json overseer-0-898c87646-8747d:/results/orders/order.json 2025-05-15 23:17:29,152 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-05-15 23:17:30,494 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:30,494 DEBUG --- stdout --- 2025-05-15 23:17:30,494 DEBUG 2025-05-15 23:17:30,494 DEBUG --- stderr --- 2025-05-15 23:17:30,494 DEBUG Order has been successfully forwarded to overseer 2025-05-15 23:17:30 - INFO: Check if Gatling simulation started successfully 2025-05-15 23:17:30,494 INFO 2025-05-15 23:17:30,494 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d -- head --lines 5000 /results/orders/cleanup/stdout.txt | grep "Simulation com.forgerock.pyrock.Loadtest started..." 2025-05-15 23:17:30,494 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2025-05-15 23:17:31,212 INFO [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-05-15 23:17:36,918 INFO [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-05-15 23:17:42,655 INFO [loop_until]: Function succeeded after 12s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2025-05-15 23:17:48,432 INFO [loop_until]: Function succeeded after 17s (rc=0) - expected pattern found 2025-05-15 23:17:48,432 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:48,432 DEBUG --- stdout --- 2025-05-15 23:17:48,433 DEBUG 2025-05-16 00:17:32,498 DEBUG Creating results folder /results/gatling/cleanup 2025-05-16 00:17:32,503 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmpwy4p4_lr 2025-05-16 00:17:32,506 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/cleanup" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-10m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dcheck_before_preload=True -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-10m.forgeblocks.com -Dlogin_cookie=3398f6a1737d73a -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=**** Gatling 3.13.5 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Set idVarName to user_id Simulation com.forgerock.pyrock.Loadtest started... ================================================================================ 2025-05-15 23:17:46 GMT 1s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=9 KO=0 ) > restLoginInitiate (OK=1 KO=0 ) > restLoginUsernamePasswordCallback (OK=1 KO=0 ) > Skip 2FA (OK=1 KO=0 ) > User Authorize PKCE stage (OK=1 KO=0 ) > User AccessToken PKCE stage (OK=1 KO=0 ) > DeleteManagedUser (OK=4 KO=0 ) ---- getToken ------------------------------------------------------------------ [##########################################################################]100% waiting: 0 / active: 0 / done: 1 ---- idm.ManagedUsersDelete ---------------------------------------------------- [##########################################################################]100% waiting: 0 / active: 0 / done: 30 ================================================================================ 2025-05-16 00:17:46,811 INFO Gatling simulation run ended successfully 2025-05-16 00:17:46,811 DEBUG No archive name provided. No archive will be created 2025-05-15 23:17:48,433 DEBUG --- stderr --- 2025-05-15 23:17:48,433 DEBUG ________________________________________________________________________________ [2025-05-15 23:17:48] cleanup wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 1h. Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING PASS : Order stopped by itself after 5s and before timeout (1h) ________________________________________________________________________________ [2025-05-15 23:17:53] cleanup stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2025-05-15 23:17:53,679 INFO 2025-05-15 23:17:53,679 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-05-15 23:17:53,679 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-05-15 23:17:53,955 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-05-15 23:17:53,955 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:53,955 DEBUG --- stdout --- 2025-05-15 23:17:53,955 DEBUG overseer-0-898c87646-8747d 2025-05-15 23:17:53,955 DEBUG --- stderr --- 2025-05-15 23:17:53,955 DEBUG 2025-05-15 23:17:53,955 INFO 2025-05-15 23:17:53,955 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d -- rm -f /results/orders/order.json 2025-05-15 23:17:53,955 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-05-15 23:17:54,678 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:54,678 DEBUG --- stdout --- 2025-05-15 23:17:54,678 DEBUG 2025-05-15 23:17:54,678 DEBUG --- stderr --- 2025-05-15 23:17:54,678 DEBUG Order ran during 24s ________________________________________________________________________________ [2025-05-15 23:17:54] cleanup download_logs : Download individual order files ________________________________________________________________________________ 2025-05-15 23:17:54 - PASS: file already downloaded /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/order.json 2025-05-15 23:17:54,679 INFO 2025-05-15 23:17:54,679 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-05-15 23:17:54,679 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-05-15 23:17:54,953 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-05-15 23:17:54,953 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:54,953 DEBUG --- stdout --- 2025-05-15 23:17:54,953 DEBUG overseer-0-898c87646-8747d 2025-05-15 23:17:54,953 DEBUG --- stderr --- 2025-05-15 23:17:54,953 DEBUG Checking if file exists on pod 'overseer-0-898c87646-8747d', wait max 3 minutes if not. 2025-05-15 23:17:54,954 INFO 2025-05-15 23:17:54,954 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d --stdin -- bash -c 'if [ -f "/results/orders/cleanup/rc.txt" ]; then echo "/results/orders/cleanup/rc.txt file found"; exit 0; else echo "/results/orders/cleanup/rc.txt file not found"; exit 1; fi' | grep "/results/orders/cleanup/rc.txt file found" 2025-05-15 23:17:54,954 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-05-15 23:17:55,740 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-05-15 23:17:55,740 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:55,740 DEBUG --- stdout --- 2025-05-15 23:17:55,740 DEBUG /results/orders/cleanup/rc.txt file found 2025-05-15 23:17:55,740 DEBUG --- stderr --- 2025-05-15 23:17:55,740 DEBUG 2025-05-15 23:17:55,740 INFO 2025-05-15 23:17:55,740 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d --stdin -- du -k /results/orders/cleanup/rc.txt | grep "^\d+ .+$" 2025-05-15 23:17:55,740 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-05-15 23:17:56,583 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-05-15 23:17:56,583 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:56,583 DEBUG --- stdout --- 2025-05-15 23:17:56,583 DEBUG 4 /results/orders/cleanup/rc.txt 2025-05-15 23:17:56,583 DEBUG --- stderr --- 2025-05-15 23:17:56,583 DEBUG - Download file (size 4kB) source : /results/orders/cleanup/rc.txt @ overseer-0-898c87646-8747d destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/rc.txt 2025-05-15 23:17:56,583 INFO 2025-05-15 23:17:56,583 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-8747d:/results/orders/cleanup/rc.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/rc.txt 2025-05-15 23:17:56,583 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-05-15 23:17:57,291 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:57,291 DEBUG --- stdout --- 2025-05-15 23:17:57,291 DEBUG tar: Removing leading `/' from member names 2025-05-15 23:17:57,291 DEBUG --- stderr --- 2025-05-15 23:17:57,291 DEBUG 2025-05-15 23:17:57,292 INFO 2025-05-15 23:17:57,292 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-05-15 23:17:57,292 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-05-15 23:17:57,567 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-05-15 23:17:57,567 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:57,568 DEBUG --- stdout --- 2025-05-15 23:17:57,568 DEBUG overseer-0-898c87646-8747d 2025-05-15 23:17:57,568 DEBUG --- stderr --- 2025-05-15 23:17:57,568 DEBUG Checking if file exists on pod 'overseer-0-898c87646-8747d', wait max 3 minutes if not. 2025-05-15 23:17:57,568 INFO 2025-05-15 23:17:57,568 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d --stdin -- bash -c 'if [ -f "/results/orders/cleanup/stderr.txt" ]; then echo "/results/orders/cleanup/stderr.txt file found"; exit 0; else echo "/results/orders/cleanup/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/cleanup/stderr.txt file found" 2025-05-15 23:17:57,568 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-05-15 23:17:58,355 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-05-15 23:17:58,355 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:58,355 DEBUG --- stdout --- 2025-05-15 23:17:58,355 DEBUG /results/orders/cleanup/stderr.txt file found 2025-05-15 23:17:58,355 DEBUG --- stderr --- 2025-05-15 23:17:58,355 DEBUG 2025-05-15 23:17:58,355 INFO 2025-05-15 23:17:58,356 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d --stdin -- du -k /results/orders/cleanup/stderr.txt | grep "^\d+ .+$" 2025-05-15 23:17:58,356 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-05-15 23:17:59,143 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-05-15 23:17:59,143 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:59,143 DEBUG --- stdout --- 2025-05-15 23:17:59,144 DEBUG 0 /results/orders/cleanup/stderr.txt 2025-05-15 23:17:59,144 DEBUG --- stderr --- 2025-05-15 23:17:59,144 DEBUG - Download file (size 0B) source : /results/orders/cleanup/stderr.txt @ overseer-0-898c87646-8747d destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/stderr.txt 2025-05-15 23:17:59,146 INFO 2025-05-15 23:17:59,146 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-8747d:/results/orders/cleanup/stderr.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/stderr.txt 2025-05-15 23:17:59,146 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-05-15 23:17:59,885 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:17:59,885 DEBUG --- stdout --- 2025-05-15 23:17:59,885 DEBUG tar: Removing leading `/' from member names 2025-05-15 23:17:59,885 DEBUG --- stderr --- 2025-05-15 23:17:59,885 DEBUG 2025-05-15 23:17:59,885 INFO 2025-05-15 23:17:59,885 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-05-15 23:17:59,885 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-05-15 23:18:00,159 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-05-15 23:18:00,159 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:18:00,159 DEBUG --- stdout --- 2025-05-15 23:18:00,160 DEBUG overseer-0-898c87646-8747d 2025-05-15 23:18:00,160 DEBUG --- stderr --- 2025-05-15 23:18:00,160 DEBUG Checking if file exists on pod 'overseer-0-898c87646-8747d', wait max 3 minutes if not. 2025-05-15 23:18:00,160 INFO 2025-05-15 23:18:00,160 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d --stdin -- bash -c 'if [ -f "/results/orders/cleanup/stdout.txt" ]; then echo "/results/orders/cleanup/stdout.txt file found"; exit 0; else echo "/results/orders/cleanup/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/cleanup/stdout.txt file found" 2025-05-15 23:18:00,160 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-05-15 23:18:00,945 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-05-15 23:18:00,945 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:18:00,945 DEBUG --- stdout --- 2025-05-15 23:18:00,946 DEBUG /results/orders/cleanup/stdout.txt file found 2025-05-15 23:18:00,946 DEBUG --- stderr --- 2025-05-15 23:18:00,946 DEBUG 2025-05-15 23:18:00,946 INFO 2025-05-15 23:18:00,946 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d --stdin -- du -k /results/orders/cleanup/stdout.txt | grep "^\d+ .+$" 2025-05-15 23:18:00,946 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-05-15 23:18:01,726 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-05-15 23:18:01,726 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:18:01,726 DEBUG --- stdout --- 2025-05-15 23:18:01,726 DEBUG 4 /results/orders/cleanup/stdout.txt 2025-05-15 23:18:01,726 DEBUG --- stderr --- 2025-05-15 23:18:01,726 DEBUG - Download file (size 4kB) source : /results/orders/cleanup/stdout.txt @ overseer-0-898c87646-8747d destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/stdout.txt 2025-05-15 23:18:01,727 INFO 2025-05-15 23:18:01,727 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-8747d:/results/orders/cleanup/stdout.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/stdout.txt 2025-05-15 23:18:01,727 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-05-15 23:18:02,478 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:18:02,478 DEBUG --- stdout --- 2025-05-15 23:18:02,478 DEBUG tar: Removing leading `/' from member names 2025-05-15 23:18:02,478 DEBUG --- stderr --- 2025-05-15 23:18:02,478 DEBUG ________________________________________________________________________________ [2025-05-15 23:18:02] cleanup show_cmd : Display order command ________________________________________________________________________________ 2025-05-15 23:18:02,479 INFO 2025-05-15 23:18:02,479 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2025-05-15 23:18:02,479 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2025-05-15 23:18:02,754 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2025-05-15 23:18:02,754 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:18:02,754 DEBUG --- stdout --- 2025-05-15 23:18:02,754 DEBUG overseer-0-898c87646-8747d 2025-05-15 23:18:02,754 DEBUG --- stderr --- 2025-05-15 23:18:02,754 DEBUG Checking if file exists on pod 'overseer-0-898c87646-8747d', wait max 3 minutes if not. 2025-05-15 23:18:02,755 INFO 2025-05-15 23:18:02,755 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d --stdin -- bash -c 'if [ -f "/results/orders/cleanup/cmd.txt" ]; then echo "/results/orders/cleanup/cmd.txt file found"; exit 0; else echo "/results/orders/cleanup/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/cleanup/cmd.txt file found" 2025-05-15 23:18:02,755 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2025-05-15 23:18:03,797 INFO [loop_until]: Function succeeded after 1s (rc=0) - expected pattern found 2025-05-15 23:18:03,797 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:18:03,797 DEBUG --- stdout --- 2025-05-15 23:18:03,797 DEBUG /results/orders/cleanup/cmd.txt file found 2025-05-15 23:18:03,797 DEBUG --- stderr --- 2025-05-15 23:18:03,797 DEBUG 2025-05-15 23:18:03,797 INFO 2025-05-15 23:18:03,798 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-898c87646-8747d --stdin -- du -k /results/orders/cleanup/cmd.txt | grep "^\d+ .+$" 2025-05-15 23:18:03,798 INFO [loop_until]: (max_time=240, interval=10, expected_rc=[0] 2025-05-15 23:18:04,578 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2025-05-15 23:18:04,578 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:18:04,578 DEBUG --- stdout --- 2025-05-15 23:18:04,578 DEBUG 4 /results/orders/cleanup/cmd.txt 2025-05-15 23:18:04,578 DEBUG --- stderr --- 2025-05-15 23:18:04,578 DEBUG - Download file (size 4kB) source : /results/orders/cleanup/cmd.txt @ overseer-0-898c87646-8747d destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/cmd.txt 2025-05-15 23:18:04,579 INFO 2025-05-15 23:18:04,579 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-898c87646-8747d:/results/orders/cleanup/cmd.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/cmd.txt 2025-05-15 23:18:04,579 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2025-05-15 23:18:05,304 INFO [loop_until]: OK (rc = 0) 2025-05-15 23:18:05,304 DEBUG --- stdout --- 2025-05-15 23:18:05,304 DEBUG tar: Removing leading `/' from member names 2025-05-15 23:18:05,304 DEBUG --- stderr --- 2025-05-15 23:18:05,304 DEBUG 2025-05-15 23:18:05,304 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/cmd.txt is small (1 lines), printing all lines: 2025-05-15 23:18:05,304 INFO ----- output ----- /lodestar/lodestar.py perf run-gatling --mvn-quiet --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idm.ManagedUsersDelete@perf-10m" --results-folder="/results/gatling/cleanup" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties=" -Dam_host=openam-perf-10m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=M@ythe4th -DamadminUsername=xiaosong.lou@pingidentity.com -Dcheck_before_preload=True -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-10m.forgeblocks.com -Dlogin_cookie=3398f6a1737d73a -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=T35!tr0c-k123" ________________________________________________________________________________ [2025-05-15 23:18:05] cleanup show_rc : Display rc order ________________________________________________________________________________ 2025-05-15 23:18:05,304 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/rc.txt is small (1 lines), printing all lines: 2025-05-15 23:18:05,304 INFO ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [2025-05-15 23:18:05] cleanup show_stdout : Display stdout order ________________________________________________________________________________ 2025-05-15 23:18:05,305 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/stdout.txt is small (29 lines), printing all lines: 2025-05-15 23:18:05,305 INFO ----- output ----- 2025-05-16 00:17:32,498 DEBUG Creating results folder /results/gatling/cleanup 2025-05-16 00:17:32,503 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmpwy4p4_lr 2025-05-16 00:17:32,506 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode --quiet -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/cleanup" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-10m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dcheck_before_preload=True -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-10m.forgeblocks.com -Dlogin_cookie=3398f6a1737d73a -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=**** Gatling 3.13.5 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Set idVarName to user_id Simulation com.forgerock.pyrock.Loadtest started... ================================================================================ 2025-05-15 23:17:46 GMT 1s elapsed ---- Requests ------------------------------------------------------------------ > Global (OK=9 KO=0 ) > restLoginInitiate (OK=1 KO=0 ) > restLoginUsernamePasswordCallback (OK=1 KO=0 ) > Skip 2FA (OK=1 KO=0 ) > User Authorize PKCE stage (OK=1 KO=0 ) > User AccessToken PKCE stage (OK=1 KO=0 ) > DeleteManagedUser (OK=4 KO=0 ) ---- getToken ------------------------------------------------------------------ [##########################################################################]100% waiting: 0 / active: 0 / done: 1 ---- idm.ManagedUsersDelete ---------------------------------------------------- [##########################################################################]100% waiting: 0 / active: 0 / done: 30 ================================================================================ 2025-05-16 00:17:46,811 INFO Gatling simulation run ended successfully 2025-05-16 00:17:46,811 DEBUG No archive name provided. No archive will be created ________________________________________________________________________________ [2025-05-15 23:18:05] cleanup show_stderr : Display stderr order ________________________________________________________________________________ 2025-05-15 23:18:05,305 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_login_tree/05_Data_Init/02_cleanup/stderr.txt is small (0 lines), printing all lines: 2025-05-15 23:18:05,305 INFO ----- output ----- ________________________________________________________________________________ [2025-05-15 23:18:05] cleanup check_rc_and_logs : Check order's outputs ________________________________________________________________________________ CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2025-05-15 23:18:05] cleanup print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2025-05-15 23:18:05] cleanup post : Post ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped