--Task-- name: cleanup enabled: True class_name: GatlingTask source_name: overseer-0 source_namespace: >default< target_name: idm target_namespace: >default< start: 0.0 stop: None timeout: 1h (3600 seconds) loop: False interval: None dependencies: ['dump-uid'] wait_for: ['Pre-check Num Users'] preceding_task: None options: {'testCase': 'idm.ManagedUsersDelete', 'concurrency': 30, 'java-opts': '-Dcsvfile=/results/idm/user_feeder.csv -Drealm=alpha'} group_name: None Current dir: /mnt/disk1/xslou/workshop/lodestar-fork/pyrock ________________________________________________________________________________ [2024-12-17 17:51:08] cleanup pre : N/A ________________________________________________________________________________ 2024-12-17 17:51:08,816 INFO Lodestar assumed that full stack is deployed so will set all products host URLs in Gatling system properties Pod log file : /results/orders/cleanup/stdout.txt Pod log : /results/orders/cleanup 2024-12-17 17:51:09,627 INFO 2024-12-17 17:51:09,627 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:09,627 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:09,900 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:09,900 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:09,900 DEBUG --- stdout --- 2024-12-17 17:51:09,900 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:09,900 DEBUG --- stderr --- 2024-12-17 17:51:09,900 DEBUG 2024-12-17 17:51:09 - INFO: Get overseer pod resources requests memory 2024-12-17 17:51:09,900 INFO 2024-12-17 17:51:09,900 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods overseer-0-5865b977f-9sbfx --output jsonpath={.spec.containers[].resources.requests.memory} 2024-12-17 17:51:09,900 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-12-17 17:51:10,172 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:10,172 DEBUG --- stdout --- 2024-12-17 17:51:10,172 DEBUG 4Gi 2024-12-17 17:51:10,172 DEBUG --- stderr --- 2024-12-17 17:51:10,172 DEBUG ________________________________________________________________________________ [2024-12-17 17:51:10] cleanup print_command : Display the command to be forwarded to overseer as an order ________________________________________________________________________________ 2024-12-17 17:51:10,173 INFO 2024-12-17 17:51:10,173 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:10,173 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:10,447 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:10,447 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:10,447 DEBUG --- stdout --- 2024-12-17 17:51:10,447 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:10,447 DEBUG --- stderr --- 2024-12-17 17:51:10,447 DEBUG /lodestar/lodestar.py perf run-gatling --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idm.ManagedUsersDelete@perf-200m" --results-folder="/results/gatling/cleanup" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-200m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=M@ythe4th -DamadminUsername=xiaosong.lou@pingidentity.com -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-200m.forgeblocks.com -Dlogin_cookie=15892ffd844ef80 -Dnum_users=200000000 -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=T35!tr0c-k123" ________________________________________________________________________________ [2024-12-17 17:51:10] cleanup create_order : Create order for overseer with the command to run ________________________________________________________________________________ 2024-12-17 17:51:10,448 INFO 2024-12-17 17:51:10,448 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:10,448 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:10,724 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:10,724 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:10,724 DEBUG --- stdout --- 2024-12-17 17:51:10,724 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:10,724 DEBUG --- stderr --- 2024-12-17 17:51:10,724 DEBUG Order file: /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/order.json { "name": "cleanup", "command": " /lodestar/lodestar.py perf run-gatling --simulation-class=\"com.forgerock.pyrock.Loadtest\" --run-description=\"idm.ManagedUsersDelete@perf-200m\" --results-folder=\"/results/gatling/cleanup\" --no-reports --simulation-jvm-args=\"-XX:+ExitOnOutOfMemoryError -Xmx4G\" --simulation-system-properties=\"-Dam_host=openam-perf-200m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-200m.forgeblocks.com -Dlogin_cookie=15892ffd844ef80 -Dnum_users=200000000 -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=T35!tr0c-k123\"", "out-dir": "/results/orders/cleanup" } ________________________________________________________________________________ [2024-12-17 17:51:10] cleanup loop_until_ready : Wait for preceding task to finish if it uses same Overseer as a source, then loop for max_time until Overseer pod status is READY to accept order :param max_time: periodically check overseer status for this many seconds at maximum :param interval: interval between two periodical checks :return: ________________________________________________________________________________ 2024-12-17 17:51:10,727 INFO 2024-12-17 17:51:10,727 INFO [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" 2024-12-17 17:51:10,727 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2024-12-17 17:51:11,008 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-12-17 17:51:11,008 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:11,008 DEBUG --- stdout --- 2024-12-17 17:51:11,008 DEBUG WAITING 2024-12-17 17:51:11,008 DEBUG --- stderr --- 2024-12-17 17:51:11,008 DEBUG % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 25 0 --:--:-- --:--:-- --:--:-- 25 ________________________________________________________________________________ [2024-12-17 17:51:11] cleanup start_order : Send the order to the overseer pod ________________________________________________________________________________ 2024-12-17 17:51:11,009 INFO 2024-12-17 17:51:11,009 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:11,009 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:11,283 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:11,283 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:11,283 DEBUG --- stdout --- 2024-12-17 17:51:11,283 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:11,283 DEBUG --- stderr --- 2024-12-17 17:51:11,283 DEBUG 2024-12-17 17:51:11,283 INFO 2024-12-17 17:51:11,284 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/order.json overseer-0-5865b977f-9sbfx:/results/orders/order.json 2024-12-17 17:51:11,284 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-12-17 17:51:12,655 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:12,655 DEBUG --- stdout --- 2024-12-17 17:51:12,655 DEBUG 2024-12-17 17:51:12,655 DEBUG --- stderr --- 2024-12-17 17:51:12,655 DEBUG Order has been successfully forwarded to overseer 2024-12-17 17:51:12 - INFO: Check if Gatling simulation started successfully 2024-12-17 17:51:12,656 INFO 2024-12-17 17:51:12,656 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:12,656 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:12,941 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:12,942 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:12,942 DEBUG --- stdout --- 2024-12-17 17:51:12,942 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:12,942 DEBUG --- stderr --- 2024-12-17 17:51:12,942 DEBUG 2024-12-17 17:51:12,942 INFO 2024-12-17 17:51:12,942 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx -- head --lines 5000 /results/orders/cleanup/stdout.txt | grep "Simulation com.forgerock.pyrock.Loadtest started..." 2024-12-17 17:51:12,942 INFO [loop_until]: (max_time=300, interval=5, expected_rc=[0] 2024-12-17 17:51:13,686 INFO [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2024-12-17 17:51:19,412 INFO [loop_until]: Function succeeded after 6s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2024-12-17 17:51:25,145 INFO [loop_until]: Function succeeded after 12s (rc=0) - failed to find expected output: Simulation com.forgerock.pyrock.Loadtest started... - retry 2024-12-17 17:51:30,877 INFO [loop_until]: Function succeeded after 17s (rc=0) - expected pattern found 2024-12-17 17:51:30,877 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:30,877 DEBUG --- stdout --- 2024-12-17 17:51:30,877 DEBUG 2024-12-17 17:51:13,973 DEBUG Creating results folder /results/gatling/cleanup 2024-12-17 17:51:13,980 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmput17t1y0 2024-12-17 17:51:13,983 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/cleanup" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-200m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-200m.forgeblocks.com -Dlogin_cookie=15892ffd844ef80 -Dnum_users=200000000 -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=**** [INFO] Scanning for projects... [INFO] [INFO] -----------< com.forgerock.pyrock:pyrock-gatling-simulation >----------- [INFO] Building pyrock-gatling-simulation 3.12.0 [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] >>> gatling:4.9.6:test (default-cli) > test-compile @ pyrock-gatling-simulation >>> [INFO] [INFO] --- resources:3.3.1:resources (default-resources) @ pyrock-gatling-simulation --- [INFO] skip non existing resourceDirectory /tmp/tmput17t1y0/src/main/resources [INFO] [INFO] --- compiler:3.13.0:compile (default-compile) @ pyrock-gatling-simulation --- [INFO] No sources to compile [INFO] [INFO] --- resources:3.3.1:testResources (default-testResources) @ pyrock-gatling-simulation --- [INFO] Copying 2 resources from src/test/resources to target/test-classes [INFO] [INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ pyrock-gatling-simulation --- [INFO] Nothing to compile - all classes are up to date. [INFO] [INFO] --- scala:4.9.2:testCompile (default) @ pyrock-gatling-simulation --- [INFO] compiling 38 Scala sources to /tmp/tmput17t1y0/target/test-classes ... [INFO] compile in 10.1 s [INFO] [INFO] <<< gatling:4.9.6:test (default-cli) < test-compile @ pyrock-gatling-simulation <<< [INFO] [INFO] [INFO] --- gatling:4.9.6:test (default-cli) @ pyrock-gatling-simulation --- [INFO] Running simulation com.forgerock.pyrock.Loadtest. Gatling 3.13.1 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Set idVarName to user_id (This line must be printed to let controller stop the overseer task) Simulation com.forgerock.pyrock.Loadtest started... Selected test case is file feeder based and no entries were found in provided data file (this might happen when removing users before a test and no users are present). Nothing to do, exit now... [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 13.894 s [INFO] Finished at: 2024-12-17T17:51:28Z [INFO] ------------------------------------------------------------------------ 2024-12-17 17:51:28,940 INFO Gatling simulation run ended successfully 2024-12-17 17:51:28,941 DEBUG No archive name provided. No archive will be created 2024-12-17 17:51:30,877 DEBUG --- stderr --- 2024-12-17 17:51:30,877 DEBUG ________________________________________________________________________________ [2024-12-17 17:51:30] cleanup wait_stop_by_itself : Wait for the order to stop by itself ________________________________________________________________________________ Task may stop by itself (no stop defined in configuration file), wait until overseer ends the order - timeout is 1h. Waiting for status (curl --fail --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt) == WAITING PASS : Order stopped by itself after 5s and before timeout (1h) ________________________________________________________________________________ [2024-12-17 17:51:36] cleanup stop_order : Stop the order on overseer pod ________________________________________________________________________________ 2024-12-17 17:51:36,115 INFO 2024-12-17 17:51:36,115 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:36,115 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:36,388 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:36,388 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:36,388 DEBUG --- stdout --- 2024-12-17 17:51:36,388 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:36,389 DEBUG --- stderr --- 2024-12-17 17:51:36,389 DEBUG 2024-12-17 17:51:36,389 INFO 2024-12-17 17:51:36,389 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx -- rm -f /results/orders/order.json 2024-12-17 17:51:36,389 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-12-17 17:51:37,109 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:37,109 DEBUG --- stdout --- 2024-12-17 17:51:37,109 DEBUG 2024-12-17 17:51:37,109 DEBUG --- stderr --- 2024-12-17 17:51:37,109 DEBUG Order ran during 24s ________________________________________________________________________________ [2024-12-17 17:51:37] cleanup download_logs : Download individual order files ________________________________________________________________________________ 2024-12-17 17:51:37 - PASS: file already downloaded /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/order.json 2024-12-17 17:51:37,109 INFO 2024-12-17 17:51:37,109 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:37,109 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:37,382 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:37,382 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:37,382 DEBUG --- stdout --- 2024-12-17 17:51:37,382 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:37,382 DEBUG --- stderr --- 2024-12-17 17:51:37,382 DEBUG Checking pod: overseer-0-5865b977f-9sbfx for file exists, if not wait. 2024-12-17 17:51:37,383 INFO 2024-12-17 17:51:37,383 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:37,383 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:37,657 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:37,657 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:37,657 DEBUG --- stdout --- 2024-12-17 17:51:37,657 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:37,657 DEBUG --- stderr --- 2024-12-17 17:51:37,657 DEBUG 2024-12-17 17:51:37,657 INFO 2024-12-17 17:51:37,657 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx --stdin -- bash -c 'if [ -f "/results/orders/cleanup/rc.txt" ]; then echo "/results/orders/cleanup/rc.txt file found"; exit 0; else echo "/results/orders/cleanup/rc.txt file not found"; exit 1; fi' | grep "/results/orders/cleanup/rc.txt file found" 2024-12-17 17:51:37,657 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-12-17 17:51:38,441 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-12-17 17:51:38,442 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:38,442 DEBUG --- stdout --- 2024-12-17 17:51:38,442 DEBUG /results/orders/cleanup/rc.txt file found 2024-12-17 17:51:38,442 DEBUG --- stderr --- 2024-12-17 17:51:38,442 DEBUG 2024-12-17 17:51:38,442 INFO 2024-12-17 17:51:38,442 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:38,442 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:38,731 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:38,731 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:38,731 DEBUG --- stdout --- 2024-12-17 17:51:38,731 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:38,731 DEBUG --- stderr --- 2024-12-17 17:51:38,731 DEBUG 2024-12-17 17:51:38,731 INFO 2024-12-17 17:51:38,732 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx --stdin -- du -k /results/orders/cleanup/rc.txt 2024-12-17 17:51:39,523 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-12-17 17:51:39,523 DEBUG --- stdout --- 2024-12-17 17:51:39,523 DEBUG 4 /results/orders/cleanup/rc.txt 2024-12-17 17:51:39,523 DEBUG --- stderr --- 2024-12-17 17:51:39,523 DEBUG 2024-12-17 17:51:39,523 INFO - Download file (size 4k) 2024-12-17 17:51:39,523 INFO 2024-12-17 17:51:39,523 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:39,523 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:39,797 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:39,797 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:39,797 DEBUG --- stdout --- 2024-12-17 17:51:39,797 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:39,797 DEBUG --- stderr --- 2024-12-17 17:51:39,797 DEBUG source : /results/orders/cleanup/rc.txt @ overseer-0-5865b977f-9sbfx destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/rc.txt 2024-12-17 17:51:39,797 INFO 2024-12-17 17:51:39,797 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:39,797 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:40,072 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:40,072 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:40,072 DEBUG --- stdout --- 2024-12-17 17:51:40,072 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:40,072 DEBUG --- stderr --- 2024-12-17 17:51:40,072 DEBUG 2024-12-17 17:51:40,073 INFO 2024-12-17 17:51:40,073 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-5865b977f-9sbfx:/results/orders/cleanup/rc.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/rc.txt 2024-12-17 17:51:40,073 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-12-17 17:51:40,791 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:40,791 DEBUG --- stdout --- 2024-12-17 17:51:40,791 DEBUG tar: Removing leading `/' from member names 2024-12-17 17:51:40,791 DEBUG --- stderr --- 2024-12-17 17:51:40,791 DEBUG 2024-12-17 17:51:40,792 INFO 2024-12-17 17:51:40,792 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:40,792 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:41,067 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:41,067 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:41,067 DEBUG --- stdout --- 2024-12-17 17:51:41,067 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:41,067 DEBUG --- stderr --- 2024-12-17 17:51:41,067 DEBUG Checking pod: overseer-0-5865b977f-9sbfx for file exists, if not wait. 2024-12-17 17:51:41,068 INFO 2024-12-17 17:51:41,068 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:41,068 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:41,392 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:41,392 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:41,393 DEBUG --- stdout --- 2024-12-17 17:51:41,393 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:41,393 DEBUG --- stderr --- 2024-12-17 17:51:41,393 DEBUG 2024-12-17 17:51:41,394 INFO 2024-12-17 17:51:41,394 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx --stdin -- bash -c 'if [ -f "/results/orders/cleanup/stderr.txt" ]; then echo "/results/orders/cleanup/stderr.txt file found"; exit 0; else echo "/results/orders/cleanup/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/cleanup/stderr.txt file found" 2024-12-17 17:51:41,394 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-12-17 17:51:42,186 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-12-17 17:51:42,186 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:42,186 DEBUG --- stdout --- 2024-12-17 17:51:42,186 DEBUG /results/orders/cleanup/stderr.txt file found 2024-12-17 17:51:42,186 DEBUG --- stderr --- 2024-12-17 17:51:42,186 DEBUG 2024-12-17 17:51:42,186 INFO 2024-12-17 17:51:42,186 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:42,186 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:42,462 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:42,462 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:42,462 DEBUG --- stdout --- 2024-12-17 17:51:42,462 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:42,462 DEBUG --- stderr --- 2024-12-17 17:51:42,462 DEBUG 2024-12-17 17:51:42,462 INFO 2024-12-17 17:51:42,462 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx --stdin -- du -k /results/orders/cleanup/stderr.txt 2024-12-17 17:51:43,284 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-12-17 17:51:43,284 DEBUG --- stdout --- 2024-12-17 17:51:43,284 DEBUG 0 /results/orders/cleanup/stderr.txt 2024-12-17 17:51:43,284 DEBUG --- stderr --- 2024-12-17 17:51:43,284 DEBUG 2024-12-17 17:51:43,284 INFO - Download file (size 0k) 2024-12-17 17:51:43,284 INFO 2024-12-17 17:51:43,284 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:43,284 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:43,561 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:43,561 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:43,561 DEBUG --- stdout --- 2024-12-17 17:51:43,561 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:43,561 DEBUG --- stderr --- 2024-12-17 17:51:43,561 DEBUG source : /results/orders/cleanup/stderr.txt @ overseer-0-5865b977f-9sbfx destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/stderr.txt 2024-12-17 17:51:43,561 INFO 2024-12-17 17:51:43,561 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:43,561 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:43,837 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:43,837 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:43,837 DEBUG --- stdout --- 2024-12-17 17:51:43,837 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:43,837 DEBUG --- stderr --- 2024-12-17 17:51:43,837 DEBUG 2024-12-17 17:51:43,837 INFO 2024-12-17 17:51:43,837 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-5865b977f-9sbfx:/results/orders/cleanup/stderr.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/stderr.txt 2024-12-17 17:51:43,837 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-12-17 17:51:44,585 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:44,585 DEBUG --- stdout --- 2024-12-17 17:51:44,585 DEBUG tar: Removing leading `/' from member names 2024-12-17 17:51:44,585 DEBUG --- stderr --- 2024-12-17 17:51:44,585 DEBUG 2024-12-17 17:51:44,585 INFO 2024-12-17 17:51:44,585 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:44,585 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:44,861 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:44,861 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:44,861 DEBUG --- stdout --- 2024-12-17 17:51:44,861 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:44,861 DEBUG --- stderr --- 2024-12-17 17:51:44,861 DEBUG Checking pod: overseer-0-5865b977f-9sbfx for file exists, if not wait. 2024-12-17 17:51:44,861 INFO 2024-12-17 17:51:44,861 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:44,861 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:45,135 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:45,136 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:45,136 DEBUG --- stdout --- 2024-12-17 17:51:45,136 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:45,136 DEBUG --- stderr --- 2024-12-17 17:51:45,136 DEBUG 2024-12-17 17:51:45,136 INFO 2024-12-17 17:51:45,136 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx --stdin -- bash -c 'if [ -f "/results/orders/cleanup/stdout.txt" ]; then echo "/results/orders/cleanup/stdout.txt file found"; exit 0; else echo "/results/orders/cleanup/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/cleanup/stdout.txt file found" 2024-12-17 17:51:45,136 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-12-17 17:51:45,922 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-12-17 17:51:45,922 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:45,923 DEBUG --- stdout --- 2024-12-17 17:51:45,923 DEBUG /results/orders/cleanup/stdout.txt file found 2024-12-17 17:51:45,923 DEBUG --- stderr --- 2024-12-17 17:51:45,923 DEBUG 2024-12-17 17:51:45,923 INFO 2024-12-17 17:51:45,923 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:45,923 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:46,197 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:46,198 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:46,198 DEBUG --- stdout --- 2024-12-17 17:51:46,198 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:46,198 DEBUG --- stderr --- 2024-12-17 17:51:46,198 DEBUG 2024-12-17 17:51:46,198 INFO 2024-12-17 17:51:46,198 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx --stdin -- du -k /results/orders/cleanup/stdout.txt 2024-12-17 17:51:46,986 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-12-17 17:51:46,987 DEBUG --- stdout --- 2024-12-17 17:51:46,987 DEBUG 4 /results/orders/cleanup/stdout.txt 2024-12-17 17:51:46,987 DEBUG --- stderr --- 2024-12-17 17:51:46,987 DEBUG 2024-12-17 17:51:46,987 INFO - Download file (size 4k) 2024-12-17 17:51:46,987 INFO 2024-12-17 17:51:46,987 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:46,987 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:47,285 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:47,285 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:47,285 DEBUG --- stdout --- 2024-12-17 17:51:47,285 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:47,285 DEBUG --- stderr --- 2024-12-17 17:51:47,285 DEBUG source : /results/orders/cleanup/stdout.txt @ overseer-0-5865b977f-9sbfx destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/stdout.txt 2024-12-17 17:51:47,285 INFO 2024-12-17 17:51:47,285 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:47,285 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:47,560 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:47,560 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:47,560 DEBUG --- stdout --- 2024-12-17 17:51:47,560 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:47,560 DEBUG --- stderr --- 2024-12-17 17:51:47,560 DEBUG 2024-12-17 17:51:47,560 INFO 2024-12-17 17:51:47,560 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-5865b977f-9sbfx:/results/orders/cleanup/stdout.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/stdout.txt 2024-12-17 17:51:47,560 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-12-17 17:51:48,314 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:48,314 DEBUG --- stdout --- 2024-12-17 17:51:48,314 DEBUG tar: Removing leading `/' from member names 2024-12-17 17:51:48,314 DEBUG --- stderr --- 2024-12-17 17:51:48,314 DEBUG ________________________________________________________________________________ [2024-12-17 17:51:48] cleanup show_cmd : Display order command ________________________________________________________________________________ 2024-12-17 17:51:48,314 INFO 2024-12-17 17:51:48,314 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:48,314 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:48,589 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:48,589 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:48,589 DEBUG --- stdout --- 2024-12-17 17:51:48,589 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:48,589 DEBUG --- stderr --- 2024-12-17 17:51:48,589 DEBUG Checking pod: overseer-0-5865b977f-9sbfx for file exists, if not wait. 2024-12-17 17:51:48,589 INFO 2024-12-17 17:51:48,589 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:48,589 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:48,863 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:48,863 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:48,863 DEBUG --- stdout --- 2024-12-17 17:51:48,863 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:48,863 DEBUG --- stderr --- 2024-12-17 17:51:48,863 DEBUG 2024-12-17 17:51:48,864 INFO 2024-12-17 17:51:48,864 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx --stdin -- bash -c 'if [ -f "/results/orders/cleanup/cmd.txt" ]; then echo "/results/orders/cleanup/cmd.txt file found"; exit 0; else echo "/results/orders/cleanup/cmd.txt file not found"; exit 1; fi' | grep "/results/orders/cleanup/cmd.txt file found" 2024-12-17 17:51:48,864 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-12-17 17:51:49,652 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-12-17 17:51:49,652 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:49,652 DEBUG --- stdout --- 2024-12-17 17:51:49,652 DEBUG /results/orders/cleanup/cmd.txt file found 2024-12-17 17:51:49,652 DEBUG --- stderr --- 2024-12-17 17:51:49,652 DEBUG 2024-12-17 17:51:49,653 INFO 2024-12-17 17:51:49,653 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:49,653 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:49,928 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:49,928 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:49,928 DEBUG --- stdout --- 2024-12-17 17:51:49,928 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:49,928 DEBUG --- stderr --- 2024-12-17 17:51:49,928 DEBUG 2024-12-17 17:51:49,928 INFO 2024-12-17 17:51:49,928 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-5865b977f-9sbfx --stdin -- du -k /results/orders/cleanup/cmd.txt 2024-12-17 17:51:50,718 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-12-17 17:51:50,718 DEBUG --- stdout --- 2024-12-17 17:51:50,718 DEBUG 4 /results/orders/cleanup/cmd.txt 2024-12-17 17:51:50,718 DEBUG --- stderr --- 2024-12-17 17:51:50,718 DEBUG 2024-12-17 17:51:50,718 INFO - Download file (size 4k) 2024-12-17 17:51:50,718 INFO 2024-12-17 17:51:50,718 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:50,718 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:50,992 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:50,992 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:50,992 DEBUG --- stdout --- 2024-12-17 17:51:50,993 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:50,993 DEBUG --- stderr --- 2024-12-17 17:51:50,993 DEBUG source : /results/orders/cleanup/cmd.txt @ overseer-0-5865b977f-9sbfx destination : /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/cmd.txt 2024-12-17 17:51:50,993 INFO 2024-12-17 17:51:50,993 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 2024-12-17 17:51:50,993 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-12-17 17:51:51,304 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-12-17 17:51:51,304 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:51,304 DEBUG --- stdout --- 2024-12-17 17:51:51,304 DEBUG overseer-0-5865b977f-9sbfx 2024-12-17 17:51:51,304 DEBUG --- stderr --- 2024-12-17 17:51:51,305 DEBUG 2024-12-17 17:51:51,305 INFO 2024-12-17 17:51:51,305 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-5865b977f-9sbfx:/results/orders/cleanup/cmd.txt /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/cmd.txt 2024-12-17 17:51:51,305 INFO [loop_until]: (max_time=-1, interval=5, expected_rc=[0] 2024-12-17 17:51:52,087 INFO [loop_until]: OK (rc = 0) 2024-12-17 17:51:52,087 DEBUG --- stdout --- 2024-12-17 17:51:52,087 DEBUG tar: Removing leading `/' from member names 2024-12-17 17:51:52,087 DEBUG --- stderr --- 2024-12-17 17:51:52,087 DEBUG 2024-12-17 17:51:52,087 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/cmd.txt is small (1 lines), printing all lines: 2024-12-17 17:51:52,087 DEBUG ----- output ----- 2024-12-17 17:51:52,087 DEBUG /lodestar/lodestar.py perf run-gatling --simulation-class="com.forgerock.pyrock.Loadtest" --run-description="idm.ManagedUsersDelete@perf-200m" --results-folder="/results/gatling/cleanup" --no-reports --simulation-jvm-args="-XX:+ExitOnOutOfMemoryError -Xmx4G" --simulation-system-properties="-Dam_host=openam-perf-200m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=M@ythe4th -DamadminUsername=xiaosong.lou@pingidentity.com -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-200m.forgeblocks.com -Dlogin_cookie=15892ffd844ef80 -Dnum_users=200000000 -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=T35!tr0c-k123" 2024-12-17 17:51:52,087 INFO ________________________________________________________________________________ [2024-12-17 17:51:52] cleanup show_rc : Display rc order ________________________________________________________________________________ 2024-12-17 17:51:52,088 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/rc.txt is small (1 lines), printing all lines: 2024-12-17 17:51:52,088 DEBUG ----- output ----- 2024-12-17 17:51:52,088 DEBUG CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) 2024-12-17 17:51:52,088 INFO ________________________________________________________________________________ [2024-12-17 17:51:52] cleanup show_stdout : Display stdout order ________________________________________________________________________________ 2024-12-17 17:51:52,088 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/stdout.txt is small (46 lines), printing all lines: 2024-12-17 17:51:52,088 DEBUG ----- output ----- 2024-12-17 17:51:52,088 DEBUG 2024-12-17 17:51:13,973 DEBUG Creating results folder /results/gatling/cleanup 2024-12-17 17:51:13,980 INFO pom.xml and Gatling Scala files copied from /home/jenkins/lodestar/pyrock/shared/clients/gatling to temporary folder /tmp/tmput17t1y0 2024-12-17 17:51:13,983 INFO Running Gatling simulation with command: mvn gatling:test --batch-mode -Dgatling.simulationClass="com.forgerock.pyrock.Loadtest" -Dgatling.runDescription="..." -Dgatling.resultsFolder="/results/gatling/cleanup" -Dgatling.noReports=true -Dgatling.jvmArgs="-XX:+ExitOnOutOfMemoryError,-Xmx4G" -Dgatling.propagateSystemProperties=true -Dam_host=openam-perf-200m.forgeblocks.com -Dam_oauth2_scope=openid^fr:idm:* -DamadminPassword=**** -DamadminUsername=xiaosong.lou@pingidentity.com -Dconcurrency=30 -Dcsvfile=/results/idm/user_feeder.csv -Didc=true -Didm_host=openam-perf-200m.forgeblocks.com -Dlogin_cookie=15892ffd844ef80 -Dnum_users=200000000 -Doauth2_client_id=idmAdminClient -Doauth2_redirect_uri=/platform/appAuthHelperRedirect.html -Drealm=alpha -DtestCase=idm.ManagedUsersDelete -Duser_password=**** [INFO] Scanning for projects... [INFO] [INFO] -----------< com.forgerock.pyrock:pyrock-gatling-simulation >----------- [INFO] Building pyrock-gatling-simulation 3.12.0 [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] >>> gatling:4.9.6:test (default-cli) > test-compile @ pyrock-gatling-simulation >>> [INFO] [INFO] --- resources:3.3.1:resources (default-resources) @ pyrock-gatling-simulation --- [INFO] skip non existing resourceDirectory /tmp/tmput17t1y0/src/main/resources [INFO] [INFO] --- compiler:3.13.0:compile (default-compile) @ pyrock-gatling-simulation --- [INFO] No sources to compile [INFO] [INFO] --- resources:3.3.1:testResources (default-testResources) @ pyrock-gatling-simulation --- [INFO] Copying 2 resources from src/test/resources to target/test-classes [INFO] [INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ pyrock-gatling-simulation --- [INFO] Nothing to compile - all classes are up to date. [INFO] [INFO] --- scala:4.9.2:testCompile (default) @ pyrock-gatling-simulation --- [INFO] compiling 38 Scala sources to /tmp/tmput17t1y0/target/test-classes ... [INFO] compile in 10.1 s [INFO] [INFO] <<< gatling:4.9.6:test (default-cli) < test-compile @ pyrock-gatling-simulation <<< [INFO] [INFO] [INFO] --- gatling:4.9.6:test (default-cli) @ pyrock-gatling-simulation --- [INFO] Running simulation com.forgerock.pyrock.Loadtest. Gatling 3.13.1 is available! (you're using 3.12.0) System property think_time_multiplier not set. No think time emulation will be done in the simulation. Set idVarName to user_id (This line must be printed to let controller stop the overseer task) Simulation com.forgerock.pyrock.Loadtest started... Selected test case is file feeder based and no entries were found in provided data file (this might happen when removing users before a test and no users are present). Nothing to do, exit now... [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 13.894 s [INFO] Finished at: 2024-12-17T17:51:28Z [INFO] ------------------------------------------------------------------------ 2024-12-17 17:51:28,940 INFO Gatling simulation run ended successfully 2024-12-17 17:51:28,941 DEBUG No archive name provided. No archive will be created 2024-12-17 17:51:52,088 INFO ________________________________________________________________________________ [2024-12-17 17:51:52] cleanup show_stderr : Display stderr order ________________________________________________________________________________ 2024-12-17 17:51:52,088 DEBUG [print_head_tail]: Input file /mnt/disk1/xslou/workshop/lodestar-fork/results/pyrock/idc_idm_recon/05_Data_Init/02_cleanup/stderr.txt is small (0 lines), printing all lines: 2024-12-17 17:51:52,088 DEBUG ----- output ----- 2024-12-17 17:51:52,088 DEBUG 2024-12-17 17:51:52,088 INFO ________________________________________________________________________________ [2024-12-17 17:51:52] cleanup check_rc_and_logs : Check order's outputs ________________________________________________________________________________ PASS: no error found in rc log PASS: no error found in stdout log PASS: no error found in stderr log ________________________________________________________________________________ [2024-12-17 17:51:52] cleanup print_overseer_logs : Print overseer logs from the K8s pod ________________________________________________________________________________ Task result is INIT no need to display logs. ________________________________________________________________________________ [2024-12-17 17:51:52] cleanup post : Post ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped