--Task-- name: Reporting-login-tree-350 enabled: True class_name: GatlingResultTask source_name: overseer-0 source_namespace: >default< target_name: overseer-0 target_namespace: >default< start: 0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: [] wait_for: ['Reporting-login-tree-10', 'Reporting-login-tree-50', 'Reporting-login-tree-100', 'Reporting-login-tree-150', 'Reporting-login-tree-200', 'Reporting-login-tree-250', 'Reporting-login-tree-300'] options: {'task_name': 'login-tree-350'} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [2024-02-25 06:32:00] Reporting-login-tree-350 pre : N/A ________________________________________________________________________________ source (overseer-0) is a component [2024-02-25 06:32:00] - INFO: Timeout for this Task has changed to 4h (14400 seconds) [2024-02-25 06:32:00] - INFO: timeout was set to GatlingResultTask default (half of global_duration or 3600 seconds (whatever is higher)), because it was unset Pod stdout file : /results/orders/Reporting-login-tree-350/stdout.txt Pod stderr file : /results/orders/Reporting-login-tree-350/stderr.txt Pod log : /results/orders/Reporting-login-tree-350 ________________________________________________________________________________ [2024-02-25 06:32:00] Reporting-login-tree-350 step1 : Write order for overseer to generate the report ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- Order file: /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/order.json { "name": "Reporting-login-tree-350", "command": "/lodestar/clients/gatling-report-only-for-pyrock.sh login-tree-350 --extra-run-jvm-options \"-Xmx4G\"", "out-dir": "/results/orders/Reporting-login-tree-350" } ________________________________________________________________________________ [2024-02-25 06:32:00] Reporting-login-tree-350 step2 : Make sure overseer is ready to accept order (not already busy) ________________________________________________________________________________ [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" [loop_until]: (max_time=120, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- WAITING --- stderr --- % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 33 0 --:--:-- --:--:-- --:--:-- 33 ________________________________________________________________________________ [2024-02-25 06:32:00] Reporting-login-tree-350 step3 : Send order to overseer ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/order.json overseer-0-86b9999bcb-xv8f9:/results/orders/order.json [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- Order has been successfully forwarded to overseer ________________________________________________________________________________ [2024-02-25 06:32:02] Reporting-login-tree-350 step4 : Wait order is done ________________________________________________________________________________ Workaround for PERF-1966. Sleeping for 5 seconds. [loop_until]: curl --insecure https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com/orders/status.txt | grep "WAITING" [loop_until]: (max_time=14400, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: WAITING - retry [loop_until]: Function succeeded after 5s (rc=0) - failed to find expected output: WAITING - retry [loop_until]: Function succeeded after 10s (rc=0) - failed to find expected output: WAITING - retry [loop_until]: Function succeeded after 16s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- WAITING --- stderr --- % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 7 100 7 0 0 33 0 --:--:-- --:--:-- --:--:-- 33 ________________________________________________________________________________ [2024-02-25 06:32:23] Reporting-login-tree-350 step5 : Wait until gatling report is generated on overseer ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- Checking pod: overseer-0-86b9999bcb-xv8f9 for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-86b9999bcb-xv8f9 --stdin -- bash -c 'if [ -f "/results/gatling/login-tree-350.tar.gz" ]; then echo "/results/gatling/login-tree-350.tar.gz file found"; exit 0; else echo "/results/gatling/login-tree-350.tar.gz file not found"; exit 1; fi' | grep "/results/gatling/login-tree-350.tar.gz file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/gatling/login-tree-350.tar.gz file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-86b9999bcb-xv8f9 --stdin -- du -k /results/gatling/login-tree-350.tar.gz [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 368 /results/gatling/login-tree-350.tar.gz --- stderr --- - Download file (size 368k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- source : /results/gatling/login-tree-350.tar.gz @ overseer-0-86b9999bcb-xv8f9 destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/shared/login-tree-350.tar.gz [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-86b9999bcb-xv8f9:/results/gatling/login-tree-350.tar.gz /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/shared/login-tree-350.tar.gz [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- ________________________________________________________________________________ [2024-02-25 06:32:27] Reporting-login-tree-350 step6 : Unzip the result ________________________________________________________________________________ [run_command]: tar zxf /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/shared/login-tree-350.tar.gz -C /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/07_scenario/08_login-tree-350/gatling [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- --- stderr --- ________________________________________________________________________________ [2024-02-25 06:32:27] Reporting-login-tree-350 step7 : Print rc order ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- Checking pod: overseer-0-86b9999bcb-xv8f9 for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-86b9999bcb-xv8f9 --stdin -- bash -c 'if [ -f "/results/orders/Reporting-login-tree-350/rc.txt" ]; then echo "/results/orders/Reporting-login-tree-350/rc.txt file found"; exit 0; else echo "/results/orders/Reporting-login-tree-350/rc.txt file not found"; exit 1; fi' | grep "/results/orders/Reporting-login-tree-350/rc.txt file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/Reporting-login-tree-350/rc.txt file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-86b9999bcb-xv8f9 --stdin -- du -k /results/orders/Reporting-login-tree-350/rc.txt [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 4 /results/orders/Reporting-login-tree-350/rc.txt --- stderr --- - Download file (size 4k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- source : /results/orders/Reporting-login-tree-350/rc.txt @ overseer-0-86b9999bcb-xv8f9 destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/rc.txt [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-86b9999bcb-xv8f9:/results/orders/Reporting-login-tree-350/rc.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/rc.txt [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/rc.txt is small (1 lines), printing all lines: ----- output ----- CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) ________________________________________________________________________________ [2024-02-25 06:32:31] Reporting-login-tree-350 step8 : Check rc log ________________________________________________________________________________ PASS: no error found in rc log ________________________________________________________________________________ [2024-02-25 06:32:31] Reporting-login-tree-350 step9 : Print stdout order ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- Checking pod: overseer-0-86b9999bcb-xv8f9 for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-86b9999bcb-xv8f9 --stdin -- bash -c 'if [ -f "/results/orders/Reporting-login-tree-350/stdout.txt" ]; then echo "/results/orders/Reporting-login-tree-350/stdout.txt file found"; exit 0; else echo "/results/orders/Reporting-login-tree-350/stdout.txt file not found"; exit 1; fi' | grep "/results/orders/Reporting-login-tree-350/stdout.txt file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/Reporting-login-tree-350/stdout.txt file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-86b9999bcb-xv8f9 --stdin -- du -k /results/orders/Reporting-login-tree-350/stdout.txt [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 4 /results/orders/Reporting-login-tree-350/stdout.txt --- stderr --- - Download file (size 4k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- source : /results/orders/Reporting-login-tree-350/stdout.txt @ overseer-0-86b9999bcb-xv8f9 destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/stdout.txt [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-86b9999bcb-xv8f9:/results/orders/Reporting-login-tree-350/stdout.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/stdout.txt [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/stdout.txt is small (84 lines), printing all lines: ----- output ----- [cmd] /lodestar/clients/gatling/bin/gatling.sh --reports-only /results/gatling/login-tree-350/loadtest-20240225031318339 --extra-run-jvm-options -Xmx4G GATLING_HOME is set to /home/jenkins/lodestar/clients/gatling Generating Gatling reports locally Parsing log file(s)... Parsing log file(s) done Generating reports... ================================================================================ ---- Global Information -------------------------------------------------------- > request count 2555571 (OK=2513232 KO=42339 ) > min response time 33 (OK=39 KO=33 ) > max response time 60001 (OK=59248 KO=60001 ) > mean response time 9285 (OK=9374 KO=4001 ) > std deviation 7323 (OK=7314 KO=5713 ) > response time 50th percentile 8436 (OK=8554 KO=42 ) > response time 75th percentile 13851 (OK=13927 KO=8036 ) > response time 95th percentile 22628 (OK=22694 KO=17062 ) > response time 99th percentile 30094 (OK=30143 KO=22463 ) > mean requests/sec 700.925 (OK=689.312 KO=11.612) ---- Response Time Distribution ------------------------------------------------ > t < 800 ms 393160 ( 15%) > 800 ms <= t < 1200 ms 39894 ( 2%) > t >= 1200 ms 2080178 ( 81%) > failed 42339 ( 2%) ---- Errors -------------------------------------------------------------------- > status.find.is(200), but actually found 503 22618 (35.98%) > restLoginUsernamePasswordCallback: Failed to build request: No 20530 (32.66%) attribute named 'callbacks' is defined > status.find.is(200), but actually found 502 16947 (26.96%) > status.find.is(200), but actually found 401 2737 ( 4.35%) > Request timeout to openam-ema-perf-benchmark.forgeblocks.com/3 36 ( 0.06%) 5.190.24.146:443 after 60000 ms > i.n.h.s.SslHandshakeTimeoutException: handshake timed out afte 1 ( 0.00%) r 10000ms ================================================================================ Reports generated in 0s. Please open the following file: file:///results/gatling/login-tree-350/loadtest-20240225031318339/index.html ---------------------- [25/Feb/2024 06:32:23] Create tar file /results/gatling/login-tree-350.tar.gz ---------------------- SIMULATION_PATH=/results/gatling/login-tree-350/loadtest-20240225031318339 RESULT_FOLDER=login-tree-350 RESULT_PATH=/results/gatling/login-tree-350 PWD=/results/gatling/login-tree-350/loadtest-20240225031318339 [cmd] tar -czvf /results/gatling/login-tree-350.tar.gz --exclude=simulation.log * index.html js/ js/ellipsis.js js/unpack.js js/global_stats.json js/highstock.js js/gatling.js js/bootstrap.min.js js/theme.js js/menu.js js/all_sessions.js js/highcharts-more.js js/stats.js js/jquery-3.5.1.min.js js/assertions.json js/stats.json js/assertions.xml req_restlogininitia-e2b9b.html req_restloginprogre-9fc93.html req_restloginuserna-d98ea.html style/ style/stat-l-temps.png style/stat-l-roue.png style/favicon.ico style/sorted-up.png style/little_arrow_right.png style/logo-enterprise.svg style/sorted-down.png style/sortable.png style/stat-fleche-bas.png style/arrow_right_black.png style/logo.svg style/arrow_down_black.png style/style.css style/bootstrap.min.css style/arrow_right.png style/arrow_down.png ________________________________________________________________________________ [2024-02-25 06:32:35] Reporting-login-tree-350 step10 : Check stdout log ________________________________________________________________________________ PASS: no error found in stdout log ________________________________________________________________________________ [2024-02-25 06:32:35] Reporting-login-tree-350 step11 : Print stderr order ________________________________________________________________________________ [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- Checking pod: overseer-0-86b9999bcb-xv8f9 for file exists, if not wait. [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-86b9999bcb-xv8f9 --stdin -- bash -c 'if [ -f "/results/orders/Reporting-login-tree-350/stderr.txt" ]; then echo "/results/orders/Reporting-login-tree-350/stderr.txt file found"; exit 0; else echo "/results/orders/Reporting-login-tree-350/stderr.txt file not found"; exit 1; fi' | grep "/results/orders/Reporting-login-tree-350/stderr.txt file found" [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found [loop_until]: OK (rc = 0) --- stdout --- /results/orders/Reporting-login-tree-350/stderr.txt file found --- stderr --- [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec overseer-0-86b9999bcb-xv8f9 --stdin -- du -k /results/orders/Reporting-login-tree-350/stderr.txt [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- 0 /results/orders/Reporting-login-tree-350/stderr.txt --- stderr --- - Download file (size 0k) [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- source : /results/orders/Reporting-login-tree-350/stderr.txt @ overseer-0-86b9999bcb-xv8f9 destination : /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/stderr.txt [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1 [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found [loop_until]: OK (rc = 0) --- stdout --- overseer-0-86b9999bcb-xv8f9 --- stderr --- [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp --retries=100 overseer-0-86b9999bcb-xv8f9:/results/orders/Reporting-login-tree-350/stderr.txt /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/stderr.txt [loop_until]: (max_time=-1, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- tar: Removing leading `/' from member names --- stderr --- [print_head_tail]: Input file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/08_Verification/16_Reporting-login-tree-350/stderr.txt is small (0 lines), printing all lines: ----- output ----- ________________________________________________________________________________ [2024-02-25 06:32:39] Reporting-login-tree-350 step12 : Check stderr log ________________________________________________________________________________ PASS: no error found in stderr log ________________________________________________________________________________ [2024-02-25 06:32:39] Reporting-login-tree-350 step13 : Analyze graphs for spikes ________________________________________________________________________________ Analyze Gatling graphs for spikes checking parameters open overall stats JSON file checking simulation request restLoginInitiate [warn] responseTimePercentiles was not found in /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/07_scenario/08_login-tree-350/gatling/req_restlogininitia-e2b9b.html - cannot check for spikes checking simulation request restLoginUsernamePasswordCallback [warn] responseTimePercentiles was not found in /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/07_scenario/08_login-tree-350/gatling/req_restloginuserna-d98ea.html - cannot check for spikes checking simulation request restloginProgressiveProfileCallback [warn] responseTimePercentiles was not found in /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/07_scenario/08_login-tree-350/gatling/req_restloginprogre-9fc93.html - cannot check for spikes no spikes found ________________________________________________________________________________ [2024-02-25 06:32:39] Reporting-login-tree-350 step14 : Collect gatling simulation data ________________________________________________________________________________ collected stats : { "global": { "name": "All Requests", "numberOfRequests": { "total": 2555571, "ok": 2513232, "ko": 42339 }, "minResponseTime": { "total": 33, "ok": 39, "ko": 33 }, "maxResponseTime": { "total": 60001, "ok": 59248, "ko": 60001 }, "meanResponseTime": { "total": 9285, "ok": 9374, "ko": 4001 }, "standardDeviation": { "total": 7323, "ok": 7314, "ko": 5713 }, "percentiles1": { "total": 8436, "ok": 8554, "ko": 42 }, "percentiles2": { "total": 13851, "ok": 13927, "ko": 8036 }, "percentiles3": { "total": 22628, "ok": 22694, "ko": 17062 }, "percentiles4": { "total": 30094, "ok": 30143, "ko": 22463 }, "group1": { "name": "t < 800 ms", "htmlName": "t < 800 ms", "count": 393160, "percentage": 15 }, "group2": { "name": "800 ms <= t < 1200 ms", "htmlName": "t >= 800 ms
t < 1200 ms", "count": 39894, "percentage": 2 }, "group3": { "name": "t >= 1200 ms", "htmlName": "t >= 1200 ms", "count": 2080178, "percentage": 81 }, "group4": { "name": "failed", "htmlName": "failed", "count": 42339, "percentage": 2 }, "meanNumberOfRequestsPerSecond": { "total": 700.924574876577, "ok": 689.3121228743829, "ko": 11.612452002194185 } }, "details": [ { "name": "restLoginInitiate", "numberOfRequests": { "total": 1264970, "ok": 1244440, "ko": 20530 }, "minResponseTime": { "total": 36, "ok": 39, "ko": 36 }, "maxResponseTime": { "total": 60001, "ok": 53065, "ko": 60001 }, "meanResponseTime": { "total": 6225, "ok": 6267, "ko": 3683 }, "standardDeviation": { "total": 6065, "ok": 6079, "ko": 4427 }, "percentiles1": { "total": 5164, "ok": 5221, "ko": 3788 }, "percentiles2": { "total": 9388, "ok": 9420, "ko": 8039 }, "percentiles3": { "total": 18025, "ok": 18148, "ko": 12042 }, "percentiles4": { "total": 25383, "ok": 25436, "ko": 12045 }, "group1": { "name": "t < 800 ms", "htmlName": "t < 800 ms", "count": 319753, "percentage": 25 }, "group2": { "name": "800 ms <= t < 1200 ms", "htmlName": "t >= 800 ms
t < 1200 ms", "count": 22961, "percentage": 2 }, "group3": { "name": "t >= 1200 ms", "htmlName": "t >= 1200 ms", "count": 901726, "percentage": 71 }, "group4": { "name": "failed", "htmlName": "failed", "count": 20530, "percentage": 2 }, "meanNumberOfRequestsPerSecond": { "total": 346.947339550192, "ok": 341.3165112452002, "ko": 5.630828304991772 }, "spikes": [] }, { "name": "restLoginUsernamePasswordCallback", "numberOfRequests": { "total": 1244440, "ok": 1223513, "ko": 20927 }, "minResponseTime": { "total": 33, "ok": 100, "ko": 33 }, "maxResponseTime": { "total": 60001, "ok": 59248, "ko": 60001 }, "meanResponseTime": { "total": 12292, "ok": 12429, "ko": 4262 }, "standardDeviation": { "total": 7212, "ok": 7142, "ko": 6722 }, "percentiles1": { "total": 11818, "ok": 11906, "ko": 37 }, "percentiles2": { "total": 16430, "ok": 16491, "ko": 7704 }, "percentiles3": { "total": 25030, "ok": 25095, "ko": 18680 }, "percentiles4": { "total": 32848, "ok": 32877, "ko": 23563 }, "group1": { "name": "t < 800 ms", "htmlName": "t < 800 ms", "count": 71007, "percentage": 6 }, "group2": { "name": "800 ms <= t < 1200 ms", "htmlName": "t >= 800 ms
t < 1200 ms", "count": 16304, "percentage": 1 }, "group3": { "name": "t >= 1200 ms", "htmlName": "t >= 1200 ms", "count": 1136202, "percentage": 91 }, "group4": { "name": "failed", "htmlName": "failed", "count": 20927, "percentage": 2 }, "meanNumberOfRequestsPerSecond": { "total": 341.3165112452002, "ok": 335.5767964893033, "ko": 5.739714755896873 }, "spikes": [] }, { "name": "restloginProgressiveProfileCallback", "numberOfRequests": { "total": 46161, "ok": 45279, "ko": 882 }, "minResponseTime": { "total": 33, "ok": 137, "ko": 33 }, "maxResponseTime": { "total": 50722, "ok": 50722, "ko": 22626 }, "meanResponseTime": { "total": 12101, "ok": 12235, "ko": 5190 }, "standardDeviation": { "total": 7082, "ok": 7038, "ko": 5747 }, "percentiles1": { "total": 11563, "ok": 11660, "ko": 4036 }, "percentiles2": { "total": 16240, "ok": 16328, "ko": 8041 }, "percentiles3": { "total": 24643, "ok": 24766, "ko": 15956 }, "percentiles4": { "total": 32498, "ok": 32581, "ko": 21120 }, "group1": { "name": "t < 800 ms", "htmlName": "t < 800 ms", "count": 2400, "percentage": 5 }, "group2": { "name": "800 ms <= t < 1200 ms", "htmlName": "t >= 800 ms
t < 1200 ms", "count": 629, "percentage": 1 }, "group3": { "name": "t >= 1200 ms", "htmlName": "t >= 1200 ms", "count": 42250, "percentage": 92 }, "group4": { "name": "failed", "htmlName": "failed", "count": 882, "percentage": 2 }, "meanNumberOfRequestsPerSecond": { "total": 12.66072408118486, "ok": 12.41881513987932, "ko": 0.24190894130554033 }, "spikes": [] } ] } { "concurrency": null, "generic_stats": { "avg_num_of_requests_per_second": 701, "avg_response_time": 9285, "max_response_time": 59248, "min_response_time": 39, "num_requests": 2555571, "num_requests_pass": 2513232, "num_requests_percent_pass": 98.343 }, "options": "; testCase:idc.login; testMode:open; throughput:350; rampup-duration:30; duration:3600", "stats": { "details": [ { "group1": { "count": 319753, "htmlName": "t < 800 ms", "name": "t < 800 ms", "percentage": 25 }, "group2": { "count": 22961, "htmlName": "t >= 800 ms
t < 1200 ms", "name": "800 ms <= t < 1200 ms", "percentage": 2 }, "group3": { "count": 901726, "htmlName": "t >= 1200 ms", "name": "t >= 1200 ms", "percentage": 71 }, "group4": { "count": 20530, "htmlName": "failed", "name": "failed", "percentage": 2 }, "maxResponseTime": { "ko": 60001, "ok": 53065, "total": 60001 }, "meanNumberOfRequestsPerSecond": { "ko": 5.630828304991772, "ok": 341.3165112452002, "total": 346.947339550192 }, "meanResponseTime": { "ko": 3683, "ok": 6267, "total": 6225 }, "minResponseTime": { "ko": 36, "ok": 39, "total": 36 }, "name": "restLoginInitiate", "numberOfRequests": { "ko": 20530, "ok": 1244440, "total": 1264970 }, "percentiles1": { "ko": 3788, "ok": 5221, "total": 5164 }, "percentiles2": { "ko": 8039, "ok": 9420, "total": 9388 }, "percentiles3": { "ko": 12042, "ok": 18148, "total": 18025 }, "percentiles4": { "ko": 12045, "ok": 25436, "total": 25383 }, "spikes": [], "standardDeviation": { "ko": 4427, "ok": 6079, "total": 6065 } }, { "group1": { "count": 71007, "htmlName": "t < 800 ms", "name": "t < 800 ms", "percentage": 6 }, "group2": { "count": 16304, "htmlName": "t >= 800 ms
t < 1200 ms", "name": "800 ms <= t < 1200 ms", "percentage": 1 }, "group3": { "count": 1136202, "htmlName": "t >= 1200 ms", "name": "t >= 1200 ms", "percentage": 91 }, "group4": { "count": 20927, "htmlName": "failed", "name": "failed", "percentage": 2 }, "maxResponseTime": { "ko": 60001, "ok": 59248, "total": 60001 }, "meanNumberOfRequestsPerSecond": { "ko": 5.739714755896873, "ok": 335.5767964893033, "total": 341.3165112452002 }, "meanResponseTime": { "ko": 4262, "ok": 12429, "total": 12292 }, "minResponseTime": { "ko": 33, "ok": 100, "total": 33 }, "name": "restLoginUsernamePasswordCallback", "numberOfRequests": { "ko": 20927, "ok": 1223513, "total": 1244440 }, "percentiles1": { "ko": 37, "ok": 11906, "total": 11818 }, "percentiles2": { "ko": 7704, "ok": 16491, "total": 16430 }, "percentiles3": { "ko": 18680, "ok": 25095, "total": 25030 }, "percentiles4": { "ko": 23563, "ok": 32877, "total": 32848 }, "spikes": [], "standardDeviation": { "ko": 6722, "ok": 7142, "total": 7212 } }, { "group1": { "count": 2400, "htmlName": "t < 800 ms", "name": "t < 800 ms", "percentage": 5 }, "group2": { "count": 629, "htmlName": "t >= 800 ms
t < 1200 ms", "name": "800 ms <= t < 1200 ms", "percentage": 1 }, "group3": { "count": 42250, "htmlName": "t >= 1200 ms", "name": "t >= 1200 ms", "percentage": 92 }, "group4": { "count": 882, "htmlName": "failed", "name": "failed", "percentage": 2 }, "maxResponseTime": { "ko": 22626, "ok": 50722, "total": 50722 }, "meanNumberOfRequestsPerSecond": { "ko": 0.24190894130554033, "ok": 12.41881513987932, "total": 12.66072408118486 }, "meanResponseTime": { "ko": 5190, "ok": 12235, "total": 12101 }, "minResponseTime": { "ko": 33, "ok": 137, "total": 33 }, "name": "restloginProgressiveProfileCallback", "numberOfRequests": { "ko": 882, "ok": 45279, "total": 46161 }, "percentiles1": { "ko": 4036, "ok": 11660, "total": 11563 }, "percentiles2": { "ko": 8041, "ok": 16328, "total": 16240 }, "percentiles3": { "ko": 15956, "ok": 24766, "total": 24643 }, "percentiles4": { "ko": 21120, "ok": 32581, "total": 32498 }, "spikes": [], "standardDeviation": { "ko": 5747, "ok": 7038, "total": 7082 } } ], "global": { "group1": { "count": 393160, "htmlName": "t < 800 ms", "name": "t < 800 ms", "percentage": 15 }, "group2": { "count": 39894, "htmlName": "t >= 800 ms
t < 1200 ms", "name": "800 ms <= t < 1200 ms", "percentage": 2 }, "group3": { "count": 2080178, "htmlName": "t >= 1200 ms", "name": "t >= 1200 ms", "percentage": 81 }, "group4": { "count": 42339, "htmlName": "failed", "name": "failed", "percentage": 2 }, "maxResponseTime": { "ko": 60001, "ok": 59248, "total": 60001 }, "meanNumberOfRequestsPerSecond": { "ko": 11.612452002194185, "ok": 689.3121228743829, "total": 700.924574876577 }, "meanResponseTime": { "ko": 4001, "ok": 9374, "total": 9285 }, "minResponseTime": { "ko": 33, "ok": 39, "total": 33 }, "name": "All Requests", "numberOfRequests": { "ko": 42339, "ok": 2513232, "total": 2555571 }, "percentiles1": { "ko": 42, "ok": 8554, "total": 8436 }, "percentiles2": { "ko": 8036, "ok": 13927, "total": 13851 }, "percentiles3": { "ko": 17062, "ok": 22694, "total": 22628 }, "percentiles4": { "ko": 22463, "ok": 30143, "total": 30094 }, "standardDeviation": { "ko": 5713, "ok": 7314, "total": 7323 } } }, "target_hostname": "openam-ema-perf-benchmark.forgeblocks.com", "tool_name": "gatling" } ________________________________________________________________________________ [2024-02-25 06:32:39] Reporting-login-tree-350 step15 : Remove gatling task's stdout file ________________________________________________________________________________ Cleanup of Gatling stdout file /mnt/disks/data/xslou/lodestar-fork/results/pyrock/idc_login_tree/07_scenario/08_login-tree-350/stdout.txt Compressing Gatling stdout.txt file.. ________________________________________________________________________________ [2024-02-25 06:32:43] Reporting-login-tree-350 post : Check output ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped