--Task--
name: Discover_all_components
enabled: True
class_name: DiscoverComponentsTask
source_name: controller
source_namespace: >default<
target_name: controller
target_namespace: >default<
start: 0.0
stop: None
timeout: no timeout
loop: False
interval: None
dependencies: []
wait_for: []
preceding_task: None
options: {}
group_name: None
Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock
________________________________________________________________________________
[2024-04-16 21:06:52] Discover_all_components pre : Initialising task parameters
________________________________________________________________________________
task will be executed on controller (localhost)
________________________________________________________________________________
[2024-04-16 21:06:52] Discover_all_components step1 : Discover components
________________________________________________________________________________
2024-04-16 21:06:52,671 INFO Deployment was skipped.
2024-04-16 21:06:53,014 INFO
2024-04-16 21:06:53,014 INFO ************************************* Creating deployment *************************************
2024-04-16 21:06:53,014 INFO Creating normal (forgeops) type deployment for deployment: stack
2024-04-16 21:06:53,030 INFO
2024-04-16 21:06:53,030 INFO ------- Custom component configuration present. Loading values -------
2024-04-16 21:06:53,030 INFO
2024-04-16 21:06:53,031 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds cts
2024-04-16 21:06:53,104 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:06:53,104 DEBUG --- stdout ---
2024-04-16 21:06:53,104 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts
2024-04-16 21:06:53,104 DEBUG --- stderr ---
2024-04-16 21:06:53,104 DEBUG
2024-04-16 21:06:53,104 INFO
2024-04-16 21:06:53,104 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903
2024-04-16 21:06:53,105 INFO
2024-04-16 21:06:53,105 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds idrepo
2024-04-16 21:06:53,179 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:06:53,179 DEBUG --- stdout ---
2024-04-16 21:06:53,179 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo
2024-04-16 21:06:53,179 DEBUG --- stderr ---
2024-04-16 21:06:53,179 DEBUG
2024-04-16 21:06:53,179 INFO
2024-04-16 21:06:53,180 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903
2024-04-16 21:06:53,180 INFO
2024-04-16 21:06:53,180 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker am
2024-04-16 21:06:53,253 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:06:53,253 DEBUG --- stdout ---
2024-04-16 21:06:53,253 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am
2024-04-16 21:06:53,253 DEBUG --- stderr ---
2024-04-16 21:06:53,253 DEBUG
2024-04-16 21:06:53,253 INFO
2024-04-16 21:06:53,253 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am/Dockerfile: gcr.io/forgerock-io/am-cdk/pit1:7.6.0-bbf6791c961b39ef101ebad36686ee2c3cde94bb
2024-04-16 21:06:53,254 INFO
2024-04-16 21:06:53,254 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker amster
2024-04-16 21:06:53,329 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:06:53,329 DEBUG --- stdout ---
2024-04-16 21:06:53,330 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster
2024-04-16 21:06:53,330 DEBUG --- stderr ---
2024-04-16 21:06:53,330 DEBUG
2024-04-16 21:06:53,330 INFO
2024-04-16 21:06:53,330 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster/Dockerfile: gcr.io/forgerock-io/amster/pit1:7.6.0-bbf6791c961b39ef101ebad36686ee2c3cde94bb
2024-04-16 21:06:53,330 INFO
2024-04-16 21:06:53,330 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker idm
2024-04-16 21:06:53,404 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:06:53,405 DEBUG --- stdout ---
2024-04-16 21:06:53,405 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm
2024-04-16 21:06:53,405 DEBUG --- stderr ---
2024-04-16 21:06:53,405 DEBUG
2024-04-16 21:06:53,405 INFO
2024-04-16 21:06:53,405 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm/Dockerfile: gcr.io/forgerock-io/idm-cdk/pit1:7.6.0-8554e0d1c5044f4f6c6504cd366196848874e695
2024-04-16 21:06:53,405 INFO
2024-04-16 21:06:53,405 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base end-user-ui
2024-04-16 21:06:53,478 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:06:53,478 DEBUG --- stdout ---
2024-04-16 21:06:53,478 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui
2024-04-16 21:06:53,478 DEBUG --- stderr ---
2024-04-16 21:06:53,478 DEBUG
2024-04-16 21:06:53,478 INFO
2024-04-16 21:06:53,481 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui/kustomization.yaml: gcr.io/forgerock-io/platform-enduser-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-16 21:06:53,481 INFO
2024-04-16 21:06:53,481 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base login-ui
2024-04-16 21:06:53,554 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:06:53,554 DEBUG --- stdout ---
2024-04-16 21:06:53,554 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui
2024-04-16 21:06:53,554 DEBUG --- stderr ---
2024-04-16 21:06:53,555 DEBUG
2024-04-16 21:06:53,555 INFO
2024-04-16 21:06:53,558 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui/kustomization.yaml: gcr.io/forgerock-io/platform-login-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-16 21:06:53,558 INFO
2024-04-16 21:06:53,558 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base admin-ui
2024-04-16 21:06:53,631 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:06:53,632 DEBUG --- stdout ---
2024-04-16 21:06:53,632 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui
2024-04-16 21:06:53,632 DEBUG --- stderr ---
2024-04-16 21:06:53,632 DEBUG
2024-04-16 21:06:53,632 INFO
2024-04-16 21:06:53,635 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui/kustomization.yaml: gcr.io/forgerock-io/platform-admin-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-16 21:06:53,635 INFO Loading external components from config.yaml
2024-04-16 21:06:53,755 INFO No external components defined.
2024-04-16 21:06:53,871 INFO Setting context for components to: gke_engineeringpit_us-east1-d_xlou-cdm
2024-04-16 21:06:53,871 INFO
2024-04-16 21:06:53,871 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-16 21:06:53,871 INFO
2024-04-16 21:06:53,871 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:06:53,872 INFO
2024-04-16 21:06:53,872 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:06:53,872 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:06:54,262 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:54,262 DEBUG --- stdout ---
2024-04-16 21:06:54,262 DEBUG 3
2024-04-16 21:06:54,262 DEBUG --- stderr ---
2024-04-16 21:06:54,262 DEBUG
2024-04-16 21:06:54,262 INFO
2024-04-16 21:06:54,262 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:06:54,262 INFO
2024-04-16 21:06:54,262 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-16 21:06:54,262 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:06:54,701 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-16 21:06:54,701 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:54,701 DEBUG --- stdout ---
2024-04-16 21:06:54,701 DEBUG ds-cts-0 ds-cts-1 ds-cts-2
2024-04-16 21:06:54,701 DEBUG --- stderr ---
2024-04-16 21:06:54,701 DEBUG
2024-04-16 21:06:54,701 INFO
2024-04-16 21:06:54,702 INFO -------------------- Check pod ds-cts-0 is running --------------------
2024-04-16 21:06:54,702 INFO
2024-04-16 21:06:54,702 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:06:54,702 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:06:55,061 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:06:55,062 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:55,062 DEBUG --- stdout ---
2024-04-16 21:06:55,062 DEBUG Running
2024-04-16 21:06:55,062 DEBUG --- stderr ---
2024-04-16 21:06:55,062 DEBUG
2024-04-16 21:06:55,062 INFO
2024-04-16 21:06:55,062 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:06:55,062 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:06:55,423 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:06:55,423 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:55,423 DEBUG --- stdout ---
2024-04-16 21:06:55,423 DEBUG true
2024-04-16 21:06:55,423 DEBUG --- stderr ---
2024-04-16 21:06:55,423 DEBUG
2024-04-16 21:06:55,424 INFO
2024-04-16 21:06:55,424 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.startTime}
2024-04-16 21:06:55,424 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:06:55,777 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:55,777 DEBUG --- stdout ---
2024-04-16 21:06:55,777 DEBUG 2024-04-13T22:38:33Z
2024-04-16 21:06:55,777 DEBUG --- stderr ---
2024-04-16 21:06:55,777 DEBUG
2024-04-16 21:06:55,778 INFO
2024-04-16 21:06:55,778 INFO ------------- Check pod ds-cts-0 filesystem is accessible -------------
2024-04-16 21:06:56,605 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:06:56,605 INFO
2024-04-16 21:06:56,605 INFO ------------------ Check pod ds-cts-0 restart count ------------------
2024-04-16 21:06:56,605 INFO
2024-04-16 21:06:56,605 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:06:56,605 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:06:56,963 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:56,963 DEBUG --- stdout ---
2024-04-16 21:06:56,963 DEBUG 0
2024-04-16 21:06:56,963 DEBUG --- stderr ---
2024-04-16 21:06:56,963 DEBUG
2024-04-16 21:06:56,963 INFO Pod ds-cts-0 has been restarted 0 times.
2024-04-16 21:06:56,964 INFO
2024-04-16 21:06:56,964 INFO -------------------- Check pod ds-cts-1 is running --------------------
2024-04-16 21:06:56,964 INFO
2024-04-16 21:06:56,964 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:06:56,964 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:06:57,323 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:06:57,324 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:57,324 DEBUG --- stdout ---
2024-04-16 21:06:57,324 DEBUG Running
2024-04-16 21:06:57,324 DEBUG --- stderr ---
2024-04-16 21:06:57,324 DEBUG
2024-04-16 21:06:57,324 INFO
2024-04-16 21:06:57,324 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:06:57,324 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:06:57,679 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:06:57,679 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:57,679 DEBUG --- stdout ---
2024-04-16 21:06:57,679 DEBUG true
2024-04-16 21:06:57,679 DEBUG --- stderr ---
2024-04-16 21:06:57,679 DEBUG
2024-04-16 21:06:57,679 INFO
2024-04-16 21:06:57,680 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.startTime}
2024-04-16 21:06:57,680 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:06:58,050 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:58,050 DEBUG --- stdout ---
2024-04-16 21:06:58,050 DEBUG 2024-04-15T16:51:45Z
2024-04-16 21:06:58,050 DEBUG --- stderr ---
2024-04-16 21:06:58,050 DEBUG
2024-04-16 21:06:58,051 INFO
2024-04-16 21:06:58,051 INFO ------------- Check pod ds-cts-1 filesystem is accessible -------------
2024-04-16 21:06:58,966 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:06:58,967 INFO
2024-04-16 21:06:58,967 INFO ------------------ Check pod ds-cts-1 restart count ------------------
2024-04-16 21:06:58,967 INFO
2024-04-16 21:06:58,967 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:06:58,967 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:06:59,328 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:59,328 DEBUG --- stdout ---
2024-04-16 21:06:59,328 DEBUG 0
2024-04-16 21:06:59,328 DEBUG --- stderr ---
2024-04-16 21:06:59,328 DEBUG
2024-04-16 21:06:59,328 INFO Pod ds-cts-1 has been restarted 0 times.
2024-04-16 21:06:59,328 INFO
2024-04-16 21:06:59,328 INFO -------------------- Check pod ds-cts-2 is running --------------------
2024-04-16 21:06:59,329 INFO
2024-04-16 21:06:59,329 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:06:59,329 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:06:59,682 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:06:59,682 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:06:59,682 DEBUG --- stdout ---
2024-04-16 21:06:59,683 DEBUG Running
2024-04-16 21:06:59,683 DEBUG --- stderr ---
2024-04-16 21:06:59,683 DEBUG
2024-04-16 21:06:59,683 INFO
2024-04-16 21:06:59,683 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:06:59,683 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:00,042 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:00,042 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:00,043 DEBUG --- stdout ---
2024-04-16 21:07:00,043 DEBUG true
2024-04-16 21:07:00,043 DEBUG --- stderr ---
2024-04-16 21:07:00,043 DEBUG
2024-04-16 21:07:00,044 INFO
2024-04-16 21:07:00,044 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.startTime}
2024-04-16 21:07:00,044 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:00,402 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:00,402 DEBUG --- stdout ---
2024-04-16 21:07:00,402 DEBUG 2024-04-15T16:52:10Z
2024-04-16 21:07:00,402 DEBUG --- stderr ---
2024-04-16 21:07:00,402 DEBUG
2024-04-16 21:07:00,403 INFO
2024-04-16 21:07:00,403 INFO ------------- Check pod ds-cts-2 filesystem is accessible -------------
2024-04-16 21:07:01,222 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:01,222 INFO
2024-04-16 21:07:01,222 INFO ------------------ Check pod ds-cts-2 restart count ------------------
2024-04-16 21:07:01,222 INFO
2024-04-16 21:07:01,222 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:01,222 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:01,583 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:01,583 DEBUG --- stdout ---
2024-04-16 21:07:01,583 DEBUG 0
2024-04-16 21:07:01,583 DEBUG --- stderr ---
2024-04-16 21:07:01,583 DEBUG
2024-04-16 21:07:01,583 INFO Pod ds-cts-2 has been restarted 0 times.
2024-04-16 21:07:01,583 INFO
2024-04-16 21:07:01,583 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-16 21:07:01,583 INFO
2024-04-16 21:07:01,583 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:01,584 INFO
2024-04-16 21:07:01,584 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:01,584 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:01,953 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:01,953 DEBUG --- stdout ---
2024-04-16 21:07:01,953 DEBUG 3
2024-04-16 21:07:01,953 DEBUG --- stderr ---
2024-04-16 21:07:01,953 DEBUG
2024-04-16 21:07:01,953 INFO
2024-04-16 21:07:01,953 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:07:01,953 INFO
2024-04-16 21:07:01,953 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-16 21:07:01,953 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:07:02,387 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-16 21:07:02,387 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:02,387 DEBUG --- stdout ---
2024-04-16 21:07:02,387 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2
2024-04-16 21:07:02,387 DEBUG --- stderr ---
2024-04-16 21:07:02,387 DEBUG
2024-04-16 21:07:02,387 INFO
2024-04-16 21:07:02,387 INFO ------------------ Check pod ds-idrepo-0 is running ------------------
2024-04-16 21:07:02,387 INFO
2024-04-16 21:07:02,387 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:02,388 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:02,750 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:02,750 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:02,750 DEBUG --- stdout ---
2024-04-16 21:07:02,750 DEBUG Running
2024-04-16 21:07:02,750 DEBUG --- stderr ---
2024-04-16 21:07:02,750 DEBUG
2024-04-16 21:07:02,751 INFO
2024-04-16 21:07:02,751 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:02,751 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:03,112 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:03,112 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:03,112 DEBUG --- stdout ---
2024-04-16 21:07:03,113 DEBUG true
2024-04-16 21:07:03,113 DEBUG --- stderr ---
2024-04-16 21:07:03,113 DEBUG
2024-04-16 21:07:03,113 INFO
2024-04-16 21:07:03,113 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.startTime}
2024-04-16 21:07:03,113 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:03,484 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:03,484 DEBUG --- stdout ---
2024-04-16 21:07:03,484 DEBUG 2024-04-13T22:38:34Z
2024-04-16 21:07:03,484 DEBUG --- stderr ---
2024-04-16 21:07:03,484 DEBUG
2024-04-16 21:07:03,485 INFO
2024-04-16 21:07:03,485 INFO ----------- Check pod ds-idrepo-0 filesystem is accessible -----------
2024-04-16 21:07:04,317 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:04,317 INFO
2024-04-16 21:07:04,317 INFO ----------------- Check pod ds-idrepo-0 restart count -----------------
2024-04-16 21:07:04,317 INFO
2024-04-16 21:07:04,317 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:04,317 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:04,677 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:04,677 DEBUG --- stdout ---
2024-04-16 21:07:04,677 DEBUG 0
2024-04-16 21:07:04,677 DEBUG --- stderr ---
2024-04-16 21:07:04,677 DEBUG
2024-04-16 21:07:04,677 INFO Pod ds-idrepo-0 has been restarted 0 times.
2024-04-16 21:07:04,678 INFO
2024-04-16 21:07:04,678 INFO ------------------ Check pod ds-idrepo-1 is running ------------------
2024-04-16 21:07:04,678 INFO
2024-04-16 21:07:04,678 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:04,678 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:05,084 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:05,084 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:05,086 DEBUG --- stdout ---
2024-04-16 21:07:05,086 DEBUG Running
2024-04-16 21:07:05,086 DEBUG --- stderr ---
2024-04-16 21:07:05,086 DEBUG
2024-04-16 21:07:05,086 INFO
2024-04-16 21:07:05,086 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:05,086 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:05,444 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:05,445 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:05,445 DEBUG --- stdout ---
2024-04-16 21:07:05,445 DEBUG true
2024-04-16 21:07:05,445 DEBUG --- stderr ---
2024-04-16 21:07:05,445 DEBUG
2024-04-16 21:07:05,445 INFO
2024-04-16 21:07:05,445 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.startTime}
2024-04-16 21:07:05,445 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:05,803 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:05,804 DEBUG --- stdout ---
2024-04-16 21:07:05,804 DEBUG 2024-04-15T20:48:06Z
2024-04-16 21:07:05,804 DEBUG --- stderr ---
2024-04-16 21:07:05,804 DEBUG
2024-04-16 21:07:05,804 INFO
2024-04-16 21:07:05,804 INFO ----------- Check pod ds-idrepo-1 filesystem is accessible -----------
2024-04-16 21:07:06,624 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:06,625 INFO
2024-04-16 21:07:06,625 INFO ----------------- Check pod ds-idrepo-1 restart count -----------------
2024-04-16 21:07:06,625 INFO
2024-04-16 21:07:06,625 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:06,625 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:06,996 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:06,997 DEBUG --- stdout ---
2024-04-16 21:07:06,997 DEBUG 0
2024-04-16 21:07:06,997 DEBUG --- stderr ---
2024-04-16 21:07:06,997 DEBUG
2024-04-16 21:07:06,997 INFO Pod ds-idrepo-1 has been restarted 0 times.
2024-04-16 21:07:06,997 INFO
2024-04-16 21:07:06,997 INFO ------------------ Check pod ds-idrepo-2 is running ------------------
2024-04-16 21:07:06,997 INFO
2024-04-16 21:07:06,997 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:06,997 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:07,386 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:07,386 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:07,386 DEBUG --- stdout ---
2024-04-16 21:07:07,386 DEBUG Running
2024-04-16 21:07:07,387 DEBUG --- stderr ---
2024-04-16 21:07:07,387 DEBUG
2024-04-16 21:07:07,387 INFO
2024-04-16 21:07:07,387 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:07,387 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:07,741 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:07,741 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:07,741 DEBUG --- stdout ---
2024-04-16 21:07:07,741 DEBUG true
2024-04-16 21:07:07,741 DEBUG --- stderr ---
2024-04-16 21:07:07,741 DEBUG
2024-04-16 21:07:07,741 INFO
2024-04-16 21:07:07,742 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.startTime}
2024-04-16 21:07:07,742 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:08,102 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:08,103 DEBUG --- stdout ---
2024-04-16 21:07:08,103 DEBUG 2024-04-15T20:48:42Z
2024-04-16 21:07:08,103 DEBUG --- stderr ---
2024-04-16 21:07:08,103 DEBUG
2024-04-16 21:07:08,103 INFO
2024-04-16 21:07:08,104 INFO ----------- Check pod ds-idrepo-2 filesystem is accessible -----------
2024-04-16 21:07:08,943 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:08,944 INFO
2024-04-16 21:07:08,944 INFO ----------------- Check pod ds-idrepo-2 restart count -----------------
2024-04-16 21:07:08,944 INFO
2024-04-16 21:07:08,944 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:08,944 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:09,314 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:09,314 DEBUG --- stdout ---
2024-04-16 21:07:09,314 DEBUG 0
2024-04-16 21:07:09,314 DEBUG --- stderr ---
2024-04-16 21:07:09,314 DEBUG
2024-04-16 21:07:09,314 INFO Pod ds-idrepo-2 has been restarted 0 times.
2024-04-16 21:07:09,314 INFO
2024-04-16 21:07:09,315 INFO ****************************** Initializing component pods for AM ******************************
2024-04-16 21:07:09,315 INFO
2024-04-16 21:07:09,315 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:09,315 INFO
2024-04-16 21:07:09,315 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:09,315 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:09,717 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:09,717 DEBUG --- stdout ---
2024-04-16 21:07:09,717 DEBUG 3
2024-04-16 21:07:09,717 DEBUG --- stderr ---
2024-04-16 21:07:09,717 DEBUG
2024-04-16 21:07:09,717 INFO
2024-04-16 21:07:09,717 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:07:09,718 INFO
2024-04-16 21:07:09,718 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-16 21:07:09,718 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:07:10,167 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-16 21:07:10,171 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:10,172 DEBUG --- stdout ---
2024-04-16 21:07:10,172 DEBUG am-555f4db595-4lbss am-555f4db595-fz8sn am-555f4db595-kzxsp
2024-04-16 21:07:10,172 DEBUG --- stderr ---
2024-04-16 21:07:10,172 DEBUG
2024-04-16 21:07:10,172 INFO
2024-04-16 21:07:10,172 INFO -------------- Check pod am-555f4db595-4lbss is running --------------
2024-04-16 21:07:10,173 INFO
2024-04-16 21:07:10,173 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-555f4db595-4lbss -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:10,173 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:10,540 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:10,540 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:10,540 DEBUG --- stdout ---
2024-04-16 21:07:10,540 DEBUG Running
2024-04-16 21:07:10,540 DEBUG --- stderr ---
2024-04-16 21:07:10,540 DEBUG
2024-04-16 21:07:10,540 INFO
2024-04-16 21:07:10,540 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-555f4db595-4lbss -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:10,540 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:10,911 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:10,911 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:10,911 DEBUG --- stdout ---
2024-04-16 21:07:10,911 DEBUG true
2024-04-16 21:07:10,911 DEBUG --- stderr ---
2024-04-16 21:07:10,911 DEBUG
2024-04-16 21:07:10,912 INFO
2024-04-16 21:07:10,912 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-555f4db595-4lbss --output jsonpath={.status.startTime}
2024-04-16 21:07:10,912 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:11,278 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:11,278 DEBUG --- stdout ---
2024-04-16 21:07:11,279 DEBUG 2024-04-16T17:09:56Z
2024-04-16 21:07:11,279 DEBUG --- stderr ---
2024-04-16 21:07:11,279 DEBUG
2024-04-16 21:07:11,279 INFO
2024-04-16 21:07:11,279 INFO ------- Check pod am-555f4db595-4lbss filesystem is accessible -------
2024-04-16 21:07:12,120 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:12,120 INFO
2024-04-16 21:07:12,120 INFO ------------- Check pod am-555f4db595-4lbss restart count -------------
2024-04-16 21:07:12,120 INFO
2024-04-16 21:07:12,120 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-555f4db595-4lbss --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:12,120 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:12,485 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:12,485 DEBUG --- stdout ---
2024-04-16 21:07:12,485 DEBUG 0
2024-04-16 21:07:12,485 DEBUG --- stderr ---
2024-04-16 21:07:12,485 DEBUG
2024-04-16 21:07:12,485 INFO Pod am-555f4db595-4lbss has been restarted 0 times.
2024-04-16 21:07:12,485 INFO
2024-04-16 21:07:12,485 INFO -------------- Check pod am-555f4db595-fz8sn is running --------------
2024-04-16 21:07:12,485 INFO
2024-04-16 21:07:12,486 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-555f4db595-fz8sn -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:12,486 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:12,854 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:12,855 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:12,855 DEBUG --- stdout ---
2024-04-16 21:07:12,855 DEBUG Running
2024-04-16 21:07:12,855 DEBUG --- stderr ---
2024-04-16 21:07:12,855 DEBUG
2024-04-16 21:07:12,855 INFO
2024-04-16 21:07:12,855 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-555f4db595-fz8sn -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:12,855 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:13,227 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:13,227 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:13,227 DEBUG --- stdout ---
2024-04-16 21:07:13,227 DEBUG true
2024-04-16 21:07:13,227 DEBUG --- stderr ---
2024-04-16 21:07:13,227 DEBUG
2024-04-16 21:07:13,228 INFO
2024-04-16 21:07:13,228 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-555f4db595-fz8sn --output jsonpath={.status.startTime}
2024-04-16 21:07:13,228 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:13,588 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:13,588 DEBUG --- stdout ---
2024-04-16 21:07:13,589 DEBUG 2024-04-16T16:27:42Z
2024-04-16 21:07:13,589 DEBUG --- stderr ---
2024-04-16 21:07:13,589 DEBUG
2024-04-16 21:07:13,589 INFO
2024-04-16 21:07:13,589 INFO ------- Check pod am-555f4db595-fz8sn filesystem is accessible -------
2024-04-16 21:07:14,419 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:14,420 INFO
2024-04-16 21:07:14,420 INFO ------------- Check pod am-555f4db595-fz8sn restart count -------------
2024-04-16 21:07:14,420 INFO
2024-04-16 21:07:14,420 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-555f4db595-fz8sn --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:14,420 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:14,780 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:14,780 DEBUG --- stdout ---
2024-04-16 21:07:14,780 DEBUG 0
2024-04-16 21:07:14,780 DEBUG --- stderr ---
2024-04-16 21:07:14,780 DEBUG
2024-04-16 21:07:14,780 INFO Pod am-555f4db595-fz8sn has been restarted 0 times.
2024-04-16 21:07:14,781 INFO
2024-04-16 21:07:14,781 INFO -------------- Check pod am-555f4db595-kzxsp is running --------------
2024-04-16 21:07:14,781 INFO
2024-04-16 21:07:14,781 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-555f4db595-kzxsp -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:14,781 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:15,141 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:15,141 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:15,141 DEBUG --- stdout ---
2024-04-16 21:07:15,142 DEBUG Running
2024-04-16 21:07:15,142 DEBUG --- stderr ---
2024-04-16 21:07:15,142 DEBUG
2024-04-16 21:07:15,142 INFO
2024-04-16 21:07:15,142 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-555f4db595-kzxsp -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:15,142 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:15,496 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:15,497 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:15,498 DEBUG --- stdout ---
2024-04-16 21:07:15,498 DEBUG true
2024-04-16 21:07:15,498 DEBUG --- stderr ---
2024-04-16 21:07:15,498 DEBUG
2024-04-16 21:07:15,499 INFO
2024-04-16 21:07:15,499 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-555f4db595-kzxsp --output jsonpath={.status.startTime}
2024-04-16 21:07:15,499 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:15,858 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:15,858 DEBUG --- stdout ---
2024-04-16 21:07:15,859 DEBUG 2024-04-16T17:09:56Z
2024-04-16 21:07:15,859 DEBUG --- stderr ---
2024-04-16 21:07:15,859 DEBUG
2024-04-16 21:07:15,859 INFO
2024-04-16 21:07:15,859 INFO ------- Check pod am-555f4db595-kzxsp filesystem is accessible -------
2024-04-16 21:07:16,680 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:16,680 INFO
2024-04-16 21:07:16,680 INFO ------------- Check pod am-555f4db595-kzxsp restart count -------------
2024-04-16 21:07:16,681 INFO
2024-04-16 21:07:16,681 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-555f4db595-kzxsp --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:16,681 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:17,057 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:17,057 DEBUG --- stdout ---
2024-04-16 21:07:17,057 DEBUG 0
2024-04-16 21:07:17,057 DEBUG --- stderr ---
2024-04-16 21:07:17,057 DEBUG
2024-04-16 21:07:17,057 INFO Pod am-555f4db595-kzxsp has been restarted 0 times.
2024-04-16 21:07:17,057 INFO
2024-04-16 21:07:17,057 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-16 21:07:17,057 INFO
2024-04-16 21:07:17,057 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:07:17,058 INFO
2024-04-16 21:07:17,058 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-16 21:07:17,058 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:07:17,431 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:17,432 DEBUG --- stdout ---
2024-04-16 21:07:17,432 DEBUG amster-qv96p
2024-04-16 21:07:17,432 DEBUG --- stderr ---
2024-04-16 21:07:17,432 DEBUG
2024-04-16 21:07:17,432 INFO
2024-04-16 21:07:17,432 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-16 21:07:17,432 INFO
2024-04-16 21:07:17,432 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:17,433 INFO
2024-04-16 21:07:17,433 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:17,433 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:17,793 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:17,794 DEBUG --- stdout ---
2024-04-16 21:07:17,794 DEBUG 2
2024-04-16 21:07:17,794 DEBUG --- stderr ---
2024-04-16 21:07:17,794 DEBUG
2024-04-16 21:07:17,794 INFO
2024-04-16 21:07:17,794 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:07:17,794 INFO
2024-04-16 21:07:17,795 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name}` | grep 2
2024-04-16 21:07:17,795 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:07:18,171 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-16 21:07:18,171 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:18,171 DEBUG --- stdout ---
2024-04-16 21:07:18,171 DEBUG idm-b85f9d978-j5pbm idm-b85f9d978-rl76f
2024-04-16 21:07:18,171 DEBUG --- stderr ---
2024-04-16 21:07:18,171 DEBUG
2024-04-16 21:07:18,171 INFO
2024-04-16 21:07:18,171 INFO -------------- Check pod idm-b85f9d978-j5pbm is running --------------
2024-04-16 21:07:18,172 INFO
2024-04-16 21:07:18,172 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-j5pbm -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:18,172 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:18,549 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:18,549 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:18,549 DEBUG --- stdout ---
2024-04-16 21:07:18,549 DEBUG Running
2024-04-16 21:07:18,549 DEBUG --- stderr ---
2024-04-16 21:07:18,549 DEBUG
2024-04-16 21:07:18,550 INFO
2024-04-16 21:07:18,550 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-j5pbm -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:18,550 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:18,903 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:18,903 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:18,903 DEBUG --- stdout ---
2024-04-16 21:07:18,903 DEBUG true
2024-04-16 21:07:18,903 DEBUG --- stderr ---
2024-04-16 21:07:18,903 DEBUG
2024-04-16 21:07:18,903 INFO
2024-04-16 21:07:18,903 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-j5pbm --output jsonpath={.status.startTime}
2024-04-16 21:07:18,903 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:19,257 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:19,257 DEBUG --- stdout ---
2024-04-16 21:07:19,257 DEBUG 2024-04-13T22:40:34Z
2024-04-16 21:07:19,257 DEBUG --- stderr ---
2024-04-16 21:07:19,257 DEBUG
2024-04-16 21:07:19,258 INFO
2024-04-16 21:07:19,258 INFO ------- Check pod idm-b85f9d978-j5pbm filesystem is accessible -------
2024-04-16 21:07:20,072 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:20,072 INFO
2024-04-16 21:07:20,072 INFO ------------- Check pod idm-b85f9d978-j5pbm restart count -------------
2024-04-16 21:07:20,072 INFO
2024-04-16 21:07:20,072 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-j5pbm --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:20,072 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:20,427 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:20,427 DEBUG --- stdout ---
2024-04-16 21:07:20,427 DEBUG 0
2024-04-16 21:07:20,427 DEBUG --- stderr ---
2024-04-16 21:07:20,427 DEBUG
2024-04-16 21:07:20,427 INFO Pod idm-b85f9d978-j5pbm has been restarted 0 times.
2024-04-16 21:07:20,427 INFO
2024-04-16 21:07:20,427 INFO -------------- Check pod idm-b85f9d978-rl76f is running --------------
2024-04-16 21:07:20,428 INFO
2024-04-16 21:07:20,428 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-rl76f -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:20,428 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:20,809 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:20,811 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:20,811 DEBUG --- stdout ---
2024-04-16 21:07:20,811 DEBUG Running
2024-04-16 21:07:20,811 DEBUG --- stderr ---
2024-04-16 21:07:20,811 DEBUG
2024-04-16 21:07:20,811 INFO
2024-04-16 21:07:20,812 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-rl76f -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:20,812 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:21,167 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:21,167 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:21,167 DEBUG --- stdout ---
2024-04-16 21:07:21,167 DEBUG true
2024-04-16 21:07:21,167 DEBUG --- stderr ---
2024-04-16 21:07:21,167 DEBUG
2024-04-16 21:07:21,167 INFO
2024-04-16 21:07:21,167 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-rl76f --output jsonpath={.status.startTime}
2024-04-16 21:07:21,167 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:21,523 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:21,523 DEBUG --- stdout ---
2024-04-16 21:07:21,523 DEBUG 2024-04-13T22:40:34Z
2024-04-16 21:07:21,523 DEBUG --- stderr ---
2024-04-16 21:07:21,523 DEBUG
2024-04-16 21:07:21,524 INFO
2024-04-16 21:07:21,524 INFO ------- Check pod idm-b85f9d978-rl76f filesystem is accessible -------
2024-04-16 21:07:22,356 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:22,356 INFO
2024-04-16 21:07:22,356 INFO ------------- Check pod idm-b85f9d978-rl76f restart count -------------
2024-04-16 21:07:22,357 INFO
2024-04-16 21:07:22,357 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-rl76f --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:22,357 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:22,727 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:22,727 DEBUG --- stdout ---
2024-04-16 21:07:22,727 DEBUG 0
2024-04-16 21:07:22,727 DEBUG --- stderr ---
2024-04-16 21:07:22,727 DEBUG
2024-04-16 21:07:22,727 INFO Pod idm-b85f9d978-rl76f has been restarted 0 times.
2024-04-16 21:07:22,727 INFO
2024-04-16 21:07:22,727 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-16 21:07:22,727 INFO
2024-04-16 21:07:22,727 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:22,728 INFO
2024-04-16 21:07:22,728 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:22,728 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:23,032 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:23,032 DEBUG --- stdout ---
2024-04-16 21:07:23,032 DEBUG 1
2024-04-16 21:07:23,032 DEBUG --- stderr ---
2024-04-16 21:07:23,032 DEBUG
2024-04-16 21:07:23,032 INFO
2024-04-16 21:07:23,032 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:07:23,032 INFO
2024-04-16 21:07:23,032 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-16 21:07:23,032 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:07:23,336 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-16 21:07:23,336 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:23,336 DEBUG --- stdout ---
2024-04-16 21:07:23,336 DEBUG end-user-ui-58878d7668-zg4h2
2024-04-16 21:07:23,336 DEBUG --- stderr ---
2024-04-16 21:07:23,336 DEBUG
2024-04-16 21:07:23,336 INFO
2024-04-16 21:07:23,336 INFO ---------- Check pod end-user-ui-58878d7668-zg4h2 is running ----------
2024-04-16 21:07:23,336 INFO
2024-04-16 21:07:23,336 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-58878d7668-zg4h2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:23,336 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:23,662 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:23,663 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:23,663 DEBUG --- stdout ---
2024-04-16 21:07:23,663 DEBUG Running
2024-04-16 21:07:23,663 DEBUG --- stderr ---
2024-04-16 21:07:23,663 DEBUG
2024-04-16 21:07:23,663 INFO
2024-04-16 21:07:23,663 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-58878d7668-zg4h2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:23,663 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:23,961 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:23,962 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:23,962 DEBUG --- stdout ---
2024-04-16 21:07:23,962 DEBUG true
2024-04-16 21:07:23,962 DEBUG --- stderr ---
2024-04-16 21:07:23,962 DEBUG
2024-04-16 21:07:23,962 INFO
2024-04-16 21:07:23,962 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-58878d7668-zg4h2 --output jsonpath={.status.startTime}
2024-04-16 21:07:23,962 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:24,269 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:24,270 DEBUG --- stdout ---
2024-04-16 21:07:24,270 DEBUG 2024-04-13T22:41:43Z
2024-04-16 21:07:24,270 DEBUG --- stderr ---
2024-04-16 21:07:24,270 DEBUG
2024-04-16 21:07:24,270 INFO
2024-04-16 21:07:24,270 INFO --- Check pod end-user-ui-58878d7668-zg4h2 filesystem is accessible ---
2024-04-16 21:07:25,033 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:25,033 INFO
2024-04-16 21:07:25,034 INFO -------- Check pod end-user-ui-58878d7668-zg4h2 restart count --------
2024-04-16 21:07:25,034 INFO
2024-04-16 21:07:25,034 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-58878d7668-zg4h2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:25,034 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:25,331 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:25,331 DEBUG --- stdout ---
2024-04-16 21:07:25,331 DEBUG 0
2024-04-16 21:07:25,331 DEBUG --- stderr ---
2024-04-16 21:07:25,331 DEBUG
2024-04-16 21:07:25,331 INFO Pod end-user-ui-58878d7668-zg4h2 has been restarted 0 times.
2024-04-16 21:07:25,331 INFO
2024-04-16 21:07:25,331 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-16 21:07:25,331 INFO
2024-04-16 21:07:25,331 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:25,332 INFO
2024-04-16 21:07:25,332 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:25,332 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:25,632 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:25,632 DEBUG --- stdout ---
2024-04-16 21:07:25,632 DEBUG 1
2024-04-16 21:07:25,632 DEBUG --- stderr ---
2024-04-16 21:07:25,632 DEBUG
2024-04-16 21:07:25,633 INFO
2024-04-16 21:07:25,633 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:07:25,633 INFO
2024-04-16 21:07:25,633 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-16 21:07:25,633 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:07:25,931 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-16 21:07:25,931 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:25,935 DEBUG --- stdout ---
2024-04-16 21:07:25,935 DEBUG login-ui-5fdf49fc66-2wnx7
2024-04-16 21:07:25,935 DEBUG --- stderr ---
2024-04-16 21:07:25,935 DEBUG
2024-04-16 21:07:25,935 INFO
2024-04-16 21:07:25,936 INFO ----------- Check pod login-ui-5fdf49fc66-2wnx7 is running -----------
2024-04-16 21:07:25,936 INFO
2024-04-16 21:07:25,936 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5fdf49fc66-2wnx7 -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:25,936 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:26,252 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:26,253 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:26,253 DEBUG --- stdout ---
2024-04-16 21:07:26,253 DEBUG Running
2024-04-16 21:07:26,253 DEBUG --- stderr ---
2024-04-16 21:07:26,253 DEBUG
2024-04-16 21:07:26,253 INFO
2024-04-16 21:07:26,253 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5fdf49fc66-2wnx7 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:26,253 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:26,549 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:26,549 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:26,549 DEBUG --- stdout ---
2024-04-16 21:07:26,549 DEBUG true
2024-04-16 21:07:26,549 DEBUG --- stderr ---
2024-04-16 21:07:26,549 DEBUG
2024-04-16 21:07:26,550 INFO
2024-04-16 21:07:26,550 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5fdf49fc66-2wnx7 --output jsonpath={.status.startTime}
2024-04-16 21:07:26,550 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:26,860 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:26,860 DEBUG --- stdout ---
2024-04-16 21:07:26,861 DEBUG 2024-04-13T22:41:43Z
2024-04-16 21:07:26,861 DEBUG --- stderr ---
2024-04-16 21:07:26,861 DEBUG
2024-04-16 21:07:26,861 INFO
2024-04-16 21:07:26,861 INFO ---- Check pod login-ui-5fdf49fc66-2wnx7 filesystem is accessible ----
2024-04-16 21:07:27,627 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:27,627 INFO
2024-04-16 21:07:27,627 INFO ---------- Check pod login-ui-5fdf49fc66-2wnx7 restart count ----------
2024-04-16 21:07:27,628 INFO
2024-04-16 21:07:27,628 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5fdf49fc66-2wnx7 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:27,628 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:27,938 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:27,938 DEBUG --- stdout ---
2024-04-16 21:07:27,938 DEBUG 0
2024-04-16 21:07:27,938 DEBUG --- stderr ---
2024-04-16 21:07:27,938 DEBUG
2024-04-16 21:07:27,938 INFO Pod login-ui-5fdf49fc66-2wnx7 has been restarted 0 times.
2024-04-16 21:07:27,938 INFO
2024-04-16 21:07:27,938 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-16 21:07:27,938 INFO
2024-04-16 21:07:27,938 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:27,939 INFO
2024-04-16 21:07:27,939 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:27,939 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:28,258 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:28,259 DEBUG --- stdout ---
2024-04-16 21:07:28,259 DEBUG 1
2024-04-16 21:07:28,259 DEBUG --- stderr ---
2024-04-16 21:07:28,259 DEBUG
2024-04-16 21:07:28,259 INFO
2024-04-16 21:07:28,259 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:07:28,259 INFO
2024-04-16 21:07:28,259 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-16 21:07:28,259 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:07:28,574 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-16 21:07:28,574 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:28,574 DEBUG --- stdout ---
2024-04-16 21:07:28,575 DEBUG admin-ui-c4c7bf584-mbzlq
2024-04-16 21:07:28,575 DEBUG --- stderr ---
2024-04-16 21:07:28,575 DEBUG
2024-04-16 21:07:28,575 INFO
2024-04-16 21:07:28,575 INFO ------------ Check pod admin-ui-c4c7bf584-mbzlq is running ------------
2024-04-16 21:07:28,575 INFO
2024-04-16 21:07:28,575 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-c4c7bf584-mbzlq -o=jsonpath={.status.phase} | grep "Running"
2024-04-16 21:07:28,575 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:28,928 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:28,928 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:28,928 DEBUG --- stdout ---
2024-04-16 21:07:28,929 DEBUG Running
2024-04-16 21:07:28,929 DEBUG --- stderr ---
2024-04-16 21:07:28,929 DEBUG
2024-04-16 21:07:28,929 INFO
2024-04-16 21:07:28,929 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-c4c7bf584-mbzlq -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-16 21:07:28,929 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-16 21:07:29,241 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:29,242 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:29,242 DEBUG --- stdout ---
2024-04-16 21:07:29,242 DEBUG true
2024-04-16 21:07:29,242 DEBUG --- stderr ---
2024-04-16 21:07:29,242 DEBUG
2024-04-16 21:07:29,243 INFO
2024-04-16 21:07:29,243 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-c4c7bf584-mbzlq --output jsonpath={.status.startTime}
2024-04-16 21:07:29,243 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:29,551 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:29,552 DEBUG --- stdout ---
2024-04-16 21:07:29,552 DEBUG 2024-04-13T22:41:42Z
2024-04-16 21:07:29,552 DEBUG --- stderr ---
2024-04-16 21:07:29,552 DEBUG
2024-04-16 21:07:29,552 INFO
2024-04-16 21:07:29,552 INFO ----- Check pod admin-ui-c4c7bf584-mbzlq filesystem is accessible -----
2024-04-16 21:07:30,326 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:30,327 INFO
2024-04-16 21:07:30,327 INFO ---------- Check pod admin-ui-c4c7bf584-mbzlq restart count ----------
2024-04-16 21:07:30,327 INFO
2024-04-16 21:07:30,327 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-c4c7bf584-mbzlq --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-16 21:07:30,327 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:30,634 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:30,634 DEBUG --- stdout ---
2024-04-16 21:07:30,634 DEBUG 0
2024-04-16 21:07:30,634 DEBUG --- stderr ---
2024-04-16 21:07:30,634 DEBUG
2024-04-16 21:07:30,634 INFO Pod admin-ui-c4c7bf584-mbzlq has been restarted 0 times.
2024-04-16 21:07:30,634 INFO
2024-04-16 21:07:30,634 INFO ***************************** Checking DS-CTS component is running *****************************
2024-04-16 21:07:30,634 INFO
2024-04-16 21:07:30,634 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:30,635 INFO
2024-04-16 21:07:30,635 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:30,635 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:31,007 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:31,008 DEBUG --- stdout ---
2024-04-16 21:07:31,009 DEBUG 3
2024-04-16 21:07:31,009 DEBUG --- stderr ---
2024-04-16 21:07:31,009 DEBUG
2024-04-16 21:07:31,009 INFO
2024-04-16 21:07:31,009 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-16 21:07:31,010 INFO
2024-04-16 21:07:31,010 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-cts --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3"
2024-04-16 21:07:31,010 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-16 21:07:31,411 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:31,412 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:31,412 DEBUG --- stdout ---
2024-04-16 21:07:31,412 DEBUG current:3 ready:3 replicas:3
2024-04-16 21:07:31,412 DEBUG --- stderr ---
2024-04-16 21:07:31,412 DEBUG
2024-04-16 21:07:31,412 INFO
2024-04-16 21:07:31,412 INFO *************************** Checking DS-IDREPO component is running ***************************
2024-04-16 21:07:31,412 INFO
2024-04-16 21:07:31,412 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:31,412 INFO
2024-04-16 21:07:31,412 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:31,412 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:31,790 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:31,790 DEBUG --- stdout ---
2024-04-16 21:07:31,790 DEBUG 3
2024-04-16 21:07:31,790 DEBUG --- stderr ---
2024-04-16 21:07:31,790 DEBUG
2024-04-16 21:07:31,790 INFO
2024-04-16 21:07:31,790 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-16 21:07:31,791 INFO
2024-04-16 21:07:31,791 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-idrepo --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3"
2024-04-16 21:07:31,791 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-16 21:07:32,225 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:32,225 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:32,225 DEBUG --- stdout ---
2024-04-16 21:07:32,225 DEBUG current:3 ready:3 replicas:3
2024-04-16 21:07:32,225 DEBUG --- stderr ---
2024-04-16 21:07:32,225 DEBUG
2024-04-16 21:07:32,225 INFO
2024-04-16 21:07:32,225 INFO ******************************* Checking AM component is running *******************************
2024-04-16 21:07:32,225 INFO
2024-04-16 21:07:32,225 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:32,225 INFO
2024-04-16 21:07:32,225 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:32,225 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:32,590 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:32,591 DEBUG --- stdout ---
2024-04-16 21:07:32,591 DEBUG 3
2024-04-16 21:07:32,591 DEBUG --- stderr ---
2024-04-16 21:07:32,591 DEBUG
2024-04-16 21:07:32,591 INFO
2024-04-16 21:07:32,591 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-16 21:07:32,591 INFO
2024-04-16 21:07:32,591 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments am --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:3 replicas:3"
2024-04-16 21:07:32,591 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-16 21:07:32,970 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:32,971 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:32,971 DEBUG --- stdout ---
2024-04-16 21:07:32,971 DEBUG ready:3 replicas:3
2024-04-16 21:07:32,971 DEBUG --- stderr ---
2024-04-16 21:07:32,971 DEBUG
2024-04-16 21:07:32,971 INFO
2024-04-16 21:07:32,971 INFO ***************************** Checking AMSTER component is running *****************************
2024-04-16 21:07:32,971 INFO
2024-04-16 21:07:32,971 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:32,971 INFO
2024-04-16 21:07:32,971 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-16 21:07:32,971 INFO
2024-04-16 21:07:32,971 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get jobs amster --output jsonpath="{.status.succeeded}" | grep "1"
2024-04-16 21:07:32,972 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-16 21:07:33,325 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:33,325 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:33,325 DEBUG --- stdout ---
2024-04-16 21:07:33,325 DEBUG 1
2024-04-16 21:07:33,325 DEBUG --- stderr ---
2024-04-16 21:07:33,325 DEBUG
2024-04-16 21:07:33,325 INFO
2024-04-16 21:07:33,325 INFO ****************************** Checking IDM component is running ******************************
2024-04-16 21:07:33,325 INFO
2024-04-16 21:07:33,325 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:33,326 INFO
2024-04-16 21:07:33,326 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:33,326 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:33,691 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:33,691 DEBUG --- stdout ---
2024-04-16 21:07:33,691 DEBUG 2
2024-04-16 21:07:33,691 DEBUG --- stderr ---
2024-04-16 21:07:33,691 DEBUG
2024-04-16 21:07:33,692 INFO
2024-04-16 21:07:33,692 INFO -------------- Waiting for 2 expected pod(s) to be ready --------------
2024-04-16 21:07:33,692 INFO
2024-04-16 21:07:33,692 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment idm --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:2 replicas:2"
2024-04-16 21:07:33,692 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-16 21:07:34,049 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:34,049 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:34,049 DEBUG --- stdout ---
2024-04-16 21:07:34,049 DEBUG ready:2 replicas:2
2024-04-16 21:07:34,049 DEBUG --- stderr ---
2024-04-16 21:07:34,049 DEBUG
2024-04-16 21:07:34,050 INFO
2024-04-16 21:07:34,050 INFO ************************** Checking END-USER-UI component is running **************************
2024-04-16 21:07:34,050 INFO
2024-04-16 21:07:34,050 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:34,050 INFO
2024-04-16 21:07:34,050 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:34,050 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:34,356 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:34,356 DEBUG --- stdout ---
2024-04-16 21:07:34,356 DEBUG 1
2024-04-16 21:07:34,356 DEBUG --- stderr ---
2024-04-16 21:07:34,356 DEBUG
2024-04-16 21:07:34,356 INFO
2024-04-16 21:07:34,357 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-16 21:07:34,357 INFO
2024-04-16 21:07:34,357 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments end-user-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-16 21:07:34,357 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-16 21:07:34,669 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:34,669 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:34,669 DEBUG --- stdout ---
2024-04-16 21:07:34,669 DEBUG ready:1 replicas:1
2024-04-16 21:07:34,669 DEBUG --- stderr ---
2024-04-16 21:07:34,669 DEBUG
2024-04-16 21:07:34,669 INFO
2024-04-16 21:07:34,669 INFO **************************** Checking LOGIN-UI component is running ****************************
2024-04-16 21:07:34,670 INFO
2024-04-16 21:07:34,670 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:34,670 INFO
2024-04-16 21:07:34,670 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:34,670 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:34,970 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:34,970 DEBUG --- stdout ---
2024-04-16 21:07:34,970 DEBUG 1
2024-04-16 21:07:34,970 DEBUG --- stderr ---
2024-04-16 21:07:34,970 DEBUG
2024-04-16 21:07:34,970 INFO
2024-04-16 21:07:34,970 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-16 21:07:34,971 INFO
2024-04-16 21:07:34,971 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments login-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-16 21:07:34,971 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-16 21:07:35,290 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:35,290 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:35,290 DEBUG --- stdout ---
2024-04-16 21:07:35,290 DEBUG ready:1 replicas:1
2024-04-16 21:07:35,291 DEBUG --- stderr ---
2024-04-16 21:07:35,291 DEBUG
2024-04-16 21:07:35,291 INFO
2024-04-16 21:07:35,291 INFO **************************** Checking ADMIN-UI component is running ****************************
2024-04-16 21:07:35,291 INFO
2024-04-16 21:07:35,291 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:07:35,291 INFO
2024-04-16 21:07:35,291 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:07:35,292 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:35,605 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:35,605 DEBUG --- stdout ---
2024-04-16 21:07:35,605 DEBUG 1
2024-04-16 21:07:35,605 DEBUG --- stderr ---
2024-04-16 21:07:35,605 DEBUG
2024-04-16 21:07:35,605 INFO
2024-04-16 21:07:35,605 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-16 21:07:35,606 INFO
2024-04-16 21:07:35,606 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments admin-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-16 21:07:35,606 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-16 21:07:35,970 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-16 21:07:35,970 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:35,970 DEBUG --- stdout ---
2024-04-16 21:07:35,970 DEBUG ready:1 replicas:1
2024-04-16 21:07:35,970 DEBUG --- stderr ---
2024-04-16 21:07:35,970 DEBUG
2024-04-16 21:07:35,971 INFO
2024-04-16 21:07:35,971 INFO ****************************** Livecheck stage: After deployment ******************************
2024-04-16 21:07:35,971 INFO
2024-04-16 21:07:35,971 INFO ---------------------- Running DS-CTS livecheck ----------------------
2024-04-16 21:07:35,971 INFO Livecheck to ds-cts-0
2024-04-16 21:07:35,971 INFO
2024-04-16 21:07:35,971 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}"
2024-04-16 21:07:35,971 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-16 21:07:36,316 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:36,317 DEBUG --- stdout ---
2024-04-16 21:07:36,318 DEBUG YjJFdDlYdUlSY0J3bnRZVWk3SjdmN09BU1M5WWJRcXg=
2024-04-16 21:07:36,318 DEBUG --- stderr ---
2024-04-16 21:07:36,318 DEBUG
2024-04-16 21:07:36,318 INFO
2024-04-16 21:07:36,318 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-16 21:07:38,094 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:07:38,095 DEBUG --- stdout ---
2024-04-16 21:07:38,095 DEBUG dn:
alive: true
2024-04-16 21:07:38,095 DEBUG --- stderr ---
2024-04-16 21:07:38,095 DEBUG
2024-04-16 21:07:38,095 INFO
2024-04-16 21:07:38,095 INFO Livecheck to ds-cts-1
2024-04-16 21:07:38,095 INFO
2024-04-16 21:07:38,095 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-16 21:07:39,829 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:07:39,829 DEBUG --- stdout ---
2024-04-16 21:07:39,829 DEBUG dn:
alive: true
2024-04-16 21:07:39,829 DEBUG --- stderr ---
2024-04-16 21:07:39,829 DEBUG
2024-04-16 21:07:39,829 INFO
2024-04-16 21:07:39,829 INFO Livecheck to ds-cts-2
2024-04-16 21:07:39,830 INFO
2024-04-16 21:07:39,830 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-16 21:07:41,581 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:07:41,583 DEBUG --- stdout ---
2024-04-16 21:07:41,583 DEBUG dn:
alive: true
2024-04-16 21:07:41,583 DEBUG --- stderr ---
2024-04-16 21:07:41,583 DEBUG
2024-04-16 21:07:41,583 INFO
2024-04-16 21:07:41,583 INFO
2024-04-16 21:07:41,583 INFO --------------------- Running DS-IDREPO livecheck ---------------------
2024-04-16 21:07:41,583 INFO Livecheck to ds-idrepo-0
2024-04-16 21:07:41,584 INFO
2024-04-16 21:07:41,584 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}"
2024-04-16 21:07:41,584 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-16 21:07:41,878 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:41,878 DEBUG --- stdout ---
2024-04-16 21:07:41,878 DEBUG YjJFdDlYdUlSY0J3bnRZVWk3SjdmN09BU1M5WWJRcXg=
2024-04-16 21:07:41,878 DEBUG --- stderr ---
2024-04-16 21:07:41,878 DEBUG
2024-04-16 21:07:41,879 INFO
2024-04-16 21:07:41,879 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-16 21:07:43,611 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:07:43,611 DEBUG --- stdout ---
2024-04-16 21:07:43,611 DEBUG dn:
alive: true
2024-04-16 21:07:43,611 DEBUG --- stderr ---
2024-04-16 21:07:43,611 DEBUG
2024-04-16 21:07:43,611 INFO
2024-04-16 21:07:43,611 INFO Livecheck to ds-idrepo-1
2024-04-16 21:07:43,611 INFO
2024-04-16 21:07:43,612 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-16 21:07:45,367 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:07:45,367 DEBUG --- stdout ---
2024-04-16 21:07:45,367 DEBUG dn:
alive: true
2024-04-16 21:07:45,367 DEBUG --- stderr ---
2024-04-16 21:07:45,367 DEBUG
2024-04-16 21:07:45,367 INFO
2024-04-16 21:07:45,367 INFO Livecheck to ds-idrepo-2
2024-04-16 21:07:45,367 INFO
2024-04-16 21:07:45,368 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-16 21:07:47,127 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-16 21:07:47,128 DEBUG --- stdout ---
2024-04-16 21:07:47,128 DEBUG dn:
alive: true
2024-04-16 21:07:47,128 DEBUG --- stderr ---
2024-04-16 21:07:47,128 DEBUG
2024-04-16 21:07:47,128 INFO
2024-04-16 21:07:47,128 INFO
2024-04-16 21:07:47,128 INFO ------------------------ Running AM livecheck ------------------------
2024-04-16 21:07:47,129 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready
2024-04-16 21:07:47,129 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready"
2024-04-16 21:07:47,450 INFO [http_cmd]: http status code OK
2024-04-16 21:07:47,451 DEBUG --- status code ---
2024-04-16 21:07:47,451 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:47,451 DEBUG --- http response ---
2024-04-16 21:07:47,451 DEBUG
2024-04-16 21:07:47,451 INFO
2024-04-16 21:07:47,451 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.AM_PASSWORDS_AMADMIN_CLEAR}"
2024-04-16 21:07:47,451 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-16 21:07:47,769 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:47,770 DEBUG --- stdout ---
2024-04-16 21:07:47,770 DEBUG WU53YjE0T1M1SDNjODNsV0txOWpibG5U
2024-04-16 21:07:47,770 DEBUG --- stderr ---
2024-04-16 21:07:47,770 DEBUG
2024-04-16 21:07:47,770 INFO Authenticate user amadmin via REST
2024-04-16 21:07:47,771 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
2024-04-16 21:07:47,998 INFO [http_cmd]: http status code OK
2024-04-16 21:07:47,998 DEBUG --- status code ---
2024-04-16 21:07:47,998 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:47,998 DEBUG --- http response ---
2024-04-16 21:07:47,999 DEBUG {
"tokenId": "zp5-NS5igy5b4nzmbok7rm6fs6M.*AAJTSQACMDIAAlNLABxvdDBvS3B6c0JWaEFTYjFBU212Z2Q2dXYyaFk9AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
"successUrl": "/am/console",
"realm": "/"
}
2024-04-16 21:07:48,003 INFO
2024-04-16 21:07:48,004 INFO ---------------------- Running AMSTER livecheck ----------------------
2024-04-16 21:07:48,004 INFO
2024-04-16 21:07:48,004 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:07:48,005 INFO
2024-04-16 21:07:48,005 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-16 21:07:48,005 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:07:48,366 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:48,367 DEBUG --- stdout ---
2024-04-16 21:07:48,367 DEBUG amster-qv96p
2024-04-16 21:07:48,367 DEBUG --- stderr ---
2024-04-16 21:07:48,367 DEBUG
2024-04-16 21:07:48,977 INFO Amster livecheck is passed
2024-04-16 21:07:48,977 INFO
2024-04-16 21:07:48,977 INFO ------------------------ Running IDM livecheck ------------------------
2024-04-16 21:07:48,977 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping
2024-04-16 21:07:48,978 INFO
2024-04-16 21:07:48,978 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.OPENIDM_ADMIN_PASSWORD}"
2024-04-16 21:07:48,978 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-16 21:07:49,289 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:49,289 DEBUG --- stdout ---
2024-04-16 21:07:49,289 DEBUG MVVYcW1nZmVQQjROSGdoYkdjdWFzUlhD
2024-04-16 21:07:49,289 DEBUG --- stderr ---
2024-04-16 21:07:49,289 DEBUG
2024-04-16 21:07:49,289 INFO Set admin password: 1UXqmgfePB4NHghbGcuasRXC
2024-04-16 21:07:49,290 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping"
2024-04-16 21:07:49,501 INFO [http_cmd]: http status code OK
2024-04-16 21:07:49,501 DEBUG --- status code ---
2024-04-16 21:07:49,501 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:49,502 DEBUG --- http response ---
2024-04-16 21:07:49,502 DEBUG {
"_id": "",
"_rev": "",
"shortDesc": "OpenIDM ready",
"state": "ACTIVE_READY"
}
2024-04-16 21:07:49,502 INFO Create new LoginSession for user "amadmin"
2024-04-16 21:07:49,502 INFO [LoginSession] Get "access_token" token for user "amadmin"
2024-04-16 21:07:49,502 INFO [LoginSession] Obtaining new "access_token" token from server because it was not obtained yet for the user
2024-04-16 21:07:49,502 INFO [LoginSession] Get "token_id" token for user "amadmin"
2024-04-16 21:07:49,502 INFO [LoginSession] Obtaining new "token_id" token from server because it was not obtained yet for the user
2024-04-16 21:07:49,502 INFO Authenticate user amadmin via REST
2024-04-16 21:07:49,502 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
2024-04-16 21:07:49,723 INFO [http_cmd]: http status code OK
2024-04-16 21:07:49,723 DEBUG --- status code ---
2024-04-16 21:07:49,723 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:49,723 DEBUG --- http response ---
2024-04-16 21:07:49,723 DEBUG {
"tokenId": "5wemwxEHV1mvZ89HHfTl6dhR3_E.*AAJTSQACMDIAAlNLABxVTUIxNExySk52OHh0RW1xU25yNG92OXRRZ3c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
"successUrl": "/am/console",
"realm": "/"
}
2024-04-16 21:07:49,723 INFO [LoginSession] Obtaining session info
2024-04-16 21:07:49,724 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/realms/root/sessions?_action=getSessionInfo"
2024-04-16 21:07:49,944 INFO [http_cmd]: http status code OK
2024-04-16 21:07:49,944 DEBUG --- status code ---
2024-04-16 21:07:49,944 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:49,944 DEBUG --- http response ---
2024-04-16 21:07:49,944 DEBUG {
"username": "amadmin",
"universalId": "id=amadmin,ou=user,ou=am-config",
"realm": "/",
"latestAccessTime": "2024-04-16T21:07:49Z",
"maxIdleExpirationTime": "2024-04-16T21:37:49Z",
"maxSessionExpirationTime": "2024-04-16T23:07:48Z",
"properties": {
"AMCtxId": "8faa1829-abe2-46a9-9b71-9b48e0145bfe-7276"
}
}
2024-04-16 21:07:49,948 INFO Obtain Oauth2 authz code via REST
2024-04-16 21:07:49,948 DEBUG Oauth2 authorize for access token
2024-04-16 21:07:49,948 INFO [http_cmd]: curl --request POST --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=5wemwxEHV1mvZ89HHfTl6dhR3_E.*AAJTSQACMDIAAlNLABxVTUIxNExySk52OHh0RW1xU25yNG92OXRRZ3c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" --data '{"decision": "Allow", "csrf": "5wemwxEHV1mvZ89HHfTl6dhR3_E.*AAJTSQACMDIAAlNLABxVTUIxNExySk52OHh0RW1xU25yNG92OXRRZ3c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/authorize?client_id=smokeclient&scope=fr:idm:*&redirect_uri=https://fake.com&response_type=code&realm=/"
2024-04-16 21:07:50,177 INFO [http_cmd]: http status code OK
2024-04-16 21:07:50,177 DEBUG --- status code ---
2024-04-16 21:07:50,177 DEBUG http status code is 302 (expected 302)
2024-04-16 21:07:50,177 DEBUG --- http response ---
2024-04-16 21:07:50,177 DEBUG {'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'cache-control': 'no-store', 'location': 'https://fake.com?code=RES1lsA9nBtQ35hO2LBSTgcJrEA&iss=https%3A%2F%2Fxlou.iam.xlou-cdm.engineeringpit.com%2Fam%2Foauth2&client_id=smokeclient', 'pragma': 'no-cache', 'set-cookie': 'OAUTH_REQUEST_ATTRIBUTES=DELETED; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/; Secure; HttpOnly; SameSite=none', 'content-length': '0', 'date': 'Tue, 16 Apr 2024 21:07:50 GMT'}
2024-04-16 21:07:50,178 DEBUG Oauth2 exchange authorize code for access token
2024-04-16 21:07:50,178 INFO [http_cmd]: curl -L --request POST --data '{"grant_type": "authorization_code", "redirect_uri": "https://fake.com", "client_id": "smokeclient", "code": "RES1lsA9nBtQ35hO2LBSTgcJrEA"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/access_token?realm=/"
2024-04-16 21:07:50,259 INFO [http_cmd]: http status code OK
2024-04-16 21:07:50,259 DEBUG --- status code ---
2024-04-16 21:07:50,259 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:50,259 DEBUG --- http response ---
2024-04-16 21:07:50,259 DEBUG {
"access_token": "4mzi4wYBsKeIXN9wuN4NY0D3J5I",
"scope": "fr:idm:*",
"token_type": "Bearer",
"expires_in": 3599
}
2024-04-16 21:07:50,261 INFO Additional check to sync/queue to ensure sync with DS is ok (see LODESTAR-1245)
2024-04-16 21:07:50,261 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/sync/queue?_queryFilter=true&_pageSize=1"
2024-04-16 21:07:50,484 INFO [http_cmd]: http status code OK
2024-04-16 21:07:50,484 DEBUG --- status code ---
2024-04-16 21:07:50,485 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:50,485 DEBUG --- http response ---
2024-04-16 21:07:50,485 DEBUG {
"result": [],
"resultCount": 0,
"pagedResultsCookie": null,
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}
2024-04-16 21:07:50,486 INFO Try to create, query and delete a user
2024-04-16 21:07:50,487 INFO Create user idm_l4nudswd6a in IDM
2024-04-16 21:07:50,487 INFO [http_cmd]: curl --header "Authorization: Bearer 4mzi4wYBsKeIXN9wuN4NY0D3J5I" --header "Content-Type: application/json" --insecure -L --request POST --data '{"userName": "idm_l4nudswd6a", "givenName": "Peter", "sn": "Parker", "password": "T35tr0ck123", "telephoneNumber": "6669876987", "description": "IDM Test User", "roles": [], "mail": "idm_l4nudswd6a@forgerock.com"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user?_action=create"
2024-04-16 21:07:50,740 INFO [http_cmd]: http status code OK
2024-04-16 21:07:50,740 DEBUG --- status code ---
2024-04-16 21:07:50,740 DEBUG http status code is 201 (expected [200, 201])
2024-04-16 21:07:50,740 DEBUG --- http response ---
2024-04-16 21:07:50,740 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-16 21:07:50,740 DEBUG ----- output -----
2024-04-16 21:07:50,740 DEBUG {
2024-04-16 21:07:50,740 DEBUG "_id": "e01a9b98-9fe7-41d8-9e84-25242c322ebe",
2024-04-16 21:07:50,740 DEBUG "_rev": "d2dd67f6-c775-4646-8c43-6112461729ba-1101619",
2024-04-16 21:07:50,740 DEBUG "country": null,
2024-04-16 21:07:50,740 DEBUG "mail": "idm_l4nudswd6a@forgerock.com",
2024-04-16 21:07:50,740 DEBUG "memberOfOrgIDs": [],
2024-04-16 21:07:50,741 DEBUG "postalCode": null,
2024-04-16 21:07:50,741 DEBUG "profileImage": null,
2024-04-16 21:07:50,741 DEBUG "expireAccount": null,
2024-04-16 21:07:50,741 DEBUG "activeDate": null,
2024-04-16 21:07:50,741 DEBUG
2024-04-16 21:07:50,741 DEBUG
2024-04-16 21:07:50,741 DEBUG
2024-04-16 21:07:50,741 DEBUG [...]
2024-04-16 21:07:50,741 DEBUG
2024-04-16 21:07:50,741 DEBUG
2024-04-16 21:07:50,741 DEBUG
2024-04-16 21:07:50,741 DEBUG "aliasList": [],
2024-04-16 21:07:50,741 DEBUG "kbaInfo": [],
2024-04-16 21:07:50,741 DEBUG "inactiveDate": null,
2024-04-16 21:07:50,741 DEBUG "sn": "Parker",
2024-04-16 21:07:50,741 DEBUG "preferences": null,
2024-04-16 21:07:50,741 DEBUG "userName": "idm_l4nudswd6a",
2024-04-16 21:07:50,741 DEBUG "passwordLastChangedTime": "2024-04-16T21:07:50.667Z",
2024-04-16 21:07:50,741 DEBUG "effectiveRoles": [],
2024-04-16 21:07:50,741 DEBUG "activateAccount": null
2024-04-16 21:07:50,741 DEBUG }
2024-04-16 21:07:50,741 INFO Query IDM to verify UserID e01a9b98-9fe7-41d8-9e84-25242c322ebe is visible
2024-04-16 21:07:50,741 INFO [http_cmd]: curl --insecure -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/e01a9b98-9fe7-41d8-9e84-25242c322ebe"
2024-04-16 21:07:50,949 INFO [http_cmd]: http status code OK
2024-04-16 21:07:50,949 DEBUG --- status code ---
2024-04-16 21:07:50,949 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:50,949 DEBUG --- http response ---
2024-04-16 21:07:50,949 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-16 21:07:50,949 DEBUG ----- output -----
2024-04-16 21:07:50,949 DEBUG {
2024-04-16 21:07:50,949 DEBUG "_id": "e01a9b98-9fe7-41d8-9e84-25242c322ebe",
2024-04-16 21:07:50,949 DEBUG "_rev": "d2dd67f6-c775-4646-8c43-6112461729ba-1101619",
2024-04-16 21:07:50,949 DEBUG "country": null,
2024-04-16 21:07:50,949 DEBUG "mail": "idm_l4nudswd6a@forgerock.com",
2024-04-16 21:07:50,949 DEBUG "memberOfOrgIDs": [],
2024-04-16 21:07:50,949 DEBUG "postalCode": null,
2024-04-16 21:07:50,949 DEBUG "profileImage": null,
2024-04-16 21:07:50,949 DEBUG "expireAccount": null,
2024-04-16 21:07:50,949 DEBUG "activeDate": null,
2024-04-16 21:07:50,949 DEBUG
2024-04-16 21:07:50,949 DEBUG
2024-04-16 21:07:50,949 DEBUG
2024-04-16 21:07:50,949 DEBUG [...]
2024-04-16 21:07:50,950 DEBUG
2024-04-16 21:07:50,950 DEBUG
2024-04-16 21:07:50,950 DEBUG
2024-04-16 21:07:50,950 DEBUG "aliasList": [],
2024-04-16 21:07:50,950 DEBUG "kbaInfo": [],
2024-04-16 21:07:50,950 DEBUG "inactiveDate": null,
2024-04-16 21:07:50,950 DEBUG "sn": "Parker",
2024-04-16 21:07:50,950 DEBUG "preferences": null,
2024-04-16 21:07:50,950 DEBUG "userName": "idm_l4nudswd6a",
2024-04-16 21:07:50,950 DEBUG "passwordLastChangedTime": "2024-04-16T21:07:50.667Z",
2024-04-16 21:07:50,950 DEBUG "effectiveRoles": [],
2024-04-16 21:07:50,950 DEBUG "activateAccount": null
2024-04-16 21:07:50,950 DEBUG }
2024-04-16 21:07:50,950 INFO Authenticate as user idm_l4nudswd6a
2024-04-16 21:07:50,950 INFO [http_cmd]: curl --insecure -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/authentication?_action=login"
2024-04-16 21:07:51,153 INFO [http_cmd]: http status code OK
2024-04-16 21:07:51,154 DEBUG --- status code ---
2024-04-16 21:07:51,154 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:51,154 DEBUG --- http response ---
2024-04-16 21:07:51,154 DEBUG {
"_id": "login",
"authorization": {
"id": "anonymous",
"roles": [
"internal/role/openidm-reg"
],
"component": "internal/user"
},
"authenticationId": "anonymous"
}
2024-04-16 21:07:51,155 INFO [http_cmd]: curl --insecure -L --request DELETE "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/e01a9b98-9fe7-41d8-9e84-25242c322ebe"
2024-04-16 21:07:51,379 INFO [http_cmd]: http status code OK
2024-04-16 21:07:51,379 DEBUG --- status code ---
2024-04-16 21:07:51,379 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:51,379 DEBUG --- http response ---
2024-04-16 21:07:51,379 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-16 21:07:51,379 DEBUG ----- output -----
2024-04-16 21:07:51,379 DEBUG {
2024-04-16 21:07:51,379 DEBUG "_id": "e01a9b98-9fe7-41d8-9e84-25242c322ebe",
2024-04-16 21:07:51,379 DEBUG "_rev": "d2dd67f6-c775-4646-8c43-6112461729ba-1101619",
2024-04-16 21:07:51,379 DEBUG "country": null,
2024-04-16 21:07:51,379 DEBUG "mail": "idm_l4nudswd6a@forgerock.com",
2024-04-16 21:07:51,379 DEBUG "memberOfOrgIDs": [],
2024-04-16 21:07:51,379 DEBUG "postalCode": null,
2024-04-16 21:07:51,379 DEBUG "profileImage": null,
2024-04-16 21:07:51,379 DEBUG "expireAccount": null,
2024-04-16 21:07:51,379 DEBUG "activeDate": null,
2024-04-16 21:07:51,379 DEBUG
2024-04-16 21:07:51,379 DEBUG
2024-04-16 21:07:51,379 DEBUG
2024-04-16 21:07:51,379 DEBUG [...]
2024-04-16 21:07:51,379 DEBUG
2024-04-16 21:07:51,379 DEBUG
2024-04-16 21:07:51,380 DEBUG
2024-04-16 21:07:51,380 DEBUG "aliasList": [],
2024-04-16 21:07:51,380 DEBUG "kbaInfo": [],
2024-04-16 21:07:51,380 DEBUG "inactiveDate": null,
2024-04-16 21:07:51,380 DEBUG "sn": "Parker",
2024-04-16 21:07:51,380 DEBUG "preferences": null,
2024-04-16 21:07:51,380 DEBUG "userName": "idm_l4nudswd6a",
2024-04-16 21:07:51,380 DEBUG "passwordLastChangedTime": "2024-04-16T21:07:50.667Z",
2024-04-16 21:07:51,380 DEBUG "effectiveRoles": [],
2024-04-16 21:07:51,380 DEBUG "activateAccount": null
2024-04-16 21:07:51,380 DEBUG }
2024-04-16 21:07:51,381 INFO
2024-04-16 21:07:51,382 INFO -------------------- Running END-USER-UI livecheck --------------------
2024-04-16 21:07:51,382 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/enduser
2024-04-16 21:07:51,382 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/enduser"
2024-04-16 21:07:51,659 INFO [http_cmd]: http status code OK
2024-04-16 21:07:51,659 DEBUG --- status code ---
2024-04-16 21:07:51,659 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:51,659 DEBUG --- http response ---
2024-04-16 21:07:51,660 DEBUG Identity Management
[]
2024-04-16 21:07:51,661 INFO
2024-04-16 21:07:51,662 INFO --------------------- Running LOGIN-UI livecheck ---------------------
2024-04-16 21:07:51,662 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI
2024-04-16 21:07:51,662 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI"
2024-04-16 21:07:51,938 INFO [http_cmd]: http status code OK
2024-04-16 21:07:51,939 DEBUG --- status code ---
2024-04-16 21:07:51,939 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:51,939 DEBUG --- http response ---
2024-04-16 21:07:51,939 DEBUG Login
[]
2024-04-16 21:07:51,942 INFO
2024-04-16 21:07:51,943 INFO --------------------- Running ADMIN-UI livecheck ---------------------
2024-04-16 21:07:51,943 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/platform
2024-04-16 21:07:51,944 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/platform"
2024-04-16 21:07:52,221 INFO [http_cmd]: http status code OK
2024-04-16 21:07:52,222 DEBUG --- status code ---
2024-04-16 21:07:52,223 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:52,223 DEBUG --- http response ---
2024-04-16 21:07:52,223 DEBUG Platform Admin
[]
2024-04-16 21:07:52,224 INFO LIVECHECK SUCCEEDED
2024-04-16 21:07:52,225 INFO
2024-04-16 21:07:52,225 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-16 21:07:52,225 INFO
2024-04-16 21:07:52,225 INFO --------------------- Get DS-CTS software version ---------------------
2024-04-16 21:07:52,226 INFO
2024-04-16 21:07:52,226 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- find /opt/opendj -name opendj-core.jar
2024-04-16 21:07:52,226 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-16 21:07:53,062 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:53,063 DEBUG --- stdout ---
2024-04-16 21:07:53,063 DEBUG /opt/opendj/lib/opendj-core.jar
2024-04-16 21:07:53,063 DEBUG --- stderr ---
2024-04-16 21:07:53,063 DEBUG
2024-04-16 21:07:53,063 INFO
2024-04-16 21:07:53,063 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-cts-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-cts_info/opendj-core.jar --container ds
2024-04-16 21:07:53,064 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:54,340 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:54,340 DEBUG --- stdout ---
2024-04-16 21:07:54,340 DEBUG tar: Removing leading `/' from member names
2024-04-16 21:07:54,340 DEBUG --- stderr ---
2024-04-16 21:07:54,340 DEBUG
2024-04-16 21:07:54,348 INFO
2024-04-16 21:07:54,349 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-16 21:07:54,349 INFO
2024-04-16 21:07:54,349 INFO ------------------- Get DS-IDREPO software version -------------------
2024-04-16 21:07:54,349 INFO
2024-04-16 21:07:54,349 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- find /opt/opendj -name opendj-core.jar
2024-04-16 21:07:54,349 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-16 21:07:55,173 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:55,173 DEBUG --- stdout ---
2024-04-16 21:07:55,173 DEBUG /opt/opendj/lib/opendj-core.jar
2024-04-16 21:07:55,173 DEBUG --- stderr ---
2024-04-16 21:07:55,173 DEBUG
2024-04-16 21:07:55,174 INFO
2024-04-16 21:07:55,174 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-idrepo-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-idrepo_info/opendj-core.jar --container ds
2024-04-16 21:07:55,174 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:56,401 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:56,402 DEBUG --- stdout ---
2024-04-16 21:07:56,402 DEBUG tar: Removing leading `/' from member names
2024-04-16 21:07:56,402 DEBUG --- stderr ---
2024-04-16 21:07:56,402 DEBUG
2024-04-16 21:07:56,410 INFO
2024-04-16 21:07:56,410 INFO ****************************** Initializing component pods for AM ******************************
2024-04-16 21:07:56,410 INFO
2024-04-16 21:07:56,410 INFO ----------------------- Get AM software version -----------------------
2024-04-16 21:07:56,410 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version
2024-04-16 21:07:56,411 INFO [LoginSession] Get "token_id" token for user "amadmin"
2024-04-16 21:07:56,411 DEBUG [LoginSession] re-using "token_id" token that remains valid for 29m 43s
2024-04-16 21:07:56,412 INFO [http_cmd]: curl -L --request GET --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=5wemwxEHV1mvZ89HHfTl6dhR3_E.*AAJTSQACMDIAAlNLABxVTUIxNExySk52OHh0RW1xU25yNG92OXRRZ3c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version"
2024-04-16 21:07:56,630 INFO [http_cmd]: http status code OK
2024-04-16 21:07:56,631 DEBUG --- status code ---
2024-04-16 21:07:56,631 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:56,631 DEBUG --- http response ---
2024-04-16 21:07:56,631 DEBUG {
"_id": "version",
"_rev": "2117078800",
"version": "7.6.0-SNAPSHOT",
"fullVersion": "ForgeRock Access Management 7.6.0-SNAPSHOT Build 22d357546ce5875c6a5368e1656cb87cd71537f7 (2024-April-05 16:07)",
"revision": "22d357546ce5875c6a5368e1656cb87cd71537f7",
"date": "2024-April-05 16:07"
}
2024-04-16 21:07:56,634 INFO
2024-04-16 21:07:56,635 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-16 21:07:56,635 INFO
2024-04-16 21:07:56,635 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-16 21:07:56,635 INFO
2024-04-16 21:07:56,635 INFO ---------------------- Get IDM software version ----------------------
2024-04-16 21:07:56,635 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version
2024-04-16 21:07:56,635 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version"
2024-04-16 21:07:56,846 INFO [http_cmd]: http status code OK
2024-04-16 21:07:56,847 DEBUG --- status code ---
2024-04-16 21:07:56,847 DEBUG http status code is 200 (expected 200)
2024-04-16 21:07:56,847 DEBUG --- http response ---
2024-04-16 21:07:56,847 DEBUG {
"_id": "version",
"productVersion": "7.6.0-SNAPSHOT",
"productBuildDate": "20240411202127",
"productRevision": "8554e0d1c5"
}
2024-04-16 21:07:56,850 INFO
2024-04-16 21:07:56,850 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-16 21:07:56,850 INFO
2024-04-16 21:07:56,850 INFO ------------------ Get END-USER-UI software version ------------------
2024-04-16 21:07:56,851 INFO
2024-04-16 21:07:56,851 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec end-user-ui-58878d7668-zg4h2 --container end-user-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-16 21:07:56,851 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-16 21:07:57,640 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:57,640 DEBUG --- stdout ---
2024-04-16 21:07:57,640 DEBUG /usr/share/nginx/html/js/chunk-vendors.0380a414.js
2024-04-16 21:07:57,640 DEBUG --- stderr ---
2024-04-16 21:07:57,640 DEBUG
2024-04-16 21:07:57,640 INFO
2024-04-16 21:07:57,640 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp end-user-ui-58878d7668-zg4h2:/usr/share/nginx/html/js/chunk-vendors.0380a414.js /tmp/end-user-ui_info/chunk-vendors.0380a414.js --container end-user-ui
2024-04-16 21:07:57,640 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:07:58,751 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:58,751 DEBUG --- stdout ---
2024-04-16 21:07:58,751 DEBUG tar: Removing leading `/' from member names
2024-04-16 21:07:58,751 DEBUG --- stderr ---
2024-04-16 21:07:58,751 DEBUG
2024-04-16 21:07:58,779 INFO
2024-04-16 21:07:58,779 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-16 21:07:58,779 INFO
2024-04-16 21:07:58,779 INFO -------------------- Get LOGIN-UI software version --------------------
2024-04-16 21:07:58,779 INFO
2024-04-16 21:07:58,779 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec login-ui-5fdf49fc66-2wnx7 --container login-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-16 21:07:58,779 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-16 21:07:59,553 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:07:59,553 DEBUG --- stdout ---
2024-04-16 21:07:59,553 DEBUG /usr/share/nginx/html/js/chunk-vendors.f2fa5365.js
2024-04-16 21:07:59,553 DEBUG --- stderr ---
2024-04-16 21:07:59,553 DEBUG
2024-04-16 21:07:59,553 INFO
2024-04-16 21:07:59,553 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp login-ui-5fdf49fc66-2wnx7:/usr/share/nginx/html/js/chunk-vendors.f2fa5365.js /tmp/login-ui_info/chunk-vendors.f2fa5365.js --container login-ui
2024-04-16 21:07:59,553 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:08:00,677 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:00,678 DEBUG --- stdout ---
2024-04-16 21:08:00,678 DEBUG tar: Removing leading `/' from member names
2024-04-16 21:08:00,678 DEBUG --- stderr ---
2024-04-16 21:08:00,678 DEBUG
2024-04-16 21:08:00,698 INFO
2024-04-16 21:08:00,698 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-16 21:08:00,698 INFO
2024-04-16 21:08:00,698 INFO -------------------- Get ADMIN-UI software version --------------------
2024-04-16 21:08:00,699 INFO
2024-04-16 21:08:00,699 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec admin-ui-c4c7bf584-mbzlq --container admin-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-16 21:08:00,699 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-16 21:08:01,503 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:01,503 DEBUG --- stdout ---
2024-04-16 21:08:01,503 DEBUG /usr/share/nginx/html/js/chunk-vendors.b3d879ac.js
2024-04-16 21:08:01,503 DEBUG --- stderr ---
2024-04-16 21:08:01,503 DEBUG
2024-04-16 21:08:01,503 INFO
2024-04-16 21:08:01,503 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp admin-ui-c4c7bf584-mbzlq:/usr/share/nginx/html/js/chunk-vendors.b3d879ac.js /tmp/admin-ui_info/chunk-vendors.b3d879ac.js --container admin-ui
2024-04-16 21:08:01,503 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:08:02,681 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:02,681 DEBUG --- stdout ---
2024-04-16 21:08:02,682 DEBUG tar: Removing leading `/' from member names
2024-04-16 21:08:02,682 DEBUG --- stderr ---
2024-04-16 21:08:02,683 DEBUG
2024-04-16 21:08:02,755 INFO
2024-04-16 21:08:02,756 INFO
2024-04-16 21:08:02,756 INFO ====================================================================================================
2024-04-16 21:08:02,756 INFO ================ Admin password for DS-CTS is: b2Et9XuIRcBwntYUi7J7f7OASS9YbQqx ================
2024-04-16 21:08:02,756 INFO ====================================================================================================
2024-04-16 21:08:02,756 INFO
2024-04-16 21:08:02,756 INFO ====================================================================================================
2024-04-16 21:08:02,756 INFO ============== Admin password for DS-IDREPO is: b2Et9XuIRcBwntYUi7J7f7OASS9YbQqx ==============
2024-04-16 21:08:02,756 INFO ====================================================================================================
2024-04-16 21:08:02,756 INFO
2024-04-16 21:08:02,756 INFO ====================================================================================================
2024-04-16 21:08:02,756 INFO ====================== Admin password for AM is: YNwb14OS5H3c83lWKq9jblnT ======================
2024-04-16 21:08:02,756 INFO ====================================================================================================
2024-04-16 21:08:02,756 INFO
2024-04-16 21:08:02,756 INFO ====================================================================================================
2024-04-16 21:08:02,756 INFO ===================== Admin password for IDM is: 1UXqmgfePB4NHghbGcuasRXC =====================
2024-04-16 21:08:02,757 INFO ====================================================================================================
2024-04-16 21:08:02,757 INFO
2024-04-16 21:08:02,757 INFO *************************************** Dumping pod list ***************************************
2024-04-16 21:08:02,757 INFO Dumping pod list to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/_pod-list.txt
2024-04-16 21:08:03,264 INFO Dumping platform-config configmap to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/_platform-config_configmap.txt
2024-04-16 21:08:03,662 INFO
2024-04-16 21:08:03,662 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-16 21:08:03,662 INFO
2024-04-16 21:08:03,662 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:08:03,662 INFO
2024-04-16 21:08:03,662 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:08:03,662 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:08:04,063 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:04,063 DEBUG --- stdout ---
2024-04-16 21:08:04,063 DEBUG 3
2024-04-16 21:08:04,063 DEBUG --- stderr ---
2024-04-16 21:08:04,063 DEBUG
2024-04-16 21:08:04,063 INFO
2024-04-16 21:08:04,063 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:08:04,063 INFO
2024-04-16 21:08:04,064 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name}
2024-04-16 21:08:04,064 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:08:04,532 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:04,532 DEBUG --- stdout ---
2024-04-16 21:08:04,532 DEBUG ds-cts-0 ds-cts-1 ds-cts-2
2024-04-16 21:08:04,533 DEBUG --- stderr ---
2024-04-16 21:08:04,533 DEBUG
2024-04-16 21:08:04,533 INFO
2024-04-16 21:08:04,533 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-16 21:08:04,533 INFO
2024-04-16 21:08:04,533 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:08:04,533 INFO
2024-04-16 21:08:04,533 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:08:04,533 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:08:04,907 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:04,907 DEBUG --- stdout ---
2024-04-16 21:08:04,907 DEBUG 3
2024-04-16 21:08:04,907 DEBUG --- stderr ---
2024-04-16 21:08:04,907 DEBUG
2024-04-16 21:08:04,907 INFO
2024-04-16 21:08:04,907 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:08:04,908 INFO
2024-04-16 21:08:04,908 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name}
2024-04-16 21:08:04,908 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:08:05,352 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:05,352 DEBUG --- stdout ---
2024-04-16 21:08:05,352 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2
2024-04-16 21:08:05,352 DEBUG --- stderr ---
2024-04-16 21:08:05,352 DEBUG
2024-04-16 21:08:05,352 INFO
2024-04-16 21:08:05,352 INFO ****************************** Initializing component pods for AM ******************************
2024-04-16 21:08:05,352 INFO
2024-04-16 21:08:05,352 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:08:05,352 INFO
2024-04-16 21:08:05,352 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:08:05,352 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:08:05,719 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:05,720 DEBUG --- stdout ---
2024-04-16 21:08:05,720 DEBUG 3
2024-04-16 21:08:05,720 DEBUG --- stderr ---
2024-04-16 21:08:05,720 DEBUG
2024-04-16 21:08:05,720 INFO
2024-04-16 21:08:05,720 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:08:05,720 INFO
2024-04-16 21:08:05,720 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name}
2024-04-16 21:08:05,720 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:08:06,156 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:06,156 DEBUG --- stdout ---
2024-04-16 21:08:06,156 DEBUG am-555f4db595-4lbss am-555f4db595-fz8sn am-555f4db595-kzxsp
2024-04-16 21:08:06,156 DEBUG --- stderr ---
2024-04-16 21:08:06,156 DEBUG
2024-04-16 21:08:06,156 INFO
2024-04-16 21:08:06,156 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-16 21:08:06,156 INFO
2024-04-16 21:08:06,156 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:08:06,157 INFO
2024-04-16 21:08:06,157 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-16 21:08:06,157 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:08:06,522 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:06,522 DEBUG --- stdout ---
2024-04-16 21:08:06,522 DEBUG amster-qv96p
2024-04-16 21:08:06,522 DEBUG --- stderr ---
2024-04-16 21:08:06,522 DEBUG
2024-04-16 21:08:06,522 INFO
2024-04-16 21:08:06,522 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-16 21:08:06,523 INFO
2024-04-16 21:08:06,523 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:08:06,523 INFO
2024-04-16 21:08:06,523 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:08:06,523 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:08:06,906 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:06,906 DEBUG --- stdout ---
2024-04-16 21:08:06,906 DEBUG 2
2024-04-16 21:08:06,906 DEBUG --- stderr ---
2024-04-16 21:08:06,906 DEBUG
2024-04-16 21:08:06,906 INFO
2024-04-16 21:08:06,906 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:08:06,906 INFO
2024-04-16 21:08:06,907 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name}
2024-04-16 21:08:06,907 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:08:07,293 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:07,294 DEBUG --- stdout ---
2024-04-16 21:08:07,294 DEBUG idm-b85f9d978-j5pbm idm-b85f9d978-rl76f
2024-04-16 21:08:07,294 DEBUG --- stderr ---
2024-04-16 21:08:07,294 DEBUG
2024-04-16 21:08:07,294 INFO
2024-04-16 21:08:07,294 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-16 21:08:07,294 INFO
2024-04-16 21:08:07,294 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:08:07,294 INFO
2024-04-16 21:08:07,295 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:08:07,295 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:08:07,602 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:07,603 DEBUG --- stdout ---
2024-04-16 21:08:07,603 DEBUG 1
2024-04-16 21:08:07,603 DEBUG --- stderr ---
2024-04-16 21:08:07,603 DEBUG
2024-04-16 21:08:07,603 INFO
2024-04-16 21:08:07,603 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:08:07,603 INFO
2024-04-16 21:08:07,603 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name}
2024-04-16 21:08:07,603 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:08:07,907 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:07,909 DEBUG --- stdout ---
2024-04-16 21:08:07,909 DEBUG end-user-ui-58878d7668-zg4h2
2024-04-16 21:08:07,909 DEBUG --- stderr ---
2024-04-16 21:08:07,909 DEBUG
2024-04-16 21:08:07,909 INFO
2024-04-16 21:08:07,909 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-16 21:08:07,910 INFO
2024-04-16 21:08:07,910 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:08:07,910 INFO
2024-04-16 21:08:07,910 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:08:07,910 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:08:08,214 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:08,214 DEBUG --- stdout ---
2024-04-16 21:08:08,214 DEBUG 1
2024-04-16 21:08:08,214 DEBUG --- stderr ---
2024-04-16 21:08:08,214 DEBUG
2024-04-16 21:08:08,215 INFO
2024-04-16 21:08:08,215 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:08:08,215 INFO
2024-04-16 21:08:08,215 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name}
2024-04-16 21:08:08,215 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:08:08,526 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:08,526 DEBUG --- stdout ---
2024-04-16 21:08:08,526 DEBUG login-ui-5fdf49fc66-2wnx7
2024-04-16 21:08:08,526 DEBUG --- stderr ---
2024-04-16 21:08:08,526 DEBUG
2024-04-16 21:08:08,526 INFO
2024-04-16 21:08:08,526 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-16 21:08:08,527 INFO
2024-04-16 21:08:08,527 INFO --------------------- Get expected number of pods ---------------------
2024-04-16 21:08:08,527 INFO
2024-04-16 21:08:08,527 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-16 21:08:08,527 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-16 21:08:08,837 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:08,837 DEBUG --- stdout ---
2024-04-16 21:08:08,837 DEBUG 1
2024-04-16 21:08:08,837 DEBUG --- stderr ---
2024-04-16 21:08:08,837 DEBUG
2024-04-16 21:08:08,838 INFO
2024-04-16 21:08:08,838 INFO ---------------------------- Get pod list ----------------------------
2024-04-16 21:08:08,838 INFO
2024-04-16 21:08:08,838 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name}
2024-04-16 21:08:08,838 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-16 21:08:09,199 INFO [loop_until]: OK (rc = 0)
2024-04-16 21:08:09,199 DEBUG --- stdout ---
2024-04-16 21:08:09,199 DEBUG admin-ui-c4c7bf584-mbzlq
2024-04-16 21:08:09,199 DEBUG --- stderr ---
2024-04-16 21:08:09,199 DEBUG
2024-04-16 21:08:09,200 INFO
2024-04-16 21:08:09,200 INFO *********************************** Dumping components logs ***********************************
2024-04-16 21:08:09,200 INFO
2024-04-16 21:08:09,200 INFO ----------------------- Dumping logs for DS-CTS -----------------------
2024-04-16 21:08:10,497 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-cts-0.txt
2024-04-16 21:08:10,498 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-cts-0.txt
2024-04-16 21:08:10,499 INFO Check pod logs for errors
2024-04-16 21:08:11,995 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-cts-1.txt
2024-04-16 21:08:11,996 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-cts-1.txt
2024-04-16 21:08:12,004 INFO Check pod logs for errors
2024-04-16 21:08:13,457 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-cts-2.txt
2024-04-16 21:08:13,459 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-cts-2.txt
2024-04-16 21:08:13,467 INFO Check pod logs for errors
2024-04-16 21:08:13,473 INFO
2024-04-16 21:08:13,474 INFO --------------------- Dumping logs for DS-IDREPO ---------------------
2024-04-16 21:08:15,095 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-idrepo-0.txt
2024-04-16 21:08:15,095 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-idrepo-0.txt
2024-04-16 21:08:15,146 INFO Check pod logs for errors
2024-04-16 21:08:16,726 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-idrepo-1.txt
2024-04-16 21:08:16,726 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-idrepo-1.txt
2024-04-16 21:08:16,732 INFO Check pod logs for errors
2024-04-16 21:08:18,200 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-idrepo-2.txt
2024-04-16 21:08:18,201 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/ds-idrepo-2.txt
2024-04-16 21:08:18,209 INFO Check pod logs for errors
2024-04-16 21:08:18,216 INFO
2024-04-16 21:08:18,216 INFO ------------------------- Dumping logs for AM -------------------------
2024-04-16 21:08:19,571 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/am-555f4db595-4lbss.txt
2024-04-16 21:08:19,572 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/am-555f4db595-4lbss.txt
2024-04-16 21:08:19,574 INFO Check pod logs for errors
2024-04-16 21:08:20,963 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/am-555f4db595-fz8sn.txt
2024-04-16 21:08:20,964 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/am-555f4db595-fz8sn.txt
2024-04-16 21:08:20,965 INFO Check pod logs for errors
2024-04-16 21:08:22,334 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/am-555f4db595-kzxsp.txt
2024-04-16 21:08:22,335 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/am-555f4db595-kzxsp.txt
2024-04-16 21:08:22,336 INFO Check pod logs for errors
2024-04-16 21:08:22,338 INFO
2024-04-16 21:08:22,338 INFO ----------------------- Dumping logs for AMSTER -----------------------
2024-04-16 21:08:23,449 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/amster-qv96p.txt
2024-04-16 21:08:23,450 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/amster-qv96p.txt
2024-04-16 21:08:23,450 INFO Check pod logs for errors
2024-04-16 21:08:23,450 INFO
2024-04-16 21:08:23,451 INFO ------------------------ Dumping logs for IDM ------------------------
2024-04-16 21:08:25,188 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/idm-b85f9d978-j5pbm.txt
2024-04-16 21:08:25,190 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/idm-b85f9d978-j5pbm.txt
2024-04-16 21:08:25,239 INFO Check pod logs for errors
2024-04-16 21:08:26,999 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/idm-b85f9d978-rl76f.txt
2024-04-16 21:08:27,000 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/idm-b85f9d978-rl76f.txt
2024-04-16 21:08:27,054 INFO Check pod logs for errors
2024-04-16 21:08:27,093 INFO
2024-04-16 21:08:27,094 INFO -------------------- Dumping logs for END-USER-UI --------------------
2024-04-16 21:08:28,452 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/end-user-ui-58878d7668-zg4h2.txt
2024-04-16 21:08:28,453 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/end-user-ui-58878d7668-zg4h2.txt
2024-04-16 21:08:28,468 INFO Check pod logs for errors
2024-04-16 21:08:28,485 INFO
2024-04-16 21:08:28,485 INFO ---------------------- Dumping logs for LOGIN-UI ----------------------
2024-04-16 21:08:29,866 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/login-ui-5fdf49fc66-2wnx7.txt
2024-04-16 21:08:29,866 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/login-ui-5fdf49fc66-2wnx7.txt
2024-04-16 21:08:29,882 INFO Check pod logs for errors
2024-04-16 21:08:29,909 INFO
2024-04-16 21:08:29,909 INFO ---------------------- Dumping logs for ADMIN-UI ----------------------
2024-04-16 21:08:31,330 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/admin-ui-c4c7bf584-mbzlq.txt
2024-04-16 21:08:31,330 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240416_210802-after-deployment/admin-ui-c4c7bf584-mbzlq.txt
2024-04-16 21:08:31,355 INFO Check pod logs for errors
[2024-04-16 21:08:31] - INFO: Deployment ready
________________________________________________________________________________
[2024-04-16 21:08:31] Discover_all_components post : Post method
________________________________________________________________________________
Setting result to PASS
Task has been successfully stopped