--Task--
name: Discover_all_components
enabled: True
class_name: DiscoverComponentsTask
source_name: controller
source_namespace: >default<
target_name: controller
target_namespace: >default<
start: 0.0
stop: None
timeout: no timeout
loop: False
interval: None
dependencies: []
wait_for: []
preceding_task: None
options: {}
group_name: None
Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock
________________________________________________________________________________
[2024-04-14 17:24:35] Discover_all_components pre : Initialising task parameters
________________________________________________________________________________
task will be executed on controller (localhost)
________________________________________________________________________________
[2024-04-14 17:24:35] Discover_all_components step1 : Discover components
________________________________________________________________________________
2024-04-14 17:24:35,551 INFO Deployment was skipped.
2024-04-14 17:24:37,835 INFO
2024-04-14 17:24:37,835 INFO ************************************* Creating deployment *************************************
2024-04-14 17:24:37,835 INFO Creating normal (forgeops) type deployment for deployment: stack
2024-04-14 17:24:37,863 INFO
2024-04-14 17:24:37,864 INFO ------- Custom component configuration present. Loading values -------
2024-04-14 17:24:37,864 INFO
2024-04-14 17:24:37,865 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds cts
2024-04-14 17:24:37,973 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:24:37,973 DEBUG --- stdout ---
2024-04-14 17:24:37,974 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts
2024-04-14 17:24:37,974 DEBUG --- stderr ---
2024-04-14 17:24:37,974 DEBUG
2024-04-14 17:24:37,974 INFO
2024-04-14 17:24:37,974 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903
2024-04-14 17:24:37,975 INFO
2024-04-14 17:24:37,975 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds idrepo
2024-04-14 17:24:38,080 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:24:38,080 DEBUG --- stdout ---
2024-04-14 17:24:38,080 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo
2024-04-14 17:24:38,080 DEBUG --- stderr ---
2024-04-14 17:24:38,080 DEBUG
2024-04-14 17:24:38,080 INFO
2024-04-14 17:24:38,081 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903
2024-04-14 17:24:38,081 INFO
2024-04-14 17:24:38,081 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker am
2024-04-14 17:24:38,177 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:24:38,177 DEBUG --- stdout ---
2024-04-14 17:24:38,177 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am
2024-04-14 17:24:38,177 DEBUG --- stderr ---
2024-04-14 17:24:38,177 DEBUG
2024-04-14 17:24:38,177 INFO
2024-04-14 17:24:38,178 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am/Dockerfile: gcr.io/forgerock-io/am-cdk/pit1:7.6.0-bbf6791c961b39ef101ebad36686ee2c3cde94bb
2024-04-14 17:24:38,178 INFO
2024-04-14 17:24:38,178 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker amster
2024-04-14 17:24:38,266 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:24:38,266 DEBUG --- stdout ---
2024-04-14 17:24:38,266 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster
2024-04-14 17:24:38,266 DEBUG --- stderr ---
2024-04-14 17:24:38,267 DEBUG
2024-04-14 17:24:38,267 INFO
2024-04-14 17:24:38,267 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster/Dockerfile: gcr.io/forgerock-io/amster/pit1:7.6.0-bbf6791c961b39ef101ebad36686ee2c3cde94bb
2024-04-14 17:24:38,267 INFO
2024-04-14 17:24:38,267 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker idm
2024-04-14 17:24:38,340 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:24:38,341 DEBUG --- stdout ---
2024-04-14 17:24:38,341 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm
2024-04-14 17:24:38,341 DEBUG --- stderr ---
2024-04-14 17:24:38,341 DEBUG
2024-04-14 17:24:38,341 INFO
2024-04-14 17:24:38,341 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm/Dockerfile: gcr.io/forgerock-io/idm-cdk/pit1:7.6.0-8554e0d1c5044f4f6c6504cd366196848874e695
2024-04-14 17:24:38,341 INFO
2024-04-14 17:24:38,341 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base end-user-ui
2024-04-14 17:24:38,415 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:24:38,415 DEBUG --- stdout ---
2024-04-14 17:24:38,415 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui
2024-04-14 17:24:38,415 DEBUG --- stderr ---
2024-04-14 17:24:38,415 DEBUG
2024-04-14 17:24:38,415 INFO
2024-04-14 17:24:38,418 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui/kustomization.yaml: gcr.io/forgerock-io/platform-enduser-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-14 17:24:38,418 INFO
2024-04-14 17:24:38,418 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base login-ui
2024-04-14 17:24:38,493 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:24:38,493 DEBUG --- stdout ---
2024-04-14 17:24:38,493 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui
2024-04-14 17:24:38,493 DEBUG --- stderr ---
2024-04-14 17:24:38,493 DEBUG
2024-04-14 17:24:38,493 INFO
2024-04-14 17:24:38,496 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui/kustomization.yaml: gcr.io/forgerock-io/platform-login-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-14 17:24:38,496 INFO
2024-04-14 17:24:38,496 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base admin-ui
2024-04-14 17:24:38,571 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:24:38,571 DEBUG --- stdout ---
2024-04-14 17:24:38,571 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui
2024-04-14 17:24:38,571 DEBUG --- stderr ---
2024-04-14 17:24:38,571 DEBUG
2024-04-14 17:24:38,571 INFO
2024-04-14 17:24:38,579 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui/kustomization.yaml: gcr.io/forgerock-io/platform-admin-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-14 17:24:38,579 INFO Loading external components from config.yaml
2024-04-14 17:24:38,696 INFO No external components defined.
2024-04-14 17:24:38,814 INFO Setting context for components to: gke_engineeringpit_us-east1-d_xlou-cdm
2024-04-14 17:24:38,815 INFO
2024-04-14 17:24:38,815 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-14 17:24:38,815 INFO
2024-04-14 17:24:38,815 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:24:38,815 INFO
2024-04-14 17:24:38,815 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:24:38,815 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:39,199 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:39,199 DEBUG --- stdout ---
2024-04-14 17:24:39,200 DEBUG 3
2024-04-14 17:24:39,200 DEBUG --- stderr ---
2024-04-14 17:24:39,200 DEBUG
2024-04-14 17:24:39,200 INFO
2024-04-14 17:24:39,200 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:24:39,200 INFO
2024-04-14 17:24:39,200 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-14 17:24:39,200 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:24:39,635 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-14 17:24:39,635 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:39,635 DEBUG --- stdout ---
2024-04-14 17:24:39,635 DEBUG ds-cts-0 ds-cts-1 ds-cts-2
2024-04-14 17:24:39,635 DEBUG --- stderr ---
2024-04-14 17:24:39,635 DEBUG
2024-04-14 17:24:39,635 INFO
2024-04-14 17:24:39,636 INFO -------------------- Check pod ds-cts-0 is running --------------------
2024-04-14 17:24:39,636 INFO
2024-04-14 17:24:39,636 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:24:39,636 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:40,015 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:40,016 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:40,016 DEBUG --- stdout ---
2024-04-14 17:24:40,016 DEBUG Running
2024-04-14 17:24:40,016 DEBUG --- stderr ---
2024-04-14 17:24:40,016 DEBUG
2024-04-14 17:24:40,016 INFO
2024-04-14 17:24:40,016 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:24:40,016 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:40,371 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:40,372 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:40,372 DEBUG --- stdout ---
2024-04-14 17:24:40,372 DEBUG true
2024-04-14 17:24:40,372 DEBUG --- stderr ---
2024-04-14 17:24:40,372 DEBUG
2024-04-14 17:24:40,372 INFO
2024-04-14 17:24:40,372 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.startTime}
2024-04-14 17:24:40,372 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:40,730 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:40,730 DEBUG --- stdout ---
2024-04-14 17:24:40,730 DEBUG 2024-04-13T22:38:33Z
2024-04-14 17:24:40,730 DEBUG --- stderr ---
2024-04-14 17:24:40,730 DEBUG
2024-04-14 17:24:40,731 INFO
2024-04-14 17:24:40,731 INFO ------------- Check pod ds-cts-0 filesystem is accessible -------------
2024-04-14 17:24:41,567 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:41,567 INFO
2024-04-14 17:24:41,567 INFO ------------------ Check pod ds-cts-0 restart count ------------------
2024-04-14 17:24:41,567 INFO
2024-04-14 17:24:41,567 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:24:41,567 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:41,935 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:41,936 DEBUG --- stdout ---
2024-04-14 17:24:41,936 DEBUG 0
2024-04-14 17:24:41,936 DEBUG --- stderr ---
2024-04-14 17:24:41,936 DEBUG
2024-04-14 17:24:41,936 INFO Pod ds-cts-0 has been restarted 0 times.
2024-04-14 17:24:41,936 INFO
2024-04-14 17:24:41,936 INFO -------------------- Check pod ds-cts-1 is running --------------------
2024-04-14 17:24:41,936 INFO
2024-04-14 17:24:41,936 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:24:41,936 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:42,292 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:42,292 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:42,292 DEBUG --- stdout ---
2024-04-14 17:24:42,292 DEBUG Running
2024-04-14 17:24:42,292 DEBUG --- stderr ---
2024-04-14 17:24:42,292 DEBUG
2024-04-14 17:24:42,293 INFO
2024-04-14 17:24:42,293 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:24:42,293 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:42,664 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:42,665 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:42,665 DEBUG --- stdout ---
2024-04-14 17:24:42,665 DEBUG true
2024-04-14 17:24:42,665 DEBUG --- stderr ---
2024-04-14 17:24:42,665 DEBUG
2024-04-14 17:24:42,665 INFO
2024-04-14 17:24:42,665 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.startTime}
2024-04-14 17:24:42,665 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:43,044 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:43,044 DEBUG --- stdout ---
2024-04-14 17:24:43,044 DEBUG 2024-04-13T22:39:01Z
2024-04-14 17:24:43,044 DEBUG --- stderr ---
2024-04-14 17:24:43,044 DEBUG
2024-04-14 17:24:43,045 INFO
2024-04-14 17:24:43,045 INFO ------------- Check pod ds-cts-1 filesystem is accessible -------------
2024-04-14 17:24:43,890 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:43,890 INFO
2024-04-14 17:24:43,890 INFO ------------------ Check pod ds-cts-1 restart count ------------------
2024-04-14 17:24:43,890 INFO
2024-04-14 17:24:43,890 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:24:43,890 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:44,254 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:44,255 DEBUG --- stdout ---
2024-04-14 17:24:44,255 DEBUG 0
2024-04-14 17:24:44,255 DEBUG --- stderr ---
2024-04-14 17:24:44,255 DEBUG
2024-04-14 17:24:44,255 INFO Pod ds-cts-1 has been restarted 0 times.
2024-04-14 17:24:44,255 INFO
2024-04-14 17:24:44,255 INFO -------------------- Check pod ds-cts-2 is running --------------------
2024-04-14 17:24:44,255 INFO
2024-04-14 17:24:44,255 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:24:44,255 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:44,629 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:44,629 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:44,629 DEBUG --- stdout ---
2024-04-14 17:24:44,630 DEBUG Running
2024-04-14 17:24:44,630 DEBUG --- stderr ---
2024-04-14 17:24:44,630 DEBUG
2024-04-14 17:24:44,630 INFO
2024-04-14 17:24:44,630 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:24:44,630 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:45,001 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:45,001 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:45,001 DEBUG --- stdout ---
2024-04-14 17:24:45,002 DEBUG true
2024-04-14 17:24:45,002 DEBUG --- stderr ---
2024-04-14 17:24:45,002 DEBUG
2024-04-14 17:24:45,002 INFO
2024-04-14 17:24:45,002 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.startTime}
2024-04-14 17:24:45,002 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:45,374 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:45,374 DEBUG --- stdout ---
2024-04-14 17:24:45,375 DEBUG 2024-04-13T22:39:31Z
2024-04-14 17:24:45,375 DEBUG --- stderr ---
2024-04-14 17:24:45,375 DEBUG
2024-04-14 17:24:45,375 INFO
2024-04-14 17:24:45,375 INFO ------------- Check pod ds-cts-2 filesystem is accessible -------------
2024-04-14 17:24:46,212 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:46,212 INFO
2024-04-14 17:24:46,212 INFO ------------------ Check pod ds-cts-2 restart count ------------------
2024-04-14 17:24:46,213 INFO
2024-04-14 17:24:46,213 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:24:46,213 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:46,570 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:46,570 DEBUG --- stdout ---
2024-04-14 17:24:46,571 DEBUG 0
2024-04-14 17:24:46,571 DEBUG --- stderr ---
2024-04-14 17:24:46,571 DEBUG
2024-04-14 17:24:46,571 INFO Pod ds-cts-2 has been restarted 0 times.
2024-04-14 17:24:46,571 INFO
2024-04-14 17:24:46,571 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-14 17:24:46,571 INFO
2024-04-14 17:24:46,571 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:24:46,571 INFO
2024-04-14 17:24:46,571 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:24:46,571 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:46,936 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:46,937 DEBUG --- stdout ---
2024-04-14 17:24:46,937 DEBUG 3
2024-04-14 17:24:46,937 DEBUG --- stderr ---
2024-04-14 17:24:46,937 DEBUG
2024-04-14 17:24:46,937 INFO
2024-04-14 17:24:46,937 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:24:46,937 INFO
2024-04-14 17:24:46,937 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-14 17:24:46,937 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:24:47,371 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-14 17:24:47,371 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:47,371 DEBUG --- stdout ---
2024-04-14 17:24:47,372 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2
2024-04-14 17:24:47,372 DEBUG --- stderr ---
2024-04-14 17:24:47,372 DEBUG
2024-04-14 17:24:47,372 INFO
2024-04-14 17:24:47,372 INFO ------------------ Check pod ds-idrepo-0 is running ------------------
2024-04-14 17:24:47,372 INFO
2024-04-14 17:24:47,372 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:24:47,372 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:47,727 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:47,727 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:47,727 DEBUG --- stdout ---
2024-04-14 17:24:47,727 DEBUG Running
2024-04-14 17:24:47,727 DEBUG --- stderr ---
2024-04-14 17:24:47,727 DEBUG
2024-04-14 17:24:47,727 INFO
2024-04-14 17:24:47,727 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:24:47,727 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:48,080 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:48,081 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:48,081 DEBUG --- stdout ---
2024-04-14 17:24:48,081 DEBUG true
2024-04-14 17:24:48,081 DEBUG --- stderr ---
2024-04-14 17:24:48,081 DEBUG
2024-04-14 17:24:48,082 INFO
2024-04-14 17:24:48,082 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.startTime}
2024-04-14 17:24:48,082 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:48,439 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:48,439 DEBUG --- stdout ---
2024-04-14 17:24:48,439 DEBUG 2024-04-13T22:38:34Z
2024-04-14 17:24:48,439 DEBUG --- stderr ---
2024-04-14 17:24:48,439 DEBUG
2024-04-14 17:24:48,439 INFO
2024-04-14 17:24:48,440 INFO ----------- Check pod ds-idrepo-0 filesystem is accessible -----------
2024-04-14 17:24:49,251 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:49,251 INFO
2024-04-14 17:24:49,251 INFO ----------------- Check pod ds-idrepo-0 restart count -----------------
2024-04-14 17:24:49,251 INFO
2024-04-14 17:24:49,251 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:24:49,251 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:49,621 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:49,621 DEBUG --- stdout ---
2024-04-14 17:24:49,621 DEBUG 0
2024-04-14 17:24:49,621 DEBUG --- stderr ---
2024-04-14 17:24:49,621 DEBUG
2024-04-14 17:24:49,621 INFO Pod ds-idrepo-0 has been restarted 0 times.
2024-04-14 17:24:49,621 INFO
2024-04-14 17:24:49,621 INFO ------------------ Check pod ds-idrepo-1 is running ------------------
2024-04-14 17:24:49,622 INFO
2024-04-14 17:24:49,622 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:24:49,622 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:49,995 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:49,995 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:49,995 DEBUG --- stdout ---
2024-04-14 17:24:49,995 DEBUG Running
2024-04-14 17:24:49,995 DEBUG --- stderr ---
2024-04-14 17:24:49,995 DEBUG
2024-04-14 17:24:49,995 INFO
2024-04-14 17:24:49,995 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:24:49,995 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:50,375 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:50,375 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:50,375 DEBUG --- stdout ---
2024-04-14 17:24:50,376 DEBUG true
2024-04-14 17:24:50,376 DEBUG --- stderr ---
2024-04-14 17:24:50,376 DEBUG
2024-04-14 17:24:50,376 INFO
2024-04-14 17:24:50,376 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.startTime}
2024-04-14 17:24:50,376 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:50,751 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:50,751 DEBUG --- stdout ---
2024-04-14 17:24:50,752 DEBUG 2024-04-13T22:39:10Z
2024-04-14 17:24:50,752 DEBUG --- stderr ---
2024-04-14 17:24:50,752 DEBUG
2024-04-14 17:24:50,752 INFO
2024-04-14 17:24:50,752 INFO ----------- Check pod ds-idrepo-1 filesystem is accessible -----------
2024-04-14 17:24:51,586 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:51,587 INFO
2024-04-14 17:24:51,587 INFO ----------------- Check pod ds-idrepo-1 restart count -----------------
2024-04-14 17:24:51,587 INFO
2024-04-14 17:24:51,587 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:24:51,587 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:51,969 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:51,969 DEBUG --- stdout ---
2024-04-14 17:24:51,969 DEBUG 0
2024-04-14 17:24:51,970 DEBUG --- stderr ---
2024-04-14 17:24:51,970 DEBUG
2024-04-14 17:24:51,970 INFO Pod ds-idrepo-1 has been restarted 0 times.
2024-04-14 17:24:51,970 INFO
2024-04-14 17:24:51,970 INFO ------------------ Check pod ds-idrepo-2 is running ------------------
2024-04-14 17:24:51,970 INFO
2024-04-14 17:24:51,970 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:24:51,970 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:52,347 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:52,347 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:52,347 DEBUG --- stdout ---
2024-04-14 17:24:52,347 DEBUG Running
2024-04-14 17:24:52,347 DEBUG --- stderr ---
2024-04-14 17:24:52,347 DEBUG
2024-04-14 17:24:52,348 INFO
2024-04-14 17:24:52,348 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:24:52,348 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:52,709 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:52,710 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:52,710 DEBUG --- stdout ---
2024-04-14 17:24:52,710 DEBUG true
2024-04-14 17:24:52,710 DEBUG --- stderr ---
2024-04-14 17:24:52,710 DEBUG
2024-04-14 17:24:52,710 INFO
2024-04-14 17:24:52,710 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.startTime}
2024-04-14 17:24:52,710 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:53,068 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:53,068 DEBUG --- stdout ---
2024-04-14 17:24:53,068 DEBUG 2024-04-13T22:39:50Z
2024-04-14 17:24:53,068 DEBUG --- stderr ---
2024-04-14 17:24:53,068 DEBUG
2024-04-14 17:24:53,069 INFO
2024-04-14 17:24:53,069 INFO ----------- Check pod ds-idrepo-2 filesystem is accessible -----------
2024-04-14 17:24:53,889 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:53,889 INFO
2024-04-14 17:24:53,889 INFO ----------------- Check pod ds-idrepo-2 restart count -----------------
2024-04-14 17:24:53,890 INFO
2024-04-14 17:24:53,890 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:24:53,890 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:54,265 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:54,266 DEBUG --- stdout ---
2024-04-14 17:24:54,266 DEBUG 0
2024-04-14 17:24:54,266 DEBUG --- stderr ---
2024-04-14 17:24:54,266 DEBUG
2024-04-14 17:24:54,266 INFO Pod ds-idrepo-2 has been restarted 0 times.
2024-04-14 17:24:54,266 INFO
2024-04-14 17:24:54,266 INFO ****************************** Initializing component pods for AM ******************************
2024-04-14 17:24:54,266 INFO
2024-04-14 17:24:54,266 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:24:54,266 INFO
2024-04-14 17:24:54,266 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:24:54,266 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:54,674 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:54,675 DEBUG --- stdout ---
2024-04-14 17:24:54,675 DEBUG 3
2024-04-14 17:24:54,675 DEBUG --- stderr ---
2024-04-14 17:24:54,675 DEBUG
2024-04-14 17:24:54,675 INFO
2024-04-14 17:24:54,675 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:24:54,675 INFO
2024-04-14 17:24:54,675 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-14 17:24:54,675 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:24:55,109 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-14 17:24:55,109 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:55,109 DEBUG --- stdout ---
2024-04-14 17:24:55,109 DEBUG am-7d597cf7d6-8ctft am-7d597cf7d6-8vr8f am-7d597cf7d6-9wz9h
2024-04-14 17:24:55,109 DEBUG --- stderr ---
2024-04-14 17:24:55,109 DEBUG
2024-04-14 17:24:55,109 INFO
2024-04-14 17:24:55,109 INFO -------------- Check pod am-7d597cf7d6-8ctft is running --------------
2024-04-14 17:24:55,110 INFO
2024-04-14 17:24:55,110 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-7d597cf7d6-8ctft -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:24:55,110 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:55,490 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:55,490 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:55,490 DEBUG --- stdout ---
2024-04-14 17:24:55,490 DEBUG Running
2024-04-14 17:24:55,490 DEBUG --- stderr ---
2024-04-14 17:24:55,490 DEBUG
2024-04-14 17:24:55,491 INFO
2024-04-14 17:24:55,491 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-7d597cf7d6-8ctft -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:24:55,491 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:55,864 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:55,864 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:55,864 DEBUG --- stdout ---
2024-04-14 17:24:55,864 DEBUG true
2024-04-14 17:24:55,864 DEBUG --- stderr ---
2024-04-14 17:24:55,864 DEBUG
2024-04-14 17:24:55,864 INFO
2024-04-14 17:24:55,864 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-7d597cf7d6-8ctft --output jsonpath={.status.startTime}
2024-04-14 17:24:55,864 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:56,243 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:56,244 DEBUG --- stdout ---
2024-04-14 17:24:56,244 DEBUG 2024-04-13T22:40:34Z
2024-04-14 17:24:56,244 DEBUG --- stderr ---
2024-04-14 17:24:56,244 DEBUG
2024-04-14 17:24:56,244 INFO
2024-04-14 17:24:56,245 INFO ------- Check pod am-7d597cf7d6-8ctft filesystem is accessible -------
2024-04-14 17:24:57,087 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:57,087 INFO
2024-04-14 17:24:57,087 INFO ------------- Check pod am-7d597cf7d6-8ctft restart count -------------
2024-04-14 17:24:57,088 INFO
2024-04-14 17:24:57,088 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-7d597cf7d6-8ctft --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:24:57,088 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:57,462 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:57,463 DEBUG --- stdout ---
2024-04-14 17:24:57,463 DEBUG 0
2024-04-14 17:24:57,463 DEBUG --- stderr ---
2024-04-14 17:24:57,463 DEBUG
2024-04-14 17:24:57,463 INFO Pod am-7d597cf7d6-8ctft has been restarted 0 times.
2024-04-14 17:24:57,463 INFO
2024-04-14 17:24:57,463 INFO -------------- Check pod am-7d597cf7d6-8vr8f is running --------------
2024-04-14 17:24:57,463 INFO
2024-04-14 17:24:57,463 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-7d597cf7d6-8vr8f -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:24:57,463 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:57,821 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:57,821 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:57,821 DEBUG --- stdout ---
2024-04-14 17:24:57,821 DEBUG Running
2024-04-14 17:24:57,821 DEBUG --- stderr ---
2024-04-14 17:24:57,821 DEBUG
2024-04-14 17:24:57,821 INFO
2024-04-14 17:24:57,821 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-7d597cf7d6-8vr8f -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:24:57,821 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:24:58,184 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:58,184 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:58,184 DEBUG --- stdout ---
2024-04-14 17:24:58,184 DEBUG true
2024-04-14 17:24:58,184 DEBUG --- stderr ---
2024-04-14 17:24:58,184 DEBUG
2024-04-14 17:24:58,184 INFO
2024-04-14 17:24:58,184 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-7d597cf7d6-8vr8f --output jsonpath={.status.startTime}
2024-04-14 17:24:58,184 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:58,547 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:58,547 DEBUG --- stdout ---
2024-04-14 17:24:58,547 DEBUG 2024-04-13T22:40:34Z
2024-04-14 17:24:58,547 DEBUG --- stderr ---
2024-04-14 17:24:58,547 DEBUG
2024-04-14 17:24:58,548 INFO
2024-04-14 17:24:58,548 INFO ------- Check pod am-7d597cf7d6-8vr8f filesystem is accessible -------
2024-04-14 17:24:59,423 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:24:59,423 INFO
2024-04-14 17:24:59,425 INFO ------------- Check pod am-7d597cf7d6-8vr8f restart count -------------
2024-04-14 17:24:59,425 INFO
2024-04-14 17:24:59,425 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-7d597cf7d6-8vr8f --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:24:59,425 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:24:59,794 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:24:59,794 DEBUG --- stdout ---
2024-04-14 17:24:59,794 DEBUG 0
2024-04-14 17:24:59,794 DEBUG --- stderr ---
2024-04-14 17:24:59,794 DEBUG
2024-04-14 17:24:59,794 INFO Pod am-7d597cf7d6-8vr8f has been restarted 0 times.
2024-04-14 17:24:59,794 INFO
2024-04-14 17:24:59,794 INFO -------------- Check pod am-7d597cf7d6-9wz9h is running --------------
2024-04-14 17:24:59,795 INFO
2024-04-14 17:24:59,795 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-7d597cf7d6-9wz9h -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:24:59,795 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:00,150 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:00,150 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:00,150 DEBUG --- stdout ---
2024-04-14 17:25:00,150 DEBUG Running
2024-04-14 17:25:00,150 DEBUG --- stderr ---
2024-04-14 17:25:00,150 DEBUG
2024-04-14 17:25:00,151 INFO
2024-04-14 17:25:00,151 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-7d597cf7d6-9wz9h -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:25:00,151 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:00,506 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:00,506 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:00,506 DEBUG --- stdout ---
2024-04-14 17:25:00,506 DEBUG true
2024-04-14 17:25:00,506 DEBUG --- stderr ---
2024-04-14 17:25:00,506 DEBUG
2024-04-14 17:25:00,507 INFO
2024-04-14 17:25:00,507 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-7d597cf7d6-9wz9h --output jsonpath={.status.startTime}
2024-04-14 17:25:00,507 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:00,864 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:00,864 DEBUG --- stdout ---
2024-04-14 17:25:00,864 DEBUG 2024-04-13T22:40:34Z
2024-04-14 17:25:00,865 DEBUG --- stderr ---
2024-04-14 17:25:00,865 DEBUG
2024-04-14 17:25:00,865 INFO
2024-04-14 17:25:00,865 INFO ------- Check pod am-7d597cf7d6-9wz9h filesystem is accessible -------
2024-04-14 17:25:01,682 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:01,683 INFO
2024-04-14 17:25:01,683 INFO ------------- Check pod am-7d597cf7d6-9wz9h restart count -------------
2024-04-14 17:25:01,683 INFO
2024-04-14 17:25:01,683 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-7d597cf7d6-9wz9h --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:25:01,683 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:02,046 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:02,047 DEBUG --- stdout ---
2024-04-14 17:25:02,047 DEBUG 0
2024-04-14 17:25:02,047 DEBUG --- stderr ---
2024-04-14 17:25:02,047 DEBUG
2024-04-14 17:25:02,047 INFO Pod am-7d597cf7d6-9wz9h has been restarted 0 times.
2024-04-14 17:25:02,047 INFO
2024-04-14 17:25:02,047 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-14 17:25:02,047 INFO
2024-04-14 17:25:02,047 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:02,047 INFO
2024-04-14 17:25:02,047 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:02,047 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:02,404 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:02,405 DEBUG --- stdout ---
2024-04-14 17:25:02,405 DEBUG amster-qv96p
2024-04-14 17:25:02,405 DEBUG --- stderr ---
2024-04-14 17:25:02,405 DEBUG
2024-04-14 17:25:02,405 INFO
2024-04-14 17:25:02,405 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-14 17:25:02,405 INFO
2024-04-14 17:25:02,405 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:02,405 INFO
2024-04-14 17:25:02,405 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:02,405 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:02,783 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:02,783 DEBUG --- stdout ---
2024-04-14 17:25:02,783 DEBUG 2
2024-04-14 17:25:02,783 DEBUG --- stderr ---
2024-04-14 17:25:02,783 DEBUG
2024-04-14 17:25:02,783 INFO
2024-04-14 17:25:02,783 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:02,784 INFO
2024-04-14 17:25:02,784 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name}` | grep 2
2024-04-14 17:25:02,784 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:03,177 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-14 17:25:03,178 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:03,178 DEBUG --- stdout ---
2024-04-14 17:25:03,178 DEBUG idm-b85f9d978-j5pbm idm-b85f9d978-rl76f
2024-04-14 17:25:03,178 DEBUG --- stderr ---
2024-04-14 17:25:03,178 DEBUG
2024-04-14 17:25:03,178 INFO
2024-04-14 17:25:03,178 INFO -------------- Check pod idm-b85f9d978-j5pbm is running --------------
2024-04-14 17:25:03,178 INFO
2024-04-14 17:25:03,178 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-j5pbm -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:25:03,178 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:03,551 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:03,551 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:03,551 DEBUG --- stdout ---
2024-04-14 17:25:03,551 DEBUG Running
2024-04-14 17:25:03,551 DEBUG --- stderr ---
2024-04-14 17:25:03,551 DEBUG
2024-04-14 17:25:03,551 INFO
2024-04-14 17:25:03,551 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-j5pbm -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:25:03,551 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:03,920 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:03,920 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:03,920 DEBUG --- stdout ---
2024-04-14 17:25:03,920 DEBUG true
2024-04-14 17:25:03,921 DEBUG --- stderr ---
2024-04-14 17:25:03,921 DEBUG
2024-04-14 17:25:03,921 INFO
2024-04-14 17:25:03,921 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-j5pbm --output jsonpath={.status.startTime}
2024-04-14 17:25:03,921 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:04,277 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:04,278 DEBUG --- stdout ---
2024-04-14 17:25:04,278 DEBUG 2024-04-13T22:40:34Z
2024-04-14 17:25:04,278 DEBUG --- stderr ---
2024-04-14 17:25:04,278 DEBUG
2024-04-14 17:25:04,278 INFO
2024-04-14 17:25:04,278 INFO ------- Check pod idm-b85f9d978-j5pbm filesystem is accessible -------
2024-04-14 17:25:05,096 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:05,096 INFO
2024-04-14 17:25:05,096 INFO ------------- Check pod idm-b85f9d978-j5pbm restart count -------------
2024-04-14 17:25:05,096 INFO
2024-04-14 17:25:05,097 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-j5pbm --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:25:05,097 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:05,478 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:05,478 DEBUG --- stdout ---
2024-04-14 17:25:05,478 DEBUG 0
2024-04-14 17:25:05,478 DEBUG --- stderr ---
2024-04-14 17:25:05,478 DEBUG
2024-04-14 17:25:05,478 INFO Pod idm-b85f9d978-j5pbm has been restarted 0 times.
2024-04-14 17:25:05,479 INFO
2024-04-14 17:25:05,479 INFO -------------- Check pod idm-b85f9d978-rl76f is running --------------
2024-04-14 17:25:05,479 INFO
2024-04-14 17:25:05,479 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-rl76f -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:25:05,479 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:05,853 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:05,853 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:05,853 DEBUG --- stdout ---
2024-04-14 17:25:05,853 DEBUG Running
2024-04-14 17:25:05,853 DEBUG --- stderr ---
2024-04-14 17:25:05,853 DEBUG
2024-04-14 17:25:05,854 INFO
2024-04-14 17:25:05,854 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-rl76f -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:25:05,854 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:06,261 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:06,261 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:06,261 DEBUG --- stdout ---
2024-04-14 17:25:06,261 DEBUG true
2024-04-14 17:25:06,261 DEBUG --- stderr ---
2024-04-14 17:25:06,262 DEBUG
2024-04-14 17:25:06,262 INFO
2024-04-14 17:25:06,262 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-rl76f --output jsonpath={.status.startTime}
2024-04-14 17:25:06,262 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:06,645 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:06,645 DEBUG --- stdout ---
2024-04-14 17:25:06,645 DEBUG 2024-04-13T22:40:34Z
2024-04-14 17:25:06,645 DEBUG --- stderr ---
2024-04-14 17:25:06,645 DEBUG
2024-04-14 17:25:06,646 INFO
2024-04-14 17:25:06,646 INFO ------- Check pod idm-b85f9d978-rl76f filesystem is accessible -------
2024-04-14 17:25:07,487 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:07,488 INFO
2024-04-14 17:25:07,488 INFO ------------- Check pod idm-b85f9d978-rl76f restart count -------------
2024-04-14 17:25:07,488 INFO
2024-04-14 17:25:07,488 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-rl76f --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:25:07,488 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:07,866 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:07,866 DEBUG --- stdout ---
2024-04-14 17:25:07,866 DEBUG 0
2024-04-14 17:25:07,866 DEBUG --- stderr ---
2024-04-14 17:25:07,866 DEBUG
2024-04-14 17:25:07,866 INFO Pod idm-b85f9d978-rl76f has been restarted 0 times.
2024-04-14 17:25:07,866 INFO
2024-04-14 17:25:07,866 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-14 17:25:07,866 INFO
2024-04-14 17:25:07,867 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:07,867 INFO
2024-04-14 17:25:07,867 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:07,867 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:08,186 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:08,187 DEBUG --- stdout ---
2024-04-14 17:25:08,187 DEBUG 1
2024-04-14 17:25:08,187 DEBUG --- stderr ---
2024-04-14 17:25:08,187 DEBUG
2024-04-14 17:25:08,187 INFO
2024-04-14 17:25:08,187 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:08,187 INFO
2024-04-14 17:25:08,187 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-14 17:25:08,187 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:08,513 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-14 17:25:08,513 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:08,513 DEBUG --- stdout ---
2024-04-14 17:25:08,513 DEBUG end-user-ui-58878d7668-zg4h2
2024-04-14 17:25:08,513 DEBUG --- stderr ---
2024-04-14 17:25:08,513 DEBUG
2024-04-14 17:25:08,514 INFO
2024-04-14 17:25:08,514 INFO ---------- Check pod end-user-ui-58878d7668-zg4h2 is running ----------
2024-04-14 17:25:08,514 INFO
2024-04-14 17:25:08,514 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-58878d7668-zg4h2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:25:08,514 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:08,843 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:08,843 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:08,843 DEBUG --- stdout ---
2024-04-14 17:25:08,843 DEBUG Running
2024-04-14 17:25:08,843 DEBUG --- stderr ---
2024-04-14 17:25:08,843 DEBUG
2024-04-14 17:25:08,844 INFO
2024-04-14 17:25:08,844 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-58878d7668-zg4h2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:25:08,844 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:09,139 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:09,139 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:09,139 DEBUG --- stdout ---
2024-04-14 17:25:09,139 DEBUG true
2024-04-14 17:25:09,139 DEBUG --- stderr ---
2024-04-14 17:25:09,139 DEBUG
2024-04-14 17:25:09,139 INFO
2024-04-14 17:25:09,139 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-58878d7668-zg4h2 --output jsonpath={.status.startTime}
2024-04-14 17:25:09,139 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:09,437 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:09,437 DEBUG --- stdout ---
2024-04-14 17:25:09,437 DEBUG 2024-04-13T22:41:43Z
2024-04-14 17:25:09,437 DEBUG --- stderr ---
2024-04-14 17:25:09,437 DEBUG
2024-04-14 17:25:09,437 INFO
2024-04-14 17:25:09,438 INFO --- Check pod end-user-ui-58878d7668-zg4h2 filesystem is accessible ---
2024-04-14 17:25:10,202 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:10,202 INFO
2024-04-14 17:25:10,207 INFO -------- Check pod end-user-ui-58878d7668-zg4h2 restart count --------
2024-04-14 17:25:10,208 INFO
2024-04-14 17:25:10,208 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-58878d7668-zg4h2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:25:10,208 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:10,520 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:10,520 DEBUG --- stdout ---
2024-04-14 17:25:10,521 DEBUG 0
2024-04-14 17:25:10,521 DEBUG --- stderr ---
2024-04-14 17:25:10,521 DEBUG
2024-04-14 17:25:10,521 INFO Pod end-user-ui-58878d7668-zg4h2 has been restarted 0 times.
2024-04-14 17:25:10,521 INFO
2024-04-14 17:25:10,521 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-14 17:25:10,521 INFO
2024-04-14 17:25:10,521 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:10,521 INFO
2024-04-14 17:25:10,521 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:10,521 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:10,835 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:10,836 DEBUG --- stdout ---
2024-04-14 17:25:10,836 DEBUG 1
2024-04-14 17:25:10,836 DEBUG --- stderr ---
2024-04-14 17:25:10,836 DEBUG
2024-04-14 17:25:10,836 INFO
2024-04-14 17:25:10,836 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:10,836 INFO
2024-04-14 17:25:10,836 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-14 17:25:10,836 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:11,140 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-14 17:25:11,140 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:11,140 DEBUG --- stdout ---
2024-04-14 17:25:11,140 DEBUG login-ui-5fdf49fc66-2wnx7
2024-04-14 17:25:11,140 DEBUG --- stderr ---
2024-04-14 17:25:11,140 DEBUG
2024-04-14 17:25:11,140 INFO
2024-04-14 17:25:11,141 INFO ----------- Check pod login-ui-5fdf49fc66-2wnx7 is running -----------
2024-04-14 17:25:11,141 INFO
2024-04-14 17:25:11,141 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5fdf49fc66-2wnx7 -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:25:11,141 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:11,459 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:11,459 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:11,459 DEBUG --- stdout ---
2024-04-14 17:25:11,459 DEBUG Running
2024-04-14 17:25:11,459 DEBUG --- stderr ---
2024-04-14 17:25:11,459 DEBUG
2024-04-14 17:25:11,460 INFO
2024-04-14 17:25:11,460 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5fdf49fc66-2wnx7 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:25:11,460 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:11,756 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:11,757 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:11,757 DEBUG --- stdout ---
2024-04-14 17:25:11,757 DEBUG true
2024-04-14 17:25:11,757 DEBUG --- stderr ---
2024-04-14 17:25:11,757 DEBUG
2024-04-14 17:25:11,757 INFO
2024-04-14 17:25:11,757 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5fdf49fc66-2wnx7 --output jsonpath={.status.startTime}
2024-04-14 17:25:11,757 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:12,061 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:12,061 DEBUG --- stdout ---
2024-04-14 17:25:12,062 DEBUG 2024-04-13T22:41:43Z
2024-04-14 17:25:12,062 DEBUG --- stderr ---
2024-04-14 17:25:12,062 DEBUG
2024-04-14 17:25:12,062 INFO
2024-04-14 17:25:12,062 INFO ---- Check pod login-ui-5fdf49fc66-2wnx7 filesystem is accessible ----
2024-04-14 17:25:12,835 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:12,835 INFO
2024-04-14 17:25:12,835 INFO ---------- Check pod login-ui-5fdf49fc66-2wnx7 restart count ----------
2024-04-14 17:25:12,835 INFO
2024-04-14 17:25:12,835 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5fdf49fc66-2wnx7 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:25:12,835 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:13,152 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:13,152 DEBUG --- stdout ---
2024-04-14 17:25:13,152 DEBUG 0
2024-04-14 17:25:13,153 DEBUG --- stderr ---
2024-04-14 17:25:13,153 DEBUG
2024-04-14 17:25:13,153 INFO Pod login-ui-5fdf49fc66-2wnx7 has been restarted 0 times.
2024-04-14 17:25:13,153 INFO
2024-04-14 17:25:13,153 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-14 17:25:13,153 INFO
2024-04-14 17:25:13,153 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:13,153 INFO
2024-04-14 17:25:13,153 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:13,153 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:13,468 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:13,468 DEBUG --- stdout ---
2024-04-14 17:25:13,469 DEBUG 1
2024-04-14 17:25:13,469 DEBUG --- stderr ---
2024-04-14 17:25:13,469 DEBUG
2024-04-14 17:25:13,469 INFO
2024-04-14 17:25:13,469 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:13,469 INFO
2024-04-14 17:25:13,469 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-14 17:25:13,469 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:13,816 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-14 17:25:13,816 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:13,816 DEBUG --- stdout ---
2024-04-14 17:25:13,816 DEBUG admin-ui-c4c7bf584-mbzlq
2024-04-14 17:25:13,816 DEBUG --- stderr ---
2024-04-14 17:25:13,816 DEBUG
2024-04-14 17:25:13,816 INFO
2024-04-14 17:25:13,817 INFO ------------ Check pod admin-ui-c4c7bf584-mbzlq is running ------------
2024-04-14 17:25:13,817 INFO
2024-04-14 17:25:13,817 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-c4c7bf584-mbzlq -o=jsonpath={.status.phase} | grep "Running"
2024-04-14 17:25:13,817 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:14,148 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:14,148 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:14,148 DEBUG --- stdout ---
2024-04-14 17:25:14,148 DEBUG Running
2024-04-14 17:25:14,148 DEBUG --- stderr ---
2024-04-14 17:25:14,148 DEBUG
2024-04-14 17:25:14,148 INFO
2024-04-14 17:25:14,148 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-c4c7bf584-mbzlq -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-14 17:25:14,149 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-14 17:25:14,444 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:14,444 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:14,444 DEBUG --- stdout ---
2024-04-14 17:25:14,444 DEBUG true
2024-04-14 17:25:14,444 DEBUG --- stderr ---
2024-04-14 17:25:14,444 DEBUG
2024-04-14 17:25:14,445 INFO
2024-04-14 17:25:14,445 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-c4c7bf584-mbzlq --output jsonpath={.status.startTime}
2024-04-14 17:25:14,445 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:14,750 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:14,750 DEBUG --- stdout ---
2024-04-14 17:25:14,750 DEBUG 2024-04-13T22:41:42Z
2024-04-14 17:25:14,750 DEBUG --- stderr ---
2024-04-14 17:25:14,750 DEBUG
2024-04-14 17:25:14,751 INFO
2024-04-14 17:25:14,751 INFO ----- Check pod admin-ui-c4c7bf584-mbzlq filesystem is accessible -----
2024-04-14 17:25:15,499 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:15,501 INFO
2024-04-14 17:25:15,501 INFO ---------- Check pod admin-ui-c4c7bf584-mbzlq restart count ----------
2024-04-14 17:25:15,501 INFO
2024-04-14 17:25:15,501 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-c4c7bf584-mbzlq --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-14 17:25:15,501 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:15,800 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:15,800 DEBUG --- stdout ---
2024-04-14 17:25:15,800 DEBUG 0
2024-04-14 17:25:15,800 DEBUG --- stderr ---
2024-04-14 17:25:15,800 DEBUG
2024-04-14 17:25:15,800 INFO Pod admin-ui-c4c7bf584-mbzlq has been restarted 0 times.
2024-04-14 17:25:15,800 INFO
2024-04-14 17:25:15,800 INFO ***************************** Checking DS-CTS component is running *****************************
2024-04-14 17:25:15,801 INFO
2024-04-14 17:25:15,801 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:15,801 INFO
2024-04-14 17:25:15,801 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:15,801 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:16,169 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:16,169 DEBUG --- stdout ---
2024-04-14 17:25:16,169 DEBUG 3
2024-04-14 17:25:16,169 DEBUG --- stderr ---
2024-04-14 17:25:16,169 DEBUG
2024-04-14 17:25:16,169 INFO
2024-04-14 17:25:16,169 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-14 17:25:16,170 INFO
2024-04-14 17:25:16,170 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-cts --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3"
2024-04-14 17:25:16,170 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-14 17:25:16,573 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:16,573 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:16,573 DEBUG --- stdout ---
2024-04-14 17:25:16,573 DEBUG current:3 ready:3 replicas:3
2024-04-14 17:25:16,573 DEBUG --- stderr ---
2024-04-14 17:25:16,573 DEBUG
2024-04-14 17:25:16,573 INFO
2024-04-14 17:25:16,573 INFO *************************** Checking DS-IDREPO component is running ***************************
2024-04-14 17:25:16,573 INFO
2024-04-14 17:25:16,573 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:16,574 INFO
2024-04-14 17:25:16,574 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:16,574 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:16,968 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:16,968 DEBUG --- stdout ---
2024-04-14 17:25:16,968 DEBUG 3
2024-04-14 17:25:16,969 DEBUG --- stderr ---
2024-04-14 17:25:16,969 DEBUG
2024-04-14 17:25:16,969 INFO
2024-04-14 17:25:16,969 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-14 17:25:16,969 INFO
2024-04-14 17:25:16,969 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-idrepo --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3"
2024-04-14 17:25:16,969 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-14 17:25:17,372 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:17,372 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:17,372 DEBUG --- stdout ---
2024-04-14 17:25:17,372 DEBUG current:3 ready:3 replicas:3
2024-04-14 17:25:17,372 DEBUG --- stderr ---
2024-04-14 17:25:17,372 DEBUG
2024-04-14 17:25:17,372 INFO
2024-04-14 17:25:17,372 INFO ******************************* Checking AM component is running *******************************
2024-04-14 17:25:17,372 INFO
2024-04-14 17:25:17,372 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:17,372 INFO
2024-04-14 17:25:17,372 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:17,372 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:17,754 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:17,754 DEBUG --- stdout ---
2024-04-14 17:25:17,754 DEBUG 3
2024-04-14 17:25:17,754 DEBUG --- stderr ---
2024-04-14 17:25:17,754 DEBUG
2024-04-14 17:25:17,755 INFO
2024-04-14 17:25:17,755 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-14 17:25:17,755 INFO
2024-04-14 17:25:17,755 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments am --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:3 replicas:3"
2024-04-14 17:25:17,755 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-14 17:25:18,157 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:18,157 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:18,157 DEBUG --- stdout ---
2024-04-14 17:25:18,158 DEBUG ready:3 replicas:3
2024-04-14 17:25:18,158 DEBUG --- stderr ---
2024-04-14 17:25:18,158 DEBUG
2024-04-14 17:25:18,158 INFO
2024-04-14 17:25:18,158 INFO ***************************** Checking AMSTER component is running *****************************
2024-04-14 17:25:18,158 INFO
2024-04-14 17:25:18,158 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:18,158 INFO
2024-04-14 17:25:18,158 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-14 17:25:18,158 INFO
2024-04-14 17:25:18,158 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get jobs amster --output jsonpath="{.status.succeeded}" | grep "1"
2024-04-14 17:25:18,158 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-14 17:25:18,519 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:18,519 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:18,519 DEBUG --- stdout ---
2024-04-14 17:25:18,519 DEBUG 1
2024-04-14 17:25:18,519 DEBUG --- stderr ---
2024-04-14 17:25:18,519 DEBUG
2024-04-14 17:25:18,519 INFO
2024-04-14 17:25:18,519 INFO ****************************** Checking IDM component is running ******************************
2024-04-14 17:25:18,519 INFO
2024-04-14 17:25:18,519 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:18,520 INFO
2024-04-14 17:25:18,520 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:18,520 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:18,883 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:18,883 DEBUG --- stdout ---
2024-04-14 17:25:18,883 DEBUG 2
2024-04-14 17:25:18,884 DEBUG --- stderr ---
2024-04-14 17:25:18,884 DEBUG
2024-04-14 17:25:18,884 INFO
2024-04-14 17:25:18,884 INFO -------------- Waiting for 2 expected pod(s) to be ready --------------
2024-04-14 17:25:18,884 INFO
2024-04-14 17:25:18,884 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment idm --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:2 replicas:2"
2024-04-14 17:25:18,884 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-14 17:25:19,263 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:19,263 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:19,263 DEBUG --- stdout ---
2024-04-14 17:25:19,263 DEBUG ready:2 replicas:2
2024-04-14 17:25:19,263 DEBUG --- stderr ---
2024-04-14 17:25:19,263 DEBUG
2024-04-14 17:25:19,263 INFO
2024-04-14 17:25:19,263 INFO ************************** Checking END-USER-UI component is running **************************
2024-04-14 17:25:19,263 INFO
2024-04-14 17:25:19,263 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:19,263 INFO
2024-04-14 17:25:19,263 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:19,263 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:19,564 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:19,564 DEBUG --- stdout ---
2024-04-14 17:25:19,564 DEBUG 1
2024-04-14 17:25:19,564 DEBUG --- stderr ---
2024-04-14 17:25:19,565 DEBUG
2024-04-14 17:25:19,565 INFO
2024-04-14 17:25:19,565 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-14 17:25:19,565 INFO
2024-04-14 17:25:19,565 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments end-user-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-14 17:25:19,565 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-14 17:25:19,883 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:19,883 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:19,883 DEBUG --- stdout ---
2024-04-14 17:25:19,883 DEBUG ready:1 replicas:1
2024-04-14 17:25:19,883 DEBUG --- stderr ---
2024-04-14 17:25:19,883 DEBUG
2024-04-14 17:25:19,883 INFO
2024-04-14 17:25:19,884 INFO **************************** Checking LOGIN-UI component is running ****************************
2024-04-14 17:25:19,884 INFO
2024-04-14 17:25:19,884 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:19,884 INFO
2024-04-14 17:25:19,884 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:19,884 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:20,188 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:20,188 DEBUG --- stdout ---
2024-04-14 17:25:20,188 DEBUG 1
2024-04-14 17:25:20,188 DEBUG --- stderr ---
2024-04-14 17:25:20,188 DEBUG
2024-04-14 17:25:20,188 INFO
2024-04-14 17:25:20,188 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-14 17:25:20,188 INFO
2024-04-14 17:25:20,188 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments login-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-14 17:25:20,188 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-14 17:25:20,549 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:20,551 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:20,551 DEBUG --- stdout ---
2024-04-14 17:25:20,551 DEBUG ready:1 replicas:1
2024-04-14 17:25:20,551 DEBUG --- stderr ---
2024-04-14 17:25:20,551 DEBUG
2024-04-14 17:25:20,551 INFO
2024-04-14 17:25:20,551 INFO **************************** Checking ADMIN-UI component is running ****************************
2024-04-14 17:25:20,551 INFO
2024-04-14 17:25:20,551 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:20,552 INFO
2024-04-14 17:25:20,552 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:20,552 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:20,873 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:20,873 DEBUG --- stdout ---
2024-04-14 17:25:20,873 DEBUG 1
2024-04-14 17:25:20,873 DEBUG --- stderr ---
2024-04-14 17:25:20,873 DEBUG
2024-04-14 17:25:20,873 INFO
2024-04-14 17:25:20,874 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-14 17:25:20,874 INFO
2024-04-14 17:25:20,874 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments admin-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-14 17:25:20,874 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-14 17:25:21,224 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-14 17:25:21,224 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:21,224 DEBUG --- stdout ---
2024-04-14 17:25:21,224 DEBUG ready:1 replicas:1
2024-04-14 17:25:21,224 DEBUG --- stderr ---
2024-04-14 17:25:21,224 DEBUG
2024-04-14 17:25:21,224 INFO
2024-04-14 17:25:21,224 INFO ****************************** Livecheck stage: After deployment ******************************
2024-04-14 17:25:21,225 INFO
2024-04-14 17:25:21,225 INFO ---------------------- Running DS-CTS livecheck ----------------------
2024-04-14 17:25:21,225 INFO Livecheck to ds-cts-0
2024-04-14 17:25:21,225 INFO
2024-04-14 17:25:21,225 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}"
2024-04-14 17:25:21,225 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-14 17:25:21,537 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:21,537 DEBUG --- stdout ---
2024-04-14 17:25:21,537 DEBUG YjJFdDlYdUlSY0J3bnRZVWk3SjdmN09BU1M5WWJRcXg=
2024-04-14 17:25:21,537 DEBUG --- stderr ---
2024-04-14 17:25:21,537 DEBUG
2024-04-14 17:25:21,537 INFO
2024-04-14 17:25:21,538 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-14 17:25:23,317 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:25:23,317 DEBUG --- stdout ---
2024-04-14 17:25:23,317 DEBUG dn:
alive: true
2024-04-14 17:25:23,317 DEBUG --- stderr ---
2024-04-14 17:25:23,317 DEBUG
2024-04-14 17:25:23,317 INFO
2024-04-14 17:25:23,317 INFO Livecheck to ds-cts-1
2024-04-14 17:25:23,317 INFO
2024-04-14 17:25:23,318 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-14 17:25:25,069 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:25:25,069 DEBUG --- stdout ---
2024-04-14 17:25:25,069 DEBUG dn:
alive: true
2024-04-14 17:25:25,069 DEBUG --- stderr ---
2024-04-14 17:25:25,069 DEBUG
2024-04-14 17:25:25,069 INFO
2024-04-14 17:25:25,069 INFO Livecheck to ds-cts-2
2024-04-14 17:25:25,069 INFO
2024-04-14 17:25:25,070 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-14 17:25:26,856 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:25:26,857 DEBUG --- stdout ---
2024-04-14 17:25:26,857 DEBUG dn:
alive: true
2024-04-14 17:25:26,857 DEBUG --- stderr ---
2024-04-14 17:25:26,857 DEBUG
2024-04-14 17:25:26,857 INFO
2024-04-14 17:25:26,857 INFO
2024-04-14 17:25:26,857 INFO --------------------- Running DS-IDREPO livecheck ---------------------
2024-04-14 17:25:26,857 INFO Livecheck to ds-idrepo-0
2024-04-14 17:25:26,858 INFO
2024-04-14 17:25:26,858 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}"
2024-04-14 17:25:26,858 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-14 17:25:27,236 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:27,236 DEBUG --- stdout ---
2024-04-14 17:25:27,236 DEBUG YjJFdDlYdUlSY0J3bnRZVWk3SjdmN09BU1M5WWJRcXg=
2024-04-14 17:25:27,236 DEBUG --- stderr ---
2024-04-14 17:25:27,236 DEBUG
2024-04-14 17:25:27,236 INFO
2024-04-14 17:25:27,237 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-14 17:25:29,006 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:25:29,006 DEBUG --- stdout ---
2024-04-14 17:25:29,006 DEBUG dn:
alive: true
2024-04-14 17:25:29,006 DEBUG --- stderr ---
2024-04-14 17:25:29,006 DEBUG
2024-04-14 17:25:29,006 INFO
2024-04-14 17:25:29,007 INFO Livecheck to ds-idrepo-1
2024-04-14 17:25:29,007 INFO
2024-04-14 17:25:29,007 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-14 17:25:30,733 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:25:30,733 DEBUG --- stdout ---
2024-04-14 17:25:30,733 DEBUG dn:
alive: true
2024-04-14 17:25:30,733 DEBUG --- stderr ---
2024-04-14 17:25:30,733 DEBUG
2024-04-14 17:25:30,733 INFO
2024-04-14 17:25:30,734 INFO Livecheck to ds-idrepo-2
2024-04-14 17:25:30,734 INFO
2024-04-14 17:25:30,734 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-14 17:25:32,477 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-14 17:25:32,477 DEBUG --- stdout ---
2024-04-14 17:25:32,479 DEBUG dn:
alive: true
2024-04-14 17:25:32,479 DEBUG --- stderr ---
2024-04-14 17:25:32,479 DEBUG
2024-04-14 17:25:32,479 INFO
2024-04-14 17:25:32,479 INFO
2024-04-14 17:25:32,479 INFO ------------------------ Running AM livecheck ------------------------
2024-04-14 17:25:32,479 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready
2024-04-14 17:25:32,480 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready"
2024-04-14 17:25:32,803 INFO [http_cmd]: http status code OK
2024-04-14 17:25:32,803 DEBUG --- status code ---
2024-04-14 17:25:32,803 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:32,803 DEBUG --- http response ---
2024-04-14 17:25:32,803 DEBUG
2024-04-14 17:25:32,803 INFO
2024-04-14 17:25:32,803 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.AM_PASSWORDS_AMADMIN_CLEAR}"
2024-04-14 17:25:32,803 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-14 17:25:33,163 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:33,163 DEBUG --- stdout ---
2024-04-14 17:25:33,163 DEBUG WU53YjE0T1M1SDNjODNsV0txOWpibG5U
2024-04-14 17:25:33,163 DEBUG --- stderr ---
2024-04-14 17:25:33,163 DEBUG
2024-04-14 17:25:33,163 INFO Authenticate user amadmin via REST
2024-04-14 17:25:33,164 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
2024-04-14 17:25:33,398 INFO [http_cmd]: http status code OK
2024-04-14 17:25:33,398 DEBUG --- status code ---
2024-04-14 17:25:33,398 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:33,398 DEBUG --- http response ---
2024-04-14 17:25:33,399 DEBUG {
"tokenId": "7p3S9FuFqjHNwWdIjT2-QeISgNY.*AAJTSQACMDIAAlNLABxjSlJiWFBhcXJBdFdpZzY2VVdPWm5IbDN4dUE9AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
"successUrl": "/am/console",
"realm": "/"
}
2024-04-14 17:25:33,402 INFO
2024-04-14 17:25:33,403 INFO ---------------------- Running AMSTER livecheck ----------------------
2024-04-14 17:25:33,403 INFO
2024-04-14 17:25:33,403 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:33,403 INFO
2024-04-14 17:25:33,403 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:33,403 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:33,768 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:33,768 DEBUG --- stdout ---
2024-04-14 17:25:33,768 DEBUG amster-qv96p
2024-04-14 17:25:33,768 DEBUG --- stderr ---
2024-04-14 17:25:33,768 DEBUG
2024-04-14 17:25:34,383 INFO Amster livecheck is passed
2024-04-14 17:25:34,383 INFO
2024-04-14 17:25:34,383 INFO ------------------------ Running IDM livecheck ------------------------
2024-04-14 17:25:34,383 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping
2024-04-14 17:25:34,383 INFO
2024-04-14 17:25:34,383 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.OPENIDM_ADMIN_PASSWORD}"
2024-04-14 17:25:34,383 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-14 17:25:34,714 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:34,714 DEBUG --- stdout ---
2024-04-14 17:25:34,714 DEBUG MVVYcW1nZmVQQjROSGdoYkdjdWFzUlhD
2024-04-14 17:25:34,714 DEBUG --- stderr ---
2024-04-14 17:25:34,715 DEBUG
2024-04-14 17:25:34,715 INFO Set admin password: 1UXqmgfePB4NHghbGcuasRXC
2024-04-14 17:25:34,715 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping"
2024-04-14 17:25:34,932 INFO [http_cmd]: http status code OK
2024-04-14 17:25:34,932 DEBUG --- status code ---
2024-04-14 17:25:34,932 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:34,932 DEBUG --- http response ---
2024-04-14 17:25:34,933 DEBUG {
"_id": "",
"_rev": "",
"shortDesc": "OpenIDM ready",
"state": "ACTIVE_READY"
}
2024-04-14 17:25:34,933 INFO Create new LoginSession for user "amadmin"
2024-04-14 17:25:34,933 INFO [LoginSession] Get "access_token" token for user "amadmin"
2024-04-14 17:25:34,933 INFO [LoginSession] Obtaining new "access_token" token from server because it was not obtained yet for the user
2024-04-14 17:25:34,933 INFO [LoginSession] Get "token_id" token for user "amadmin"
2024-04-14 17:25:34,933 INFO [LoginSession] Obtaining new "token_id" token from server because it was not obtained yet for the user
2024-04-14 17:25:34,933 INFO Authenticate user amadmin via REST
2024-04-14 17:25:34,933 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
2024-04-14 17:25:35,155 INFO [http_cmd]: http status code OK
2024-04-14 17:25:35,155 DEBUG --- status code ---
2024-04-14 17:25:35,155 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:35,155 DEBUG --- http response ---
2024-04-14 17:25:35,155 DEBUG {
"tokenId": "6ecCleDCOLEuF_XTx04dREsY6HY.*AAJTSQACMDIAAlNLABxJV2hYbk41R0JDZWp5bDFWMkw3MUQySVhid009AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
"successUrl": "/am/console",
"realm": "/"
}
2024-04-14 17:25:35,156 INFO [LoginSession] Obtaining session info
2024-04-14 17:25:35,156 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/realms/root/sessions?_action=getSessionInfo"
2024-04-14 17:25:35,370 INFO [http_cmd]: http status code OK
2024-04-14 17:25:35,370 DEBUG --- status code ---
2024-04-14 17:25:35,370 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:35,370 DEBUG --- http response ---
2024-04-14 17:25:35,371 DEBUG {
"username": "amadmin",
"universalId": "id=amadmin,ou=user,ou=am-config",
"realm": "/",
"latestAccessTime": "2024-04-14T17:25:35Z",
"maxIdleExpirationTime": "2024-04-14T17:28:35Z",
"maxSessionExpirationTime": "2024-04-14T17:28:34Z",
"properties": {
"AMCtxId": "c8fd95f4-08d1-4303-ae9b-1eea9f4d0cf7-496543479"
}
}
2024-04-14 17:25:35,375 INFO Obtain Oauth2 authz code via REST
2024-04-14 17:25:35,375 DEBUG Oauth2 authorize for access token
2024-04-14 17:25:35,375 INFO [http_cmd]: curl --request POST --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=6ecCleDCOLEuF_XTx04dREsY6HY.*AAJTSQACMDIAAlNLABxJV2hYbk41R0JDZWp5bDFWMkw3MUQySVhid009AAR0eXBlAANDVFMAAlMxAAIwMQ..*" --cookie "route=1713115536.111.49470.49623|f60edb382037eb2df1e800d563ad78a7" --data '{"decision": "Allow", "csrf": "6ecCleDCOLEuF_XTx04dREsY6HY.*AAJTSQACMDIAAlNLABxJV2hYbk41R0JDZWp5bDFWMkw3MUQySVhid009AAR0eXBlAANDVFMAAlMxAAIwMQ..*"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/authorize?client_id=smokeclient&scope=fr:idm:*&redirect_uri=https://fake.com&response_type=code&realm=/"
2024-04-14 17:25:35,601 INFO [http_cmd]: http status code OK
2024-04-14 17:25:35,601 DEBUG --- status code ---
2024-04-14 17:25:35,601 DEBUG http status code is 302 (expected 302)
2024-04-14 17:25:35,601 DEBUG --- http response ---
2024-04-14 17:25:35,601 DEBUG {'Date': 'Sun, 14 Apr 2024 17:25:35 GMT', 'Content-Length': '0', 'Connection': 'keep-alive', 'X-Frame-Options': 'SAMEORIGIN', 'X-Content-Type-Options': 'nosniff', 'Cache-Control': 'no-store', 'Location': 'https://fake.com?code=SiIJKd6WOXm2TC8_xzOMrFfB6Cc&iss=https%3A%2F%2Fxlou.iam.xlou-cdm.engineeringpit.com%2Fam%2Foauth2&client_id=smokeclient', 'Pragma': 'no-cache', 'Set-Cookie': 'OAUTH_REQUEST_ATTRIBUTES=DELETED; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/; Secure; HttpOnly; SameSite=none', 'Strict-Transport-Security': 'max-age=15724800; includeSubDomains'}
2024-04-14 17:25:35,601 DEBUG Oauth2 exchange authorize code for access token
2024-04-14 17:25:35,602 INFO [http_cmd]: curl -L --request POST --data '{"grant_type": "authorization_code", "redirect_uri": "https://fake.com", "client_id": "smokeclient", "code": "SiIJKd6WOXm2TC8_xzOMrFfB6Cc"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/access_token?realm=/"
2024-04-14 17:25:35,679 INFO [http_cmd]: http status code OK
2024-04-14 17:25:35,680 DEBUG --- status code ---
2024-04-14 17:25:35,680 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:35,680 DEBUG --- http response ---
2024-04-14 17:25:35,680 DEBUG {
"access_token": "OGUK3CCXrfuQEjf7EqmwPjWH04E",
"scope": "fr:idm:*",
"token_type": "Bearer",
"expires_in": 3599
}
2024-04-14 17:25:35,681 INFO Additional check to sync/queue to ensure sync with DS is ok (see LODESTAR-1245)
2024-04-14 17:25:35,682 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/sync/queue?_queryFilter=true&_pageSize=1"
2024-04-14 17:25:35,910 INFO [http_cmd]: http status code OK
2024-04-14 17:25:35,910 DEBUG --- status code ---
2024-04-14 17:25:35,910 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:35,910 DEBUG --- http response ---
2024-04-14 17:25:35,910 DEBUG {
"result": [],
"resultCount": 0,
"pagedResultsCookie": null,
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}
2024-04-14 17:25:35,912 INFO Try to create, query and delete a user
2024-04-14 17:25:35,912 INFO Create user idm_0lq70o8mkm in IDM
2024-04-14 17:25:35,912 INFO [http_cmd]: curl --header "Authorization: Bearer OGUK3CCXrfuQEjf7EqmwPjWH04E" --header "Content-Type: application/json" --insecure -L --request POST --data '{"userName": "idm_0lq70o8mkm", "givenName": "Peter", "sn": "Parker", "password": "T35tr0ck123", "telephoneNumber": "6669876987", "description": "IDM Test User", "roles": [], "mail": "idm_0lq70o8mkm@forgerock.com"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user?_action=create"
2024-04-14 17:25:36,186 INFO [http_cmd]: http status code OK
2024-04-14 17:25:36,186 DEBUG --- status code ---
2024-04-14 17:25:36,186 DEBUG http status code is 201 (expected [200, 201])
2024-04-14 17:25:36,186 DEBUG --- http response ---
2024-04-14 17:25:36,186 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-14 17:25:36,186 DEBUG ----- output -----
2024-04-14 17:25:36,186 DEBUG {
2024-04-14 17:25:36,186 DEBUG "_id": "d3b24c82-e1ef-47bf-b4ba-36ef0022070d",
2024-04-14 17:25:36,186 DEBUG "_rev": "d2dd67f6-c775-4646-8c43-6112461729ba-1027100",
2024-04-14 17:25:36,186 DEBUG "country": null,
2024-04-14 17:25:36,186 DEBUG "mail": "idm_0lq70o8mkm@forgerock.com",
2024-04-14 17:25:36,186 DEBUG "memberOfOrgIDs": [],
2024-04-14 17:25:36,186 DEBUG "postalCode": null,
2024-04-14 17:25:36,186 DEBUG "profileImage": null,
2024-04-14 17:25:36,186 DEBUG "expireAccount": null,
2024-04-14 17:25:36,186 DEBUG "activeDate": null,
2024-04-14 17:25:36,186 DEBUG
2024-04-14 17:25:36,187 DEBUG
2024-04-14 17:25:36,187 DEBUG
2024-04-14 17:25:36,187 DEBUG [...]
2024-04-14 17:25:36,187 DEBUG
2024-04-14 17:25:36,187 DEBUG
2024-04-14 17:25:36,187 DEBUG
2024-04-14 17:25:36,187 DEBUG "aliasList": [],
2024-04-14 17:25:36,187 DEBUG "kbaInfo": [],
2024-04-14 17:25:36,187 DEBUG "inactiveDate": null,
2024-04-14 17:25:36,187 DEBUG "sn": "Parker",
2024-04-14 17:25:36,187 DEBUG "preferences": null,
2024-04-14 17:25:36,187 DEBUG "userName": "idm_0lq70o8mkm",
2024-04-14 17:25:36,187 DEBUG "passwordLastChangedTime": "2024-04-14T17:25:36.040Z",
2024-04-14 17:25:36,187 DEBUG "effectiveRoles": [],
2024-04-14 17:25:36,187 DEBUG "activateAccount": null
2024-04-14 17:25:36,187 DEBUG }
2024-04-14 17:25:36,187 INFO Query IDM to verify UserID d3b24c82-e1ef-47bf-b4ba-36ef0022070d is visible
2024-04-14 17:25:36,187 INFO [http_cmd]: curl --insecure -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/d3b24c82-e1ef-47bf-b4ba-36ef0022070d"
2024-04-14 17:25:36,397 INFO [http_cmd]: http status code OK
2024-04-14 17:25:36,398 DEBUG --- status code ---
2024-04-14 17:25:36,398 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:36,398 DEBUG --- http response ---
2024-04-14 17:25:36,398 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-14 17:25:36,398 DEBUG ----- output -----
2024-04-14 17:25:36,398 DEBUG {
2024-04-14 17:25:36,398 DEBUG "_id": "d3b24c82-e1ef-47bf-b4ba-36ef0022070d",
2024-04-14 17:25:36,398 DEBUG "_rev": "d2dd67f6-c775-4646-8c43-6112461729ba-1027100",
2024-04-14 17:25:36,398 DEBUG "country": null,
2024-04-14 17:25:36,398 DEBUG "mail": "idm_0lq70o8mkm@forgerock.com",
2024-04-14 17:25:36,398 DEBUG "memberOfOrgIDs": [],
2024-04-14 17:25:36,398 DEBUG "postalCode": null,
2024-04-14 17:25:36,398 DEBUG "profileImage": null,
2024-04-14 17:25:36,398 DEBUG "expireAccount": null,
2024-04-14 17:25:36,398 DEBUG "activeDate": null,
2024-04-14 17:25:36,398 DEBUG
2024-04-14 17:25:36,398 DEBUG
2024-04-14 17:25:36,398 DEBUG
2024-04-14 17:25:36,398 DEBUG [...]
2024-04-14 17:25:36,398 DEBUG
2024-04-14 17:25:36,398 DEBUG
2024-04-14 17:25:36,398 DEBUG
2024-04-14 17:25:36,398 DEBUG "aliasList": [],
2024-04-14 17:25:36,398 DEBUG "kbaInfo": [],
2024-04-14 17:25:36,398 DEBUG "inactiveDate": null,
2024-04-14 17:25:36,398 DEBUG "sn": "Parker",
2024-04-14 17:25:36,398 DEBUG "preferences": null,
2024-04-14 17:25:36,398 DEBUG "userName": "idm_0lq70o8mkm",
2024-04-14 17:25:36,398 DEBUG "passwordLastChangedTime": "2024-04-14T17:25:36.040Z",
2024-04-14 17:25:36,398 DEBUG "effectiveRoles": [],
2024-04-14 17:25:36,398 DEBUG "activateAccount": null
2024-04-14 17:25:36,398 DEBUG }
2024-04-14 17:25:36,399 INFO Authenticate as user idm_0lq70o8mkm
2024-04-14 17:25:36,399 INFO [http_cmd]: curl --insecure -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/authentication?_action=login"
2024-04-14 17:25:36,603 INFO [http_cmd]: http status code OK
2024-04-14 17:25:36,603 DEBUG --- status code ---
2024-04-14 17:25:36,603 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:36,603 DEBUG --- http response ---
2024-04-14 17:25:36,603 DEBUG {
"_id": "login",
"authorization": {
"id": "anonymous",
"roles": [
"internal/role/openidm-reg"
],
"component": "internal/user"
},
"authenticationId": "anonymous"
}
2024-04-14 17:25:36,604 INFO [http_cmd]: curl --insecure -L --request DELETE "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/d3b24c82-e1ef-47bf-b4ba-36ef0022070d"
2024-04-14 17:25:36,839 INFO [http_cmd]: http status code OK
2024-04-14 17:25:36,839 DEBUG --- status code ---
2024-04-14 17:25:36,839 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:36,839 DEBUG --- http response ---
2024-04-14 17:25:36,840 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-14 17:25:36,840 DEBUG ----- output -----
2024-04-14 17:25:36,840 DEBUG {
2024-04-14 17:25:36,840 DEBUG "_id": "d3b24c82-e1ef-47bf-b4ba-36ef0022070d",
2024-04-14 17:25:36,840 DEBUG "_rev": "d2dd67f6-c775-4646-8c43-6112461729ba-1027100",
2024-04-14 17:25:36,840 DEBUG "country": null,
2024-04-14 17:25:36,840 DEBUG "mail": "idm_0lq70o8mkm@forgerock.com",
2024-04-14 17:25:36,840 DEBUG "memberOfOrgIDs": [],
2024-04-14 17:25:36,840 DEBUG "postalCode": null,
2024-04-14 17:25:36,840 DEBUG "profileImage": null,
2024-04-14 17:25:36,840 DEBUG "expireAccount": null,
2024-04-14 17:25:36,840 DEBUG "activeDate": null,
2024-04-14 17:25:36,840 DEBUG
2024-04-14 17:25:36,840 DEBUG
2024-04-14 17:25:36,840 DEBUG
2024-04-14 17:25:36,840 DEBUG [...]
2024-04-14 17:25:36,840 DEBUG
2024-04-14 17:25:36,840 DEBUG
2024-04-14 17:25:36,840 DEBUG
2024-04-14 17:25:36,841 DEBUG "aliasList": [],
2024-04-14 17:25:36,841 DEBUG "kbaInfo": [],
2024-04-14 17:25:36,841 DEBUG "inactiveDate": null,
2024-04-14 17:25:36,841 DEBUG "sn": "Parker",
2024-04-14 17:25:36,841 DEBUG "preferences": null,
2024-04-14 17:25:36,841 DEBUG "userName": "idm_0lq70o8mkm",
2024-04-14 17:25:36,841 DEBUG "passwordLastChangedTime": "2024-04-14T17:25:36.040Z",
2024-04-14 17:25:36,841 DEBUG "effectiveRoles": [],
2024-04-14 17:25:36,841 DEBUG "activateAccount": null
2024-04-14 17:25:36,841 DEBUG }
2024-04-14 17:25:36,844 INFO
2024-04-14 17:25:36,845 INFO -------------------- Running END-USER-UI livecheck --------------------
2024-04-14 17:25:36,845 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/enduser
2024-04-14 17:25:36,846 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/enduser"
2024-04-14 17:25:37,125 INFO [http_cmd]: http status code OK
2024-04-14 17:25:37,125 DEBUG --- status code ---
2024-04-14 17:25:37,125 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:37,125 DEBUG --- http response ---
2024-04-14 17:25:37,125 DEBUG Identity Management
[]
2024-04-14 17:25:37,126 INFO
2024-04-14 17:25:37,127 INFO --------------------- Running LOGIN-UI livecheck ---------------------
2024-04-14 17:25:37,127 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI
2024-04-14 17:25:37,127 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI"
2024-04-14 17:25:37,413 INFO [http_cmd]: http status code OK
2024-04-14 17:25:37,414 DEBUG --- status code ---
2024-04-14 17:25:37,414 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:37,414 DEBUG --- http response ---
2024-04-14 17:25:37,414 DEBUG Login
[]
2024-04-14 17:25:37,416 INFO
2024-04-14 17:25:37,416 INFO --------------------- Running ADMIN-UI livecheck ---------------------
2024-04-14 17:25:37,416 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/platform
2024-04-14 17:25:37,417 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/platform"
2024-04-14 17:25:37,701 INFO [http_cmd]: http status code OK
2024-04-14 17:25:37,702 DEBUG --- status code ---
2024-04-14 17:25:37,702 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:37,702 DEBUG --- http response ---
2024-04-14 17:25:37,702 DEBUG Platform Admin
[]
2024-04-14 17:25:37,705 INFO LIVECHECK SUCCEEDED
2024-04-14 17:25:37,705 INFO
2024-04-14 17:25:37,705 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-14 17:25:37,705 INFO
2024-04-14 17:25:37,705 INFO --------------------- Get DS-CTS software version ---------------------
2024-04-14 17:25:37,706 INFO
2024-04-14 17:25:37,706 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- find /opt/opendj -name opendj-core.jar
2024-04-14 17:25:37,706 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-14 17:25:38,538 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:38,538 DEBUG --- stdout ---
2024-04-14 17:25:38,539 DEBUG /opt/opendj/lib/opendj-core.jar
2024-04-14 17:25:38,539 DEBUG --- stderr ---
2024-04-14 17:25:38,539 DEBUG
2024-04-14 17:25:38,539 INFO
2024-04-14 17:25:38,539 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-cts-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-cts_info/opendj-core.jar --container ds
2024-04-14 17:25:38,539 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:39,767 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:39,768 DEBUG --- stdout ---
2024-04-14 17:25:39,768 DEBUG tar: Removing leading `/' from member names
2024-04-14 17:25:39,768 DEBUG --- stderr ---
2024-04-14 17:25:39,768 DEBUG
2024-04-14 17:25:39,778 INFO
2024-04-14 17:25:39,778 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-14 17:25:39,778 INFO
2024-04-14 17:25:39,778 INFO ------------------- Get DS-IDREPO software version -------------------
2024-04-14 17:25:39,779 INFO
2024-04-14 17:25:39,779 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- find /opt/opendj -name opendj-core.jar
2024-04-14 17:25:39,779 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-14 17:25:40,595 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:40,595 DEBUG --- stdout ---
2024-04-14 17:25:40,595 DEBUG /opt/opendj/lib/opendj-core.jar
2024-04-14 17:25:40,595 DEBUG --- stderr ---
2024-04-14 17:25:40,595 DEBUG
2024-04-14 17:25:40,596 INFO
2024-04-14 17:25:40,596 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-idrepo-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-idrepo_info/opendj-core.jar --container ds
2024-04-14 17:25:40,596 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:41,828 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:41,828 DEBUG --- stdout ---
2024-04-14 17:25:41,828 DEBUG tar: Removing leading `/' from member names
2024-04-14 17:25:41,828 DEBUG --- stderr ---
2024-04-14 17:25:41,828 DEBUG
2024-04-14 17:25:41,836 INFO
2024-04-14 17:25:41,836 INFO ****************************** Initializing component pods for AM ******************************
2024-04-14 17:25:41,836 INFO
2024-04-14 17:25:41,836 INFO ----------------------- Get AM software version -----------------------
2024-04-14 17:25:41,837 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version
2024-04-14 17:25:41,837 INFO [LoginSession] Get "token_id" token for user "amadmin"
2024-04-14 17:25:41,837 DEBUG [LoginSession] re-using "token_id" token that remains valid for 2m 42s
2024-04-14 17:25:41,837 INFO [http_cmd]: curl -L --request GET --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=6ecCleDCOLEuF_XTx04dREsY6HY.*AAJTSQACMDIAAlNLABxJV2hYbk41R0JDZWp5bDFWMkw3MUQySVhid009AAR0eXBlAANDVFMAAlMxAAIwMQ..*" --cookie "route=1713115536.111.49470.49623|f60edb382037eb2df1e800d563ad78a7" "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version"
2024-04-14 17:25:42,052 INFO [http_cmd]: http status code OK
2024-04-14 17:25:42,052 DEBUG --- status code ---
2024-04-14 17:25:42,052 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:42,052 DEBUG --- http response ---
2024-04-14 17:25:42,053 DEBUG {
"_id": "version",
"_rev": "509589602",
"version": "7.6.0-SNAPSHOT",
"fullVersion": "ForgeRock Access Management 7.6.0-SNAPSHOT Build bbf6791c961b39ef101ebad36686ee2c3cde94bb (2024-April-12 18:28)",
"revision": "bbf6791c961b39ef101ebad36686ee2c3cde94bb",
"date": "2024-April-12 18:28"
}
2024-04-14 17:25:42,056 INFO
2024-04-14 17:25:42,056 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-14 17:25:42,056 INFO
2024-04-14 17:25:42,056 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-14 17:25:42,056 INFO
2024-04-14 17:25:42,056 INFO ---------------------- Get IDM software version ----------------------
2024-04-14 17:25:42,056 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version
2024-04-14 17:25:42,056 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version"
2024-04-14 17:25:42,268 INFO [http_cmd]: http status code OK
2024-04-14 17:25:42,268 DEBUG --- status code ---
2024-04-14 17:25:42,268 DEBUG http status code is 200 (expected 200)
2024-04-14 17:25:42,268 DEBUG --- http response ---
2024-04-14 17:25:42,268 DEBUG {
"_id": "version",
"productVersion": "7.6.0-SNAPSHOT",
"productBuildDate": "20240411202127",
"productRevision": "8554e0d1c5"
}
2024-04-14 17:25:42,270 INFO
2024-04-14 17:25:42,270 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-14 17:25:42,270 INFO
2024-04-14 17:25:42,270 INFO ------------------ Get END-USER-UI software version ------------------
2024-04-14 17:25:42,271 INFO
2024-04-14 17:25:42,271 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec end-user-ui-58878d7668-zg4h2 --container end-user-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-14 17:25:42,271 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-14 17:25:43,037 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:43,038 DEBUG --- stdout ---
2024-04-14 17:25:43,039 DEBUG /usr/share/nginx/html/js/chunk-vendors.0380a414.js
2024-04-14 17:25:43,039 DEBUG --- stderr ---
2024-04-14 17:25:43,040 DEBUG
2024-04-14 17:25:43,041 INFO
2024-04-14 17:25:43,041 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp end-user-ui-58878d7668-zg4h2:/usr/share/nginx/html/js/chunk-vendors.0380a414.js /tmp/end-user-ui_info/chunk-vendors.0380a414.js --container end-user-ui
2024-04-14 17:25:43,041 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:44,216 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:44,216 DEBUG --- stdout ---
2024-04-14 17:25:44,216 DEBUG tar: Removing leading `/' from member names
2024-04-14 17:25:44,216 DEBUG --- stderr ---
2024-04-14 17:25:44,216 DEBUG
2024-04-14 17:25:44,256 INFO
2024-04-14 17:25:44,256 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-14 17:25:44,256 INFO
2024-04-14 17:25:44,256 INFO -------------------- Get LOGIN-UI software version --------------------
2024-04-14 17:25:44,257 INFO
2024-04-14 17:25:44,257 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec login-ui-5fdf49fc66-2wnx7 --container login-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-14 17:25:44,257 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-14 17:25:45,039 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:45,039 DEBUG --- stdout ---
2024-04-14 17:25:45,039 DEBUG /usr/share/nginx/html/js/chunk-vendors.f2fa5365.js
2024-04-14 17:25:45,039 DEBUG --- stderr ---
2024-04-14 17:25:45,039 DEBUG
2024-04-14 17:25:45,039 INFO
2024-04-14 17:25:45,039 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp login-ui-5fdf49fc66-2wnx7:/usr/share/nginx/html/js/chunk-vendors.f2fa5365.js /tmp/login-ui_info/chunk-vendors.f2fa5365.js --container login-ui
2024-04-14 17:25:45,039 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:46,169 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:46,170 DEBUG --- stdout ---
2024-04-14 17:25:46,170 DEBUG tar: Removing leading `/' from member names
2024-04-14 17:25:46,170 DEBUG --- stderr ---
2024-04-14 17:25:46,170 DEBUG
2024-04-14 17:25:46,190 INFO
2024-04-14 17:25:46,191 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-14 17:25:46,191 INFO
2024-04-14 17:25:46,191 INFO -------------------- Get ADMIN-UI software version --------------------
2024-04-14 17:25:46,191 INFO
2024-04-14 17:25:46,191 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec admin-ui-c4c7bf584-mbzlq --container admin-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-14 17:25:46,191 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-14 17:25:46,945 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:46,945 DEBUG --- stdout ---
2024-04-14 17:25:46,945 DEBUG /usr/share/nginx/html/js/chunk-vendors.b3d879ac.js
2024-04-14 17:25:46,945 DEBUG --- stderr ---
2024-04-14 17:25:46,945 DEBUG
2024-04-14 17:25:46,945 INFO
2024-04-14 17:25:46,946 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp admin-ui-c4c7bf584-mbzlq:/usr/share/nginx/html/js/chunk-vendors.b3d879ac.js /tmp/admin-ui_info/chunk-vendors.b3d879ac.js --container admin-ui
2024-04-14 17:25:46,946 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:48,117 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:48,119 DEBUG --- stdout ---
2024-04-14 17:25:48,119 DEBUG tar: Removing leading `/' from member names
2024-04-14 17:25:48,119 DEBUG --- stderr ---
2024-04-14 17:25:48,119 DEBUG
2024-04-14 17:25:48,163 INFO
2024-04-14 17:25:48,163 INFO
2024-04-14 17:25:48,164 INFO ====================================================================================================
2024-04-14 17:25:48,164 INFO ================ Admin password for DS-CTS is: b2Et9XuIRcBwntYUi7J7f7OASS9YbQqx ================
2024-04-14 17:25:48,164 INFO ====================================================================================================
2024-04-14 17:25:48,164 INFO
2024-04-14 17:25:48,164 INFO ====================================================================================================
2024-04-14 17:25:48,164 INFO ============== Admin password for DS-IDREPO is: b2Et9XuIRcBwntYUi7J7f7OASS9YbQqx ==============
2024-04-14 17:25:48,164 INFO ====================================================================================================
2024-04-14 17:25:48,164 INFO
2024-04-14 17:25:48,164 INFO ====================================================================================================
2024-04-14 17:25:48,164 INFO ====================== Admin password for AM is: YNwb14OS5H3c83lWKq9jblnT ======================
2024-04-14 17:25:48,164 INFO ====================================================================================================
2024-04-14 17:25:48,164 INFO
2024-04-14 17:25:48,164 INFO ====================================================================================================
2024-04-14 17:25:48,164 INFO ===================== Admin password for IDM is: 1UXqmgfePB4NHghbGcuasRXC =====================
2024-04-14 17:25:48,164 INFO ====================================================================================================
2024-04-14 17:25:48,164 INFO
2024-04-14 17:25:48,164 INFO *************************************** Dumping pod list ***************************************
2024-04-14 17:25:48,164 INFO Dumping pod list to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/_pod-list.txt
2024-04-14 17:25:48,686 INFO Dumping platform-config configmap to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/_platform-config_configmap.txt
2024-04-14 17:25:49,052 INFO
2024-04-14 17:25:49,053 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-14 17:25:49,053 INFO
2024-04-14 17:25:49,053 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:49,053 INFO
2024-04-14 17:25:49,053 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:49,053 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:49,493 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:49,493 DEBUG --- stdout ---
2024-04-14 17:25:49,493 DEBUG 3
2024-04-14 17:25:49,494 DEBUG --- stderr ---
2024-04-14 17:25:49,494 DEBUG
2024-04-14 17:25:49,494 INFO
2024-04-14 17:25:49,494 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:49,494 INFO
2024-04-14 17:25:49,494 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:49,494 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:50,013 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:50,013 DEBUG --- stdout ---
2024-04-14 17:25:50,013 DEBUG ds-cts-0 ds-cts-1 ds-cts-2
2024-04-14 17:25:50,013 DEBUG --- stderr ---
2024-04-14 17:25:50,013 DEBUG
2024-04-14 17:25:50,014 INFO
2024-04-14 17:25:50,014 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-14 17:25:50,014 INFO
2024-04-14 17:25:50,014 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:50,014 INFO
2024-04-14 17:25:50,014 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:50,014 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:50,405 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:50,405 DEBUG --- stdout ---
2024-04-14 17:25:50,405 DEBUG 3
2024-04-14 17:25:50,405 DEBUG --- stderr ---
2024-04-14 17:25:50,406 DEBUG
2024-04-14 17:25:50,406 INFO
2024-04-14 17:25:50,406 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:50,406 INFO
2024-04-14 17:25:50,406 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:50,406 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:50,864 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:50,864 DEBUG --- stdout ---
2024-04-14 17:25:50,864 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2
2024-04-14 17:25:50,864 DEBUG --- stderr ---
2024-04-14 17:25:50,864 DEBUG
2024-04-14 17:25:50,864 INFO
2024-04-14 17:25:50,864 INFO ****************************** Initializing component pods for AM ******************************
2024-04-14 17:25:50,864 INFO
2024-04-14 17:25:50,864 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:50,864 INFO
2024-04-14 17:25:50,864 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:50,865 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:51,252 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:51,252 DEBUG --- stdout ---
2024-04-14 17:25:51,252 DEBUG 3
2024-04-14 17:25:51,252 DEBUG --- stderr ---
2024-04-14 17:25:51,252 DEBUG
2024-04-14 17:25:51,252 INFO
2024-04-14 17:25:51,252 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:51,253 INFO
2024-04-14 17:25:51,253 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:51,253 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:51,709 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:51,710 DEBUG --- stdout ---
2024-04-14 17:25:51,710 DEBUG am-7d597cf7d6-8ctft am-7d597cf7d6-8vr8f am-7d597cf7d6-9wz9h
2024-04-14 17:25:51,710 DEBUG --- stderr ---
2024-04-14 17:25:51,710 DEBUG
2024-04-14 17:25:51,710 INFO
2024-04-14 17:25:51,710 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-14 17:25:51,710 INFO
2024-04-14 17:25:51,710 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:51,710 INFO
2024-04-14 17:25:51,710 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:51,710 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:52,090 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:52,091 DEBUG --- stdout ---
2024-04-14 17:25:52,091 DEBUG amster-qv96p
2024-04-14 17:25:52,091 DEBUG --- stderr ---
2024-04-14 17:25:52,091 DEBUG
2024-04-14 17:25:52,091 INFO
2024-04-14 17:25:52,091 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-14 17:25:52,091 INFO
2024-04-14 17:25:52,091 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:52,091 INFO
2024-04-14 17:25:52,091 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:52,091 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:52,483 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:52,483 DEBUG --- stdout ---
2024-04-14 17:25:52,483 DEBUG 2
2024-04-14 17:25:52,483 DEBUG --- stderr ---
2024-04-14 17:25:52,483 DEBUG
2024-04-14 17:25:52,483 INFO
2024-04-14 17:25:52,484 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:52,484 INFO
2024-04-14 17:25:52,484 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:52,484 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:52,875 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:52,875 DEBUG --- stdout ---
2024-04-14 17:25:52,875 DEBUG idm-b85f9d978-j5pbm idm-b85f9d978-rl76f
2024-04-14 17:25:52,876 DEBUG --- stderr ---
2024-04-14 17:25:52,876 DEBUG
2024-04-14 17:25:52,876 INFO
2024-04-14 17:25:52,876 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-14 17:25:52,876 INFO
2024-04-14 17:25:52,876 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:52,876 INFO
2024-04-14 17:25:52,876 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:52,876 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:53,174 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:53,175 DEBUG --- stdout ---
2024-04-14 17:25:53,175 DEBUG 1
2024-04-14 17:25:53,175 DEBUG --- stderr ---
2024-04-14 17:25:53,175 DEBUG
2024-04-14 17:25:53,176 INFO
2024-04-14 17:25:53,176 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:53,176 INFO
2024-04-14 17:25:53,176 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:53,176 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:53,481 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:53,481 DEBUG --- stdout ---
2024-04-14 17:25:53,481 DEBUG end-user-ui-58878d7668-zg4h2
2024-04-14 17:25:53,481 DEBUG --- stderr ---
2024-04-14 17:25:53,481 DEBUG
2024-04-14 17:25:53,482 INFO
2024-04-14 17:25:53,482 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-14 17:25:53,482 INFO
2024-04-14 17:25:53,482 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:53,482 INFO
2024-04-14 17:25:53,482 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:53,482 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:53,819 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:53,819 DEBUG --- stdout ---
2024-04-14 17:25:53,819 DEBUG 1
2024-04-14 17:25:53,820 DEBUG --- stderr ---
2024-04-14 17:25:53,820 DEBUG
2024-04-14 17:25:53,820 INFO
2024-04-14 17:25:53,820 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:53,820 INFO
2024-04-14 17:25:53,820 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:53,820 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:54,154 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:54,154 DEBUG --- stdout ---
2024-04-14 17:25:54,154 DEBUG login-ui-5fdf49fc66-2wnx7
2024-04-14 17:25:54,154 DEBUG --- stderr ---
2024-04-14 17:25:54,154 DEBUG
2024-04-14 17:25:54,155 INFO
2024-04-14 17:25:54,155 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-14 17:25:54,155 INFO
2024-04-14 17:25:54,155 INFO --------------------- Get expected number of pods ---------------------
2024-04-14 17:25:54,155 INFO
2024-04-14 17:25:54,155 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-14 17:25:54,155 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-14 17:25:54,482 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:54,482 DEBUG --- stdout ---
2024-04-14 17:25:54,482 DEBUG 1
2024-04-14 17:25:54,482 DEBUG --- stderr ---
2024-04-14 17:25:54,482 DEBUG
2024-04-14 17:25:54,482 INFO
2024-04-14 17:25:54,482 INFO ---------------------------- Get pod list ----------------------------
2024-04-14 17:25:54,483 INFO
2024-04-14 17:25:54,483 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name}
2024-04-14 17:25:54,483 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-14 17:25:54,809 INFO [loop_until]: OK (rc = 0)
2024-04-14 17:25:54,810 DEBUG --- stdout ---
2024-04-14 17:25:54,810 DEBUG admin-ui-c4c7bf584-mbzlq
2024-04-14 17:25:54,810 DEBUG --- stderr ---
2024-04-14 17:25:54,810 DEBUG
2024-04-14 17:25:54,810 INFO
2024-04-14 17:25:54,810 INFO *********************************** Dumping components logs ***********************************
2024-04-14 17:25:54,811 INFO
2024-04-14 17:25:54,811 INFO ----------------------- Dumping logs for DS-CTS -----------------------
2024-04-14 17:25:56,295 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-cts-0.txt
2024-04-14 17:25:56,296 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-cts-0.txt
2024-04-14 17:25:56,304 INFO Check pod logs for errors
2024-04-14 17:25:57,769 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-cts-1.txt
2024-04-14 17:25:57,770 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-cts-1.txt
2024-04-14 17:25:57,776 INFO Check pod logs for errors
2024-04-14 17:25:59,290 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-cts-2.txt
2024-04-14 17:25:59,291 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-cts-2.txt
2024-04-14 17:25:59,297 INFO Check pod logs for errors
2024-04-14 17:25:59,302 INFO
2024-04-14 17:25:59,303 INFO --------------------- Dumping logs for DS-IDREPO ---------------------
2024-04-14 17:26:00,724 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-idrepo-0.txt
2024-04-14 17:26:00,725 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-idrepo-0.txt
2024-04-14 17:26:00,730 INFO Check pod logs for errors
2024-04-14 17:26:02,168 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-idrepo-1.txt
2024-04-14 17:26:02,169 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-idrepo-1.txt
2024-04-14 17:26:02,173 INFO Check pod logs for errors
2024-04-14 17:26:03,566 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-idrepo-2.txt
2024-04-14 17:26:03,566 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/ds-idrepo-2.txt
2024-04-14 17:26:03,571 INFO Check pod logs for errors
2024-04-14 17:26:03,576 INFO
2024-04-14 17:26:03,577 INFO ------------------------- Dumping logs for AM -------------------------
2024-04-14 17:26:04,913 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/am-7d597cf7d6-8ctft.txt
2024-04-14 17:26:04,914 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/am-7d597cf7d6-8ctft.txt
2024-04-14 17:26:04,915 INFO Check pod logs for errors
2024-04-14 17:26:06,569 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/am-7d597cf7d6-8vr8f.txt
2024-04-14 17:26:06,570 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/am-7d597cf7d6-8vr8f.txt
2024-04-14 17:26:06,587 INFO Check pod logs for errors
2024-04-14 17:26:07,949 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/am-7d597cf7d6-9wz9h.txt
2024-04-14 17:26:07,950 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/am-7d597cf7d6-9wz9h.txt
2024-04-14 17:26:07,951 INFO Check pod logs for errors
2024-04-14 17:26:07,952 INFO
2024-04-14 17:26:07,952 INFO ----------------------- Dumping logs for AMSTER -----------------------
2024-04-14 17:26:09,066 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/amster-qv96p.txt
2024-04-14 17:26:09,067 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/amster-qv96p.txt
2024-04-14 17:26:09,067 INFO Check pod logs for errors
2024-04-14 17:26:09,067 INFO
2024-04-14 17:26:09,067 INFO ------------------------ Dumping logs for IDM ------------------------
2024-04-14 17:26:10,633 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/idm-b85f9d978-j5pbm.txt
2024-04-14 17:26:10,633 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/idm-b85f9d978-j5pbm.txt
2024-04-14 17:26:10,645 INFO Check pod logs for errors
2024-04-14 17:26:12,205 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/idm-b85f9d978-rl76f.txt
2024-04-14 17:26:12,206 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/idm-b85f9d978-rl76f.txt
2024-04-14 17:26:12,217 INFO Check pod logs for errors
2024-04-14 17:26:12,223 INFO
2024-04-14 17:26:12,224 INFO -------------------- Dumping logs for END-USER-UI --------------------
2024-04-14 17:26:13,464 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/end-user-ui-58878d7668-zg4h2.txt
2024-04-14 17:26:13,465 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/end-user-ui-58878d7668-zg4h2.txt
2024-04-14 17:26:13,470 INFO Check pod logs for errors
2024-04-14 17:26:13,475 INFO
2024-04-14 17:26:13,475 INFO ---------------------- Dumping logs for LOGIN-UI ----------------------
2024-04-14 17:26:14,744 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/login-ui-5fdf49fc66-2wnx7.txt
2024-04-14 17:26:14,744 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/login-ui-5fdf49fc66-2wnx7.txt
2024-04-14 17:26:14,748 INFO Check pod logs for errors
2024-04-14 17:26:14,753 INFO
2024-04-14 17:26:14,753 INFO ---------------------- Dumping logs for ADMIN-UI ----------------------
2024-04-14 17:26:15,935 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/admin-ui-c4c7bf584-mbzlq.txt
2024-04-14 17:26:15,938 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240414_172548-after-deployment/admin-ui-c4c7bf584-mbzlq.txt
2024-04-14 17:26:15,943 INFO Check pod logs for errors
[2024-04-14 17:26:15] - INFO: Deployment ready
________________________________________________________________________________
[2024-04-14 17:26:15] Discover_all_components post : Post method
________________________________________________________________________________
Setting result to PASS
Task has been successfully stopped