--Task--
name: Discover_all_components
enabled: True
class_name: DiscoverComponentsTask
source_name: controller
source_namespace: >default<
target_name: controller
target_namespace: >default<
start: 0.0
stop: None
timeout: no timeout
loop: False
interval: None
dependencies: []
wait_for: []
preceding_task: None
options: {}
group_name: None
Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock
________________________________________________________________________________
[2024-04-19 05:42:38] Discover_all_components pre : Initialising task parameters
________________________________________________________________________________
task will be executed on controller (localhost)
________________________________________________________________________________
[2024-04-19 05:42:38] Discover_all_components step1 : Discover components
________________________________________________________________________________
2024-04-19 05:42:39,570 INFO Deployment was skipped.
2024-04-19 05:42:45,812 INFO
2024-04-19 05:42:45,812 INFO ************************************* Creating deployment *************************************
2024-04-19 05:42:45,812 INFO Creating normal (forgeops) type deployment for deployment: stack
2024-04-19 05:42:45,974 INFO
2024-04-19 05:42:45,975 INFO ------- Custom component configuration present. Loading values -------
2024-04-19 05:42:45,975 INFO
2024-04-19 05:42:45,977 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds cts
2024-04-19 05:42:46,889 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:42:46,891 DEBUG --- stdout ---
2024-04-19 05:42:46,892 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts
2024-04-19 05:42:46,892 DEBUG --- stderr ---
2024-04-19 05:42:46,893 DEBUG
2024-04-19 05:42:46,893 INFO
2024-04-19 05:42:46,895 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-db97906b8eff882e42d79d5b86aca43562470e8d
2024-04-19 05:42:46,896 INFO
2024-04-19 05:42:46,898 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds idrepo
2024-04-19 05:42:48,055 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:42:48,057 DEBUG --- stdout ---
2024-04-19 05:42:48,059 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo
2024-04-19 05:42:48,059 DEBUG --- stderr ---
2024-04-19 05:42:48,060 DEBUG
2024-04-19 05:42:48,060 INFO
2024-04-19 05:42:48,064 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-db97906b8eff882e42d79d5b86aca43562470e8d
2024-04-19 05:42:48,065 INFO
2024-04-19 05:42:48,067 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker am
2024-04-19 05:42:48,292 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:42:48,292 DEBUG --- stdout ---
2024-04-19 05:42:48,292 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am
2024-04-19 05:42:48,292 DEBUG --- stderr ---
2024-04-19 05:42:48,292 DEBUG
2024-04-19 05:42:48,292 INFO
2024-04-19 05:42:48,293 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am/Dockerfile: gcr.io/forgerock-io/am-cdk/pit1:7.6.0-b3f32bcf2825f595a381aae2f4ddab14a92da8c1
2024-04-19 05:42:48,293 INFO
2024-04-19 05:42:48,293 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker amster
2024-04-19 05:42:48,448 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:42:48,448 DEBUG --- stdout ---
2024-04-19 05:42:48,451 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster
2024-04-19 05:42:48,452 DEBUG --- stderr ---
2024-04-19 05:42:48,455 DEBUG
2024-04-19 05:42:48,455 INFO
2024-04-19 05:42:48,459 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster/Dockerfile: gcr.io/forgerock-io/amster/pit1:7.6.0-b3f32bcf2825f595a381aae2f4ddab14a92da8c1
2024-04-19 05:42:48,461 INFO
2024-04-19 05:42:48,462 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker idm
2024-04-19 05:42:48,798 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:42:48,798 DEBUG --- stdout ---
2024-04-19 05:42:48,798 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm
2024-04-19 05:42:48,798 DEBUG --- stderr ---
2024-04-19 05:42:48,798 DEBUG
2024-04-19 05:42:48,798 INFO
2024-04-19 05:42:48,799 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm/Dockerfile: gcr.io/forgerock-io/idm-cdk/pit1:7.6.0-8554e0d1c5044f4f6c6504cd366196848874e695
2024-04-19 05:42:48,799 INFO
2024-04-19 05:42:48,799 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base end-user-ui
2024-04-19 05:42:48,970 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:42:48,970 DEBUG --- stdout ---
2024-04-19 05:42:48,970 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui
2024-04-19 05:42:48,970 DEBUG --- stderr ---
2024-04-19 05:42:48,970 DEBUG
2024-04-19 05:42:48,970 INFO
2024-04-19 05:42:48,973 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui/kustomization.yaml: gcr.io/forgerock-io/platform-enduser-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-19 05:42:48,973 INFO
2024-04-19 05:42:48,973 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base login-ui
2024-04-19 05:42:49,154 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:42:49,155 DEBUG --- stdout ---
2024-04-19 05:42:49,155 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui
2024-04-19 05:42:49,155 DEBUG --- stderr ---
2024-04-19 05:42:49,155 DEBUG
2024-04-19 05:42:49,155 INFO
2024-04-19 05:42:49,163 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui/kustomization.yaml: gcr.io/forgerock-io/platform-login-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-19 05:42:49,163 INFO
2024-04-19 05:42:49,164 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base admin-ui
2024-04-19 05:42:49,316 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:42:49,317 DEBUG --- stdout ---
2024-04-19 05:42:49,317 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui
2024-04-19 05:42:49,317 DEBUG --- stderr ---
2024-04-19 05:42:49,317 DEBUG
2024-04-19 05:42:49,317 INFO
2024-04-19 05:42:49,324 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui/kustomization.yaml: gcr.io/forgerock-io/platform-admin-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-19 05:42:49,324 INFO Loading external components from config.yaml
2024-04-19 05:42:49,605 INFO No external components defined.
2024-04-19 05:42:49,874 INFO Setting context for components to: gke_engineeringpit_us-east1-d_xlou-cdm
2024-04-19 05:42:49,875 INFO
2024-04-19 05:42:49,875 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-19 05:42:49,875 INFO
2024-04-19 05:42:49,875 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:42:49,876 INFO
2024-04-19 05:42:49,876 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:42:49,876 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:42:50,405 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:50,406 DEBUG --- stdout ---
2024-04-19 05:42:50,406 DEBUG 3
2024-04-19 05:42:50,406 DEBUG --- stderr ---
2024-04-19 05:42:50,406 DEBUG
2024-04-19 05:42:50,406 INFO
2024-04-19 05:42:50,406 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:42:50,406 INFO
2024-04-19 05:42:50,406 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-19 05:42:50,407 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:42:51,028 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-19 05:42:51,031 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:51,032 DEBUG --- stdout ---
2024-04-19 05:42:51,032 DEBUG ds-cts-0 ds-cts-1 ds-cts-2
2024-04-19 05:42:51,032 DEBUG --- stderr ---
2024-04-19 05:42:51,033 DEBUG
2024-04-19 05:42:51,033 INFO
2024-04-19 05:42:51,033 INFO -------------------- Check pod ds-cts-0 is running --------------------
2024-04-19 05:42:51,033 INFO
2024-04-19 05:42:51,034 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:42:51,034 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:42:51,585 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:42:51,586 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:51,586 DEBUG --- stdout ---
2024-04-19 05:42:51,586 DEBUG Running
2024-04-19 05:42:51,586 DEBUG --- stderr ---
2024-04-19 05:42:51,587 DEBUG
2024-04-19 05:42:51,588 INFO
2024-04-19 05:42:51,588 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:42:51,588 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:42:52,206 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:42:52,207 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:52,207 DEBUG --- stdout ---
2024-04-19 05:42:52,207 DEBUG true
2024-04-19 05:42:52,207 DEBUG --- stderr ---
2024-04-19 05:42:52,208 DEBUG
2024-04-19 05:42:52,208 INFO
2024-04-19 05:42:52,209 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.startTime}
2024-04-19 05:42:52,209 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:42:52,736 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:52,736 DEBUG --- stdout ---
2024-04-19 05:42:52,736 DEBUG 2024-04-17T18:33:21Z
2024-04-19 05:42:52,736 DEBUG --- stderr ---
2024-04-19 05:42:52,736 DEBUG
2024-04-19 05:42:52,737 INFO
2024-04-19 05:42:52,737 INFO ------------- Check pod ds-cts-0 filesystem is accessible -------------
2024-04-19 05:42:53,711 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:42:53,712 INFO
2024-04-19 05:42:53,712 INFO ------------------ Check pod ds-cts-0 restart count ------------------
2024-04-19 05:42:53,712 INFO
2024-04-19 05:42:53,712 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:42:53,712 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:42:54,202 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:54,203 DEBUG --- stdout ---
2024-04-19 05:42:54,203 DEBUG 0
2024-04-19 05:42:54,203 DEBUG --- stderr ---
2024-04-19 05:42:54,203 DEBUG
2024-04-19 05:42:54,203 INFO Pod ds-cts-0 has been restarted 0 times.
2024-04-19 05:42:54,203 INFO
2024-04-19 05:42:54,203 INFO -------------------- Check pod ds-cts-1 is running --------------------
2024-04-19 05:42:54,203 INFO
2024-04-19 05:42:54,204 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:42:54,204 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:42:54,700 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:42:54,701 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:54,701 DEBUG --- stdout ---
2024-04-19 05:42:54,701 DEBUG Running
2024-04-19 05:42:54,701 DEBUG --- stderr ---
2024-04-19 05:42:54,701 DEBUG
2024-04-19 05:42:54,701 INFO
2024-04-19 05:42:54,701 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:42:54,701 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:42:55,214 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:42:55,214 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:55,214 DEBUG --- stdout ---
2024-04-19 05:42:55,214 DEBUG true
2024-04-19 05:42:55,214 DEBUG --- stderr ---
2024-04-19 05:42:55,214 DEBUG
2024-04-19 05:42:55,215 INFO
2024-04-19 05:42:55,215 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.startTime}
2024-04-19 05:42:55,215 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:42:55,731 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:55,731 DEBUG --- stdout ---
2024-04-19 05:42:55,731 DEBUG 2024-04-17T18:33:00Z
2024-04-19 05:42:55,731 DEBUG --- stderr ---
2024-04-19 05:42:55,732 DEBUG
2024-04-19 05:42:55,732 INFO
2024-04-19 05:42:55,732 INFO ------------- Check pod ds-cts-1 filesystem is accessible -------------
2024-04-19 05:42:56,720 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:42:56,720 INFO
2024-04-19 05:42:56,720 INFO ------------------ Check pod ds-cts-1 restart count ------------------
2024-04-19 05:42:56,721 INFO
2024-04-19 05:42:56,721 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:42:56,721 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:42:57,214 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:57,214 DEBUG --- stdout ---
2024-04-19 05:42:57,214 DEBUG 0
2024-04-19 05:42:57,214 DEBUG --- stderr ---
2024-04-19 05:42:57,214 DEBUG
2024-04-19 05:42:57,214 INFO Pod ds-cts-1 has been restarted 0 times.
2024-04-19 05:42:57,214 INFO
2024-04-19 05:42:57,214 INFO -------------------- Check pod ds-cts-2 is running --------------------
2024-04-19 05:42:57,215 INFO
2024-04-19 05:42:57,215 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:42:57,215 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:42:57,711 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:42:57,711 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:57,711 DEBUG --- stdout ---
2024-04-19 05:42:57,712 DEBUG Running
2024-04-19 05:42:57,712 DEBUG --- stderr ---
2024-04-19 05:42:57,712 DEBUG
2024-04-19 05:42:57,712 INFO
2024-04-19 05:42:57,712 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:42:57,713 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:42:58,200 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:42:58,200 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:58,200 DEBUG --- stdout ---
2024-04-19 05:42:58,200 DEBUG true
2024-04-19 05:42:58,201 DEBUG --- stderr ---
2024-04-19 05:42:58,201 DEBUG
2024-04-19 05:42:58,201 INFO
2024-04-19 05:42:58,201 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.startTime}
2024-04-19 05:42:58,201 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:42:58,682 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:42:58,683 DEBUG --- stdout ---
2024-04-19 05:42:58,684 DEBUG 2024-04-17T18:32:38Z
2024-04-19 05:42:58,684 DEBUG --- stderr ---
2024-04-19 05:42:58,684 DEBUG
2024-04-19 05:42:58,685 INFO
2024-04-19 05:42:58,686 INFO ------------- Check pod ds-cts-2 filesystem is accessible -------------
2024-04-19 05:42:59,663 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:42:59,663 INFO
2024-04-19 05:42:59,663 INFO ------------------ Check pod ds-cts-2 restart count ------------------
2024-04-19 05:42:59,664 INFO
2024-04-19 05:42:59,664 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:42:59,664 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:00,229 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:00,229 DEBUG --- stdout ---
2024-04-19 05:43:00,230 DEBUG 0
2024-04-19 05:43:00,230 DEBUG --- stderr ---
2024-04-19 05:43:00,230 DEBUG
2024-04-19 05:43:00,230 INFO Pod ds-cts-2 has been restarted 0 times.
2024-04-19 05:43:00,230 INFO
2024-04-19 05:43:00,230 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-19 05:43:00,230 INFO
2024-04-19 05:43:00,230 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:00,230 INFO
2024-04-19 05:43:00,230 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:00,230 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:00,810 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:00,812 DEBUG --- stdout ---
2024-04-19 05:43:00,812 DEBUG 3
2024-04-19 05:43:00,812 DEBUG --- stderr ---
2024-04-19 05:43:00,812 DEBUG
2024-04-19 05:43:00,812 INFO
2024-04-19 05:43:00,812 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:43:00,812 INFO
2024-04-19 05:43:00,813 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-19 05:43:00,813 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:43:01,446 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-19 05:43:01,450 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:01,451 DEBUG --- stdout ---
2024-04-19 05:43:01,451 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2
2024-04-19 05:43:01,451 DEBUG --- stderr ---
2024-04-19 05:43:01,451 DEBUG
2024-04-19 05:43:01,451 INFO
2024-04-19 05:43:01,451 INFO ------------------ Check pod ds-idrepo-0 is running ------------------
2024-04-19 05:43:01,452 INFO
2024-04-19 05:43:01,452 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:01,452 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:01,989 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:01,990 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:01,990 DEBUG --- stdout ---
2024-04-19 05:43:01,990 DEBUG Running
2024-04-19 05:43:01,990 DEBUG --- stderr ---
2024-04-19 05:43:01,990 DEBUG
2024-04-19 05:43:01,990 INFO
2024-04-19 05:43:01,990 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:01,990 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:02,503 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:02,503 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:02,503 DEBUG --- stdout ---
2024-04-19 05:43:02,503 DEBUG true
2024-04-19 05:43:02,503 DEBUG --- stderr ---
2024-04-19 05:43:02,503 DEBUG
2024-04-19 05:43:02,503 INFO
2024-04-19 05:43:02,503 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.startTime}
2024-04-19 05:43:02,503 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:03,075 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:03,076 DEBUG --- stdout ---
2024-04-19 05:43:03,076 DEBUG 2024-04-17T18:34:28Z
2024-04-19 05:43:03,076 DEBUG --- stderr ---
2024-04-19 05:43:03,076 DEBUG
2024-04-19 05:43:03,077 INFO
2024-04-19 05:43:03,077 INFO ----------- Check pod ds-idrepo-0 filesystem is accessible -----------
2024-04-19 05:43:04,106 INFO [loop_until]: Function succeeded after 1s (rc=0) - expected pattern found
2024-04-19 05:43:04,107 INFO
2024-04-19 05:43:04,107 INFO ----------------- Check pod ds-idrepo-0 restart count -----------------
2024-04-19 05:43:04,107 INFO
2024-04-19 05:43:04,107 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:04,107 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:04,784 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:04,785 DEBUG --- stdout ---
2024-04-19 05:43:04,785 DEBUG 0
2024-04-19 05:43:04,786 DEBUG --- stderr ---
2024-04-19 05:43:04,786 DEBUG
2024-04-19 05:43:04,786 INFO Pod ds-idrepo-0 has been restarted 0 times.
2024-04-19 05:43:04,787 INFO
2024-04-19 05:43:04,787 INFO ------------------ Check pod ds-idrepo-1 is running ------------------
2024-04-19 05:43:04,787 INFO
2024-04-19 05:43:04,787 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:04,787 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:05,320 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:05,321 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:05,321 DEBUG --- stdout ---
2024-04-19 05:43:05,321 DEBUG Running
2024-04-19 05:43:05,321 DEBUG --- stderr ---
2024-04-19 05:43:05,321 DEBUG
2024-04-19 05:43:05,321 INFO
2024-04-19 05:43:05,321 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:05,321 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:05,865 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:05,866 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:05,866 DEBUG --- stdout ---
2024-04-19 05:43:05,866 DEBUG true
2024-04-19 05:43:05,866 DEBUG --- stderr ---
2024-04-19 05:43:05,866 DEBUG
2024-04-19 05:43:05,866 INFO
2024-04-19 05:43:05,867 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.startTime}
2024-04-19 05:43:05,867 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:06,358 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:06,358 DEBUG --- stdout ---
2024-04-19 05:43:06,358 DEBUG 2024-04-17T18:33:37Z
2024-04-19 05:43:06,359 DEBUG --- stderr ---
2024-04-19 05:43:06,359 DEBUG
2024-04-19 05:43:06,361 INFO
2024-04-19 05:43:06,361 INFO ----------- Check pod ds-idrepo-1 filesystem is accessible -----------
2024-04-19 05:43:07,329 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:07,329 INFO
2024-04-19 05:43:07,330 INFO ----------------- Check pod ds-idrepo-1 restart count -----------------
2024-04-19 05:43:07,330 INFO
2024-04-19 05:43:07,331 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:07,331 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:07,854 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:07,855 DEBUG --- stdout ---
2024-04-19 05:43:07,855 DEBUG 0
2024-04-19 05:43:07,855 DEBUG --- stderr ---
2024-04-19 05:43:07,855 DEBUG
2024-04-19 05:43:07,855 INFO Pod ds-idrepo-1 has been restarted 0 times.
2024-04-19 05:43:07,855 INFO
2024-04-19 05:43:07,855 INFO ------------------ Check pod ds-idrepo-2 is running ------------------
2024-04-19 05:43:07,856 INFO
2024-04-19 05:43:07,856 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:07,856 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:08,438 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:08,438 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:08,438 DEBUG --- stdout ---
2024-04-19 05:43:08,438 DEBUG Running
2024-04-19 05:43:08,438 DEBUG --- stderr ---
2024-04-19 05:43:08,439 DEBUG
2024-04-19 05:43:08,439 INFO
2024-04-19 05:43:08,439 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:08,439 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:08,981 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:08,982 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:08,982 DEBUG --- stdout ---
2024-04-19 05:43:08,983 DEBUG true
2024-04-19 05:43:08,983 DEBUG --- stderr ---
2024-04-19 05:43:08,983 DEBUG
2024-04-19 05:43:08,984 INFO
2024-04-19 05:43:08,984 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.startTime}
2024-04-19 05:43:08,984 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:09,577 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:09,578 DEBUG --- stdout ---
2024-04-19 05:43:09,578 DEBUG 2024-04-17T18:32:49Z
2024-04-19 05:43:09,578 DEBUG --- stderr ---
2024-04-19 05:43:09,578 DEBUG
2024-04-19 05:43:09,578 INFO
2024-04-19 05:43:09,578 INFO ----------- Check pod ds-idrepo-2 filesystem is accessible -----------
2024-04-19 05:43:10,575 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:10,575 INFO
2024-04-19 05:43:10,575 INFO ----------------- Check pod ds-idrepo-2 restart count -----------------
2024-04-19 05:43:10,576 INFO
2024-04-19 05:43:10,576 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:10,576 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:11,002 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:11,002 DEBUG --- stdout ---
2024-04-19 05:43:11,002 DEBUG 0
2024-04-19 05:43:11,002 DEBUG --- stderr ---
2024-04-19 05:43:11,002 DEBUG
2024-04-19 05:43:11,003 INFO Pod ds-idrepo-2 has been restarted 0 times.
2024-04-19 05:43:11,003 INFO
2024-04-19 05:43:11,003 INFO ****************************** Initializing component pods for AM ******************************
2024-04-19 05:43:11,003 INFO
2024-04-19 05:43:11,003 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:11,003 INFO
2024-04-19 05:43:11,003 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:11,003 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:11,458 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:11,458 DEBUG --- stdout ---
2024-04-19 05:43:11,459 DEBUG 3
2024-04-19 05:43:11,459 DEBUG --- stderr ---
2024-04-19 05:43:11,459 DEBUG
2024-04-19 05:43:11,459 INFO
2024-04-19 05:43:11,459 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:43:11,459 INFO
2024-04-19 05:43:11,459 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-19 05:43:11,459 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:43:11,946 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-19 05:43:11,946 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:11,946 DEBUG --- stdout ---
2024-04-19 05:43:11,946 DEBUG am-6fd554f8f7-6rhpf am-6fd554f8f7-hvvkw am-6fd554f8f7-pmj4b
2024-04-19 05:43:11,947 DEBUG --- stderr ---
2024-04-19 05:43:11,947 DEBUG
2024-04-19 05:43:11,947 INFO
2024-04-19 05:43:11,947 INFO -------------- Check pod am-6fd554f8f7-6rhpf is running --------------
2024-04-19 05:43:11,947 INFO
2024-04-19 05:43:11,947 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-6fd554f8f7-6rhpf -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:11,948 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:12,375 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:12,377 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:12,377 DEBUG --- stdout ---
2024-04-19 05:43:12,377 DEBUG Running
2024-04-19 05:43:12,377 DEBUG --- stderr ---
2024-04-19 05:43:12,377 DEBUG
2024-04-19 05:43:12,378 INFO
2024-04-19 05:43:12,378 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-6fd554f8f7-6rhpf -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:12,378 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:12,767 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:12,768 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:12,768 DEBUG --- stdout ---
2024-04-19 05:43:12,768 DEBUG true
2024-04-19 05:43:12,768 DEBUG --- stderr ---
2024-04-19 05:43:12,768 DEBUG
2024-04-19 05:43:12,768 INFO
2024-04-19 05:43:12,768 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-6fd554f8f7-6rhpf --output jsonpath={.status.startTime}
2024-04-19 05:43:12,768 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:13,163 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:13,163 DEBUG --- stdout ---
2024-04-19 05:43:13,163 DEBUG 2024-04-18T23:35:12Z
2024-04-19 05:43:13,163 DEBUG --- stderr ---
2024-04-19 05:43:13,163 DEBUG
2024-04-19 05:43:13,164 INFO
2024-04-19 05:43:13,164 INFO ------- Check pod am-6fd554f8f7-6rhpf filesystem is accessible -------
2024-04-19 05:43:14,102 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:14,102 INFO
2024-04-19 05:43:14,102 INFO ------------- Check pod am-6fd554f8f7-6rhpf restart count -------------
2024-04-19 05:43:14,103 INFO
2024-04-19 05:43:14,103 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-6fd554f8f7-6rhpf --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:14,103 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:14,467 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:14,467 DEBUG --- stdout ---
2024-04-19 05:43:14,468 DEBUG 0
2024-04-19 05:43:14,468 DEBUG --- stderr ---
2024-04-19 05:43:14,468 DEBUG
2024-04-19 05:43:14,468 INFO Pod am-6fd554f8f7-6rhpf has been restarted 0 times.
2024-04-19 05:43:14,468 INFO
2024-04-19 05:43:14,468 INFO -------------- Check pod am-6fd554f8f7-hvvkw is running --------------
2024-04-19 05:43:14,468 INFO
2024-04-19 05:43:14,468 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-6fd554f8f7-hvvkw -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:14,468 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:14,832 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:14,832 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:14,832 DEBUG --- stdout ---
2024-04-19 05:43:14,832 DEBUG Running
2024-04-19 05:43:14,832 DEBUG --- stderr ---
2024-04-19 05:43:14,832 DEBUG
2024-04-19 05:43:14,833 INFO
2024-04-19 05:43:14,833 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-6fd554f8f7-hvvkw -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:14,833 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:15,205 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:15,205 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:15,205 DEBUG --- stdout ---
2024-04-19 05:43:15,205 DEBUG true
2024-04-19 05:43:15,205 DEBUG --- stderr ---
2024-04-19 05:43:15,205 DEBUG
2024-04-19 05:43:15,206 INFO
2024-04-19 05:43:15,206 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-6fd554f8f7-hvvkw --output jsonpath={.status.startTime}
2024-04-19 05:43:15,206 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:15,577 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:15,577 DEBUG --- stdout ---
2024-04-19 05:43:15,577 DEBUG 2024-04-18T23:35:53Z
2024-04-19 05:43:15,577 DEBUG --- stderr ---
2024-04-19 05:43:15,577 DEBUG
2024-04-19 05:43:15,578 INFO
2024-04-19 05:43:15,578 INFO ------- Check pod am-6fd554f8f7-hvvkw filesystem is accessible -------
2024-04-19 05:43:16,412 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:16,412 INFO
2024-04-19 05:43:16,412 INFO ------------- Check pod am-6fd554f8f7-hvvkw restart count -------------
2024-04-19 05:43:16,412 INFO
2024-04-19 05:43:16,412 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-6fd554f8f7-hvvkw --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:16,412 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:16,796 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:16,796 DEBUG --- stdout ---
2024-04-19 05:43:16,796 DEBUG 0
2024-04-19 05:43:16,796 DEBUG --- stderr ---
2024-04-19 05:43:16,796 DEBUG
2024-04-19 05:43:16,796 INFO Pod am-6fd554f8f7-hvvkw has been restarted 0 times.
2024-04-19 05:43:16,796 INFO
2024-04-19 05:43:16,796 INFO -------------- Check pod am-6fd554f8f7-pmj4b is running --------------
2024-04-19 05:43:16,797 INFO
2024-04-19 05:43:16,797 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-6fd554f8f7-pmj4b -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:16,797 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:17,167 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:17,168 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:17,168 DEBUG --- stdout ---
2024-04-19 05:43:17,168 DEBUG Running
2024-04-19 05:43:17,168 DEBUG --- stderr ---
2024-04-19 05:43:17,168 DEBUG
2024-04-19 05:43:17,168 INFO
2024-04-19 05:43:17,168 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-6fd554f8f7-pmj4b -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:17,168 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:17,535 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:17,537 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:17,537 DEBUG --- stdout ---
2024-04-19 05:43:17,537 DEBUG true
2024-04-19 05:43:17,537 DEBUG --- stderr ---
2024-04-19 05:43:17,537 DEBUG
2024-04-19 05:43:17,537 INFO
2024-04-19 05:43:17,537 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-6fd554f8f7-pmj4b --output jsonpath={.status.startTime}
2024-04-19 05:43:17,537 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:17,901 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:17,902 DEBUG --- stdout ---
2024-04-19 05:43:17,902 DEBUG 2024-04-18T23:36:23Z
2024-04-19 05:43:17,902 DEBUG --- stderr ---
2024-04-19 05:43:17,902 DEBUG
2024-04-19 05:43:17,903 INFO
2024-04-19 05:43:17,903 INFO ------- Check pod am-6fd554f8f7-pmj4b filesystem is accessible -------
2024-04-19 05:43:18,732 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:18,732 INFO
2024-04-19 05:43:18,732 INFO ------------- Check pod am-6fd554f8f7-pmj4b restart count -------------
2024-04-19 05:43:18,733 INFO
2024-04-19 05:43:18,733 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-6fd554f8f7-pmj4b --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:18,733 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:19,107 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:19,107 DEBUG --- stdout ---
2024-04-19 05:43:19,108 DEBUG 0
2024-04-19 05:43:19,108 DEBUG --- stderr ---
2024-04-19 05:43:19,108 DEBUG
2024-04-19 05:43:19,108 INFO Pod am-6fd554f8f7-pmj4b has been restarted 0 times.
2024-04-19 05:43:19,108 INFO
2024-04-19 05:43:19,108 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-19 05:43:19,108 INFO
2024-04-19 05:43:19,108 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:43:19,108 INFO
2024-04-19 05:43:19,108 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-19 05:43:19,108 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:43:19,490 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:19,490 DEBUG --- stdout ---
2024-04-19 05:43:19,490 DEBUG amster-grkh7
2024-04-19 05:43:19,490 DEBUG --- stderr ---
2024-04-19 05:43:19,490 DEBUG
2024-04-19 05:43:19,491 INFO
2024-04-19 05:43:19,491 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-19 05:43:19,491 INFO
2024-04-19 05:43:19,491 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:19,491 INFO
2024-04-19 05:43:19,491 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:19,491 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:19,875 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:19,875 DEBUG --- stdout ---
2024-04-19 05:43:19,875 DEBUG 2
2024-04-19 05:43:19,875 DEBUG --- stderr ---
2024-04-19 05:43:19,875 DEBUG
2024-04-19 05:43:19,875 INFO
2024-04-19 05:43:19,875 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:43:19,876 INFO
2024-04-19 05:43:19,876 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name}` | grep 2
2024-04-19 05:43:19,876 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:43:20,269 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-19 05:43:20,270 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:20,270 DEBUG --- stdout ---
2024-04-19 05:43:20,270 DEBUG idm-b85f9d978-txpwv idm-b85f9d978-wnhxb
2024-04-19 05:43:20,270 DEBUG --- stderr ---
2024-04-19 05:43:20,270 DEBUG
2024-04-19 05:43:20,270 INFO
2024-04-19 05:43:20,270 INFO -------------- Check pod idm-b85f9d978-txpwv is running --------------
2024-04-19 05:43:20,270 INFO
2024-04-19 05:43:20,270 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-txpwv -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:20,270 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:20,624 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:20,624 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:20,624 DEBUG --- stdout ---
2024-04-19 05:43:20,624 DEBUG Running
2024-04-19 05:43:20,624 DEBUG --- stderr ---
2024-04-19 05:43:20,624 DEBUG
2024-04-19 05:43:20,624 INFO
2024-04-19 05:43:20,624 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-txpwv -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:20,624 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:20,979 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:20,979 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:20,979 DEBUG --- stdout ---
2024-04-19 05:43:20,979 DEBUG true
2024-04-19 05:43:20,979 DEBUG --- stderr ---
2024-04-19 05:43:20,979 DEBUG
2024-04-19 05:43:20,979 INFO
2024-04-19 05:43:20,979 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-txpwv --output jsonpath={.status.startTime}
2024-04-19 05:43:20,979 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:21,345 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:21,345 DEBUG --- stdout ---
2024-04-19 05:43:21,346 DEBUG 2024-04-17T16:00:45Z
2024-04-19 05:43:21,346 DEBUG --- stderr ---
2024-04-19 05:43:21,346 DEBUG
2024-04-19 05:43:21,346 INFO
2024-04-19 05:43:21,346 INFO ------- Check pod idm-b85f9d978-txpwv filesystem is accessible -------
2024-04-19 05:43:22,165 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:22,165 INFO
2024-04-19 05:43:22,165 INFO ------------- Check pod idm-b85f9d978-txpwv restart count -------------
2024-04-19 05:43:22,165 INFO
2024-04-19 05:43:22,165 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-txpwv --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:22,165 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:22,520 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:22,520 DEBUG --- stdout ---
2024-04-19 05:43:22,520 DEBUG 0
2024-04-19 05:43:22,520 DEBUG --- stderr ---
2024-04-19 05:43:22,520 DEBUG
2024-04-19 05:43:22,521 INFO Pod idm-b85f9d978-txpwv has been restarted 0 times.
2024-04-19 05:43:22,521 INFO
2024-04-19 05:43:22,521 INFO -------------- Check pod idm-b85f9d978-wnhxb is running --------------
2024-04-19 05:43:22,521 INFO
2024-04-19 05:43:22,521 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-wnhxb -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:22,521 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:22,876 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:22,877 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:22,877 DEBUG --- stdout ---
2024-04-19 05:43:22,878 DEBUG Running
2024-04-19 05:43:22,878 DEBUG --- stderr ---
2024-04-19 05:43:22,878 DEBUG
2024-04-19 05:43:22,878 INFO
2024-04-19 05:43:22,878 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-wnhxb -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:22,878 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:23,238 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:23,238 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:23,238 DEBUG --- stdout ---
2024-04-19 05:43:23,238 DEBUG true
2024-04-19 05:43:23,238 DEBUG --- stderr ---
2024-04-19 05:43:23,238 DEBUG
2024-04-19 05:43:23,238 INFO
2024-04-19 05:43:23,238 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-wnhxb --output jsonpath={.status.startTime}
2024-04-19 05:43:23,238 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:23,595 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:23,595 DEBUG --- stdout ---
2024-04-19 05:43:23,595 DEBUG 2024-04-17T16:00:45Z
2024-04-19 05:43:23,595 DEBUG --- stderr ---
2024-04-19 05:43:23,595 DEBUG
2024-04-19 05:43:23,596 INFO
2024-04-19 05:43:23,596 INFO ------- Check pod idm-b85f9d978-wnhxb filesystem is accessible -------
2024-04-19 05:43:24,412 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:24,413 INFO
2024-04-19 05:43:24,413 INFO ------------- Check pod idm-b85f9d978-wnhxb restart count -------------
2024-04-19 05:43:24,413 INFO
2024-04-19 05:43:24,413 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-wnhxb --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:24,413 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:24,784 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:24,784 DEBUG --- stdout ---
2024-04-19 05:43:24,784 DEBUG 0
2024-04-19 05:43:24,784 DEBUG --- stderr ---
2024-04-19 05:43:24,785 DEBUG
2024-04-19 05:43:24,785 INFO Pod idm-b85f9d978-wnhxb has been restarted 0 times.
2024-04-19 05:43:24,785 INFO
2024-04-19 05:43:24,785 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-19 05:43:24,785 INFO
2024-04-19 05:43:24,785 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:24,785 INFO
2024-04-19 05:43:24,785 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:24,785 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:25,105 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:25,105 DEBUG --- stdout ---
2024-04-19 05:43:25,105 DEBUG 1
2024-04-19 05:43:25,105 DEBUG --- stderr ---
2024-04-19 05:43:25,105 DEBUG
2024-04-19 05:43:25,105 INFO
2024-04-19 05:43:25,105 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:43:25,106 INFO
2024-04-19 05:43:25,106 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-19 05:43:25,106 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:43:25,428 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-19 05:43:25,428 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:25,428 DEBUG --- stdout ---
2024-04-19 05:43:25,428 DEBUG end-user-ui-58878d7668-z8s6j
2024-04-19 05:43:25,428 DEBUG --- stderr ---
2024-04-19 05:43:25,428 DEBUG
2024-04-19 05:43:25,429 INFO
2024-04-19 05:43:25,429 INFO ---------- Check pod end-user-ui-58878d7668-z8s6j is running ----------
2024-04-19 05:43:25,429 INFO
2024-04-19 05:43:25,429 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-58878d7668-z8s6j -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:25,429 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:25,759 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:25,760 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:25,760 DEBUG --- stdout ---
2024-04-19 05:43:25,760 DEBUG Running
2024-04-19 05:43:25,760 DEBUG --- stderr ---
2024-04-19 05:43:25,760 DEBUG
2024-04-19 05:43:25,760 INFO
2024-04-19 05:43:25,760 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-58878d7668-z8s6j -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:25,760 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:26,072 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:26,072 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:26,072 DEBUG --- stdout ---
2024-04-19 05:43:26,072 DEBUG true
2024-04-19 05:43:26,072 DEBUG --- stderr ---
2024-04-19 05:43:26,072 DEBUG
2024-04-19 05:43:26,072 INFO
2024-04-19 05:43:26,072 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-58878d7668-z8s6j --output jsonpath={.status.startTime}
2024-04-19 05:43:26,072 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:26,386 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:26,387 DEBUG --- stdout ---
2024-04-19 05:43:26,387 DEBUG 2024-04-17T16:01:55Z
2024-04-19 05:43:26,387 DEBUG --- stderr ---
2024-04-19 05:43:26,387 DEBUG
2024-04-19 05:43:26,387 INFO
2024-04-19 05:43:26,387 INFO --- Check pod end-user-ui-58878d7668-z8s6j filesystem is accessible ---
2024-04-19 05:43:27,175 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:27,176 INFO
2024-04-19 05:43:27,176 INFO -------- Check pod end-user-ui-58878d7668-z8s6j restart count --------
2024-04-19 05:43:27,176 INFO
2024-04-19 05:43:27,176 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-58878d7668-z8s6j --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:27,176 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:27,490 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:27,490 DEBUG --- stdout ---
2024-04-19 05:43:27,490 DEBUG 0
2024-04-19 05:43:27,490 DEBUG --- stderr ---
2024-04-19 05:43:27,490 DEBUG
2024-04-19 05:43:27,491 INFO Pod end-user-ui-58878d7668-z8s6j has been restarted 0 times.
2024-04-19 05:43:27,491 INFO
2024-04-19 05:43:27,491 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-19 05:43:27,491 INFO
2024-04-19 05:43:27,491 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:27,491 INFO
2024-04-19 05:43:27,491 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:27,491 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:27,805 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:27,805 DEBUG --- stdout ---
2024-04-19 05:43:27,805 DEBUG 1
2024-04-19 05:43:27,805 DEBUG --- stderr ---
2024-04-19 05:43:27,805 DEBUG
2024-04-19 05:43:27,806 INFO
2024-04-19 05:43:27,806 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:43:27,806 INFO
2024-04-19 05:43:27,806 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-19 05:43:27,806 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:43:28,129 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-19 05:43:28,131 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:28,131 DEBUG --- stdout ---
2024-04-19 05:43:28,131 DEBUG login-ui-5fdf49fc66-dlhjd
2024-04-19 05:43:28,131 DEBUG --- stderr ---
2024-04-19 05:43:28,131 DEBUG
2024-04-19 05:43:28,131 INFO
2024-04-19 05:43:28,131 INFO ----------- Check pod login-ui-5fdf49fc66-dlhjd is running -----------
2024-04-19 05:43:28,132 INFO
2024-04-19 05:43:28,132 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5fdf49fc66-dlhjd -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:28,132 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:28,462 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:28,463 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:28,463 DEBUG --- stdout ---
2024-04-19 05:43:28,463 DEBUG Running
2024-04-19 05:43:28,463 DEBUG --- stderr ---
2024-04-19 05:43:28,463 DEBUG
2024-04-19 05:43:28,463 INFO
2024-04-19 05:43:28,463 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5fdf49fc66-dlhjd -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:28,463 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:28,776 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:28,776 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:28,776 DEBUG --- stdout ---
2024-04-19 05:43:28,776 DEBUG true
2024-04-19 05:43:28,776 DEBUG --- stderr ---
2024-04-19 05:43:28,776 DEBUG
2024-04-19 05:43:28,777 INFO
2024-04-19 05:43:28,777 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5fdf49fc66-dlhjd --output jsonpath={.status.startTime}
2024-04-19 05:43:28,777 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:29,087 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:29,087 DEBUG --- stdout ---
2024-04-19 05:43:29,087 DEBUG 2024-04-17T16:01:55Z
2024-04-19 05:43:29,087 DEBUG --- stderr ---
2024-04-19 05:43:29,087 DEBUG
2024-04-19 05:43:29,088 INFO
2024-04-19 05:43:29,088 INFO ---- Check pod login-ui-5fdf49fc66-dlhjd filesystem is accessible ----
2024-04-19 05:43:29,857 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:29,857 INFO
2024-04-19 05:43:29,857 INFO ---------- Check pod login-ui-5fdf49fc66-dlhjd restart count ----------
2024-04-19 05:43:29,858 INFO
2024-04-19 05:43:29,858 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5fdf49fc66-dlhjd --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:29,858 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:30,150 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:30,151 DEBUG --- stdout ---
2024-04-19 05:43:30,151 DEBUG 0
2024-04-19 05:43:30,151 DEBUG --- stderr ---
2024-04-19 05:43:30,151 DEBUG
2024-04-19 05:43:30,151 INFO Pod login-ui-5fdf49fc66-dlhjd has been restarted 0 times.
2024-04-19 05:43:30,151 INFO
2024-04-19 05:43:30,151 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-19 05:43:30,151 INFO
2024-04-19 05:43:30,151 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:30,151 INFO
2024-04-19 05:43:30,151 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:30,151 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:30,461 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:30,461 DEBUG --- stdout ---
2024-04-19 05:43:30,461 DEBUG 1
2024-04-19 05:43:30,461 DEBUG --- stderr ---
2024-04-19 05:43:30,461 DEBUG
2024-04-19 05:43:30,461 INFO
2024-04-19 05:43:30,461 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:43:30,462 INFO
2024-04-19 05:43:30,462 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-19 05:43:30,462 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:43:30,773 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-19 05:43:30,773 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:30,773 DEBUG --- stdout ---
2024-04-19 05:43:30,774 DEBUG admin-ui-c4c7bf584-tlgkc
2024-04-19 05:43:30,774 DEBUG --- stderr ---
2024-04-19 05:43:30,774 DEBUG
2024-04-19 05:43:30,774 INFO
2024-04-19 05:43:30,774 INFO ------------ Check pod admin-ui-c4c7bf584-tlgkc is running ------------
2024-04-19 05:43:30,774 INFO
2024-04-19 05:43:30,774 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-c4c7bf584-tlgkc -o=jsonpath={.status.phase} | grep "Running"
2024-04-19 05:43:30,774 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:31,102 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:31,102 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:31,102 DEBUG --- stdout ---
2024-04-19 05:43:31,102 DEBUG Running
2024-04-19 05:43:31,102 DEBUG --- stderr ---
2024-04-19 05:43:31,102 DEBUG
2024-04-19 05:43:31,103 INFO
2024-04-19 05:43:31,103 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-c4c7bf584-tlgkc -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-19 05:43:31,103 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-19 05:43:31,408 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:31,408 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:31,408 DEBUG --- stdout ---
2024-04-19 05:43:31,408 DEBUG true
2024-04-19 05:43:31,408 DEBUG --- stderr ---
2024-04-19 05:43:31,408 DEBUG
2024-04-19 05:43:31,409 INFO
2024-04-19 05:43:31,409 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-c4c7bf584-tlgkc --output jsonpath={.status.startTime}
2024-04-19 05:43:31,409 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:31,708 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:31,708 DEBUG --- stdout ---
2024-04-19 05:43:31,708 DEBUG 2024-04-17T16:01:54Z
2024-04-19 05:43:31,708 DEBUG --- stderr ---
2024-04-19 05:43:31,708 DEBUG
2024-04-19 05:43:31,709 INFO
2024-04-19 05:43:31,709 INFO ----- Check pod admin-ui-c4c7bf584-tlgkc filesystem is accessible -----
2024-04-19 05:43:32,466 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:32,466 INFO
2024-04-19 05:43:32,466 INFO ---------- Check pod admin-ui-c4c7bf584-tlgkc restart count ----------
2024-04-19 05:43:32,466 INFO
2024-04-19 05:43:32,466 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-c4c7bf584-tlgkc --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-19 05:43:32,466 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:32,772 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:32,772 DEBUG --- stdout ---
2024-04-19 05:43:32,772 DEBUG 0
2024-04-19 05:43:32,772 DEBUG --- stderr ---
2024-04-19 05:43:32,772 DEBUG
2024-04-19 05:43:32,772 INFO Pod admin-ui-c4c7bf584-tlgkc has been restarted 0 times.
2024-04-19 05:43:32,772 INFO
2024-04-19 05:43:32,773 INFO ***************************** Checking DS-CTS component is running *****************************
2024-04-19 05:43:32,773 INFO
2024-04-19 05:43:32,773 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:32,773 INFO
2024-04-19 05:43:32,773 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:32,773 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:33,160 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:33,162 DEBUG --- stdout ---
2024-04-19 05:43:33,162 DEBUG 3
2024-04-19 05:43:33,162 DEBUG --- stderr ---
2024-04-19 05:43:33,162 DEBUG
2024-04-19 05:43:33,162 INFO
2024-04-19 05:43:33,163 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-19 05:43:33,163 INFO
2024-04-19 05:43:33,163 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-cts --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3"
2024-04-19 05:43:33,163 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-19 05:43:33,554 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:33,554 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:33,554 DEBUG --- stdout ---
2024-04-19 05:43:33,554 DEBUG current:3 ready:3 replicas:3
2024-04-19 05:43:33,554 DEBUG --- stderr ---
2024-04-19 05:43:33,554 DEBUG
2024-04-19 05:43:33,554 INFO
2024-04-19 05:43:33,554 INFO *************************** Checking DS-IDREPO component is running ***************************
2024-04-19 05:43:33,554 INFO
2024-04-19 05:43:33,555 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:33,555 INFO
2024-04-19 05:43:33,555 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:33,555 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:33,931 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:33,931 DEBUG --- stdout ---
2024-04-19 05:43:33,931 DEBUG 3
2024-04-19 05:43:33,931 DEBUG --- stderr ---
2024-04-19 05:43:33,931 DEBUG
2024-04-19 05:43:33,932 INFO
2024-04-19 05:43:33,932 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-19 05:43:33,932 INFO
2024-04-19 05:43:33,932 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-idrepo --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3"
2024-04-19 05:43:33,932 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-19 05:43:34,319 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:34,319 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:34,319 DEBUG --- stdout ---
2024-04-19 05:43:34,319 DEBUG current:3 ready:3 replicas:3
2024-04-19 05:43:34,319 DEBUG --- stderr ---
2024-04-19 05:43:34,319 DEBUG
2024-04-19 05:43:34,320 INFO
2024-04-19 05:43:34,320 INFO ******************************* Checking AM component is running *******************************
2024-04-19 05:43:34,320 INFO
2024-04-19 05:43:34,320 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:34,320 INFO
2024-04-19 05:43:34,320 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:34,320 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:34,711 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:34,712 DEBUG --- stdout ---
2024-04-19 05:43:34,712 DEBUG 3
2024-04-19 05:43:34,712 DEBUG --- stderr ---
2024-04-19 05:43:34,712 DEBUG
2024-04-19 05:43:34,712 INFO
2024-04-19 05:43:34,712 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-19 05:43:34,712 INFO
2024-04-19 05:43:34,712 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments am --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:3 replicas:3"
2024-04-19 05:43:34,712 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-19 05:43:35,110 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:35,111 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:35,111 DEBUG --- stdout ---
2024-04-19 05:43:35,111 DEBUG ready:3 replicas:3
2024-04-19 05:43:35,111 DEBUG --- stderr ---
2024-04-19 05:43:35,111 DEBUG
2024-04-19 05:43:35,111 INFO
2024-04-19 05:43:35,111 INFO ***************************** Checking AMSTER component is running *****************************
2024-04-19 05:43:35,111 INFO
2024-04-19 05:43:35,111 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:35,111 INFO
2024-04-19 05:43:35,111 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-19 05:43:35,111 INFO
2024-04-19 05:43:35,111 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get jobs amster --output jsonpath="{.status.succeeded}" | grep "1"
2024-04-19 05:43:35,111 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-19 05:43:35,478 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:35,478 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:35,478 DEBUG --- stdout ---
2024-04-19 05:43:35,478 DEBUG 1
2024-04-19 05:43:35,478 DEBUG --- stderr ---
2024-04-19 05:43:35,479 DEBUG
2024-04-19 05:43:35,479 INFO
2024-04-19 05:43:35,479 INFO ****************************** Checking IDM component is running ******************************
2024-04-19 05:43:35,479 INFO
2024-04-19 05:43:35,479 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:35,479 INFO
2024-04-19 05:43:35,479 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:35,479 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:35,868 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:35,869 DEBUG --- stdout ---
2024-04-19 05:43:35,869 DEBUG 2
2024-04-19 05:43:35,869 DEBUG --- stderr ---
2024-04-19 05:43:35,869 DEBUG
2024-04-19 05:43:35,869 INFO
2024-04-19 05:43:35,869 INFO -------------- Waiting for 2 expected pod(s) to be ready --------------
2024-04-19 05:43:35,869 INFO
2024-04-19 05:43:35,869 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment idm --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:2 replicas:2"
2024-04-19 05:43:35,869 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-19 05:43:36,264 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:36,265 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:36,265 DEBUG --- stdout ---
2024-04-19 05:43:36,265 DEBUG ready:2 replicas:2
2024-04-19 05:43:36,265 DEBUG --- stderr ---
2024-04-19 05:43:36,265 DEBUG
2024-04-19 05:43:36,265 INFO
2024-04-19 05:43:36,265 INFO ************************** Checking END-USER-UI component is running **************************
2024-04-19 05:43:36,265 INFO
2024-04-19 05:43:36,265 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:36,266 INFO
2024-04-19 05:43:36,266 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:36,266 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:36,585 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:36,585 DEBUG --- stdout ---
2024-04-19 05:43:36,586 DEBUG 1
2024-04-19 05:43:36,586 DEBUG --- stderr ---
2024-04-19 05:43:36,586 DEBUG
2024-04-19 05:43:36,586 INFO
2024-04-19 05:43:36,586 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-19 05:43:36,586 INFO
2024-04-19 05:43:36,586 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments end-user-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-19 05:43:36,586 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-19 05:43:36,901 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:36,901 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:36,901 DEBUG --- stdout ---
2024-04-19 05:43:36,901 DEBUG ready:1 replicas:1
2024-04-19 05:43:36,901 DEBUG --- stderr ---
2024-04-19 05:43:36,901 DEBUG
2024-04-19 05:43:36,901 INFO
2024-04-19 05:43:36,901 INFO **************************** Checking LOGIN-UI component is running ****************************
2024-04-19 05:43:36,901 INFO
2024-04-19 05:43:36,902 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:36,902 INFO
2024-04-19 05:43:36,902 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:36,902 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:37,203 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:37,203 DEBUG --- stdout ---
2024-04-19 05:43:37,203 DEBUG 1
2024-04-19 05:43:37,203 DEBUG --- stderr ---
2024-04-19 05:43:37,203 DEBUG
2024-04-19 05:43:37,204 INFO
2024-04-19 05:43:37,204 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-19 05:43:37,204 INFO
2024-04-19 05:43:37,204 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments login-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-19 05:43:37,204 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-19 05:43:37,520 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:37,520 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:37,520 DEBUG --- stdout ---
2024-04-19 05:43:37,520 DEBUG ready:1 replicas:1
2024-04-19 05:43:37,521 DEBUG --- stderr ---
2024-04-19 05:43:37,521 DEBUG
2024-04-19 05:43:37,521 INFO
2024-04-19 05:43:37,521 INFO **************************** Checking ADMIN-UI component is running ****************************
2024-04-19 05:43:37,521 INFO
2024-04-19 05:43:37,521 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:43:37,521 INFO
2024-04-19 05:43:37,521 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:43:37,521 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:37,817 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:37,817 DEBUG --- stdout ---
2024-04-19 05:43:37,817 DEBUG 1
2024-04-19 05:43:37,817 DEBUG --- stderr ---
2024-04-19 05:43:37,817 DEBUG
2024-04-19 05:43:37,817 INFO
2024-04-19 05:43:37,817 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-19 05:43:37,818 INFO
2024-04-19 05:43:37,818 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments admin-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-19 05:43:37,818 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-19 05:43:38,134 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-19 05:43:38,135 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:38,135 DEBUG --- stdout ---
2024-04-19 05:43:38,135 DEBUG ready:1 replicas:1
2024-04-19 05:43:38,135 DEBUG --- stderr ---
2024-04-19 05:43:38,135 DEBUG
2024-04-19 05:43:38,135 INFO
2024-04-19 05:43:38,135 INFO ****************************** Livecheck stage: After deployment ******************************
2024-04-19 05:43:38,135 INFO
2024-04-19 05:43:38,135 INFO ---------------------- Running DS-CTS livecheck ----------------------
2024-04-19 05:43:38,135 INFO Livecheck to ds-cts-0
2024-04-19 05:43:38,135 INFO
2024-04-19 05:43:38,135 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}"
2024-04-19 05:43:38,136 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-19 05:43:38,450 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:38,452 DEBUG --- stdout ---
2024-04-19 05:43:38,452 DEBUG RGlLaTRoUUY2dlkyc3hUWTM0enhpbUt0dW1yU2QxelM=
2024-04-19 05:43:38,453 DEBUG --- stderr ---
2024-04-19 05:43:38,453 DEBUG
2024-04-19 05:43:38,453 INFO
2024-04-19 05:43:38,453 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-19 05:43:40,227 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:43:40,227 DEBUG --- stdout ---
2024-04-19 05:43:40,228 DEBUG dn:
alive: true
2024-04-19 05:43:40,228 DEBUG --- stderr ---
2024-04-19 05:43:40,228 DEBUG
2024-04-19 05:43:40,228 INFO
2024-04-19 05:43:40,228 INFO Livecheck to ds-cts-1
2024-04-19 05:43:40,228 INFO
2024-04-19 05:43:40,229 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-19 05:43:42,045 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:43:42,046 DEBUG --- stdout ---
2024-04-19 05:43:42,046 DEBUG dn:
alive: true
2024-04-19 05:43:42,046 DEBUG --- stderr ---
2024-04-19 05:43:42,046 DEBUG
2024-04-19 05:43:42,046 INFO
2024-04-19 05:43:42,046 INFO Livecheck to ds-cts-2
2024-04-19 05:43:42,046 INFO
2024-04-19 05:43:42,047 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-19 05:43:43,811 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:43:43,811 DEBUG --- stdout ---
2024-04-19 05:43:43,811 DEBUG dn:
alive: true
2024-04-19 05:43:43,812 DEBUG --- stderr ---
2024-04-19 05:43:43,812 DEBUG
2024-04-19 05:43:43,812 INFO
2024-04-19 05:43:43,812 INFO
2024-04-19 05:43:43,812 INFO --------------------- Running DS-IDREPO livecheck ---------------------
2024-04-19 05:43:43,812 INFO Livecheck to ds-idrepo-0
2024-04-19 05:43:43,812 INFO
2024-04-19 05:43:43,812 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}"
2024-04-19 05:43:43,812 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-19 05:43:44,109 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:44,109 DEBUG --- stdout ---
2024-04-19 05:43:44,109 DEBUG RGlLaTRoUUY2dlkyc3hUWTM0enhpbUt0dW1yU2QxelM=
2024-04-19 05:43:44,109 DEBUG --- stderr ---
2024-04-19 05:43:44,109 DEBUG
2024-04-19 05:43:44,109 INFO
2024-04-19 05:43:44,110 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-19 05:43:45,938 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:43:45,938 DEBUG --- stdout ---
2024-04-19 05:43:45,938 DEBUG dn:
alive: true
2024-04-19 05:43:45,938 DEBUG --- stderr ---
2024-04-19 05:43:45,938 DEBUG
2024-04-19 05:43:45,938 INFO
2024-04-19 05:43:45,938 INFO Livecheck to ds-idrepo-1
2024-04-19 05:43:45,938 INFO
2024-04-19 05:43:45,939 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-19 05:43:47,684 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:43:47,684 DEBUG --- stdout ---
2024-04-19 05:43:47,684 DEBUG dn:
alive: true
2024-04-19 05:43:47,684 DEBUG --- stderr ---
2024-04-19 05:43:47,684 DEBUG
2024-04-19 05:43:47,684 INFO
2024-04-19 05:43:47,684 INFO Livecheck to ds-idrepo-2
2024-04-19 05:43:47,684 INFO
2024-04-19 05:43:47,685 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-19 05:43:49,426 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-19 05:43:49,427 DEBUG --- stdout ---
2024-04-19 05:43:49,428 DEBUG dn:
alive: true
2024-04-19 05:43:49,428 DEBUG --- stderr ---
2024-04-19 05:43:49,428 DEBUG
2024-04-19 05:43:49,428 INFO
2024-04-19 05:43:49,428 INFO
2024-04-19 05:43:49,428 INFO ------------------------ Running AM livecheck ------------------------
2024-04-19 05:43:49,428 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready
2024-04-19 05:43:49,429 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready"
2024-04-19 05:43:49,732 INFO [http_cmd]: http status code OK
2024-04-19 05:43:49,732 DEBUG --- status code ---
2024-04-19 05:43:49,732 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:49,732 DEBUG --- http response ---
2024-04-19 05:43:49,732 DEBUG
2024-04-19 05:43:49,732 INFO
2024-04-19 05:43:49,732 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.AM_PASSWORDS_AMADMIN_CLEAR}"
2024-04-19 05:43:49,732 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-19 05:43:50,066 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:50,067 DEBUG --- stdout ---
2024-04-19 05:43:50,067 DEBUG dUl1ZjBBejNacjd5Nk94akxLYjF6eWZj
2024-04-19 05:43:50,067 DEBUG --- stderr ---
2024-04-19 05:43:50,067 DEBUG
2024-04-19 05:43:50,067 INFO Authenticate user amadmin via REST
2024-04-19 05:43:50,068 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
2024-04-19 05:43:50,287 INFO [http_cmd]: http status code OK
2024-04-19 05:43:50,287 DEBUG --- status code ---
2024-04-19 05:43:50,287 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:50,287 DEBUG --- http response ---
2024-04-19 05:43:50,288 DEBUG {
"tokenId": "-cFhyQ1TFzFKsEHy8uMEDpMlAVs.*AAJTSQACMDIAAlNLABxCYm8yOVFCbFFCK2xJQlRqR0JkNll1WU5lRUk9AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
"successUrl": "/am/console",
"realm": "/"
}
2024-04-19 05:43:50,291 INFO
2024-04-19 05:43:50,292 INFO ---------------------- Running AMSTER livecheck ----------------------
2024-04-19 05:43:50,292 INFO
2024-04-19 05:43:50,292 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:43:50,292 INFO
2024-04-19 05:43:50,292 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-19 05:43:50,292 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:43:50,651 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:50,651 DEBUG --- stdout ---
2024-04-19 05:43:50,651 DEBUG amster-grkh7
2024-04-19 05:43:50,651 DEBUG --- stderr ---
2024-04-19 05:43:50,651 DEBUG
2024-04-19 05:43:51,333 INFO Amster livecheck is passed
2024-04-19 05:43:51,334 INFO
2024-04-19 05:43:51,334 INFO ------------------------ Running IDM livecheck ------------------------
2024-04-19 05:43:51,334 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping
2024-04-19 05:43:51,334 INFO
2024-04-19 05:43:51,334 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.OPENIDM_ADMIN_PASSWORD}"
2024-04-19 05:43:51,334 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-19 05:43:51,669 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:51,669 DEBUG --- stdout ---
2024-04-19 05:43:51,669 DEBUG TzlLMVZYWVZKaXlpbVNtbTRWUU54clFV
2024-04-19 05:43:51,669 DEBUG --- stderr ---
2024-04-19 05:43:51,669 DEBUG
2024-04-19 05:43:51,669 INFO Set admin password: O9K1VXYVJiyimSmm4VQNxrQU
2024-04-19 05:43:51,670 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping"
2024-04-19 05:43:51,886 INFO [http_cmd]: http status code OK
2024-04-19 05:43:51,887 DEBUG --- status code ---
2024-04-19 05:43:51,887 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:51,887 DEBUG --- http response ---
2024-04-19 05:43:51,887 DEBUG {
"_id": "",
"_rev": "",
"shortDesc": "OpenIDM ready",
"state": "ACTIVE_READY"
}
2024-04-19 05:43:51,887 INFO Create new LoginSession for user "amadmin"
2024-04-19 05:43:51,888 INFO [LoginSession] Get "access_token" token for user "amadmin"
2024-04-19 05:43:51,888 INFO [LoginSession] Obtaining new "access_token" token from server because it was not obtained yet for the user
2024-04-19 05:43:51,888 INFO [LoginSession] Get "token_id" token for user "amadmin"
2024-04-19 05:43:51,888 INFO [LoginSession] Obtaining new "token_id" token from server because it was not obtained yet for the user
2024-04-19 05:43:51,888 INFO Authenticate user amadmin via REST
2024-04-19 05:43:51,888 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
2024-04-19 05:43:52,120 INFO [http_cmd]: http status code OK
2024-04-19 05:43:52,120 DEBUG --- status code ---
2024-04-19 05:43:52,120 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:52,120 DEBUG --- http response ---
2024-04-19 05:43:52,120 DEBUG {
"tokenId": "zao0lEThVsLEzHhbZcTZD5lmRMk.*AAJTSQACMDIAAlNLABxGWlNsbWI2Vi82YUhHZGVlZ2RIeWNzWkFsbjQ9AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
"successUrl": "/am/console",
"realm": "/"
}
2024-04-19 05:43:52,121 INFO [LoginSession] Obtaining session info
2024-04-19 05:43:52,121 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/realms/root/sessions?_action=getSessionInfo"
2024-04-19 05:43:52,396 INFO [http_cmd]: http status code OK
2024-04-19 05:43:52,396 DEBUG --- status code ---
2024-04-19 05:43:52,396 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:52,396 DEBUG --- http response ---
2024-04-19 05:43:52,397 DEBUG {
"username": "amadmin",
"universalId": "id=amadmin,ou=user,ou=am-config",
"realm": "/",
"latestAccessTime": "2024-04-19T05:43:52Z",
"maxIdleExpirationTime": "2024-04-19T06:13:52Z",
"maxSessionExpirationTime": "2024-04-19T07:43:51Z",
"properties": {
"AMCtxId": "a9387508-985d-4daf-8595-10fa5ef3e565-56920539"
}
}
2024-04-19 05:43:52,403 INFO Obtain Oauth2 authz code via REST
2024-04-19 05:43:52,404 DEBUG Oauth2 authorize for access token
2024-04-19 05:43:52,404 INFO [http_cmd]: curl --request POST --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=zao0lEThVsLEzHhbZcTZD5lmRMk.*AAJTSQACMDIAAlNLABxGWlNsbWI2Vi82YUhHZGVlZ2RIeWNzWkFsbjQ9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" --data '{"decision": "Allow", "csrf": "zao0lEThVsLEzHhbZcTZD5lmRMk.*AAJTSQACMDIAAlNLABxGWlNsbWI2Vi82YUhHZGVlZ2RIeWNzWkFsbjQ9AAR0eXBlAANDVFMAAlMxAAIwMQ..*"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/authorize?client_id=smokeclient&scope=fr:idm:*&redirect_uri=https://fake.com&response_type=code&realm=/"
2024-04-19 05:43:52,676 INFO [http_cmd]: http status code OK
2024-04-19 05:43:52,676 DEBUG --- status code ---
2024-04-19 05:43:52,676 DEBUG http status code is 302 (expected 302)
2024-04-19 05:43:52,676 DEBUG --- http response ---
2024-04-19 05:43:52,677 DEBUG {'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'cache-control': 'no-store', 'location': 'https://fake.com?code=mo9yN_1ADrxmPdogTpr2DB_r6Pc&iss=https%3A%2F%2Fxlou.iam.xlou-cdm.engineeringpit.com%2Fam%2Foauth2&client_id=smokeclient', 'pragma': 'no-cache', 'set-cookie': 'OAUTH_REQUEST_ATTRIBUTES=DELETED; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/; Secure; HttpOnly; SameSite=none', 'content-length': '0', 'date': 'Fri, 19 Apr 2024 05:43:52 GMT'}
2024-04-19 05:43:52,677 DEBUG Oauth2 exchange authorize code for access token
2024-04-19 05:43:52,678 INFO [http_cmd]: curl -L --request POST --data '{"grant_type": "authorization_code", "redirect_uri": "https://fake.com", "client_id": "smokeclient", "code": "mo9yN_1ADrxmPdogTpr2DB_r6Pc"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/access_token?realm=/"
2024-04-19 05:43:52,947 INFO [http_cmd]: http status code OK
2024-04-19 05:43:52,947 DEBUG --- status code ---
2024-04-19 05:43:52,947 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:52,947 DEBUG --- http response ---
2024-04-19 05:43:52,947 DEBUG {
"access_token": "7tYitjndZXMVrniyIBA-FnqhqH0",
"scope": "fr:idm:*",
"token_type": "Bearer",
"expires_in": 3599
}
2024-04-19 05:43:52,950 INFO Additional check to sync/queue to ensure sync with DS is ok (see LODESTAR-1245)
2024-04-19 05:43:52,950 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/sync/queue?_queryFilter=true&_pageSize=1"
2024-04-19 05:43:53,181 INFO [http_cmd]: http status code OK
2024-04-19 05:43:53,182 DEBUG --- status code ---
2024-04-19 05:43:53,182 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:53,182 DEBUG --- http response ---
2024-04-19 05:43:53,182 DEBUG {
"result": [],
"resultCount": 0,
"pagedResultsCookie": null,
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}
2024-04-19 05:43:53,183 INFO Try to create, query and delete a user
2024-04-19 05:43:53,184 INFO Create user idm_fydevmwd18 in IDM
2024-04-19 05:43:53,184 INFO [http_cmd]: curl --header "Authorization: Bearer 7tYitjndZXMVrniyIBA-FnqhqH0" --header "Content-Type: application/json" --insecure -L --request POST --data '{"userName": "idm_fydevmwd18", "givenName": "Peter", "sn": "Parker", "password": "T35tr0ck123", "telephoneNumber": "6669876987", "description": "IDM Test User", "roles": [], "mail": "idm_fydevmwd18@forgerock.com"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user?_action=create"
2024-04-19 05:43:53,435 INFO [http_cmd]: http status code OK
2024-04-19 05:43:53,435 DEBUG --- status code ---
2024-04-19 05:43:53,435 DEBUG http status code is 201 (expected [200, 201])
2024-04-19 05:43:53,435 DEBUG --- http response ---
2024-04-19 05:43:53,436 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-19 05:43:53,436 DEBUG ----- output -----
2024-04-19 05:43:53,436 DEBUG {
2024-04-19 05:43:53,436 DEBUG "_id": "6f53511d-502b-457f-85fe-f7ae8f87231b",
2024-04-19 05:43:53,436 DEBUG "_rev": "34651f31-1c43-4774-828c-270d64a997a7-50633",
2024-04-19 05:43:53,436 DEBUG "country": null,
2024-04-19 05:43:53,436 DEBUG "mail": "idm_fydevmwd18@forgerock.com",
2024-04-19 05:43:53,436 DEBUG "memberOfOrgIDs": [],
2024-04-19 05:43:53,436 DEBUG "postalCode": null,
2024-04-19 05:43:53,436 DEBUG "profileImage": null,
2024-04-19 05:43:53,436 DEBUG "expireAccount": null,
2024-04-19 05:43:53,436 DEBUG "activeDate": null,
2024-04-19 05:43:53,436 DEBUG
2024-04-19 05:43:53,436 DEBUG
2024-04-19 05:43:53,436 DEBUG
2024-04-19 05:43:53,436 DEBUG [...]
2024-04-19 05:43:53,436 DEBUG
2024-04-19 05:43:53,436 DEBUG
2024-04-19 05:43:53,436 DEBUG
2024-04-19 05:43:53,436 DEBUG "aliasList": [],
2024-04-19 05:43:53,436 DEBUG "kbaInfo": [],
2024-04-19 05:43:53,436 DEBUG "inactiveDate": null,
2024-04-19 05:43:53,436 DEBUG "sn": "Parker",
2024-04-19 05:43:53,436 DEBUG "preferences": null,
2024-04-19 05:43:53,436 DEBUG "userName": "idm_fydevmwd18",
2024-04-19 05:43:53,436 DEBUG "passwordLastChangedTime": "2024-04-19T05:43:53.332Z",
2024-04-19 05:43:53,436 DEBUG "effectiveRoles": [],
2024-04-19 05:43:53,436 DEBUG "activateAccount": null
2024-04-19 05:43:53,436 DEBUG }
2024-04-19 05:43:53,436 INFO Query IDM to verify UserID 6f53511d-502b-457f-85fe-f7ae8f87231b is visible
2024-04-19 05:43:53,437 INFO [http_cmd]: curl --insecure -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/6f53511d-502b-457f-85fe-f7ae8f87231b"
2024-04-19 05:43:53,644 INFO [http_cmd]: http status code OK
2024-04-19 05:43:53,644 DEBUG --- status code ---
2024-04-19 05:43:53,644 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:53,644 DEBUG --- http response ---
2024-04-19 05:43:53,644 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-19 05:43:53,644 DEBUG ----- output -----
2024-04-19 05:43:53,644 DEBUG {
2024-04-19 05:43:53,644 DEBUG "_id": "6f53511d-502b-457f-85fe-f7ae8f87231b",
2024-04-19 05:43:53,644 DEBUG "_rev": "34651f31-1c43-4774-828c-270d64a997a7-50633",
2024-04-19 05:43:53,644 DEBUG "country": null,
2024-04-19 05:43:53,645 DEBUG "mail": "idm_fydevmwd18@forgerock.com",
2024-04-19 05:43:53,645 DEBUG "memberOfOrgIDs": [],
2024-04-19 05:43:53,645 DEBUG "postalCode": null,
2024-04-19 05:43:53,645 DEBUG "profileImage": null,
2024-04-19 05:43:53,645 DEBUG "expireAccount": null,
2024-04-19 05:43:53,645 DEBUG "activeDate": null,
2024-04-19 05:43:53,645 DEBUG
2024-04-19 05:43:53,645 DEBUG
2024-04-19 05:43:53,645 DEBUG
2024-04-19 05:43:53,645 DEBUG [...]
2024-04-19 05:43:53,645 DEBUG
2024-04-19 05:43:53,645 DEBUG
2024-04-19 05:43:53,645 DEBUG
2024-04-19 05:43:53,645 DEBUG "aliasList": [],
2024-04-19 05:43:53,645 DEBUG "kbaInfo": [],
2024-04-19 05:43:53,645 DEBUG "inactiveDate": null,
2024-04-19 05:43:53,645 DEBUG "sn": "Parker",
2024-04-19 05:43:53,645 DEBUG "preferences": null,
2024-04-19 05:43:53,645 DEBUG "userName": "idm_fydevmwd18",
2024-04-19 05:43:53,645 DEBUG "passwordLastChangedTime": "2024-04-19T05:43:53.332Z",
2024-04-19 05:43:53,645 DEBUG "effectiveRoles": [],
2024-04-19 05:43:53,645 DEBUG "activateAccount": null
2024-04-19 05:43:53,645 DEBUG }
2024-04-19 05:43:53,645 INFO Authenticate as user idm_fydevmwd18
2024-04-19 05:43:53,646 INFO [http_cmd]: curl --insecure -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/authentication?_action=login"
2024-04-19 05:43:53,849 INFO [http_cmd]: http status code OK
2024-04-19 05:43:53,849 DEBUG --- status code ---
2024-04-19 05:43:53,849 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:53,849 DEBUG --- http response ---
2024-04-19 05:43:53,849 DEBUG {
"_id": "login",
"authorization": {
"id": "anonymous",
"roles": [
"internal/role/openidm-reg"
],
"component": "internal/user"
},
"authenticationId": "anonymous"
}
2024-04-19 05:43:53,850 INFO [http_cmd]: curl --insecure -L --request DELETE "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/6f53511d-502b-457f-85fe-f7ae8f87231b"
2024-04-19 05:43:54,074 INFO [http_cmd]: http status code OK
2024-04-19 05:43:54,074 DEBUG --- status code ---
2024-04-19 05:43:54,074 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:54,074 DEBUG --- http response ---
2024-04-19 05:43:54,075 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-19 05:43:54,075 DEBUG ----- output -----
2024-04-19 05:43:54,075 DEBUG {
2024-04-19 05:43:54,075 DEBUG "_id": "6f53511d-502b-457f-85fe-f7ae8f87231b",
2024-04-19 05:43:54,075 DEBUG "_rev": "34651f31-1c43-4774-828c-270d64a997a7-50633",
2024-04-19 05:43:54,075 DEBUG "country": null,
2024-04-19 05:43:54,075 DEBUG "mail": "idm_fydevmwd18@forgerock.com",
2024-04-19 05:43:54,075 DEBUG "memberOfOrgIDs": [],
2024-04-19 05:43:54,075 DEBUG "postalCode": null,
2024-04-19 05:43:54,075 DEBUG "profileImage": null,
2024-04-19 05:43:54,075 DEBUG "expireAccount": null,
2024-04-19 05:43:54,075 DEBUG "activeDate": null,
2024-04-19 05:43:54,075 DEBUG
2024-04-19 05:43:54,075 DEBUG
2024-04-19 05:43:54,075 DEBUG
2024-04-19 05:43:54,075 DEBUG [...]
2024-04-19 05:43:54,075 DEBUG
2024-04-19 05:43:54,075 DEBUG
2024-04-19 05:43:54,075 DEBUG
2024-04-19 05:43:54,075 DEBUG "aliasList": [],
2024-04-19 05:43:54,075 DEBUG "kbaInfo": [],
2024-04-19 05:43:54,075 DEBUG "inactiveDate": null,
2024-04-19 05:43:54,075 DEBUG "sn": "Parker",
2024-04-19 05:43:54,075 DEBUG "preferences": null,
2024-04-19 05:43:54,075 DEBUG "userName": "idm_fydevmwd18",
2024-04-19 05:43:54,075 DEBUG "passwordLastChangedTime": "2024-04-19T05:43:53.332Z",
2024-04-19 05:43:54,075 DEBUG "effectiveRoles": [],
2024-04-19 05:43:54,075 DEBUG "activateAccount": null
2024-04-19 05:43:54,075 DEBUG }
2024-04-19 05:43:54,077 INFO
2024-04-19 05:43:54,077 INFO -------------------- Running END-USER-UI livecheck --------------------
2024-04-19 05:43:54,077 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/enduser
2024-04-19 05:43:54,078 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/enduser"
2024-04-19 05:43:54,353 INFO [http_cmd]: http status code OK
2024-04-19 05:43:54,354 DEBUG --- status code ---
2024-04-19 05:43:54,354 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:54,354 DEBUG --- http response ---
2024-04-19 05:43:54,354 DEBUG Identity Management
[]
2024-04-19 05:43:54,355 INFO
2024-04-19 05:43:54,355 INFO --------------------- Running LOGIN-UI livecheck ---------------------
2024-04-19 05:43:54,355 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI
2024-04-19 05:43:54,356 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI"
2024-04-19 05:43:54,630 INFO [http_cmd]: http status code OK
2024-04-19 05:43:54,630 DEBUG --- status code ---
2024-04-19 05:43:54,630 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:54,630 DEBUG --- http response ---
2024-04-19 05:43:54,630 DEBUG Login
[]
2024-04-19 05:43:54,632 INFO
2024-04-19 05:43:54,632 INFO --------------------- Running ADMIN-UI livecheck ---------------------
2024-04-19 05:43:54,632 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/platform
2024-04-19 05:43:54,632 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/platform"
2024-04-19 05:43:54,907 INFO [http_cmd]: http status code OK
2024-04-19 05:43:54,907 DEBUG --- status code ---
2024-04-19 05:43:54,907 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:54,907 DEBUG --- http response ---
2024-04-19 05:43:54,907 DEBUG Platform Admin
[]
2024-04-19 05:43:54,908 INFO LIVECHECK SUCCEEDED
2024-04-19 05:43:54,909 INFO
2024-04-19 05:43:54,909 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-19 05:43:54,909 INFO
2024-04-19 05:43:54,909 INFO --------------------- Get DS-CTS software version ---------------------
2024-04-19 05:43:54,909 INFO
2024-04-19 05:43:54,909 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- find /opt/opendj -name opendj-core.jar
2024-04-19 05:43:54,909 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-19 05:43:55,763 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:55,764 DEBUG --- stdout ---
2024-04-19 05:43:55,764 DEBUG /opt/opendj/lib/opendj-core.jar
2024-04-19 05:43:55,764 DEBUG --- stderr ---
2024-04-19 05:43:55,764 DEBUG
2024-04-19 05:43:55,764 INFO
2024-04-19 05:43:55,764 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-cts-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-cts_info/opendj-core.jar --container ds
2024-04-19 05:43:55,764 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:56,990 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:56,990 DEBUG --- stdout ---
2024-04-19 05:43:56,990 DEBUG tar: Removing leading `/' from member names
2024-04-19 05:43:56,990 DEBUG --- stderr ---
2024-04-19 05:43:56,990 DEBUG
2024-04-19 05:43:56,998 INFO
2024-04-19 05:43:56,998 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-19 05:43:56,999 INFO
2024-04-19 05:43:56,999 INFO ------------------- Get DS-IDREPO software version -------------------
2024-04-19 05:43:56,999 INFO
2024-04-19 05:43:56,999 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- find /opt/opendj -name opendj-core.jar
2024-04-19 05:43:56,999 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-19 05:43:57,825 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:57,825 DEBUG --- stdout ---
2024-04-19 05:43:57,826 DEBUG /opt/opendj/lib/opendj-core.jar
2024-04-19 05:43:57,826 DEBUG --- stderr ---
2024-04-19 05:43:57,826 DEBUG
2024-04-19 05:43:57,826 INFO
2024-04-19 05:43:57,826 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-idrepo-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-idrepo_info/opendj-core.jar --container ds
2024-04-19 05:43:57,826 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:43:59,100 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:43:59,100 DEBUG --- stdout ---
2024-04-19 05:43:59,100 DEBUG tar: Removing leading `/' from member names
2024-04-19 05:43:59,100 DEBUG --- stderr ---
2024-04-19 05:43:59,100 DEBUG
2024-04-19 05:43:59,107 INFO
2024-04-19 05:43:59,107 INFO ****************************** Initializing component pods for AM ******************************
2024-04-19 05:43:59,107 INFO
2024-04-19 05:43:59,107 INFO ----------------------- Get AM software version -----------------------
2024-04-19 05:43:59,107 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version
2024-04-19 05:43:59,108 INFO [LoginSession] Get "token_id" token for user "amadmin"
2024-04-19 05:43:59,108 DEBUG [LoginSession] re-using "token_id" token that remains valid for 29m 43s
2024-04-19 05:43:59,108 INFO [http_cmd]: curl -L --request GET --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=zao0lEThVsLEzHhbZcTZD5lmRMk.*AAJTSQACMDIAAlNLABxGWlNsbWI2Vi82YUhHZGVlZ2RIeWNzWkFsbjQ9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version"
2024-04-19 05:43:59,351 INFO [http_cmd]: http status code OK
2024-04-19 05:43:59,352 DEBUG --- status code ---
2024-04-19 05:43:59,352 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:59,352 DEBUG --- http response ---
2024-04-19 05:43:59,352 DEBUG {
"_id": "version",
"_rev": "509589602",
"version": "7.6.0-SNAPSHOT",
"fullVersion": "ForgeRock Access Management 7.6.0-SNAPSHOT Build bbf6791c961b39ef101ebad36686ee2c3cde94bb (2024-April-12 18:28)",
"revision": "bbf6791c961b39ef101ebad36686ee2c3cde94bb",
"date": "2024-April-12 18:28"
}
2024-04-19 05:43:59,354 INFO
2024-04-19 05:43:59,354 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-19 05:43:59,355 INFO
2024-04-19 05:43:59,355 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-19 05:43:59,355 INFO
2024-04-19 05:43:59,355 INFO ---------------------- Get IDM software version ----------------------
2024-04-19 05:43:59,355 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version
2024-04-19 05:43:59,355 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version"
2024-04-19 05:43:59,565 INFO [http_cmd]: http status code OK
2024-04-19 05:43:59,565 DEBUG --- status code ---
2024-04-19 05:43:59,565 DEBUG http status code is 200 (expected 200)
2024-04-19 05:43:59,565 DEBUG --- http response ---
2024-04-19 05:43:59,565 DEBUG {
"_id": "version",
"productVersion": "7.6.0-SNAPSHOT",
"productBuildDate": "20240411202127",
"productRevision": "8554e0d1c5"
}
2024-04-19 05:43:59,568 INFO
2024-04-19 05:43:59,568 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-19 05:43:59,568 INFO
2024-04-19 05:43:59,568 INFO ------------------ Get END-USER-UI software version ------------------
2024-04-19 05:43:59,568 INFO
2024-04-19 05:43:59,568 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec end-user-ui-58878d7668-z8s6j --container end-user-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-19 05:43:59,568 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-19 05:44:00,304 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:00,306 DEBUG --- stdout ---
2024-04-19 05:44:00,306 DEBUG /usr/share/nginx/html/js/chunk-vendors.0380a414.js
2024-04-19 05:44:00,306 DEBUG --- stderr ---
2024-04-19 05:44:00,306 DEBUG
2024-04-19 05:44:00,306 INFO
2024-04-19 05:44:00,307 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp end-user-ui-58878d7668-z8s6j:/usr/share/nginx/html/js/chunk-vendors.0380a414.js /tmp/end-user-ui_info/chunk-vendors.0380a414.js --container end-user-ui
2024-04-19 05:44:00,307 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:01,415 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:01,415 DEBUG --- stdout ---
2024-04-19 05:44:01,415 DEBUG tar: Removing leading `/' from member names
2024-04-19 05:44:01,415 DEBUG --- stderr ---
2024-04-19 05:44:01,415 DEBUG
2024-04-19 05:44:01,444 INFO
2024-04-19 05:44:01,444 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-19 05:44:01,444 INFO
2024-04-19 05:44:01,444 INFO -------------------- Get LOGIN-UI software version --------------------
2024-04-19 05:44:01,445 INFO
2024-04-19 05:44:01,445 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec login-ui-5fdf49fc66-dlhjd --container login-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-19 05:44:01,445 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-19 05:44:02,177 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:02,177 DEBUG --- stdout ---
2024-04-19 05:44:02,178 DEBUG /usr/share/nginx/html/js/chunk-vendors.f2fa5365.js
2024-04-19 05:44:02,178 DEBUG --- stderr ---
2024-04-19 05:44:02,178 DEBUG
2024-04-19 05:44:02,178 INFO
2024-04-19 05:44:02,178 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp login-ui-5fdf49fc66-dlhjd:/usr/share/nginx/html/js/chunk-vendors.f2fa5365.js /tmp/login-ui_info/chunk-vendors.f2fa5365.js --container login-ui
2024-04-19 05:44:02,178 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:03,362 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:03,363 DEBUG --- stdout ---
2024-04-19 05:44:03,363 DEBUG tar: Removing leading `/' from member names
2024-04-19 05:44:03,363 DEBUG --- stderr ---
2024-04-19 05:44:03,363 DEBUG
2024-04-19 05:44:03,396 INFO
2024-04-19 05:44:03,397 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-19 05:44:03,397 INFO
2024-04-19 05:44:03,397 INFO -------------------- Get ADMIN-UI software version --------------------
2024-04-19 05:44:03,398 INFO
2024-04-19 05:44:03,398 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec admin-ui-c4c7bf584-tlgkc --container admin-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-19 05:44:03,398 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-19 05:44:04,213 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:04,213 DEBUG --- stdout ---
2024-04-19 05:44:04,213 DEBUG /usr/share/nginx/html/js/chunk-vendors.b3d879ac.js
2024-04-19 05:44:04,213 DEBUG --- stderr ---
2024-04-19 05:44:04,213 DEBUG
2024-04-19 05:44:04,214 INFO
2024-04-19 05:44:04,214 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp admin-ui-c4c7bf584-tlgkc:/usr/share/nginx/html/js/chunk-vendors.b3d879ac.js /tmp/admin-ui_info/chunk-vendors.b3d879ac.js --container admin-ui
2024-04-19 05:44:04,214 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:05,417 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:05,418 DEBUG --- stdout ---
2024-04-19 05:44:05,418 DEBUG tar: Removing leading `/' from member names
2024-04-19 05:44:05,419 DEBUG --- stderr ---
2024-04-19 05:44:05,419 DEBUG
2024-04-19 05:44:05,471 INFO
2024-04-19 05:44:05,471 INFO
2024-04-19 05:44:05,471 INFO ====================================================================================================
2024-04-19 05:44:05,471 INFO ================ Admin password for DS-CTS is: DiKi4hQF6vY2sxTY34zximKtumrSd1zS ================
2024-04-19 05:44:05,471 INFO ====================================================================================================
2024-04-19 05:44:05,471 INFO
2024-04-19 05:44:05,471 INFO ====================================================================================================
2024-04-19 05:44:05,471 INFO ============== Admin password for DS-IDREPO is: DiKi4hQF6vY2sxTY34zximKtumrSd1zS ==============
2024-04-19 05:44:05,471 INFO ====================================================================================================
2024-04-19 05:44:05,471 INFO
2024-04-19 05:44:05,471 INFO ====================================================================================================
2024-04-19 05:44:05,471 INFO ====================== Admin password for AM is: uIuf0Az3Zr7y6OxjLKb1zyfc ======================
2024-04-19 05:44:05,471 INFO ====================================================================================================
2024-04-19 05:44:05,471 INFO
2024-04-19 05:44:05,471 INFO ====================================================================================================
2024-04-19 05:44:05,471 INFO ===================== Admin password for IDM is: O9K1VXYVJiyimSmm4VQNxrQU =====================
2024-04-19 05:44:05,471 INFO ====================================================================================================
2024-04-19 05:44:05,472 INFO
2024-04-19 05:44:05,472 INFO *************************************** Dumping pod list ***************************************
2024-04-19 05:44:05,472 INFO Dumping pod list to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/_pod-list.txt
2024-04-19 05:44:05,992 INFO Dumping platform-config configmap to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/_platform-config_configmap.txt
2024-04-19 05:44:06,314 INFO
2024-04-19 05:44:06,315 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-19 05:44:06,315 INFO
2024-04-19 05:44:06,315 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:44:06,315 INFO
2024-04-19 05:44:06,315 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:44:06,315 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:06,681 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:06,681 DEBUG --- stdout ---
2024-04-19 05:44:06,682 DEBUG 3
2024-04-19 05:44:06,682 DEBUG --- stderr ---
2024-04-19 05:44:06,682 DEBUG
2024-04-19 05:44:06,682 INFO
2024-04-19 05:44:06,682 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:44:06,682 INFO
2024-04-19 05:44:06,682 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name}
2024-04-19 05:44:06,682 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:44:07,121 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:07,121 DEBUG --- stdout ---
2024-04-19 05:44:07,121 DEBUG ds-cts-0 ds-cts-1 ds-cts-2
2024-04-19 05:44:07,121 DEBUG --- stderr ---
2024-04-19 05:44:07,121 DEBUG
2024-04-19 05:44:07,122 INFO
2024-04-19 05:44:07,122 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-19 05:44:07,122 INFO
2024-04-19 05:44:07,122 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:44:07,122 INFO
2024-04-19 05:44:07,122 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:44:07,122 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:07,511 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:07,511 DEBUG --- stdout ---
2024-04-19 05:44:07,511 DEBUG 3
2024-04-19 05:44:07,512 DEBUG --- stderr ---
2024-04-19 05:44:07,512 DEBUG
2024-04-19 05:44:07,512 INFO
2024-04-19 05:44:07,512 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:44:07,512 INFO
2024-04-19 05:44:07,512 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name}
2024-04-19 05:44:07,512 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:44:08,019 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:08,019 DEBUG --- stdout ---
2024-04-19 05:44:08,020 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2
2024-04-19 05:44:08,020 DEBUG --- stderr ---
2024-04-19 05:44:08,020 DEBUG
2024-04-19 05:44:08,020 INFO
2024-04-19 05:44:08,020 INFO ****************************** Initializing component pods for AM ******************************
2024-04-19 05:44:08,020 INFO
2024-04-19 05:44:08,020 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:44:08,020 INFO
2024-04-19 05:44:08,021 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:44:08,021 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:08,420 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:08,420 DEBUG --- stdout ---
2024-04-19 05:44:08,420 DEBUG 3
2024-04-19 05:44:08,420 DEBUG --- stderr ---
2024-04-19 05:44:08,420 DEBUG
2024-04-19 05:44:08,420 INFO
2024-04-19 05:44:08,420 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:44:08,421 INFO
2024-04-19 05:44:08,421 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name}
2024-04-19 05:44:08,421 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:44:08,894 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:08,894 DEBUG --- stdout ---
2024-04-19 05:44:08,894 DEBUG am-6fd554f8f7-6rhpf am-6fd554f8f7-hvvkw am-6fd554f8f7-pmj4b
2024-04-19 05:44:08,894 DEBUG --- stderr ---
2024-04-19 05:44:08,894 DEBUG
2024-04-19 05:44:08,895 INFO
2024-04-19 05:44:08,895 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-19 05:44:08,895 INFO
2024-04-19 05:44:08,895 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:44:08,895 INFO
2024-04-19 05:44:08,895 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-19 05:44:08,895 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:44:09,263 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:09,263 DEBUG --- stdout ---
2024-04-19 05:44:09,263 DEBUG amster-grkh7
2024-04-19 05:44:09,263 DEBUG --- stderr ---
2024-04-19 05:44:09,263 DEBUG
2024-04-19 05:44:09,263 INFO
2024-04-19 05:44:09,263 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-19 05:44:09,263 INFO
2024-04-19 05:44:09,263 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:44:09,264 INFO
2024-04-19 05:44:09,264 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:44:09,264 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:09,635 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:09,636 DEBUG --- stdout ---
2024-04-19 05:44:09,636 DEBUG 2
2024-04-19 05:44:09,636 DEBUG --- stderr ---
2024-04-19 05:44:09,636 DEBUG
2024-04-19 05:44:09,636 INFO
2024-04-19 05:44:09,636 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:44:09,636 INFO
2024-04-19 05:44:09,636 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name}
2024-04-19 05:44:09,636 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:44:10,021 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:10,022 DEBUG --- stdout ---
2024-04-19 05:44:10,022 DEBUG idm-b85f9d978-txpwv idm-b85f9d978-wnhxb
2024-04-19 05:44:10,022 DEBUG --- stderr ---
2024-04-19 05:44:10,022 DEBUG
2024-04-19 05:44:10,022 INFO
2024-04-19 05:44:10,022 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-19 05:44:10,022 INFO
2024-04-19 05:44:10,022 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:44:10,022 INFO
2024-04-19 05:44:10,022 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:44:10,022 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:10,326 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:10,326 DEBUG --- stdout ---
2024-04-19 05:44:10,327 DEBUG 1
2024-04-19 05:44:10,327 DEBUG --- stderr ---
2024-04-19 05:44:10,327 DEBUG
2024-04-19 05:44:10,327 INFO
2024-04-19 05:44:10,327 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:44:10,327 INFO
2024-04-19 05:44:10,327 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name}
2024-04-19 05:44:10,327 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:44:10,633 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:10,634 DEBUG --- stdout ---
2024-04-19 05:44:10,634 DEBUG end-user-ui-58878d7668-z8s6j
2024-04-19 05:44:10,634 DEBUG --- stderr ---
2024-04-19 05:44:10,634 DEBUG
2024-04-19 05:44:10,635 INFO
2024-04-19 05:44:10,635 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-19 05:44:10,635 INFO
2024-04-19 05:44:10,635 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:44:10,635 INFO
2024-04-19 05:44:10,635 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:44:10,635 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:10,930 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:10,930 DEBUG --- stdout ---
2024-04-19 05:44:10,930 DEBUG 1
2024-04-19 05:44:10,930 DEBUG --- stderr ---
2024-04-19 05:44:10,930 DEBUG
2024-04-19 05:44:10,930 INFO
2024-04-19 05:44:10,930 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:44:10,930 INFO
2024-04-19 05:44:10,930 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name}
2024-04-19 05:44:10,930 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:44:11,227 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:11,227 DEBUG --- stdout ---
2024-04-19 05:44:11,228 DEBUG login-ui-5fdf49fc66-dlhjd
2024-04-19 05:44:11,228 DEBUG --- stderr ---
2024-04-19 05:44:11,228 DEBUG
2024-04-19 05:44:11,228 INFO
2024-04-19 05:44:11,228 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-19 05:44:11,228 INFO
2024-04-19 05:44:11,228 INFO --------------------- Get expected number of pods ---------------------
2024-04-19 05:44:11,228 INFO
2024-04-19 05:44:11,228 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-19 05:44:11,228 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-19 05:44:11,525 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:11,526 DEBUG --- stdout ---
2024-04-19 05:44:11,526 DEBUG 1
2024-04-19 05:44:11,526 DEBUG --- stderr ---
2024-04-19 05:44:11,526 DEBUG
2024-04-19 05:44:11,526 INFO
2024-04-19 05:44:11,526 INFO ---------------------------- Get pod list ----------------------------
2024-04-19 05:44:11,526 INFO
2024-04-19 05:44:11,526 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name}
2024-04-19 05:44:11,526 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-19 05:44:11,825 INFO [loop_until]: OK (rc = 0)
2024-04-19 05:44:11,825 DEBUG --- stdout ---
2024-04-19 05:44:11,825 DEBUG admin-ui-c4c7bf584-tlgkc
2024-04-19 05:44:11,825 DEBUG --- stderr ---
2024-04-19 05:44:11,825 DEBUG
2024-04-19 05:44:11,826 INFO
2024-04-19 05:44:11,826 INFO *********************************** Dumping components logs ***********************************
2024-04-19 05:44:11,826 INFO
2024-04-19 05:44:11,826 INFO ----------------------- Dumping logs for DS-CTS -----------------------
2024-04-19 05:44:13,376 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-cts-0.txt
2024-04-19 05:44:13,376 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-cts-0.txt
2024-04-19 05:44:13,389 INFO Check pod logs for errors
2024-04-19 05:44:15,028 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-cts-1.txt
2024-04-19 05:44:15,028 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-cts-1.txt
2024-04-19 05:44:15,048 INFO Check pod logs for errors
2024-04-19 05:44:16,663 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-cts-2.txt
2024-04-19 05:44:16,663 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-cts-2.txt
2024-04-19 05:44:16,697 INFO Check pod logs for errors
2024-04-19 05:44:16,709 INFO
2024-04-19 05:44:16,710 INFO --------------------- Dumping logs for DS-IDREPO ---------------------
2024-04-19 05:44:18,213 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-idrepo-0.txt
2024-04-19 05:44:18,213 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-idrepo-0.txt
2024-04-19 05:44:18,223 INFO Check pod logs for errors
2024-04-19 05:44:19,734 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-idrepo-1.txt
2024-04-19 05:44:19,735 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-idrepo-1.txt
2024-04-19 05:44:19,743 INFO Check pod logs for errors
2024-04-19 05:44:21,300 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-idrepo-2.txt
2024-04-19 05:44:21,300 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/ds-idrepo-2.txt
2024-04-19 05:44:21,308 INFO Check pod logs for errors
2024-04-19 05:44:21,316 INFO
2024-04-19 05:44:21,316 INFO ------------------------- Dumping logs for AM -------------------------
2024-04-19 05:44:22,815 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/am-6fd554f8f7-6rhpf.txt
2024-04-19 05:44:22,816 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/am-6fd554f8f7-6rhpf.txt
2024-04-19 05:44:22,820 INFO Check pod logs for errors
2024-04-19 05:44:24,014 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/am-6fd554f8f7-hvvkw.txt
2024-04-19 05:44:24,015 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/am-6fd554f8f7-hvvkw.txt
2024-04-19 05:44:24,015 INFO Check pod logs for errors
2024-04-19 05:44:25,465 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/am-6fd554f8f7-pmj4b.txt
2024-04-19 05:44:25,466 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/am-6fd554f8f7-pmj4b.txt
2024-04-19 05:44:25,470 INFO Check pod logs for errors
2024-04-19 05:44:25,474 INFO
2024-04-19 05:44:25,475 INFO ----------------------- Dumping logs for AMSTER -----------------------
2024-04-19 05:44:26,585 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/amster-grkh7.txt
2024-04-19 05:44:26,586 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/amster-grkh7.txt
2024-04-19 05:44:26,586 INFO Check pod logs for errors
2024-04-19 05:44:26,586 INFO
2024-04-19 05:44:26,586 INFO ------------------------ Dumping logs for IDM ------------------------
2024-04-19 05:44:28,154 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/idm-b85f9d978-txpwv.txt
2024-04-19 05:44:28,155 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/idm-b85f9d978-txpwv.txt
2024-04-19 05:44:28,177 INFO Check pod logs for errors
2024-04-19 05:44:30,152 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/idm-b85f9d978-wnhxb.txt
2024-04-19 05:44:30,153 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/idm-b85f9d978-wnhxb.txt
2024-04-19 05:44:30,180 INFO Check pod logs for errors
2024-04-19 05:44:30,202 INFO
2024-04-19 05:44:30,202 INFO -------------------- Dumping logs for END-USER-UI --------------------
2024-04-19 05:44:31,567 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/end-user-ui-58878d7668-z8s6j.txt
2024-04-19 05:44:31,568 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/end-user-ui-58878d7668-z8s6j.txt
2024-04-19 05:44:31,579 INFO Check pod logs for errors
2024-04-19 05:44:31,589 INFO
2024-04-19 05:44:31,591 INFO ---------------------- Dumping logs for LOGIN-UI ----------------------
2024-04-19 05:44:32,936 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/login-ui-5fdf49fc66-dlhjd.txt
2024-04-19 05:44:32,937 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/login-ui-5fdf49fc66-dlhjd.txt
2024-04-19 05:44:32,944 INFO Check pod logs for errors
2024-04-19 05:44:32,953 INFO
2024-04-19 05:44:32,953 INFO ---------------------- Dumping logs for ADMIN-UI ----------------------
2024-04-19 05:44:34,205 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/admin-ui-c4c7bf584-tlgkc.txt
2024-04-19 05:44:34,206 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240419_054405-after-deployment/admin-ui-c4c7bf584-tlgkc.txt
2024-04-19 05:44:34,213 INFO Check pod logs for errors
[2024-04-19 05:44:34] - INFO: Deployment ready
________________________________________________________________________________
[2024-04-19 05:44:34] Discover_all_components post : Post method
________________________________________________________________________________
Setting result to PASS
Task has been successfully stopped