--Task--
name: Discover_all_components
enabled: True
class_name: DiscoverComponentsTask
source_name: controller
source_namespace: >default<
target_name: controller
target_namespace: >default<
start: 0.0
stop: None
timeout: no timeout
loop: False
interval: None
dependencies: []
wait_for: []
preceding_task: None
options: {}
group_name: None
Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock
________________________________________________________________________________
[2024-04-21 22:08:47] Discover_all_components pre : Initialising task parameters
________________________________________________________________________________
task will be executed on controller (localhost)
________________________________________________________________________________
[2024-04-21 22:08:47] Discover_all_components step1 : Discover components
________________________________________________________________________________
2024-04-21 22:08:49,306 INFO Deployment was skipped.
2024-04-21 22:08:57,134 INFO
2024-04-21 22:08:57,135 INFO ************************************* Creating deployment *************************************
2024-04-21 22:08:57,135 INFO Creating normal (forgeops) type deployment for deployment: stack
2024-04-21 22:08:57,304 INFO
2024-04-21 22:08:57,305 INFO ------- Custom component configuration present. Loading values -------
2024-04-21 22:08:57,306 INFO
2024-04-21 22:08:57,309 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds cts
2024-04-21 22:08:58,014 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:08:58,015 DEBUG --- stdout ---
2024-04-21 22:08:58,015 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts
2024-04-21 22:08:58,016 DEBUG --- stderr ---
2024-04-21 22:08:58,016 DEBUG
2024-04-21 22:08:58,016 INFO
2024-04-21 22:08:58,017 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-db97906b8eff882e42d79d5b86aca43562470e8d
2024-04-21 22:08:58,017 INFO
2024-04-21 22:08:58,020 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds idrepo
2024-04-21 22:08:58,537 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:08:58,538 DEBUG --- stdout ---
2024-04-21 22:08:58,538 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo
2024-04-21 22:08:58,538 DEBUG --- stderr ---
2024-04-21 22:08:58,539 DEBUG
2024-04-21 22:08:58,539 INFO
2024-04-21 22:08:58,541 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-db97906b8eff882e42d79d5b86aca43562470e8d
2024-04-21 22:08:58,541 INFO
2024-04-21 22:08:58,541 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker am
2024-04-21 22:08:59,129 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:08:59,129 DEBUG --- stdout ---
2024-04-21 22:08:59,130 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am
2024-04-21 22:08:59,130 DEBUG --- stderr ---
2024-04-21 22:08:59,130 DEBUG
2024-04-21 22:08:59,130 INFO
2024-04-21 22:08:59,130 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am/Dockerfile: gcr.io/forgerock-io/am-cdk/pit1:7.6.0-b3f32bcf2825f595a381aae2f4ddab14a92da8c1
2024-04-21 22:08:59,131 INFO
2024-04-21 22:08:59,133 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker amster
2024-04-21 22:08:59,614 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:08:59,614 DEBUG --- stdout ---
2024-04-21 22:08:59,615 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster
2024-04-21 22:08:59,615 DEBUG --- stderr ---
2024-04-21 22:08:59,615 DEBUG
2024-04-21 22:08:59,616 INFO
2024-04-21 22:08:59,618 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster/Dockerfile: gcr.io/forgerock-io/amster/pit1:7.6.0-b3f32bcf2825f595a381aae2f4ddab14a92da8c1
2024-04-21 22:08:59,618 INFO
2024-04-21 22:08:59,619 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker idm
2024-04-21 22:08:59,938 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:08:59,938 DEBUG --- stdout ---
2024-04-21 22:08:59,939 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm
2024-04-21 22:08:59,939 DEBUG --- stderr ---
2024-04-21 22:08:59,939 DEBUG
2024-04-21 22:08:59,939 INFO
2024-04-21 22:08:59,940 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm/Dockerfile: gcr.io/forgerock-io/idm-cdk/pit1:7.6.0-8554e0d1c5044f4f6c6504cd366196848874e695
2024-04-21 22:08:59,941 INFO
2024-04-21 22:08:59,942 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base end-user-ui
2024-04-21 22:09:00,179 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:09:00,180 DEBUG --- stdout ---
2024-04-21 22:09:00,180 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui
2024-04-21 22:09:00,180 DEBUG --- stderr ---
2024-04-21 22:09:00,180 DEBUG
2024-04-21 22:09:00,180 INFO
2024-04-21 22:09:00,194 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui/kustomization.yaml: gcr.io/forgerock-io/platform-enduser-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-21 22:09:00,194 INFO
2024-04-21 22:09:00,194 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base login-ui
2024-04-21 22:09:00,549 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:09:00,551 DEBUG --- stdout ---
2024-04-21 22:09:00,552 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui
2024-04-21 22:09:00,552 DEBUG --- stderr ---
2024-04-21 22:09:00,552 DEBUG
2024-04-21 22:09:00,552 INFO
2024-04-21 22:09:00,563 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui/kustomization.yaml: gcr.io/forgerock-io/platform-login-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-21 22:09:00,564 INFO
2024-04-21 22:09:00,565 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base admin-ui
2024-04-21 22:09:00,887 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:09:00,888 DEBUG --- stdout ---
2024-04-21 22:09:00,888 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui
2024-04-21 22:09:00,888 DEBUG --- stderr ---
2024-04-21 22:09:00,888 DEBUG
2024-04-21 22:09:00,888 INFO
2024-04-21 22:09:00,898 INFO Read newTag info from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui/kustomization.yaml: gcr.io/forgerock-io/platform-admin-ui/docker-build:7.6.0-SNAPSHOT-e28d3cbe3f9d1b58834dbaa041c4c5d0f2cae626
2024-04-21 22:09:00,901 INFO Loading external components from config.yaml
2024-04-21 22:09:01,412 INFO No external components defined.
2024-04-21 22:09:01,741 INFO Setting context for components to: gke_engineeringpit_us-east1-d_xlou-cdm
2024-04-21 22:09:01,741 INFO
2024-04-21 22:09:01,741 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-21 22:09:01,741 INFO
2024-04-21 22:09:01,741 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:01,741 INFO
2024-04-21 22:09:01,741 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:01,742 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:02,372 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:02,373 DEBUG --- stdout ---
2024-04-21 22:09:02,375 DEBUG 3
2024-04-21 22:09:02,375 DEBUG --- stderr ---
2024-04-21 22:09:02,375 DEBUG
2024-04-21 22:09:02,376 INFO
2024-04-21 22:09:02,377 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:09:02,378 INFO
2024-04-21 22:09:02,378 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-21 22:09:02,378 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:09:03,061 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-21 22:09:03,062 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:03,063 DEBUG --- stdout ---
2024-04-21 22:09:03,063 DEBUG ds-cts-0 ds-cts-1 ds-cts-2
2024-04-21 22:09:03,063 DEBUG --- stderr ---
2024-04-21 22:09:03,063 DEBUG
2024-04-21 22:09:03,064 INFO
2024-04-21 22:09:03,064 INFO -------------------- Check pod ds-cts-0 is running --------------------
2024-04-21 22:09:03,064 INFO
2024-04-21 22:09:03,064 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:03,065 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:03,717 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:03,718 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:03,718 DEBUG --- stdout ---
2024-04-21 22:09:03,719 DEBUG Running
2024-04-21 22:09:03,719 DEBUG --- stderr ---
2024-04-21 22:09:03,719 DEBUG
2024-04-21 22:09:03,719 INFO
2024-04-21 22:09:03,719 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:03,719 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:04,272 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:04,272 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:04,272 DEBUG --- stdout ---
2024-04-21 22:09:04,272 DEBUG true
2024-04-21 22:09:04,272 DEBUG --- stderr ---
2024-04-21 22:09:04,272 DEBUG
2024-04-21 22:09:04,273 INFO
2024-04-21 22:09:04,273 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.startTime}
2024-04-21 22:09:04,273 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:04,829 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:04,829 DEBUG --- stdout ---
2024-04-21 22:09:04,829 DEBUG 2024-04-17T18:33:21Z
2024-04-21 22:09:04,829 DEBUG --- stderr ---
2024-04-21 22:09:04,829 DEBUG
2024-04-21 22:09:04,830 INFO
2024-04-21 22:09:04,830 INFO ------------- Check pod ds-cts-0 filesystem is accessible -------------
2024-04-21 22:09:05,845 INFO [loop_until]: Function succeeded after 1s (rc=0) - expected pattern found
2024-04-21 22:09:05,845 INFO
2024-04-21 22:09:05,845 INFO ------------------ Check pod ds-cts-0 restart count ------------------
2024-04-21 22:09:05,845 INFO
2024-04-21 22:09:05,845 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:05,845 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:06,425 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:06,426 DEBUG --- stdout ---
2024-04-21 22:09:06,426 DEBUG 0
2024-04-21 22:09:06,426 DEBUG --- stderr ---
2024-04-21 22:09:06,426 DEBUG
2024-04-21 22:09:06,426 INFO Pod ds-cts-0 has been restarted 0 times.
2024-04-21 22:09:06,426 INFO
2024-04-21 22:09:06,426 INFO -------------------- Check pod ds-cts-1 is running --------------------
2024-04-21 22:09:06,426 INFO
2024-04-21 22:09:06,426 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:06,426 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:06,952 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:06,952 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:06,952 DEBUG --- stdout ---
2024-04-21 22:09:06,952 DEBUG Running
2024-04-21 22:09:06,952 DEBUG --- stderr ---
2024-04-21 22:09:06,952 DEBUG
2024-04-21 22:09:06,952 INFO
2024-04-21 22:09:06,952 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:06,952 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:07,508 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:07,509 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:07,512 DEBUG --- stdout ---
2024-04-21 22:09:07,512 DEBUG true
2024-04-21 22:09:07,512 DEBUG --- stderr ---
2024-04-21 22:09:07,512 DEBUG
2024-04-21 22:09:07,513 INFO
2024-04-21 22:09:07,513 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.startTime}
2024-04-21 22:09:07,513 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:08,044 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:08,045 DEBUG --- stdout ---
2024-04-21 22:09:08,045 DEBUG 2024-04-17T18:33:00Z
2024-04-21 22:09:08,045 DEBUG --- stderr ---
2024-04-21 22:09:08,045 DEBUG
2024-04-21 22:09:08,045 INFO
2024-04-21 22:09:08,045 INFO ------------- Check pod ds-cts-1 filesystem is accessible -------------
2024-04-21 22:09:09,045 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:09,047 INFO
2024-04-21 22:09:09,047 INFO ------------------ Check pod ds-cts-1 restart count ------------------
2024-04-21 22:09:09,048 INFO
2024-04-21 22:09:09,048 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:09,048 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:09,621 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:09,622 DEBUG --- stdout ---
2024-04-21 22:09:09,622 DEBUG 0
2024-04-21 22:09:09,622 DEBUG --- stderr ---
2024-04-21 22:09:09,624 DEBUG
2024-04-21 22:09:09,624 INFO Pod ds-cts-1 has been restarted 0 times.
2024-04-21 22:09:09,625 INFO
2024-04-21 22:09:09,625 INFO -------------------- Check pod ds-cts-2 is running --------------------
2024-04-21 22:09:09,625 INFO
2024-04-21 22:09:09,625 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:09,625 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:10,169 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:10,169 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:10,169 DEBUG --- stdout ---
2024-04-21 22:09:10,169 DEBUG Running
2024-04-21 22:09:10,169 DEBUG --- stderr ---
2024-04-21 22:09:10,169 DEBUG
2024-04-21 22:09:10,169 INFO
2024-04-21 22:09:10,169 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:10,169 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:10,768 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:10,768 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:10,768 DEBUG --- stdout ---
2024-04-21 22:09:10,768 DEBUG true
2024-04-21 22:09:10,768 DEBUG --- stderr ---
2024-04-21 22:09:10,768 DEBUG
2024-04-21 22:09:10,768 INFO
2024-04-21 22:09:10,768 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.startTime}
2024-04-21 22:09:10,768 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:11,370 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:11,370 DEBUG --- stdout ---
2024-04-21 22:09:11,370 DEBUG 2024-04-17T18:32:38Z
2024-04-21 22:09:11,370 DEBUG --- stderr ---
2024-04-21 22:09:11,370 DEBUG
2024-04-21 22:09:11,371 INFO
2024-04-21 22:09:11,372 INFO ------------- Check pod ds-cts-2 filesystem is accessible -------------
2024-04-21 22:09:12,407 INFO [loop_until]: Function succeeded after 1s (rc=0) - expected pattern found
2024-04-21 22:09:12,409 INFO
2024-04-21 22:09:12,409 INFO ------------------ Check pod ds-cts-2 restart count ------------------
2024-04-21 22:09:12,409 INFO
2024-04-21 22:09:12,410 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:12,410 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:12,988 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:12,990 DEBUG --- stdout ---
2024-04-21 22:09:12,991 DEBUG 0
2024-04-21 22:09:12,991 DEBUG --- stderr ---
2024-04-21 22:09:12,991 DEBUG
2024-04-21 22:09:12,992 INFO Pod ds-cts-2 has been restarted 0 times.
2024-04-21 22:09:12,992 INFO
2024-04-21 22:09:12,992 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-21 22:09:12,993 INFO
2024-04-21 22:09:12,993 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:12,993 INFO
2024-04-21 22:09:12,994 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:12,994 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:13,670 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:13,671 DEBUG --- stdout ---
2024-04-21 22:09:13,671 DEBUG 3
2024-04-21 22:09:13,671 DEBUG --- stderr ---
2024-04-21 22:09:13,671 DEBUG
2024-04-21 22:09:13,672 INFO
2024-04-21 22:09:13,672 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:09:13,673 INFO
2024-04-21 22:09:13,673 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-21 22:09:13,673 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:09:14,370 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-21 22:09:14,370 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:14,371 DEBUG --- stdout ---
2024-04-21 22:09:14,371 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2
2024-04-21 22:09:14,371 DEBUG --- stderr ---
2024-04-21 22:09:14,371 DEBUG
2024-04-21 22:09:14,372 INFO
2024-04-21 22:09:14,372 INFO ------------------ Check pod ds-idrepo-0 is running ------------------
2024-04-21 22:09:14,372 INFO
2024-04-21 22:09:14,372 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:14,372 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:14,929 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:14,929 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:14,929 DEBUG --- stdout ---
2024-04-21 22:09:14,929 DEBUG Running
2024-04-21 22:09:14,929 DEBUG --- stderr ---
2024-04-21 22:09:14,930 DEBUG
2024-04-21 22:09:14,930 INFO
2024-04-21 22:09:14,930 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:14,930 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:15,500 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:15,500 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:15,501 DEBUG --- stdout ---
2024-04-21 22:09:15,501 DEBUG true
2024-04-21 22:09:15,501 DEBUG --- stderr ---
2024-04-21 22:09:15,501 DEBUG
2024-04-21 22:09:15,501 INFO
2024-04-21 22:09:15,501 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.startTime}
2024-04-21 22:09:15,501 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:16,089 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:16,090 DEBUG --- stdout ---
2024-04-21 22:09:16,090 DEBUG 2024-04-17T18:34:28Z
2024-04-21 22:09:16,090 DEBUG --- stderr ---
2024-04-21 22:09:16,090 DEBUG
2024-04-21 22:09:16,091 INFO
2024-04-21 22:09:16,091 INFO ----------- Check pod ds-idrepo-0 filesystem is accessible -----------
2024-04-21 22:09:17,090 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:17,091 INFO
2024-04-21 22:09:17,091 INFO ----------------- Check pod ds-idrepo-0 restart count -----------------
2024-04-21 22:09:17,092 INFO
2024-04-21 22:09:17,092 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:17,092 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:17,661 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:17,662 DEBUG --- stdout ---
2024-04-21 22:09:17,662 DEBUG 0
2024-04-21 22:09:17,662 DEBUG --- stderr ---
2024-04-21 22:09:17,662 DEBUG
2024-04-21 22:09:17,662 INFO Pod ds-idrepo-0 has been restarted 0 times.
2024-04-21 22:09:17,663 INFO
2024-04-21 22:09:17,663 INFO ------------------ Check pod ds-idrepo-1 is running ------------------
2024-04-21 22:09:17,663 INFO
2024-04-21 22:09:17,663 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:17,663 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:18,289 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:18,292 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:18,292 DEBUG --- stdout ---
2024-04-21 22:09:18,293 DEBUG Running
2024-04-21 22:09:18,293 DEBUG --- stderr ---
2024-04-21 22:09:18,293 DEBUG
2024-04-21 22:09:18,294 INFO
2024-04-21 22:09:18,294 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:18,294 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:18,882 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:18,882 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:18,882 DEBUG --- stdout ---
2024-04-21 22:09:18,883 DEBUG true
2024-04-21 22:09:18,883 DEBUG --- stderr ---
2024-04-21 22:09:18,883 DEBUG
2024-04-21 22:09:18,884 INFO
2024-04-21 22:09:18,884 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.startTime}
2024-04-21 22:09:18,884 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:19,483 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:19,484 DEBUG --- stdout ---
2024-04-21 22:09:19,484 DEBUG 2024-04-17T18:33:37Z
2024-04-21 22:09:19,484 DEBUG --- stderr ---
2024-04-21 22:09:19,484 DEBUG
2024-04-21 22:09:19,485 INFO
2024-04-21 22:09:19,486 INFO ----------- Check pod ds-idrepo-1 filesystem is accessible -----------
2024-04-21 22:09:20,548 INFO [loop_until]: Function succeeded after 1s (rc=0) - expected pattern found
2024-04-21 22:09:20,548 INFO
2024-04-21 22:09:20,548 INFO ----------------- Check pod ds-idrepo-1 restart count -----------------
2024-04-21 22:09:20,549 INFO
2024-04-21 22:09:20,549 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:20,549 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:21,127 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:21,128 DEBUG --- stdout ---
2024-04-21 22:09:21,128 DEBUG 0
2024-04-21 22:09:21,128 DEBUG --- stderr ---
2024-04-21 22:09:21,128 DEBUG
2024-04-21 22:09:21,128 INFO Pod ds-idrepo-1 has been restarted 0 times.
2024-04-21 22:09:21,128 INFO
2024-04-21 22:09:21,128 INFO ------------------ Check pod ds-idrepo-2 is running ------------------
2024-04-21 22:09:21,129 INFO
2024-04-21 22:09:21,129 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:21,129 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:21,704 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:21,704 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:21,704 DEBUG --- stdout ---
2024-04-21 22:09:21,704 DEBUG Running
2024-04-21 22:09:21,704 DEBUG --- stderr ---
2024-04-21 22:09:21,704 DEBUG
2024-04-21 22:09:21,704 INFO
2024-04-21 22:09:21,704 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:21,704 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:22,276 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:22,277 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:22,277 DEBUG --- stdout ---
2024-04-21 22:09:22,277 DEBUG true
2024-04-21 22:09:22,277 DEBUG --- stderr ---
2024-04-21 22:09:22,277 DEBUG
2024-04-21 22:09:22,278 INFO
2024-04-21 22:09:22,278 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.startTime}
2024-04-21 22:09:22,278 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:22,903 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:22,904 DEBUG --- stdout ---
2024-04-21 22:09:22,904 DEBUG 2024-04-17T18:32:49Z
2024-04-21 22:09:22,904 DEBUG --- stderr ---
2024-04-21 22:09:22,904 DEBUG
2024-04-21 22:09:22,905 INFO
2024-04-21 22:09:22,905 INFO ----------- Check pod ds-idrepo-2 filesystem is accessible -----------
2024-04-21 22:09:23,962 INFO [loop_until]: Function succeeded after 1s (rc=0) - expected pattern found
2024-04-21 22:09:23,963 INFO
2024-04-21 22:09:23,964 INFO ----------------- Check pod ds-idrepo-2 restart count -----------------
2024-04-21 22:09:23,964 INFO
2024-04-21 22:09:23,965 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:23,966 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:24,524 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:24,524 DEBUG --- stdout ---
2024-04-21 22:09:24,524 DEBUG 0
2024-04-21 22:09:24,524 DEBUG --- stderr ---
2024-04-21 22:09:24,524 DEBUG
2024-04-21 22:09:24,524 INFO Pod ds-idrepo-2 has been restarted 0 times.
2024-04-21 22:09:24,525 INFO
2024-04-21 22:09:24,525 INFO ****************************** Initializing component pods for AM ******************************
2024-04-21 22:09:24,525 INFO
2024-04-21 22:09:24,525 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:24,525 INFO
2024-04-21 22:09:24,525 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:24,525 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:25,098 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:25,099 DEBUG --- stdout ---
2024-04-21 22:09:25,099 DEBUG 3
2024-04-21 22:09:25,099 DEBUG --- stderr ---
2024-04-21 22:09:25,099 DEBUG
2024-04-21 22:09:25,099 INFO
2024-04-21 22:09:25,099 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:09:25,100 INFO
2024-04-21 22:09:25,100 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name}` | grep 3
2024-04-21 22:09:25,100 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:09:25,725 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-21 22:09:25,725 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:25,725 DEBUG --- stdout ---
2024-04-21 22:09:25,726 DEBUG am-767f6b67b5-9zw22 am-767f6b67b5-qcqz2 am-767f6b67b5-vbltz
2024-04-21 22:09:25,726 DEBUG --- stderr ---
2024-04-21 22:09:25,726 DEBUG
2024-04-21 22:09:25,726 INFO
2024-04-21 22:09:25,726 INFO -------------- Check pod am-767f6b67b5-9zw22 is running --------------
2024-04-21 22:09:25,727 INFO
2024-04-21 22:09:25,727 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-767f6b67b5-9zw22 -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:25,727 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:26,295 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:26,296 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:26,296 DEBUG --- stdout ---
2024-04-21 22:09:26,296 DEBUG Running
2024-04-21 22:09:26,296 DEBUG --- stderr ---
2024-04-21 22:09:26,296 DEBUG
2024-04-21 22:09:26,297 INFO
2024-04-21 22:09:26,297 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-767f6b67b5-9zw22 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:26,297 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:26,830 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:26,831 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:26,831 DEBUG --- stdout ---
2024-04-21 22:09:26,831 DEBUG true
2024-04-21 22:09:26,831 DEBUG --- stderr ---
2024-04-21 22:09:26,831 DEBUG
2024-04-21 22:09:26,831 INFO
2024-04-21 22:09:26,831 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-767f6b67b5-9zw22 --output jsonpath={.status.startTime}
2024-04-21 22:09:26,831 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:27,362 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:27,362 DEBUG --- stdout ---
2024-04-21 22:09:27,362 DEBUG 2024-04-19T18:05:54Z
2024-04-21 22:09:27,362 DEBUG --- stderr ---
2024-04-21 22:09:27,362 DEBUG
2024-04-21 22:09:27,363 INFO
2024-04-21 22:09:27,363 INFO ------- Check pod am-767f6b67b5-9zw22 filesystem is accessible -------
2024-04-21 22:09:28,325 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:28,326 INFO
2024-04-21 22:09:28,326 INFO ------------- Check pod am-767f6b67b5-9zw22 restart count -------------
2024-04-21 22:09:28,327 INFO
2024-04-21 22:09:28,327 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-767f6b67b5-9zw22 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:28,327 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:28,847 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:28,848 DEBUG --- stdout ---
2024-04-21 22:09:28,848 DEBUG 0
2024-04-21 22:09:28,848 DEBUG --- stderr ---
2024-04-21 22:09:28,848 DEBUG
2024-04-21 22:09:28,848 INFO Pod am-767f6b67b5-9zw22 has been restarted 0 times.
2024-04-21 22:09:28,849 INFO
2024-04-21 22:09:28,849 INFO -------------- Check pod am-767f6b67b5-qcqz2 is running --------------
2024-04-21 22:09:28,849 INFO
2024-04-21 22:09:28,849 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-767f6b67b5-qcqz2 -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:28,849 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:29,406 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:29,406 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:29,408 DEBUG --- stdout ---
2024-04-21 22:09:29,408 DEBUG Running
2024-04-21 22:09:29,408 DEBUG --- stderr ---
2024-04-21 22:09:29,408 DEBUG
2024-04-21 22:09:29,409 INFO
2024-04-21 22:09:29,409 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-767f6b67b5-qcqz2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:29,409 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:29,981 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:29,982 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:29,982 DEBUG --- stdout ---
2024-04-21 22:09:29,982 DEBUG true
2024-04-21 22:09:29,982 DEBUG --- stderr ---
2024-04-21 22:09:29,982 DEBUG
2024-04-21 22:09:29,982 INFO
2024-04-21 22:09:29,982 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-767f6b67b5-qcqz2 --output jsonpath={.status.startTime}
2024-04-21 22:09:29,982 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:30,624 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:30,624 DEBUG --- stdout ---
2024-04-21 22:09:30,624 DEBUG 2024-04-19T18:05:54Z
2024-04-21 22:09:30,625 DEBUG --- stderr ---
2024-04-21 22:09:30,625 DEBUG
2024-04-21 22:09:30,625 INFO
2024-04-21 22:09:30,625 INFO ------- Check pod am-767f6b67b5-qcqz2 filesystem is accessible -------
2024-04-21 22:09:31,719 INFO [loop_until]: Function succeeded after 1s (rc=0) - expected pattern found
2024-04-21 22:09:31,720 INFO
2024-04-21 22:09:31,720 INFO ------------- Check pod am-767f6b67b5-qcqz2 restart count -------------
2024-04-21 22:09:31,720 INFO
2024-04-21 22:09:31,720 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-767f6b67b5-qcqz2 --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:31,720 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:32,282 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:32,282 DEBUG --- stdout ---
2024-04-21 22:09:32,283 DEBUG 0
2024-04-21 22:09:32,283 DEBUG --- stderr ---
2024-04-21 22:09:32,283 DEBUG
2024-04-21 22:09:32,283 INFO Pod am-767f6b67b5-qcqz2 has been restarted 0 times.
2024-04-21 22:09:32,283 INFO
2024-04-21 22:09:32,283 INFO -------------- Check pod am-767f6b67b5-vbltz is running --------------
2024-04-21 22:09:32,284 INFO
2024-04-21 22:09:32,284 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-767f6b67b5-vbltz -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:32,284 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:32,931 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:32,931 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:32,931 DEBUG --- stdout ---
2024-04-21 22:09:32,931 DEBUG Running
2024-04-21 22:09:32,931 DEBUG --- stderr ---
2024-04-21 22:09:32,931 DEBUG
2024-04-21 22:09:32,932 INFO
2024-04-21 22:09:32,932 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-767f6b67b5-vbltz -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:32,932 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:33,423 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:33,423 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:33,423 DEBUG --- stdout ---
2024-04-21 22:09:33,423 DEBUG true
2024-04-21 22:09:33,423 DEBUG --- stderr ---
2024-04-21 22:09:33,423 DEBUG
2024-04-21 22:09:33,424 INFO
2024-04-21 22:09:33,424 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-767f6b67b5-vbltz --output jsonpath={.status.startTime}
2024-04-21 22:09:33,424 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:33,887 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:33,887 DEBUG --- stdout ---
2024-04-21 22:09:33,888 DEBUG 2024-04-19T18:03:58Z
2024-04-21 22:09:33,888 DEBUG --- stderr ---
2024-04-21 22:09:33,888 DEBUG
2024-04-21 22:09:33,888 INFO
2024-04-21 22:09:33,888 INFO ------- Check pod am-767f6b67b5-vbltz filesystem is accessible -------
2024-04-21 22:09:34,826 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:34,827 INFO
2024-04-21 22:09:34,827 INFO ------------- Check pod am-767f6b67b5-vbltz restart count -------------
2024-04-21 22:09:34,827 INFO
2024-04-21 22:09:34,827 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-767f6b67b5-vbltz --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:34,827 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:35,297 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:35,297 DEBUG --- stdout ---
2024-04-21 22:09:35,298 DEBUG 0
2024-04-21 22:09:35,298 DEBUG --- stderr ---
2024-04-21 22:09:35,298 DEBUG
2024-04-21 22:09:35,298 INFO Pod am-767f6b67b5-vbltz has been restarted 0 times.
2024-04-21 22:09:35,298 INFO
2024-04-21 22:09:35,298 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-21 22:09:35,298 INFO
2024-04-21 22:09:35,298 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:09:35,299 INFO
2024-04-21 22:09:35,299 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-21 22:09:35,299 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:09:35,745 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:35,745 DEBUG --- stdout ---
2024-04-21 22:09:35,746 DEBUG amster-grkh7
2024-04-21 22:09:35,746 DEBUG --- stderr ---
2024-04-21 22:09:35,746 DEBUG
2024-04-21 22:09:35,746 INFO
2024-04-21 22:09:35,746 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-21 22:09:35,746 INFO
2024-04-21 22:09:35,746 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:35,746 INFO
2024-04-21 22:09:35,747 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:35,747 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:36,139 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:36,140 DEBUG --- stdout ---
2024-04-21 22:09:36,140 DEBUG 2
2024-04-21 22:09:36,140 DEBUG --- stderr ---
2024-04-21 22:09:36,140 DEBUG
2024-04-21 22:09:36,140 INFO
2024-04-21 22:09:36,140 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:09:36,141 INFO
2024-04-21 22:09:36,141 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name}` | grep 2
2024-04-21 22:09:36,141 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:09:36,550 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-21 22:09:36,550 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:36,550 DEBUG --- stdout ---
2024-04-21 22:09:36,550 DEBUG idm-b85f9d978-txpwv idm-b85f9d978-wnhxb
2024-04-21 22:09:36,550 DEBUG --- stderr ---
2024-04-21 22:09:36,550 DEBUG
2024-04-21 22:09:36,550 INFO
2024-04-21 22:09:36,550 INFO -------------- Check pod idm-b85f9d978-txpwv is running --------------
2024-04-21 22:09:36,551 INFO
2024-04-21 22:09:36,551 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-txpwv -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:36,551 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:36,930 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:36,931 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:36,931 DEBUG --- stdout ---
2024-04-21 22:09:36,931 DEBUG Running
2024-04-21 22:09:36,931 DEBUG --- stderr ---
2024-04-21 22:09:36,931 DEBUG
2024-04-21 22:09:36,931 INFO
2024-04-21 22:09:36,931 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-txpwv -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:36,931 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:37,315 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:37,316 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:37,316 DEBUG --- stdout ---
2024-04-21 22:09:37,316 DEBUG true
2024-04-21 22:09:37,316 DEBUG --- stderr ---
2024-04-21 22:09:37,316 DEBUG
2024-04-21 22:09:37,316 INFO
2024-04-21 22:09:37,316 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-txpwv --output jsonpath={.status.startTime}
2024-04-21 22:09:37,316 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:37,711 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:37,711 DEBUG --- stdout ---
2024-04-21 22:09:37,711 DEBUG 2024-04-17T16:00:45Z
2024-04-21 22:09:37,711 DEBUG --- stderr ---
2024-04-21 22:09:37,711 DEBUG
2024-04-21 22:09:37,712 INFO
2024-04-21 22:09:37,712 INFO ------- Check pod idm-b85f9d978-txpwv filesystem is accessible -------
2024-04-21 22:09:38,547 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:38,547 INFO
2024-04-21 22:09:38,547 INFO ------------- Check pod idm-b85f9d978-txpwv restart count -------------
2024-04-21 22:09:38,547 INFO
2024-04-21 22:09:38,547 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-txpwv --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:38,547 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:38,934 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:38,934 DEBUG --- stdout ---
2024-04-21 22:09:38,934 DEBUG 0
2024-04-21 22:09:38,934 DEBUG --- stderr ---
2024-04-21 22:09:38,934 DEBUG
2024-04-21 22:09:38,934 INFO Pod idm-b85f9d978-txpwv has been restarted 0 times.
2024-04-21 22:09:38,935 INFO
2024-04-21 22:09:38,935 INFO -------------- Check pod idm-b85f9d978-wnhxb is running --------------
2024-04-21 22:09:38,935 INFO
2024-04-21 22:09:38,935 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-wnhxb -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:38,935 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:39,331 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:39,332 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:39,332 DEBUG --- stdout ---
2024-04-21 22:09:39,332 DEBUG Running
2024-04-21 22:09:39,332 DEBUG --- stderr ---
2024-04-21 22:09:39,332 DEBUG
2024-04-21 22:09:39,333 INFO
2024-04-21 22:09:39,333 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-b85f9d978-wnhxb -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:39,333 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:39,722 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:39,722 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:39,723 DEBUG --- stdout ---
2024-04-21 22:09:39,723 DEBUG true
2024-04-21 22:09:39,723 DEBUG --- stderr ---
2024-04-21 22:09:39,723 DEBUG
2024-04-21 22:09:39,723 INFO
2024-04-21 22:09:39,723 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-wnhxb --output jsonpath={.status.startTime}
2024-04-21 22:09:39,723 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:40,109 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:40,110 DEBUG --- stdout ---
2024-04-21 22:09:40,110 DEBUG 2024-04-17T16:00:45Z
2024-04-21 22:09:40,111 DEBUG --- stderr ---
2024-04-21 22:09:40,111 DEBUG
2024-04-21 22:09:40,111 INFO
2024-04-21 22:09:40,112 INFO ------- Check pod idm-b85f9d978-wnhxb filesystem is accessible -------
2024-04-21 22:09:40,959 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:40,959 INFO
2024-04-21 22:09:40,959 INFO ------------- Check pod idm-b85f9d978-wnhxb restart count -------------
2024-04-21 22:09:40,960 INFO
2024-04-21 22:09:40,960 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-b85f9d978-wnhxb --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:40,960 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:41,336 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:41,336 DEBUG --- stdout ---
2024-04-21 22:09:41,336 DEBUG 0
2024-04-21 22:09:41,336 DEBUG --- stderr ---
2024-04-21 22:09:41,336 DEBUG
2024-04-21 22:09:41,336 INFO Pod idm-b85f9d978-wnhxb has been restarted 0 times.
2024-04-21 22:09:41,337 INFO
2024-04-21 22:09:41,337 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-21 22:09:41,337 INFO
2024-04-21 22:09:41,337 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:41,337 INFO
2024-04-21 22:09:41,337 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:41,337 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:41,669 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:41,670 DEBUG --- stdout ---
2024-04-21 22:09:41,670 DEBUG 1
2024-04-21 22:09:41,670 DEBUG --- stderr ---
2024-04-21 22:09:41,670 DEBUG
2024-04-21 22:09:41,670 INFO
2024-04-21 22:09:41,670 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:09:41,670 INFO
2024-04-21 22:09:41,671 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-21 22:09:41,671 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:09:41,997 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-21 22:09:41,997 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:41,997 DEBUG --- stdout ---
2024-04-21 22:09:41,997 DEBUG end-user-ui-58878d7668-z8s6j
2024-04-21 22:09:41,997 DEBUG --- stderr ---
2024-04-21 22:09:41,997 DEBUG
2024-04-21 22:09:41,997 INFO
2024-04-21 22:09:41,997 INFO ---------- Check pod end-user-ui-58878d7668-z8s6j is running ----------
2024-04-21 22:09:41,998 INFO
2024-04-21 22:09:41,998 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-58878d7668-z8s6j -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:41,998 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:42,340 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:42,340 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:42,340 DEBUG --- stdout ---
2024-04-21 22:09:42,340 DEBUG Running
2024-04-21 22:09:42,340 DEBUG --- stderr ---
2024-04-21 22:09:42,340 DEBUG
2024-04-21 22:09:42,341 INFO
2024-04-21 22:09:42,341 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-58878d7668-z8s6j -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:42,341 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:42,652 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:42,653 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:42,653 DEBUG --- stdout ---
2024-04-21 22:09:42,653 DEBUG true
2024-04-21 22:09:42,653 DEBUG --- stderr ---
2024-04-21 22:09:42,653 DEBUG
2024-04-21 22:09:42,653 INFO
2024-04-21 22:09:42,653 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-58878d7668-z8s6j --output jsonpath={.status.startTime}
2024-04-21 22:09:42,653 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:42,976 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:42,976 DEBUG --- stdout ---
2024-04-21 22:09:42,976 DEBUG 2024-04-17T16:01:55Z
2024-04-21 22:09:42,976 DEBUG --- stderr ---
2024-04-21 22:09:42,976 DEBUG
2024-04-21 22:09:42,977 INFO
2024-04-21 22:09:42,977 INFO --- Check pod end-user-ui-58878d7668-z8s6j filesystem is accessible ---
2024-04-21 22:09:43,758 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:43,759 INFO
2024-04-21 22:09:43,759 INFO -------- Check pod end-user-ui-58878d7668-z8s6j restart count --------
2024-04-21 22:09:43,759 INFO
2024-04-21 22:09:43,759 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-58878d7668-z8s6j --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:43,759 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:44,089 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:44,089 DEBUG --- stdout ---
2024-04-21 22:09:44,089 DEBUG 0
2024-04-21 22:09:44,089 DEBUG --- stderr ---
2024-04-21 22:09:44,089 DEBUG
2024-04-21 22:09:44,089 INFO Pod end-user-ui-58878d7668-z8s6j has been restarted 0 times.
2024-04-21 22:09:44,089 INFO
2024-04-21 22:09:44,090 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-21 22:09:44,090 INFO
2024-04-21 22:09:44,090 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:44,090 INFO
2024-04-21 22:09:44,090 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:44,090 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:44,420 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:44,420 DEBUG --- stdout ---
2024-04-21 22:09:44,420 DEBUG 1
2024-04-21 22:09:44,420 DEBUG --- stderr ---
2024-04-21 22:09:44,420 DEBUG
2024-04-21 22:09:44,420 INFO
2024-04-21 22:09:44,420 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:09:44,420 INFO
2024-04-21 22:09:44,421 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-21 22:09:44,421 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:09:44,751 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-21 22:09:44,751 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:44,751 DEBUG --- stdout ---
2024-04-21 22:09:44,751 DEBUG login-ui-5fdf49fc66-dlhjd
2024-04-21 22:09:44,751 DEBUG --- stderr ---
2024-04-21 22:09:44,751 DEBUG
2024-04-21 22:09:44,751 INFO
2024-04-21 22:09:44,751 INFO ----------- Check pod login-ui-5fdf49fc66-dlhjd is running -----------
2024-04-21 22:09:44,752 INFO
2024-04-21 22:09:44,752 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5fdf49fc66-dlhjd -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:44,752 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:45,091 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:45,092 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:45,092 DEBUG --- stdout ---
2024-04-21 22:09:45,092 DEBUG Running
2024-04-21 22:09:45,092 DEBUG --- stderr ---
2024-04-21 22:09:45,092 DEBUG
2024-04-21 22:09:45,092 INFO
2024-04-21 22:09:45,093 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5fdf49fc66-dlhjd -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:45,093 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:45,413 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:45,414 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:45,414 DEBUG --- stdout ---
2024-04-21 22:09:45,414 DEBUG true
2024-04-21 22:09:45,414 DEBUG --- stderr ---
2024-04-21 22:09:45,414 DEBUG
2024-04-21 22:09:45,415 INFO
2024-04-21 22:09:45,415 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5fdf49fc66-dlhjd --output jsonpath={.status.startTime}
2024-04-21 22:09:45,415 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:45,741 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:45,742 DEBUG --- stdout ---
2024-04-21 22:09:45,742 DEBUG 2024-04-17T16:01:55Z
2024-04-21 22:09:45,742 DEBUG --- stderr ---
2024-04-21 22:09:45,742 DEBUG
2024-04-21 22:09:45,742 INFO
2024-04-21 22:09:45,742 INFO ---- Check pod login-ui-5fdf49fc66-dlhjd filesystem is accessible ----
2024-04-21 22:09:46,515 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:46,515 INFO
2024-04-21 22:09:46,515 INFO ---------- Check pod login-ui-5fdf49fc66-dlhjd restart count ----------
2024-04-21 22:09:46,516 INFO
2024-04-21 22:09:46,516 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5fdf49fc66-dlhjd --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:46,516 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:46,832 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:46,832 DEBUG --- stdout ---
2024-04-21 22:09:46,832 DEBUG 0
2024-04-21 22:09:46,832 DEBUG --- stderr ---
2024-04-21 22:09:46,832 DEBUG
2024-04-21 22:09:46,832 INFO Pod login-ui-5fdf49fc66-dlhjd has been restarted 0 times.
2024-04-21 22:09:46,832 INFO
2024-04-21 22:09:46,832 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-21 22:09:46,832 INFO
2024-04-21 22:09:46,833 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:46,833 INFO
2024-04-21 22:09:46,833 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:46,833 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:47,155 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:47,155 DEBUG --- stdout ---
2024-04-21 22:09:47,155 DEBUG 1
2024-04-21 22:09:47,155 DEBUG --- stderr ---
2024-04-21 22:09:47,155 DEBUG
2024-04-21 22:09:47,155 INFO
2024-04-21 22:09:47,155 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:09:47,156 INFO
2024-04-21 22:09:47,156 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name}` | grep 1
2024-04-21 22:09:47,156 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:09:47,500 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2024-04-21 22:09:47,501 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:47,501 DEBUG --- stdout ---
2024-04-21 22:09:47,501 DEBUG admin-ui-c4c7bf584-tlgkc
2024-04-21 22:09:47,501 DEBUG --- stderr ---
2024-04-21 22:09:47,501 DEBUG
2024-04-21 22:09:47,501 INFO
2024-04-21 22:09:47,501 INFO ------------ Check pod admin-ui-c4c7bf584-tlgkc is running ------------
2024-04-21 22:09:47,501 INFO
2024-04-21 22:09:47,501 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-c4c7bf584-tlgkc -o=jsonpath={.status.phase} | grep "Running"
2024-04-21 22:09:47,501 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:47,851 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:47,851 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:47,851 DEBUG --- stdout ---
2024-04-21 22:09:47,851 DEBUG Running
2024-04-21 22:09:47,851 DEBUG --- stderr ---
2024-04-21 22:09:47,851 DEBUG
2024-04-21 22:09:47,852 INFO
2024-04-21 22:09:47,852 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-c4c7bf584-tlgkc -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2024-04-21 22:09:47,852 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2024-04-21 22:09:48,180 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:48,180 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:48,180 DEBUG --- stdout ---
2024-04-21 22:09:48,180 DEBUG true
2024-04-21 22:09:48,180 DEBUG --- stderr ---
2024-04-21 22:09:48,180 DEBUG
2024-04-21 22:09:48,180 INFO
2024-04-21 22:09:48,181 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-c4c7bf584-tlgkc --output jsonpath={.status.startTime}
2024-04-21 22:09:48,181 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:48,503 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:48,503 DEBUG --- stdout ---
2024-04-21 22:09:48,504 DEBUG 2024-04-17T16:01:54Z
2024-04-21 22:09:48,504 DEBUG --- stderr ---
2024-04-21 22:09:48,504 DEBUG
2024-04-21 22:09:48,504 INFO
2024-04-21 22:09:48,504 INFO ----- Check pod admin-ui-c4c7bf584-tlgkc filesystem is accessible -----
2024-04-21 22:09:49,280 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:49,281 INFO
2024-04-21 22:09:49,281 INFO ---------- Check pod admin-ui-c4c7bf584-tlgkc restart count ----------
2024-04-21 22:09:49,281 INFO
2024-04-21 22:09:49,281 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-c4c7bf584-tlgkc --output jsonpath={.status.containerStatuses[*].restartCount}
2024-04-21 22:09:49,281 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:49,600 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:49,601 DEBUG --- stdout ---
2024-04-21 22:09:49,601 DEBUG 0
2024-04-21 22:09:49,601 DEBUG --- stderr ---
2024-04-21 22:09:49,601 DEBUG
2024-04-21 22:09:49,601 INFO Pod admin-ui-c4c7bf584-tlgkc has been restarted 0 times.
2024-04-21 22:09:49,601 INFO
2024-04-21 22:09:49,601 INFO ***************************** Checking DS-CTS component is running *****************************
2024-04-21 22:09:49,601 INFO
2024-04-21 22:09:49,601 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:49,602 INFO
2024-04-21 22:09:49,602 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:49,602 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:49,984 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:49,985 DEBUG --- stdout ---
2024-04-21 22:09:49,985 DEBUG 3
2024-04-21 22:09:49,985 DEBUG --- stderr ---
2024-04-21 22:09:49,985 DEBUG
2024-04-21 22:09:49,985 INFO
2024-04-21 22:09:49,985 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-21 22:09:49,985 INFO
2024-04-21 22:09:49,985 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-cts --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3"
2024-04-21 22:09:49,985 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-21 22:09:50,386 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:50,386 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:50,386 DEBUG --- stdout ---
2024-04-21 22:09:50,386 DEBUG current:3 ready:3 replicas:3
2024-04-21 22:09:50,386 DEBUG --- stderr ---
2024-04-21 22:09:50,386 DEBUG
2024-04-21 22:09:50,386 INFO
2024-04-21 22:09:50,386 INFO *************************** Checking DS-IDREPO component is running ***************************
2024-04-21 22:09:50,387 INFO
2024-04-21 22:09:50,387 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:50,387 INFO
2024-04-21 22:09:50,387 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:50,387 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:50,776 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:50,776 DEBUG --- stdout ---
2024-04-21 22:09:50,776 DEBUG 3
2024-04-21 22:09:50,776 DEBUG --- stderr ---
2024-04-21 22:09:50,776 DEBUG
2024-04-21 22:09:50,776 INFO
2024-04-21 22:09:50,776 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-21 22:09:50,777 INFO
2024-04-21 22:09:50,777 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-idrepo --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3"
2024-04-21 22:09:50,777 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-21 22:09:51,188 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:51,188 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:51,188 DEBUG --- stdout ---
2024-04-21 22:09:51,188 DEBUG current:3 ready:3 replicas:3
2024-04-21 22:09:51,188 DEBUG --- stderr ---
2024-04-21 22:09:51,188 DEBUG
2024-04-21 22:09:51,188 INFO
2024-04-21 22:09:51,188 INFO ******************************* Checking AM component is running *******************************
2024-04-21 22:09:51,189 INFO
2024-04-21 22:09:51,189 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:51,189 INFO
2024-04-21 22:09:51,189 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:51,189 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:51,577 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:51,578 DEBUG --- stdout ---
2024-04-21 22:09:51,578 DEBUG 3
2024-04-21 22:09:51,578 DEBUG --- stderr ---
2024-04-21 22:09:51,578 DEBUG
2024-04-21 22:09:51,578 INFO
2024-04-21 22:09:51,578 INFO -------------- Waiting for 3 expected pod(s) to be ready --------------
2024-04-21 22:09:51,579 INFO
2024-04-21 22:09:51,579 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments am --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:3 replicas:3"
2024-04-21 22:09:51,579 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-21 22:09:52,023 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:52,023 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:52,023 DEBUG --- stdout ---
2024-04-21 22:09:52,023 DEBUG ready:3 replicas:3
2024-04-21 22:09:52,023 DEBUG --- stderr ---
2024-04-21 22:09:52,024 DEBUG
2024-04-21 22:09:52,024 INFO
2024-04-21 22:09:52,024 INFO ***************************** Checking AMSTER component is running *****************************
2024-04-21 22:09:52,024 INFO
2024-04-21 22:09:52,024 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:52,024 INFO
2024-04-21 22:09:52,024 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-21 22:09:52,024 INFO
2024-04-21 22:09:52,024 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get jobs amster --output jsonpath="{.status.succeeded}" | grep "1"
2024-04-21 22:09:52,024 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-21 22:09:52,437 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:52,438 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:52,438 DEBUG --- stdout ---
2024-04-21 22:09:52,438 DEBUG 1
2024-04-21 22:09:52,438 DEBUG --- stderr ---
2024-04-21 22:09:52,439 DEBUG
2024-04-21 22:09:52,439 INFO
2024-04-21 22:09:52,439 INFO ****************************** Checking IDM component is running ******************************
2024-04-21 22:09:52,439 INFO
2024-04-21 22:09:52,439 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:52,439 INFO
2024-04-21 22:09:52,439 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:52,439 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:52,880 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:52,880 DEBUG --- stdout ---
2024-04-21 22:09:52,881 DEBUG 2
2024-04-21 22:09:52,881 DEBUG --- stderr ---
2024-04-21 22:09:52,881 DEBUG
2024-04-21 22:09:52,881 INFO
2024-04-21 22:09:52,881 INFO -------------- Waiting for 2 expected pod(s) to be ready --------------
2024-04-21 22:09:52,881 INFO
2024-04-21 22:09:52,881 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment idm --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:2 replicas:2"
2024-04-21 22:09:52,881 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-21 22:09:53,262 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:53,262 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:53,262 DEBUG --- stdout ---
2024-04-21 22:09:53,262 DEBUG ready:2 replicas:2
2024-04-21 22:09:53,262 DEBUG --- stderr ---
2024-04-21 22:09:53,262 DEBUG
2024-04-21 22:09:53,262 INFO
2024-04-21 22:09:53,262 INFO ************************** Checking END-USER-UI component is running **************************
2024-04-21 22:09:53,262 INFO
2024-04-21 22:09:53,262 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:53,263 INFO
2024-04-21 22:09:53,263 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:53,263 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:53,587 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:53,587 DEBUG --- stdout ---
2024-04-21 22:09:53,587 DEBUG 1
2024-04-21 22:09:53,587 DEBUG --- stderr ---
2024-04-21 22:09:53,588 DEBUG
2024-04-21 22:09:53,588 INFO
2024-04-21 22:09:53,588 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-21 22:09:53,588 INFO
2024-04-21 22:09:53,588 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments end-user-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-21 22:09:53,588 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-21 22:09:53,923 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:53,923 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:53,923 DEBUG --- stdout ---
2024-04-21 22:09:53,924 DEBUG ready:1 replicas:1
2024-04-21 22:09:53,924 DEBUG --- stderr ---
2024-04-21 22:09:53,924 DEBUG
2024-04-21 22:09:53,924 INFO
2024-04-21 22:09:53,924 INFO **************************** Checking LOGIN-UI component is running ****************************
2024-04-21 22:09:53,924 INFO
2024-04-21 22:09:53,924 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:53,924 INFO
2024-04-21 22:09:53,924 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:53,924 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:54,244 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:54,244 DEBUG --- stdout ---
2024-04-21 22:09:54,244 DEBUG 1
2024-04-21 22:09:54,244 DEBUG --- stderr ---
2024-04-21 22:09:54,244 DEBUG
2024-04-21 22:09:54,244 INFO
2024-04-21 22:09:54,244 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-21 22:09:54,245 INFO
2024-04-21 22:09:54,245 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments login-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-21 22:09:54,245 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-21 22:09:54,582 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:54,582 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:54,582 DEBUG --- stdout ---
2024-04-21 22:09:54,582 DEBUG ready:1 replicas:1
2024-04-21 22:09:54,583 DEBUG --- stderr ---
2024-04-21 22:09:54,583 DEBUG
2024-04-21 22:09:54,583 INFO
2024-04-21 22:09:54,583 INFO **************************** Checking ADMIN-UI component is running ****************************
2024-04-21 22:09:54,583 INFO
2024-04-21 22:09:54,583 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:09:54,583 INFO
2024-04-21 22:09:54,583 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:09:54,583 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:09:54,899 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:54,900 DEBUG --- stdout ---
2024-04-21 22:09:54,900 DEBUG 1
2024-04-21 22:09:54,900 DEBUG --- stderr ---
2024-04-21 22:09:54,900 DEBUG
2024-04-21 22:09:54,900 INFO
2024-04-21 22:09:54,900 INFO -------------- Waiting for 1 expected pod(s) to be ready --------------
2024-04-21 22:09:54,901 INFO
2024-04-21 22:09:54,901 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments admin-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2024-04-21 22:09:54,901 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2024-04-21 22:09:55,245 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2024-04-21 22:09:55,246 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:55,246 DEBUG --- stdout ---
2024-04-21 22:09:55,246 DEBUG ready:1 replicas:1
2024-04-21 22:09:55,246 DEBUG --- stderr ---
2024-04-21 22:09:55,246 DEBUG
2024-04-21 22:09:55,246 INFO
2024-04-21 22:09:55,246 INFO ****************************** Livecheck stage: After deployment ******************************
2024-04-21 22:09:55,246 INFO
2024-04-21 22:09:55,246 INFO ---------------------- Running DS-CTS livecheck ----------------------
2024-04-21 22:09:55,246 INFO Livecheck to ds-cts-0
2024-04-21 22:09:55,247 INFO
2024-04-21 22:09:55,247 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}"
2024-04-21 22:09:55,247 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-21 22:09:55,583 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:09:55,583 DEBUG --- stdout ---
2024-04-21 22:09:55,583 DEBUG RGlLaTRoUUY2dlkyc3hUWTM0enhpbUt0dW1yU2QxelM=
2024-04-21 22:09:55,583 DEBUG --- stderr ---
2024-04-21 22:09:55,583 DEBUG
2024-04-21 22:09:55,583 INFO
2024-04-21 22:09:55,584 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-21 22:09:57,378 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:09:57,379 DEBUG --- stdout ---
2024-04-21 22:09:57,379 DEBUG dn:
alive: true
2024-04-21 22:09:57,379 DEBUG --- stderr ---
2024-04-21 22:09:57,379 DEBUG
2024-04-21 22:09:57,379 INFO
2024-04-21 22:09:57,379 INFO Livecheck to ds-cts-1
2024-04-21 22:09:57,379 INFO
2024-04-21 22:09:57,380 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-21 22:09:59,243 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:09:59,244 DEBUG --- stdout ---
2024-04-21 22:09:59,244 DEBUG dn:
alive: true
2024-04-21 22:09:59,244 DEBUG --- stderr ---
2024-04-21 22:09:59,244 DEBUG
2024-04-21 22:09:59,244 INFO
2024-04-21 22:09:59,244 INFO Livecheck to ds-cts-2
2024-04-21 22:09:59,244 INFO
2024-04-21 22:09:59,244 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-21 22:10:01,036 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:10:01,037 DEBUG --- stdout ---
2024-04-21 22:10:01,037 DEBUG dn:
alive: true
2024-04-21 22:10:01,037 DEBUG --- stderr ---
2024-04-21 22:10:01,037 DEBUG
2024-04-21 22:10:01,037 INFO
2024-04-21 22:10:01,037 INFO
2024-04-21 22:10:01,037 INFO --------------------- Running DS-IDREPO livecheck ---------------------
2024-04-21 22:10:01,037 INFO Livecheck to ds-idrepo-0
2024-04-21 22:10:01,038 INFO
2024-04-21 22:10:01,038 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}"
2024-04-21 22:10:01,038 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-21 22:10:01,354 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:01,355 DEBUG --- stdout ---
2024-04-21 22:10:01,355 DEBUG RGlLaTRoUUY2dlkyc3hUWTM0enhpbUt0dW1yU2QxelM=
2024-04-21 22:10:01,355 DEBUG --- stderr ---
2024-04-21 22:10:01,355 DEBUG
2024-04-21 22:10:01,355 INFO
2024-04-21 22:10:01,356 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-21 22:10:03,130 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:10:03,130 DEBUG --- stdout ---
2024-04-21 22:10:03,130 DEBUG dn:
alive: true
2024-04-21 22:10:03,130 DEBUG --- stderr ---
2024-04-21 22:10:03,130 DEBUG
2024-04-21 22:10:03,130 INFO
2024-04-21 22:10:03,131 INFO Livecheck to ds-idrepo-1
2024-04-21 22:10:03,131 INFO
2024-04-21 22:10:03,131 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-21 22:10:04,890 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:10:04,890 DEBUG --- stdout ---
2024-04-21 22:10:04,890 DEBUG dn:
alive: true
2024-04-21 22:10:04,890 DEBUG --- stderr ---
2024-04-21 22:10:04,891 DEBUG
2024-04-21 22:10:04,891 INFO
2024-04-21 22:10:04,891 INFO Livecheck to ds-idrepo-2
2024-04-21 22:10:04,891 INFO
2024-04-21 22:10:04,891 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive
2024-04-21 22:10:06,657 INFO [run_command]: OK (rc = 0 - expected to be in [0])
2024-04-21 22:10:06,657 DEBUG --- stdout ---
2024-04-21 22:10:06,657 DEBUG dn:
alive: true
2024-04-21 22:10:06,657 DEBUG --- stderr ---
2024-04-21 22:10:06,657 DEBUG
2024-04-21 22:10:06,657 INFO
2024-04-21 22:10:06,657 INFO
2024-04-21 22:10:06,658 INFO ------------------------ Running AM livecheck ------------------------
2024-04-21 22:10:06,658 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready
2024-04-21 22:10:06,659 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready"
2024-04-21 22:10:06,954 INFO [http_cmd]: http status code OK
2024-04-21 22:10:06,954 DEBUG --- status code ---
2024-04-21 22:10:06,954 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:06,954 DEBUG --- http response ---
2024-04-21 22:10:06,955 DEBUG
2024-04-21 22:10:06,955 INFO
2024-04-21 22:10:06,955 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.AM_PASSWORDS_AMADMIN_CLEAR}"
2024-04-21 22:10:06,955 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-21 22:10:07,308 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:07,308 DEBUG --- stdout ---
2024-04-21 22:10:07,308 DEBUG dUl1ZjBBejNacjd5Nk94akxLYjF6eWZj
2024-04-21 22:10:07,308 DEBUG --- stderr ---
2024-04-21 22:10:07,308 DEBUG
2024-04-21 22:10:07,308 INFO Authenticate user amadmin via REST
2024-04-21 22:10:07,309 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
2024-04-21 22:10:07,535 INFO [http_cmd]: http status code OK
2024-04-21 22:10:07,535 DEBUG --- status code ---
2024-04-21 22:10:07,536 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:07,536 DEBUG --- http response ---
2024-04-21 22:10:07,536 DEBUG {
"tokenId": "YSw4EFbXzB5GOMaKMo72KL-Iv2A.*AAJTSQACMDIAAlNLABxxOGFWSUhzVHRDK2drMVE2UWExdzVqaEl6cE09AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
"successUrl": "/am/console",
"realm": "/"
}
2024-04-21 22:10:07,543 INFO
2024-04-21 22:10:07,544 INFO ---------------------- Running AMSTER livecheck ----------------------
2024-04-21 22:10:07,544 INFO
2024-04-21 22:10:07,545 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:10:07,545 INFO
2024-04-21 22:10:07,545 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-21 22:10:07,545 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:10:07,937 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:07,937 DEBUG --- stdout ---
2024-04-21 22:10:07,937 DEBUG amster-grkh7
2024-04-21 22:10:07,938 DEBUG --- stderr ---
2024-04-21 22:10:07,938 DEBUG
2024-04-21 22:10:08,584 INFO Amster livecheck is passed
2024-04-21 22:10:08,586 INFO
2024-04-21 22:10:08,586 INFO ------------------------ Running IDM livecheck ------------------------
2024-04-21 22:10:08,586 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping
2024-04-21 22:10:08,587 INFO
2024-04-21 22:10:08,587 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.OPENIDM_ADMIN_PASSWORD}"
2024-04-21 22:10:08,587 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0]
2024-04-21 22:10:08,941 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:08,941 DEBUG --- stdout ---
2024-04-21 22:10:08,941 DEBUG TzlLMVZYWVZKaXlpbVNtbTRWUU54clFV
2024-04-21 22:10:08,941 DEBUG --- stderr ---
2024-04-21 22:10:08,941 DEBUG
2024-04-21 22:10:08,941 INFO Set admin password: O9K1VXYVJiyimSmm4VQNxrQU
2024-04-21 22:10:08,942 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping"
2024-04-21 22:10:09,171 INFO [http_cmd]: http status code OK
2024-04-21 22:10:09,171 DEBUG --- status code ---
2024-04-21 22:10:09,171 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:09,171 DEBUG --- http response ---
2024-04-21 22:10:09,172 DEBUG {
"_id": "",
"_rev": "",
"shortDesc": "OpenIDM ready",
"state": "ACTIVE_READY"
}
2024-04-21 22:10:09,172 INFO Create new LoginSession for user "amadmin"
2024-04-21 22:10:09,172 INFO [LoginSession] Get "access_token" token for user "amadmin"
2024-04-21 22:10:09,172 INFO [LoginSession] Obtaining new "access_token" token from server because it was not obtained yet for the user
2024-04-21 22:10:09,172 INFO [LoginSession] Get "token_id" token for user "amadmin"
2024-04-21 22:10:09,172 INFO [LoginSession] Obtaining new "token_id" token from server because it was not obtained yet for the user
2024-04-21 22:10:09,172 INFO Authenticate user amadmin via REST
2024-04-21 22:10:09,173 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
2024-04-21 22:10:09,391 INFO [http_cmd]: http status code OK
2024-04-21 22:10:09,391 DEBUG --- status code ---
2024-04-21 22:10:09,391 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:09,391 DEBUG --- http response ---
2024-04-21 22:10:09,392 DEBUG {
"tokenId": "6Bgu_Q7DYdpee25kLL83C-9eblk.*AAJTSQACMDIAAlNLABwrZkFZaEpZOGs2VEphcVlZa2JhUHV4elB4Y3c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
"successUrl": "/am/console",
"realm": "/"
}
2024-04-21 22:10:09,392 INFO [LoginSession] Obtaining session info
2024-04-21 22:10:09,393 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/realms/root/sessions?_action=getSessionInfo"
2024-04-21 22:10:09,616 INFO [http_cmd]: http status code OK
2024-04-21 22:10:09,616 DEBUG --- status code ---
2024-04-21 22:10:09,616 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:09,616 DEBUG --- http response ---
2024-04-21 22:10:09,617 DEBUG {
"username": "amadmin",
"universalId": "id=amadmin,ou=user,ou=am-config",
"realm": "/",
"latestAccessTime": "2024-04-21T22:10:09Z",
"maxIdleExpirationTime": "2024-04-21T22:13:09Z",
"maxSessionExpirationTime": "2024-04-21T22:13:08Z",
"properties": {
"AMCtxId": "38b081d1-ac52-40a8-b552-1c06acaca71e-550075092"
}
}
2024-04-21 22:10:09,622 INFO Obtain Oauth2 authz code via REST
2024-04-21 22:10:09,622 DEBUG Oauth2 authorize for access token
2024-04-21 22:10:09,623 INFO [http_cmd]: curl --request POST --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=6Bgu_Q7DYdpee25kLL83C-9eblk.*AAJTSQACMDIAAlNLABwrZkFZaEpZOGs2VEphcVlZa2JhUHV4elB4Y3c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" --data '{"decision": "Allow", "csrf": "6Bgu_Q7DYdpee25kLL83C-9eblk.*AAJTSQACMDIAAlNLABwrZkFZaEpZOGs2VEphcVlZa2JhUHV4elB4Y3c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/authorize?client_id=smokeclient&scope=fr:idm:*&redirect_uri=https://fake.com&response_type=code&realm=/"
2024-04-21 22:10:09,854 INFO [http_cmd]: http status code OK
2024-04-21 22:10:09,854 DEBUG --- status code ---
2024-04-21 22:10:09,854 DEBUG http status code is 302 (expected 302)
2024-04-21 22:10:09,854 DEBUG --- http response ---
2024-04-21 22:10:09,854 DEBUG {'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'cache-control': 'no-store', 'location': 'https://fake.com?code=o6wqp9T7S8bq3NR_oWTQzNue1YY&iss=https%3A%2F%2Fxlou.iam.xlou-cdm.engineeringpit.com%2Fam%2Foauth2&client_id=smokeclient', 'pragma': 'no-cache', 'set-cookie': 'OAUTH_REQUEST_ATTRIBUTES=DELETED; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/; Secure; HttpOnly; SameSite=none', 'content-length': '0', 'date': 'Sun, 21 Apr 2024 22:10:09 GMT'}
2024-04-21 22:10:09,855 DEBUG Oauth2 exchange authorize code for access token
2024-04-21 22:10:09,855 INFO [http_cmd]: curl -L --request POST --data '{"grant_type": "authorization_code", "redirect_uri": "https://fake.com", "client_id": "smokeclient", "code": "o6wqp9T7S8bq3NR_oWTQzNue1YY"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/access_token?realm=/"
2024-04-21 22:10:09,936 INFO [http_cmd]: http status code OK
2024-04-21 22:10:09,936 DEBUG --- status code ---
2024-04-21 22:10:09,936 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:09,936 DEBUG --- http response ---
2024-04-21 22:10:09,937 DEBUG {
"access_token": "pO9EWUAskSBYlpAlvENRkpKyB9c",
"scope": "fr:idm:*",
"token_type": "Bearer",
"expires_in": 3599
}
2024-04-21 22:10:09,938 INFO Additional check to sync/queue to ensure sync with DS is ok (see LODESTAR-1245)
2024-04-21 22:10:09,940 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/sync/queue?_queryFilter=true&_pageSize=1"
2024-04-21 22:10:10,162 INFO [http_cmd]: http status code OK
2024-04-21 22:10:10,162 DEBUG --- status code ---
2024-04-21 22:10:10,162 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:10,162 DEBUG --- http response ---
2024-04-21 22:10:10,162 DEBUG {
"result": [],
"resultCount": 0,
"pagedResultsCookie": null,
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}
2024-04-21 22:10:10,164 INFO Try to create, query and delete a user
2024-04-21 22:10:10,165 INFO Create user idm_3y9ihikz8z in IDM
2024-04-21 22:10:10,165 INFO [http_cmd]: curl --header "Authorization: Bearer pO9EWUAskSBYlpAlvENRkpKyB9c" --header "Content-Type: application/json" --insecure -L --request POST --data '{"userName": "idm_3y9ihikz8z", "givenName": "Peter", "sn": "Parker", "password": "T35tr0ck123", "telephoneNumber": "6669876987", "description": "IDM Test User", "roles": [], "mail": "idm_3y9ihikz8z@forgerock.com"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user?_action=create"
2024-04-21 22:10:10,427 INFO [http_cmd]: http status code OK
2024-04-21 22:10:10,427 DEBUG --- status code ---
2024-04-21 22:10:10,427 DEBUG http status code is 201 (expected [200, 201])
2024-04-21 22:10:10,427 DEBUG --- http response ---
2024-04-21 22:10:10,428 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-21 22:10:10,428 DEBUG ----- output -----
2024-04-21 22:10:10,428 DEBUG {
2024-04-21 22:10:10,428 DEBUG "_id": "f4019124-d136-4888-b547-5fe33c268d06",
2024-04-21 22:10:10,428 DEBUG "_rev": "34651f31-1c43-4774-828c-270d64a997a7-143535",
2024-04-21 22:10:10,428 DEBUG "country": null,
2024-04-21 22:10:10,428 DEBUG "mail": "idm_3y9ihikz8z@forgerock.com",
2024-04-21 22:10:10,428 DEBUG "memberOfOrgIDs": [],
2024-04-21 22:10:10,428 DEBUG "postalCode": null,
2024-04-21 22:10:10,428 DEBUG "profileImage": null,
2024-04-21 22:10:10,428 DEBUG "expireAccount": null,
2024-04-21 22:10:10,428 DEBUG "activeDate": null,
2024-04-21 22:10:10,428 DEBUG
2024-04-21 22:10:10,428 DEBUG
2024-04-21 22:10:10,428 DEBUG
2024-04-21 22:10:10,428 DEBUG [...]
2024-04-21 22:10:10,428 DEBUG
2024-04-21 22:10:10,428 DEBUG
2024-04-21 22:10:10,428 DEBUG
2024-04-21 22:10:10,428 DEBUG "aliasList": [],
2024-04-21 22:10:10,428 DEBUG "kbaInfo": [],
2024-04-21 22:10:10,428 DEBUG "inactiveDate": null,
2024-04-21 22:10:10,428 DEBUG "sn": "Parker",
2024-04-21 22:10:10,428 DEBUG "preferences": null,
2024-04-21 22:10:10,428 DEBUG "userName": "idm_3y9ihikz8z",
2024-04-21 22:10:10,428 DEBUG "passwordLastChangedTime": "2024-04-21T22:10:10.187Z",
2024-04-21 22:10:10,428 DEBUG "effectiveRoles": [],
2024-04-21 22:10:10,428 DEBUG "activateAccount": null
2024-04-21 22:10:10,428 DEBUG }
2024-04-21 22:10:10,429 INFO Query IDM to verify UserID f4019124-d136-4888-b547-5fe33c268d06 is visible
2024-04-21 22:10:10,429 INFO [http_cmd]: curl --insecure -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/f4019124-d136-4888-b547-5fe33c268d06"
2024-04-21 22:10:10,637 INFO [http_cmd]: http status code OK
2024-04-21 22:10:10,637 DEBUG --- status code ---
2024-04-21 22:10:10,637 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:10,637 DEBUG --- http response ---
2024-04-21 22:10:10,637 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-21 22:10:10,638 DEBUG ----- output -----
2024-04-21 22:10:10,638 DEBUG {
2024-04-21 22:10:10,638 DEBUG "_id": "f4019124-d136-4888-b547-5fe33c268d06",
2024-04-21 22:10:10,638 DEBUG "_rev": "34651f31-1c43-4774-828c-270d64a997a7-143535",
2024-04-21 22:10:10,638 DEBUG "country": null,
2024-04-21 22:10:10,638 DEBUG "mail": "idm_3y9ihikz8z@forgerock.com",
2024-04-21 22:10:10,638 DEBUG "memberOfOrgIDs": [],
2024-04-21 22:10:10,638 DEBUG "postalCode": null,
2024-04-21 22:10:10,638 DEBUG "profileImage": null,
2024-04-21 22:10:10,638 DEBUG "expireAccount": null,
2024-04-21 22:10:10,638 DEBUG "activeDate": null,
2024-04-21 22:10:10,638 DEBUG
2024-04-21 22:10:10,638 DEBUG
2024-04-21 22:10:10,638 DEBUG
2024-04-21 22:10:10,638 DEBUG [...]
2024-04-21 22:10:10,638 DEBUG
2024-04-21 22:10:10,638 DEBUG
2024-04-21 22:10:10,638 DEBUG
2024-04-21 22:10:10,638 DEBUG "aliasList": [],
2024-04-21 22:10:10,638 DEBUG "kbaInfo": [],
2024-04-21 22:10:10,638 DEBUG "inactiveDate": null,
2024-04-21 22:10:10,638 DEBUG "sn": "Parker",
2024-04-21 22:10:10,638 DEBUG "preferences": null,
2024-04-21 22:10:10,638 DEBUG "userName": "idm_3y9ihikz8z",
2024-04-21 22:10:10,638 DEBUG "passwordLastChangedTime": "2024-04-21T22:10:10.187Z",
2024-04-21 22:10:10,638 DEBUG "effectiveRoles": [],
2024-04-21 22:10:10,638 DEBUG "activateAccount": null
2024-04-21 22:10:10,638 DEBUG }
2024-04-21 22:10:10,639 INFO Authenticate as user idm_3y9ihikz8z
2024-04-21 22:10:10,639 INFO [http_cmd]: curl --insecure -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/authentication?_action=login"
2024-04-21 22:10:10,843 INFO [http_cmd]: http status code OK
2024-04-21 22:10:10,843 DEBUG --- status code ---
2024-04-21 22:10:10,843 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:10,843 DEBUG --- http response ---
2024-04-21 22:10:10,844 DEBUG {
"_id": "login",
"authorization": {
"id": "anonymous",
"roles": [
"internal/role/openidm-reg"
],
"component": "internal/user"
},
"authenticationId": "anonymous"
}
2024-04-21 22:10:10,845 INFO [http_cmd]: curl --insecure -L --request DELETE "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/f4019124-d136-4888-b547-5fe33c268d06"
2024-04-21 22:10:11,072 INFO [http_cmd]: http status code OK
2024-04-21 22:10:11,072 DEBUG --- status code ---
2024-04-21 22:10:11,072 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:11,072 DEBUG --- http response ---
2024-04-21 22:10:11,072 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines):
2024-04-21 22:10:11,073 DEBUG ----- output -----
2024-04-21 22:10:11,073 DEBUG {
2024-04-21 22:10:11,073 DEBUG "_id": "f4019124-d136-4888-b547-5fe33c268d06",
2024-04-21 22:10:11,073 DEBUG "_rev": "34651f31-1c43-4774-828c-270d64a997a7-143535",
2024-04-21 22:10:11,073 DEBUG "country": null,
2024-04-21 22:10:11,073 DEBUG "mail": "idm_3y9ihikz8z@forgerock.com",
2024-04-21 22:10:11,073 DEBUG "memberOfOrgIDs": [],
2024-04-21 22:10:11,073 DEBUG "postalCode": null,
2024-04-21 22:10:11,073 DEBUG "profileImage": null,
2024-04-21 22:10:11,073 DEBUG "expireAccount": null,
2024-04-21 22:10:11,073 DEBUG "activeDate": null,
2024-04-21 22:10:11,073 DEBUG
2024-04-21 22:10:11,073 DEBUG
2024-04-21 22:10:11,073 DEBUG
2024-04-21 22:10:11,073 DEBUG [...]
2024-04-21 22:10:11,073 DEBUG
2024-04-21 22:10:11,073 DEBUG
2024-04-21 22:10:11,073 DEBUG
2024-04-21 22:10:11,073 DEBUG "aliasList": [],
2024-04-21 22:10:11,073 DEBUG "kbaInfo": [],
2024-04-21 22:10:11,073 DEBUG "inactiveDate": null,
2024-04-21 22:10:11,073 DEBUG "sn": "Parker",
2024-04-21 22:10:11,073 DEBUG "preferences": null,
2024-04-21 22:10:11,073 DEBUG "userName": "idm_3y9ihikz8z",
2024-04-21 22:10:11,073 DEBUG "passwordLastChangedTime": "2024-04-21T22:10:10.187Z",
2024-04-21 22:10:11,073 DEBUG "effectiveRoles": [],
2024-04-21 22:10:11,073 DEBUG "activateAccount": null
2024-04-21 22:10:11,073 DEBUG }
2024-04-21 22:10:11,076 INFO
2024-04-21 22:10:11,076 INFO -------------------- Running END-USER-UI livecheck --------------------
2024-04-21 22:10:11,076 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/enduser
2024-04-21 22:10:11,076 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/enduser"
2024-04-21 22:10:11,355 INFO [http_cmd]: http status code OK
2024-04-21 22:10:11,355 DEBUG --- status code ---
2024-04-21 22:10:11,355 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:11,355 DEBUG --- http response ---
2024-04-21 22:10:11,355 DEBUG Identity Management
[]
2024-04-21 22:10:11,357 INFO
2024-04-21 22:10:11,358 INFO --------------------- Running LOGIN-UI livecheck ---------------------
2024-04-21 22:10:11,358 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI
2024-04-21 22:10:11,358 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI"
2024-04-21 22:10:11,636 INFO [http_cmd]: http status code OK
2024-04-21 22:10:11,636 DEBUG --- status code ---
2024-04-21 22:10:11,636 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:11,636 DEBUG --- http response ---
2024-04-21 22:10:11,636 DEBUG Login
[]
2024-04-21 22:10:11,637 INFO
2024-04-21 22:10:11,638 INFO --------------------- Running ADMIN-UI livecheck ---------------------
2024-04-21 22:10:11,638 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/platform
2024-04-21 22:10:11,638 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/platform"
2024-04-21 22:10:11,916 INFO [http_cmd]: http status code OK
2024-04-21 22:10:11,916 DEBUG --- status code ---
2024-04-21 22:10:11,916 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:11,916 DEBUG --- http response ---
2024-04-21 22:10:11,917 DEBUG Platform Admin
[]
2024-04-21 22:10:11,918 INFO LIVECHECK SUCCEEDED
2024-04-21 22:10:11,919 INFO
2024-04-21 22:10:11,919 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-21 22:10:11,919 INFO
2024-04-21 22:10:11,919 INFO --------------------- Get DS-CTS software version ---------------------
2024-04-21 22:10:11,919 INFO
2024-04-21 22:10:11,919 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- find /opt/opendj -name opendj-core.jar
2024-04-21 22:10:11,920 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-21 22:10:12,744 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:12,744 DEBUG --- stdout ---
2024-04-21 22:10:12,744 DEBUG /opt/opendj/lib/opendj-core.jar
2024-04-21 22:10:12,745 DEBUG --- stderr ---
2024-04-21 22:10:12,745 DEBUG
2024-04-21 22:10:12,745 INFO
2024-04-21 22:10:12,745 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-cts-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-cts_info/opendj-core.jar --container ds
2024-04-21 22:10:12,745 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:14,034 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:14,036 DEBUG --- stdout ---
2024-04-21 22:10:14,036 DEBUG tar: Removing leading `/' from member names
2024-04-21 22:10:14,036 DEBUG --- stderr ---
2024-04-21 22:10:14,036 DEBUG
2024-04-21 22:10:14,049 INFO
2024-04-21 22:10:14,049 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-21 22:10:14,049 INFO
2024-04-21 22:10:14,049 INFO ------------------- Get DS-IDREPO software version -------------------
2024-04-21 22:10:14,049 INFO
2024-04-21 22:10:14,049 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- find /opt/opendj -name opendj-core.jar
2024-04-21 22:10:14,049 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-21 22:10:14,861 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:14,861 DEBUG --- stdout ---
2024-04-21 22:10:14,861 DEBUG /opt/opendj/lib/opendj-core.jar
2024-04-21 22:10:14,861 DEBUG --- stderr ---
2024-04-21 22:10:14,861 DEBUG
2024-04-21 22:10:14,862 INFO
2024-04-21 22:10:14,862 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-idrepo-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-idrepo_info/opendj-core.jar --container ds
2024-04-21 22:10:14,862 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:16,109 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:16,109 DEBUG --- stdout ---
2024-04-21 22:10:16,109 DEBUG tar: Removing leading `/' from member names
2024-04-21 22:10:16,109 DEBUG --- stderr ---
2024-04-21 22:10:16,109 DEBUG
2024-04-21 22:10:16,118 INFO
2024-04-21 22:10:16,118 INFO ****************************** Initializing component pods for AM ******************************
2024-04-21 22:10:16,118 INFO
2024-04-21 22:10:16,118 INFO ----------------------- Get AM software version -----------------------
2024-04-21 22:10:16,119 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version
2024-04-21 22:10:16,119 INFO [LoginSession] Get "token_id" token for user "amadmin"
2024-04-21 22:10:16,119 DEBUG [LoginSession] re-using "token_id" token that remains valid for 2m 42s
2024-04-21 22:10:16,120 INFO [http_cmd]: curl -L --request GET --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=6Bgu_Q7DYdpee25kLL83C-9eblk.*AAJTSQACMDIAAlNLABwrZkFZaEpZOGs2VEphcVlZa2JhUHV4elB4Y3c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version"
2024-04-21 22:10:16,335 INFO [http_cmd]: http status code OK
2024-04-21 22:10:16,335 DEBUG --- status code ---
2024-04-21 22:10:16,335 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:16,335 DEBUG --- http response ---
2024-04-21 22:10:16,335 DEBUG {
"_id": "version",
"_rev": "-1298450700",
"version": "7.6.0-SNAPSHOT",
"fullVersion": "ForgeRock Access Management 7.6.0-SNAPSHOT Build b3f32bcf2825f595a381aae2f4ddab14a92da8c1 (2024-April-16 16:19)",
"revision": "b3f32bcf2825f595a381aae2f4ddab14a92da8c1",
"date": "2024-April-16 16:19"
}
2024-04-21 22:10:16,338 INFO
2024-04-21 22:10:16,339 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-21 22:10:16,339 INFO
2024-04-21 22:10:16,339 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-21 22:10:16,339 INFO
2024-04-21 22:10:16,339 INFO ---------------------- Get IDM software version ----------------------
2024-04-21 22:10:16,339 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version
2024-04-21 22:10:16,340 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version"
2024-04-21 22:10:16,551 INFO [http_cmd]: http status code OK
2024-04-21 22:10:16,551 DEBUG --- status code ---
2024-04-21 22:10:16,551 DEBUG http status code is 200 (expected 200)
2024-04-21 22:10:16,551 DEBUG --- http response ---
2024-04-21 22:10:16,551 DEBUG {
"_id": "version",
"productVersion": "7.6.0-SNAPSHOT",
"productBuildDate": "20240411202127",
"productRevision": "8554e0d1c5"
}
2024-04-21 22:10:16,554 INFO
2024-04-21 22:10:16,554 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-21 22:10:16,555 INFO
2024-04-21 22:10:16,555 INFO ------------------ Get END-USER-UI software version ------------------
2024-04-21 22:10:16,555 INFO
2024-04-21 22:10:16,555 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec end-user-ui-58878d7668-z8s6j --container end-user-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-21 22:10:16,555 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-21 22:10:17,314 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:17,314 DEBUG --- stdout ---
2024-04-21 22:10:17,314 DEBUG /usr/share/nginx/html/js/chunk-vendors.0380a414.js
2024-04-21 22:10:17,314 DEBUG --- stderr ---
2024-04-21 22:10:17,314 DEBUG
2024-04-21 22:10:17,315 INFO
2024-04-21 22:10:17,315 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp end-user-ui-58878d7668-z8s6j:/usr/share/nginx/html/js/chunk-vendors.0380a414.js /tmp/end-user-ui_info/chunk-vendors.0380a414.js --container end-user-ui
2024-04-21 22:10:17,315 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:18,456 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:18,457 DEBUG --- stdout ---
2024-04-21 22:10:18,457 DEBUG tar: Removing leading `/' from member names
2024-04-21 22:10:18,457 DEBUG --- stderr ---
2024-04-21 22:10:18,457 DEBUG
2024-04-21 22:10:18,492 INFO
2024-04-21 22:10:18,492 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-21 22:10:18,492 INFO
2024-04-21 22:10:18,492 INFO -------------------- Get LOGIN-UI software version --------------------
2024-04-21 22:10:18,493 INFO
2024-04-21 22:10:18,493 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec login-ui-5fdf49fc66-dlhjd --container login-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-21 22:10:18,493 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-21 22:10:19,257 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:19,259 DEBUG --- stdout ---
2024-04-21 22:10:19,259 DEBUG /usr/share/nginx/html/js/chunk-vendors.f2fa5365.js
2024-04-21 22:10:19,259 DEBUG --- stderr ---
2024-04-21 22:10:19,259 DEBUG
2024-04-21 22:10:19,260 INFO
2024-04-21 22:10:19,260 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp login-ui-5fdf49fc66-dlhjd:/usr/share/nginx/html/js/chunk-vendors.f2fa5365.js /tmp/login-ui_info/chunk-vendors.f2fa5365.js --container login-ui
2024-04-21 22:10:19,260 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:20,404 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:20,404 DEBUG --- stdout ---
2024-04-21 22:10:20,404 DEBUG tar: Removing leading `/' from member names
2024-04-21 22:10:20,404 DEBUG --- stderr ---
2024-04-21 22:10:20,404 DEBUG
2024-04-21 22:10:20,426 INFO
2024-04-21 22:10:20,427 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-21 22:10:20,427 INFO
2024-04-21 22:10:20,427 INFO -------------------- Get ADMIN-UI software version --------------------
2024-04-21 22:10:20,427 INFO
2024-04-21 22:10:20,427 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec admin-ui-c4c7bf584-tlgkc --container admin-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js
2024-04-21 22:10:20,427 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0]
2024-04-21 22:10:21,191 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:21,192 DEBUG --- stdout ---
2024-04-21 22:10:21,192 DEBUG /usr/share/nginx/html/js/chunk-vendors.b3d879ac.js
2024-04-21 22:10:21,192 DEBUG --- stderr ---
2024-04-21 22:10:21,192 DEBUG
2024-04-21 22:10:21,192 INFO
2024-04-21 22:10:21,192 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp admin-ui-c4c7bf584-tlgkc:/usr/share/nginx/html/js/chunk-vendors.b3d879ac.js /tmp/admin-ui_info/chunk-vendors.b3d879ac.js --container admin-ui
2024-04-21 22:10:21,192 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:22,386 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:22,387 DEBUG --- stdout ---
2024-04-21 22:10:22,387 DEBUG tar: Removing leading `/' from member names
2024-04-21 22:10:22,387 DEBUG --- stderr ---
2024-04-21 22:10:22,387 DEBUG
2024-04-21 22:10:22,444 INFO
2024-04-21 22:10:22,444 INFO
2024-04-21 22:10:22,444 INFO ====================================================================================================
2024-04-21 22:10:22,444 INFO ================ Admin password for DS-CTS is: DiKi4hQF6vY2sxTY34zximKtumrSd1zS ================
2024-04-21 22:10:22,444 INFO ====================================================================================================
2024-04-21 22:10:22,444 INFO
2024-04-21 22:10:22,444 INFO ====================================================================================================
2024-04-21 22:10:22,444 INFO ============== Admin password for DS-IDREPO is: DiKi4hQF6vY2sxTY34zximKtumrSd1zS ==============
2024-04-21 22:10:22,444 INFO ====================================================================================================
2024-04-21 22:10:22,444 INFO
2024-04-21 22:10:22,444 INFO ====================================================================================================
2024-04-21 22:10:22,444 INFO ====================== Admin password for AM is: uIuf0Az3Zr7y6OxjLKb1zyfc ======================
2024-04-21 22:10:22,444 INFO ====================================================================================================
2024-04-21 22:10:22,444 INFO
2024-04-21 22:10:22,444 INFO ====================================================================================================
2024-04-21 22:10:22,444 INFO ===================== Admin password for IDM is: O9K1VXYVJiyimSmm4VQNxrQU =====================
2024-04-21 22:10:22,445 INFO ====================================================================================================
2024-04-21 22:10:22,445 INFO
2024-04-21 22:10:22,445 INFO *************************************** Dumping pod list ***************************************
2024-04-21 22:10:22,445 INFO Dumping pod list to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/_pod-list.txt
2024-04-21 22:10:22,973 INFO Dumping platform-config configmap to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/_platform-config_configmap.txt
2024-04-21 22:10:23,317 INFO
2024-04-21 22:10:23,318 INFO **************************** Initializing component pods for DS-CTS ****************************
2024-04-21 22:10:23,318 INFO
2024-04-21 22:10:23,318 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:10:23,318 INFO
2024-04-21 22:10:23,318 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:10:23,318 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:23,706 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:23,706 DEBUG --- stdout ---
2024-04-21 22:10:23,706 DEBUG 3
2024-04-21 22:10:23,706 DEBUG --- stderr ---
2024-04-21 22:10:23,706 DEBUG
2024-04-21 22:10:23,706 INFO
2024-04-21 22:10:23,707 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:10:23,707 INFO
2024-04-21 22:10:23,707 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name}
2024-04-21 22:10:23,707 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:10:24,171 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:24,171 DEBUG --- stdout ---
2024-04-21 22:10:24,171 DEBUG ds-cts-0 ds-cts-1 ds-cts-2
2024-04-21 22:10:24,171 DEBUG --- stderr ---
2024-04-21 22:10:24,171 DEBUG
2024-04-21 22:10:24,171 INFO
2024-04-21 22:10:24,171 INFO ************************** Initializing component pods for DS-IDREPO **************************
2024-04-21 22:10:24,171 INFO
2024-04-21 22:10:24,171 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:10:24,172 INFO
2024-04-21 22:10:24,172 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:10:24,172 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:24,560 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:24,561 DEBUG --- stdout ---
2024-04-21 22:10:24,562 DEBUG 3
2024-04-21 22:10:24,562 DEBUG --- stderr ---
2024-04-21 22:10:24,562 DEBUG
2024-04-21 22:10:24,562 INFO
2024-04-21 22:10:24,563 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:10:24,563 INFO
2024-04-21 22:10:24,563 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name}
2024-04-21 22:10:24,563 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:10:25,024 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:25,024 DEBUG --- stdout ---
2024-04-21 22:10:25,024 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2
2024-04-21 22:10:25,024 DEBUG --- stderr ---
2024-04-21 22:10:25,024 DEBUG
2024-04-21 22:10:25,024 INFO
2024-04-21 22:10:25,024 INFO ****************************** Initializing component pods for AM ******************************
2024-04-21 22:10:25,024 INFO
2024-04-21 22:10:25,024 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:10:25,025 INFO
2024-04-21 22:10:25,025 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:10:25,025 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:25,410 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:25,410 DEBUG --- stdout ---
2024-04-21 22:10:25,410 DEBUG 3
2024-04-21 22:10:25,410 DEBUG --- stderr ---
2024-04-21 22:10:25,410 DEBUG
2024-04-21 22:10:25,410 INFO
2024-04-21 22:10:25,410 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:10:25,411 INFO
2024-04-21 22:10:25,411 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name}
2024-04-21 22:10:25,411 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:10:25,873 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:25,873 DEBUG --- stdout ---
2024-04-21 22:10:25,873 DEBUG am-767f6b67b5-9zw22 am-767f6b67b5-qcqz2 am-767f6b67b5-vbltz
2024-04-21 22:10:25,873 DEBUG --- stderr ---
2024-04-21 22:10:25,873 DEBUG
2024-04-21 22:10:25,873 INFO
2024-04-21 22:10:25,873 INFO **************************** Initializing component pods for AMSTER ****************************
2024-04-21 22:10:25,873 INFO
2024-04-21 22:10:25,873 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:10:25,874 INFO
2024-04-21 22:10:25,874 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=amster --output jsonpath={.items[*].metadata.name}
2024-04-21 22:10:25,874 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:10:26,310 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:26,310 DEBUG --- stdout ---
2024-04-21 22:10:26,310 DEBUG amster-grkh7
2024-04-21 22:10:26,311 DEBUG --- stderr ---
2024-04-21 22:10:26,311 DEBUG
2024-04-21 22:10:26,311 INFO
2024-04-21 22:10:26,311 INFO ***************************** Initializing component pods for IDM *****************************
2024-04-21 22:10:26,311 INFO
2024-04-21 22:10:26,311 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:10:26,311 INFO
2024-04-21 22:10:26,311 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:10:26,311 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:26,740 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:26,740 DEBUG --- stdout ---
2024-04-21 22:10:26,741 DEBUG 2
2024-04-21 22:10:26,741 DEBUG --- stderr ---
2024-04-21 22:10:26,741 DEBUG
2024-04-21 22:10:26,741 INFO
2024-04-21 22:10:26,741 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:10:26,741 INFO
2024-04-21 22:10:26,741 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name}
2024-04-21 22:10:26,741 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:10:27,139 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:27,139 DEBUG --- stdout ---
2024-04-21 22:10:27,139 DEBUG idm-b85f9d978-txpwv idm-b85f9d978-wnhxb
2024-04-21 22:10:27,139 DEBUG --- stderr ---
2024-04-21 22:10:27,139 DEBUG
2024-04-21 22:10:27,139 INFO
2024-04-21 22:10:27,139 INFO ************************* Initializing component pods for END-USER-UI *************************
2024-04-21 22:10:27,139 INFO
2024-04-21 22:10:27,140 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:10:27,140 INFO
2024-04-21 22:10:27,140 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:10:27,140 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:27,460 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:27,461 DEBUG --- stdout ---
2024-04-21 22:10:27,461 DEBUG 1
2024-04-21 22:10:27,461 DEBUG --- stderr ---
2024-04-21 22:10:27,461 DEBUG
2024-04-21 22:10:27,461 INFO
2024-04-21 22:10:27,461 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:10:27,461 INFO
2024-04-21 22:10:27,461 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name}
2024-04-21 22:10:27,461 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:10:27,781 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:27,781 DEBUG --- stdout ---
2024-04-21 22:10:27,782 DEBUG end-user-ui-58878d7668-z8s6j
2024-04-21 22:10:27,782 DEBUG --- stderr ---
2024-04-21 22:10:27,782 DEBUG
2024-04-21 22:10:27,782 INFO
2024-04-21 22:10:27,782 INFO *************************** Initializing component pods for LOGIN-UI ***************************
2024-04-21 22:10:27,782 INFO
2024-04-21 22:10:27,782 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:10:27,782 INFO
2024-04-21 22:10:27,782 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:10:27,782 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:28,114 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:28,114 DEBUG --- stdout ---
2024-04-21 22:10:28,114 DEBUG 1
2024-04-21 22:10:28,114 DEBUG --- stderr ---
2024-04-21 22:10:28,114 DEBUG
2024-04-21 22:10:28,114 INFO
2024-04-21 22:10:28,114 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:10:28,114 INFO
2024-04-21 22:10:28,114 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name}
2024-04-21 22:10:28,115 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:10:28,435 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:28,436 DEBUG --- stdout ---
2024-04-21 22:10:28,436 DEBUG login-ui-5fdf49fc66-dlhjd
2024-04-21 22:10:28,436 DEBUG --- stderr ---
2024-04-21 22:10:28,436 DEBUG
2024-04-21 22:10:28,436 INFO
2024-04-21 22:10:28,436 INFO *************************** Initializing component pods for ADMIN-UI ***************************
2024-04-21 22:10:28,436 INFO
2024-04-21 22:10:28,436 INFO --------------------- Get expected number of pods ---------------------
2024-04-21 22:10:28,436 INFO
2024-04-21 22:10:28,436 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas}
2024-04-21 22:10:28,436 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2024-04-21 22:10:28,778 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:28,778 DEBUG --- stdout ---
2024-04-21 22:10:28,779 DEBUG 1
2024-04-21 22:10:28,779 DEBUG --- stderr ---
2024-04-21 22:10:28,779 DEBUG
2024-04-21 22:10:28,779 INFO
2024-04-21 22:10:28,779 INFO ---------------------------- Get pod list ----------------------------
2024-04-21 22:10:28,779 INFO
2024-04-21 22:10:28,779 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name}
2024-04-21 22:10:28,779 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0]
2024-04-21 22:10:29,103 INFO [loop_until]: OK (rc = 0)
2024-04-21 22:10:29,103 DEBUG --- stdout ---
2024-04-21 22:10:29,103 DEBUG admin-ui-c4c7bf584-tlgkc
2024-04-21 22:10:29,103 DEBUG --- stderr ---
2024-04-21 22:10:29,103 DEBUG
2024-04-21 22:10:29,104 INFO
2024-04-21 22:10:29,104 INFO *********************************** Dumping components logs ***********************************
2024-04-21 22:10:29,104 INFO
2024-04-21 22:10:29,104 INFO ----------------------- Dumping logs for DS-CTS -----------------------
2024-04-21 22:10:30,641 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-cts-0.txt
2024-04-21 22:10:30,642 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-cts-0.txt
2024-04-21 22:10:30,651 INFO Check pod logs for errors
2024-04-21 22:10:32,109 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-cts-1.txt
2024-04-21 22:10:32,110 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-cts-1.txt
2024-04-21 22:10:32,114 INFO Check pod logs for errors
2024-04-21 22:10:33,599 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-cts-2.txt
2024-04-21 22:10:33,600 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-cts-2.txt
2024-04-21 22:10:33,609 INFO Check pod logs for errors
2024-04-21 22:10:33,617 INFO
2024-04-21 22:10:33,617 INFO --------------------- Dumping logs for DS-IDREPO ---------------------
2024-04-21 22:10:35,286 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-idrepo-0.txt
2024-04-21 22:10:35,295 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-idrepo-0.txt
2024-04-21 22:10:35,368 INFO Check pod logs for errors
2024-04-21 22:10:37,157 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-idrepo-1.txt
2024-04-21 22:10:37,158 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-idrepo-1.txt
2024-04-21 22:10:37,204 INFO Check pod logs for errors
2024-04-21 22:10:38,929 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-idrepo-2.txt
2024-04-21 22:10:38,930 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/ds-idrepo-2.txt
2024-04-21 22:10:38,973 INFO Check pod logs for errors
2024-04-21 22:10:39,005 INFO
2024-04-21 22:10:39,005 INFO ------------------------- Dumping logs for AM -------------------------
2024-04-21 22:10:40,652 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/am-767f6b67b5-9zw22.txt
2024-04-21 22:10:40,653 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/am-767f6b67b5-9zw22.txt
2024-04-21 22:10:40,671 INFO Check pod logs for errors
2024-04-21 22:10:42,440 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/am-767f6b67b5-qcqz2.txt
2024-04-21 22:10:42,442 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/am-767f6b67b5-qcqz2.txt
2024-04-21 22:10:42,476 INFO Check pod logs for errors
2024-04-21 22:10:44,162 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/am-767f6b67b5-vbltz.txt
2024-04-21 22:10:44,164 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/am-767f6b67b5-vbltz.txt
2024-04-21 22:10:44,178 INFO Check pod logs for errors
2024-04-21 22:10:44,190 INFO
2024-04-21 22:10:44,191 INFO ----------------------- Dumping logs for AMSTER -----------------------
2024-04-21 22:10:45,319 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/amster-grkh7.txt
2024-04-21 22:10:45,320 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/amster-grkh7.txt
2024-04-21 22:10:45,320 INFO Check pod logs for errors
2024-04-21 22:10:45,320 INFO
2024-04-21 22:10:45,320 INFO ------------------------ Dumping logs for IDM ------------------------
2024-04-21 22:10:46,888 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/idm-b85f9d978-txpwv.txt
2024-04-21 22:10:46,888 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/idm-b85f9d978-txpwv.txt
2024-04-21 22:10:46,907 INFO Check pod logs for errors
2024-04-21 22:10:48,529 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/idm-b85f9d978-wnhxb.txt
2024-04-21 22:10:48,530 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/idm-b85f9d978-wnhxb.txt
2024-04-21 22:10:48,544 INFO Check pod logs for errors
2024-04-21 22:10:48,553 INFO
2024-04-21 22:10:48,554 INFO -------------------- Dumping logs for END-USER-UI --------------------
2024-04-21 22:10:50,077 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/end-user-ui-58878d7668-z8s6j.txt
2024-04-21 22:10:50,077 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/end-user-ui-58878d7668-z8s6j.txt
2024-04-21 22:10:50,105 INFO Check pod logs for errors
2024-04-21 22:10:50,137 INFO
2024-04-21 22:10:50,137 INFO ---------------------- Dumping logs for LOGIN-UI ----------------------
2024-04-21 22:10:51,619 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/login-ui-5fdf49fc66-dlhjd.txt
2024-04-21 22:10:51,620 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/login-ui-5fdf49fc66-dlhjd.txt
2024-04-21 22:10:51,662 INFO Check pod logs for errors
2024-04-21 22:10:51,697 INFO
2024-04-21 22:10:51,697 INFO ---------------------- Dumping logs for ADMIN-UI ----------------------
2024-04-21 22:10:53,173 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/admin-ui-c4c7bf584-tlgkc.txt
2024-04-21 22:10:53,173 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240421_221022-after-deployment/admin-ui-c4c7bf584-tlgkc.txt
2024-04-21 22:10:53,216 INFO Check pod logs for errors
[2024-04-21 22:10:53] - INFO: Deployment ready
________________________________________________________________________________
[2024-04-21 22:10:53] Discover_all_components post : Post method
________________________________________________________________________________
Setting result to PASS
Task has been successfully stopped