--Task-- name: Deploy_all_forgerock_components enabled: True class_name: DeployComponentsTask source_name: controller source_namespace: >default< target_name: controller target_namespace: >default< start: 0.0 stop: None timeout: no timeout loop: False interval: None dependencies: ['Enable_prometheus_admin_api'] wait_for: [] preceding_task: None options: {} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [2024-04-09 21:54:59] Deploy_all_forgerock_components pre : Initialising task parameters ________________________________________________________________________________ task will be executed on controller (localhost) ________________________________________________________________________________ [2024-04-09 21:54:59] Deploy_all_forgerock_components step1 : Deploy components ________________________________________________________________________________ 2024-04-09 21:54:59,294 INFO 2024-04-09 21:54:59,295 INFO ******************************** Cleaning up existing namespace ******************************** 2024-04-09 21:54:59,630 INFO 2024-04-09 21:54:59,631 INFO ------------------ Deleting secret agent controller ------------------ 2024-04-09 21:54:59,958 INFO 2024-04-09 21:54:59,958 INFO [loop_until]: kubectl --namespace=xlou delete sac --all 2024-04-09 21:54:59,958 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:55:00,548 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:55:00,549 DEBUG --- stdout --- 2024-04-09 21:55:00,549 DEBUG secretagentconfiguration.secret-agent.secrets.forgerock.io "forgerock-sac" deleted 2024-04-09 21:55:00,549 DEBUG --- stderr --- 2024-04-09 21:55:00,549 DEBUG 2024-04-09 21:55:00,549 INFO 2024-04-09 21:55:00,549 INFO ----------------------- Deleting all resources ----------------------- 2024-04-09 21:55:00,549 INFO 2024-04-09 21:55:00,549 INFO [loop_until]: kubectl --namespace=xlou delete all --all --grace-period=0 --force 2024-04-09 21:55:00,549 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:55:12,102 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:55:12,102 DEBUG --- stdout --- 2024-04-09 21:55:12,102 DEBUG pod "admin-ui-7cf664f88d-mfwkm" force deleted pod "am-654c4fcbcf-78tfh" force deleted pod "am-654c4fcbcf-8p5dq" force deleted pod "am-654c4fcbcf-pm4w9" force deleted pod "amster-r6nvt" force deleted pod "ds-cts-0" force deleted pod "ds-cts-1" force deleted pod "ds-cts-2" force deleted pod "ds-idrepo-0" force deleted pod "ds-idrepo-1" force deleted pod "ds-idrepo-2" force deleted pod "end-user-ui-6d9fd5487b-rl5hw" force deleted pod "idm-76bdcbcb4c-bqxbz" force deleted pod "idm-76bdcbcb4c-xf2nb" force deleted pod "ldif-importer-898w9" force deleted pod "lodemon-6b949548d-xvhgs" force deleted pod "login-ui-5f964fb894-jzdc8" force deleted pod "overseer-0-66746dfb7d-cjf44" force deleted service "admin-ui" force deleted service "am" force deleted service "ds-cts" force deleted service "ds-idrepo" force deleted service "end-user-ui" force deleted service "idm" force deleted service "login-ui" force deleted service "overseer-0" force deleted deployment.apps "admin-ui" force deleted deployment.apps "am" force deleted deployment.apps "end-user-ui" force deleted deployment.apps "idm" force deleted deployment.apps "lodemon" force deleted deployment.apps "login-ui" force deleted deployment.apps "overseer-0" force deleted statefulset.apps "ds-cts" force deleted statefulset.apps "ds-idrepo" force deleted job.batch "amster" force deleted job.batch "ldif-importer" force deleted 2024-04-09 21:55:12,103 DEBUG --- stderr --- 2024-04-09 21:55:12,103 DEBUG Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. 2024-04-09 21:55:12,103 INFO 2024-04-09 21:55:12,103 INFO [loop_until]: kubectl --namespace xlou get pods | grep "No resources found" 2024-04-09 21:55:12,103 INFO [loop_until]: (max_time=360, interval=10, expected_rc=[0] 2024-04-09 21:55:12,562 INFO [loop_until]: Function succeeded after 0s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:55:23,026 INFO [loop_until]: Function succeeded after 10s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:55:33,494 INFO [loop_until]: Function succeeded after 21s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:55:43,968 INFO [loop_until]: Function succeeded after 31s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:55:54,420 INFO [loop_until]: Function succeeded after 42s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:56:04,881 INFO [loop_until]: Function succeeded after 52s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:56:15,328 INFO [loop_until]: Function succeeded after 1m 03s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:56:25,792 INFO [loop_until]: Function succeeded after 1m 13s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:56:36,259 INFO [loop_until]: Function succeeded after 1m 24s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:56:46,706 INFO [loop_until]: Function succeeded after 1m 34s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:56:57,156 INFO [loop_until]: Function succeeded after 1m 45s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:57:07,601 INFO [loop_until]: Function succeeded after 1m 55s (rc=0) - failed to find expected output: No resources found - retry 2024-04-09 21:57:17,927 INFO [loop_until]: Function succeeded after 2m 05s (rc=0) - expected pattern found 2024-04-09 21:57:17,927 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:17,928 DEBUG --- stdout --- 2024-04-09 21:57:17,928 DEBUG 2024-04-09 21:57:17,928 DEBUG --- stderr --- 2024-04-09 21:57:17,928 DEBUG No resources found in xlou namespace. 2024-04-09 21:57:17,928 INFO 2024-04-09 21:57:17,928 INFO ------------------------- Deleting configmap ------------------------- 2024-04-09 21:57:17,928 INFO 2024-04-09 21:57:17,928 INFO [loop_until]: kubectl --namespace=xlou get configmap --output jsonpath={.items[*].metadata.name} 2024-04-09 21:57:17,928 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:18,398 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:18,398 DEBUG --- stdout --- 2024-04-09 21:57:18,399 DEBUG am-logback amster-files amster-retain idm idm-logging-properties kube-root-ca.crt lodemon-config lodemon-deployments overseer-config-0 platform-config 2024-04-09 21:57:18,399 DEBUG --- stderr --- 2024-04-09 21:57:18,399 DEBUG 2024-04-09 21:57:18,399 INFO 2024-04-09 21:57:18,399 INFO [loop_until]: kubectl --namespace=xlou delete configmap am-logback --ignore-not-found 2024-04-09 21:57:18,399 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:18,810 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:18,811 DEBUG --- stdout --- 2024-04-09 21:57:18,811 DEBUG configmap "am-logback" deleted 2024-04-09 21:57:18,811 DEBUG --- stderr --- 2024-04-09 21:57:18,811 DEBUG 2024-04-09 21:57:18,811 INFO 2024-04-09 21:57:18,811 INFO [loop_until]: kubectl --namespace=xlou delete configmap amster-files --ignore-not-found 2024-04-09 21:57:18,811 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:19,235 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:19,235 DEBUG --- stdout --- 2024-04-09 21:57:19,236 DEBUG configmap "amster-files" deleted 2024-04-09 21:57:19,236 DEBUG --- stderr --- 2024-04-09 21:57:19,236 DEBUG 2024-04-09 21:57:19,236 INFO 2024-04-09 21:57:19,236 INFO [loop_until]: kubectl --namespace=xlou delete configmap amster-retain --ignore-not-found 2024-04-09 21:57:19,236 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:19,661 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:19,661 DEBUG --- stdout --- 2024-04-09 21:57:19,661 DEBUG configmap "amster-retain" deleted 2024-04-09 21:57:19,661 DEBUG --- stderr --- 2024-04-09 21:57:19,661 DEBUG 2024-04-09 21:57:19,661 INFO 2024-04-09 21:57:19,661 INFO [loop_until]: kubectl --namespace=xlou delete configmap idm --ignore-not-found 2024-04-09 21:57:19,661 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:20,117 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:20,117 DEBUG --- stdout --- 2024-04-09 21:57:20,118 DEBUG configmap "idm" deleted 2024-04-09 21:57:20,118 DEBUG --- stderr --- 2024-04-09 21:57:20,118 DEBUG 2024-04-09 21:57:20,118 INFO 2024-04-09 21:57:20,118 INFO [loop_until]: kubectl --namespace=xlou delete configmap idm-logging-properties --ignore-not-found 2024-04-09 21:57:20,118 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:20,546 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:20,546 DEBUG --- stdout --- 2024-04-09 21:57:20,546 DEBUG configmap "idm-logging-properties" deleted 2024-04-09 21:57:20,546 DEBUG --- stderr --- 2024-04-09 21:57:20,546 DEBUG 2024-04-09 21:57:20,547 INFO 2024-04-09 21:57:20,547 INFO [loop_until]: kubectl --namespace=xlou delete configmap kube-root-ca.crt --ignore-not-found 2024-04-09 21:57:20,547 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:20,980 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:20,980 DEBUG --- stdout --- 2024-04-09 21:57:20,980 DEBUG configmap "kube-root-ca.crt" deleted 2024-04-09 21:57:20,980 DEBUG --- stderr --- 2024-04-09 21:57:20,980 DEBUG 2024-04-09 21:57:20,980 INFO 2024-04-09 21:57:20,980 INFO [loop_until]: kubectl --namespace=xlou delete configmap lodemon-config --ignore-not-found 2024-04-09 21:57:20,981 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:21,407 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:21,407 DEBUG --- stdout --- 2024-04-09 21:57:21,407 DEBUG configmap "lodemon-config" deleted 2024-04-09 21:57:21,407 DEBUG --- stderr --- 2024-04-09 21:57:21,407 DEBUG 2024-04-09 21:57:21,407 INFO 2024-04-09 21:57:21,407 INFO [loop_until]: kubectl --namespace=xlou delete configmap lodemon-deployments --ignore-not-found 2024-04-09 21:57:21,407 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:21,836 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:21,836 DEBUG --- stdout --- 2024-04-09 21:57:21,836 DEBUG configmap "lodemon-deployments" deleted 2024-04-09 21:57:21,836 DEBUG --- stderr --- 2024-04-09 21:57:21,836 DEBUG 2024-04-09 21:57:21,836 INFO 2024-04-09 21:57:21,836 INFO [loop_until]: kubectl --namespace=xlou delete configmap overseer-config-0 --ignore-not-found 2024-04-09 21:57:21,836 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:22,263 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:22,263 DEBUG --- stdout --- 2024-04-09 21:57:22,264 DEBUG configmap "overseer-config-0" deleted 2024-04-09 21:57:22,264 DEBUG --- stderr --- 2024-04-09 21:57:22,264 DEBUG 2024-04-09 21:57:22,264 INFO 2024-04-09 21:57:22,264 INFO [loop_until]: kubectl --namespace=xlou delete configmap platform-config --ignore-not-found 2024-04-09 21:57:22,264 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:22,678 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:22,678 DEBUG --- stdout --- 2024-04-09 21:57:22,678 DEBUG configmap "platform-config" deleted 2024-04-09 21:57:22,678 DEBUG --- stderr --- 2024-04-09 21:57:22,678 DEBUG 2024-04-09 21:57:22,678 INFO 2024-04-09 21:57:22,678 INFO --------------------------- Deleting secret --------------------------- 2024-04-09 21:57:22,678 INFO 2024-04-09 21:57:22,678 INFO [loop_until]: kubectl --namespace=xlou get secret **** 2024-04-09 21:57:22,678 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:23,072 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:23,072 DEBUG --- stdout --- 2024-04-09 21:57:23,072 DEBUG cloud-storage-credentials-cts cloud-storage-credentials-idrepo 2024-04-09 21:57:23,072 DEBUG --- stderr --- 2024-04-09 21:57:23,072 DEBUG 2024-04-09 21:57:23,072 INFO 2024-04-09 21:57:23,072 INFO [loop_until]: kubectl --namespace=xlou delete secret **** --ignore-not-found 2024-04-09 21:57:23,073 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:23,514 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:23,514 DEBUG --- stdout --- 2024-04-09 21:57:23,514 DEBUG secret "cloud-storage-credentials-cts" deleted 2024-04-09 21:57:23,514 DEBUG --- stderr --- 2024-04-09 21:57:23,514 DEBUG 2024-04-09 21:57:23,515 INFO 2024-04-09 21:57:23,515 INFO [loop_until]: kubectl --namespace=xlou delete secret **** --ignore-not-found 2024-04-09 21:57:23,515 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:23,942 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:23,943 DEBUG --- stdout --- 2024-04-09 21:57:23,943 DEBUG secret "cloud-storage-credentials-idrepo" deleted 2024-04-09 21:57:23,943 DEBUG --- stderr --- 2024-04-09 21:57:23,943 DEBUG 2024-04-09 21:57:23,943 INFO 2024-04-09 21:57:23,943 INFO -------------------------- Deleting ingress -------------------------- 2024-04-09 21:57:23,943 INFO 2024-04-09 21:57:23,943 INFO [loop_until]: kubectl --namespace=xlou get ingress --output jsonpath={.items[*].metadata.name} 2024-04-09 21:57:23,943 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:24,331 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:24,331 DEBUG --- stdout --- 2024-04-09 21:57:24,332 DEBUG forgerock ig overseer-0 2024-04-09 21:57:24,332 DEBUG --- stderr --- 2024-04-09 21:57:24,332 DEBUG 2024-04-09 21:57:24,332 INFO 2024-04-09 21:57:24,332 INFO [loop_until]: kubectl --namespace=xlou delete ingress forgerock --ignore-not-found 2024-04-09 21:57:24,332 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:24,770 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:24,770 DEBUG --- stdout --- 2024-04-09 21:57:24,770 DEBUG ingress.networking.k8s.io "forgerock" deleted 2024-04-09 21:57:24,770 DEBUG --- stderr --- 2024-04-09 21:57:24,770 DEBUG 2024-04-09 21:57:24,770 INFO 2024-04-09 21:57:24,770 INFO [loop_until]: kubectl --namespace=xlou delete ingress ig --ignore-not-found 2024-04-09 21:57:24,770 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:25,179 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:25,179 DEBUG --- stdout --- 2024-04-09 21:57:25,179 DEBUG ingress.networking.k8s.io "ig" deleted 2024-04-09 21:57:25,179 DEBUG --- stderr --- 2024-04-09 21:57:25,179 DEBUG 2024-04-09 21:57:25,180 INFO 2024-04-09 21:57:25,180 INFO [loop_until]: kubectl --namespace=xlou delete ingress overseer-0 --ignore-not-found 2024-04-09 21:57:25,180 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:25,596 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:25,597 DEBUG --- stdout --- 2024-04-09 21:57:25,597 DEBUG ingress.networking.k8s.io "overseer-0" deleted 2024-04-09 21:57:25,597 DEBUG --- stderr --- 2024-04-09 21:57:25,597 DEBUG 2024-04-09 21:57:25,597 INFO 2024-04-09 21:57:25,597 INFO ---------------------------- Deleting pvc ---------------------------- 2024-04-09 21:57:25,597 INFO 2024-04-09 21:57:25,597 INFO [loop_until]: kubectl --namespace=xlou get pvc --output jsonpath={.items[*].metadata.name} 2024-04-09 21:57:25,597 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:25,987 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:25,987 DEBUG --- stdout --- 2024-04-09 21:57:25,987 DEBUG data-ds-cts-0 data-ds-cts-1 data-ds-cts-2 data-ds-idrepo-0 data-ds-idrepo-1 data-ds-idrepo-2 overseer-0 2024-04-09 21:57:25,987 DEBUG --- stderr --- 2024-04-09 21:57:25,987 DEBUG 2024-04-09 21:57:25,987 INFO 2024-04-09 21:57:25,987 INFO [loop_until]: kubectl --namespace=xlou delete pvc data-ds-cts-0 --ignore-not-found 2024-04-09 21:57:25,987 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:26,405 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:26,405 DEBUG --- stdout --- 2024-04-09 21:57:26,405 DEBUG persistentvolumeclaim "data-ds-cts-0" deleted 2024-04-09 21:57:26,405 DEBUG --- stderr --- 2024-04-09 21:57:26,405 DEBUG 2024-04-09 21:57:26,406 INFO 2024-04-09 21:57:26,406 INFO [loop_until]: kubectl --namespace=xlou delete pvc data-ds-cts-1 --ignore-not-found 2024-04-09 21:57:26,406 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:26,840 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:26,840 DEBUG --- stdout --- 2024-04-09 21:57:26,840 DEBUG persistentvolumeclaim "data-ds-cts-1" deleted 2024-04-09 21:57:26,840 DEBUG --- stderr --- 2024-04-09 21:57:26,840 DEBUG 2024-04-09 21:57:26,841 INFO 2024-04-09 21:57:26,841 INFO [loop_until]: kubectl --namespace=xlou delete pvc data-ds-cts-2 --ignore-not-found 2024-04-09 21:57:26,841 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:27,270 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:27,271 DEBUG --- stdout --- 2024-04-09 21:57:27,271 DEBUG persistentvolumeclaim "data-ds-cts-2" deleted 2024-04-09 21:57:27,271 DEBUG --- stderr --- 2024-04-09 21:57:27,271 DEBUG 2024-04-09 21:57:27,271 INFO 2024-04-09 21:57:27,271 INFO [loop_until]: kubectl --namespace=xlou delete pvc data-ds-idrepo-0 --ignore-not-found 2024-04-09 21:57:27,271 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:27,707 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:27,709 DEBUG --- stdout --- 2024-04-09 21:57:27,709 DEBUG persistentvolumeclaim "data-ds-idrepo-0" deleted 2024-04-09 21:57:27,709 DEBUG --- stderr --- 2024-04-09 21:57:27,709 DEBUG 2024-04-09 21:57:27,710 INFO 2024-04-09 21:57:27,710 INFO [loop_until]: kubectl --namespace=xlou delete pvc data-ds-idrepo-1 --ignore-not-found 2024-04-09 21:57:27,710 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:28,137 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:28,137 DEBUG --- stdout --- 2024-04-09 21:57:28,137 DEBUG persistentvolumeclaim "data-ds-idrepo-1" deleted 2024-04-09 21:57:28,137 DEBUG --- stderr --- 2024-04-09 21:57:28,137 DEBUG 2024-04-09 21:57:28,137 INFO 2024-04-09 21:57:28,137 INFO [loop_until]: kubectl --namespace=xlou delete pvc data-ds-idrepo-2 --ignore-not-found 2024-04-09 21:57:28,137 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:28,564 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:28,564 DEBUG --- stdout --- 2024-04-09 21:57:28,564 DEBUG persistentvolumeclaim "data-ds-idrepo-2" deleted 2024-04-09 21:57:28,564 DEBUG --- stderr --- 2024-04-09 21:57:28,564 DEBUG 2024-04-09 21:57:28,564 INFO 2024-04-09 21:57:28,564 INFO [loop_until]: kubectl --namespace=xlou delete pvc overseer-0 --ignore-not-found 2024-04-09 21:57:28,564 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:29,041 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:29,041 DEBUG --- stdout --- 2024-04-09 21:57:29,041 DEBUG persistentvolumeclaim "overseer-0" deleted 2024-04-09 21:57:29,041 DEBUG --- stderr --- 2024-04-09 21:57:29,042 DEBUG 2024-04-09 21:57:29,042 INFO 2024-04-09 21:57:29,042 INFO [loop_until]: kubectl --namespace=xlou delete pv ds-backup-xlou --ignore-not-found 2024-04-09 21:57:29,042 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:29,476 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:29,476 DEBUG --- stdout --- 2024-04-09 21:57:29,476 DEBUG 2024-04-09 21:57:29,476 DEBUG --- stderr --- 2024-04-09 21:57:29,476 DEBUG Warning: deleting cluster-scoped resources, not scoped to the provided namespace 2024-04-09 21:57:29,476 INFO 2024-04-09 21:57:29,476 INFO ----------------- Deleting admin clusterrolebindings ----------------- 2024-04-09 21:57:29,476 INFO 2024-04-09 21:57:29,476 INFO [loop_until]: kubectl get clusterrolebinding --output jsonpath="{range .items[?(@.subjects[0].namespace=='xlou')]}{.metadata.name} {end}" 2024-04-09 21:57:29,476 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:30,039 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:30,040 DEBUG --- stdout --- 2024-04-09 21:57:30,040 DEBUG k8s-svc-acct-crb-xlou k8s-svc-acct-crb-xlou-0 2024-04-09 21:57:30,040 DEBUG --- stderr --- 2024-04-09 21:57:30,040 DEBUG 2024-04-09 21:57:30,040 INFO Deleting clusterrolebinding k8s-svc-acct-crb-xlou associated with xlou namespace 2024-04-09 21:57:30,040 INFO 2024-04-09 21:57:30,040 INFO [loop_until]: kubectl delete clusterrolebinding k8s-svc-acct-crb-xlou 2024-04-09 21:57:30,040 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:30,450 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:30,450 DEBUG --- stdout --- 2024-04-09 21:57:30,450 DEBUG clusterrolebinding.rbac.authorization.k8s.io "k8s-svc-acct-crb-xlou" deleted 2024-04-09 21:57:30,450 DEBUG --- stderr --- 2024-04-09 21:57:30,451 DEBUG 2024-04-09 21:57:30,451 INFO Deleting clusterrolebinding k8s-svc-acct-crb-xlou-0 associated with xlou namespace 2024-04-09 21:57:30,451 INFO 2024-04-09 21:57:30,451 INFO [loop_until]: kubectl delete clusterrolebinding k8s-svc-acct-crb-xlou-0 2024-04-09 21:57:30,451 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:30,874 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:30,874 DEBUG --- stdout --- 2024-04-09 21:57:30,874 DEBUG clusterrolebinding.rbac.authorization.k8s.io "k8s-svc-acct-crb-xlou-0" deleted 2024-04-09 21:57:30,875 DEBUG --- stderr --- 2024-04-09 21:57:30,875 DEBUG 2024-04-09 21:57:30,875 INFO 2024-04-09 21:57:30,875 INFO ------------------------- Deleting namespace ------------------------- 2024-04-09 21:57:31,202 INFO 2024-04-09 21:57:31,202 INFO [loop_until]: kubectl delete namespaces xlou --grace-period=0 --force 2024-04-09 21:57:31,203 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 21:57:37,941 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:37,941 DEBUG --- stdout --- 2024-04-09 21:57:37,941 DEBUG namespace "xlou" force deleted 2024-04-09 21:57:37,942 DEBUG --- stderr --- 2024-04-09 21:57:37,942 DEBUG Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. 2024-04-09 21:57:57,304 INFO 2024-04-09 21:57:57,304 INFO ************************************* Creating deployment ************************************* 2024-04-09 21:57:57,304 INFO Creating normal (forgeops) type deployment for deployment: stack 2024-04-09 21:57:57,322 INFO 2024-04-09 21:57:57,322 INFO ------- Custom component configuration present. Loading values ------- 2024-04-09 21:57:57,322 INFO 2024-04-09 21:57:57,322 INFO ************************************ Configuring components ************************************ 2024-04-09 21:57:57,629 INFO Namespace xlou does not exist, no resources to delete 2024-04-09 21:57:57,630 INFO 2024-04-09 21:57:57,630 INFO [loop_until]: kubectl create namespace xlou 2024-04-09 21:57:57,630 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:57,947 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:57,947 DEBUG --- stdout --- 2024-04-09 21:57:57,947 DEBUG namespace/xlou created 2024-04-09 21:57:57,947 DEBUG --- stderr --- 2024-04-09 21:57:57,947 DEBUG 2024-04-09 21:57:57,947 INFO 2024-04-09 21:57:57,947 INFO [loop_until]: kubectl label namespace xlou self-service=false timeout=48 2024-04-09 21:57:57,947 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:57:58,356 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:57:58,357 DEBUG --- stdout --- 2024-04-09 21:57:58,357 DEBUG namespace/xlou labeled 2024-04-09 21:57:58,357 DEBUG --- stderr --- 2024-04-09 21:57:58,357 DEBUG 2024-04-09 21:57:58,357 INFO 2024-04-09 21:57:58,357 INFO Applying custom configuration, dockerfiles to deployment and custom lodestar component configuration 2024-04-09 21:57:58,358 INFO Copying /mnt/disks/data/xslou/lodestar-fork/shared/config/custom/session_timeout_3minutes/docker/am to /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am 2024-04-09 21:57:58,360 INFO No custom features provided. Nothing to do. 2024-04-09 21:57:58,360 INFO 2024-04-09 21:57:58,360 INFO ---- Updating components image tag/repo from platform-images repo ---- 2024-04-09 21:57:58,360 INFO 2024-04-09 21:57:58,360 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/set-images --clean 2024-04-09 21:57:58,448 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:57:58,448 DEBUG --- stdout --- 2024-04-09 21:57:58,448 DEBUG [INFO] Cleaning up. [WARNING] Found nothing to clean. 2024-04-09 21:57:58,448 DEBUG --- stderr --- 2024-04-09 21:57:58,448 DEBUG 2024-04-09 21:57:58,448 INFO 2024-04-09 21:57:58,448 INFO 2024-04-09 21:57:58,449 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/set-images --ref master-ready-for-dev-pipelines --products ds 2024-04-09 21:58:04,207 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:04,207 DEBUG --- stdout --- 2024-04-09 21:58:04,207 DEBUG [INFO] Setting repo up in /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/build/platform-images. [INFO] Repo is at 4800a873e33c1cbca21ac913ad9b95a41320974b on branch HEAD [INFO] Updating products ds [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/dsutil/Dockerfile to use gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903 [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/ds-new/Dockerfile to use gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903 [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/proxy/Dockerfile to use gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903 [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts/Dockerfile to use gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903 [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo/Dockerfile to use gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903 2024-04-09 21:58:04,207 DEBUG --- stderr --- 2024-04-09 21:58:04,207 DEBUG 2024-04-09 21:58:04,207 INFO 2024-04-09 21:58:04,207 INFO 2024-04-09 21:58:04,208 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/set-images --ref master-ready-for-dev-pipelines --products am 2024-04-09 21:58:04,319 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:04,319 DEBUG --- stdout --- 2024-04-09 21:58:04,320 DEBUG [INFO] Setting repo up in /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/build/platform-images. [INFO] Found existing files, attempting to not clone. [INFO] Repo is at 4800a873e33c1cbca21ac913ad9b95a41320974b on branch HEAD [INFO] Updating products am [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am/Dockerfile to use gcr.io/forgerock-io/am-cdk/pit1:7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 2024-04-09 21:58:04,320 DEBUG --- stderr --- 2024-04-09 21:58:04,320 DEBUG 2024-04-09 21:58:04,320 INFO 2024-04-09 21:58:04,320 INFO 2024-04-09 21:58:04,320 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/set-images --ref master-ready-for-dev-pipelines --products amster 2024-04-09 21:58:04,428 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:04,429 DEBUG --- stdout --- 2024-04-09 21:58:04,429 DEBUG [INFO] Setting repo up in /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/build/platform-images. [INFO] Found existing files, attempting to not clone. [INFO] Repo is at 4800a873e33c1cbca21ac913ad9b95a41320974b on branch HEAD [INFO] Updating products amster [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster/Dockerfile to use gcr.io/forgerock-io/amster/pit1:7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 2024-04-09 21:58:04,429 DEBUG --- stderr --- 2024-04-09 21:58:04,429 DEBUG 2024-04-09 21:58:04,429 INFO 2024-04-09 21:58:04,429 INFO 2024-04-09 21:58:04,429 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/set-images --ref master-ready-for-dev-pipelines --products idm 2024-04-09 21:58:04,539 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:04,539 DEBUG --- stdout --- 2024-04-09 21:58:04,539 DEBUG [INFO] Setting repo up in /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/build/platform-images. [INFO] Found existing files, attempting to not clone. [INFO] Repo is at 4800a873e33c1cbca21ac913ad9b95a41320974b on branch HEAD [INFO] Updating products idm [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm/Dockerfile to use gcr.io/forgerock-io/idm-cdk/pit1:7.6.0-5a54de0b49885680977e8ef4134f710026e856bf 2024-04-09 21:58:04,539 DEBUG --- stderr --- 2024-04-09 21:58:04,539 DEBUG 2024-04-09 21:58:04,540 INFO 2024-04-09 21:58:04,540 INFO 2024-04-09 21:58:04,540 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/set-images --ref master-ready-for-dev-pipelines --products ui 2024-04-09 21:58:04,717 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:04,717 DEBUG --- stdout --- 2024-04-09 21:58:04,717 DEBUG [INFO] Setting repo up in /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/build/platform-images. [INFO] Found existing files, attempting to not clone. [INFO] Repo is at 4800a873e33c1cbca21ac913ad9b95a41320974b on branch HEAD [INFO] Updating products ui [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui to use gcr.io/forgerock-io/platform-admin-ui/docker-build:7.6.0-SNAPSHOT-7f0cffe7f264543681efac51219ef9cfc50cecea [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui to use gcr.io/forgerock-io/platform-enduser-ui/docker-build:7.6.0-SNAPSHOT-7f0cffe7f264543681efac51219ef9cfc50cecea [INFO] Updating /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui to use gcr.io/forgerock-io/platform-login-ui/docker-build:7.6.0-SNAPSHOT-7f0cffe7f264543681efac51219ef9cfc50cecea 2024-04-09 21:58:04,717 DEBUG --- stderr --- 2024-04-09 21:58:04,717 DEBUG 2024-04-09 21:58:04,717 INFO 2024-04-09 21:58:04,717 INFO 2024-04-09 21:58:04,717 INFO - Checking if component Dockerfile/kustomize needs additional update - 2024-04-09 21:58:04,717 INFO 2024-04-09 21:58:04,718 INFO 2024-04-09 21:58:04,718 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds cts 2024-04-09 21:58:04,799 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:04,800 DEBUG --- stdout --- 2024-04-09 21:58:04,800 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts 2024-04-09 21:58:04,800 DEBUG --- stderr --- 2024-04-09 21:58:04,800 DEBUG 2024-04-09 21:58:04,800 INFO 2024-04-09 21:58:04,800 INFO Checking if ds-cts dockerfile /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts/Dockerfile needs additional update (custom tag/repo from config.yaml or image name resolution) 2024-04-09 21:58:04,800 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903 2024-04-09 21:58:07,457 INFO No need to update FROM line for /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/cts/Dockerfile 2024-04-09 21:58:07,457 INFO 2024-04-09 21:58:07,458 INFO 2024-04-09 21:58:07,458 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker ds idrepo 2024-04-09 21:58:07,574 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:07,574 DEBUG --- stdout --- 2024-04-09 21:58:07,574 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo 2024-04-09 21:58:07,574 DEBUG --- stderr --- 2024-04-09 21:58:07,575 DEBUG 2024-04-09 21:58:07,575 INFO 2024-04-09 21:58:07,575 INFO Checking if ds-idrepo dockerfile /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo/Dockerfile needs additional update (custom tag/repo from config.yaml or image name resolution) 2024-04-09 21:58:07,575 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo/Dockerfile: gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903 2024-04-09 21:58:10,382 INFO No need to update FROM line for /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/ds/idrepo/Dockerfile 2024-04-09 21:58:10,382 INFO 2024-04-09 21:58:10,382 INFO 2024-04-09 21:58:10,383 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker am 2024-04-09 21:58:10,517 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:10,518 DEBUG --- stdout --- 2024-04-09 21:58:10,518 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am 2024-04-09 21:58:10,518 DEBUG --- stderr --- 2024-04-09 21:58:10,518 DEBUG 2024-04-09 21:58:10,518 INFO 2024-04-09 21:58:10,518 INFO Checking if am dockerfile /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am/Dockerfile needs additional update (custom tag/repo from config.yaml or image name resolution) 2024-04-09 21:58:10,519 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am/Dockerfile: gcr.io/forgerock-io/am-cdk/pit1:7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 2024-04-09 21:58:13,491 INFO No need to update FROM line for /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/am/Dockerfile 2024-04-09 21:58:13,493 INFO 2024-04-09 21:58:13,493 INFO 2024-04-09 21:58:13,493 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker amster 2024-04-09 21:58:13,613 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:13,613 DEBUG --- stdout --- 2024-04-09 21:58:13,614 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster 2024-04-09 21:58:13,614 DEBUG --- stderr --- 2024-04-09 21:58:13,614 DEBUG 2024-04-09 21:58:13,614 INFO 2024-04-09 21:58:13,614 INFO Checking if amster dockerfile /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster/Dockerfile needs additional update (custom tag/repo from config.yaml or image name resolution) 2024-04-09 21:58:13,614 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster/Dockerfile: gcr.io/forgerock-io/amster/pit1:7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 2024-04-09 21:58:16,407 INFO No need to update FROM line for /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/amster/Dockerfile 2024-04-09 21:58:16,407 INFO 2024-04-09 21:58:16,407 INFO 2024-04-09 21:58:16,407 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path docker idm 2024-04-09 21:58:16,517 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:16,517 DEBUG --- stdout --- 2024-04-09 21:58:16,517 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm 2024-04-09 21:58:16,517 DEBUG --- stderr --- 2024-04-09 21:58:16,517 DEBUG 2024-04-09 21:58:16,517 INFO 2024-04-09 21:58:16,517 INFO Checking if idm dockerfile /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm/Dockerfile needs additional update (custom tag/repo from config.yaml or image name resolution) 2024-04-09 21:58:16,518 INFO Read FROM line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm/Dockerfile: gcr.io/forgerock-io/idm-cdk/pit1:7.6.0-5a54de0b49885680977e8ef4134f710026e856bf 2024-04-09 21:58:19,336 INFO No need to update FROM line for /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/docker/idm/Dockerfile 2024-04-09 21:58:19,338 INFO 2024-04-09 21:58:19,338 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base end-user-ui 2024-04-09 21:58:19,455 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:19,455 DEBUG --- stdout --- 2024-04-09 21:58:19,455 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui 2024-04-09 21:58:19,456 DEBUG --- stderr --- 2024-04-09 21:58:19,456 DEBUG 2024-04-09 21:58:19,456 INFO 2024-04-09 21:58:19,460 INFO 2024-04-09 21:58:19,460 INFO Checking if end-user-ui kustomize file /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui/kustomization.yaml needs additional update (custom tag/repo from config.yaml or image name resolution) 2024-04-09 21:58:19,460 INFO Read image line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui/kustomization.yaml: gcr.io/forgerock-io/platform-enduser-ui/docker-build:7.6.0-SNAPSHOT-7f0cffe7f264543681efac51219ef9cfc50cecea 2024-04-09 21:58:22,586 INFO No need to update the image line for /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/end-user-ui/kustomization.yaml 2024-04-09 21:58:22,586 INFO 2024-04-09 21:58:22,586 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base login-ui 2024-04-09 21:58:22,697 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:22,698 DEBUG --- stdout --- 2024-04-09 21:58:22,698 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui 2024-04-09 21:58:22,698 DEBUG --- stderr --- 2024-04-09 21:58:22,698 DEBUG 2024-04-09 21:58:22,698 INFO 2024-04-09 21:58:22,701 INFO 2024-04-09 21:58:22,701 INFO Checking if login-ui kustomize file /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui/kustomization.yaml needs additional update (custom tag/repo from config.yaml or image name resolution) 2024-04-09 21:58:22,701 INFO Read image line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui/kustomization.yaml: gcr.io/forgerock-io/platform-login-ui/docker-build:7.6.0-SNAPSHOT-7f0cffe7f264543681efac51219ef9cfc50cecea 2024-04-09 21:58:25,778 INFO No need to update the image line for /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/login-ui/kustomization.yaml 2024-04-09 21:58:25,778 INFO 2024-04-09 21:58:25,778 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize base admin-ui 2024-04-09 21:58:25,894 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:25,895 DEBUG --- stdout --- 2024-04-09 21:58:25,895 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui 2024-04-09 21:58:25,895 DEBUG --- stderr --- 2024-04-09 21:58:25,895 DEBUG 2024-04-09 21:58:25,895 INFO 2024-04-09 21:58:25,900 INFO 2024-04-09 21:58:25,900 INFO Checking if admin-ui kustomize file /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui/kustomization.yaml needs additional update (custom tag/repo from config.yaml or image name resolution) 2024-04-09 21:58:25,900 INFO Read image line from /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui/kustomization.yaml: gcr.io/forgerock-io/platform-admin-ui/docker-build:7.6.0-SNAPSHOT-7f0cffe7f264543681efac51219ef9cfc50cecea 2024-04-09 21:58:28,910 INFO No need to update the image line for /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/base/admin-ui/kustomization.yaml 2024-04-09 21:58:28,910 INFO 2024-04-09 21:58:28,910 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/config path kustomize overlay medium 2024-04-09 21:58:29,028 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:29,028 DEBUG --- stdout --- 2024-04-09 21:58:29,028 DEBUG /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/overlay/medium 2024-04-09 21:58:29,028 DEBUG --- stderr --- 2024-04-09 21:58:29,028 DEBUG 2024-04-09 21:58:29,029 INFO 2024-04-09 21:58:29,388 INFO 2024-04-09 21:58:29,388 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm delete --filename /tmp/tmpu9e0ilhg 2024-04-09 21:58:29,388 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0, 1] 2024-04-09 21:58:29,709 INFO [loop_until]: OK (rc = 1) 2024-04-09 21:58:29,709 DEBUG --- stdout --- 2024-04-09 21:58:29,709 DEBUG 2024-04-09 21:58:29,709 DEBUG --- stderr --- 2024-04-09 21:58:29,709 DEBUG Error from server (NotFound): error when deleting "/tmp/tmpu9e0ilhg": secrets "sslcert" not found 2024-04-09 21:58:29,709 INFO 2024-04-09 21:58:29,709 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm apply --filename /tmp/tmpu9e0ilhg 2024-04-09 21:58:29,709 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 21:58:30,935 INFO [loop_until]: OK (rc = 0) 2024-04-09 21:58:30,937 DEBUG --- stdout --- 2024-04-09 21:58:30,937 DEBUG secret/sslcert created 2024-04-09 21:58:30,937 DEBUG --- stderr --- 2024-04-09 21:58:30,937 DEBUG 2024-04-09 21:58:30,949 INFO Loading external components from config.yaml 2024-04-09 21:58:31,079 INFO No external components defined. 2024-04-09 21:58:31,216 INFO 2024-04-09 21:58:31,216 INFO *********************************** Cluster readiness check *********************************** 2024-04-09 21:58:31,216 INFO 2024-04-09 21:58:31,216 INFO ------------------------- Checking K8s api... ------------------------- 2024-04-09 21:58:31,933 INFO 2024-04-09 21:58:31,934 INFO ----------------- Checking DNS/Konnectivity agent... ----------------- 2024-04-09 21:58:35,309 INFO All pre-deployment cluster checks passed. 2024-04-09 21:58:35,309 INFO 2024-04-09 21:58:35,309 INFO *********************************** Cluster resources check *********************************** 2024-04-09 21:58:35,309 INFO 2024-04-09 21:58:35,309 INFO -------------- Checking if cert-manager is installed... -------------- 2024-04-09 21:58:36,005 INFO cert-manager is installed 2024-04-09 21:58:36,010 INFO 2024-04-09 21:58:36,010 INFO ----------- Checking if secret-agent-system is installed... ----------- 2024-04-09 21:58:36,647 INFO secret-agent-system is installed 2024-04-09 21:58:36,648 INFO 2024-04-09 21:58:36,648 INFO ------------------ Checking if nginx is installed... ------------------ 2024-04-09 21:58:37,305 INFO nginx is installed 2024-04-09 21:58:37,439 INFO 2024-04-09 21:58:37,439 INFO ------ Checking if network services global dns is configured... ------ 2024-04-09 21:58:37,517 INFO Running ping against: default-http-backend.iam.xlou-cdm.engineeringpit.com 2024-04-09 21:58:41,647 INFO network services global dns is configured 2024-04-09 21:58:41,647 INFO 2024-04-09 21:58:41,647 INFO ---- Checking if disk storage type has been updated to use ssd... ---- 2024-04-09 21:58:41,973 INFO disk storage type has been updated to use ssd 2024-04-09 21:58:42,089 INFO 2024-04-09 21:58:42,089 INFO ------------ Checking if lodestar-ssl-cert is installed... ------------ 2024-04-09 21:58:42,719 INFO lodestar-ssl-cert is installed 2024-04-09 21:58:42,720 INFO 2024-04-09 21:58:42,720 INFO ------------- Checking if intezer-protect is installed... ------------- 2024-04-09 21:58:43,424 INFO intezer-protect is installed 2024-04-09 21:58:43,424 INFO 2024-04-09 21:58:43,424 INFO ******************************* Verify docker daemon is running ******************************* 2024-04-09 21:58:43,492 INFO Docker daemon is running 2024-04-09 21:58:43,492 INFO 2024-04-09 21:58:43,492 INFO ************************** The following components will be deployed ************************** 2024-04-09 21:58:43,492 INFO - ds-cts (DS) - ds-idrepo (DS) - am (AM) - amster (Amster) - idm (IDM) - end-user-ui (EndUserUi) - login-ui (LoginUi) - admin-ui (AdminUi) 2024-04-09 21:58:43,492 INFO 2024-04-09 21:58:43,493 INFO ---------- Build product images using forgeops build command ---------- 2024-04-09 21:58:43,611 INFO 2024-04-09 21:58:43,611 INFO [run_command]: kubectl config current-context 2024-04-09 21:58:43,685 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:43,685 DEBUG --- stdout --- 2024-04-09 21:58:43,686 DEBUG gke_engineeringpit_us-east1-d_xlou-cdm 2024-04-09 21:58:43,686 DEBUG --- stderr --- 2024-04-09 21:58:43,686 DEBUG 2024-04-09 21:58:43,686 INFO 2024-04-09 21:58:43,686 INFO 2024-04-09 21:58:43,686 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/forgeops build ds-cts --config-profile=cdk --push-to gcr.io/engineeringpit/lodestar-images --tag=xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903 2024-04-09 21:58:43,686 DEBUG [run_command]: env={'HOME': '/home/xslou', 'PATH': '/mnt/disks/data/xslou/lodestar-fork/ext/bin:/home/xslou/.local/bin:/home/xslou/bin:~/bin:~/workshop/lodestar-fork/ext/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'DOCKER_SCAN_SUGGEST': 'false'} 2024-04-09 21:58:46,804 INFO The push refers to repository [gcr.io/engineeringpit/lodestar-images/ds-cts] 2024-04-09 21:58:46,907 INFO 1f875f357bc3: Preparing 2024-04-09 21:58:47,007 INFO a19dba0be6db: Preparing 2024-04-09 21:58:47,108 INFO 01c80f9cd31c: Preparing 2024-04-09 21:58:47,208 INFO 54977640d8ff: Preparing 2024-04-09 21:58:47,309 INFO 064abaee2efe: Preparing 2024-04-09 21:58:47,409 INFO cbd9008fcc14: Preparing 2024-04-09 21:58:47,510 INFO 1ce7f08455d4: Preparing 2024-04-09 21:58:47,610 INFO 5f70bf18a086: Preparing 2024-04-09 21:58:47,711 INFO 0f72daa50aea: Preparing 2024-04-09 21:58:47,811 INFO 7a4845b451a9: Preparing 2024-04-09 21:58:47,912 INFO 1fc9aae9e6e1: Preparing 2024-04-09 21:58:48,013 INFO 044ee1204247: Preparing 2024-04-09 21:58:48,113 INFO 6832d9c30d64: Preparing 2024-04-09 21:58:48,213 INFO 3c8879ab2cf2: Preparing 2024-04-09 21:58:48,314 INFO cbd9008fcc14: Waiting 2024-04-09 21:58:48,415 INFO 1ce7f08455d4: Waiting 2024-04-09 21:58:48,515 INFO 5f70bf18a086: Waiting 2024-04-09 21:58:48,616 INFO 0f72daa50aea: Waiting 2024-04-09 21:58:48,716 INFO 7a4845b451a9: Waiting 2024-04-09 21:58:48,817 INFO 1fc9aae9e6e1: Waiting 2024-04-09 21:58:48,917 INFO 044ee1204247: Waiting 2024-04-09 21:58:49,018 INFO 6832d9c30d64: Waiting 2024-04-09 21:58:49,118 INFO 3c8879ab2cf2: Waiting 2024-04-09 21:58:49,219 INFO 01c80f9cd31c: Layer already exists 2024-04-09 21:58:49,319 INFO 54977640d8ff: Layer already exists 2024-04-09 21:58:49,420 INFO a19dba0be6db: Layer already exists 2024-04-09 21:58:49,521 INFO 1f875f357bc3: Layer already exists 2024-04-09 21:58:49,621 INFO 064abaee2efe: Layer already exists 2024-04-09 21:58:49,722 INFO 1ce7f08455d4: Layer already exists 2024-04-09 21:58:49,823 INFO cbd9008fcc14: Layer already exists 2024-04-09 21:58:49,924 INFO 5f70bf18a086: Layer already exists 2024-04-09 21:58:50,024 INFO 0f72daa50aea: Layer already exists 2024-04-09 21:58:50,125 INFO 7a4845b451a9: Layer already exists 2024-04-09 21:58:50,226 INFO 1fc9aae9e6e1: Layer already exists 2024-04-09 21:58:50,327 INFO 6832d9c30d64: Layer already exists 2024-04-09 21:58:50,427 INFO 044ee1204247: Layer already exists 2024-04-09 21:58:50,528 INFO 3c8879ab2cf2: Layer already exists 2024-04-09 21:58:50,629 INFO xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903: digest: sha256:d4625dfdd91929be72a5064d8bf9b53167e17f81160a4fb99e14abf4d21541a7 size: 3252 2024-04-09 21:58:50,729 INFO Updated the image_defaulter with your new image for ds-cts: "gcr.io/engineeringpit/lodestar-images/ds-cts:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903". 2024-04-09 21:58:50,830 ERROR #0 building with "default" instance using docker driver 2024-04-09 21:58:50,830 ERROR 2024-04-09 21:58:50,830 ERROR #1 [internal] load build definition from Dockerfile 2024-04-09 21:58:50,831 ERROR #1 transferring dockerfile: 529B done 2024-04-09 21:58:50,831 ERROR #1 DONE 0.0s 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #2 [internal] load metadata for gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903 2024-04-09 21:58:50,831 ERROR #2 DONE 0.5s 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #3 [internal] load .dockerignore 2024-04-09 21:58:50,831 ERROR #3 transferring context: 2B done 2024-04-09 21:58:50,831 ERROR #3 DONE 0.0s 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #4 [1/7] FROM gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903@sha256:f5da421b3fa9a262d8d8a3d36e9d3777a421bbec723f2522c417b942702fe62a 2024-04-09 21:58:50,831 ERROR #4 DONE 0.0s 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #5 [internal] load build context 2024-04-09 21:58:50,831 ERROR #5 transferring context: 71.67kB 0.0s done 2024-04-09 21:58:50,831 ERROR #5 DONE 0.0s 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #6 [5/7] COPY --chown=forgerock:root cts /opt/opendj/ 2024-04-09 21:58:50,831 ERROR #6 CACHED 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #7 [2/7] COPY debian-buster-sources.list /etc/apt/sources.list 2024-04-09 21:58:50,831 ERROR #7 CACHED 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #8 [6/7] COPY --chown=forgerock:root scripts /opt/opendj/scripts 2024-04-09 21:58:50,831 ERROR #8 CACHED 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #9 [3/7] RUN chown -R forgerock:root /opt/opendj 2024-04-09 21:58:50,831 ERROR #9 CACHED 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #10 [4/7] COPY --chown=forgerock:root common /opt/opendj/ 2024-04-09 21:58:50,831 ERROR #10 CACHED 2024-04-09 21:58:50,831 ERROR 2024-04-09 21:58:50,831 ERROR #11 [7/7] RUN bin/setup.sh && bin/relax-security-settings.sh && rm bin/setup.sh bin/relax-security-settings.sh 2024-04-09 21:58:50,832 ERROR #11 CACHED 2024-04-09 21:58:50,832 ERROR 2024-04-09 21:58:50,832 ERROR #12 exporting to image 2024-04-09 21:58:50,832 ERROR #12 exporting layers done 2024-04-09 21:58:50,832 ERROR #12 writing image sha256:26995a65f78983e9368bc7ee813fc02cbf78d7d19d38b071b099347cfcffe225 done 2024-04-09 21:58:50,832 ERROR #12 naming to gcr.io/engineeringpit/lodestar-images/ds-cts:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903 done 2024-04-09 21:58:50,832 ERROR #12 DONE 0.0s 2024-04-09 21:58:50,976 INFO 2024-04-09 21:58:50,977 INFO [run_command]: kubectl config current-context 2024-04-09 21:58:51,059 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:51,059 DEBUG --- stdout --- 2024-04-09 21:58:51,059 DEBUG gke_engineeringpit_us-east1-d_xlou-cdm 2024-04-09 21:58:51,059 DEBUG --- stderr --- 2024-04-09 21:58:51,059 DEBUG 2024-04-09 21:58:51,059 INFO 2024-04-09 21:58:51,060 INFO 2024-04-09 21:58:51,060 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/forgeops build ds-idrepo --config-profile=cdk --push-to gcr.io/engineeringpit/lodestar-images --tag=xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903 2024-04-09 21:58:51,060 DEBUG [run_command]: env={'HOME': '/home/xslou', 'PATH': '/mnt/disks/data/xslou/lodestar-fork/ext/bin:/home/xslou/.local/bin:/home/xslou/bin:~/bin:~/workshop/lodestar-fork/ext/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'DOCKER_SCAN_SUGGEST': 'false'} 2024-04-09 21:58:53,379 INFO The push refers to repository [gcr.io/engineeringpit/lodestar-images/ds-idrepo] 2024-04-09 21:58:53,480 INFO 10653c6cbcd5: Preparing 2024-04-09 21:58:53,580 INFO 0ca5d3d26b83: Preparing 2024-04-09 21:58:53,681 INFO e144dba4a797: Preparing 2024-04-09 21:58:53,781 INFO 5df20a8aea5c: Preparing 2024-04-09 21:58:53,882 INFO e3ba025c5405: Preparing 2024-04-09 21:58:53,982 INFO 751baa1f03ff: Preparing 2024-04-09 21:58:54,082 INFO 54977640d8ff: Preparing 2024-04-09 21:58:54,183 INFO 751baa1f03ff: Waiting 2024-04-09 21:58:54,283 INFO 5f70bf18a086: Preparing 2024-04-09 21:58:54,384 INFO cbd9008fcc14: Preparing 2024-04-09 21:58:54,484 INFO 1ce7f08455d4: Preparing 2024-04-09 21:58:54,585 INFO 5f70bf18a086: Preparing 2024-04-09 21:58:54,685 INFO 0f72daa50aea: Preparing 2024-04-09 21:58:54,786 INFO 7a4845b451a9: Preparing 2024-04-09 21:58:54,886 INFO 1fc9aae9e6e1: Preparing 2024-04-09 21:58:54,987 INFO 044ee1204247: Preparing 2024-04-09 21:58:55,087 INFO 6832d9c30d64: Preparing 2024-04-09 21:58:55,188 INFO 3c8879ab2cf2: Preparing 2024-04-09 21:58:55,288 INFO 54977640d8ff: Waiting 2024-04-09 21:58:55,389 INFO 5f70bf18a086: Waiting 2024-04-09 21:58:55,489 INFO cbd9008fcc14: Waiting 2024-04-09 21:58:55,590 INFO 1ce7f08455d4: Waiting 2024-04-09 21:58:55,690 INFO 0f72daa50aea: Waiting 2024-04-09 21:58:55,791 INFO 7a4845b451a9: Waiting 2024-04-09 21:58:55,891 INFO 1fc9aae9e6e1: Waiting 2024-04-09 21:58:55,992 INFO 044ee1204247: Waiting 2024-04-09 21:58:56,092 INFO 6832d9c30d64: Waiting 2024-04-09 21:58:56,193 INFO 3c8879ab2cf2: Waiting 2024-04-09 21:58:56,293 INFO e144dba4a797: Layer already exists 2024-04-09 21:58:56,394 INFO 0ca5d3d26b83: Layer already exists 2024-04-09 21:58:56,494 INFO 10653c6cbcd5: Layer already exists 2024-04-09 21:58:56,595 INFO 5df20a8aea5c: Layer already exists 2024-04-09 21:58:56,695 INFO e3ba025c5405: Layer already exists 2024-04-09 21:58:56,796 INFO 751baa1f03ff: Layer already exists 2024-04-09 21:58:56,896 INFO 54977640d8ff: Layer already exists 2024-04-09 21:58:56,997 INFO 5f70bf18a086: Layer already exists 2024-04-09 21:58:57,097 INFO cbd9008fcc14: Layer already exists 2024-04-09 21:58:57,198 INFO 1ce7f08455d4: Layer already exists 2024-04-09 21:58:57,300 INFO 0f72daa50aea: Layer already exists 2024-04-09 21:58:57,400 INFO 7a4845b451a9: Layer already exists 2024-04-09 21:58:57,501 INFO 1fc9aae9e6e1: Layer already exists 2024-04-09 21:58:57,602 INFO 044ee1204247: Layer already exists 2024-04-09 21:58:57,702 INFO 6832d9c30d64: Layer already exists 2024-04-09 21:58:57,803 INFO 3c8879ab2cf2: Layer already exists 2024-04-09 21:58:57,904 INFO xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903: digest: sha256:3d88403ace504e53841404bc005ad2bcc95df2a6a1f2eaceef779646ed5034b8 size: 3869 2024-04-09 21:58:58,004 INFO Updated the image_defaulter with your new image for ds-idrepo: "gcr.io/engineeringpit/lodestar-images/ds-idrepo:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903". 2024-04-09 21:58:58,105 ERROR #0 building with "default" instance using docker driver 2024-04-09 21:58:58,105 ERROR 2024-04-09 21:58:58,105 ERROR #1 [internal] load build definition from Dockerfile 2024-04-09 21:58:58,105 ERROR #1 transferring dockerfile: 2.45kB done 2024-04-09 21:58:58,105 ERROR #1 DONE 0.0s 2024-04-09 21:58:58,105 ERROR 2024-04-09 21:58:58,105 ERROR #2 [internal] load metadata for gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903 2024-04-09 21:58:58,106 ERROR #2 DONE 0.3s 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,106 ERROR #3 [internal] load .dockerignore 2024-04-09 21:58:58,106 ERROR #3 transferring context: 2B done 2024-04-09 21:58:58,106 ERROR #3 DONE 0.0s 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,106 ERROR #4 [ 1/10] FROM gcr.io/forgerock-io/ds/pit1:7.6.0-d49ad6a5879fab3530729d8773297783d042f903@sha256:f5da421b3fa9a262d8d8a3d36e9d3777a421bbec723f2522c417b942702fe62a 2024-04-09 21:58:58,106 ERROR #4 DONE 0.0s 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,106 ERROR #5 [internal] load build context 2024-04-09 21:58:58,106 ERROR #5 transferring context: 37.97kB done 2024-04-09 21:58:58,106 ERROR #5 DONE 0.0s 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,106 ERROR #6 [ 8/10] COPY --chown=forgerock:root idrepo/*.ldif /var/tmp/ 2024-04-09 21:58:58,106 ERROR #6 CACHED 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,106 ERROR #7 [ 9/10] RUN chmod +w template/setup-profiles/AM/config/6.5/base-entries.ldif && cat scripts/external-am-datastore.ldif >> template/setup-profiles/AM/config/6.5/base-entries.ldif && cat uma/opendj_uma_audit.ldif >> template/setup-profiles/AM/config/6.5/base-entries.ldif && cat uma/opendj_uma_pending_requests.ldif >> template/setup-profiles/AM/config/6.5/base-entries.ldif && cat uma/opendj_uma_resource_set_labels.ldif >> template/setup-profiles/AM/config/6.5/base-entries.ldif && cat uma/opendj_uma_resource_sets.ldif >> template/setup-profiles/AM/config/6.5/base-entries.ldif && cat /var/tmp/alpha_bravo.ldif >> template/setup-profiles/AM/config/6.5/base-entries.ldif && chmod +w template/setup-profiles/AM/identity-store/7.0/base-entries.ldif && cat /var/tmp/orgs.ldif >> template/setup-profiles/AM/identity-store/7.0/base-entries.ldif && rm /var/tmp/*ldif 2024-04-09 21:58:58,106 ERROR #7 CACHED 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,106 ERROR #8 [ 6/10] COPY --chown=forgerock:root scripts /opt/opendj/scripts 2024-04-09 21:58:58,106 ERROR #8 CACHED 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,106 ERROR #9 [ 2/10] COPY debian-buster-sources.list /etc/apt/sources.list 2024-04-09 21:58:58,106 ERROR #9 CACHED 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,106 ERROR #10 [ 4/10] COPY --chown=forgerock:root common /opt/opendj/ 2024-04-09 21:58:58,106 ERROR #10 CACHED 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,106 ERROR #11 [ 5/10] COPY --chown=forgerock:root idrepo /opt/opendj/ 2024-04-09 21:58:58,106 ERROR #11 CACHED 2024-04-09 21:58:58,106 ERROR 2024-04-09 21:58:58,107 ERROR #12 [ 7/10] COPY --chown=forgerock:root uma /opt/opendj/uma 2024-04-09 21:58:58,107 ERROR #12 CACHED 2024-04-09 21:58:58,107 ERROR 2024-04-09 21:58:58,107 ERROR #13 [ 3/10] WORKDIR /opt/opendj 2024-04-09 21:58:58,107 ERROR #13 CACHED 2024-04-09 21:58:58,107 ERROR 2024-04-09 21:58:58,107 ERROR #14 [10/10] RUN bin/setup.sh && bin/relax-security-settings.sh && rm bin/setup.sh bin/relax-security-settings.sh 2024-04-09 21:58:58,107 ERROR #14 CACHED 2024-04-09 21:58:58,107 ERROR 2024-04-09 21:58:58,107 ERROR #15 exporting to image 2024-04-09 21:58:58,107 ERROR #15 exporting layers done 2024-04-09 21:58:58,107 ERROR #15 writing image sha256:3c81eae135483e4583b6566a2da81ffbd3ff2e9b9c35273f182bee26c85defe1 done 2024-04-09 21:58:58,107 ERROR #15 naming to gcr.io/engineeringpit/lodestar-images/ds-idrepo:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903 done 2024-04-09 21:58:58,107 ERROR #15 DONE 0.0s 2024-04-09 21:58:58,256 INFO 2024-04-09 21:58:58,256 INFO [run_command]: kubectl config current-context 2024-04-09 21:58:58,349 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:58:58,350 DEBUG --- stdout --- 2024-04-09 21:58:58,350 DEBUG gke_engineeringpit_us-east1-d_xlou-cdm 2024-04-09 21:58:58,350 DEBUG --- stderr --- 2024-04-09 21:58:58,350 DEBUG 2024-04-09 21:58:58,350 INFO 2024-04-09 21:58:58,350 INFO 2024-04-09 21:58:58,350 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/forgeops build am --config-profile=cdk --push-to gcr.io/engineeringpit/lodestar-images --tag=xlou-7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 2024-04-09 21:58:58,350 DEBUG [run_command]: env={'HOME': '/home/xslou', 'PATH': '/mnt/disks/data/xslou/lodestar-fork/ext/bin:/home/xslou/.local/bin:/home/xslou/bin:~/bin:~/workshop/lodestar-fork/ext/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'DOCKER_SCAN_SUGGEST': 'false'} 2024-04-09 21:59:01,168 INFO The push refers to repository [gcr.io/engineeringpit/lodestar-images/am] 2024-04-09 21:59:01,269 INFO 5f70bf18a086: Preparing 2024-04-09 21:59:01,369 INFO fdccc8455149: Preparing 2024-04-09 21:59:01,470 INFO a9b45256d71b: Preparing 2024-04-09 21:59:01,570 INFO 5f70bf18a086: Preparing 2024-04-09 21:59:01,671 INFO 9b359eece558: Preparing 2024-04-09 21:59:01,772 INFO f8ed66db8640: Preparing 2024-04-09 21:59:01,872 INFO f8a13de73039: Preparing 2024-04-09 21:59:01,973 INFO ca1d1f53a9a4: Preparing 2024-04-09 21:59:02,073 INFO 5f70bf18a086: Preparing 2024-04-09 21:59:02,174 INFO fc2e53d69d90: Preparing 2024-04-09 21:59:02,275 INFO b6a67c1306d1: Preparing 2024-04-09 21:59:02,375 INFO 2a2bf9915da1: Preparing 2024-04-09 21:59:02,476 INFO e36f47a1fb68: Preparing 2024-04-09 21:59:02,577 INFO 5f70bf18a086: Preparing 2024-04-09 21:59:02,677 INFO 678d36695c66: Preparing 2024-04-09 21:59:02,778 INFO f3d362cb39a5: Preparing 2024-04-09 21:59:02,878 INFO 0e7fe5273fc8: Preparing 2024-04-09 21:59:02,979 INFO d095ec7c4a83: Preparing 2024-04-09 21:59:03,079 INFO 8ff404ad3685: Preparing 2024-04-09 21:59:03,180 INFO 5f70bf18a086: Preparing 2024-04-09 21:59:03,280 INFO a41e73d4de9b: Preparing 2024-04-09 21:59:03,381 INFO b7eb19b56fa1: Preparing 2024-04-09 21:59:03,481 INFO fdbf4ba0353b: Preparing 2024-04-09 21:59:03,582 INFO ac3383977054: Preparing 2024-04-09 21:59:03,682 INFO 4b71a6a73039: Preparing 2024-04-09 21:59:03,783 INFO 9119e3851877: Preparing 2024-04-09 21:59:03,884 INFO a7e71257196f: Preparing 2024-04-09 21:59:03,984 INFO f4756e9dcbbe: Preparing 2024-04-09 21:59:04,085 INFO ab29bfc1c5dc: Preparing 2024-04-09 21:59:04,186 INFO c3ae5d2e71e7: Preparing 2024-04-09 21:59:04,286 INFO 5437c9a49a2a: Preparing 2024-04-09 21:59:04,387 INFO ea84394c924c: Preparing 2024-04-09 21:59:04,488 INFO e512862c1c51: Preparing 2024-04-09 21:59:04,588 INFO 5f70bf18a086: Preparing 2024-04-09 21:59:04,689 INFO c68cc2df49b5: Preparing 2024-04-09 21:59:04,789 INFO ae3c983339ae: Preparing 2024-04-09 21:59:04,890 INFO 5f70bf18a086: Preparing 2024-04-09 21:59:04,990 INFO 3538abc054e4: Preparing 2024-04-09 21:59:05,091 INFO cd6ebe3e7379: Preparing 2024-04-09 21:59:05,191 INFO dc2d5546d301: Preparing 2024-04-09 21:59:05,292 INFO 0db6428be520: Preparing 2024-04-09 21:59:05,392 INFO 293d674cfca1: Preparing 2024-04-09 21:59:05,493 INFO 2e34c6d2bb66: Preparing 2024-04-09 21:59:05,593 INFO 72361db6ff3c: Preparing 2024-04-09 21:59:05,694 INFO 8a2d41061b80: Preparing 2024-04-09 21:59:05,794 INFO 167d9a67eab6: Preparing 2024-04-09 21:59:05,895 INFO 17031737aa00: Preparing 2024-04-09 21:59:05,995 INFO 5498e8c22f69: Preparing 2024-04-09 21:59:06,096 INFO f8a13de73039: Waiting 2024-04-09 21:59:06,196 INFO ca1d1f53a9a4: Waiting 2024-04-09 21:59:06,297 INFO fc2e53d69d90: Waiting 2024-04-09 21:59:06,397 INFO b6a67c1306d1: Waiting 2024-04-09 21:59:06,498 INFO 2a2bf9915da1: Waiting 2024-04-09 21:59:06,599 INFO e36f47a1fb68: Waiting 2024-04-09 21:59:06,699 INFO 678d36695c66: Waiting 2024-04-09 21:59:06,800 INFO f3d362cb39a5: Waiting 2024-04-09 21:59:06,900 INFO 0e7fe5273fc8: Waiting 2024-04-09 21:59:07,001 INFO d095ec7c4a83: Waiting 2024-04-09 21:59:07,101 INFO 8ff404ad3685: Waiting 2024-04-09 21:59:07,202 INFO a41e73d4de9b: Waiting 2024-04-09 21:59:07,302 INFO b7eb19b56fa1: Waiting 2024-04-09 21:59:07,403 INFO fdbf4ba0353b: Waiting 2024-04-09 21:59:07,504 INFO ac3383977054: Waiting 2024-04-09 21:59:07,605 INFO 4b71a6a73039: Waiting 2024-04-09 21:59:07,705 INFO 9119e3851877: Waiting 2024-04-09 21:59:07,806 INFO a7e71257196f: Waiting 2024-04-09 21:59:07,906 INFO f4756e9dcbbe: Waiting 2024-04-09 21:59:08,007 INFO ab29bfc1c5dc: Waiting 2024-04-09 21:59:08,107 INFO c3ae5d2e71e7: Waiting 2024-04-09 21:59:08,208 INFO 5437c9a49a2a: Waiting 2024-04-09 21:59:08,308 INFO ea84394c924c: Waiting 2024-04-09 21:59:08,409 INFO e512862c1c51: Waiting 2024-04-09 21:59:08,509 INFO c68cc2df49b5: Waiting 2024-04-09 21:59:08,610 INFO ae3c983339ae: Waiting 2024-04-09 21:59:08,710 INFO 3538abc054e4: Waiting 2024-04-09 21:59:08,811 INFO cd6ebe3e7379: Waiting 2024-04-09 21:59:08,911 INFO dc2d5546d301: Waiting 2024-04-09 21:59:09,012 INFO 0db6428be520: Waiting 2024-04-09 21:59:09,112 INFO 293d674cfca1: Waiting 2024-04-09 21:59:09,213 INFO 2e34c6d2bb66: Waiting 2024-04-09 21:59:09,314 INFO 72361db6ff3c: Waiting 2024-04-09 21:59:09,414 INFO 8a2d41061b80: Waiting 2024-04-09 21:59:09,515 INFO 167d9a67eab6: Waiting 2024-04-09 21:59:09,615 INFO 17031737aa00: Waiting 2024-04-09 21:59:09,716 INFO 5498e8c22f69: Waiting 2024-04-09 21:59:09,816 INFO a9b45256d71b: Layer already exists 2024-04-09 21:59:09,917 INFO 5f70bf18a086: Layer already exists 2024-04-09 21:59:10,017 INFO f8ed66db8640: Layer already exists 2024-04-09 21:59:10,118 INFO 9b359eece558: Layer already exists 2024-04-09 21:59:10,218 INFO fdccc8455149: Layer already exists 2024-04-09 21:59:10,319 INFO f8a13de73039: Layer already exists 2024-04-09 21:59:10,419 INFO fc2e53d69d90: Layer already exists 2024-04-09 21:59:10,520 INFO ca1d1f53a9a4: Layer already exists 2024-04-09 21:59:10,620 INFO b6a67c1306d1: Layer already exists 2024-04-09 21:59:10,721 INFO 2a2bf9915da1: Layer already exists 2024-04-09 21:59:10,821 INFO e36f47a1fb68: Layer already exists 2024-04-09 21:59:10,922 INFO 678d36695c66: Layer already exists 2024-04-09 21:59:11,022 INFO d095ec7c4a83: Layer already exists 2024-04-09 21:59:11,123 INFO f3d362cb39a5: Layer already exists 2024-04-09 21:59:11,223 INFO 0e7fe5273fc8: Layer already exists 2024-04-09 21:59:11,324 INFO 8ff404ad3685: Layer already exists 2024-04-09 21:59:11,424 INFO a41e73d4de9b: Layer already exists 2024-04-09 21:59:11,525 INFO b7eb19b56fa1: Layer already exists 2024-04-09 21:59:11,626 INFO fdbf4ba0353b: Layer already exists 2024-04-09 21:59:11,726 INFO ac3383977054: Layer already exists 2024-04-09 21:59:11,827 INFO 4b71a6a73039: Layer already exists 2024-04-09 21:59:11,928 INFO 9119e3851877: Layer already exists 2024-04-09 21:59:12,028 INFO a7e71257196f: Layer already exists 2024-04-09 21:59:12,129 INFO f4756e9dcbbe: Layer already exists 2024-04-09 21:59:12,229 INFO ab29bfc1c5dc: Layer already exists 2024-04-09 21:59:12,330 INFO c3ae5d2e71e7: Layer already exists 2024-04-09 21:59:12,431 INFO 5437c9a49a2a: Layer already exists 2024-04-09 21:59:12,533 INFO ea84394c924c: Layer already exists 2024-04-09 21:59:12,634 INFO e512862c1c51: Layer already exists 2024-04-09 21:59:12,734 INFO c68cc2df49b5: Layer already exists 2024-04-09 21:59:12,835 INFO ae3c983339ae: Layer already exists 2024-04-09 21:59:12,936 INFO 3538abc054e4: Layer already exists 2024-04-09 21:59:13,036 INFO cd6ebe3e7379: Layer already exists 2024-04-09 21:59:13,137 INFO dc2d5546d301: Layer already exists 2024-04-09 21:59:13,238 INFO 0db6428be520: Layer already exists 2024-04-09 21:59:13,338 INFO 293d674cfca1: Layer already exists 2024-04-09 21:59:13,439 INFO 2e34c6d2bb66: Layer already exists 2024-04-09 21:59:13,540 INFO 72361db6ff3c: Layer already exists 2024-04-09 21:59:13,640 INFO 167d9a67eab6: Layer already exists 2024-04-09 21:59:13,741 INFO 8a2d41061b80: Layer already exists 2024-04-09 21:59:13,842 INFO 17031737aa00: Layer already exists 2024-04-09 21:59:13,942 INFO 5498e8c22f69: Layer already exists 2024-04-09 21:59:14,043 INFO xlou-7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88: digest: sha256:2be3c1123081b3154309094b4ae185679e1a413717c7efc78b0163cad7ddbc7f size: 10325 2024-04-09 21:59:14,144 INFO Updated the image_defaulter with your new image for am: "gcr.io/engineeringpit/lodestar-images/am:xlou-7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88". 2024-04-09 21:59:14,244 ERROR #0 building with "default" instance using docker driver 2024-04-09 21:59:14,245 ERROR 2024-04-09 21:59:14,245 ERROR #1 [internal] load build definition from Dockerfile 2024-04-09 21:59:14,245 ERROR #1 transferring dockerfile: 489B done 2024-04-09 21:59:14,245 ERROR #1 DONE 0.0s 2024-04-09 21:59:14,245 ERROR 2024-04-09 21:59:14,245 ERROR #2 [internal] load metadata for gcr.io/forgerock-io/am-cdk/pit1:7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 2024-04-09 21:59:14,245 ERROR #2 DONE 0.4s 2024-04-09 21:59:14,245 ERROR 2024-04-09 21:59:14,245 ERROR #3 [internal] load .dockerignore 2024-04-09 21:59:14,245 ERROR #3 transferring context: 2B done 2024-04-09 21:59:14,245 ERROR #3 DONE 0.0s 2024-04-09 21:59:14,245 ERROR 2024-04-09 21:59:14,245 ERROR #4 [1/5] FROM gcr.io/forgerock-io/am-cdk/pit1:7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88@sha256:4e100a71900966d29bac8ee0c95dfa55d9a011872a9409720b83e60d819a0b22 2024-04-09 21:59:14,245 ERROR #4 DONE 0.0s 2024-04-09 21:59:14,245 ERROR 2024-04-09 21:59:14,245 ERROR #5 [internal] load build context 2024-04-09 21:59:14,245 ERROR #5 transferring context: 3.83kB 0.0s done 2024-04-09 21:59:14,245 ERROR #5 DONE 0.0s 2024-04-09 21:59:14,245 ERROR 2024-04-09 21:59:14,245 ERROR #6 [2/5] RUN echo "\033[0;36m*** Building 'cdk' profile ***\033[0m" 2024-04-09 21:59:14,245 ERROR #6 CACHED 2024-04-09 21:59:14,245 ERROR 2024-04-09 21:59:14,245 ERROR #7 [3/5] COPY --chown=forgerock:root config-profiles/cdk/ /home/forgerock/openam/ 2024-04-09 21:59:14,245 ERROR #7 CACHED 2024-04-09 21:59:14,245 ERROR 2024-04-09 21:59:14,245 ERROR #8 [4/5] COPY --chown=forgerock:root *.sh /home/forgerock/ 2024-04-09 21:59:14,245 ERROR #8 CACHED 2024-04-09 21:59:14,246 ERROR 2024-04-09 21:59:14,246 ERROR #9 [5/5] WORKDIR /home/forgerock 2024-04-09 21:59:14,246 ERROR #9 CACHED 2024-04-09 21:59:14,246 ERROR 2024-04-09 21:59:14,246 ERROR #10 exporting to image 2024-04-09 21:59:14,246 ERROR #10 exporting layers done 2024-04-09 21:59:14,246 ERROR #10 writing image sha256:ba69dd7d7b3dea29440a969d3ff63f9192a9fdae0cf65bec08fa695ffd007af9 done 2024-04-09 21:59:14,246 ERROR #10 naming to gcr.io/engineeringpit/lodestar-images/am:xlou-7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 done 2024-04-09 21:59:14,246 ERROR #10 DONE 0.0s 2024-04-09 21:59:14,373 INFO 2024-04-09 21:59:14,373 INFO [run_command]: kubectl config current-context 2024-04-09 21:59:14,457 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:59:14,457 DEBUG --- stdout --- 2024-04-09 21:59:14,457 DEBUG gke_engineeringpit_us-east1-d_xlou-cdm 2024-04-09 21:59:14,457 DEBUG --- stderr --- 2024-04-09 21:59:14,457 DEBUG 2024-04-09 21:59:14,457 INFO 2024-04-09 21:59:14,458 INFO 2024-04-09 21:59:14,458 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/forgeops build amster --config-profile=cdk --push-to gcr.io/engineeringpit/lodestar-images --tag=xlou-7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 2024-04-09 21:59:14,458 DEBUG [run_command]: env={'HOME': '/home/xslou', 'PATH': '/mnt/disks/data/xslou/lodestar-fork/ext/bin:/home/xslou/.local/bin:/home/xslou/bin:~/bin:~/workshop/lodestar-fork/ext/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'DOCKER_SCAN_SUGGEST': 'false'} 2024-04-09 21:59:17,181 INFO The push refers to repository [gcr.io/engineeringpit/lodestar-images/amster] 2024-04-09 21:59:17,281 INFO bc81b41626ba: Preparing 2024-04-09 21:59:17,382 INFO 7bfb461145c1: Preparing 2024-04-09 21:59:17,483 INFO ba523ed879bb: Preparing 2024-04-09 21:59:17,583 INFO 38e5be0e5a27: Preparing 2024-04-09 21:59:17,684 INFO 5ab155e27bf9: Preparing 2024-04-09 21:59:17,784 INFO 5f70bf18a086: Preparing 2024-04-09 21:59:17,885 INFO a8e50f4baccd: Preparing 2024-04-09 21:59:17,985 INFO 872f2f208320: Preparing 2024-04-09 21:59:18,086 INFO 5ec14973d142: Preparing 2024-04-09 21:59:18,186 INFO ac22e735f40d: Preparing 2024-04-09 21:59:18,287 INFO b8639291fa25: Preparing 2024-04-09 21:59:18,388 INFO ccb2c633d524: Preparing 2024-04-09 21:59:18,488 INFO 62622daf7781: Preparing 2024-04-09 21:59:18,589 INFO c407a0393a48: Preparing 2024-04-09 21:59:18,689 INFO 78a2fe2d8a5e: Preparing 2024-04-09 21:59:18,790 INFO 3c8879ab2cf2: Preparing 2024-04-09 21:59:18,890 INFO 5f70bf18a086: Waiting 2024-04-09 21:59:18,991 INFO a8e50f4baccd: Waiting 2024-04-09 21:59:19,091 INFO 872f2f208320: Waiting 2024-04-09 21:59:19,192 INFO 5ec14973d142: Waiting 2024-04-09 21:59:19,292 INFO ac22e735f40d: Waiting 2024-04-09 21:59:19,393 INFO b8639291fa25: Waiting 2024-04-09 21:59:19,493 INFO ccb2c633d524: Waiting 2024-04-09 21:59:19,594 INFO 62622daf7781: Waiting 2024-04-09 21:59:19,694 INFO c407a0393a48: Waiting 2024-04-09 21:59:19,795 INFO 78a2fe2d8a5e: Waiting 2024-04-09 21:59:19,895 INFO 3c8879ab2cf2: Waiting 2024-04-09 21:59:19,996 INFO 38e5be0e5a27: Layer already exists 2024-04-09 21:59:20,097 INFO ba523ed879bb: Layer already exists 2024-04-09 21:59:20,197 INFO bc81b41626ba: Layer already exists 2024-04-09 21:59:20,298 INFO 5ab155e27bf9: Layer already exists 2024-04-09 21:59:20,399 INFO 7bfb461145c1: Layer already exists 2024-04-09 21:59:20,499 INFO a8e50f4baccd: Layer already exists 2024-04-09 21:59:20,600 INFO 5f70bf18a086: Layer already exists 2024-04-09 21:59:20,700 INFO 872f2f208320: Layer already exists 2024-04-09 21:59:20,801 INFO 5ec14973d142: Layer already exists 2024-04-09 21:59:20,902 INFO ac22e735f40d: Layer already exists 2024-04-09 21:59:21,002 INFO b8639291fa25: Layer already exists 2024-04-09 21:59:21,103 INFO 62622daf7781: Layer already exists 2024-04-09 21:59:21,203 INFO ccb2c633d524: Layer already exists 2024-04-09 21:59:21,304 INFO c407a0393a48: Layer already exists 2024-04-09 21:59:21,404 INFO 78a2fe2d8a5e: Layer already exists 2024-04-09 21:59:21,505 INFO 3c8879ab2cf2: Layer already exists 2024-04-09 21:59:21,605 INFO xlou-7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88: digest: sha256:86c065ae4fb93153381fcc2f2ee5cf2c6b6db6b726ed4514d6145325e2377d02 size: 3666 2024-04-09 21:59:21,706 INFO Embedding AM dynamic configs in the amster image is deprecated. Manage AM dynamic config with the bin/amster cmd instead. 2024-04-09 21:59:21,806 INFO Updated the image_defaulter with your new image for amster: "gcr.io/engineeringpit/lodestar-images/amster:xlou-7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88". 2024-04-09 21:59:21,907 ERROR #0 building with "default" instance using docker driver 2024-04-09 21:59:21,907 ERROR 2024-04-09 21:59:21,907 ERROR #1 [internal] load build definition from Dockerfile 2024-04-09 21:59:21,907 ERROR #1 transferring dockerfile: 802B done 2024-04-09 21:59:21,907 ERROR #1 DONE 0.0s 2024-04-09 21:59:21,907 ERROR 2024-04-09 21:59:21,907 ERROR #2 [internal] load metadata for gcr.io/forgerock-io/amster/pit1:7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 2024-04-09 21:59:21,907 ERROR #2 DONE 0.4s 2024-04-09 21:59:21,907 ERROR 2024-04-09 21:59:21,907 ERROR #3 [internal] load .dockerignore 2024-04-09 21:59:21,907 ERROR #3 transferring context: 2B done 2024-04-09 21:59:21,908 ERROR #3 DONE 0.0s 2024-04-09 21:59:21,908 ERROR 2024-04-09 21:59:21,908 ERROR #4 [1/6] FROM gcr.io/forgerock-io/amster/pit1:7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88@sha256:b6ec1a79d909578eb16120b83565c11234cd822c312dea8f776359bb15178dc5 2024-04-09 21:59:21,908 ERROR #4 DONE 0.0s 2024-04-09 21:59:21,908 ERROR 2024-04-09 21:59:21,908 ERROR #5 [internal] load build context 2024-04-09 21:59:21,908 ERROR #5 transferring context: 41.18kB done 2024-04-09 21:59:21,908 ERROR #5 DONE 0.0s 2024-04-09 21:59:21,908 ERROR 2024-04-09 21:59:21,908 ERROR #6 [4/6] COPY --chown=forgerock:root config-profiles/cdk/ /opt/amster 2024-04-09 21:59:21,908 ERROR #6 CACHED 2024-04-09 21:59:21,908 ERROR 2024-04-09 21:59:21,908 ERROR #7 [5/6] COPY --chown=forgerock:root scripts /opt/amster 2024-04-09 21:59:21,908 ERROR #7 CACHED 2024-04-09 21:59:21,908 ERROR 2024-04-09 21:59:21,908 ERROR #8 [2/6] RUN apt-get update && apt-get install -y openldap-utils jq inotify-tools && apt-get clean && rm -r /var/lib/apt/lists /var/cache/apt/archives 2024-04-09 21:59:21,908 ERROR #8 CACHED 2024-04-09 21:59:21,908 ERROR 2024-04-09 21:59:21,908 ERROR #9 [3/6] RUN echo "\033[0;36m*** Building 'cdk' profile ***\033[0m" 2024-04-09 21:59:21,908 ERROR #9 CACHED 2024-04-09 21:59:21,908 ERROR 2024-04-09 21:59:21,908 ERROR #10 [6/6] RUN chmod 777 /opt/amster 2024-04-09 21:59:21,908 ERROR #10 CACHED 2024-04-09 21:59:21,908 ERROR 2024-04-09 21:59:21,908 ERROR #11 exporting to image 2024-04-09 21:59:21,908 ERROR #11 exporting layers done 2024-04-09 21:59:21,908 ERROR #11 writing image sha256:7c61dda5c38009637eb900ee57dadbbc41eb88053a594cda236935caabf2125a done 2024-04-09 21:59:21,908 ERROR #11 naming to gcr.io/engineeringpit/lodestar-images/amster:xlou-7.6.0-46abf9899004dcd77f0fdad8a152a3749b4a9c88 done 2024-04-09 21:59:21,908 ERROR #11 DONE 0.0s 2024-04-09 21:59:22,035 INFO 2024-04-09 21:59:22,035 INFO [run_command]: kubectl config current-context 2024-04-09 21:59:22,108 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:59:22,109 DEBUG --- stdout --- 2024-04-09 21:59:22,109 DEBUG gke_engineeringpit_us-east1-d_xlou-cdm 2024-04-09 21:59:22,109 DEBUG --- stderr --- 2024-04-09 21:59:22,109 DEBUG 2024-04-09 21:59:22,109 INFO 2024-04-09 21:59:22,109 INFO 2024-04-09 21:59:22,109 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/forgeops build idm --config-profile=cdk --push-to gcr.io/engineeringpit/lodestar-images --tag=xlou-7.6.0-5a54de0b49885680977e8ef4134f710026e856bf 2024-04-09 21:59:22,109 DEBUG [run_command]: env={'HOME': '/home/xslou', 'PATH': '/mnt/disks/data/xslou/lodestar-fork/ext/bin:/home/xslou/.local/bin:/home/xslou/bin:~/bin:~/workshop/lodestar-fork/ext/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'DOCKER_SCAN_SUGGEST': 'false'} 2024-04-09 21:59:41,775 INFO The push refers to repository [gcr.io/engineeringpit/lodestar-images/idm] 2024-04-09 21:59:41,876 INFO a2c78c79d85f: Preparing 2024-04-09 21:59:41,976 INFO 163e3668d52a: Preparing 2024-04-09 21:59:42,077 INFO e2776e630fa3: Preparing 2024-04-09 21:59:42,177 INFO 9081a8f0037e: Preparing 2024-04-09 21:59:42,278 INFO 9081a8f0037e: Preparing 2024-04-09 21:59:42,378 INFO d5c7f3d02845: Preparing 2024-04-09 21:59:42,479 INFO 5f70bf18a086: Preparing 2024-04-09 21:59:42,579 INFO 4c15795abb6e: Preparing 2024-04-09 21:59:42,680 INFO a4419a24c4d1: Preparing 2024-04-09 21:59:42,781 INFO 7b53f1195e30: Preparing 2024-04-09 21:59:42,881 INFO 10b76d5003a7: Preparing 2024-04-09 21:59:42,982 INFO a1d1648cfe17: Preparing 2024-04-09 21:59:43,082 INFO ad62640b3ee3: Preparing 2024-04-09 21:59:43,183 INFO 3c8879ab2cf2: Preparing 2024-04-09 21:59:43,283 INFO 5f70bf18a086: Waiting 2024-04-09 21:59:43,384 INFO 4c15795abb6e: Waiting 2024-04-09 21:59:43,484 INFO a4419a24c4d1: Waiting 2024-04-09 21:59:43,585 INFO 7b53f1195e30: Waiting 2024-04-09 21:59:43,685 INFO 10b76d5003a7: Waiting 2024-04-09 21:59:43,786 INFO a1d1648cfe17: Waiting 2024-04-09 21:59:43,887 INFO ad62640b3ee3: Waiting 2024-04-09 21:59:43,987 INFO 3c8879ab2cf2: Waiting 2024-04-09 21:59:44,088 INFO a2c78c79d85f: Pushed 2024-04-09 21:59:44,188 INFO 9081a8f0037e: Pushed 2024-04-09 21:59:44,289 INFO e2776e630fa3: Pushed 2024-04-09 21:59:44,390 INFO 163e3668d52a: Pushed 2024-04-09 21:59:44,490 INFO d5c7f3d02845: Pushed 2024-04-09 21:59:44,591 INFO 5f70bf18a086: Layer already exists 2024-04-09 21:59:44,692 INFO 4c15795abb6e: Layer already exists 2024-04-09 21:59:44,792 INFO a4419a24c4d1: Layer already exists 2024-04-09 21:59:44,893 INFO 7b53f1195e30: Layer already exists 2024-04-09 21:59:44,994 INFO 10b76d5003a7: Layer already exists 2024-04-09 21:59:45,094 INFO a1d1648cfe17: Layer already exists 2024-04-09 21:59:45,195 INFO ad62640b3ee3: Layer already exists 2024-04-09 21:59:45,296 INFO 3c8879ab2cf2: Layer already exists 2024-04-09 21:59:46,099 INFO xlou-7.6.0-5a54de0b49885680977e8ef4134f710026e856bf: digest: sha256:4f8997e76d4f1490a58163bb7de33005103b0fd63d957b3690d40c98734ef1f3 size: 3240 2024-04-09 21:59:46,199 INFO Updated the image_defaulter with your new image for idm: "gcr.io/engineeringpit/lodestar-images/idm:xlou-7.6.0-5a54de0b49885680977e8ef4134f710026e856bf". 2024-04-09 21:59:46,300 ERROR #0 building with "default" instance using docker driver 2024-04-09 21:59:46,300 ERROR 2024-04-09 21:59:46,300 ERROR #1 [internal] load build definition from Dockerfile 2024-04-09 21:59:46,300 ERROR #1 transferring dockerfile: 1.08kB done 2024-04-09 21:59:46,301 ERROR #1 DONE 0.0s 2024-04-09 21:59:46,301 ERROR 2024-04-09 21:59:46,301 ERROR #2 [internal] load metadata for gcr.io/forgerock-io/idm-cdk/pit1:7.6.0-5a54de0b49885680977e8ef4134f710026e856bf 2024-04-09 21:59:46,301 ERROR #2 DONE 1.5s 2024-04-09 21:59:46,301 ERROR 2024-04-09 21:59:46,301 ERROR #3 [internal] load .dockerignore 2024-04-09 21:59:46,301 ERROR #3 transferring context: 2B done 2024-04-09 21:59:46,301 ERROR #3 DONE 0.0s 2024-04-09 21:59:46,301 ERROR 2024-04-09 21:59:46,301 ERROR #4 [internal] load build context 2024-04-09 21:59:46,301 ERROR #4 transferring context: 281.63kB 0.0s done 2024-04-09 21:59:46,301 ERROR #4 DONE 0.0s 2024-04-09 21:59:46,301 ERROR 2024-04-09 21:59:46,301 ERROR #5 [1/7] FROM gcr.io/forgerock-io/idm-cdk/pit1:7.6.0-5a54de0b49885680977e8ef4134f710026e856bf@sha256:6fa15a37bde1921974d371721c0ba3288ffa25d32b897a3c2502f35b3818d40c 2024-04-09 21:59:46,301 ERROR #5 resolve gcr.io/forgerock-io/idm-cdk/pit1:7.6.0-5a54de0b49885680977e8ef4134f710026e856bf@sha256:6fa15a37bde1921974d371721c0ba3288ffa25d32b897a3c2502f35b3818d40c 0.0s done 2024-04-09 21:59:46,301 ERROR #5 sha256:0ff0c13e1208252f10b09095b01a76b0d77a3c9fe2641e03fef062deb082afd1 0B / 601B 0.1s 2024-04-09 21:59:46,301 ERROR #5 sha256:6fa15a37bde1921974d371721c0ba3288ffa25d32b897a3c2502f35b3818d40c 685B / 685B done 2024-04-09 21:59:46,301 ERROR #5 sha256:7c1e814f1e069aed26c5d3a3f6aca16f7619bbc3988671247de3d54a3b50087d 1.88kB / 1.88kB done 2024-04-09 21:59:46,301 ERROR #5 sha256:46077d1d952ba64868ea424b7ea731bffbee073d0d77673e7292e23983f8c449 0B / 21.49MB 0.1s 2024-04-09 21:59:46,301 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 0B / 336.38MB 0.1s 2024-04-09 21:59:46,301 ERROR #5 sha256:5f5a7602c3bce89902111eb682329a17b2cb88c3aa9d3464c546a049fd814ffe 3.63kB / 3.63kB done 2024-04-09 21:59:46,301 ERROR #5 sha256:0ff0c13e1208252f10b09095b01a76b0d77a3c9fe2641e03fef062deb082afd1 601B / 601B 0.3s done 2024-04-09 21:59:46,301 ERROR #5 sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0B / 32B 0.4s 2024-04-09 21:59:46,301 ERROR #5 sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 32B / 32B 0.5s done 2024-04-09 21:59:46,301 ERROR #5 sha256:46077d1d952ba64868ea424b7ea731bffbee073d0d77673e7292e23983f8c449 2.10MB / 21.49MB 0.7s 2024-04-09 21:59:46,301 ERROR #5 sha256:46077d1d952ba64868ea424b7ea731bffbee073d0d77673e7292e23983f8c449 6.29MB / 21.49MB 0.9s 2024-04-09 21:59:46,301 ERROR #5 sha256:46077d1d952ba64868ea424b7ea731bffbee073d0d77673e7292e23983f8c449 8.13MB / 21.49MB 1.1s 2024-04-09 21:59:46,301 ERROR #5 sha256:46077d1d952ba64868ea424b7ea731bffbee073d0d77673e7292e23983f8c449 16.78MB / 21.49MB 1.3s 2024-04-09 21:59:46,301 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 39.85MB / 336.38MB 1.3s 2024-04-09 21:59:46,301 ERROR #5 sha256:46077d1d952ba64868ea424b7ea731bffbee073d0d77673e7292e23983f8c449 21.49MB / 21.49MB 1.4s 2024-04-09 21:59:46,301 ERROR #5 extracting sha256:46077d1d952ba64868ea424b7ea731bffbee073d0d77673e7292e23983f8c449 2024-04-09 21:59:46,302 ERROR #5 sha256:46077d1d952ba64868ea424b7ea731bffbee073d0d77673e7292e23983f8c449 21.49MB / 21.49MB 1.4s done 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 61.87MB / 336.38MB 1.7s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 80.74MB / 336.38MB 2.0s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 99.80MB / 336.38MB 2.4s 2024-04-09 21:59:46,302 ERROR #5 extracting sha256:46077d1d952ba64868ea424b7ea731bffbee073d0d77673e7292e23983f8c449 0.9s done 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 121.63MB / 336.38MB 2.6s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 147.73MB / 336.38MB 3.0s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 173.33MB / 336.38MB 3.4s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 198.18MB / 336.38MB 3.8s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 226.49MB / 336.38MB 4.2s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 251.66MB / 336.38MB 4.7s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 271.58MB / 336.38MB 5.0s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 296.75MB / 336.38MB 5.4s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 327.16MB / 336.38MB 5.9s 2024-04-09 21:59:46,302 ERROR #5 sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 336.38MB / 336.38MB 7.4s done 2024-04-09 21:59:46,302 ERROR #5 extracting sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 0.1s 2024-04-09 21:59:46,302 ERROR #5 extracting sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 5.1s 2024-04-09 21:59:46,302 ERROR #5 extracting sha256:4e8638633e0ba72e12d5ce2749cae29900413786de4d56463792db152644304a 5.4s done 2024-04-09 21:59:46,302 ERROR #5 extracting sha256:0ff0c13e1208252f10b09095b01a76b0d77a3c9fe2641e03fef062deb082afd1 2024-04-09 21:59:46,302 ERROR #5 extracting sha256:0ff0c13e1208252f10b09095b01a76b0d77a3c9fe2641e03fef062deb082afd1 done 2024-04-09 21:59:46,302 ERROR #5 extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 done 2024-04-09 21:59:46,302 ERROR #5 DONE 13.1s 2024-04-09 21:59:46,302 ERROR 2024-04-09 21:59:46,302 ERROR #6 [2/7] COPY debian-buster-sources.list /etc/apt/sources.list 2024-04-09 21:59:46,302 ERROR #6 DONE 1.7s 2024-04-09 21:59:46,302 ERROR 2024-04-09 21:59:46,302 ERROR #7 [3/7] RUN rm -f bundle/org.apache.felix.webconsole*.jar && rm -f bundle/openidm-felix-webconsole-*.jar 2024-04-09 21:59:46,302 ERROR #7 DONE 0.5s 2024-04-09 21:59:46,302 ERROR 2024-04-09 21:59:46,303 ERROR #8 [4/7] RUN echo "\033[0;36m*** Building 'cdk' profile ***\033[0m" 2024-04-09 21:59:46,303 ERROR #8 0.310 *** Building 'cdk' profile *** 2024-04-09 21:59:46,303 ERROR #8 DONE 0.4s 2024-04-09 21:59:46,303 ERROR 2024-04-09 21:59:46,303 ERROR #9 [5/7] COPY --chown=forgerock:root config-profiles/cdk/ /opt/openidm 2024-04-09 21:59:46,303 ERROR #9 DONE 0.1s 2024-04-09 21:59:46,303 ERROR 2024-04-09 21:59:46,303 ERROR #10 [6/7] COPY --chown=forgerock:root bin/import-pem-certs.sh /opt/openidm/bin 2024-04-09 21:59:46,303 ERROR #10 DONE 0.1s 2024-04-09 21:59:46,303 ERROR 2024-04-09 21:59:46,303 ERROR #11 [7/7] COPY --chown=forgerock:root . /opt/openidm 2024-04-09 21:59:46,303 ERROR #11 DONE 0.1s 2024-04-09 21:59:46,303 ERROR 2024-04-09 21:59:46,303 ERROR #12 exporting to image 2024-04-09 21:59:46,303 ERROR #12 exporting layers 0.1s done 2024-04-09 21:59:46,303 ERROR #12 writing image sha256:b17d9d8b592b64e50b86b326185159ba908491c73ffed27df642694668df9d36 done 2024-04-09 21:59:46,303 ERROR #12 naming to gcr.io/engineeringpit/lodestar-images/idm:xlou-7.6.0-5a54de0b49885680977e8ef4134f710026e856bf done 2024-04-09 21:59:46,303 ERROR #12 DONE 0.1s 2024-04-09 21:59:46,303 INFO 2024-04-09 21:59:46,303 INFO -------- Deploy product images using forgeops install command -------- 2024-04-09 21:59:46,496 INFO 2024-04-09 21:59:46,497 INFO [run_command]: kubectl config current-context 2024-04-09 21:59:46,570 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 21:59:46,571 DEBUG --- stdout --- 2024-04-09 21:59:46,571 DEBUG gke_engineeringpit_us-east1-d_xlou-cdm 2024-04-09 21:59:46,571 DEBUG --- stderr --- 2024-04-09 21:59:46,571 DEBUG 2024-04-09 21:59:46,571 INFO 2024-04-09 21:59:46,571 INFO 2024-04-09 21:59:46,571 INFO [run_command]: /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/bin/forgeops install --namespace=xlou --fqdn xlou.iam.xlou-cdm.engineeringpit.com --custom /mnt/disks/data/xslou/lodestar-fork/ext/forgeops_stack/kustomize/overlay/internal-profiles/medium-old --legacy all 2024-04-09 21:59:46,571 DEBUG [run_command]: env={'HOME': '/home/xslou', 'PATH': '/mnt/disks/data/xslou/lodestar-fork/ext/bin:/home/xslou/.local/bin:/home/xslou/bin:~/bin:~/workshop/lodestar-fork/ext/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'DOCKER_SCAN_SUGGEST': 'false'} 2024-04-09 21:59:50,591 INFO customresourcedefinition.apiextensions.k8s.io/secretagentconfigurations.secret-agent.secrets.forgerock.io condition met 2024-04-09 21:59:51,093 INFO deployment.apps/secret-agent-controller-manager condition met 2024-04-09 21:59:51,394 INFO NAME READY STATUS RESTARTS AGE 2024-04-09 21:59:51,495 INFO secret-agent-controller-manager-85df555854-lf2b8 2/2 Running 0 82d 2024-04-09 21:59:53,302 INFO configmap/platform-config created 2024-04-09 21:59:53,604 INFO ingress.networking.k8s.io/forgerock created 2024-04-09 21:59:53,905 INFO ingress.networking.k8s.io/ig created 2024-04-09 21:59:55,310 INFO certificate.cert-manager.io/ds-master-cert created 2024-04-09 21:59:55,511 INFO certificate.cert-manager.io/ds-ssl-cert created 2024-04-09 21:59:55,912 INFO issuer.cert-manager.io/selfsigned-issuer created 2024-04-09 21:59:56,213 INFO secretagentconfiguration.secret-agent.secrets.forgerock.io/forgerock-sac created 2024-04-09 21:59:56,314 INFO Checking cert-manager and related CRDs: cert-manager CRD found in cluster. 2024-04-09 21:59:56,414 INFO Checking secret-agent operator and related CRDs: secret-agent CRD found in cluster. 2024-04-09 21:59:56,515 INFO  2024-04-09 21:59:56,615 INFO Checking secret-agent operator is running... 2024-04-09 21:59:56,716 INFO secret-agent operator is running 2024-04-09 21:59:56,816 INFO 2024-04-09 21:59:56,917 INFO Installing component(s): ['all'] platform: "custom-old" in namespace: "xlou" from deployment manifests in "None". 2024-04-09 21:59:57,018 INFO  2024-04-09 21:59:57,119 INFO Deploying base.yaml. This is a one time activity. 2024-04-09 21:59:57,219 INFO  2024-04-09 21:59:57,320 INFO Waiting for K8s secrets. 2024-04-09 21:59:57,922 INFO Waiting for secret "am-env-secrets" to exist in the cluster: .done 2024-04-09 22:00:03,843 INFO Waiting for secret "idm-env-secrets" to exist in the cluster: ....done 2024-04-09 22:00:04,144 INFO Waiting for secret "ds-passwords" to exist in the cluster: done 2024-04-09 22:00:06,250 INFO Waiting for secret "ds-env-secrets" to exist in the cluster: secret/cloud-storage-credentials-cts created 2024-04-09 22:00:06,551 INFO secret/cloud-storage-credentials-idrepo created 2024-04-09 22:00:06,752 INFO service/ds-cts created 2024-04-09 22:00:07,054 INFO service/ds-idrepo created 2024-04-09 22:00:07,355 INFO statefulset.apps/ds-cts created 2024-04-09 22:00:07,657 INFO statefulset.apps/ds-idrepo created 2024-04-09 22:00:07,858 INFO job.batch/ldif-importer created 2024-04-09 22:00:07,958 INFO done 2024-04-09 22:00:08,059 INFO  2024-04-09 22:00:08,159 INFO Deploying ds.yaml. This includes all directory resources. 2024-04-09 22:00:08,260 INFO  2024-04-09 22:00:08,360 INFO Waiting for DS deployment. This can take a few minutes. First installation takes longer. 2024-04-09 22:00:08,761 INFO Waiting for statefulset "ds-idrepo" to exist in the cluster: Waiting for 3 pods to be ready... 2024-04-09 22:00:40,566 INFO Waiting for 2 pods to be ready... 2024-04-09 22:00:40,668 INFO Waiting for 2 pods to be ready... 2024-04-09 22:01:17,078 INFO Waiting for 1 pods to be ready... 2024-04-09 22:01:17,178 INFO Waiting for 1 pods to be ready... 2024-04-09 22:01:54,692 INFO statefulset rolling update complete 3 pods at revision ds-idrepo-76f9f698cf... 2024-04-09 22:01:54,792 INFO done 2024-04-09 22:01:55,594 INFO Waiting for Service Account Password Update: done 2024-04-09 22:01:56,497 INFO Waiting for statefulset "ds-cts" to exist in the cluster: statefulset rolling update complete 3 pods at revision ds-cts-bf5d4667... 2024-04-09 22:01:56,598 INFO done 2024-04-09 22:02:01,214 INFO Waiting for Service Account Password Update: configmap/am-logback created 2024-04-09 22:02:01,414 INFO configmap/amster-files created 2024-04-09 22:02:01,716 INFO configmap/idm created 2024-04-09 22:02:01,917 INFO configmap/idm-logging-properties created 2024-04-09 22:02:02,218 INFO service/am created 2024-04-09 22:02:02,419 INFO service/idm created 2024-04-09 22:02:02,720 INFO deployment.apps/am created 2024-04-09 22:02:03,021 INFO deployment.apps/idm created 2024-04-09 22:02:03,222 INFO job.batch/amster created 2024-04-09 22:02:05,228 INFO done 2024-04-09 22:02:05,328 INFO Cleaning up amster components. 2024-04-09 22:02:05,429 INFO  2024-04-09 22:02:05,529 INFO Deploying apps. 2024-04-09 22:02:05,630 INFO  2024-04-09 22:02:05,730 INFO Waiting for AM deployment. This can take a few minutes. First installation takes longer. 2024-04-09 22:02:33,915 INFO Waiting for deployment "am" to exist in the cluster: deployment.apps/am condition met 2024-04-09 22:02:34,216 INFO configmap/amster-retain created 2024-04-09 22:02:34,317 INFO done 2024-04-09 22:02:34,417 INFO  2024-04-09 22:02:34,518 INFO Waiting for amster job to complete. This can take several minutes. 2024-04-09 22:02:56,894 INFO Waiting for job "amster" to exist in the cluster: job.batch/amster condition met 2024-04-09 22:02:56,994 INFO done 2024-04-09 22:02:57,095 INFO  2024-04-09 22:02:57,196 INFO Waiting for IDM deployment. This can take a few minutes. First installation takes longer. 2024-04-09 22:03:03,716 INFO Waiting for deployment "idm" to exist in the cluster: pod/idm-6bfb7c96cc-qklk9 condition met 2024-04-09 22:03:03,917 INFO pod/idm-6bfb7c96cc-vsb58 condition met 2024-04-09 22:03:05,521 INFO service/admin-ui created 2024-04-09 22:03:05,722 INFO service/end-user-ui created 2024-04-09 22:03:06,027 INFO service/login-ui created 2024-04-09 22:03:06,227 INFO deployment.apps/admin-ui created 2024-04-09 22:03:06,529 INFO deployment.apps/end-user-ui created 2024-04-09 22:03:06,730 INFO deployment.apps/login-ui created 2024-04-09 22:03:06,830 INFO done 2024-04-09 22:03:06,931 INFO  2024-04-09 22:03:07,032 INFO Deploying UI. 2024-04-09 22:03:07,132 INFO  2024-04-09 22:03:07,233 INFO Waiting for K8s secrets. 2024-04-09 22:03:07,333 INFO Waiting for secret "am-env-secrets" to exist in the cluster: done 2024-04-09 22:03:07,434 INFO Waiting for secret "idm-env-secrets" to exist in the cluster: done 2024-04-09 22:03:07,735 INFO Waiting for secret "ds-passwords" to exist in the cluster: done 2024-04-09 22:03:10,142 INFO Waiting for secret "ds-env-secrets" to exist in the cluster: done 2024-04-09 22:03:10,242 INFO  2024-04-09 22:03:10,343 INFO Relevant passwords: 2024-04-09 22:03:10,443 INFO EhSFnlsetVdSH6qjWDy5Xa0L (amadmin user) 2024-04-09 22:03:10,544 INFO A9N6V2kMl7aufgvBdZQgGHthQF3bOZSj (uid=admin user) 2024-04-09 22:03:10,644 INFO iTIFMZKrIoSMmkVyYvR6w9OMtUxwn3KG (App str svc acct (uid=am-config,ou=admins,ou=am-config)) 2024-04-09 22:03:10,745 INFO MxAvjItaWZ2RbYTxF1j3zjcaGQxdlLQn (CTS svc acct (uid=openam_cts,ou=admins,ou=famrecords,ou=openam-session,ou=tokens)) 2024-04-09 22:03:10,847 INFO AxGpHZydf9RTOmyfn1hK1RagorV5wLee (ID repo svc acct (uid=am-identity-bind-account,ou=admins,ou=identities)) 2024-04-09 22:03:10,947 INFO  2024-04-09 22:03:11,048 INFO Relevant URLs: 2024-04-09 22:03:11,148 INFO https://xlou.iam.xlou-cdm.engineeringpit.com/platform 2024-04-09 22:03:11,249 INFO https://xlou.iam.xlou-cdm.engineeringpit.com/admin 2024-04-09 22:03:11,349 INFO https://xlou.iam.xlou-cdm.engineeringpit.com/am 2024-04-09 22:03:11,450 INFO https://xlou.iam.xlou-cdm.engineeringpit.com/enduser 2024-04-09 22:03:11,551 INFO  2024-04-09 22:03:11,651 INFO Enjoy your deployment! 2024-04-09 22:03:11,752 ERROR # Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically. 2024-04-09 22:03:11,753 ERROR # Warning: 'patchesJson6902' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically. 2024-04-09 22:03:11,753 ERROR # Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically. 2024-04-09 22:03:11,753 INFO Setting context for components to: gke_engineeringpit_us-east1-d_xlou-cdm 2024-04-09 22:03:11,753 INFO 2024-04-09 22:03:11,753 INFO **************************** Initializing component pods for DS-CTS **************************** 2024-04-09 22:03:11,753 INFO 2024-04-09 22:03:11,753 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:11,753 INFO 2024-04-09 22:03:11,753 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:11,753 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:12,162 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:12,162 DEBUG --- stdout --- 2024-04-09 22:03:12,162 DEBUG 3 2024-04-09 22:03:12,163 DEBUG --- stderr --- 2024-04-09 22:03:12,163 DEBUG 2024-04-09 22:03:12,163 INFO 2024-04-09 22:03:12,163 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:03:12,163 INFO 2024-04-09 22:03:12,163 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name}` | grep 3 2024-04-09 22:03:12,163 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:03:12,622 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-09 22:03:12,623 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:12,623 DEBUG --- stdout --- 2024-04-09 22:03:12,623 DEBUG ds-cts-0 ds-cts-1 ds-cts-2 2024-04-09 22:03:12,623 DEBUG --- stderr --- 2024-04-09 22:03:12,623 DEBUG 2024-04-09 22:03:12,623 INFO 2024-04-09 22:03:12,623 INFO -------------------- Check pod ds-cts-0 is running -------------------- 2024-04-09 22:03:12,623 INFO 2024-04-09 22:03:12,623 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:12,623 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:12,995 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:12,996 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:12,996 DEBUG --- stdout --- 2024-04-09 22:03:12,996 DEBUG Running 2024-04-09 22:03:12,996 DEBUG --- stderr --- 2024-04-09 22:03:12,996 DEBUG 2024-04-09 22:03:12,996 INFO 2024-04-09 22:03:12,996 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:12,996 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:13,384 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:13,384 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:13,384 DEBUG --- stdout --- 2024-04-09 22:03:13,384 DEBUG true 2024-04-09 22:03:13,384 DEBUG --- stderr --- 2024-04-09 22:03:13,384 DEBUG 2024-04-09 22:03:13,384 INFO 2024-04-09 22:03:13,384 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.startTime} 2024-04-09 22:03:13,384 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:13,744 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:13,745 DEBUG --- stdout --- 2024-04-09 22:03:13,745 DEBUG 2024-04-09T22:00:11Z 2024-04-09 22:03:13,745 DEBUG --- stderr --- 2024-04-09 22:03:13,745 DEBUG 2024-04-09 22:03:13,746 INFO 2024-04-09 22:03:13,746 INFO ------------- Check pod ds-cts-0 filesystem is accessible ------------- 2024-04-09 22:03:14,638 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:14,638 INFO 2024-04-09 22:03:14,638 INFO ------------------ Check pod ds-cts-0 restart count ------------------ 2024-04-09 22:03:14,639 INFO 2024-04-09 22:03:14,639 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-0 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:14,639 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:15,012 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:15,012 DEBUG --- stdout --- 2024-04-09 22:03:15,013 DEBUG 0 2024-04-09 22:03:15,013 DEBUG --- stderr --- 2024-04-09 22:03:15,013 DEBUG 2024-04-09 22:03:15,013 INFO Pod ds-cts-0 has been restarted 0 times. 2024-04-09 22:03:15,013 INFO 2024-04-09 22:03:15,013 INFO -------------------- Check pod ds-cts-1 is running -------------------- 2024-04-09 22:03:15,013 INFO 2024-04-09 22:03:15,013 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:15,013 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:15,392 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:15,392 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:15,392 DEBUG --- stdout --- 2024-04-09 22:03:15,392 DEBUG Running 2024-04-09 22:03:15,392 DEBUG --- stderr --- 2024-04-09 22:03:15,392 DEBUG 2024-04-09 22:03:15,393 INFO 2024-04-09 22:03:15,393 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:15,393 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:15,756 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:15,757 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:15,758 DEBUG --- stdout --- 2024-04-09 22:03:15,758 DEBUG true 2024-04-09 22:03:15,758 DEBUG --- stderr --- 2024-04-09 22:03:15,758 DEBUG 2024-04-09 22:03:15,758 INFO 2024-04-09 22:03:15,758 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.startTime} 2024-04-09 22:03:15,758 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:16,120 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:16,120 DEBUG --- stdout --- 2024-04-09 22:03:16,120 DEBUG 2024-04-09T22:00:37Z 2024-04-09 22:03:16,120 DEBUG --- stderr --- 2024-04-09 22:03:16,120 DEBUG 2024-04-09 22:03:16,121 INFO 2024-04-09 22:03:16,121 INFO ------------- Check pod ds-cts-1 filesystem is accessible ------------- 2024-04-09 22:03:16,970 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:16,970 INFO 2024-04-09 22:03:16,970 INFO ------------------ Check pod ds-cts-1 restart count ------------------ 2024-04-09 22:03:16,970 INFO 2024-04-09 22:03:16,971 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-1 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:16,971 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:17,348 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:17,348 DEBUG --- stdout --- 2024-04-09 22:03:17,348 DEBUG 0 2024-04-09 22:03:17,348 DEBUG --- stderr --- 2024-04-09 22:03:17,348 DEBUG 2024-04-09 22:03:17,348 INFO Pod ds-cts-1 has been restarted 0 times. 2024-04-09 22:03:17,349 INFO 2024-04-09 22:03:17,349 INFO -------------------- Check pod ds-cts-2 is running -------------------- 2024-04-09 22:03:17,349 INFO 2024-04-09 22:03:17,349 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:17,349 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:17,716 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:17,716 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:17,716 DEBUG --- stdout --- 2024-04-09 22:03:17,716 DEBUG Running 2024-04-09 22:03:17,716 DEBUG --- stderr --- 2024-04-09 22:03:17,716 DEBUG 2024-04-09 22:03:17,717 INFO 2024-04-09 22:03:17,717 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-cts-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:17,717 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:18,079 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:18,080 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:18,080 DEBUG --- stdout --- 2024-04-09 22:03:18,080 DEBUG true 2024-04-09 22:03:18,080 DEBUG --- stderr --- 2024-04-09 22:03:18,080 DEBUG 2024-04-09 22:03:18,080 INFO 2024-04-09 22:03:18,080 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.startTime} 2024-04-09 22:03:18,080 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:18,446 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:18,447 DEBUG --- stdout --- 2024-04-09 22:03:18,447 DEBUG 2024-04-09T22:01:05Z 2024-04-09 22:03:18,447 DEBUG --- stderr --- 2024-04-09 22:03:18,447 DEBUG 2024-04-09 22:03:18,447 INFO 2024-04-09 22:03:18,447 INFO ------------- Check pod ds-cts-2 filesystem is accessible ------------- 2024-04-09 22:03:19,301 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:19,301 INFO 2024-04-09 22:03:19,301 INFO ------------------ Check pod ds-cts-2 restart count ------------------ 2024-04-09 22:03:19,301 INFO 2024-04-09 22:03:19,301 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-cts-2 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:19,301 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:19,673 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:19,673 DEBUG --- stdout --- 2024-04-09 22:03:19,673 DEBUG 0 2024-04-09 22:03:19,673 DEBUG --- stderr --- 2024-04-09 22:03:19,674 DEBUG 2024-04-09 22:03:19,674 INFO Pod ds-cts-2 has been restarted 0 times. 2024-04-09 22:03:19,674 INFO 2024-04-09 22:03:19,674 INFO ************************** Initializing component pods for DS-IDREPO ************************** 2024-04-09 22:03:19,674 INFO 2024-04-09 22:03:19,674 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:19,674 INFO 2024-04-09 22:03:19,674 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:19,674 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:20,072 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:20,072 DEBUG --- stdout --- 2024-04-09 22:03:20,072 DEBUG 3 2024-04-09 22:03:20,072 DEBUG --- stderr --- 2024-04-09 22:03:20,072 DEBUG 2024-04-09 22:03:20,072 INFO 2024-04-09 22:03:20,072 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:03:20,073 INFO 2024-04-09 22:03:20,073 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name}` | grep 3 2024-04-09 22:03:20,073 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:03:20,513 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-09 22:03:20,514 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:20,514 DEBUG --- stdout --- 2024-04-09 22:03:20,514 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2 2024-04-09 22:03:20,514 DEBUG --- stderr --- 2024-04-09 22:03:20,514 DEBUG 2024-04-09 22:03:20,514 INFO 2024-04-09 22:03:20,514 INFO ------------------ Check pod ds-idrepo-0 is running ------------------ 2024-04-09 22:03:20,514 INFO 2024-04-09 22:03:20,514 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:20,514 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:20,872 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:20,873 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:20,874 DEBUG --- stdout --- 2024-04-09 22:03:20,874 DEBUG Running 2024-04-09 22:03:20,874 DEBUG --- stderr --- 2024-04-09 22:03:20,874 DEBUG 2024-04-09 22:03:20,874 INFO 2024-04-09 22:03:20,874 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-0 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:20,875 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:21,237 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:21,237 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:21,237 DEBUG --- stdout --- 2024-04-09 22:03:21,237 DEBUG true 2024-04-09 22:03:21,237 DEBUG --- stderr --- 2024-04-09 22:03:21,237 DEBUG 2024-04-09 22:03:21,238 INFO 2024-04-09 22:03:21,238 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.startTime} 2024-04-09 22:03:21,238 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:21,601 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:21,601 DEBUG --- stdout --- 2024-04-09 22:03:21,602 DEBUG 2024-04-09T22:00:11Z 2024-04-09 22:03:21,602 DEBUG --- stderr --- 2024-04-09 22:03:21,602 DEBUG 2024-04-09 22:03:21,602 INFO 2024-04-09 22:03:21,602 INFO ----------- Check pod ds-idrepo-0 filesystem is accessible ----------- 2024-04-09 22:03:22,453 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:22,453 INFO 2024-04-09 22:03:22,453 INFO ----------------- Check pod ds-idrepo-0 restart count ----------------- 2024-04-09 22:03:22,454 INFO 2024-04-09 22:03:22,454 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-0 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:22,454 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:22,828 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:22,828 DEBUG --- stdout --- 2024-04-09 22:03:22,828 DEBUG 0 2024-04-09 22:03:22,828 DEBUG --- stderr --- 2024-04-09 22:03:22,828 DEBUG 2024-04-09 22:03:22,828 INFO Pod ds-idrepo-0 has been restarted 0 times. 2024-04-09 22:03:22,828 INFO 2024-04-09 22:03:22,828 INFO ------------------ Check pod ds-idrepo-1 is running ------------------ 2024-04-09 22:03:22,828 INFO 2024-04-09 22:03:22,828 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:22,829 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:23,200 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:23,200 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:23,200 DEBUG --- stdout --- 2024-04-09 22:03:23,200 DEBUG Running 2024-04-09 22:03:23,200 DEBUG --- stderr --- 2024-04-09 22:03:23,200 DEBUG 2024-04-09 22:03:23,201 INFO 2024-04-09 22:03:23,201 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-1 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:23,201 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:23,579 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:23,579 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:23,579 DEBUG --- stdout --- 2024-04-09 22:03:23,579 DEBUG true 2024-04-09 22:03:23,579 DEBUG --- stderr --- 2024-04-09 22:03:23,579 DEBUG 2024-04-09 22:03:23,580 INFO 2024-04-09 22:03:23,580 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.startTime} 2024-04-09 22:03:23,580 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:23,942 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:23,942 DEBUG --- stdout --- 2024-04-09 22:03:23,942 DEBUG 2024-04-09T22:00:44Z 2024-04-09 22:03:23,942 DEBUG --- stderr --- 2024-04-09 22:03:23,942 DEBUG 2024-04-09 22:03:23,943 INFO 2024-04-09 22:03:23,943 INFO ----------- Check pod ds-idrepo-1 filesystem is accessible ----------- 2024-04-09 22:03:24,771 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:24,771 INFO 2024-04-09 22:03:24,771 INFO ----------------- Check pod ds-idrepo-1 restart count ----------------- 2024-04-09 22:03:24,771 INFO 2024-04-09 22:03:24,771 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-1 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:24,771 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:25,131 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:25,132 DEBUG --- stdout --- 2024-04-09 22:03:25,132 DEBUG 0 2024-04-09 22:03:25,132 DEBUG --- stderr --- 2024-04-09 22:03:25,132 DEBUG 2024-04-09 22:03:25,132 INFO Pod ds-idrepo-1 has been restarted 0 times. 2024-04-09 22:03:25,132 INFO 2024-04-09 22:03:25,132 INFO ------------------ Check pod ds-idrepo-2 is running ------------------ 2024-04-09 22:03:25,132 INFO 2024-04-09 22:03:25,132 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:25,132 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:25,518 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:25,518 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:25,518 DEBUG --- stdout --- 2024-04-09 22:03:25,518 DEBUG Running 2024-04-09 22:03:25,519 DEBUG --- stderr --- 2024-04-09 22:03:25,519 DEBUG 2024-04-09 22:03:25,519 INFO 2024-04-09 22:03:25,519 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods ds-idrepo-2 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:25,519 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:25,886 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:25,888 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:25,889 DEBUG --- stdout --- 2024-04-09 22:03:25,889 DEBUG true 2024-04-09 22:03:25,889 DEBUG --- stderr --- 2024-04-09 22:03:25,889 DEBUG 2024-04-09 22:03:25,889 INFO 2024-04-09 22:03:25,889 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.startTime} 2024-04-09 22:03:25,889 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:26,267 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:26,268 DEBUG --- stdout --- 2024-04-09 22:03:26,268 DEBUG 2024-04-09T22:01:21Z 2024-04-09 22:03:26,268 DEBUG --- stderr --- 2024-04-09 22:03:26,268 DEBUG 2024-04-09 22:03:26,268 INFO 2024-04-09 22:03:26,269 INFO ----------- Check pod ds-idrepo-2 filesystem is accessible ----------- 2024-04-09 22:03:27,130 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:27,130 INFO 2024-04-09 22:03:27,130 INFO ----------------- Check pod ds-idrepo-2 restart count ----------------- 2024-04-09 22:03:27,130 INFO 2024-04-09 22:03:27,130 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod ds-idrepo-2 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:27,130 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:27,509 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:27,509 DEBUG --- stdout --- 2024-04-09 22:03:27,509 DEBUG 0 2024-04-09 22:03:27,509 DEBUG --- stderr --- 2024-04-09 22:03:27,509 DEBUG 2024-04-09 22:03:27,509 INFO Pod ds-idrepo-2 has been restarted 0 times. 2024-04-09 22:03:27,509 INFO 2024-04-09 22:03:27,509 INFO ****************************** Initializing component pods for AM ****************************** 2024-04-09 22:03:27,509 INFO 2024-04-09 22:03:27,509 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:27,510 INFO 2024-04-09 22:03:27,510 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:27,510 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:27,892 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:27,892 DEBUG --- stdout --- 2024-04-09 22:03:27,892 DEBUG 3 2024-04-09 22:03:27,892 DEBUG --- stderr --- 2024-04-09 22:03:27,893 DEBUG 2024-04-09 22:03:27,893 INFO 2024-04-09 22:03:27,893 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:03:27,893 INFO 2024-04-09 22:03:27,893 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name}` | grep 3 2024-04-09 22:03:27,893 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:03:28,356 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-09 22:03:28,356 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:28,356 DEBUG --- stdout --- 2024-04-09 22:03:28,356 DEBUG am-654c4fcbcf-487cv am-654c4fcbcf-9scrj am-654c4fcbcf-v5qb5 2024-04-09 22:03:28,356 DEBUG --- stderr --- 2024-04-09 22:03:28,356 DEBUG 2024-04-09 22:03:28,356 INFO 2024-04-09 22:03:28,357 INFO -------------- Check pod am-654c4fcbcf-487cv is running -------------- 2024-04-09 22:03:28,357 INFO 2024-04-09 22:03:28,357 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-654c4fcbcf-487cv -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:28,357 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:28,741 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:28,741 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:28,741 DEBUG --- stdout --- 2024-04-09 22:03:28,741 DEBUG Running 2024-04-09 22:03:28,741 DEBUG --- stderr --- 2024-04-09 22:03:28,741 DEBUG 2024-04-09 22:03:28,742 INFO 2024-04-09 22:03:28,742 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-654c4fcbcf-487cv -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:28,742 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:29,120 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:29,121 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:29,121 DEBUG --- stdout --- 2024-04-09 22:03:29,121 DEBUG true 2024-04-09 22:03:29,121 DEBUG --- stderr --- 2024-04-09 22:03:29,121 DEBUG 2024-04-09 22:03:29,121 INFO 2024-04-09 22:03:29,121 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-654c4fcbcf-487cv --output jsonpath={.status.startTime} 2024-04-09 22:03:29,121 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:29,494 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:29,494 DEBUG --- stdout --- 2024-04-09 22:03:29,495 DEBUG 2024-04-09T22:02:02Z 2024-04-09 22:03:29,495 DEBUG --- stderr --- 2024-04-09 22:03:29,495 DEBUG 2024-04-09 22:03:29,495 INFO 2024-04-09 22:03:29,495 INFO ------- Check pod am-654c4fcbcf-487cv filesystem is accessible ------- 2024-04-09 22:03:30,350 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:30,350 INFO 2024-04-09 22:03:30,350 INFO ------------- Check pod am-654c4fcbcf-487cv restart count ------------- 2024-04-09 22:03:30,350 INFO 2024-04-09 22:03:30,350 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-654c4fcbcf-487cv --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:30,350 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:30,728 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:30,728 DEBUG --- stdout --- 2024-04-09 22:03:30,728 DEBUG 0 2024-04-09 22:03:30,728 DEBUG --- stderr --- 2024-04-09 22:03:30,728 DEBUG 2024-04-09 22:03:30,728 INFO Pod am-654c4fcbcf-487cv has been restarted 0 times. 2024-04-09 22:03:30,728 INFO 2024-04-09 22:03:30,728 INFO -------------- Check pod am-654c4fcbcf-9scrj is running -------------- 2024-04-09 22:03:30,729 INFO 2024-04-09 22:03:30,729 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-654c4fcbcf-9scrj -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:30,729 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:31,104 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:31,104 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:31,106 DEBUG --- stdout --- 2024-04-09 22:03:31,106 DEBUG Running 2024-04-09 22:03:31,106 DEBUG --- stderr --- 2024-04-09 22:03:31,106 DEBUG 2024-04-09 22:03:31,106 INFO 2024-04-09 22:03:31,107 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-654c4fcbcf-9scrj -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:31,107 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:31,480 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:31,481 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:31,481 DEBUG --- stdout --- 2024-04-09 22:03:31,481 DEBUG true 2024-04-09 22:03:31,481 DEBUG --- stderr --- 2024-04-09 22:03:31,481 DEBUG 2024-04-09 22:03:31,481 INFO 2024-04-09 22:03:31,481 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-654c4fcbcf-9scrj --output jsonpath={.status.startTime} 2024-04-09 22:03:31,481 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:31,853 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:31,853 DEBUG --- stdout --- 2024-04-09 22:03:31,853 DEBUG 2024-04-09T22:02:02Z 2024-04-09 22:03:31,854 DEBUG --- stderr --- 2024-04-09 22:03:31,854 DEBUG 2024-04-09 22:03:31,854 INFO 2024-04-09 22:03:31,854 INFO ------- Check pod am-654c4fcbcf-9scrj filesystem is accessible ------- 2024-04-09 22:03:32,690 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:32,690 INFO 2024-04-09 22:03:32,690 INFO ------------- Check pod am-654c4fcbcf-9scrj restart count ------------- 2024-04-09 22:03:32,691 INFO 2024-04-09 22:03:32,691 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-654c4fcbcf-9scrj --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:32,691 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:33,057 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:33,057 DEBUG --- stdout --- 2024-04-09 22:03:33,057 DEBUG 0 2024-04-09 22:03:33,057 DEBUG --- stderr --- 2024-04-09 22:03:33,057 DEBUG 2024-04-09 22:03:33,057 INFO Pod am-654c4fcbcf-9scrj has been restarted 0 times. 2024-04-09 22:03:33,057 INFO 2024-04-09 22:03:33,057 INFO -------------- Check pod am-654c4fcbcf-v5qb5 is running -------------- 2024-04-09 22:03:33,058 INFO 2024-04-09 22:03:33,058 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-654c4fcbcf-v5qb5 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:33,058 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:33,437 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:33,437 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:33,437 DEBUG --- stdout --- 2024-04-09 22:03:33,437 DEBUG Running 2024-04-09 22:03:33,437 DEBUG --- stderr --- 2024-04-09 22:03:33,437 DEBUG 2024-04-09 22:03:33,437 INFO 2024-04-09 22:03:33,437 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods am-654c4fcbcf-v5qb5 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:33,437 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:33,819 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:33,819 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:33,819 DEBUG --- stdout --- 2024-04-09 22:03:33,819 DEBUG true 2024-04-09 22:03:33,819 DEBUG --- stderr --- 2024-04-09 22:03:33,819 DEBUG 2024-04-09 22:03:33,819 INFO 2024-04-09 22:03:33,819 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-654c4fcbcf-v5qb5 --output jsonpath={.status.startTime} 2024-04-09 22:03:33,819 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:34,195 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:34,195 DEBUG --- stdout --- 2024-04-09 22:03:34,195 DEBUG 2024-04-09T22:02:02Z 2024-04-09 22:03:34,195 DEBUG --- stderr --- 2024-04-09 22:03:34,195 DEBUG 2024-04-09 22:03:34,196 INFO 2024-04-09 22:03:34,196 INFO ------- Check pod am-654c4fcbcf-v5qb5 filesystem is accessible ------- 2024-04-09 22:03:35,065 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:35,065 INFO 2024-04-09 22:03:35,065 INFO ------------- Check pod am-654c4fcbcf-v5qb5 restart count ------------- 2024-04-09 22:03:35,066 INFO 2024-04-09 22:03:35,066 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod am-654c4fcbcf-v5qb5 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:35,066 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:35,450 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:35,450 DEBUG --- stdout --- 2024-04-09 22:03:35,450 DEBUG 0 2024-04-09 22:03:35,450 DEBUG --- stderr --- 2024-04-09 22:03:35,450 DEBUG 2024-04-09 22:03:35,450 INFO Pod am-654c4fcbcf-v5qb5 has been restarted 0 times. 2024-04-09 22:03:35,451 INFO 2024-04-09 22:03:35,451 INFO **************************** Initializing component pods for AMSTER **************************** 2024-04-09 22:03:35,451 INFO 2024-04-09 22:03:35,451 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:03:35,451 INFO 2024-04-09 22:03:35,451 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name} 2024-04-09 22:03:35,451 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:03:35,834 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:35,834 DEBUG --- stdout --- 2024-04-09 22:03:35,834 DEBUG amster-l8fng 2024-04-09 22:03:35,834 DEBUG --- stderr --- 2024-04-09 22:03:35,834 DEBUG 2024-04-09 22:03:35,834 INFO 2024-04-09 22:03:35,834 INFO ***************************** Initializing component pods for IDM ***************************** 2024-04-09 22:03:35,834 INFO 2024-04-09 22:03:35,834 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:35,835 INFO 2024-04-09 22:03:35,835 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:35,835 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:36,223 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:36,223 DEBUG --- stdout --- 2024-04-09 22:03:36,223 DEBUG 2 2024-04-09 22:03:36,223 DEBUG --- stderr --- 2024-04-09 22:03:36,223 DEBUG 2024-04-09 22:03:36,224 INFO 2024-04-09 22:03:36,224 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:03:36,224 INFO 2024-04-09 22:03:36,224 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name}` | grep 2 2024-04-09 22:03:36,224 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:03:36,628 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-09 22:03:36,628 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:36,628 DEBUG --- stdout --- 2024-04-09 22:03:36,628 DEBUG idm-6bfb7c96cc-qklk9 idm-6bfb7c96cc-vsb58 2024-04-09 22:03:36,628 DEBUG --- stderr --- 2024-04-09 22:03:36,628 DEBUG 2024-04-09 22:03:36,628 INFO 2024-04-09 22:03:36,629 INFO -------------- Check pod idm-6bfb7c96cc-qklk9 is running -------------- 2024-04-09 22:03:36,629 INFO 2024-04-09 22:03:36,629 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-6bfb7c96cc-qklk9 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:36,629 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:37,015 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:37,016 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:37,016 DEBUG --- stdout --- 2024-04-09 22:03:37,016 DEBUG Running 2024-04-09 22:03:37,016 DEBUG --- stderr --- 2024-04-09 22:03:37,016 DEBUG 2024-04-09 22:03:37,017 INFO 2024-04-09 22:03:37,017 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-6bfb7c96cc-qklk9 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:37,017 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:37,390 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:37,390 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:37,390 DEBUG --- stdout --- 2024-04-09 22:03:37,390 DEBUG true 2024-04-09 22:03:37,390 DEBUG --- stderr --- 2024-04-09 22:03:37,391 DEBUG 2024-04-09 22:03:37,391 INFO 2024-04-09 22:03:37,391 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-6bfb7c96cc-qklk9 --output jsonpath={.status.startTime} 2024-04-09 22:03:37,391 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:37,771 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:37,771 DEBUG --- stdout --- 2024-04-09 22:03:37,771 DEBUG 2024-04-09T22:02:02Z 2024-04-09 22:03:37,771 DEBUG --- stderr --- 2024-04-09 22:03:37,771 DEBUG 2024-04-09 22:03:37,772 INFO 2024-04-09 22:03:37,772 INFO ------- Check pod idm-6bfb7c96cc-qklk9 filesystem is accessible ------- 2024-04-09 22:03:38,623 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:38,623 INFO 2024-04-09 22:03:38,623 INFO ------------ Check pod idm-6bfb7c96cc-qklk9 restart count ------------ 2024-04-09 22:03:38,624 INFO 2024-04-09 22:03:38,624 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-6bfb7c96cc-qklk9 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:38,624 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:38,988 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:38,988 DEBUG --- stdout --- 2024-04-09 22:03:38,988 DEBUG 0 2024-04-09 22:03:38,988 DEBUG --- stderr --- 2024-04-09 22:03:38,988 DEBUG 2024-04-09 22:03:38,988 INFO Pod idm-6bfb7c96cc-qklk9 has been restarted 0 times. 2024-04-09 22:03:38,989 INFO 2024-04-09 22:03:38,989 INFO -------------- Check pod idm-6bfb7c96cc-vsb58 is running -------------- 2024-04-09 22:03:38,989 INFO 2024-04-09 22:03:38,989 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-6bfb7c96cc-vsb58 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:38,989 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:39,352 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:39,352 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:39,352 DEBUG --- stdout --- 2024-04-09 22:03:39,352 DEBUG Running 2024-04-09 22:03:39,352 DEBUG --- stderr --- 2024-04-09 22:03:39,352 DEBUG 2024-04-09 22:03:39,353 INFO 2024-04-09 22:03:39,353 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods idm-6bfb7c96cc-vsb58 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:39,353 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:39,716 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:39,716 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:39,716 DEBUG --- stdout --- 2024-04-09 22:03:39,716 DEBUG true 2024-04-09 22:03:39,716 DEBUG --- stderr --- 2024-04-09 22:03:39,716 DEBUG 2024-04-09 22:03:39,717 INFO 2024-04-09 22:03:39,717 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-6bfb7c96cc-vsb58 --output jsonpath={.status.startTime} 2024-04-09 22:03:39,717 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:40,080 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:40,080 DEBUG --- stdout --- 2024-04-09 22:03:40,080 DEBUG 2024-04-09T22:02:03Z 2024-04-09 22:03:40,080 DEBUG --- stderr --- 2024-04-09 22:03:40,080 DEBUG 2024-04-09 22:03:40,081 INFO 2024-04-09 22:03:40,081 INFO ------- Check pod idm-6bfb7c96cc-vsb58 filesystem is accessible ------- 2024-04-09 22:03:40,943 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:40,943 INFO 2024-04-09 22:03:40,943 INFO ------------ Check pod idm-6bfb7c96cc-vsb58 restart count ------------ 2024-04-09 22:03:40,943 INFO 2024-04-09 22:03:40,943 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod idm-6bfb7c96cc-vsb58 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:40,943 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:41,300 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:41,301 DEBUG --- stdout --- 2024-04-09 22:03:41,301 DEBUG 0 2024-04-09 22:03:41,301 DEBUG --- stderr --- 2024-04-09 22:03:41,301 DEBUG 2024-04-09 22:03:41,301 INFO Pod idm-6bfb7c96cc-vsb58 has been restarted 0 times. 2024-04-09 22:03:41,301 INFO 2024-04-09 22:03:41,302 INFO ************************* Initializing component pods for END-USER-UI ************************* 2024-04-09 22:03:41,302 INFO 2024-04-09 22:03:41,302 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:41,302 INFO 2024-04-09 22:03:41,302 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:41,302 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:41,604 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:41,605 DEBUG --- stdout --- 2024-04-09 22:03:41,605 DEBUG 1 2024-04-09 22:03:41,605 DEBUG --- stderr --- 2024-04-09 22:03:41,605 DEBUG 2024-04-09 22:03:41,605 INFO 2024-04-09 22:03:41,605 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:03:41,605 INFO 2024-04-09 22:03:41,605 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-09 22:03:41,605 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:03:41,915 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-09 22:03:41,915 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:41,915 DEBUG --- stdout --- 2024-04-09 22:03:41,915 DEBUG end-user-ui-6d9fd5487b-2s47b 2024-04-09 22:03:41,915 DEBUG --- stderr --- 2024-04-09 22:03:41,915 DEBUG 2024-04-09 22:03:41,916 INFO 2024-04-09 22:03:41,916 INFO ---------- Check pod end-user-ui-6d9fd5487b-2s47b is running ---------- 2024-04-09 22:03:41,916 INFO 2024-04-09 22:03:41,916 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-6d9fd5487b-2s47b -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:41,916 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:42,248 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:42,248 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:42,248 DEBUG --- stdout --- 2024-04-09 22:03:42,248 DEBUG Running 2024-04-09 22:03:42,248 DEBUG --- stderr --- 2024-04-09 22:03:42,248 DEBUG 2024-04-09 22:03:42,248 INFO 2024-04-09 22:03:42,249 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods end-user-ui-6d9fd5487b-2s47b -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:42,249 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:42,564 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:42,565 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:42,565 DEBUG --- stdout --- 2024-04-09 22:03:42,565 DEBUG true 2024-04-09 22:03:42,565 DEBUG --- stderr --- 2024-04-09 22:03:42,565 DEBUG 2024-04-09 22:03:42,565 INFO 2024-04-09 22:03:42,565 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-6d9fd5487b-2s47b --output jsonpath={.status.startTime} 2024-04-09 22:03:42,565 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:42,876 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:42,877 DEBUG --- stdout --- 2024-04-09 22:03:42,877 DEBUG 2024-04-09T22:03:06Z 2024-04-09 22:03:42,877 DEBUG --- stderr --- 2024-04-09 22:03:42,877 DEBUG 2024-04-09 22:03:42,878 INFO 2024-04-09 22:03:42,878 INFO --- Check pod end-user-ui-6d9fd5487b-2s47b filesystem is accessible --- 2024-04-09 22:03:43,672 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:43,672 INFO 2024-04-09 22:03:43,672 INFO -------- Check pod end-user-ui-6d9fd5487b-2s47b restart count -------- 2024-04-09 22:03:43,672 INFO 2024-04-09 22:03:43,672 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod end-user-ui-6d9fd5487b-2s47b --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:43,672 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:44,019 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:44,019 DEBUG --- stdout --- 2024-04-09 22:03:44,019 DEBUG 0 2024-04-09 22:03:44,019 DEBUG --- stderr --- 2024-04-09 22:03:44,019 DEBUG 2024-04-09 22:03:44,019 INFO Pod end-user-ui-6d9fd5487b-2s47b has been restarted 0 times. 2024-04-09 22:03:44,019 INFO 2024-04-09 22:03:44,019 INFO *************************** Initializing component pods for LOGIN-UI *************************** 2024-04-09 22:03:44,019 INFO 2024-04-09 22:03:44,019 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:44,020 INFO 2024-04-09 22:03:44,020 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:44,020 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:44,346 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:44,346 DEBUG --- stdout --- 2024-04-09 22:03:44,346 DEBUG 1 2024-04-09 22:03:44,346 DEBUG --- stderr --- 2024-04-09 22:03:44,346 DEBUG 2024-04-09 22:03:44,347 INFO 2024-04-09 22:03:44,347 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:03:44,347 INFO 2024-04-09 22:03:44,347 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-09 22:03:44,347 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:03:44,670 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-09 22:03:44,671 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:44,671 DEBUG --- stdout --- 2024-04-09 22:03:44,671 DEBUG login-ui-5f964fb894-5zpvq 2024-04-09 22:03:44,671 DEBUG --- stderr --- 2024-04-09 22:03:44,671 DEBUG 2024-04-09 22:03:44,672 INFO 2024-04-09 22:03:44,672 INFO ----------- Check pod login-ui-5f964fb894-5zpvq is running ----------- 2024-04-09 22:03:44,672 INFO 2024-04-09 22:03:44,672 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5f964fb894-5zpvq -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:44,672 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:45,014 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:45,014 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:45,014 DEBUG --- stdout --- 2024-04-09 22:03:45,014 DEBUG Running 2024-04-09 22:03:45,014 DEBUG --- stderr --- 2024-04-09 22:03:45,014 DEBUG 2024-04-09 22:03:45,015 INFO 2024-04-09 22:03:45,015 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods login-ui-5f964fb894-5zpvq -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:45,015 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:45,320 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:45,320 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:45,320 DEBUG --- stdout --- 2024-04-09 22:03:45,320 DEBUG true 2024-04-09 22:03:45,320 DEBUG --- stderr --- 2024-04-09 22:03:45,320 DEBUG 2024-04-09 22:03:45,320 INFO 2024-04-09 22:03:45,320 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5f964fb894-5zpvq --output jsonpath={.status.startTime} 2024-04-09 22:03:45,320 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:45,622 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:45,622 DEBUG --- stdout --- 2024-04-09 22:03:45,622 DEBUG 2024-04-09T22:03:06Z 2024-04-09 22:03:45,622 DEBUG --- stderr --- 2024-04-09 22:03:45,623 DEBUG 2024-04-09 22:03:45,623 INFO 2024-04-09 22:03:45,623 INFO ---- Check pod login-ui-5f964fb894-5zpvq filesystem is accessible ---- 2024-04-09 22:03:46,402 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:46,404 INFO 2024-04-09 22:03:46,404 INFO ---------- Check pod login-ui-5f964fb894-5zpvq restart count ---------- 2024-04-09 22:03:46,404 INFO 2024-04-09 22:03:46,405 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod login-ui-5f964fb894-5zpvq --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:46,405 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:46,721 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:46,721 DEBUG --- stdout --- 2024-04-09 22:03:46,721 DEBUG 0 2024-04-09 22:03:46,721 DEBUG --- stderr --- 2024-04-09 22:03:46,721 DEBUG 2024-04-09 22:03:46,722 INFO Pod login-ui-5f964fb894-5zpvq has been restarted 0 times. 2024-04-09 22:03:46,722 INFO 2024-04-09 22:03:46,722 INFO *************************** Initializing component pods for ADMIN-UI *************************** 2024-04-09 22:03:46,722 INFO 2024-04-09 22:03:46,722 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:46,722 INFO 2024-04-09 22:03:46,722 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:46,722 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:47,042 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:47,043 DEBUG --- stdout --- 2024-04-09 22:03:47,043 DEBUG 1 2024-04-09 22:03:47,043 DEBUG --- stderr --- 2024-04-09 22:03:47,043 DEBUG 2024-04-09 22:03:47,043 INFO 2024-04-09 22:03:47,043 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:03:47,043 INFO 2024-04-09 22:03:47,043 INFO [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name}` | grep 1 2024-04-09 22:03:47,043 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:03:47,353 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found 2024-04-09 22:03:47,354 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:47,354 DEBUG --- stdout --- 2024-04-09 22:03:47,354 DEBUG admin-ui-7cf664f88d-pnt57 2024-04-09 22:03:47,354 DEBUG --- stderr --- 2024-04-09 22:03:47,354 DEBUG 2024-04-09 22:03:47,354 INFO 2024-04-09 22:03:47,354 INFO ----------- Check pod admin-ui-7cf664f88d-pnt57 is running ----------- 2024-04-09 22:03:47,354 INFO 2024-04-09 22:03:47,354 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-7cf664f88d-pnt57 -o=jsonpath={.status.phase} | grep "Running" 2024-04-09 22:03:47,354 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:47,677 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:47,677 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:47,677 DEBUG --- stdout --- 2024-04-09 22:03:47,678 DEBUG Running 2024-04-09 22:03:47,678 DEBUG --- stderr --- 2024-04-09 22:03:47,678 DEBUG 2024-04-09 22:03:47,678 INFO 2024-04-09 22:03:47,678 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods admin-ui-7cf664f88d-pnt57 -o=jsonpath={.status.containerStatuses[*].ready} | grep "true" 2024-04-09 22:03:47,678 INFO [loop_until]: (max_time=360, interval=5, expected_rc=[0] 2024-04-09 22:03:47,977 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:47,978 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:47,978 DEBUG --- stdout --- 2024-04-09 22:03:47,978 DEBUG true 2024-04-09 22:03:47,978 DEBUG --- stderr --- 2024-04-09 22:03:47,978 DEBUG 2024-04-09 22:03:47,978 INFO 2024-04-09 22:03:47,978 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-7cf664f88d-pnt57 --output jsonpath={.status.startTime} 2024-04-09 22:03:47,978 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:48,276 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:48,276 DEBUG --- stdout --- 2024-04-09 22:03:48,276 DEBUG 2024-04-09T22:03:06Z 2024-04-09 22:03:48,276 DEBUG --- stderr --- 2024-04-09 22:03:48,276 DEBUG 2024-04-09 22:03:48,277 INFO 2024-04-09 22:03:48,277 INFO ---- Check pod admin-ui-7cf664f88d-pnt57 filesystem is accessible ---- 2024-04-09 22:03:49,058 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:49,059 INFO 2024-04-09 22:03:49,059 INFO ---------- Check pod admin-ui-7cf664f88d-pnt57 restart count ---------- 2024-04-09 22:03:49,059 INFO 2024-04-09 22:03:49,059 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod admin-ui-7cf664f88d-pnt57 --output jsonpath={.status.containerStatuses[*].restartCount} 2024-04-09 22:03:49,059 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:49,373 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:49,373 DEBUG --- stdout --- 2024-04-09 22:03:49,373 DEBUG 0 2024-04-09 22:03:49,373 DEBUG --- stderr --- 2024-04-09 22:03:49,373 DEBUG 2024-04-09 22:03:49,374 INFO Pod admin-ui-7cf664f88d-pnt57 has been restarted 0 times. 2024-04-09 22:03:49,374 INFO 2024-04-09 22:03:49,374 INFO ***************************** Checking DS-CTS component is running ***************************** 2024-04-09 22:03:49,374 INFO 2024-04-09 22:03:49,374 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:49,374 INFO 2024-04-09 22:03:49,374 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:49,374 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:49,761 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:49,762 DEBUG --- stdout --- 2024-04-09 22:03:49,762 DEBUG 3 2024-04-09 22:03:49,762 DEBUG --- stderr --- 2024-04-09 22:03:49,762 DEBUG 2024-04-09 22:03:49,762 INFO 2024-04-09 22:03:49,762 INFO -------------- Waiting for 3 expected pod(s) to be ready -------------- 2024-04-09 22:03:49,763 INFO 2024-04-09 22:03:49,763 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-cts --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3" 2024-04-09 22:03:49,763 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0] 2024-04-09 22:03:50,152 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:50,152 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:50,152 DEBUG --- stdout --- 2024-04-09 22:03:50,152 DEBUG current:3 ready:3 replicas:3 2024-04-09 22:03:50,152 DEBUG --- stderr --- 2024-04-09 22:03:50,152 DEBUG 2024-04-09 22:03:50,152 INFO 2024-04-09 22:03:50,152 INFO *************************** Checking DS-IDREPO component is running *************************** 2024-04-09 22:03:50,152 INFO 2024-04-09 22:03:50,152 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:50,152 INFO 2024-04-09 22:03:50,152 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:50,153 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:50,539 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:50,539 DEBUG --- stdout --- 2024-04-09 22:03:50,539 DEBUG 3 2024-04-09 22:03:50,539 DEBUG --- stderr --- 2024-04-09 22:03:50,539 DEBUG 2024-04-09 22:03:50,539 INFO 2024-04-09 22:03:50,539 INFO -------------- Waiting for 3 expected pod(s) to be ready -------------- 2024-04-09 22:03:50,540 INFO 2024-04-09 22:03:50,540 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets ds-idrepo --output jsonpath="current:{.status.currentReplicas} ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "current:3 ready:3 replicas:3" 2024-04-09 22:03:50,540 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0] 2024-04-09 22:03:50,925 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:50,925 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:50,925 DEBUG --- stdout --- 2024-04-09 22:03:50,925 DEBUG current:3 ready:3 replicas:3 2024-04-09 22:03:50,925 DEBUG --- stderr --- 2024-04-09 22:03:50,925 DEBUG 2024-04-09 22:03:50,925 INFO 2024-04-09 22:03:50,925 INFO ******************************* Checking AM component is running ******************************* 2024-04-09 22:03:50,925 INFO 2024-04-09 22:03:50,925 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:50,926 INFO 2024-04-09 22:03:50,926 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:50,926 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:51,311 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:51,312 DEBUG --- stdout --- 2024-04-09 22:03:51,312 DEBUG 3 2024-04-09 22:03:51,312 DEBUG --- stderr --- 2024-04-09 22:03:51,312 DEBUG 2024-04-09 22:03:51,312 INFO 2024-04-09 22:03:51,312 INFO -------------- Waiting for 3 expected pod(s) to be ready -------------- 2024-04-09 22:03:51,312 INFO 2024-04-09 22:03:51,312 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments am --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:3 replicas:3" 2024-04-09 22:03:51,312 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0] 2024-04-09 22:03:51,689 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:51,689 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:51,689 DEBUG --- stdout --- 2024-04-09 22:03:51,689 DEBUG ready:3 replicas:3 2024-04-09 22:03:51,689 DEBUG --- stderr --- 2024-04-09 22:03:51,689 DEBUG 2024-04-09 22:03:51,689 INFO 2024-04-09 22:03:51,689 INFO ***************************** Checking AMSTER component is running ***************************** 2024-04-09 22:03:51,689 INFO 2024-04-09 22:03:51,689 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:51,689 INFO 2024-04-09 22:03:51,690 INFO -------------- Waiting for 1 expected pod(s) to be ready -------------- 2024-04-09 22:03:51,690 INFO 2024-04-09 22:03:51,690 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get jobs amster --output jsonpath="{.status.succeeded}" | grep "1" 2024-04-09 22:03:51,690 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0] 2024-04-09 22:03:52,057 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:52,057 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:52,057 DEBUG --- stdout --- 2024-04-09 22:03:52,057 DEBUG 1 2024-04-09 22:03:52,058 DEBUG --- stderr --- 2024-04-09 22:03:52,058 DEBUG 2024-04-09 22:03:52,058 INFO 2024-04-09 22:03:52,058 INFO ****************************** Checking IDM component is running ****************************** 2024-04-09 22:03:52,058 INFO 2024-04-09 22:03:52,058 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:52,058 INFO 2024-04-09 22:03:52,058 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:52,058 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:52,433 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:52,433 DEBUG --- stdout --- 2024-04-09 22:03:52,433 DEBUG 2 2024-04-09 22:03:52,433 DEBUG --- stderr --- 2024-04-09 22:03:52,433 DEBUG 2024-04-09 22:03:52,433 INFO 2024-04-09 22:03:52,433 INFO -------------- Waiting for 2 expected pod(s) to be ready -------------- 2024-04-09 22:03:52,433 INFO 2024-04-09 22:03:52,433 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment idm --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:2 replicas:2" 2024-04-09 22:03:52,433 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0] 2024-04-09 22:03:52,801 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:52,801 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:52,801 DEBUG --- stdout --- 2024-04-09 22:03:52,801 DEBUG ready:2 replicas:2 2024-04-09 22:03:52,801 DEBUG --- stderr --- 2024-04-09 22:03:52,801 DEBUG 2024-04-09 22:03:52,801 INFO 2024-04-09 22:03:52,801 INFO ************************** Checking END-USER-UI component is running ************************** 2024-04-09 22:03:52,801 INFO 2024-04-09 22:03:52,801 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:52,801 INFO 2024-04-09 22:03:52,801 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:52,802 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:53,110 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:53,110 DEBUG --- stdout --- 2024-04-09 22:03:53,110 DEBUG 1 2024-04-09 22:03:53,110 DEBUG --- stderr --- 2024-04-09 22:03:53,110 DEBUG 2024-04-09 22:03:53,111 INFO 2024-04-09 22:03:53,111 INFO -------------- Waiting for 1 expected pod(s) to be ready -------------- 2024-04-09 22:03:53,111 INFO 2024-04-09 22:03:53,111 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments end-user-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1" 2024-04-09 22:03:53,111 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0] 2024-04-09 22:03:53,414 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:53,415 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:53,415 DEBUG --- stdout --- 2024-04-09 22:03:53,415 DEBUG ready:1 replicas:1 2024-04-09 22:03:53,415 DEBUG --- stderr --- 2024-04-09 22:03:53,415 DEBUG 2024-04-09 22:03:53,415 INFO 2024-04-09 22:03:53,415 INFO **************************** Checking LOGIN-UI component is running **************************** 2024-04-09 22:03:53,415 INFO 2024-04-09 22:03:53,415 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:53,416 INFO 2024-04-09 22:03:53,416 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:53,416 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:53,726 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:53,726 DEBUG --- stdout --- 2024-04-09 22:03:53,726 DEBUG 1 2024-04-09 22:03:53,726 DEBUG --- stderr --- 2024-04-09 22:03:53,726 DEBUG 2024-04-09 22:03:53,726 INFO 2024-04-09 22:03:53,726 INFO -------------- Waiting for 1 expected pod(s) to be ready -------------- 2024-04-09 22:03:53,727 INFO 2024-04-09 22:03:53,727 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments login-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1" 2024-04-09 22:03:53,727 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0] 2024-04-09 22:03:54,027 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:54,027 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:54,027 DEBUG --- stdout --- 2024-04-09 22:03:54,027 DEBUG ready:1 replicas:1 2024-04-09 22:03:54,027 DEBUG --- stderr --- 2024-04-09 22:03:54,028 DEBUG 2024-04-09 22:03:54,028 INFO 2024-04-09 22:03:54,028 INFO **************************** Checking ADMIN-UI component is running **************************** 2024-04-09 22:03:54,028 INFO 2024-04-09 22:03:54,028 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:03:54,028 INFO 2024-04-09 22:03:54,028 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:03:54,028 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:03:54,336 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:54,336 DEBUG --- stdout --- 2024-04-09 22:03:54,336 DEBUG 1 2024-04-09 22:03:54,336 DEBUG --- stderr --- 2024-04-09 22:03:54,336 DEBUG 2024-04-09 22:03:54,336 INFO 2024-04-09 22:03:54,336 INFO -------------- Waiting for 1 expected pod(s) to be ready -------------- 2024-04-09 22:03:54,337 INFO 2024-04-09 22:03:54,337 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments admin-ui --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1" 2024-04-09 22:03:54,337 INFO [loop_until]: (max_time=900, interval=30, expected_rc=[0] 2024-04-09 22:03:54,639 INFO [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found 2024-04-09 22:03:54,640 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:54,640 DEBUG --- stdout --- 2024-04-09 22:03:54,640 DEBUG ready:1 replicas:1 2024-04-09 22:03:54,640 DEBUG --- stderr --- 2024-04-09 22:03:54,640 DEBUG 2024-04-09 22:03:54,640 INFO 2024-04-09 22:03:54,640 INFO ****************************** Livecheck stage: After deployment ****************************** 2024-04-09 22:03:54,640 INFO 2024-04-09 22:03:54,640 INFO ---------------------- Running DS-CTS livecheck ---------------------- 2024-04-09 22:03:54,640 INFO Livecheck to ds-cts-0 2024-04-09 22:03:54,641 INFO 2024-04-09 22:03:54,641 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}" 2024-04-09 22:03:54,641 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0] 2024-04-09 22:03:54,941 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:03:54,941 DEBUG --- stdout --- 2024-04-09 22:03:54,942 DEBUG QTlONlYya01sN2F1Zmd2QmRaUWdHSHRoUUYzYk9aU2o= 2024-04-09 22:03:54,942 DEBUG --- stderr --- 2024-04-09 22:03:54,942 DEBUG 2024-04-09 22:03:54,942 INFO 2024-04-09 22:03:54,942 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive 2024-04-09 22:03:56,749 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 22:03:56,749 DEBUG --- stdout --- 2024-04-09 22:03:56,751 DEBUG dn: alive: true 2024-04-09 22:03:56,751 DEBUG --- stderr --- 2024-04-09 22:03:56,751 DEBUG 2024-04-09 22:03:56,751 INFO 2024-04-09 22:03:56,751 INFO Livecheck to ds-cts-1 2024-04-09 22:03:56,752 INFO 2024-04-09 22:03:56,752 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive 2024-04-09 22:03:58,631 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 22:03:58,631 DEBUG --- stdout --- 2024-04-09 22:03:58,631 DEBUG dn: alive: true 2024-04-09 22:03:58,631 DEBUG --- stderr --- 2024-04-09 22:03:58,631 DEBUG 2024-04-09 22:03:58,631 INFO 2024-04-09 22:03:58,631 INFO Livecheck to ds-cts-2 2024-04-09 22:03:58,631 INFO 2024-04-09 22:03:58,632 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive 2024-04-09 22:04:00,533 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 22:04:00,534 DEBUG --- stdout --- 2024-04-09 22:04:00,534 DEBUG dn: alive: true 2024-04-09 22:04:00,534 DEBUG --- stderr --- 2024-04-09 22:04:00,534 DEBUG 2024-04-09 22:04:00,534 INFO 2024-04-09 22:04:00,534 INFO 2024-04-09 22:04:00,534 INFO --------------------- Running DS-IDREPO livecheck --------------------- 2024-04-09 22:04:00,534 INFO Livecheck to ds-idrepo-0 2024-04-09 22:04:00,534 INFO 2024-04-09 22:04:00,534 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.dirmanager\.pw}" 2024-04-09 22:04:00,534 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0] 2024-04-09 22:04:00,871 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:00,871 DEBUG --- stdout --- 2024-04-09 22:04:00,871 DEBUG QTlONlYya01sN2F1Zmd2QmRaUWdHSHRoUUYzYk9aU2o= 2024-04-09 22:04:00,871 DEBUG --- stderr --- 2024-04-09 22:04:00,871 DEBUG 2024-04-09 22:04:00,872 INFO 2024-04-09 22:04:00,872 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive 2024-04-09 22:04:02,688 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 22:04:02,688 DEBUG --- stdout --- 2024-04-09 22:04:02,688 DEBUG dn: alive: true 2024-04-09 22:04:02,688 DEBUG --- stderr --- 2024-04-09 22:04:02,688 DEBUG 2024-04-09 22:04:02,688 INFO 2024-04-09 22:04:02,689 INFO Livecheck to ds-idrepo-1 2024-04-09 22:04:02,689 INFO 2024-04-09 22:04:02,689 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-1 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive 2024-04-09 22:04:04,566 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 22:04:04,566 DEBUG --- stdout --- 2024-04-09 22:04:04,566 DEBUG dn: alive: true 2024-04-09 22:04:04,566 DEBUG --- stderr --- 2024-04-09 22:04:04,566 DEBUG 2024-04-09 22:04:04,566 INFO 2024-04-09 22:04:04,566 INFO Livecheck to ds-idrepo-2 2024-04-09 22:04:04,566 INFO 2024-04-09 22:04:04,567 INFO [run_command]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-2 --container ds -- ldapsearch --noPropertiesFile --port 1389 --useStartTls --trustAll --bindDn "uid=admin" --bindPassword **** --baseDn "" --searchScope base "(&)" alive 2024-04-09 22:04:06,426 INFO [run_command]: OK (rc = 0 - expected to be in [0]) 2024-04-09 22:04:06,426 DEBUG --- stdout --- 2024-04-09 22:04:06,427 DEBUG dn: alive: true 2024-04-09 22:04:06,427 DEBUG --- stderr --- 2024-04-09 22:04:06,427 DEBUG 2024-04-09 22:04:06,427 INFO 2024-04-09 22:04:06,427 INFO 2024-04-09 22:04:06,427 INFO ------------------------ Running AM livecheck ------------------------ 2024-04-09 22:04:06,427 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready 2024-04-09 22:04:06,428 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready" 2024-04-09 22:04:06,705 INFO [http_cmd]: http status code OK 2024-04-09 22:04:06,706 DEBUG --- status code --- 2024-04-09 22:04:06,706 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:06,706 DEBUG --- http response --- 2024-04-09 22:04:06,706 DEBUG 2024-04-09 22:04:06,706 INFO 2024-04-09 22:04:06,706 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.AM_PASSWORDS_AMADMIN_CLEAR}" 2024-04-09 22:04:06,706 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0] 2024-04-09 22:04:07,089 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:07,089 DEBUG --- stdout --- 2024-04-09 22:04:07,089 DEBUG RWhTRm5sc2V0VmRTSDZxaldEeTVYYTBM 2024-04-09 22:04:07,089 DEBUG --- stderr --- 2024-04-09 22:04:07,089 DEBUG 2024-04-09 22:04:07,089 INFO Authenticate user amadmin via REST 2024-04-09 22:04:07,090 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/" 2024-04-09 22:04:07,413 INFO [http_cmd]: http status code OK 2024-04-09 22:04:07,413 DEBUG --- status code --- 2024-04-09 22:04:07,413 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:07,413 DEBUG --- http response --- 2024-04-09 22:04:07,413 DEBUG { "tokenId": "ezGJHEAuM3y4SQfH2lhzU_2f1iE.*AAJTSQACMDIAAlNLABxpc3Y4TlJRSThmNmhWZHdRR01ET0c1MytUOW89AAR0eXBlAANDVFMAAlMxAAIwMQ..*", "successUrl": "/am/console", "realm": "/" } 2024-04-09 22:04:07,416 INFO 2024-04-09 22:04:07,417 INFO ---------------------- Running AMSTER livecheck ---------------------- 2024-04-09 22:04:07,417 INFO 2024-04-09 22:04:07,417 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:04:07,417 INFO 2024-04-09 22:04:07,417 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm --field-selector status.phase!=Failed get pods --selector app=amster --output jsonpath={.items[*].metadata.name} 2024-04-09 22:04:07,417 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:04:07,781 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:07,782 DEBUG --- stdout --- 2024-04-09 22:04:07,782 DEBUG amster-l8fng 2024-04-09 22:04:07,783 DEBUG --- stderr --- 2024-04-09 22:04:07,783 DEBUG 2024-04-09 22:04:08,401 INFO Amster livecheck is passed 2024-04-09 22:04:08,402 INFO 2024-04-09 22:04:08,402 INFO ------------------------ Running IDM livecheck ------------------------ 2024-04-09 22:04:08,402 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping 2024-04-09 22:04:08,402 INFO 2024-04-09 22:04:08,402 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get secret **** --output jsonpath="{.data.OPENIDM_ADMIN_PASSWORD}" 2024-04-09 22:04:08,402 INFO [loop_until]: (max_time=60, interval=5, expected_rc=[0] 2024-04-09 22:04:08,719 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:08,719 DEBUG --- stdout --- 2024-04-09 22:04:08,719 DEBUG cnI1YXJuRDM5UUFsU0Q3U0JmbHRlU0N6 2024-04-09 22:04:08,719 DEBUG --- stderr --- 2024-04-09 22:04:08,719 DEBUG 2024-04-09 22:04:08,719 INFO Set admin password: rr5arnD39QAlSD7SBflteSCz 2024-04-09 22:04:08,720 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping" 2024-04-09 22:04:08,940 INFO [http_cmd]: http status code OK 2024-04-09 22:04:08,940 DEBUG --- status code --- 2024-04-09 22:04:08,940 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:08,940 DEBUG --- http response --- 2024-04-09 22:04:08,940 DEBUG { "_id": "", "_rev": "", "shortDesc": "OpenIDM ready", "state": "ACTIVE_READY" } 2024-04-09 22:04:08,940 INFO Create new LoginSession for user "amadmin" 2024-04-09 22:04:08,940 INFO [LoginSession] Get "access_token" token for user "amadmin" 2024-04-09 22:04:08,940 INFO [LoginSession] Obtaining new "access_token" token from server because it was not obtained yet for the user 2024-04-09 22:04:08,940 INFO [LoginSession] Get "token_id" token for user "amadmin" 2024-04-09 22:04:08,940 INFO [LoginSession] Obtaining new "token_id" token from server because it was not obtained yet for the user 2024-04-09 22:04:08,940 INFO Authenticate user amadmin via REST 2024-04-09 22:04:08,941 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/" 2024-04-09 22:04:09,654 INFO [http_cmd]: http status code OK 2024-04-09 22:04:09,655 DEBUG --- status code --- 2024-04-09 22:04:09,655 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:09,655 DEBUG --- http response --- 2024-04-09 22:04:09,655 DEBUG { "tokenId": "_hcWIyaaM0-7JS20bws5sys-Fic.*AAJTSQACMDIAAlNLABwvUitCcFBvS2cxT2wwZ013c2FIUTNkVFZXdEE9AAR0eXBlAANDVFMAAlMxAAIwMQ..*", "successUrl": "/am/console", "realm": "/" } 2024-04-09 22:04:09,655 INFO [LoginSession] Obtaining session info 2024-04-09 22:04:09,655 INFO [http_cmd]: curl -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/realms/root/sessions?_action=getSessionInfo" 2024-04-09 22:04:09,916 INFO [http_cmd]: http status code OK 2024-04-09 22:04:09,916 DEBUG --- status code --- 2024-04-09 22:04:09,917 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:09,917 DEBUG --- http response --- 2024-04-09 22:04:09,917 DEBUG { "username": "amadmin", "universalId": "id=amadmin,ou=user,ou=am-config", "realm": "/", "latestAccessTime": "2024-04-09T22:04:09Z", "maxIdleExpirationTime": "2024-04-09T22:07:09Z", "maxSessionExpirationTime": "2024-04-09T22:07:08Z", "properties": { "AMCtxId": "73c94e89-f792-4e9b-93b9-6ed32f125e0e-97" } } 2024-04-09 22:04:09,922 INFO Obtain Oauth2 authz code via REST 2024-04-09 22:04:09,922 DEBUG Oauth2 authorize for access token 2024-04-09 22:04:09,922 INFO [http_cmd]: curl --request POST --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=_hcWIyaaM0-7JS20bws5sys-Fic.*AAJTSQACMDIAAlNLABwvUitCcFBvS2cxT2wwZ013c2FIUTNkVFZXdEE9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" --cookie "route=1712700250.117.161643.741551|f60edb382037eb2df1e800d563ad78a7" --data '{"decision": "Allow", "csrf": "_hcWIyaaM0-7JS20bws5sys-Fic.*AAJTSQACMDIAAlNLABwvUitCcFBvS2cxT2wwZ013c2FIUTNkVFZXdEE9AAR0eXBlAANDVFMAAlMxAAIwMQ..*"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/authorize?client_id=smokeclient&scope=fr:idm:*&redirect_uri=https://fake.com&response_type=code&realm=/" 2024-04-09 22:04:10,279 INFO [http_cmd]: http status code OK 2024-04-09 22:04:10,279 DEBUG --- status code --- 2024-04-09 22:04:10,279 DEBUG http status code is 302 (expected 302) 2024-04-09 22:04:10,279 DEBUG --- http response --- 2024-04-09 22:04:10,279 DEBUG {'Date': 'Tue, 09 Apr 2024 22:04:10 GMT', 'Content-Length': '0', 'Connection': 'keep-alive', 'X-Frame-Options': 'SAMEORIGIN', 'X-Content-Type-Options': 'nosniff', 'Cache-Control': 'no-store', 'Location': 'https://fake.com?code=DXBEbbSupT-NG9YW1OQtgIVDjuY&iss=https%3A%2F%2Fxlou.iam.xlou-cdm.engineeringpit.com%2Fam%2Foauth2&client_id=smokeclient', 'Pragma': 'no-cache', 'Set-Cookie': 'OAUTH_REQUEST_ATTRIBUTES=DELETED; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/; Secure; HttpOnly; SameSite=none', 'Strict-Transport-Security': 'max-age=15724800; includeSubDomains'} 2024-04-09 22:04:10,280 DEBUG Oauth2 exchange authorize code for access token 2024-04-09 22:04:10,280 INFO [http_cmd]: curl -L --request POST --data '{"grant_type": "authorization_code", "redirect_uri": "https://fake.com", "client_id": "smokeclient", "code": "DXBEbbSupT-NG9YW1OQtgIVDjuY"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/am/oauth2/access_token?realm=/" 2024-04-09 22:04:10,621 INFO [http_cmd]: http status code OK 2024-04-09 22:04:10,621 DEBUG --- status code --- 2024-04-09 22:04:10,621 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:10,621 DEBUG --- http response --- 2024-04-09 22:04:10,622 DEBUG { "access_token": "1CVoX4K_jyNaDbIboL7UaC_RaHs", "scope": "fr:idm:*", "token_type": "Bearer", "expires_in": 3599 } 2024-04-09 22:04:10,623 INFO Additional check to sync/queue to ensure sync with DS is ok (see LODESTAR-1245) 2024-04-09 22:04:10,624 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/sync/queue?_queryFilter=true&_pageSize=1" 2024-04-09 22:04:10,958 INFO [http_cmd]: http status code OK 2024-04-09 22:04:10,959 DEBUG --- status code --- 2024-04-09 22:04:10,959 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:10,959 DEBUG --- http response --- 2024-04-09 22:04:10,959 DEBUG { "result": [], "resultCount": 0, "pagedResultsCookie": null, "totalPagedResultsPolicy": "NONE", "totalPagedResults": -1, "remainingPagedResults": -1 } 2024-04-09 22:04:10,960 INFO Try to create, query and delete a user 2024-04-09 22:04:10,960 INFO Create user idm_84kuw0y0kj in IDM 2024-04-09 22:04:10,961 INFO [http_cmd]: curl --header "Authorization: Bearer 1CVoX4K_jyNaDbIboL7UaC_RaHs" --header "Content-Type: application/json" --insecure -L --request POST --data '{"userName": "idm_84kuw0y0kj", "givenName": "Peter", "sn": "Parker", "password": "T35tr0ck123", "telephoneNumber": "6669876987", "description": "IDM Test User", "roles": [], "mail": "idm_84kuw0y0kj@forgerock.com"}' "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user?_action=create" 2024-04-09 22:04:11,376 INFO [http_cmd]: http status code OK 2024-04-09 22:04:11,376 DEBUG --- status code --- 2024-04-09 22:04:11,376 DEBUG http status code is 201 (expected [200, 201]) 2024-04-09 22:04:11,376 DEBUG --- http response --- 2024-04-09 22:04:11,376 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines): 2024-04-09 22:04:11,376 DEBUG ----- output ----- 2024-04-09 22:04:11,376 DEBUG { 2024-04-09 22:04:11,376 DEBUG "_id": "048f9799-d5d4-4282-9d35-6b1d53e3eda5", 2024-04-09 22:04:11,377 DEBUG "_rev": "19b49001-1866-4e35-b460-9fb49c7cc3be-68", 2024-04-09 22:04:11,377 DEBUG "country": null, 2024-04-09 22:04:11,377 DEBUG "mail": "idm_84kuw0y0kj@forgerock.com", 2024-04-09 22:04:11,377 DEBUG "memberOfOrgIDs": [], 2024-04-09 22:04:11,377 DEBUG "postalCode": null, 2024-04-09 22:04:11,377 DEBUG "profileImage": null, 2024-04-09 22:04:11,377 DEBUG "expireAccount": null, 2024-04-09 22:04:11,377 DEBUG "activeDate": null, 2024-04-09 22:04:11,377 DEBUG 2024-04-09 22:04:11,377 DEBUG 2024-04-09 22:04:11,377 DEBUG 2024-04-09 22:04:11,377 DEBUG [...] 2024-04-09 22:04:11,377 DEBUG 2024-04-09 22:04:11,377 DEBUG 2024-04-09 22:04:11,377 DEBUG 2024-04-09 22:04:11,377 DEBUG "aliasList": [], 2024-04-09 22:04:11,377 DEBUG "kbaInfo": [], 2024-04-09 22:04:11,377 DEBUG "inactiveDate": null, 2024-04-09 22:04:11,377 DEBUG "sn": "Parker", 2024-04-09 22:04:11,377 DEBUG "preferences": null, 2024-04-09 22:04:11,377 DEBUG "userName": "idm_84kuw0y0kj", 2024-04-09 22:04:11,377 DEBUG "passwordLastChangedTime": "2024-04-09T22:04:11.066Z", 2024-04-09 22:04:11,377 DEBUG "effectiveRoles": [], 2024-04-09 22:04:11,377 DEBUG "activateAccount": null 2024-04-09 22:04:11,377 DEBUG } 2024-04-09 22:04:11,377 INFO Query IDM to verify UserID 048f9799-d5d4-4282-9d35-6b1d53e3eda5 is visible 2024-04-09 22:04:11,377 INFO [http_cmd]: curl --insecure -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/048f9799-d5d4-4282-9d35-6b1d53e3eda5" 2024-04-09 22:04:11,595 INFO [http_cmd]: http status code OK 2024-04-09 22:04:11,595 DEBUG --- status code --- 2024-04-09 22:04:11,595 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:11,595 DEBUG --- http response --- 2024-04-09 22:04:11,595 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines): 2024-04-09 22:04:11,595 DEBUG ----- output ----- 2024-04-09 22:04:11,595 DEBUG { 2024-04-09 22:04:11,595 DEBUG "_id": "048f9799-d5d4-4282-9d35-6b1d53e3eda5", 2024-04-09 22:04:11,595 DEBUG "_rev": "19b49001-1866-4e35-b460-9fb49c7cc3be-68", 2024-04-09 22:04:11,596 DEBUG "country": null, 2024-04-09 22:04:11,596 DEBUG "mail": "idm_84kuw0y0kj@forgerock.com", 2024-04-09 22:04:11,596 DEBUG "memberOfOrgIDs": [], 2024-04-09 22:04:11,596 DEBUG "postalCode": null, 2024-04-09 22:04:11,596 DEBUG "profileImage": null, 2024-04-09 22:04:11,596 DEBUG "expireAccount": null, 2024-04-09 22:04:11,596 DEBUG "activeDate": null, 2024-04-09 22:04:11,596 DEBUG 2024-04-09 22:04:11,596 DEBUG 2024-04-09 22:04:11,596 DEBUG 2024-04-09 22:04:11,596 DEBUG [...] 2024-04-09 22:04:11,596 DEBUG 2024-04-09 22:04:11,596 DEBUG 2024-04-09 22:04:11,596 DEBUG 2024-04-09 22:04:11,596 DEBUG "aliasList": [], 2024-04-09 22:04:11,596 DEBUG "kbaInfo": [], 2024-04-09 22:04:11,596 DEBUG "inactiveDate": null, 2024-04-09 22:04:11,596 DEBUG "sn": "Parker", 2024-04-09 22:04:11,596 DEBUG "preferences": null, 2024-04-09 22:04:11,596 DEBUG "userName": "idm_84kuw0y0kj", 2024-04-09 22:04:11,596 DEBUG "passwordLastChangedTime": "2024-04-09T22:04:11.066Z", 2024-04-09 22:04:11,596 DEBUG "effectiveRoles": [], 2024-04-09 22:04:11,596 DEBUG "activateAccount": null 2024-04-09 22:04:11,596 DEBUG } 2024-04-09 22:04:11,596 INFO Authenticate as user idm_84kuw0y0kj 2024-04-09 22:04:11,597 INFO [http_cmd]: curl --insecure -L --request POST "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/authentication?_action=login" 2024-04-09 22:04:11,813 INFO [http_cmd]: http status code OK 2024-04-09 22:04:11,813 DEBUG --- status code --- 2024-04-09 22:04:11,813 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:11,813 DEBUG --- http response --- 2024-04-09 22:04:11,813 DEBUG { "_id": "login", "authorization": { "id": "anonymous", "roles": [ "internal/role/openidm-reg" ], "component": "internal/user" }, "authenticationId": "anonymous" } 2024-04-09 22:04:11,814 INFO [http_cmd]: curl --insecure -L --request DELETE "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/managed/user/048f9799-d5d4-4282-9d35-6b1d53e3eda5" 2024-04-09 22:04:12,107 INFO [http_cmd]: http status code OK 2024-04-09 22:04:12,108 DEBUG --- status code --- 2024-04-09 22:04:12,108 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:12,108 DEBUG --- http response --- 2024-04-09 22:04:12,108 DEBUG [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (34 lines): 2024-04-09 22:04:12,108 DEBUG ----- output ----- 2024-04-09 22:04:12,108 DEBUG { 2024-04-09 22:04:12,108 DEBUG "_id": "048f9799-d5d4-4282-9d35-6b1d53e3eda5", 2024-04-09 22:04:12,108 DEBUG "_rev": "19b49001-1866-4e35-b460-9fb49c7cc3be-68", 2024-04-09 22:04:12,108 DEBUG "country": null, 2024-04-09 22:04:12,109 DEBUG "mail": "idm_84kuw0y0kj@forgerock.com", 2024-04-09 22:04:12,109 DEBUG "memberOfOrgIDs": [], 2024-04-09 22:04:12,109 DEBUG "postalCode": null, 2024-04-09 22:04:12,109 DEBUG "profileImage": null, 2024-04-09 22:04:12,109 DEBUG "expireAccount": null, 2024-04-09 22:04:12,109 DEBUG "activeDate": null, 2024-04-09 22:04:12,109 DEBUG 2024-04-09 22:04:12,109 DEBUG 2024-04-09 22:04:12,109 DEBUG 2024-04-09 22:04:12,109 DEBUG [...] 2024-04-09 22:04:12,109 DEBUG 2024-04-09 22:04:12,109 DEBUG 2024-04-09 22:04:12,109 DEBUG 2024-04-09 22:04:12,109 DEBUG "aliasList": [], 2024-04-09 22:04:12,109 DEBUG "kbaInfo": [], 2024-04-09 22:04:12,109 DEBUG "inactiveDate": null, 2024-04-09 22:04:12,109 DEBUG "sn": "Parker", 2024-04-09 22:04:12,109 DEBUG "preferences": null, 2024-04-09 22:04:12,109 DEBUG "userName": "idm_84kuw0y0kj", 2024-04-09 22:04:12,109 DEBUG "passwordLastChangedTime": "2024-04-09T22:04:11.066Z", 2024-04-09 22:04:12,109 DEBUG "effectiveRoles": [], 2024-04-09 22:04:12,109 DEBUG "activateAccount": null 2024-04-09 22:04:12,109 DEBUG } 2024-04-09 22:04:12,112 INFO 2024-04-09 22:04:12,112 INFO -------------------- Running END-USER-UI livecheck -------------------- 2024-04-09 22:04:12,112 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/enduser 2024-04-09 22:04:12,112 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/enduser" 2024-04-09 22:04:12,395 INFO [http_cmd]: http status code OK 2024-04-09 22:04:12,395 DEBUG --- status code --- 2024-04-09 22:04:12,395 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:12,395 DEBUG --- http response --- 2024-04-09 22:04:12,396 DEBUG Identity Management
[] 2024-04-09 22:04:12,398 INFO 2024-04-09 22:04:12,398 INFO --------------------- Running LOGIN-UI livecheck --------------------- 2024-04-09 22:04:12,398 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI 2024-04-09 22:04:12,399 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/am/XUI" 2024-04-09 22:04:12,686 INFO [http_cmd]: http status code OK 2024-04-09 22:04:12,686 DEBUG --- status code --- 2024-04-09 22:04:12,686 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:12,686 DEBUG --- http response --- 2024-04-09 22:04:12,686 DEBUG Login
[] 2024-04-09 22:04:12,689 INFO 2024-04-09 22:04:12,690 INFO --------------------- Running ADMIN-UI livecheck --------------------- 2024-04-09 22:04:12,690 INFO Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/platform 2024-04-09 22:04:12,691 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/platform" 2024-04-09 22:04:12,971 INFO [http_cmd]: http status code OK 2024-04-09 22:04:12,972 DEBUG --- status code --- 2024-04-09 22:04:12,972 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:12,972 DEBUG --- http response --- 2024-04-09 22:04:12,972 DEBUG Platform Admin
[] 2024-04-09 22:04:12,974 INFO LIVECHECK SUCCEEDED 2024-04-09 22:04:12,975 INFO 2024-04-09 22:04:12,975 INFO **************************** Initializing component pods for DS-CTS **************************** 2024-04-09 22:04:12,975 INFO 2024-04-09 22:04:12,975 INFO --------------------- Get DS-CTS software version --------------------- 2024-04-09 22:04:12,975 INFO 2024-04-09 22:04:12,975 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-cts-0 --container ds -- find /opt/opendj -name opendj-core.jar 2024-04-09 22:04:12,975 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0] 2024-04-09 22:04:13,974 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:13,974 DEBUG --- stdout --- 2024-04-09 22:04:13,974 DEBUG /opt/opendj/lib/opendj-core.jar 2024-04-09 22:04:13,974 DEBUG --- stderr --- 2024-04-09 22:04:13,974 DEBUG 2024-04-09 22:04:13,975 INFO 2024-04-09 22:04:13,975 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-cts-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-cts_info/opendj-core.jar --container ds 2024-04-09 22:04:13,975 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:15,209 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:15,209 DEBUG --- stdout --- 2024-04-09 22:04:15,209 DEBUG tar: Removing leading `/' from member names 2024-04-09 22:04:15,209 DEBUG --- stderr --- 2024-04-09 22:04:15,210 DEBUG 2024-04-09 22:04:15,218 INFO 2024-04-09 22:04:15,219 INFO ************************** Initializing component pods for DS-IDREPO ************************** 2024-04-09 22:04:15,219 INFO 2024-04-09 22:04:15,219 INFO ------------------- Get DS-IDREPO software version ------------------- 2024-04-09 22:04:15,219 INFO 2024-04-09 22:04:15,219 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec ds-idrepo-0 --container ds -- find /opt/opendj -name opendj-core.jar 2024-04-09 22:04:15,219 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0] 2024-04-09 22:04:16,056 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:16,056 DEBUG --- stdout --- 2024-04-09 22:04:16,056 DEBUG /opt/opendj/lib/opendj-core.jar 2024-04-09 22:04:16,056 DEBUG --- stderr --- 2024-04-09 22:04:16,056 DEBUG 2024-04-09 22:04:16,057 INFO 2024-04-09 22:04:16,057 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp ds-idrepo-0:/opt/opendj/lib/opendj-core.jar /tmp/ds-idrepo_info/opendj-core.jar --container ds 2024-04-09 22:04:16,057 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:17,296 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:17,296 DEBUG --- stdout --- 2024-04-09 22:04:17,296 DEBUG tar: Removing leading `/' from member names 2024-04-09 22:04:17,297 DEBUG --- stderr --- 2024-04-09 22:04:17,297 DEBUG 2024-04-09 22:04:17,304 INFO 2024-04-09 22:04:17,304 INFO ****************************** Initializing component pods for AM ****************************** 2024-04-09 22:04:17,305 INFO 2024-04-09 22:04:17,305 INFO ----------------------- Get AM software version ----------------------- 2024-04-09 22:04:17,305 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version 2024-04-09 22:04:17,305 INFO [LoginSession] Get "token_id" token for user "amadmin" 2024-04-09 22:04:17,305 DEBUG [LoginSession] re-using "token_id" token that remains valid for 2m 41s 2024-04-09 22:04:17,306 INFO [http_cmd]: curl -L --request GET --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=_hcWIyaaM0-7JS20bws5sys-Fic.*AAJTSQACMDIAAlNLABwvUitCcFBvS2cxT2wwZ013c2FIUTNkVFZXdEE9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" --cookie "route=1712700250.117.161643.741551|f60edb382037eb2df1e800d563ad78a7" "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version" 2024-04-09 22:04:17,559 INFO [http_cmd]: http status code OK 2024-04-09 22:04:17,559 DEBUG --- status code --- 2024-04-09 22:04:17,559 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:17,559 DEBUG --- http response --- 2024-04-09 22:04:17,560 DEBUG { "_id": "version", "_rev": "-1131776936", "version": "7.6.0-SNAPSHOT", "fullVersion": "ForgeRock Access Management 7.6.0-SNAPSHOT Build 46abf9899004dcd77f0fdad8a152a3749b4a9c88 (2024-April-08 17:51)", "revision": "46abf9899004dcd77f0fdad8a152a3749b4a9c88", "date": "2024-April-08 17:51" } 2024-04-09 22:04:17,563 INFO 2024-04-09 22:04:17,563 INFO **************************** Initializing component pods for AMSTER **************************** 2024-04-09 22:04:17,563 INFO 2024-04-09 22:04:17,563 INFO ***************************** Initializing component pods for IDM ***************************** 2024-04-09 22:04:17,563 INFO 2024-04-09 22:04:17,563 INFO ---------------------- Get IDM software version ---------------------- 2024-04-09 22:04:17,563 INFO Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version 2024-04-09 22:04:17,564 INFO [http_cmd]: curl -L --request GET "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version" 2024-04-09 22:04:17,781 INFO [http_cmd]: http status code OK 2024-04-09 22:04:17,781 DEBUG --- status code --- 2024-04-09 22:04:17,781 DEBUG http status code is 200 (expected 200) 2024-04-09 22:04:17,781 DEBUG --- http response --- 2024-04-09 22:04:17,781 DEBUG { "_id": "version", "productVersion": "7.6.0-SNAPSHOT", "productBuildDate": "20240409083519", "productRevision": "5a54de0b49" } 2024-04-09 22:04:17,783 INFO 2024-04-09 22:04:17,784 INFO ************************* Initializing component pods for END-USER-UI ************************* 2024-04-09 22:04:17,784 INFO 2024-04-09 22:04:17,784 INFO ------------------ Get END-USER-UI software version ------------------ 2024-04-09 22:04:17,784 INFO 2024-04-09 22:04:17,784 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec end-user-ui-6d9fd5487b-2s47b --container end-user-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js 2024-04-09 22:04:17,784 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0] 2024-04-09 22:04:18,658 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:18,658 DEBUG --- stdout --- 2024-04-09 22:04:18,659 DEBUG /usr/share/nginx/html/js/chunk-vendors.a19ff3b3.js 2024-04-09 22:04:18,659 DEBUG --- stderr --- 2024-04-09 22:04:18,659 DEBUG 2024-04-09 22:04:18,659 INFO 2024-04-09 22:04:18,659 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp end-user-ui-6d9fd5487b-2s47b:/usr/share/nginx/html/js/chunk-vendors.a19ff3b3.js /tmp/end-user-ui_info/chunk-vendors.a19ff3b3.js --container end-user-ui 2024-04-09 22:04:18,659 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:19,857 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:19,857 DEBUG --- stdout --- 2024-04-09 22:04:19,857 DEBUG tar: Removing leading `/' from member names 2024-04-09 22:04:19,857 DEBUG --- stderr --- 2024-04-09 22:04:19,857 DEBUG 2024-04-09 22:04:19,886 INFO 2024-04-09 22:04:19,886 INFO *************************** Initializing component pods for LOGIN-UI *************************** 2024-04-09 22:04:19,886 INFO 2024-04-09 22:04:19,886 INFO -------------------- Get LOGIN-UI software version -------------------- 2024-04-09 22:04:19,887 INFO 2024-04-09 22:04:19,887 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec login-ui-5f964fb894-5zpvq --container login-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js 2024-04-09 22:04:19,887 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0] 2024-04-09 22:04:20,679 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:20,679 DEBUG --- stdout --- 2024-04-09 22:04:20,679 DEBUG /usr/share/nginx/html/js/chunk-vendors.4b29eede.js 2024-04-09 22:04:20,679 DEBUG --- stderr --- 2024-04-09 22:04:20,679 DEBUG 2024-04-09 22:04:20,680 INFO 2024-04-09 22:04:20,680 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp login-ui-5f964fb894-5zpvq:/usr/share/nginx/html/js/chunk-vendors.4b29eede.js /tmp/login-ui_info/chunk-vendors.4b29eede.js --container login-ui 2024-04-09 22:04:20,680 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:21,836 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:21,836 DEBUG --- stdout --- 2024-04-09 22:04:21,836 DEBUG tar: Removing leading `/' from member names 2024-04-09 22:04:21,836 DEBUG --- stderr --- 2024-04-09 22:04:21,836 DEBUG 2024-04-09 22:04:21,857 INFO 2024-04-09 22:04:21,857 INFO *************************** Initializing component pods for ADMIN-UI *************************** 2024-04-09 22:04:21,857 INFO 2024-04-09 22:04:21,857 INFO -------------------- Get ADMIN-UI software version -------------------- 2024-04-09 22:04:21,858 INFO 2024-04-09 22:04:21,858 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm exec admin-ui-7cf664f88d-pnt57 --container admin-ui -- find /usr/share/nginx/html -name chunk-vendors.*.js 2024-04-09 22:04:21,858 INFO [loop_until]: (max_time=30, interval=5, expected_rc=[0] 2024-04-09 22:04:22,655 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:22,655 DEBUG --- stdout --- 2024-04-09 22:04:22,655 DEBUG /usr/share/nginx/html/js/chunk-vendors.66372e8c.js 2024-04-09 22:04:22,655 DEBUG --- stderr --- 2024-04-09 22:04:22,655 DEBUG 2024-04-09 22:04:22,656 INFO 2024-04-09 22:04:22,656 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm cp admin-ui-7cf664f88d-pnt57:/usr/share/nginx/html/js/chunk-vendors.66372e8c.js /tmp/admin-ui_info/chunk-vendors.66372e8c.js --container admin-ui 2024-04-09 22:04:22,656 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:23,857 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:23,857 DEBUG --- stdout --- 2024-04-09 22:04:23,858 DEBUG tar: Removing leading `/' from member names 2024-04-09 22:04:23,858 DEBUG --- stderr --- 2024-04-09 22:04:23,859 DEBUG 2024-04-09 22:04:23,923 INFO 2024-04-09 22:04:23,923 INFO 2024-04-09 22:04:23,923 INFO ==================================================================================================== 2024-04-09 22:04:23,923 INFO ================ Admin password for DS-CTS is: A9N6V2kMl7aufgvBdZQgGHthQF3bOZSj ================ 2024-04-09 22:04:23,923 INFO ==================================================================================================== 2024-04-09 22:04:23,923 INFO 2024-04-09 22:04:23,923 INFO ==================================================================================================== 2024-04-09 22:04:23,923 INFO ============== Admin password for DS-IDREPO is: A9N6V2kMl7aufgvBdZQgGHthQF3bOZSj ============== 2024-04-09 22:04:23,924 INFO ==================================================================================================== 2024-04-09 22:04:23,924 INFO 2024-04-09 22:04:23,924 INFO ==================================================================================================== 2024-04-09 22:04:23,924 INFO ====================== Admin password for AM is: EhSFnlsetVdSH6qjWDy5Xa0L ====================== 2024-04-09 22:04:23,924 INFO ==================================================================================================== 2024-04-09 22:04:23,924 INFO 2024-04-09 22:04:23,924 INFO ==================================================================================================== 2024-04-09 22:04:23,924 INFO ===================== Admin password for IDM is: rr5arnD39QAlSD7SBflteSCz ===================== 2024-04-09 22:04:23,924 INFO ==================================================================================================== 2024-04-09 22:04:23,924 INFO 2024-04-09 22:04:23,924 INFO *************************************** Dumping pod list *************************************** 2024-04-09 22:04:23,924 INFO Dumping pod list to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/_pod-list.txt 2024-04-09 22:04:24,428 INFO Dumping platform-config configmap to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/_platform-config_configmap.txt 2024-04-09 22:04:24,768 INFO 2024-04-09 22:04:24,768 INFO **************************** Initializing component pods for DS-CTS **************************** 2024-04-09 22:04:24,768 INFO 2024-04-09 22:04:24,768 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:04:24,769 INFO 2024-04-09 22:04:24,769 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:04:24,769 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:25,141 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:25,141 DEBUG --- stdout --- 2024-04-09 22:04:25,141 DEBUG 3 2024-04-09 22:04:25,141 DEBUG --- stderr --- 2024-04-09 22:04:25,141 DEBUG 2024-04-09 22:04:25,141 INFO 2024-04-09 22:04:25,141 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:04:25,141 INFO 2024-04-09 22:04:25,141 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name} 2024-04-09 22:04:25,141 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:04:25,615 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:25,615 DEBUG --- stdout --- 2024-04-09 22:04:25,615 DEBUG ds-cts-0 ds-cts-1 ds-cts-2 2024-04-09 22:04:25,616 DEBUG --- stderr --- 2024-04-09 22:04:25,616 DEBUG 2024-04-09 22:04:25,616 INFO 2024-04-09 22:04:25,616 INFO ************************** Initializing component pods for DS-IDREPO ************************** 2024-04-09 22:04:25,616 INFO 2024-04-09 22:04:25,616 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:04:25,616 INFO 2024-04-09 22:04:25,616 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:04:25,616 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:25,987 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:25,988 DEBUG --- stdout --- 2024-04-09 22:04:25,988 DEBUG 3 2024-04-09 22:04:25,988 DEBUG --- stderr --- 2024-04-09 22:04:25,988 DEBUG 2024-04-09 22:04:25,988 INFO 2024-04-09 22:04:25,988 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:04:25,989 INFO 2024-04-09 22:04:25,989 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name} 2024-04-09 22:04:25,989 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:04:26,444 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:26,444 DEBUG --- stdout --- 2024-04-09 22:04:26,444 DEBUG ds-idrepo-0 ds-idrepo-1 ds-idrepo-2 2024-04-09 22:04:26,444 DEBUG --- stderr --- 2024-04-09 22:04:26,444 DEBUG 2024-04-09 22:04:26,445 INFO 2024-04-09 22:04:26,445 INFO ****************************** Initializing component pods for AM ****************************** 2024-04-09 22:04:26,445 INFO 2024-04-09 22:04:26,445 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:04:26,445 INFO 2024-04-09 22:04:26,445 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=am --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:04:26,445 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:26,837 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:26,837 DEBUG --- stdout --- 2024-04-09 22:04:26,837 DEBUG 3 2024-04-09 22:04:26,837 DEBUG --- stderr --- 2024-04-09 22:04:26,837 DEBUG 2024-04-09 22:04:26,837 INFO 2024-04-09 22:04:26,837 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:04:26,837 INFO 2024-04-09 22:04:26,838 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=am --output jsonpath={.items[*].metadata.name} 2024-04-09 22:04:26,838 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:04:27,286 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:27,286 DEBUG --- stdout --- 2024-04-09 22:04:27,286 DEBUG am-654c4fcbcf-487cv am-654c4fcbcf-9scrj am-654c4fcbcf-v5qb5 2024-04-09 22:04:27,287 DEBUG --- stderr --- 2024-04-09 22:04:27,287 DEBUG 2024-04-09 22:04:27,287 INFO 2024-04-09 22:04:27,287 INFO **************************** Initializing component pods for AMSTER **************************** 2024-04-09 22:04:27,287 INFO 2024-04-09 22:04:27,287 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:04:27,287 INFO 2024-04-09 22:04:27,287 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=amster --output jsonpath={.items[*].metadata.name} 2024-04-09 22:04:27,287 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:04:27,666 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:27,666 DEBUG --- stdout --- 2024-04-09 22:04:27,666 DEBUG amster-l8fng 2024-04-09 22:04:27,666 DEBUG --- stderr --- 2024-04-09 22:04:27,666 DEBUG 2024-04-09 22:04:27,666 INFO 2024-04-09 22:04:27,666 INFO ***************************** Initializing component pods for IDM ***************************** 2024-04-09 22:04:27,666 INFO 2024-04-09 22:04:27,667 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:04:27,667 INFO 2024-04-09 22:04:27,667 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:04:27,667 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:28,039 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:28,039 DEBUG --- stdout --- 2024-04-09 22:04:28,040 DEBUG 2 2024-04-09 22:04:28,040 DEBUG --- stderr --- 2024-04-09 22:04:28,040 DEBUG 2024-04-09 22:04:28,040 INFO 2024-04-09 22:04:28,040 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:04:28,040 INFO 2024-04-09 22:04:28,040 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=idm --output jsonpath={.items[*].metadata.name} 2024-04-09 22:04:28,040 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:04:28,414 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:28,415 DEBUG --- stdout --- 2024-04-09 22:04:28,415 DEBUG idm-6bfb7c96cc-qklk9 idm-6bfb7c96cc-vsb58 2024-04-09 22:04:28,415 DEBUG --- stderr --- 2024-04-09 22:04:28,415 DEBUG 2024-04-09 22:04:28,415 INFO 2024-04-09 22:04:28,415 INFO ************************* Initializing component pods for END-USER-UI ************************* 2024-04-09 22:04:28,415 INFO 2024-04-09 22:04:28,415 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:04:28,415 INFO 2024-04-09 22:04:28,415 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:04:28,415 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:28,726 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:28,726 DEBUG --- stdout --- 2024-04-09 22:04:28,726 DEBUG 1 2024-04-09 22:04:28,726 DEBUG --- stderr --- 2024-04-09 22:04:28,726 DEBUG 2024-04-09 22:04:28,727 INFO 2024-04-09 22:04:28,727 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:04:28,727 INFO 2024-04-09 22:04:28,727 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name} 2024-04-09 22:04:28,727 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:04:29,037 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:29,039 DEBUG --- stdout --- 2024-04-09 22:04:29,039 DEBUG end-user-ui-6d9fd5487b-2s47b 2024-04-09 22:04:29,039 DEBUG --- stderr --- 2024-04-09 22:04:29,039 DEBUG 2024-04-09 22:04:29,039 INFO 2024-04-09 22:04:29,039 INFO *************************** Initializing component pods for LOGIN-UI *************************** 2024-04-09 22:04:29,039 INFO 2024-04-09 22:04:29,039 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:04:29,040 INFO 2024-04-09 22:04:29,040 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:04:29,040 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:29,346 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:29,346 DEBUG --- stdout --- 2024-04-09 22:04:29,346 DEBUG 1 2024-04-09 22:04:29,346 DEBUG --- stderr --- 2024-04-09 22:04:29,346 DEBUG 2024-04-09 22:04:29,346 INFO 2024-04-09 22:04:29,347 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:04:29,347 INFO 2024-04-09 22:04:29,347 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name} 2024-04-09 22:04:29,347 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:04:29,689 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:29,689 DEBUG --- stdout --- 2024-04-09 22:04:29,689 DEBUG login-ui-5f964fb894-5zpvq 2024-04-09 22:04:29,690 DEBUG --- stderr --- 2024-04-09 22:04:29,690 DEBUG 2024-04-09 22:04:29,690 INFO 2024-04-09 22:04:29,690 INFO *************************** Initializing component pods for ADMIN-UI *************************** 2024-04-09 22:04:29,690 INFO 2024-04-09 22:04:29,690 INFO --------------------- Get expected number of pods --------------------- 2024-04-09 22:04:29,690 INFO 2024-04-09 22:04:29,690 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas} 2024-04-09 22:04:29,690 INFO [loop_until]: (max_time=180, interval=5, expected_rc=[0] 2024-04-09 22:04:30,063 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:30,063 DEBUG --- stdout --- 2024-04-09 22:04:30,063 DEBUG 1 2024-04-09 22:04:30,063 DEBUG --- stderr --- 2024-04-09 22:04:30,063 DEBUG 2024-04-09 22:04:30,063 INFO 2024-04-09 22:04:30,063 INFO ---------------------------- Get pod list ---------------------------- 2024-04-09 22:04:30,064 INFO 2024-04-09 22:04:30,064 INFO [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name} 2024-04-09 22:04:30,064 INFO [loop_until]: (max_time=180, interval=10, expected_rc=[0] 2024-04-09 22:04:30,410 INFO [loop_until]: OK (rc = 0) 2024-04-09 22:04:30,410 DEBUG --- stdout --- 2024-04-09 22:04:30,411 DEBUG admin-ui-7cf664f88d-pnt57 2024-04-09 22:04:30,411 DEBUG --- stderr --- 2024-04-09 22:04:30,411 DEBUG 2024-04-09 22:04:30,411 INFO 2024-04-09 22:04:30,411 INFO *********************************** Dumping components logs *********************************** 2024-04-09 22:04:30,411 INFO 2024-04-09 22:04:30,411 INFO ----------------------- Dumping logs for DS-CTS ----------------------- 2024-04-09 22:04:31,522 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-cts-0.txt 2024-04-09 22:04:31,522 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-cts-0.txt 2024-04-09 22:04:31,523 INFO Check pod logs for errors 2024-04-09 22:04:32,622 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-cts-1.txt 2024-04-09 22:04:32,622 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-cts-1.txt 2024-04-09 22:04:32,623 INFO Check pod logs for errors 2024-04-09 22:04:33,685 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-cts-2.txt 2024-04-09 22:04:33,685 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-cts-2.txt 2024-04-09 22:04:33,685 INFO Check pod logs for errors 2024-04-09 22:04:33,686 INFO 2024-04-09 22:04:33,686 INFO --------------------- Dumping logs for DS-IDREPO --------------------- 2024-04-09 22:04:34,770 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-idrepo-0.txt 2024-04-09 22:04:34,771 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-idrepo-0.txt 2024-04-09 22:04:34,771 INFO Check pod logs for errors 2024-04-09 22:04:35,887 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-idrepo-1.txt 2024-04-09 22:04:35,887 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-idrepo-1.txt 2024-04-09 22:04:35,888 INFO Check pod logs for errors 2024-04-09 22:04:36,993 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-idrepo-2.txt 2024-04-09 22:04:36,993 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/ds-idrepo-2.txt 2024-04-09 22:04:36,994 INFO Check pod logs for errors 2024-04-09 22:04:36,994 INFO 2024-04-09 22:04:36,994 INFO ------------------------- Dumping logs for AM ------------------------- 2024-04-09 22:04:38,232 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/am-654c4fcbcf-487cv.txt 2024-04-09 22:04:38,233 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/am-654c4fcbcf-487cv.txt 2024-04-09 22:04:38,233 INFO Check pod logs for errors 2024-04-09 22:04:39,577 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/am-654c4fcbcf-9scrj.txt 2024-04-09 22:04:39,578 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/am-654c4fcbcf-9scrj.txt 2024-04-09 22:04:39,579 INFO Check pod logs for errors 2024-04-09 22:04:40,832 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/am-654c4fcbcf-v5qb5.txt 2024-04-09 22:04:40,833 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/am-654c4fcbcf-v5qb5.txt 2024-04-09 22:04:40,833 INFO Check pod logs for errors 2024-04-09 22:04:40,833 INFO 2024-04-09 22:04:40,833 INFO ----------------------- Dumping logs for AMSTER ----------------------- 2024-04-09 22:04:42,009 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/amster-l8fng.txt 2024-04-09 22:04:42,010 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/amster-l8fng.txt 2024-04-09 22:04:42,010 INFO Check pod logs for errors 2024-04-09 22:04:42,010 INFO 2024-04-09 22:04:42,010 INFO ------------------------ Dumping logs for IDM ------------------------ 2024-04-09 22:04:43,391 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/idm-6bfb7c96cc-qklk9.txt 2024-04-09 22:04:43,392 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/idm-6bfb7c96cc-qklk9.txt 2024-04-09 22:04:43,393 INFO Check pod logs for errors 2024-04-09 22:04:44,737 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/idm-6bfb7c96cc-vsb58.txt 2024-04-09 22:04:44,738 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/idm-6bfb7c96cc-vsb58.txt 2024-04-09 22:04:44,740 INFO Check pod logs for errors 2024-04-09 22:04:44,741 INFO 2024-04-09 22:04:44,741 INFO -------------------- Dumping logs for END-USER-UI -------------------- 2024-04-09 22:04:45,687 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/end-user-ui-6d9fd5487b-2s47b.txt 2024-04-09 22:04:45,688 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/end-user-ui-6d9fd5487b-2s47b.txt 2024-04-09 22:04:45,688 INFO Check pod logs for errors 2024-04-09 22:04:45,688 INFO 2024-04-09 22:04:45,688 INFO ---------------------- Dumping logs for LOGIN-UI ---------------------- 2024-04-09 22:04:46,626 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/login-ui-5f964fb894-5zpvq.txt 2024-04-09 22:04:46,626 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/login-ui-5f964fb894-5zpvq.txt 2024-04-09 22:04:46,627 INFO Check pod logs for errors 2024-04-09 22:04:46,627 INFO 2024-04-09 22:04:46,627 INFO ---------------------- Dumping logs for ADMIN-UI ---------------------- 2024-04-09 22:04:47,517 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/admin-ui-7cf664f88d-pnt57.txt 2024-04-09 22:04:47,518 INFO Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/authn_rest/pod-logs/stack/20240409_220423-after-deployment/admin-ui-7cf664f88d-pnt57.txt 2024-04-09 22:04:47,518 INFO Check pod logs for errors [2024-04-09 22:04:47] - INFO: Deployment successful ________________________________________________________________________________ [2024-04-09 22:04:47] Deploy_all_forgerock_components post : Post method ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped