--Task-- name: Dump_logs_scenario enabled: True class_name: DumpLogs source_name: controller source_namespace: >default< target_name: controller target_namespace: >default< start: 0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: [] wait_for: ['mixedquery-10', 'mixedquery-20', 'mixedquery-30', 'mixedquery-40', 'mixedquery-50', 'mixedquery-60', 'mixedquery-70', 'mixedquery-80'] options: {'log-suffix': 'scenario'} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [11/Aug/2023 20:05:49] Dump_logs_scenario pre : N/A ________________________________________________________________________________ [11/Aug/2023 20:05:49] - INFO: Timeout for this Task has changed to 8h (28800 seconds) [11/Aug/2023 20:05:49] - INFO: timeout was set to Task default (2x global_duration or 300 seconds (whatever is higher)), because it was unset ________________________________________________________________________________ [11/Aug/2023 20:05:49] Dump_logs_scenario step1 : Dump component logs ________________________________________________________________________________ *************************************** Dumping pod list *************************************** Dumping pod list to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/_pod-list.txt **************************** Initializing component pods for DS-CTS **************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 3 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- ds-cts-0 ds-cts-1 ds-cts-2 --- stderr --- ************************** Initializing component pods for DS-IDREPO ************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 3 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- ds-idrepo-0 ds-idrepo-1 ds-idrepo-2 --- stderr --- ****************************** Initializing component pods for AM ****************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app=am --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 3 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=am --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- am-55f77847b7-7qk7g am-55f77847b7-ngpns am-55f77847b7-q6zcv --- stderr --- **************************** Initializing component pods for AMSTER **************************** ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=amster --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- amster-7mxkl --- stderr --- ***************************** Initializing component pods for IDM ***************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 2 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=idm --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- idm-65858d8c4c-5kwbg idm-65858d8c4c-8ff69 --- stderr --- ************************* Initializing component pods for END-USER-UI ************************* --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 1 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- end-user-ui-6845bc78c7-m5k2c --- stderr --- *************************** Initializing component pods for LOGIN-UI *************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 1 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- login-ui-74d6fb46c-2qx2r --- stderr --- *************************** Initializing component pods for ADMIN-UI *************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 1 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- admin-ui-587fc66dd5-4p7xb --- stderr --- ************************** Initializing component pods for OVERSEER-0 ************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app=overseer-0 --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 1 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- overseer-0-5fcfb8f45c-v6ck5 --- stderr --- *********************************** Dumping components logs *********************************** ----------------------- Dumping logs for DS-CTS ----------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/ds-cts-0.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/ds-cts-1.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/ds-cts-2.txt Check pod logs for errors --------------------- Dumping logs for DS-IDREPO --------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/ds-idrepo-0.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/ds-idrepo-1.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/ds-idrepo-2.txt Check pod logs for errors Ignored error: {"eventName":"DJ-SERVER","timestamp":"2023-08-11T15:20:16.298Z","category":"SYNC.CONNECTIONS","severity":"ERROR","msgId":377,"message":"Changelog on replication server 'ds-idrepo-2' no longer contains changes for domain 'dc=openidm,dc=forgerock,dc=io' which are required by replica 'ds-idrepo-2'. The replica will no longer receive replicated changes and must be re-initialized."} Ignored error: {"eventName":"DJ-SERVER","timestamp":"2023-08-11T15:20:16.699Z","category":"SYNC.CONNECTIONS","severity":"ERROR","msgId":377,"message":"Changelog on replication server 'ds-idrepo-2' no longer contains changes for domain 'ou=identities' which are required by replica 'ds-idrepo-2'. The replica will no longer receive replicated changes and must be re-initialized."} ------------------------- Dumping logs for AM ------------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/am-55f77847b7-7qk7g.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/am-55f77847b7-ngpns.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/am-55f77847b7-q6zcv.txt Check pod logs for errors ----------------------- Dumping logs for AMSTER ----------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/amster-7mxkl.txt Check pod logs for errors ------------------------ Dumping logs for IDM ------------------------ Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/idm-65858d8c4c-5kwbg.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/idm-65858d8c4c-8ff69.txt Check pod logs for errors -------------------- Dumping logs for END-USER-UI -------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/end-user-ui-6845bc78c7-m5k2c.txt Check pod logs for errors ---------------------- Dumping logs for LOGIN-UI ---------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/login-ui-74d6fb46c-2qx2r.txt Check pod logs for errors ---------------------- Dumping logs for ADMIN-UI ---------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/admin-ui-587fc66dd5-4p7xb.txt Check pod logs for errors --------------------- Dumping logs for OVERSEER-0 --------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230811-200549-after-scenario/overseer-0-5fcfb8f45c-v6ck5.txt Check pod logs for errors ________________________________________________________________________________ [11/Aug/2023 20:06:15] Dump_logs_scenario post : Post method ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped