--Task-- name: Dump_logs_scenario enabled: True class_name: DumpLogs source_name: controller source_namespace: >default< target_name: controller target_namespace: >default< start: 0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: [] wait_for: ['mixedquery-10', 'mixedquery-20', 'mixedquery-30', 'mixedquery-40', 'mixedquery-50', 'mixedquery-60', 'mixedquery-70', 'mixedquery-80'] options: {'log-suffix': 'scenario'} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [16/Aug/2023 06:08:47] Dump_logs_scenario pre : N/A ________________________________________________________________________________ [16/Aug/2023 06:08:47] - INFO: Timeout for this Task has changed to 8h (28800 seconds) [16/Aug/2023 06:08:47] - INFO: timeout was set to Task default (2x global_duration or 300 seconds (whatever is higher)), because it was unset ________________________________________________________________________________ [16/Aug/2023 06:08:47] Dump_logs_scenario step1 : Dump component logs ________________________________________________________________________________ *************************************** Dumping pod list *************************************** Dumping pod list to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/_pod-list.txt **************************** Initializing component pods for DS-CTS **************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get statefulsets --selector app=ds-cts --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 3 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=ds-cts --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- ds-cts-0 ds-cts-1 ds-cts-2 --- stderr --- ************************** Initializing component pods for DS-IDREPO ************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get statefulsets --selector app=ds-idrepo --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 3 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=ds-idrepo --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- ds-idrepo-0 ds-idrepo-1 ds-idrepo-2 --- stderr --- ****************************** Initializing component pods for AM ****************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app=am --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 3 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=am --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- am-869fdb5db9-5j69v am-869fdb5db9-8dg94 am-869fdb5db9-wt7sg --- stderr --- **************************** Initializing component pods for AMSTER **************************** ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=amster --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- amster-85vcv --- stderr --- ***************************** Initializing component pods for IDM ***************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 2 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=idm --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- idm-65858d8c4c-d6c9h idm-65858d8c4c-pt5s9 --- stderr --- ************************* Initializing component pods for END-USER-UI ************************* --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 1 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app.kubernetes.io/name=end-user-ui --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- end-user-ui-6845bc78c7-sqnhx --- stderr --- *************************** Initializing component pods for LOGIN-UI *************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 1 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app.kubernetes.io/name=login-ui --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- login-ui-74d6fb46c-qcg59 --- stderr --- *************************** Initializing component pods for ADMIN-UI *************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 1 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app.kubernetes.io/name=admin-ui --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- admin-ui-587fc66dd5-w2rbv --- stderr --- ************************** Initializing component pods for OVERSEER-0 ************************** --------------------- Get expected number of pods --------------------- [loop_until]: kubectl --namespace=xlou get deployments --selector app=overseer-0 --output jsonpath={.items[*].spec.replicas} [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- 1 --- stderr --- ---------------------------- Get pod list ---------------------------- [loop_until]: kubectl --namespace=xlou get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name} [loop_until]: (max_time=180, interval=10, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- overseer-0-788b4494cc-bdwtm --- stderr --- *********************************** Dumping components logs *********************************** ----------------------- Dumping logs for DS-CTS ----------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/ds-cts-0.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/ds-cts-1.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/ds-cts-2.txt Check pod logs for errors --------------------- Dumping logs for DS-IDREPO --------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/ds-idrepo-0.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/ds-idrepo-1.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/ds-idrepo-2.txt Check pod logs for errors Ignored error: {"eventName":"DJ-SERVER","timestamp":"2023-08-16T01:22:29.094Z","category":"SYNC.CONNECTIONS","severity":"ERROR","msgId":377,"message":"Changelog on replication server 'ds-idrepo-2' no longer contains changes for domain 'dc=openidm,dc=forgerock,dc=io' which are required by replica 'ds-idrepo-2'. The replica will no longer receive replicated changes and must be re-initialized."} Ignored error: {"eventName":"DJ-SERVER","timestamp":"2023-08-16T01:22:29.495Z","category":"SYNC.CONNECTIONS","severity":"ERROR","msgId":377,"message":"Changelog on replication server 'ds-idrepo-2' no longer contains changes for domain 'ou=identities' which are required by replica 'ds-idrepo-2'. The replica will no longer receive replicated changes and must be re-initialized."} ------------------------- Dumping logs for AM ------------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/am-869fdb5db9-5j69v.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/am-869fdb5db9-8dg94.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/am-869fdb5db9-wt7sg.txt Check pod logs for errors ----------------------- Dumping logs for AMSTER ----------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/amster-85vcv.txt Check pod logs for errors ------------------------ Dumping logs for IDM ------------------------ Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/idm-65858d8c4c-d6c9h.txt Check pod logs for errors Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/idm-65858d8c4c-pt5s9.txt Check pod logs for errors -------------------- Dumping logs for END-USER-UI -------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/end-user-ui-6845bc78c7-sqnhx.txt Check pod logs for errors ---------------------- Dumping logs for LOGIN-UI ---------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/login-ui-74d6fb46c-qcg59.txt Check pod logs for errors ---------------------- Dumping logs for ADMIN-UI ---------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/admin-ui-587fc66dd5-w2rbv.txt Check pod logs for errors --------------------- Dumping logs for OVERSEER-0 --------------------- Dumping pod description and logs to /mnt/disks/data/xslou/lodestar-fork/results/pyrock/simple_managed_users/pod-logs/stack/20230816-060847-after-scenario/overseer-0-788b4494cc-bdwtm.txt Check pod logs for errors ________________________________________________________________________________ [16/Aug/2023 06:09:12] Dump_logs_scenario post : Post method ________________________________________________________________________________ Setting result to PASS Task has been successfully stopped