--Task--
              name: Verification_idm
           enabled: True
        class_name: VerificationTask
       source_name: idm
  source_namespace: >default<
       target_name: idm
  target_namespace: >default<
             start: 0
              stop: None
           timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow
              loop: False
          interval: None
      dependencies: []
          wait_for: []
           options: {}
        group_name: None
       Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock

________________________________________________________________________________

[13/Jun/2023 22:10:27] Verification_idm pre : N/A
________________________________________________________________________________

[13/Jun/2023 22:10:27] - INFO: Timeout for this Task has changed to 12h (43200 seconds)
[13/Jun/2023 22:10:27] - INFO: timeout was set to Task default (2x global_duration or 300 seconds (whatever is higher)), because it was unset
________________________________________________________________________________

[13/Jun/2023 22:10:27] Verification_idm step1 : Wait until pods are in Running state and check number of running pods
________________________________________________________________________________


---------------------   Get expected number of pods   ---------------------

[loop_until]: kubectl --namespace=xlou get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
[loop_until]: (max_time=180, interval=5, expected_rc=[0]
[loop_until]: OK (rc = 0)
--- stdout ---
2
--- stderr ---


--------------   Waiting for 2 expected pod(s) to be ready   --------------

[loop_until]: kubectl --namespace=xlou get deployment idm --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:2 replicas:2"
[loop_until]: (max_time=900, interval=30, expected_rc=[0]
[loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
[loop_until]: OK (rc = 0)
--- stdout ---
ready:2 replicas:2
--- stderr ---

________________________________________________________________________________

[13/Jun/2023 22:10:28] Verification_idm step2 : Discovering pods
________________________________________________________________________________


---------------------   Get expected number of pods   ---------------------

[loop_until]: kubectl --namespace=xlou get deployment --selector app=idm --output jsonpath={.items[*].spec.replicas}
[loop_until]: (max_time=180, interval=5, expected_rc=[0]
[loop_until]: OK (rc = 0)
--- stdout ---
2
--- stderr ---


----------------------------    Get pod list   ----------------------------

[loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=idm --output jsonpath={.items[*].metadata.name}` | grep 2
[loop_until]: (max_time=180, interval=10, expected_rc=[0]
[loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
[loop_until]: OK (rc = 0)
--- stdout ---
idm-58dc667486-ql6zm idm-58dc667486-rgz9j
--- stderr ---

________________________________________________________________________________

[13/Jun/2023 22:10:29] Verification_idm step3 : Check component is alive
________________________________________________________________________________

Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping
Set admin password: 1V1ONdTPKlxSJ80DChfr84jq
[http_cmd]: curl --header "X-OpenIDM-Username: anonymous" --header "X-OpenIDM-Password: anonymous" -L --request GET  "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/ping"
[http_cmd]: http status code OK
---  status code  ---
http status code is 200 (expected 200)
--- http response ---
{
    "_id": "",
    "_rev": "",
    "shortDesc": "OpenIDM ready",
    "state": "ACTIVE_READY"
}

PASS : component idm is alive
________________________________________________________________________________

[13/Jun/2023 22:10:29] Verification_idm step4 : Record version
________________________________________________________________________________


----------------------    Get IDM software version   ----------------------
Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version
[http_cmd]: curl --header "X-OpenIDM-Username: anonymous" --header "X-OpenIDM-Password: anonymous" -L --request GET  "https://xlou.iam.xlou-cdm.engineeringpit.com/openidm/info/version"
[http_cmd]: http status code OK
---  status code  ---
http status code is 200 (expected 200)
--- http response ---
{
    "_id": "version",
    "productVersion": "7.4.0-SNAPSHOT",
    "productBuildDate": "20230606162121",
    "productRevision": "e1b90e8693"
}

Version is 7.4.0-SNAPSHOT
Revision is e1b90e8693
________________________________________________________________________________

[13/Jun/2023 22:10:29] Verification_idm step5 :  Checking pod restart counts 
________________________________________________________________________________


------------    Check pod idm-58dc667486-ql6zm restart count   ------------

[loop_until]: kubectl --namespace=xlou get pod idm-58dc667486-ql6zm --output jsonpath={.status.containerStatuses[*].restartCount}
[loop_until]: (max_time=180, interval=5, expected_rc=[0]
[loop_until]: OK (rc = 0)
--- stdout ---
1
--- stderr ---

Pod idm-58dc667486-ql6zm has been restarted 1 times.
ERROR: Pod "idm-58dc667486-ql6zm" was restarted 1 times after validation phase.
Here are its restarted containers and their reason for restart:
- openidm: terminated - Error

Setting result to FAIL

------------    Check pod idm-58dc667486-rgz9j restart count   ------------

[loop_until]: kubectl --namespace=xlou get pod idm-58dc667486-rgz9j --output jsonpath={.status.containerStatuses[*].restartCount}
[loop_until]: (max_time=180, interval=5, expected_rc=[0]
[loop_until]: OK (rc = 0)
--- stdout ---
0
--- stderr ---

Pod idm-58dc667486-rgz9j has been restarted 0 times.
SUCCESS : pod restart count (0) did not change between validation and verification phases
________________________________________________________________________________

[13/Jun/2023 22:10:30] Verification_idm post : Post method
________________________________________________________________________________

Task has been successfully stopped