--Task--
              name: Verification_am
           enabled: True
        class_name: VerificationTask
       source_name: am
  source_namespace: >default<
       target_name: am
  target_namespace: >default<
             start: 0
              stop: None
           timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow
              loop: False
          interval: None
      dependencies: []
          wait_for: []
           options: {}
        group_name: None
       Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock

________________________________________________________________________________

[16/Aug/2023 06:09:13] Verification_am pre : N/A
________________________________________________________________________________

[16/Aug/2023 06:09:13] - INFO: Timeout for this Task has changed to 8h (28800 seconds)
[16/Aug/2023 06:09:13] - INFO: timeout was set to Task default (2x global_duration or 300 seconds (whatever is higher)), because it was unset
________________________________________________________________________________

[16/Aug/2023 06:09:13] Verification_am step1 : Wait until pods are in Running state and check number of running pods
________________________________________________________________________________


---------------------   Get expected number of pods   ---------------------

[loop_until]: kubectl --namespace=xlou get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
[loop_until]: (max_time=180, interval=5, expected_rc=[0]
[loop_until]: OK (rc = 0)
--- stdout ---
3
--- stderr ---


--------------   Waiting for 3 expected pod(s) to be ready   --------------

[loop_until]: kubectl --namespace=xlou get deployments am --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:3 replicas:3"
[loop_until]: (max_time=900, interval=30, expected_rc=[0]
[loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
[loop_until]: OK (rc = 0)
--- stdout ---
ready:3 replicas:3
--- stderr ---

________________________________________________________________________________

[16/Aug/2023 06:09:15] Verification_am step2 : Discovering pods
________________________________________________________________________________


---------------------   Get expected number of pods   ---------------------

[loop_until]: kubectl --namespace=xlou get deployments --selector app=am --output jsonpath={.items[*].spec.replicas}
[loop_until]: (max_time=180, interval=5, expected_rc=[0]
[loop_until]: OK (rc = 0)
--- stdout ---
3
--- stderr ---


----------------------------    Get pod list   ----------------------------

[loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou get pods --selector app=am --output jsonpath={.items[*].metadata.name}` | grep 3
[loop_until]: (max_time=180, interval=10, expected_rc=[0]
[loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
[loop_until]: OK (rc = 0)
--- stdout ---
am-869fdb5db9-5j69v am-869fdb5db9-8dg94 am-869fdb5db9-wt7sg
--- stderr ---

________________________________________________________________________________

[16/Aug/2023 06:09:16] Verification_am step3 : Check component is alive
________________________________________________________________________________

Livecheck to https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready
[http_cmd]: curl -L --request GET  "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/health/ready"
[http_cmd]: http status code OK
---  status code  ---
http status code is 200 (expected 200)
--- http response ---


Authenticate user amadmin via REST
[http_cmd]: curl --header "X-OpenAM-Username: amadmin" --header "X-OpenAM-Password: jiJ0ZhJJjEp6J90eFtnzIOXZ" --header "Content-Type: application/json" --header "Accept-API-Version: resource=2.0, protocol=1.0" -L --request POST  "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
[http_cmd]: http status code OK
---  status code  ---
http status code is 200 (expected 200)
--- http response ---
{
    "tokenId": "hjDSbTVmcDZV40PxxBxOL1VfcDo.*AAJTSQACMDIAAlNLABxnWm1zR2k5cnl5cWN5dkUvNHdEMUtoNUJqS3c9AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
    "successUrl": "/am/console",
    "realm": "/"
}

PASS : component am is alive
________________________________________________________________________________

[16/Aug/2023 06:09:16] Verification_am step4 : Record version
________________________________________________________________________________


-----------------------   Get AM software version   -----------------------
Getting product version from https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version
[LoginSession] Get "token_id" token for user "amadmin"
[LoginSession] Obtaining new "token_id" token from server because it has expired
Authenticate user amadmin via REST
[http_cmd]: curl --header "X-OpenAM-Username: amadmin" --header "X-OpenAM-Password: jiJ0ZhJJjEp6J90eFtnzIOXZ" --header "Content-Type: application/json" --header "Accept-API-Version: resource=2.0, protocol=1.0" -L --request POST  "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/authenticate?realm=/"
[http_cmd]: http status code OK
---  status code  ---
http status code is 200 (expected 200)
--- http response ---
{
    "tokenId": "qQJxkYdR-J3L_hkiIWN04PuJooc.*AAJTSQACMDIAAlNLABxqNXZUVFFYV1FpZ1ZRbkVzTDF3eSt2WWsrS0E9AAR0eXBlAANDVFMAAlMxAAIwMQ..*",
    "successUrl": "/am/console",
    "realm": "/"
}

[LoginSession] Obtaining session info
[http_cmd]: curl --header "iPlanetDirectoryPro: qQJxkYdR-J3L_hkiIWN04PuJooc.*AAJTSQACMDIAAlNLABxqNXZUVFFYV1FpZ1ZRbkVzTDF3eSt2WWsrS0E9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" --header "Content-Type: application/json" --header "Accept-API-Version: resource=3.0, protocol=2.1" --header "filters_cookie: qQJxkYdR-J3L_hkiIWN04PuJooc.*AAJTSQACMDIAAlNLABxqNXZUVFFYV1FpZ1ZRbkVzTDF3eSt2WWsrS0E9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" -L --request POST  "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/realms/root/sessions?_action=getSessionInfo"
[http_cmd]: http status code OK
---  status code  ---
http status code is 200 (expected 200)
--- http response ---
{
    "username": "amadmin",
    "universalId": "id=amadmin,ou=user,ou=am-config",
    "realm": "/",
    "latestAccessTime": "2023-08-16T06:09:17Z",
    "maxIdleExpirationTime": "2023-08-16T06:39:17Z",
    "maxSessionExpirationTime": "2023-08-16T08:09:16Z",
    "properties": {
        "AMCtxId": "b52d83a6-2a86-4db1-874c-ba0c3bcee018-7548"
    }
}

[http_cmd]: curl -L --request GET --cookie "amlbcookie=01" --cookie "iPlanetDirectoryPro=qQJxkYdR-J3L_hkiIWN04PuJooc.*AAJTSQACMDIAAlNLABxqNXZUVFFYV1FpZ1ZRbkVzTDF3eSt2WWsrS0E9AAR0eXBlAANDVFMAAlMxAAIwMQ..*" --cookie "route=1692166158.15.21013.797963|f60edb382037eb2df1e800d563ad78a7" "https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/serverinfo/version"
[http_cmd]: http status code OK
---  status code  ---
http status code is 200 (expected 200)
--- http response ---
{
    "_id": "version",
    "_rev": "575777018",
    "version": "7.4.0-SNAPSHOT",
    "fullVersion": "ForgeRock Access Management 7.4.0-SNAPSHOT Build 66777f275067f01bf7084f6278d26174d122e555 (2023-August-14 10:22)",
    "revision": "66777f275067f01bf7084f6278d26174d122e555",
    "date": "2023-August-14 10:22"
}

Version is 7.4.0-SNAPSHOT
Revision is 66777f275067f01bf7084f6278d26174d122e555
________________________________________________________________________________

[16/Aug/2023 06:09:17] Verification_am step5 :  Checking pod restart counts 
________________________________________________________________________________


-------------   Check pod am-869fdb5db9-5j69v restart count   -------------

[loop_until]: kubectl --namespace=xlou get pod am-869fdb5db9-5j69v --output jsonpath={.status.containerStatuses[*].restartCount}
[loop_until]: (max_time=180, interval=5, expected_rc=[0]
[loop_until]: OK (rc = 0)
--- stdout ---
0
--- stderr ---

Pod am-869fdb5db9-5j69v has been restarted 0 times.
SUCCESS : pod restart count (0) did not change between validation and verification phases

-------------   Check pod am-869fdb5db9-8dg94 restart count   -------------

[loop_until]: kubectl --namespace=xlou get pod am-869fdb5db9-8dg94 --output jsonpath={.status.containerStatuses[*].restartCount}
[loop_until]: (max_time=180, interval=5, expected_rc=[0]
[loop_until]: OK (rc = 0)
--- stdout ---
0
--- stderr ---

Pod am-869fdb5db9-8dg94 has been restarted 0 times.
SUCCESS : pod restart count (0) did not change between validation and verification phases

-------------   Check pod am-869fdb5db9-wt7sg restart count   -------------

[loop_until]: kubectl --namespace=xlou get pod am-869fdb5db9-wt7sg --output jsonpath={.status.containerStatuses[*].restartCount}
[loop_until]: (max_time=180, interval=5, expected_rc=[0]
[loop_until]: OK (rc = 0)
--- stdout ---
0
--- stderr ---

Pod am-869fdb5db9-wt7sg has been restarted 0 times.
SUCCESS : pod restart count (0) did not change between validation and verification phases
________________________________________________________________________________

[16/Aug/2023 06:09:18] Verification_am post : Post method
________________________________________________________________________________

Setting result to PASS
Task has been successfully stopped