--Task--
              name: Deploy_or_discover_overseer
           enabled: True
        class_name: DeployOrDiscoverOverseerTask
       source_name: controller
  source_namespace: >default<
       target_name: controller
  target_namespace: >default<
             start: 0.0
              stop: None
           timeout: no timeout
              loop: False
          interval: None
      dependencies: ['Discover_Tenant']
          wait_for: []
    preceding_task: None
           options: {}
        group_name: None
       Current dir: /mnt/disk1/xslou/workshop/lodestar-fork/pyrock

________________________________________________________________________________

[2025-01-17 04:21:27] Deploy_or_discover_overseer pre : Initialising task parameters
________________________________________________________________________________

task will be executed on controller (localhost)
________________________________________________________________________________

[2025-01-17 04:21:27] Deploy_or_discover_overseer step1 : Discover or deploy overseer
________________________________________________________________________________

2025-01-17 04:21:27,783 INFO    
2025-01-17 04:21:27,783 INFO    [loop_until]: kubectl --namespace=xlou get deployments -l "release=overseer" -o name
2025-01-17 04:21:27,783 INFO    [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2025-01-17 04:21:28,089 INFO    [loop_until]: OK (rc = 0)
2025-01-17 04:21:28,089 DEBUG   --- stdout ---
2025-01-17 04:21:28,089 DEBUG   deployment.apps/overseer-0
2025-01-17 04:21:28,089 DEBUG   --- stderr ---
2025-01-17 04:21:28,089 DEBUG   
Discover overseer pod: overseer-0
2025-01-17 04:21:28,251 INFO    
2025-01-17 04:21:28,251 INFO    ---------------------   Get expected number of pods   ---------------------
2025-01-17 04:21:28,252 INFO    
2025-01-17 04:21:28,252 INFO    [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=overseer-0 --output jsonpath={.items[*].spec.replicas}
2025-01-17 04:21:28,252 INFO    [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2025-01-17 04:21:28,527 INFO    [loop_until]: OK (rc = 0)
2025-01-17 04:21:28,527 DEBUG   --- stdout ---
2025-01-17 04:21:28,527 DEBUG   1
2025-01-17 04:21:28,527 DEBUG   --- stderr ---
2025-01-17 04:21:28,527 DEBUG   
2025-01-17 04:21:28,527 INFO    
2025-01-17 04:21:28,527 INFO    -------------   Reloading pod list for product "overseer-0"   -------------
2025-01-17 04:21:28,528 INFO    
2025-01-17 04:21:28,528 INFO    [loop_until]: awk -F" " "{print NF}" <<< `kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods --selector app=overseer-0 --output jsonpath={.items[*].metadata.name}` | grep 1
2025-01-17 04:21:28,528 INFO    [loop_until]: (max_time=360.0, interval=10, expected_rc=[0]
2025-01-17 04:21:28,831 INFO    [loop_until]: Function succeeded after 0s (rc=0) - expected number of elements found
2025-01-17 04:21:28,831 INFO    [loop_until]: OK (rc = 0)
2025-01-17 04:21:28,831 DEBUG   --- stdout ---
2025-01-17 04:21:28,831 DEBUG   overseer-0-5865b977f-pk2tz
2025-01-17 04:21:28,831 DEBUG   --- stderr ---
2025-01-17 04:21:28,831 DEBUG   
2025-01-17 04:21:28,831 INFO    
2025-01-17 04:21:28,831 INFO    -----------   Check pod overseer-0-5865b977f-pk2tz is running   -----------
2025-01-17 04:21:28,832 INFO    
2025-01-17 04:21:28,832 INFO    [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods overseer-0-5865b977f-pk2tz -o=jsonpath={.status.phase} | grep "Running"
2025-01-17 04:21:28,832 INFO    [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2025-01-17 04:21:29,131 INFO    [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2025-01-17 04:21:29,131 INFO    [loop_until]: OK (rc = 0)
2025-01-17 04:21:29,131 DEBUG   --- stdout ---
2025-01-17 04:21:29,131 DEBUG   Running
2025-01-17 04:21:29,131 DEBUG   --- stderr ---
2025-01-17 04:21:29,131 DEBUG   
2025-01-17 04:21:29,132 INFO    
2025-01-17 04:21:29,132 INFO    [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pods overseer-0-5865b977f-pk2tz -o=jsonpath={.status.containerStatuses[*].ready} | grep "true"
2025-01-17 04:21:29,132 INFO    [loop_until]: (max_time=360, interval=5, expected_rc=[0]
2025-01-17 04:21:29,407 INFO    [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2025-01-17 04:21:29,408 INFO    [loop_until]: OK (rc = 0)
2025-01-17 04:21:29,408 DEBUG   --- stdout ---
2025-01-17 04:21:29,408 DEBUG   true
2025-01-17 04:21:29,408 DEBUG   --- stderr ---
2025-01-17 04:21:29,408 DEBUG   
2025-01-17 04:21:29,408 INFO    
2025-01-17 04:21:29,408 INFO    [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod overseer-0-5865b977f-pk2tz --output jsonpath={.status.startTime}
2025-01-17 04:21:29,408 INFO    [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2025-01-17 04:21:29,684 INFO    [loop_until]: OK (rc = 0)
2025-01-17 04:21:29,684 DEBUG   --- stdout ---
2025-01-17 04:21:29,684 DEBUG   2025-01-08T13:06:01Z
2025-01-17 04:21:29,684 DEBUG   --- stderr ---
2025-01-17 04:21:29,684 DEBUG   
2025-01-17 04:21:29,685 INFO    
2025-01-17 04:21:29,685 INFO    ----   Check pod overseer-0-5865b977f-pk2tz filesystem is accessible   ----
2025-01-17 04:21:30,450 INFO    [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2025-01-17 04:21:30,450 INFO    
2025-01-17 04:21:30,450 INFO    ---------    Check pod overseer-0-5865b977f-pk2tz restart count   ---------
2025-01-17 04:21:30,450 INFO    
2025-01-17 04:21:30,450 INFO    [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get pod overseer-0-5865b977f-pk2tz --output jsonpath={.status.containerStatuses[*].restartCount}
2025-01-17 04:21:30,450 INFO    [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2025-01-17 04:21:30,727 INFO    [loop_until]: OK (rc = 0)
2025-01-17 04:21:30,727 DEBUG   --- stdout ---
2025-01-17 04:21:30,727 DEBUG   0
2025-01-17 04:21:30,727 DEBUG   --- stderr ---
2025-01-17 04:21:30,727 DEBUG   
2025-01-17 04:21:30,727 INFO    Pod overseer-0-5865b977f-pk2tz has been restarted 0 times.
2025-01-17 04:21:30,727 INFO    
2025-01-17 04:21:30,727 INFO    ---------------------   Get expected number of pods   ---------------------
2025-01-17 04:21:30,727 INFO    
2025-01-17 04:21:30,727 INFO    [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments --selector app=overseer-0 --output jsonpath={.items[*].spec.replicas}
2025-01-17 04:21:30,727 INFO    [loop_until]: (max_time=180, interval=5, expected_rc=[0]
2025-01-17 04:21:31,007 INFO    [loop_until]: OK (rc = 0)
2025-01-17 04:21:31,007 DEBUG   --- stdout ---
2025-01-17 04:21:31,007 DEBUG   1
2025-01-17 04:21:31,007 DEBUG   --- stderr ---
2025-01-17 04:21:31,007 DEBUG   
2025-01-17 04:21:31,007 INFO    
2025-01-17 04:21:31,007 INFO    --------------   Waiting for 1 expected pod(s) to be ready   --------------
2025-01-17 04:21:31,008 INFO    
2025-01-17 04:21:31,008 INFO    [loop_until]: kubectl --namespace=xlou --context=gke_engineeringpit_us-east1-d_xlou-cdm get deployments overseer-0 --output jsonpath="ready:{.status.readyReplicas} replicas:{.status.replicas}" | grep "ready:1 replicas:1"
2025-01-17 04:21:31,008 INFO    [loop_until]: (max_time=900, interval=30, expected_rc=[0]
2025-01-17 04:21:31,346 INFO    [loop_until]: Function succeeded after 0s (rc=0) - expected pattern found
2025-01-17 04:21:31,346 INFO    [loop_until]: OK (rc = 0)
2025-01-17 04:21:31,346 DEBUG   --- stdout ---
2025-01-17 04:21:31,346 DEBUG   ready:1 replicas:1
2025-01-17 04:21:31,346 DEBUG   --- stderr ---
2025-01-17 04:21:31,346 DEBUG   
2025-01-17 04:21:31,346 INFO    Livecheck to https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com
2025-01-17 04:21:31,346 INFO    [http_cmd]: curl --insecure -L --request GET  "https://overseer-0.xlou.iam.xlou-cdm.engineeringpit.com"
2025-01-17 04:21:31,687 INFO    [http_cmd]: http status code OK
2025-01-17 04:21:31,688 DEBUG   ---  status code  ---
2025-01-17 04:21:31,688 DEBUG   http status code is 200 (expected 200)
2025-01-17 04:21:31,688 DEBUG   --- http response ---
2025-01-17 04:21:31,688 DEBUG   [print_head_tail]: Print head (10 lines) and tail (10 lines) of input string (21 lines):
2025-01-17 04:21:31,688 DEBUG   ----- output -----
2025-01-17 04:21:31,688 DEBUG   <!DOCTYPE HTML>
2025-01-17 04:21:31,688 DEBUG   <html lang="en">
2025-01-17 04:21:31,688 DEBUG   <head>
2025-01-17 04:21:31,688 DEBUG   <meta charset="utf-8">
2025-01-17 04:21:31,688 DEBUG   <title>Directory listing for /</title>
2025-01-17 04:21:31,688 DEBUG   </head>
2025-01-17 04:21:31,688 DEBUG   <body>
2025-01-17 04:21:31,688 DEBUG   <h1>Directory listing for /</h1>
2025-01-17 04:21:31,688 DEBUG   <hr>
2025-01-17 04:21:31,688 DEBUG   <ul>
2025-01-17 04:21:31,688 DEBUG   
2025-01-17 04:21:31,688 DEBUG   
2025-01-17 04:21:31,688 DEBUG   
2025-01-17 04:21:31,688 DEBUG   [...]
2025-01-17 04:21:31,688 DEBUG   
2025-01-17 04:21:31,688 DEBUG   
2025-01-17 04:21:31,688 DEBUG   
2025-01-17 04:21:31,688 DEBUG   <li><a href="gatling/">gatling/</a></li>
2025-01-17 04:21:31,688 DEBUG   <li><a href="idm/">idm/</a></li>
2025-01-17 04:21:31,688 DEBUG   <li><a href="lost%2Bfound/">lost+found/</a></li>
2025-01-17 04:21:31,688 DEBUG   <li><a href="orders/">orders/</a></li>
2025-01-17 04:21:31,688 DEBUG   <li><a href="users.ldif.gz">users.ldif.gz</a></li>
2025-01-17 04:21:31,688 DEBUG   </ul>
2025-01-17 04:21:31,688 DEBUG   <hr>
2025-01-17 04:21:31,689 DEBUG   </body>
2025-01-17 04:21:31,689 DEBUG   </html>
2025-01-17 04:21:31,689 DEBUG   
________________________________________________________________________________

[2025-01-17 04:21:31] Deploy_or_discover_overseer post : Post method
________________________________________________________________________________

Setting result to PASS
Task has been successfully stopped