==================================================================================================== ========================================= Pod describe ========================================= ==================================================================================================== Name: overseer-0-56c5c978d9-v8wm9 Namespace: xlou Priority: 0 Service Account: k8s-svc-acct-0 Node: gke-xlou-cdm-frontend-a8771548-i82n/10.142.0.26 Start Time: Thu, 25 Apr 2024 17:24:20 +0000 Labels: app=overseer-0 pod-template-hash=56c5c978d9 release=overseer skaffold.dev/run-id=2752c09c-d131-40f7-96b8-8533bd8f9752 Annotations: Status: Running IP: 10.106.41.87 IPs: IP: 10.106.41.87 Controlled By: ReplicaSet/overseer-0-56c5c978d9 Containers: overseer: Container ID: containerd://7758a86d9623c6829b23ce03533b2584a0455af8ea54f0d05a02ffb7e5b4b08c Image: gcr.io/engineeringpit/lodestar-images/lodestarbox:master-stable Image ID: gcr.io/engineeringpit/lodestar-images/lodestarbox@sha256:8f3ffb07d87daf5d0bb82f4e1d6e9126b62ec155fd32b51512c2619dd67dd55c Port: Host Port: Args: /bin/sh -c PYTHONUNBUFFERED=x /lodestar/pyrock/shared/scripts/overseer/run.py State: Running Started: Thu, 25 Apr 2024 17:24:31 +0000 Ready: True Restart Count: 0 Limits: cpu: 8 memory: 8Gi Requests: cpu: 2500m memory: 4Gi Environment Variables from: overseer-config-0 ConfigMap Optional: false Environment: Mounts: /results from results (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zv2pp (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: results: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: overseer-0 ReadOnly: false kube-api-access-zv2pp: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: Burstable Node-Selectors: forgerock.io/role=frontend Tolerations: WorkerDedicatedFrontend=true:NoSchedule node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 18m default-scheduler Successfully assigned xlou/overseer-0-56c5c978d9-v8wm9 to gke-xlou-cdm-frontend-a8771548-i82n Normal SuccessfulAttachVolume 17m attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-5a93c3f1-6dcd-42f5-a070-a1c205de0c2a" Normal Pulling 17m kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/lodestarbox:master-stable" Normal Pulled 17m kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/lodestarbox:master-stable" in 235.538213ms Normal Created 17m kubelet Created container overseer Normal Started 17m kubelet Started container overseer ==================================================================================================== =========================================== Pod logs =========================================== ==================================================================================================== [2024-04-25 18:24:31] httpd web_server : Server is running at http://localhost:8080 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:24:31] overseer : Waiting for order /results/orders/order.json ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 10.106.42.16 - - [25/Apr/2024 18:24:35] "GET / HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:24:38] "GET / HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:26:43] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:38:32] "GET /orders/status.txt HTTP/1.1" 200 - [2024-04-25 18:38:34] overseer : Start new order MakeFeeders_ds-idrepo (id 1) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:38:34] order MakeFeeders_ds-idrepo : Start command ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:38:34] order MakeFeeders_ds-idrepo : Running command : python3 /lodestar/pyrock/shared/scripts/ds/generate_feeders.py --ldif-file /results/ds-idrepo/user_entries.ldif --out-dir /results/ds-idrepo --zip-file /results/ds-idrepo/feeders.tar.gz [2024-04-25 18:38:34] order MakeFeeders_ds-idrepo : Http access : http://localhost:8080/orders/MakeFeeders_ds-idrepo [2024-04-25 18:38:34] order MakeFeeders_ds-idrepo : Full command stored in : /results/orders/MakeFeeders_ds-idrepo/cmd.txt [2024-04-25 18:38:34] order MakeFeeders_ds-idrepo : RC file : /results/orders/MakeFeeders_ds-idrepo/rc.txt [2024-04-25 18:38:34] order MakeFeeders_ds-idrepo : Stdout file : /results/orders/MakeFeeders_ds-idrepo/stdout.txt [2024-04-25 18:38:34] order MakeFeeders_ds-idrepo : Stderr file : /results/orders/MakeFeeders_ds-idrepo/stderr.txt [2024-04-25 18:38:34] order MakeFeeders_ds-idrepo : Process 367 has been successfully launched 10.106.42.16 - - [25/Apr/2024 18:38:39] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:38:44] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:38:50] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:38:55] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:39:00] "GET /orders/status.txt HTTP/1.1" 200 - [2024-04-25 18:39:04] order MakeFeeders_ds-idrepo : CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) [2024-04-25 18:39:04] overseer : order stopped by itself MakeFeeders_ds-idrepo : /results/orders/MakeFeeders_ds-idrepo ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:39:05] overseer : Waiting for order /results/orders/order.json ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:39:05] overseer : Http access : http://localhost:8080 10.106.42.16 - - [25/Apr/2024 18:39:05] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:39:24] "GET /orders/status.txt HTTP/1.1" 200 - [2024-04-25 18:39:26] overseer : Start new order DSPrimeTask_ds_idrepo_0 (id 2) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:39:26] order DSPrimeTask_ds_idrepo_0 : Start command ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:39:26] order DSPrimeTask_ds_idrepo_0 : Running command : /lodestar/clients/ds/bin/searchrate --maxDuration 1 --bindPassword **** --bindDn "uid=admin" --trustAll --useStartTls --port 1389 --statInterval 5 --noPropertiesFile --hostname 'ds-idrepo-0.ds-idrepo' --noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 1000000 --argument 'inc(0, 1000000)' '(uid=user.{})' [2024-04-25 18:39:26] order DSPrimeTask_ds_idrepo_0 : Http access : http://localhost:8080/orders/DSPrimeTask_ds_idrepo_0 [2024-04-25 18:39:26] order DSPrimeTask_ds_idrepo_0 : Full command stored in : /results/orders/DSPrimeTask_ds_idrepo_0/cmd.txt [2024-04-25 18:39:26] order DSPrimeTask_ds_idrepo_0 : RC file : /results/orders/DSPrimeTask_ds_idrepo_0/rc.txt [2024-04-25 18:39:26] order DSPrimeTask_ds_idrepo_0 : Stdout file : /results/orders/DSPrimeTask_ds_idrepo_0/stdout.txt [2024-04-25 18:39:26] order DSPrimeTask_ds_idrepo_0 : Stderr file : /results/orders/DSPrimeTask_ds_idrepo_0/stderr.txt [2024-04-25 18:39:26] order DSPrimeTask_ds_idrepo_0 : Process 466 has been successfully launched 10.106.42.16 - - [25/Apr/2024 18:39:30] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:39:36] "GET /orders/status.txt HTTP/1.1" 200 - [2024-04-25 18:39:37] order DSPrimeTask_ds_idrepo_0 : CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) [2024-04-25 18:39:37] overseer : order stopped by itself DSPrimeTask_ds_idrepo_0 : /results/orders/DSPrimeTask_ds_idrepo_0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:39:38] overseer : Waiting for order /results/orders/order.json ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:39:38] overseer : Http access : http://localhost:8080 10.106.42.16 - - [25/Apr/2024 18:39:41] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:39:58] "GET /orders/status.txt HTTP/1.1" 200 - [2024-04-25 18:40:00] overseer : Start new order DSPrimeTask_ds_idrepo_1 (id 3) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:40:00] order DSPrimeTask_ds_idrepo_1 : Start command ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:40:00] order DSPrimeTask_ds_idrepo_1 : Running command : /lodestar/clients/ds/bin/searchrate --maxDuration 1 --bindPassword **** --bindDn "uid=admin" --trustAll --useStartTls --port 1389 --statInterval 5 --noPropertiesFile --hostname 'ds-idrepo-1.ds-idrepo' --noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 1000000 --argument 'inc(0, 1000000)' '(uid=user.{})' [2024-04-25 18:40:00] order DSPrimeTask_ds_idrepo_1 : Http access : http://localhost:8080/orders/DSPrimeTask_ds_idrepo_1 [2024-04-25 18:40:00] order DSPrimeTask_ds_idrepo_1 : Full command stored in : /results/orders/DSPrimeTask_ds_idrepo_1/cmd.txt [2024-04-25 18:40:00] order DSPrimeTask_ds_idrepo_1 : RC file : /results/orders/DSPrimeTask_ds_idrepo_1/rc.txt [2024-04-25 18:40:00] order DSPrimeTask_ds_idrepo_1 : Stdout file : /results/orders/DSPrimeTask_ds_idrepo_1/stdout.txt [2024-04-25 18:40:00] order DSPrimeTask_ds_idrepo_1 : Stderr file : /results/orders/DSPrimeTask_ds_idrepo_1/stderr.txt [2024-04-25 18:40:00] order DSPrimeTask_ds_idrepo_1 : Process 619 has been successfully launched 10.106.42.16 - - [25/Apr/2024 18:40:05] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:40:11] "GET /orders/status.txt HTTP/1.1" 200 - [2024-04-25 18:40:12] order DSPrimeTask_ds_idrepo_1 : CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) [2024-04-25 18:40:12] overseer : order stopped by itself DSPrimeTask_ds_idrepo_1 : /results/orders/DSPrimeTask_ds_idrepo_1 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:40:12] overseer : Waiting for order /results/orders/order.json ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:40:12] overseer : Http access : http://localhost:8080 10.106.42.16 - - [25/Apr/2024 18:40:16] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:40:34] "GET /orders/status.txt HTTP/1.1" 200 - [2024-04-25 18:40:36] overseer : Start new order DSPrimeTask_ds_idrepo_2 (id 4) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:40:36] order DSPrimeTask_ds_idrepo_2 : Start command ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:40:36] order DSPrimeTask_ds_idrepo_2 : Running command : /lodestar/clients/ds/bin/searchrate --maxDuration 1 --bindPassword **** --bindDn "uid=admin" --trustAll --useStartTls --port 1389 --statInterval 5 --noPropertiesFile --hostname 'ds-idrepo-2.ds-idrepo' --noRebind --warmUpDuration 10 --numConnections 1 --numConcurrentRequests 10 --baseDn 'ou=people,ou=identities' --maxIterations 1000000 --argument 'inc(0, 1000000)' '(uid=user.{})' [2024-04-25 18:40:36] order DSPrimeTask_ds_idrepo_2 : Http access : http://localhost:8080/orders/DSPrimeTask_ds_idrepo_2 [2024-04-25 18:40:36] order DSPrimeTask_ds_idrepo_2 : Full command stored in : /results/orders/DSPrimeTask_ds_idrepo_2/cmd.txt [2024-04-25 18:40:36] order DSPrimeTask_ds_idrepo_2 : RC file : /results/orders/DSPrimeTask_ds_idrepo_2/rc.txt [2024-04-25 18:40:36] order DSPrimeTask_ds_idrepo_2 : Stdout file : /results/orders/DSPrimeTask_ds_idrepo_2/stdout.txt [2024-04-25 18:40:36] order DSPrimeTask_ds_idrepo_2 : Stderr file : /results/orders/DSPrimeTask_ds_idrepo_2/stderr.txt [2024-04-25 18:40:36] order DSPrimeTask_ds_idrepo_2 : Process 772 has been successfully launched 10.106.42.16 - - [25/Apr/2024 18:40:41] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:40:46] "GET /orders/status.txt HTTP/1.1" 200 - [2024-04-25 18:40:47] order DSPrimeTask_ds_idrepo_2 : CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) [2024-04-25 18:40:47] overseer : order stopped by itself DSPrimeTask_ds_idrepo_2 : /results/orders/DSPrimeTask_ds_idrepo_2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:40:48] overseer : Waiting for order /results/orders/order.json ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:40:48] overseer : Http access : http://localhost:8080 10.106.42.16 - - [25/Apr/2024 18:40:51] "GET /orders/status.txt HTTP/1.1" 200 - 10.106.42.16 - - [25/Apr/2024 18:41:11] "GET /orders/status.txt HTTP/1.1" 200 - [2024-04-25 18:41:13] overseer : Start new order authn-users-debug (id 5) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:41:13] order authn-users-debug : Start command ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:41:13] order authn-users-debug : Running command : export JAVA_OPTS="-Dam_host=xlou.iam.xlou-cdm.engineeringpit.com -DamadminPassword=**** -DamadminUsername=amadmin -Dduration=10 -Dnum_users=1000000 -DtestCase=am.AMAuthN -DtestMode=debug -Dthroughput=1 -Duser_password=**** --run-mode local --simulation com.forgerock.gatling.Loadtest --simulations-folder /lodestar/pyrock/shared/clients/gatling-open/ --run-description "am.AMAuthN@medium" --results-folder /results/gatling/authn-users-debug --extra-run-jvm-options "-Xmx12G" --no-reports [2024-04-25 18:41:13] order authn-users-debug : Http access : http://localhost:8080/orders/authn-users-debug [2024-04-25 18:41:13] order authn-users-debug : Full command stored in : /results/orders/authn-users-debug/cmd.txt [2024-04-25 18:41:13] order authn-users-debug : RC file : /results/orders/authn-users-debug/rc.txt [2024-04-25 18:41:13] order authn-users-debug : Stdout file : /results/orders/authn-users-debug/stdout.txt [2024-04-25 18:41:13] order authn-users-debug : Stderr file : /results/orders/authn-users-debug/stderr.txt [2024-04-25 18:41:13] order authn-users-debug : Process 924 has been successfully launched [2024-04-25 18:41:29] order authn-users-debug : CMD SUCCESS: command returns code 0 (expected [0, -9, 143, 137, -15]) [2024-04-25 18:41:29] overseer : order stopped by itself authn-users-debug : /results/orders/authn-users-debug ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:41:29] overseer : Waiting for order /results/orders/order.json ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [2024-04-25 18:41:29] overseer : Http access : http://localhost:8080 10.106.42.16 - - [25/Apr/2024 18:41:36] "GET /orders/status.txt HTTP/1.1" 200 -