==================================================================================================== ========================================= Pod describe ========================================= ==================================================================================================== Name: ds-cts-1 Namespace: xlou Priority: 0 Service Account: default Node: gke-xlou-cdm-ds-0f601751-j488/10.142.0.39 Start Time: Sat, 13 Apr 2024 22:39:01 +0000 Labels: affinity=directory app=ds-cts app.kubernetes.io/component=directory app.kubernetes.io/instance=ds-cts app.kubernetes.io/name=ds app.kubernetes.io/part-of=forgerock controller-revision-hash=ds-cts-bf5d4667 statefulset.kubernetes.io/pod-name=ds-cts-1 tier=ds Annotations: Status: Running IP: 10.106.45.2 IPs: IP: 10.106.45.2 Controlled By: StatefulSet/ds-cts Init Containers: initialize: Container ID: containerd://f86fb257f65e8775208185725eeb60297ad61fa4e63d3306fede693283ad9493 Image: gcr.io/engineeringpit/lodestar-images/ds-cts:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903 Image ID: gcr.io/engineeringpit/lodestar-images/ds-cts@sha256:d4625dfdd91929be72a5064d8bf9b53167e17f81160a4fb99e14abf4d21541a7 Port: Host Port: Command: /opt/opendj/scripts/init-and-restore.sh State: Terminated Reason: Completed Exit Code: 0 Started: Sat, 13 Apr 2024 22:39:15 +0000 Finished: Sat, 13 Apr 2024 22:39:25 +0000 Ready: True Restart Count: 0 Limits: memory: 14Gi Requests: cpu: 8 memory: 11Gi Environment Variables from: cloud-storage-credentials-cts Secret Optional: false Environment: DS_SET_UID_ADMIN_AND_MONITOR_PASSWORDS: true DS_UID_MONITOR_PASSWORD_FILE: /var/run/secrets/opendj-passwords/monitor.pw DS_UID_ADMIN_PASSWORD_FILE: /var/run/secrets/opendj-passwords/dirmanager.pw DSBACKUP_HOSTS: Optional: false DSBACKUP_DIRECTORY: Optional: false AUTORESTORE_FROM_DSBACKUP: Optional: false DISASTER_RECOVERY_ID: Optional: false Mounts: /opt/opendj/data from data (rw) /opt/opendj/pem-keys-directory/master-key from secrets (rw,path="master-key-pair-combined.pem") /opt/opendj/pem-keys-directory/ssl-key-pair from secrets (rw,path="ssl-key-pair-combined.pem") /opt/opendj/pem-trust-directory/cacerts.pem from truststore (rw,path="cacerts") /opt/opendj/secrets from secrets (rw) /var/run/secrets/cloud-credentials-cache/ from cloud-credentials-cache (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-6qwg9 (ro) /var/run/secrets/opendj from secrets (rw) /var/run/secrets/opendj-passwords from passwords (rw) Containers: ds: Container ID: containerd://491cc6e495b25c3054866a5f0b8d7fc1e16ca9d5f5fb4d4f06b5b20a1e3daf4c Image: gcr.io/engineeringpit/lodestar-images/ds-cts:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903 Image ID: gcr.io/engineeringpit/lodestar-images/ds-cts@sha256:d4625dfdd91929be72a5064d8bf9b53167e17f81160a4fb99e14abf4d21541a7 Ports: 1389/TCP, 1636/TCP, 4444/TCP, 8989/TCP, 8080/TCP, 8443/TCP Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP Args: start-ds State: Running Started: Sat, 13 Apr 2024 22:39:26 +0000 Ready: True Restart Count: 0 Limits: memory: 14Gi Requests: cpu: 8 memory: 11Gi Environment Variables from: cloud-storage-credentials-cts Secret Optional: false Environment: NAMESPACE: xlou (v1:metadata.namespace) DSBACKUP_HOSTS: Optional: false DSBACKUP_DIRECTORY: Optional: false Mounts: /opt/opendj/data from data (rw) /opt/opendj/pem-keys-directory/master-key from secrets (rw,path="master-key-pair-combined.pem") /opt/opendj/pem-keys-directory/ssl-key-pair from secrets (rw,path="ssl-key-pair-combined.pem") /opt/opendj/pem-trust-directory/cacerts.pem from truststore (rw,path="cacerts") /opt/opendj/secrets from secrets (rw) /var/run/secrets/cloud-credentials-cache/ from cloud-credentials-cache (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-6qwg9 (ro) /var/run/secrets/opendj from secrets (rw) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: data: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: data-ds-cts-1 ReadOnly: false secrets: Type: Secret (a volume populated by a Secret) SecretName: ds Optional: false passwords: Type: Secret (a volume populated by a Secret) SecretName: ds-passwords Optional: false cloud-credentials-cache: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: truststore: Type: Secret (a volume populated by a Secret) SecretName: truststore-pem Optional: false kube-api-access-6qwg9: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: Burstable Node-Selectors: Tolerations: WorkerDedicatedDS op=Exists kubernetes.io/arch:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 4m5s default-scheduler Successfully assigned xlou/ds-cts-1 to gke-xlou-cdm-ds-0f601751-j488 Normal SuccessfulAttachVolume 4m1s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-27b0f7cd-a37a-42df-be87-4ff61925a605" Normal Pulling 3m54s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/ds-cts:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903" Normal Pulled 3m51s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/ds-cts:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903" in 2.699857937s (2.699865773s including waiting) Normal Created 3m51s kubelet Created container initialize Normal Started 3m51s kubelet Started container initialize Normal Pulling 3m40s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/ds-cts:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903" Normal Pulled 3m40s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/ds-cts:xlou-7.6.0-d49ad6a5879fab3530729d8773297783d042f903" in 220.748493ms (220.753321ms including waiting) Normal Created 3m40s kubelet Created container ds Normal Started 3m40s kubelet Started container ds ==================================================================================================== =========================================== Pod logs =========================================== ==================================================================================================== Initializing "data/db" from Docker image Initializing "data/changelogDb" from Docker image Initializing "data/import-tmp" from Docker image Initializing "data/locks" from Docker image Initializing "data/var" from Docker image Upgrading configuration and data... * OpenDJ data has already been upgraded to version 7.6.0.d49ad6a5879fab3530729d8773297783d042f903 Rebuilding degraded indexes for base DN "ou=tokens"... Updating the "uid=admin" password Updating the "uid=monitor" password Initialization completed AUTORESTORE_FROM_DSBACKUP is missing or not set to true. Skipping restore Initializing "data/changelogDb" from Docker image Initializing "data/import-tmp" from Docker image Upgrading configuration and data... * OpenDJ data has already been upgraded to version 7.6.0.d49ad6a5879fab3530729d8773297783d042f903 Rebuilding degraded indexes for base DN "ou=tokens"... Server configured with: Group ID : default Server ID : ds-cts-1 Advertised listen address : ds-cts-1.ds-cts.xlou.svc.cluster.local Bootstrap replication server(s) : ds-cts-0.ds-cts.xlou.svc.cluster.local:8989,ds-cts-1.ds-cts.xlou.svc.cluster.local:8989 {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:33.898Z","category":"JVM","severity":"NOTICE","msgId":1058,"message":"ForgeRock Directory Services 7.6.0-20240405153841-d49ad6a5879fab3530729d8773297783d042f903 (build 20240405155149, revision number d49ad6a5879fab3530729d8773297783d042f903) starting up"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:33.898Z","category":"JVM","severity":"NOTICE","msgId":2233,"message":"Installation Directory: /opt/opendj"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:33.898Z","category":"JVM","severity":"NOTICE","msgId":2235,"message":"Instance Directory: /opt/opendj"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:33.898Z","category":"JVM","severity":"NOTICE","msgId":2229,"message":"JVM Information: 21.0.2+13-LTS by Azul Systems, Inc., 64-bit architecture, 11274289152 bytes heap size"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:33.898Z","category":"JVM","severity":"NOTICE","msgId":2230,"message":"JVM Host: ds-cts-1.ds-cts.xlou.svc.cluster.local default/ds-cts-1, running Linux 5.15.133+ amd64, 15032385536 bytes physical memory size, number of processors available 30"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:33.898Z","category":"JVM","severity":"NOTICE","msgId":2231,"message":"JVM Arguments: \"-XX:+ExitOnOutOfMemoryError\", \"-XX:MaxRAMPercentage=75\", \"-XX:MaxTenuringThreshold=1\", \"-Djava.security.egd=file:/dev/urandom\", \"-Dorg.opends.server.scriptName=start-ds\""} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:34.098Z","category":"CORE","severity":"WARNING","msgId":1025,"message":"'External Access Log Publisher' is DEPRECATED for removal since 7.2.0. Its usage is highly discouraged"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:34.507Z","category":"BACKEND","severity":"NOTICE","msgId":608,"message":"The database backend amCts containing 5 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:34.908Z","category":"SYNC.PURGE","severity":"INFORMATION","msgId":2210,"message":"The purge delay has been set to: 12 h"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:34.908Z","category":"SYNC.LIFECYCLE","severity":"NOTICE","msgId":2022,"message":"Replication server RS(ds-cts-1) started listening for new connections on address 0.0.0.0 port 8989"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:34.908Z","category":"SYNC.STATE","severity":"NOTICE","msgId":2204,"message":"Directory server DS(ds-cts-1) did not find a generation ID for domain 'uid=monitor'. A new generation ID will be computed by exporting the first 1 entries in the domain"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.108Z","category":"SYNC.CONNECTIONS","severity":"INFORMATION","msgId":2023,"message":"Replication server RS(ds-cts-1) has connected to replication server RS(ds-cts-0) for domain \"uid=monitor\" at ds-cts-0.ds-cts.xlou.svc.cluster.local/10.106.41.137:8989"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.108Z","category":"SYNC.CONNECTIONS","severity":"INFORMATION","msgId":2025,"message":"Replication server RS(ds-cts-1) has accepted a connection from directory server DS(ds-cts-1) for domain \"uid=monitor\" at /10.106.45.2:46946"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.108Z","category":"SYNC.CONNECTIONS","severity":"NOTICE","msgId":2026,"message":"Directory server DS(ds-cts-1) has connected to replication server RS(ds-cts-1) for domain \"uid=monitor\" at 10.106.45.2:8989 with generation ID 12671"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.108Z","category":"SYNC.STATE","severity":"NOTICE","msgId":2204,"message":"Directory server DS(ds-cts-1) did not find a generation ID for domain 'ou=tokens'. A new generation ID will be computed by exporting the first 5 entries in the domain"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.108Z","category":"BACKEND","severity":"NOTICE","msgId":610,"message":"Exported 5 entries and skipped 0 in 0 seconds (average rate 454.5/sec)"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.108Z","category":"SYNC.CONNECTIONS","severity":"INFORMATION","msgId":2023,"message":"Replication server RS(ds-cts-1) has connected to replication server RS(ds-cts-0) for domain \"ou=tokens\" at ds-cts-0.ds-cts.xlou.svc.cluster.local/10.106.41.137:8989"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.108Z","category":"SYNC.CONNECTIONS","severity":"INFORMATION","msgId":2025,"message":"Replication server RS(ds-cts-1) has accepted a connection from directory server DS(ds-cts-1) for domain \"ou=tokens\" at /10.106.45.2:46962"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.108Z","category":"SYNC.CONNECTIONS","severity":"NOTICE","msgId":2026,"message":"Directory server DS(ds-cts-1) has connected to replication server RS(ds-cts-1) for domain \"ou=tokens\" at 10.106.45.2:8989 with generation ID 69125"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.108Z","category":"CORE","severity":"NOTICE","msgId":1409,"message":"Registered 0 static groups, 0 dynamic groups and 0 virtual static groups. The static group cache is using 0 bytes of memory"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.308Z","category":"PROTOCOL.LDAP","severity":"NOTICE","msgId":1845,"message":"Started listening for new connections on Administration Connector 0.0.0.0:4444"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.308Z","category":"PROTOCOL.LDAP","severity":"NOTICE","msgId":1845,"message":"Started listening for new connections on LDAP 0.0.0.0:1389"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.308Z","category":"PROTOCOL.HTTP","severity":"NOTICE","msgId":1845,"message":"Started listening for new connections on HTTP 0.0.0.0:8080"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.308Z","category":"PROTOCOL.HTTP","severity":"NOTICE","msgId":1845,"message":"Started listening for new connections on HTTPS 0.0.0.0:8443"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.308Z","category":"PROTOCOL.LDAP","severity":"NOTICE","msgId":1845,"message":"Started listening for new connections on LDAPS 0.0.0.0:1636"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.308Z","category":"CORE","severity":"NOTICE","msgId":1059,"message":"The Directory Server has started successfully"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:35.308Z","category":"CORE","severity":"NOTICE","msgId":1061,"message":"The Directory Server has sent an alert notification generated by class org.opends.server.core.DirectoryServer (alert type org.opends.server.DirectoryServerStarted, alert ID org.opends.messages.server-1059): The Directory Server has started successfully"} {"_id":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-57","timestamp":"2024-04-13T22:39:40.257Z","eventName":"DJ-HTTP","transactionId":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-52","client":{"ip":"10.106.41.131","port":60910},"server":{"ip":"10.106.45.2","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-1.ds-cts.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.106.45.2:8080"],"user-agent":["Prometheus/2.44.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":90,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:59.529Z","category":"SYNC.CONNECTIONS","severity":"INFORMATION","msgId":2024,"message":"Replication server RS(ds-cts-1) has accepted a connection from replication server RS(ds-cts-2) for domain \"uid=monitor\" at /10.106.43.12:60962"} {"eventName":"DJ-SERVER","timestamp":"2024-04-13T22:39:59.729Z","category":"SYNC.CONNECTIONS","severity":"INFORMATION","msgId":2024,"message":"Replication server RS(ds-cts-1) has accepted a connection from replication server RS(ds-cts-2) for domain \"ou=tokens\" at /10.106.43.12:60976"} {"_id":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-64","timestamp":"2024-04-13T22:40:10.226Z","eventName":"DJ-HTTP","transactionId":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-59","client":{"ip":"10.106.41.131","port":60910},"server":{"ip":"10.106.45.2","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-1.ds-cts.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.106.45.2:8080"],"user-agent":["Prometheus/2.44.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":36,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-71","timestamp":"2024-04-13T22:40:40.226Z","eventName":"DJ-HTTP","transactionId":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-66","client":{"ip":"10.106.41.131","port":60910},"server":{"ip":"10.106.45.2","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-1.ds-cts.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.106.45.2:8080"],"user-agent":["Prometheus/2.44.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":23,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-91","timestamp":"2024-04-13T22:41:10.226Z","eventName":"DJ-HTTP","transactionId":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-86","client":{"ip":"10.106.41.131","port":60910},"server":{"ip":"10.106.45.2","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-1.ds-cts.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.106.45.2:8080"],"user-agent":["Prometheus/2.44.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":19,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-146","timestamp":"2024-04-13T22:41:40.226Z","eventName":"DJ-HTTP","transactionId":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-141","client":{"ip":"10.106.41.131","port":60910},"server":{"ip":"10.106.45.2","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-1.ds-cts.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.106.45.2:8080"],"user-agent":["Prometheus/2.44.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":33,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-187","timestamp":"2024-04-13T22:42:10.226Z","eventName":"DJ-HTTP","transactionId":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-182","client":{"ip":"10.106.41.131","port":60910},"server":{"ip":"10.106.45.2","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-1.ds-cts.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.106.45.2:8080"],"user-agent":["Prometheus/2.44.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":22,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-234","timestamp":"2024-04-13T22:42:40.226Z","eventName":"DJ-HTTP","transactionId":"6c4a6e59-e07d-4da8-b755-5a5aef370d07-229","client":{"ip":"10.106.41.131","port":60910},"server":{"ip":"10.106.45.2","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-1.ds-cts.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.106.45.2:8080"],"user-agent":["Prometheus/2.44.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":27,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"}