==================================================================================================== ========================================= Pod describe ========================================= ==================================================================================================== Name: ds-cts-0 Namespace: xlou-rcs Priority: 0 Service Account: default Node: gke-xlou-cdm-frontend-18a44ad7-wxs9/10.142.0.11 Start Time: Tue, 17 Dec 2024 17:46:33 +0000 Labels: affinity=directory app=ds-cts app.kubernetes.io/component=directory app.kubernetes.io/instance=ds-cts app.kubernetes.io/name=ds app.kubernetes.io/part-of=forgerock apps.kubernetes.io/pod-index=0 controller-revision-hash=ds-cts-6c76767f9d statefulset.kubernetes.io/pod-name=ds-cts-0 Annotations: Status: Running IP: 10.106.40.168 IPs: IP: 10.106.40.168 Controlled By: StatefulSet/ds-cts Init Containers: init: Container ID: containerd://6f16ffbe9a4ace72cfc61c5bcbb372355819c9826b19950f870975b89dc0eb64 Image: gcr.io/engineeringpit/lodestar-images/ds:xlou-rcs-8.0.0-c9a59f45a8b654a548fd7065172bb45faedb125a Image ID: gcr.io/engineeringpit/lodestar-images/ds@sha256:6ad344293fd80829637f1c59751809b5433d07584d1932064fadc55b0a78e062 Port: Host Port: Command: /opt/opendj/runtime-scripts/init-and-restore.sh State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 17 Dec 2024 17:46:45 +0000 Finished: Tue, 17 Dec 2024 17:47:09 +0000 Ready: True Restart Count: 0 Limits: memory: 1366Mi Requests: cpu: 500m memory: 1366Mi Environment Variables from: cloud-storage-credentials Secret Optional: true Environment: POD_NAME: ds-cts-0 (v1:metadata.name) DS_SET_UID_ADMIN_AND_MONITOR_PASSWORDS: true DS_UID_MONITOR_PASSWORD_FILE: /var/run/secrets/monitor/monitor.pw DS_UID_ADMIN_PASSWORD_FILE: /var/run/secrets/admin/dirmanager.pw DSBACKUP_HOSTS: Optional: false DSBACKUP_DIRECTORY: Optional: false AUTORESTORE_FROM_DSBACKUP: Optional: false DISASTER_RECOVERY_ID: Optional: false Mounts: /opt/opendj/data from data (rw) /var/run/secrets/admin from admin-password (rw) /var/run/secrets/cloud-credentials-cache/ from cloud-credentials-cache (rw) /var/run/secrets/ds-master-keypair from ds-master-keypair (rw) /var/run/secrets/ds-ssl-keypair from ds-ssl-keypair (rw) /var/run/secrets/keys from keys (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-z42tz (ro) /var/run/secrets/monitor from monitor-password (rw) /var/run/secrets/truststore from truststore (rw) Containers: ds: Container ID: containerd://9aaa324dffa702212408e1d19baea6d58626216666970731bef4c51021af938b Image: gcr.io/engineeringpit/lodestar-images/ds:xlou-rcs-8.0.0-c9a59f45a8b654a548fd7065172bb45faedb125a Image ID: gcr.io/engineeringpit/lodestar-images/ds@sha256:6ad344293fd80829637f1c59751809b5433d07584d1932064fadc55b0a78e062 Port: 8080/TCP Host Port: 0/TCP Args: start-ds State: Running Started: Tue, 17 Dec 2024 17:47:10 +0000 Ready: True Restart Count: 0 Limits: cpu: 4 memory: 4Gi Requests: cpu: 4 memory: 4Gi Startup: http-get http://:http/healthy delay=30s timeout=1s period=10s #success=1 #failure=720 Environment Variables from: cloud-storage-credentials Secret Optional: true Environment: POD_NAME: ds-cts-0 (v1:metadata.name) DS_SET_UID_ADMIN_AND_MONITOR_PASSWORDS: true DS_UID_MONITOR_PASSWORD_FILE: /var/run/secrets/monitor/monitor.pw DS_UID_ADMIN_PASSWORD_FILE: /var/run/secrets/admin/dirmanager.pw Mounts: /opt/opendj/data from data (rw) /var/run/secrets/admin from admin-password (rw) /var/run/secrets/cloud-credentials-cache from cloud-credentials-cache (rw) /var/run/secrets/ds-master-keypair from ds-master-keypair (rw) /var/run/secrets/ds-ssl-keypair from ds-ssl-keypair (rw) /var/run/secrets/keys from keys (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-z42tz (ro) /var/run/secrets/monitor from monitor-password (rw) /var/run/secrets/truststore from truststore (rw) Conditions: Type Status PodReadyToStartContainers True Initialized True Ready True ContainersReady True PodScheduled True Volumes: data: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: data-ds-cts-0 ReadOnly: false ds-master-keypair: Type: Secret (a volume populated by a Secret) SecretName: ds-master-keypair Optional: false ds-ssl-keypair: Type: Secret (a volume populated by a Secret) SecretName: ds-ssl-keypair Optional: false truststore: Type: Secret (a volume populated by a Secret) SecretName: ds-ssl-keypair Optional: false admin-password: Type: Secret (a volume populated by a Secret) SecretName: ds-passwords Optional: false monitor-password: Type: Secret (a volume populated by a Secret) SecretName: ds-passwords Optional: false keys: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: cloud-credentials-cache: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: kube-api-access-z42tz: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: Burstable Node-Selectors: Tolerations: WorkerDedicatedDS op=Exists kubernetes.io/arch:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Topology Spread Constraints: topology.kubernetes.io/hostname:ScheduleAnyway when max skew 1 is exceeded for selector affinity=directory,app=ds-cts,app.kubernetes.io/component=directory,app.kubernetes.io/instance=ds-cts,app.kubernetes.io/name=ds,app.kubernetes.io/part-of=forgerock topology.kubernetes.io/zone:ScheduleAnyway when max skew 1 is exceeded for selector affinity=directory,app=ds-cts,app.kubernetes.io/component=directory,app.kubernetes.io/instance=ds-cts,app.kubernetes.io/name=ds,app.kubernetes.io/part-of=forgerock Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 2m4s default-scheduler Successfully assigned xlou-rcs/ds-cts-0 to gke-xlou-cdm-frontend-18a44ad7-wxs9 Normal SuccessfulAttachVolume 117s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-d2b00c4d-f4d4-4d43-9599-b1d62d6c9631" Normal Pulling 116s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/ds:xlou-rcs-8.0.0-c9a59f45a8b654a548fd7065172bb45faedb125a" Normal Pulled 112s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/ds:xlou-rcs-8.0.0-c9a59f45a8b654a548fd7065172bb45faedb125a" in 3.869s (3.869s including waiting). Image size: 195177097 bytes. Normal Created 112s kubelet Created container init Normal Started 112s kubelet Started container init Normal Pulling 87s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/ds:xlou-rcs-8.0.0-c9a59f45a8b654a548fd7065172bb45faedb125a" Normal Pulled 87s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/ds:xlou-rcs-8.0.0-c9a59f45a8b654a548fd7065172bb45faedb125a" in 258ms (258ms including waiting). Image size: 195177097 bytes. Normal Created 87s kubelet Created container ds Normal Started 87s kubelet Started container ds ==================================================================================================== =========================================== Pod logs =========================================== ==================================================================================================== Server configured with: Group ID : default Server ID : ds-cts-0-default Advertised listen address : ds-cts-0.ds-cts.xlou-rcs.svc.cluster.local Bootstrap replication server(s) : ds-cts-0.ds-cts.xlou-rcs.svc.cluster.local:8989,ds-cts-1.ds-cts.xlou-rcs.svc.cluster.local:8989 Untaring protoype setup to /opt/opendj/data bak/ changelogDb/ classes/ config/ config/MakeLDIF/ config/MakeLDIF/first.names config/MakeLDIF/addrate.template config/MakeLDIF/example.template config/MakeLDIF/last.names config/MakeLDIF/cities config/MakeLDIF/states config/MakeLDIF/streets config/MakeLDIF/people_and_groups.template config/java.properties config/keystore config/audit-handlers/ config/audit-handlers/mysql_tables-example.sql config/audit-handlers/elasticsearch-config.json-example config/audit-handlers/oracle_tables-example.sql config/audit-handlers/json-stdout-config.json-example config/audit-handlers/jdbc-config.json-example config/audit-handlers/jms-config.json-example config/audit-handlers/syslog-config.json-example config/audit-handlers/splunk-config.json-example config/audit-handlers/postgres_tables-example.sql config/audit-handlers/elasticsearch-index-setup-example.json config/audit-handlers/ldap-access-stdout.json config/audit-handlers/http-access-stdout.json config/wordlist.txt config/config.ldif config/messages/ config/messages/account-reset-locked.template config/messages/account-expired.template config/messages/password-changed.template config/messages/password-expired.template config/messages/account-permanently-locked.template config/messages/password-reset.template config/messages/password-expiring.template config/messages/account-unlocked.template config/messages/account-temporarily-locked.template config/messages/account-disabled.template config/messages/account-idle-locked.template config/messages/account-enabled.template config/rest2ldap/ config/rest2ldap/endpoints/ config/rest2ldap/endpoints/api/ config/rest2ldap/endpoints/api/example-v1.json config/keystore.pin config/common-passwords.txt db/ db/tasks/ db/rootUser/ db/rootUser/rootUser.ldif db/schema/ db/schema/00-core.ldif db/schema/05-samba.ldif db/schema/03-rfc2713.ldif db/schema/06-compat.ldif db/schema/04-rfc2307bis.ldif db/schema/01-pwpolicy.ldif db/schema/02-config.ldif db/schema/03-pwpolicyextension.ldif db/schema/03-rfc3712.ldif db/schema/05-solaris.ldif db/schema/03-rfc2926.ldif db/schema/03-changelog.ldif db/schema/03-rfc2739.ldif db/schema/03-keystore.ldif db/schema/03-rfc2714.ldif db/schema/03-uddiv3.ldif db/schema/05-rfc4876.ldif db/schema/03-rfc3112.ldif db/adminRoot/ db/adminRoot/admin-backend.ldif db/monitorUser/ db/monitorUser/monitorUser.ldif extlib/ import-tmp/ ldif/ lib/ lib/extensions locks/ logs/ var/ var/upgrade/ var/upgrade/schema.ldif.current var/data.version Configuring profile AM CTS data store...... Done Configuring profile DS proxied server..... Done Done. Upgrading configuration and data... * OpenDJ has already been upgraded to version 8.0.0.c9a59f45a8b654a548fd7065172bb45faedb125a Rebuilding degraded indexes for base DN "ou=tokens"... Updating the "uid=admin" password Updating the "uid=monitor" password AUTORESTORE_FROM_DSBACKUP is missing or not set to true. Skipping restore Server configured with: Group ID : default Server ID : ds-cts-0-default Advertised listen address : ds-cts-0.ds-cts.xlou-rcs.svc.cluster.local Bootstrap replication server(s) : ds-cts-0.ds-cts.xlou-rcs.svc.cluster.local:8989,ds-cts-1.ds-cts.xlou-rcs.svc.cluster.local:8989 [17/Dec/2024:17:47:14 +0000] category=JVM severity=NOTICE msgID=1058 msg=ForgeRock Directory Services 8.0.0-20241213145456-c9a59f45a8b654a548fd7065172bb45faedb125a (build 20241213150812, revision number c9a59f45a8b654a548fd7065172bb45faedb125a) starting up [17/Dec/2024:17:47:14 +0000] category=JVM severity=NOTICE msgID=2233 msg=Installation Directory: /opt/opendj [17/Dec/2024:17:47:14 +0000] category=JVM severity=NOTICE msgID=2235 msg=Instance Directory: /opt/opendj/data [17/Dec/2024:17:47:14 +0000] category=JVM severity=NOTICE msgID=2229 msg=JVM Information: PID(1) 21.0.5+11-LTS by Azul Systems, Inc., 64-bit architecture, 3221225472 bytes heap size [17/Dec/2024:17:47:14 +0000] category=JVM severity=NOTICE msgID=2230 msg=JVM Host: ds-cts-0.ds-cts.xlou-rcs.svc.cluster.local default/ds-cts-0-default, running Linux 6.1.100+ amd64, 4294967296 bytes physical memory size, number of processors available 4 [17/Dec/2024:17:47:14 +0000] category=JVM severity=NOTICE msgID=2231 msg=JVM Arguments: "-XX:+UseG1GC", "-XX:+ExitOnOutOfMemoryError", "-Djava.security.egd=file:/dev/urandom", "-Xlog:gc:/opt/opendj/data/gc.log:time,uptime:filecount=5,filesize=50M", "-XX:MaxGCPauseMillis=200", "-XX:MaxRAMPercentage=75", "-XX:MaxTenuringThreshold=1", "-Dorg.forgerock.opendj.useVirtualThreads=false", "-Dorg.opends.server.scriptName=start-ds" [17/Dec/2024:17:47:14 +0000] category=CORE severity=WARNING msgID=1025 msg='External Access Log Publisher' is DEPRECATED for removal since 7.2.0. Its usage is highly discouraged [17/Dec/2024:17:47:15 +0000] category=BACKEND severity=NOTICE msgID=608 msg=The database backend amCts containing 5 entries has started [17/Dec/2024:17:47:15 +0000] category=SYNC.PURGE severity=INFORMATION msgID=2210 msg=The purge delay has been set to: 1 d [17/Dec/2024:17:47:15 +0000] category=SYNC.LIFECYCLE severity=NOTICE msgID=2022 msg=Replication server RS(ds-cts-0-default) started listening for new connections on address 0.0.0.0 port 8989 [17/Dec/2024:17:47:15 +0000] category=SYNC.STATE severity=NOTICE msgID=2204 msg=Directory server DS(ds-cts-0-default) did not find a generation ID for domain 'cn=schema'. A new generation ID will be computed by exporting the first 1 entries in the domain [17/Dec/2024:17:47:16 +0000] category=SYNC.CONNECTIONS severity=INFORMATION msgID=2025 msg=Replication server RS(ds-cts-0-default) has accepted a connection from directory server DS(ds-cts-0-default) for domain "cn=schema" at /10.106.40.168:55462 [17/Dec/2024:17:47:16 +0000] category=SYNC.CONNECTIONS severity=NOTICE msgID=2026 msg=Directory server DS(ds-cts-0-default) has connected to replication server RS(ds-cts-0-default) for domain "cn=schema" at 10.106.40.168:8989 with generation ID 8408 [17/Dec/2024:17:47:16 +0000] category=SYNC.STATE severity=NOTICE msgID=2204 msg=Directory server DS(ds-cts-0-default) did not find a generation ID for domain 'uid=monitor'. A new generation ID will be computed by exporting the first 1 entries in the domain [17/Dec/2024:17:47:16 +0000] category=SYNC.CONNECTIONS severity=INFORMATION msgID=2025 msg=Replication server RS(ds-cts-0-default) has accepted a connection from directory server DS(ds-cts-0-default) for domain "uid=monitor" at /10.106.40.168:55478 [17/Dec/2024:17:47:16 +0000] category=SYNC.CONNECTIONS severity=NOTICE msgID=2026 msg=Directory server DS(ds-cts-0-default) has connected to replication server RS(ds-cts-0-default) for domain "uid=monitor" at 10.106.40.168:8989 with generation ID 12671 [17/Dec/2024:17:47:16 +0000] category=SYNC.STATE severity=NOTICE msgID=2204 msg=Directory server DS(ds-cts-0-default) did not find a generation ID for domain 'ou=tokens'. A new generation ID will be computed by exporting the first 5 entries in the domain [17/Dec/2024:17:47:16 +0000] category=BACKEND severity=NOTICE msgID=610 msg=Exported 5 entries and skipped 0 in 0 seconds (average rate 384.6/sec) [17/Dec/2024:17:47:16 +0000] category=SYNC.CONNECTIONS severity=INFORMATION msgID=2025 msg=Replication server RS(ds-cts-0-default) has accepted a connection from directory server DS(ds-cts-0-default) for domain "ou=tokens" at /10.106.40.168:55486 [17/Dec/2024:17:47:16 +0000] category=SYNC.CONNECTIONS severity=NOTICE msgID=2026 msg=Directory server DS(ds-cts-0-default) has connected to replication server RS(ds-cts-0-default) for domain "ou=tokens" at 10.106.40.168:8989 with generation ID 69125 [17/Dec/2024:17:47:16 +0000] category=CORE severity=NOTICE msgID=1409 msg=Registered 0 static groups, 0 dynamic groups and 0 virtual static groups. The static group cache is using 0 b of memory (0 bytes) [17/Dec/2024:17:47:16 +0000] category=PROTOCOL.LDAP severity=NOTICE msgID=1845 msg=Started listening for new connections on Administration Connector 0.0.0.0:4444 [17/Dec/2024:17:47:16 +0000] category=PROTOCOL.LDAP severity=NOTICE msgID=1845 msg=Started listening for new connections on LDAP 0.0.0.0:1389 [17/Dec/2024:17:47:16 +0000] category=PROTOCOL.HTTP severity=NOTICE msgID=1845 msg=Started listening for new connections on HTTP 0.0.0.0:8080 [17/Dec/2024:17:47:16 +0000] category=PROTOCOL.HTTP severity=NOTICE msgID=1845 msg=Started listening for new connections on HTTPS 0.0.0.0:8443 [17/Dec/2024:17:47:16 +0000] category=PROTOCOL.LDAP severity=NOTICE msgID=1845 msg=Started listening for new connections on LDAPS 0.0.0.0:1636 [17/Dec/2024:17:47:16 +0000] category=CORE severity=NOTICE msgID=1059 msg=The Directory Server has started successfully [17/Dec/2024:17:47:16 +0000] category=CORE severity=NOTICE msgID=1061 msg=The Directory Server has sent an alert notification generated by class org.opends.server.core.DirectoryServer (alert type org.opends.server.DirectoryServerStarted, alert ID org.opends.messages.server-1059): The Directory Server has started successfully {"_id":"7c6ac99f-4403-4940-8ef5-ed7db0eb4206-54","timestamp":"2024-12-17T17:47:41.787Z","eventName":"DJ-HTTP","transactionId":"7c6ac99f-4403-4940-8ef5-ed7db0eb4206-53","client":{"ip":"10.106.40.129","port":49932},"server":{"ip":"10.106.40.168","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-0.ds-cts.xlou-rcs.svc.cluster.local:8080/healthy","headers":{"accept":["*/*"],"host":["10.106.40.168:8080"],"user-agent":["kube-probe/1.30"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"7c6ac99f-4403-4940-8ef5-ed7db0eb4206-57","timestamp":"2024-12-17T17:47:44.035Z","eventName":"DJ-HTTP","transactionId":"7c6ac99f-4403-4940-8ef5-ed7db0eb4206-56","client":{"ip":"10.106.40.16","port":54250},"server":{"ip":"10.106.40.168","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-0.ds-cts.xlou-rcs.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.106.40.168:8080"],"user-agent":["Prometheus/2.44.0"]}}},"response":{"status":"FAILED","statusCode":"404","elapsedTime":2,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"7c6ac99f-4403-4940-8ef5-ed7db0eb4206-60","timestamp":"2024-12-17T17:48:06.098Z","eventName":"DJ-HTTP","transactionId":"7c6ac99f-4403-4940-8ef5-ed7db0eb4206-59","client":{"ip":"10.106.40.27","port":44478},"server":{"ip":"10.106.40.168","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-0.ds-cts.xlou-rcs.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-cts-0.ds-cts:8080"],"user-agent":["curl/7.64.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"7c6ac99f-4403-4940-8ef5-ed7db0eb4206-72","timestamp":"2024-12-17T17:48:14.034Z","eventName":"DJ-HTTP","transactionId":"7c6ac99f-4403-4940-8ef5-ed7db0eb4206-71","client":{"ip":"10.106.40.16","port":54250},"server":{"ip":"10.106.40.168","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-cts-0.ds-cts.xlou-rcs.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.106.40.168:8080"],"user-agent":["Prometheus/2.44.0"]}}},"response":{"status":"FAILED","statusCode":"404","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"}