==================================================================================================== ========================================= Pod describe ========================================= ==================================================================================================== Name: am-869fdb5db9-wt7sg Namespace: xlou Priority: 0 Node: gke-xlou-cdm-default-pool-f05840a3-rt14/10.142.15.199 Start Time: Wed, 16 Aug 2023 01:14:34 +0000 Labels: app=am app.kubernetes.io/component=am app.kubernetes.io/instance=am app.kubernetes.io/name=am app.kubernetes.io/part-of=forgerock pod-template-hash=869fdb5db9 tier=middle Annotations: cluster-autoscaler.kubernetes.io/safe-to-evict: true Status: Running IP: 10.106.46.4 IPs: IP: 10.106.46.4 Controlled By: ReplicaSet/am-869fdb5db9 Init Containers: fbc-init: Container ID: containerd://54b1e5cd73232fd1c57f0f3486a6b166e83e1f8ad2786664c97d5988d13afc72 Image: gcr.io/engineeringpit/lodestar-images/am:xlou-7.4.0-66777f275067f01bf7084f6278d26174d122e555 Image ID: gcr.io/engineeringpit/lodestar-images/am@sha256:ae9d735ca27c2f56df7488a5de267865fc8b4f493e6ee8fad4d71c67e61a0355 Port: Host Port: Command: /bin/bash -c if [ -d /fbc/config ]; then echo "Existing openam configuration found. Skipping copy" else echo "Copying docker image configuration files to the shared volume" cd /home/forgerock/openam cp -r .homeVersion * /fbc fi State: Terminated Reason: Completed Exit Code: 0 Started: Wed, 16 Aug 2023 01:14:44 +0000 Finished: Wed, 16 Aug 2023 01:14:44 +0000 Ready: True Restart Count: 0 Environment: Mounts: /fbc from fbc (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qt2x4 (ro) truststore-init: Container ID: containerd://84341a4526514e707c4a64b1b8c0ee40a92185e4f7646b943ab9cf00e0d0ff6a Image: gcr.io/engineeringpit/lodestar-images/am:xlou-7.4.0-66777f275067f01bf7084f6278d26174d122e555 Image ID: gcr.io/engineeringpit/lodestar-images/am@sha256:ae9d735ca27c2f56df7488a5de267865fc8b4f493e6ee8fad4d71c67e61a0355 Port: Host Port: Command: /home/forgerock/import-pem-certs.sh State: Terminated Reason: Completed Exit Code: 0 Started: Wed, 16 Aug 2023 01:14:48 +0000 Finished: Wed, 16 Aug 2023 01:14:49 +0000 Ready: True Restart Count: 0 Environment: TRUSTSTORE_PATH: /truststore/amtruststore TRUSTSTORE_PASSWORD: changeit AM_PEM_TRUSTSTORE: /var/run/secrets/truststore/cacerts AM_PEM_TRUSTSTORE_DS: /var/run/secrets/truststore/ca.crt Mounts: /truststore from new-truststore (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qt2x4 (ro) /var/run/secrets/truststore from truststore (rw) Containers: openam: Container ID: containerd://194e20240a6d1acb8a2051039931fdab22ba2eff352118ed9c2f176840d6d514 Image: gcr.io/engineeringpit/lodestar-images/am:xlou-7.4.0-66777f275067f01bf7084f6278d26174d122e555 Image ID: gcr.io/engineeringpit/lodestar-images/am@sha256:ae9d735ca27c2f56df7488a5de267865fc8b4f493e6ee8fad4d71c67e61a0355 Port: 8080/TCP Host Port: 0/TCP Command: bash /home/forgerock/docker-entrypoint.sh State: Running Started: Wed, 16 Aug 2023 01:14:49 +0000 Ready: True Restart Count: 0 Limits: memory: 10Gi Requests: cpu: 11 memory: 10Gi Liveness: http-get http://:8080/am/json/health/live delay=30s timeout=5s period=30s #success=1 #failure=3 Readiness: http-get http://:8080/am/json/health/ready delay=20s timeout=5s period=10s #success=1 #failure=3 Startup: http-get http://:8080/am/json/health/live delay=0s timeout=1s period=10s #success=1 #failure=40 Environment Variables from: am-env-secrets Secret Optional: false platform-config ConfigMap Optional: false amster-env-secrets Secret Optional: false ds-env-secrets Secret Optional: false Environment: NAMESPACE: xlou (v1:metadata.namespace) AM_STORES_SSL_ENABLED: true TRUSTSTORE_PATH: /home/forgerock/amtruststore TRUSTSTORE_PASSWORD: changeit AM_STORES_USER_TYPE: LDAPv3ForForgeRockIAM Mounts: /home/forgerock/amtruststore from new-truststore (ro,path="amtruststore") /home/forgerock/logging from am-logback (rw) /home/forgerock/openam from fbc (rw) /var/run/secrets/am from am-secrets (rw) /var/run/secrets/amster from amster-key (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qt2x4 (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: fbc: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: am-secrets: Type: Projected (a volume that contains injected data from multiple sources) SecretName: am-passwords SecretOptionalName: SecretName: am-keystore SecretOptionalName: amster-key: Type: Secret (a volume populated by a Secret) SecretName: amster Optional: false truststore: Type: Projected (a volume that contains injected data from multiple sources) SecretName: truststore-pem SecretOptionalName: SecretName: ds-ssl-keypair SecretOptionalName: new-truststore: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: am-logback: Type: ConfigMap (a volume populated by a ConfigMap) Name: am-logback Optional: false kube-api-access-qt2x4: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: Burstable Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 8m29s default-scheduler Successfully assigned xlou/am-869fdb5db9-wt7sg to gke-xlou-cdm-default-pool-f05840a3-rt14 Warning FailedMount 8m28s kubelet MountVolume.SetUp failed for volume "am-secrets" : failed to sync secret cache: timed out waiting for the condition Warning FailedMount 8m28s kubelet MountVolume.SetUp failed for volume "truststore" : failed to sync secret cache: timed out waiting for the condition Normal Pulling 8m27s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/am:xlou-7.4.0-66777f275067f01bf7084f6278d26174d122e555" Normal Pulled 8m19s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/am:xlou-7.4.0-66777f275067f01bf7084f6278d26174d122e555" in 7.575565368s Normal Created 8m19s kubelet Created container fbc-init Normal Started 8m19s kubelet Started container fbc-init Normal Pulling 8m15s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/am:xlou-7.4.0-66777f275067f01bf7084f6278d26174d122e555" Normal Pulled 8m15s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/am:xlou-7.4.0-66777f275067f01bf7084f6278d26174d122e555" in 211.5052ms Normal Created 8m15s kubelet Created container truststore-init Normal Started 8m15s kubelet Started container truststore-init Normal Pulling 8m14s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/am:xlou-7.4.0-66777f275067f01bf7084f6278d26174d122e555" Normal Pulled 8m14s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/am:xlou-7.4.0-66777f275067f01bf7084f6278d26174d122e555" in 196.737545ms Normal Created 8m14s kubelet Created container openam Normal Started 8m14s kubelet Started container openam Warning Unhealthy 7m56s (x2 over 8m6s) kubelet Startup probe failed: Get "http://10.106.46.4:8080/am/json/health/live": context deadline exceeded (Client.Timeout exceeded while awaiting headers) ==================================================================================================== =========================================== Pod logs =========================================== ==================================================================================================== Copying docker image configuration files to the shared volume Copying /opt/java/openjdk/lib/security/cacerts to /truststore/amtruststore Found (2) certificates in am_combined_truststore Importing (2) certificates into /truststore/amtruststore Certificate was added to keystore Certificate was added to keystore Import complete! Copying bootstrap files for legacy AMKeyProvider Waiting for User Store to be available Trying ds-idrepo-0.ds-idrepo:8080/alive endpoint User Store is responding Waiting for CTS to be available Trying ds-cts-0.ds-cts:8080/alive endpoint CTS is responding Waiting for Application Store to be available Trying ds-idrepo-0.ds-idrepo:8080/alive endpoint Application Store is responding Waiting for Policy Store to be available Trying ds-idrepo-0.ds-idrepo:8080/alive endpoint Policy Store is responding Waiting for UMA Store to be available Trying ds-idrepo-0.ds-idrepo:8080/alive endpoint UMA Store is responding Starting tomcat with opts: -server -Dcom.sun.services.debug.mergeall=on -Dcom.sun.identity.configuration.directory=/home/forgerock/openam -Dcom.iplanet.services.stats.state=off -Dcom.sun.identity.sm.sms_object_filebased_enabled=true -Dorg.forgerock.donotupgrade=true -DtomcatAccessLogDir=/proc/self/fd -DtomcatAccessLogFile=1 -XX:MaxRAMPercentage=75 -XX:InitialRAMPercentage=75 -XX:MaxTenuringThreshold=1 -Djava.security.egd=file:/dev/urandom -Dorg.forgerock.am.oauth2.consent.xui_path=/OAuth2_XUI -Dcom.forgerock.am.fileconfig.basepaths=/home/forgerock/base/config/services,/home/forgerock/cdk/config/services -Djavax.net.ssl.trustStore=/home/forgerock/amtruststore -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=jks NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 16-Aug-2023 01:14:51.564 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name: Apache Tomcat/9.0.78 16-Aug-2023 01:14:51.566 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Jul 4 2023 13:15:43 UTC 16-Aug-2023 01:14:51.566 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 9.0.78.0 16-Aug-2023 01:14:51.566 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux 16-Aug-2023 01:14:51.566 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 5.10.162+ 16-Aug-2023 01:14:51.566 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64 16-Aug-2023 01:14:51.566 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /opt/java/openjdk 16-Aug-2023 01:14:51.566 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 11.0.20+8 16-Aug-2023 01:14:51.567 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Eclipse Adoptium 16-Aug-2023 01:14:51.567 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /usr/local/tomcat 16-Aug-2023 01:14:51.567 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /usr/local/tomcat 16-Aug-2023 01:14:51.576 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED 16-Aug-2023 01:14:51.576 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED 16-Aug-2023 01:14:51.576 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util=ALL-UNNAMED 16-Aug-2023 01:14:51.576 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util.concurrent=ALL-UNNAMED 16-Aug-2023 01:14:51.576 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 16-Aug-2023 01:14:51.576 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties 16-Aug-2023 01:14:51.576 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 16-Aug-2023 01:14:51.576 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048 16-Aug-2023 01:14:51.577 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources 16-Aug-2023 01:14:51.577 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 16-Aug-2023 01:14:51.577 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.services.debug.mergeall=on 16-Aug-2023 01:14:51.577 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.identity.configuration.directory=/home/forgerock/openam 16-Aug-2023 01:14:51.577 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.iplanet.services.stats.state=off 16-Aug-2023 01:14:51.577 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.identity.sm.sms_object_filebased_enabled=true 16-Aug-2023 01:14:51.577 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.forgerock.donotupgrade=true 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -DtomcatAccessLogDir=/proc/self/fd 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -DtomcatAccessLogFile=1 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxRAMPercentage=75 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:InitialRAMPercentage=75 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxTenuringThreshold=1 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.security.egd=file:/dev/urandom 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.forgerock.am.oauth2.consent.xui_path=/OAuth2_XUI 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.forgerock.am.fileconfig.basepaths=/home/forgerock/base/config/services,/home/forgerock/cdk/config/services 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStore=/home/forgerock/amtruststore 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStorePassword=changeit 16-Aug-2023 01:14:51.578 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStoreType=jks 16-Aug-2023 01:14:51.579 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs= 16-Aug-2023 01:14:51.579 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/usr/local/tomcat 16-Aug-2023 01:14:51.579 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/local/tomcat 16-Aug-2023 01:14:51.579 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/usr/local/tomcat/temp 16-Aug-2023 01:14:51.582 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded Apache Tomcat Native library [1.2.37] using APR version [1.7.0]. 16-Aug-2023 01:14:51.582 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true], UDS [true]. 16-Aug-2023 01:14:51.582 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true] 16-Aug-2023 01:14:51.585 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 3.0.2 15 Mar 2022] 16-Aug-2023 01:14:51.788 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"] 16-Aug-2023 01:14:51.803 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8081"] 16-Aug-2023 01:14:51.804 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [403] milliseconds 16-Aug-2023 01:14:51.846 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina] 16-Aug-2023 01:14:51.846 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.78] 16-Aug-2023 01:14:51.851 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/am] 16-Aug-2023 01:14:58.967 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.google.inject.assistedinject.FactoryProvider2 (file:/usr/local/tomcat/webapps/am/WEB-INF/lib/guice-assistedinject-4.2.3.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int) WARNING: Please consider reporting this to the maintainers of com.google.inject.assistedinject.FactoryProvider2 WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Starting up OpenAM at Aug 16, 2023, 1:15:01 AM {"timestamp":"2023-08-16T01:15:04.306Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (0)","logger":"LDAP_CLIENT","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=10, blocked=0, ldapClient=LdapClient(host=ds-idrepo-2.ds-idrepo, port=1636, protocol=LDAPS))' is no longer operational: Connect Error: Connection refused","context":"default","transactionId":null} {"timestamp":"2023-08-16T01:15:04.563Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (7)","logger":"LDAP_CLIENT","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=10, blocked=0, ldapClient=LdapClient(host=ds-idrepo-2.ds-idrepo, port=1636, protocol=LDAPS))' is no longer operational: Connect Error: Connection refused","context":"default","transactionId":null} {"timestamp":"2023-08-16T01:15:04.608Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"org.forgerock.openam.plugins.PluginLifecycle","message":"PluginTools#upgradePlugins: Encountered plugin org.forgerock.openam.auth.nodes.CloudOnlyNodesPlugin not configured in SMS","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:06.418Z","level":"WARN","thread":"RADIUS-RadiusServerManager","logger":"org.forgerock.openam.radius.server.config.RadiusServerManager","message":"RADIUS service is not enabled.","context":"default","transactionId":null} {"timestamp":"2023-08-16T01:15:06.545Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.MonitoringConfig","message":"JDMK runtime not found - Legacy Monitoring disabled","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:06.551Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:06.826Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:08.719Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"LDAP_CLIENT","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=10, blocked=0, ldapClient=LdapClient(host=UNUSED.IN.FBC, port=9999, protocol=LDAP (clear-text)))' is no longer operational: Connect Error: Host 'UNUSED.IN.FBC' cannot be resolved","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:08.765Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.sm.ldap.LDAPEventManager","message":"LDAPEventManager: received all entries changed event from EventService","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:08.829Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.sm.ldap.SMSLdapObject","message":"SMSLDAPObject: got notifications, all objects changed","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:09.418Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.sm.ldap.LDAPEventManager","message":"LDAPEventManager: received all entries changed event from EventService","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:09.460Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.sm.ldap.SMSLdapObject","message":"SMSLDAPObject: got notifications, all objects changed","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.025Z","level":"WARN","thread":"Thread-4","logger":"org.forgerock.openam.services.push.PushNotificationService","message":"No Push Notification Service Config found for realm /","context":"default","exception":"java.util.NoSuchElementException: No value present\n\tat java.base/java.util.Optional.get(Optional.java:148)\n\tat org.forgerock.openam.services.push.PushNotificationServiceConfigHelperFactory.getConfigHelperFor(PushNotificationServiceConfigHelperFactory.java:58)\n\tat org.forgerock.openam.services.push.PushNotificationService.getConfigHelper(PushNotificationService.java:280)\n\tat org.forgerock.openam.services.push.PushNotificationService.updatePreferences(PushNotificationService.java:242)\n\tat org.forgerock.openam.services.push.PushNotificationService.init(PushNotificationService.java:157)\n\tat org.forgerock.openam.services.push.PushNotificationServiceSetupListener$1.run(PushNotificationServiceSetupListener.java:48)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\n","transactionId":null} {"timestamp":"2023-08-16T01:15:10.026Z","level":"WARN","thread":"Thread-4","logger":"org.forgerock.openam.services.push.PushNotificationService","message":"No Push Notification Delegate configured for realm /","context":"default","transactionId":null} {"timestamp":"2023-08-16T01:15:10.164Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.iplanet.services.naming.WebtopNaming","message":"WebtopNaming.configMonitoring: start generating ssoserverinfo\n Start time = 2023-08-16 01:15:10","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.167Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.MonitoringAdapters","message":"Agent.startAgent:Monitoring HTML port not enabled in configuration or JDMK not present.","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.167Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.MonitoringAdapters","message":"Agent.startAgent:Monitoring SNMP port not enabled or JDMK not present.","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.168Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.MonitoringAdapters","message":"Agent.startAgent:Monitoring RMI port not enabled.","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.188Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.sm.ldap.SMSLdapObject","message":"SMSLdapObject:entryExists: LdapSmsEntryUid{dn='ou=default,ou=OrganizationConfig,ou=1.0,ou=sunFMCOTConfigService,ou=services,ou=am-config', dataStoreId=DataStoreId{id='application-store'}} does not exist","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.195Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.sm.ldap.SMSLdapObject","message":"SMSLdapObject:entryExists: LdapSmsEntryUid{dn='ou=default,ou=OrganizationConfig,ou=1.0,ou=sunFMWSFederationMetadataService,ou=services,ou=am-config', dataStoreId=DataStoreId{id='application-store'}} does not exist","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.202Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.205Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.log.s1is.LogConfigReader","message":"LogConfigReader: DB password is null","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.206Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.log.s1is.LogConfigReader","message":"LogConfigReader: Logfile prefix string is null","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.206Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.log.s1is.LogConfigReader","message":"LogConfigReader: Filter class is null","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.211Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.229Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} {"timestamp":"2023-08-16T01:15:10.232Z","level":"WARN","thread":"main","mdc":{"transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"},"logger":"com.sun.identity.monitoring.FederationMonitoringSetupImpl","message":"JDMK runtime not found - Federation SNMP disabled","context":"default","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-0"} 16-Aug-2023 01:15:12.933 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/am] has finished in [21,080] ms 16-Aug-2023 01:15:12.936 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"] 16-Aug-2023 01:15:12.946 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8081"] 16-Aug-2023 01:15:12.948 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [21143] milliseconds {"timestamp":"2023-08-16T01:15:13.089Z","level":"WARN","thread":"http-nio-8080-exec-1","logger":"org.forgerock.openam.core.rest.docs.api.ApiDocsService","message":"API Descriptor has changed - regenerating docs","context":"default","transactionId":null} {"timestamp":"2023-08-16T01:15:18.653Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (4)","logger":"LDAP_CLIENT","message":"Connection factory 'org.forgerock.openam.ldap.MonitoredLdapClient@187c6a38' is no longer operational: Connect Error: Connection refused","context":"default","transactionId":null} 10.106.46.1 - - [16/Aug/2023:01:15:13 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 581ms 10.106.46.1 - - [16/Aug/2023:01:15:13 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 581ms 10.106.46.1 - - [16/Aug/2023:01:15:16 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:15:16 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:15:26 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:15:36 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:15:36 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:15:46 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:15:56 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:16:06 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:16:06 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:16:16 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:16:26 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:16:36 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 2ms 10.106.46.1 - - [16/Aug/2023:01:16:36 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:16:46 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:16:56 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:17:06 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:17:06 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:17:16 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:17:26 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:17:36 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:17:36 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:17:46 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:17:56 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:18:06 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 2ms 10.106.46.1 - - [16/Aug/2023:01:18:06 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:18:16 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:18:26 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:18:36 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:18:36 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:18:46 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:18:56 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:19:06 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:19:06 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.46.1 - - [16/Aug/2023:01:19:16 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:19:26 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:19:36 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:19:36 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:19:46 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:19:56 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:20:06 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 2ms 10.106.46.1 - - [16/Aug/2023:01:20:06 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:20:16 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:20:26 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:20:36 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 2ms 10.106.46.1 - - [16/Aug/2023:01:20:36 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:20:46 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:20:56 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:21:06 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 2ms 10.106.46.1 - - [16/Aug/2023:01:21:06 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:21:16 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:21:26 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:21:36 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 2ms 10.106.46.1 - - [16/Aug/2023:01:21:36 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:21:46 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:21:56 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:22:06 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:22:06 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:22:16 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.46.1 - - [16/Aug/2023:01:22:26 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms {"timestamp":"2023-08-16T01:22:42.938Z","eventName":"AM-ACCESS-OUTCOME","transactionId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-215","trackingIds":["b52d83a6-2a86-4db1-874c-ba0c3bcee018-245"],"userId":"id=amadmin,ou=user,ou=am-config","client":{"ip":"10.106.45.166","port":48634},"server":{"ip":"10.106.46.4","port":8081},"http":{"request":{"secure":true,"method":"POST","path":"https://xlou.iam.xlou-cdm.engineeringpit.com/am/json/realms/root/sessions","headers":{"accept":["*/*"],"accept-api-version":["protocol=2.1,resource=3.0"],"content-type":["application/json"],"host":["xlou.iam.xlou-cdm.engineeringpit.com"],"user-agent":["python-requests/2.28.2"],"x-forwarded-for":["35.212.145.225"],"x-forwarded-host":["xlou.iam.xlou-cdm.engineeringpit.com"],"x-forwarded-port":["443"],"x-forwarded-proto":["https"],"x-real-ip":["35.212.145.225"],"x-request-id":["5d64e0702c71fa6bce969b01286b5443"],"x-scheme":["https"]}}},"request":{"protocol":"CREST","operation":"ACTION","detail":{"action":"getSessionInfo"}},"response":{"status":"SUCCESSFUL","statusCode":"","elapsedTime":170,"elapsedTimeUnits":"MILLISECONDS"},"realm":"/","component":"Session","source":"audit","topic":"access","level":"INFO","_eventId":"1f09ec9c-774c-4c68-8696-03c0f9953f28-227"} 10.106.46.1 - - [16/Aug/2023:01:22:36 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 2ms 10.106.46.1 - - [16/Aug/2023:01:22:36 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.45.166 - - [16/Aug/2023:01:22:42 +0000] "POST /am/json/realms/root/sessions?_action=getSessionInfo HTTP/1.1" 200 289 227ms 10.106.46.1 - - [16/Aug/2023:01:22:46 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.46.1 - - [16/Aug/2023:01:22:56 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms