==================================================================================================== ========================================= Pod describe ========================================= ==================================================================================================== Name: am-7849cf7bdb-mnbt9 Namespace: xlou Priority: 0 Node: gke-xlou-cdm-default-pool-cab466d1-c9gd/10.142.0.33 Start Time: Sat, 13 May 2023 00:47:01 +0000 Labels: app=am app.kubernetes.io/component=am app.kubernetes.io/instance=am app.kubernetes.io/name=am app.kubernetes.io/part-of=forgerock pod-template-hash=7849cf7bdb tier=middle Annotations: cluster-autoscaler.kubernetes.io/safe-to-evict: true Status: Running IP: 10.106.42.139 IPs: IP: 10.106.42.139 Controlled By: ReplicaSet/am-7849cf7bdb Init Containers: fbc-init: Container ID: containerd://3e881e01e655abafb202980d4eec0c68d6b1f1fb439aa649e9a465dd5548102b Image: gcr.io/engineeringpit/lodestar-images/am:xlou Image ID: gcr.io/engineeringpit/lodestar-images/am@sha256:45f36f79f8a1976860863e2abf184e20c32f2e4750c2f7d4114b8bcf2e7566d5 Port: Host Port: Command: /bin/bash -c if [ -d /fbc/config ]; then echo "Existing openam configuration found. Skipping copy" else echo "Copying docker image configuration files to the shared volume" cd /home/forgerock/openam cp -r .homeVersion * /fbc fi State: Terminated Reason: Completed Exit Code: 0 Started: Sat, 13 May 2023 00:47:03 +0000 Finished: Sat, 13 May 2023 00:47:03 +0000 Ready: True Restart Count: 0 Environment: Mounts: /fbc from fbc (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qnzk4 (ro) truststore-init: Container ID: containerd://1d239703bbe4b4831fe3e7c2c16a1aae5bf119bf6011536a58c64912a47e399a Image: gcr.io/engineeringpit/lodestar-images/am:xlou Image ID: gcr.io/engineeringpit/lodestar-images/am@sha256:45f36f79f8a1976860863e2abf184e20c32f2e4750c2f7d4114b8bcf2e7566d5 Port: Host Port: Command: /home/forgerock/import-pem-certs.sh State: Terminated Reason: Completed Exit Code: 0 Started: Sat, 13 May 2023 00:47:03 +0000 Finished: Sat, 13 May 2023 00:47:04 +0000 Ready: True Restart Count: 0 Environment: TRUSTSTORE_PATH: /truststore/amtruststore TRUSTSTORE_PASSWORD: changeit AM_PEM_TRUSTSTORE: /var/run/secrets/truststore/cacerts Mounts: /truststore from new-truststore (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qnzk4 (ro) /var/run/secrets/truststore from truststore (rw) Containers: openam: Container ID: containerd://b1bc00945cb4951379469916480b6cdd7bbbcc02c074783a26791c3ecbd107c1 Image: gcr.io/engineeringpit/lodestar-images/am:xlou Image ID: gcr.io/engineeringpit/lodestar-images/am@sha256:45f36f79f8a1976860863e2abf184e20c32f2e4750c2f7d4114b8bcf2e7566d5 Port: 8080/TCP Host Port: 0/TCP Command: bash /home/forgerock/docker-entrypoint.sh State: Running Started: Sat, 13 May 2023 00:47:04 +0000 Ready: True Restart Count: 0 Limits: memory: 10Gi Requests: cpu: 11 memory: 10Gi Liveness: http-get http://:8080/am/json/health/live delay=30s timeout=5s period=30s #success=1 #failure=3 Readiness: http-get http://:8080/am/json/health/ready delay=20s timeout=5s period=10s #success=1 #failure=3 Startup: http-get http://:8080/am/json/health/live delay=0s timeout=1s period=10s #success=1 #failure=40 Environment Variables from: am-env-secrets Secret Optional: false platform-config ConfigMap Optional: false amster-env-secrets Secret Optional: false ds-env-secrets Secret Optional: false Environment: NAMESPACE: xlou (v1:metadata.namespace) AM_STORES_SSL_ENABLED: true TRUSTSTORE_PATH: /home/forgerock/amtruststore TRUSTSTORE_PASSWORD: changeit AM_STORES_USER_TYPE: LDAPv3ForForgeRockIAM Mounts: /home/forgerock/amtruststore from new-truststore (ro,path="amtruststore") /home/forgerock/openam from fbc (rw) /var/run/secrets/am from am-secrets (rw) /var/run/secrets/amster from amster-key (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qnzk4 (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: fbc: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: am-secrets: Type: Projected (a volume that contains injected data from multiple sources) SecretName: am-passwords SecretOptionalName: SecretName: am-keystore SecretOptionalName: amster-key: Type: Secret (a volume populated by a Secret) SecretName: amster Optional: false truststore: Type: Secret (a volume populated by a Secret) SecretName: truststore-pem Optional: false new-truststore: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: kube-api-access-qnzk4: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: Burstable Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 2m44s default-scheduler Successfully assigned xlou/am-7849cf7bdb-mnbt9 to gke-xlou-cdm-default-pool-cab466d1-c9gd Normal Pulling 2m43s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/am:xlou" Normal Pulled 2m43s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/am:xlou" in 428.329539ms Normal Created 2m42s kubelet Created container fbc-init Normal Started 2m42s kubelet Started container fbc-init Normal Pulling 2m42s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/am:xlou" Normal Pulled 2m42s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/am:xlou" in 215.40297ms Normal Created 2m42s kubelet Created container truststore-init Normal Started 2m42s kubelet Started container truststore-init Normal Pulling 2m41s kubelet Pulling image "gcr.io/engineeringpit/lodestar-images/am:xlou" Normal Pulled 2m41s kubelet Successfully pulled image "gcr.io/engineeringpit/lodestar-images/am:xlou" in 213.845671ms Normal Created 2m41s kubelet Created container openam Normal Started 2m41s kubelet Started container openam Warning Unhealthy 2m22s (x2 over 2m32s) kubelet Startup probe failed: Get "http://10.106.42.139:8080/am/json/health/live": context deadline exceeded (Client.Timeout exceeded while awaiting headers) ==================================================================================================== =========================================== Pod logs =========================================== ==================================================================================================== Copying docker image configuration files to the shared volume Copying /opt/java/openjdk/lib/security/cacerts to /truststore/amtruststore Found (1) certificates in am_combined_truststore Importing (1) certificates into /truststore/amtruststore Certificate was added to keystore Import complete! Copying bootstrap files for legacy AMKeyProvider Waiting for User Store to be available Trying ds-idrepo-0.ds-idrepo:8080/alive endpoint User Store is responding Waiting for CTS to be available Trying ds-cts-0.ds-cts:8080/alive endpoint CTS is responding Waiting for Application Store to be available Trying ds-idrepo-0.ds-idrepo:8080/alive endpoint Application Store is responding Waiting for Policy Store to be available Trying ds-idrepo-0.ds-idrepo:8080/alive endpoint Policy Store is responding Waiting for UMA Store to be available Trying ds-idrepo-0.ds-idrepo:8080/alive endpoint UMA Store is responding Starting tomcat with opts: -server -Dcom.sun.services.debug.mergeall=on -Dcom.sun.identity.configuration.directory=/home/forgerock/openam -Dcom.iplanet.services.stats.state=off -Dcom.sun.identity.sm.sms_object_filebased_enabled=true -Dorg.forgerock.donotupgrade=true -DtomcatAccessLogDir=/proc/self/fd -DtomcatAccessLogFile=1 -XX:MaxRAMPercentage=75 -XX:InitialRAMPercentage=75 -XX:MaxTenuringThreshold=1 -Djava.security.egd=file:/dev/urandom -Dorg.forgerock.am.oauth2.consent.xui_path=/OAuth2_XUI -Dcom.forgerock.am.fileconfig.basepaths=/home/forgerock/base/config/services,/home/forgerock/cdk/config/services -Djavax.net.ssl.trustStore=/home/forgerock/amtruststore -Djavax.net.ssl.trustStorePassword=changeit -Djavax.net.ssl.trustStoreType=jks NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 13-May-2023 00:47:07.041 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name: Apache Tomcat/9.0.75 13-May-2023 00:47:07.043 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: May 4 2023 13:04:05 UTC 13-May-2023 00:47:07.043 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 9.0.75.0 13-May-2023 00:47:07.043 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux 13-May-2023 00:47:07.043 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 5.10.162+ 13-May-2023 00:47:07.043 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64 13-May-2023 00:47:07.044 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /opt/java/openjdk 13-May-2023 00:47:07.044 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 11.0.19+7 13-May-2023 00:47:07.044 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Eclipse Adoptium 13-May-2023 00:47:07.044 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /usr/local/tomcat 13-May-2023 00:47:07.044 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /usr/local/tomcat 13-May-2023 00:47:07.052 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util=ALL-UNNAMED 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util.concurrent=ALL-UNNAMED 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.services.debug.mergeall=on 13-May-2023 00:47:07.053 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.identity.configuration.directory=/home/forgerock/openam 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.iplanet.services.stats.state=off 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.sun.identity.sm.sms_object_filebased_enabled=true 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.forgerock.donotupgrade=true 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -DtomcatAccessLogDir=/proc/self/fd 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -DtomcatAccessLogFile=1 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxRAMPercentage=75 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:InitialRAMPercentage=75 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxTenuringThreshold=1 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.security.egd=file:/dev/urandom 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.forgerock.am.oauth2.consent.xui_path=/OAuth2_XUI 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcom.forgerock.am.fileconfig.basepaths=/home/forgerock/base/config/services,/home/forgerock/cdk/config/services 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStore=/home/forgerock/amtruststore 13-May-2023 00:47:07.054 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStorePassword=changeit 13-May-2023 00:47:07.055 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStoreType=jks 13-May-2023 00:47:07.055 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs= 13-May-2023 00:47:07.055 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/usr/local/tomcat 13-May-2023 00:47:07.055 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/local/tomcat 13-May-2023 00:47:07.055 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/usr/local/tomcat/temp 13-May-2023 00:47:07.058 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded Apache Tomcat Native library [1.2.36] using APR version [1.7.0]. 13-May-2023 00:47:07.058 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true], UDS [true]. 13-May-2023 00:47:07.058 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true] 13-May-2023 00:47:07.061 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 3.0.2 15 Mar 2022] 13-May-2023 00:47:07.255 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"] 13-May-2023 00:47:07.272 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8081"] 13-May-2023 00:47:07.272 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [379] milliseconds 13-May-2023 00:47:07.313 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina] 13-May-2023 00:47:07.313 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.75] 13-May-2023 00:47:07.318 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/am] 13-May-2023 00:47:14.139 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.google.inject.assistedinject.FactoryProvider2 (file:/usr/local/tomcat/webapps/am/WEB-INF/lib/guice-assistedinject-4.2.3.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int) WARNING: Please consider reporting this to the maintainers of com.google.inject.assistedinject.FactoryProvider2 WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Starting up OpenAM at May 13, 2023, 12:47:16 AM {"timestamp":"2023-05-13T00:47:19.514Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (0)","logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=10, blocked=0, ldapClient=LdapClient(host=ds-idrepo-2.ds-idrepo, port=1636, protocol=LDAPS))' is no longer operational: Connect Error: Connection refused","context":"default"} {"timestamp":"2023-05-13T00:47:19.740Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (1)","logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=10, blocked=0, ldapClient=LdapClient(host=ds-idrepo-2.ds-idrepo, port=1636, protocol=LDAPS))' is no longer operational: Connect Error: Connection refused","context":"default"} {"timestamp":"2023-05-13T00:47:19.824Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"org.forgerock.openam.plugins.PluginLifecycle","message":"PluginTools#upgradePlugins: Encountered plugin org.forgerock.openam.auth.nodes.CloudOnlyNodesPlugin not configured in SMS","context":"default"} {"timestamp":"2023-05-13T00:47:21.647Z","level":"WARN","thread":"RADIUS-RadiusServerManager","logger":"org.forgerock.openam.radius.server.config.RadiusServerManager","message":"RADIUS service is not enabled.","context":"default"} {"timestamp":"2023-05-13T00:47:21.877Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.MonitoringConfig","message":"JDMK runtime not found - Legacy Monitoring disabled","context":"default"} {"timestamp":"2023-05-13T00:47:21.884Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default"} {"timestamp":"2023-05-13T00:47:22.183Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default"} {"timestamp":"2023-05-13T00:47:24.177Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=10, blocked=0, ldapClient=LdapClient(host=UNUSED.IN.FBC, port=9999, protocol=LDAP (clear-text)))' is no longer operational: Connect Error: Host 'UNUSED.IN.FBC' cannot be resolved","context":"default"} {"timestamp":"2023-05-13T00:47:24.230Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.sm.ldap.LDAPEventManager","message":"LDAPEventManager: received all entries changed event from EventService","context":"default"} {"timestamp":"2023-05-13T00:47:24.279Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.sm.ldap.SMSLdapObject","message":"SMSLDAPObject: got notifications, all objects changed","context":"default"} {"timestamp":"2023-05-13T00:47:24.944Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.sm.ldap.LDAPEventManager","message":"LDAPEventManager: received all entries changed event from EventService","context":"default"} {"timestamp":"2023-05-13T00:47:24.966Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.sm.ldap.SMSLdapObject","message":"SMSLDAPObject: got notifications, all objects changed","context":"default"} {"timestamp":"2023-05-13T00:47:25.590Z","level":"WARN","thread":"Thread-4","logger":"org.forgerock.openam.services.push.PushNotificationService","message":"No Push Notification Service Config found for realm /","context":"default","exception":"java.util.NoSuchElementException: No value present\n\tat java.base/java.util.Optional.get(Optional.java:148)\n\tat org.forgerock.openam.services.push.PushNotificationServiceConfigHelperFactory.getConfigHelperFor(PushNotificationServiceConfigHelperFactory.java:58)\n\tat org.forgerock.openam.services.push.PushNotificationService.getConfigHelper(PushNotificationService.java:280)\n\tat org.forgerock.openam.services.push.PushNotificationService.updatePreferences(PushNotificationService.java:242)\n\tat org.forgerock.openam.services.push.PushNotificationService.init(PushNotificationService.java:157)\n\tat org.forgerock.openam.services.push.PushNotificationServiceSetupListener$1.run(PushNotificationServiceSetupListener.java:48)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\n"} {"timestamp":"2023-05-13T00:47:25.591Z","level":"WARN","thread":"Thread-4","logger":"org.forgerock.openam.services.push.PushNotificationService","message":"No Push Notification Delegate configured for realm /","context":"default"} {"timestamp":"2023-05-13T00:47:25.818Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.iplanet.services.naming.WebtopNaming","message":"WebtopNaming.configMonitoring: start generating ssoserverinfo\n Start time = 2023-05-13 00:47:25","context":"default"} {"timestamp":"2023-05-13T00:47:25.822Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.MonitoringAdapters","message":"Agent.startAgent:Monitoring HTML port not enabled in configuration or JDMK not present.","context":"default"} {"timestamp":"2023-05-13T00:47:25.822Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.MonitoringAdapters","message":"Agent.startAgent:Monitoring SNMP port not enabled or JDMK not present.","context":"default"} {"timestamp":"2023-05-13T00:47:25.822Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.MonitoringAdapters","message":"Agent.startAgent:Monitoring RMI port not enabled.","context":"default"} {"timestamp":"2023-05-13T00:47:25.856Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.sm.ldap.SMSLdapObject","message":"SMSLdapObject:entryExists: LdapSmsEntryUid{dn='ou=default,ou=OrganizationConfig,ou=1.0,ou=sunFMCOTConfigService,ou=services,ou=am-config', dataStoreId=DataStoreId{id='application-store'}} does not exist","context":"default"} {"timestamp":"2023-05-13T00:47:25.864Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.sm.ldap.SMSLdapObject","message":"SMSLdapObject:entryExists: LdapSmsEntryUid{dn='ou=default,ou=OrganizationConfig,ou=1.0,ou=sunFMWSFederationMetadataService,ou=services,ou=am-config', dataStoreId=DataStoreId{id='application-store'}} does not exist","context":"default"} {"timestamp":"2023-05-13T00:47:25.874Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default"} {"timestamp":"2023-05-13T00:47:25.879Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.log.s1is.LogConfigReader","message":"LogConfigReader: DB password is null","context":"default"} {"timestamp":"2023-05-13T00:47:25.880Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.log.s1is.LogConfigReader","message":"LogConfigReader: Logfile prefix string is null","context":"default"} {"timestamp":"2023-05-13T00:47:25.880Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.log.s1is.LogConfigReader","message":"LogConfigReader: Filter class is null","context":"default"} {"timestamp":"2023-05-13T00:47:25.889Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default"} {"timestamp":"2023-05-13T00:47:25.911Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default"} {"timestamp":"2023-05-13T00:47:25.915Z","level":"WARN","thread":"main","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-0"},"logger":"com.sun.identity.monitoring.FederationMonitoringSetupImpl","message":"JDMK runtime not found - Federation SNMP disabled","context":"default"} 13-May-2023 00:47:29.187 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/am] has finished in [21,868] ms 13-May-2023 00:47:29.191 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"] 13-May-2023 00:47:29.200 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8081"] 13-May-2023 00:47:29.202 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [21929] milliseconds {"timestamp":"2023-05-13T00:47:29.408Z","level":"WARN","thread":"http-nio-8081-exec-1","logger":"org.forgerock.openam.core.rest.docs.api.ApiDocsService","message":"API Descriptor has changed - regenerating docs","context":"default"} {"timestamp":"2023-05-13T00:47:30.005Z","level":"ERROR","thread":"http-nio-8081-exec-1","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-19"},"logger":"org.forgerock.http.servlet.HttpFrameworkServlet","message":"Failed to write response - rootId:9c60b8ef-c8be-4952-8d71-b8021e52c7a9-24","context":"default","exception":"org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe\n\tat org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:351)\n\tat org.apache.catalina.connector.OutputBuffer.flushByteBuffer(OutputBuffer.java:784)\n\tat org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:687)\n\tat org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:386)\n\tat org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:364)\n\tat org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:96)\n\tat org.forgerock.http.io.IO.stream(IO.java:290)\n\tat org.forgerock.http.servlet.HttpFrameworkServlet.writeChfResponse(HttpFrameworkServlet.java:397)\n\tat org.forgerock.http.servlet.HttpFrameworkServlet.lambda$writeAndCloseChfResponse$2(HttpFrameworkServlet.java:359)\n\tat org.forgerock.http.servlet.Servlet3Adapter$SynchronousServletSynchronizer.runIfNotCompleted(Servlet3Adapter.java:151)\n\tat org.forgerock.http.servlet.HttpFrameworkServlet.writeAndCloseChfResponse(HttpFrameworkServlet.java:359)\n\tat org.forgerock.http.servlet.HttpFrameworkServlet.lambda$service$0(HttpFrameworkServlet.java:284)\n\tat org.forgerock.util.promise.Promises$CompletedPromise.thenOnResult(Promises.java:131)\n\tat org.forgerock.http.servlet.HttpFrameworkServlet.service(HttpFrameworkServlet.java:284)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:623)\n\tat org.forgerock.openam.http.OpenAMHttpFrameworkServlet.service(OpenAMHttpFrameworkServlet.java:47)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:623)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.services.datastore.DataStoreConsistencyFilter.lambda$doFilter$0(DataStoreConsistencyFilter.java:46)\n\tat org.forgerock.openam.service.datastore.ReentrantVolatileActionConsistencyController.safeExecute(ReentrantVolatileActionConsistencyController.java:37)\n\tat org.forgerock.openam.services.datastore.DataStoreConsistencyFilter.doFilter(DataStoreConsistencyFilter.java:46)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.rest.ProtocolVersionFilter.doFilter(ProtocolVersionFilter.java:63)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.headers.SecureCookieFilter.doFilter(SecureCookieFilter.java:63)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.headers.DisableSameSiteCookiesFilter.doFilter(DisableSameSiteCookiesFilter.java:106)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.validation.ResponseValidationFilter.doFilter(ResponseValidationFilter.java:59)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.headers.SetHeadersFilter.doFilter(SetHeadersFilter.java:110)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.headers.SetHeadersFilter.doFilter(SetHeadersFilter.java:110)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.headers.SetHeadersFilter.doFilter(SetHeadersFilter.java:110)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat com.sun.identity.setup.AMSetupFilter.doFilter(AMSetupFilter.java:116)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.validation.RequestEntitySizeVerificationFilter.doFilter(RequestEntitySizeVerificationFilter.java:64)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.forgerock.openam.audit.context.AuditContextFilter.doFilter(AuditContextFilter.java:47)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)\n\tat org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)\n\tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)\n\tat org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:673)\n\tat org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)\n\tat org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)\n\tat org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)\n\tat org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:926)\n\tat org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791)\n\tat org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)\n\tat org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)\n\tat org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)\n\tat org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: java.io.IOException: Broken pipe\n\tat java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method)\n\tat java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)\n\tat java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113)\n\tat java.base/sun.nio.ch.IOUtil.write(IOUtil.java:79)\n\tat java.base/sun.nio.ch.IOUtil.write(IOUtil.java:50)\n\tat java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:462)\n\tat org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:136)\n\tat org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint.java:1431)\n\tat org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:775)\n\tat org.apache.tomcat.util.net.SocketWrapperBase.writeBlocking(SocketWrapperBase.java:600)\n\tat org.apache.tomcat.util.net.SocketWrapperBase.write(SocketWrapperBase.java:544)\n\tat org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.doWrite(Http11OutputBuffer.java:540)\n\tat org.apache.coyote.http11.filters.IdentityOutputFilter.doWrite(IdentityOutputFilter.java:73)\n\tat org.apache.coyote.http11.Http11OutputBuffer.doWrite(Http11OutputBuffer.java:193)\n\tat org.apache.coyote.Response.doWrite(Response.java:603)\n\tat org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:338)\n\t... 73 common frames omitted\n"} {"timestamp":"2023-05-13T00:47:34.120Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (3)","logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'org.forgerock.openam.ldap.MonitoredLdapClient@79b15a0e' is no longer operational: Connect Error: Connection refused","context":"default"} {"timestamp":"2023-05-13T00:47:37.326Z","level":"WARN","thread":"http-nio-8081-exec-2","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-35"},"logger":"org.forgerock.openam.core.realms.impl.DefaultRealmLookup","message":"DefaultRealms:lookup Unable to find Org name for: authenticate","context":"default"} {"timestamp":"2023-05-13T00:47:37.327Z","level":"DEBUG","thread":"http-nio-8081-exec-2","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-35"},"logger":"org.forgerock.openam.cors.CorsFilter","message":"handling non-CORS request","context":"default"} {"timestamp":"2023-05-13T00:47:37.327Z","level":"DEBUG","thread":"http-nio-8081-exec-2","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-35"},"logger":"org.forgerock.openam.cors.CorsFilter","message":"handling non-CORS request","context":"default"} {"timestamp":"2023-05-13T00:47:37.388Z","level":"WARN","thread":"http-nio-8081-exec-2","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-35"},"logger":"com.sun.identity.authentication.config.AMAuthLevelManager","message":"No auth level for module OpenIdConnect","context":"default"} {"timestamp":"2023-05-13T00:47:37.400Z","level":"WARN","thread":"http-nio-8081-exec-2","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-35"},"logger":"com.sun.identity.authentication.config.AMAuthLevelManager","message":"No auth level for module AuthenticatorPushRegistration","context":"default"} {"timestamp":"2023-05-13T00:47:37.411Z","level":"WARN","thread":"http-nio-8081-exec-2","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-35"},"logger":"com.sun.identity.authentication.config.AMAuthLevelManager","message":"No auth level for module PersistentCookie","context":"default"} {"timestamp":"2023-05-13T00:47:37.549Z","eventName":"AM-ACCESS-OUTCOME","transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-35","trackingIds":["9c60b8ef-c8be-4952-8d71-b8021e52c7a9-38"],"client":{"ip":"10.106.40.154","port":49402},"http":{"request":{"secure":true,"method":"POST","path":"https://am:80/am/json/authenticate","headers":{"accept-api-version":["protocol=2.1"],"content-type":["application/json; charset=UTF-8"],"host":["am:80"],"user-agent":["Apache-HttpAsyncClient/4.1.4 (Java/11.0.17)"],"x-requested-with":["Amster"]},"queryParameters":{"authIndexType":["service"],"authIndexValue":["amsterService"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":251,"elapsedTimeUnits":"MILLISECONDS"},"realm":"/","component":"Authentication","source":"audit","topic":"access","level":"INFO","_eventId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-40"} {"timestamp":"2023-05-13T00:47:37.657Z","level":"DEBUG","thread":"http-nio-8081-exec-3","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-41"},"logger":"org.forgerock.openam.cors.CorsFilter","message":"handling non-CORS request","context":"default"} {"timestamp":"2023-05-13T00:47:37.657Z","level":"DEBUG","thread":"http-nio-8081-exec-3","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-41"},"logger":"org.forgerock.openam.cors.CorsFilter","message":"handling non-CORS request","context":"default"} {"timestamp":"2023-05-13T00:47:37.675Z","level":"WARN","thread":"http-nio-8081-exec-3","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-41"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default"} {"timestamp":"2023-05-13T00:47:37.760Z","level":"WARN","thread":"http-nio-8081-exec-3","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-41"},"logger":"com.sun.identity.idm.plugins.internal.AgentsRepo","message":"AgentsRepo.getAgentGroupConfig: Unable to get Agent Group Config due to The instance agentgroup does not exist","context":"default"} {"timestamp":"2023-05-13T00:47:37.805Z","eventName":"AM-LOGIN-MODULE-COMPLETED","transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-41","trackingIds":["9c60b8ef-c8be-4952-8d71-b8021e52c7a9-38"],"userId":"id=amadmin,ou=user,ou=am-config","principal":["amadmin"],"entries":[{"moduleId":"Amster","info":{"authIndex":"service","authControlFlag":"REQUIRED","moduleClass":"Amster","ipAddress":"10.106.40.154","authLevel":"0"}}],"result":"SUCCESSFUL","realm":"/","component":"Authentication","source":"audit","topic":"authentication","level":"INFO","_eventId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-45"} {"timestamp":"2023-05-13T00:47:37.983Z","level":"WARN","thread":"http-nio-8081-exec-3","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-41"},"logger":"com.sun.identity.monitoring.MonitoringServicesImpl","message":"JDMK runtime not found - Policy Monitoring disabled","context":"default"} {"timestamp":"2023-05-13T00:47:37.977Z","eventName":"AM-SESSION-CREATED","transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-41","trackingIds":["9c60b8ef-c8be-4952-8d71-b8021e52c7a9-38"],"userId":"id=amadmin,ou=user,ou=am-config","runAs":"id=amadmin,ou=user,ou=am-config","objectId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-38","operation":"CREATE","realm":"/","component":"Session","source":"audit","topic":"activity","level":"INFO","_eventId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-52"} {"timestamp":"2023-05-13T00:47:37.995Z","eventName":"AM-LOGIN-COMPLETED","transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-41","trackingIds":["9c60b8ef-c8be-4952-8d71-b8021e52c7a9-38"],"userId":"id=amadmin,ou=user,ou=am-config","principal":["amadmin"],"entries":[{"moduleId":"Amster","info":{"authIndex":"service","authIndexValue":"amsterService","ipAddress":"10.106.40.154","authLevel":"0"}}],"result":"SUCCESSFUL","realm":"/","component":"Authentication","source":"audit","topic":"authentication","level":"INFO","_eventId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-56"} {"timestamp":"2023-05-13T00:47:38.016Z","eventName":"AM-ACCESS-OUTCOME","transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-41","trackingIds":["9c60b8ef-c8be-4952-8d71-b8021e52c7a9-38"],"userId":"id=amadmin,ou=user,ou=am-config","client":{"ip":"10.106.40.154","port":49402},"http":{"request":{"secure":true,"method":"POST","path":"https://am:80/am/json/authenticate","headers":{"accept-api-version":["protocol=2.1"],"content-type":["application/json; charset=UTF-8"],"host":["am:80"],"user-agent":["Apache-HttpAsyncClient/4.1.4 (Java/11.0.17)"],"x-requested-with":["Amster"]},"queryParameters":{"authIndexType":["service"],"authIndexValue":["amsterService"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":360,"elapsedTimeUnits":"MILLISECONDS"},"realm":"/","component":"Authentication","source":"audit","topic":"access","level":"INFO","_eventId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-62"} {"timestamp":"2023-05-13T00:47:38.026Z","level":"WARN","thread":"http-nio-8081-exec-4","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-63"},"logger":"org.forgerock.openam.core.realms.impl.DefaultRealmLookup","message":"DefaultRealms:lookup Unable to find Org name for: serverinfo","context":"default"} {"timestamp":"2023-05-13T00:47:38.026Z","level":"DEBUG","thread":"http-nio-8081-exec-4","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-63"},"logger":"org.forgerock.openam.cors.CorsFilter","message":"handling non-CORS request","context":"default"} {"timestamp":"2023-05-13T00:47:38.026Z","level":"DEBUG","thread":"http-nio-8081-exec-4","mdc":{"transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-63"},"logger":"org.forgerock.openam.cors.CorsFilter","message":"handling non-CORS request","context":"default"} {"timestamp":"2023-05-13T00:47:38.065Z","eventName":"AM-ACCESS-OUTCOME","transactionId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-63","client":{"ip":"10.106.40.154","port":49402},"server":{"ip":"10.106.42.139","port":8081},"http":{"request":{"secure":true,"method":"GET","path":"https://am:80/am/json/serverinfo/*","headers":{"accept-api-version":["protocol=2.1"],"host":["am:80"],"user-agent":["Apache-HttpAsyncClient/4.1.4 (Java/11.0.17)"]}}},"request":{"protocol":"CREST","operation":"READ"},"response":{"status":"SUCCESSFUL","statusCode":"","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS","detail":{"revision":"-485253810"}},"realm":"/","component":"Server Info","source":"audit","topic":"access","level":"INFO","_eventId":"9c60b8ef-c8be-4952-8d71-b8021e52c7a9-67"} 10.106.42.129 - - [13/May/2023:00:47:29 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 709ms 10.106.42.129 - - [13/May/2023:00:47:29 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 709ms 10.106.43.6 - - [13/May/2023:00:47:30 +0000] "GET /am/json/metrics/prometheus HTTP/1.1" 200 73728 778ms 10.106.42.129 - - [13/May/2023:00:47:32 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.42.129 - - [13/May/2023:00:47:32 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 6ms 10.106.40.154 - - [13/May/2023:00:47:37 +0000] "POST /am/json/authenticate?authIndexType=service&authIndexValue=amsterService HTTP/1.1" 200 668 265ms 10.106.40.154 - - [13/May/2023:00:47:38 +0000] "POST /am/json/authenticate?authIndexType=service&authIndexValue=amsterService HTTP/1.1" 200 167 362ms 10.106.40.154 - - [13/May/2023:00:47:38 +0000] "GET /am/json/serverinfo/* HTTP/1.1" 200 544 55ms {"timestamp":"2023-05-13T00:47:47.703Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (4)","logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=14, blocked=0, ldapClient=LdapClient(host=ds-idrepo-2.ds-idrepo, port=1636, protocol=LDAPS))' is no longer operational: Connect Error: Connection refused","context":"default"} {"timestamp":"2023-05-13T00:47:47.705Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (4)","logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=14, blocked=0, ldapClient=LdapClient(host=ds-idrepo-2.ds-idrepo, port=1636, protocol=LDAPS))' is no longer operational: Connect Error: Connection refused","context":"default"} {"timestamp":"2023-05-13T00:47:47.719Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (0)","logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=2, blocked=0, ldapClient=LdapClient(host=ds-idrepo-2.ds-idrepo, port=1636, protocol=LDAPS))' is no longer operational: Connect Error: Connection refused","context":"default"} {"timestamp":"2023-05-13T00:47:47.721Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (3)","logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=14, blocked=0, ldapClient=LdapClient(host=ds-idrepo-1.ds-idrepo, port=1636, protocol=LDAPS))' is no longer operational: Invalid Credentials","context":"default"} {"timestamp":"2023-05-13T00:47:47.733Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (3)","logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'ConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=2, blocked=0, ldapClient=LdapClient(host=ds-idrepo-1.ds-idrepo, port=1636, protocol=LDAPS))' is no longer operational: Invalid Credentials","context":"default"} {"timestamp":"2023-05-13T00:47:47.868Z","level":"WARN","thread":"LDAP SDK Default AsyncRx I/O Thread (2)","logger":"org.forgerock.opendj.ldap.LoadBalancer","message":"Connection factory 'org.forgerock.openam.ldap.MonitoredLdapClient@5ba5fd13' is no longer operational: Connect Error: Connection refused","context":"default"} 10.106.42.129 - - [13/May/2023:00:47:42 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.43.6 - - [13/May/2023:00:47:46 +0000] "GET /am/json/metrics/prometheus HTTP/1.1" 200 356577 45ms 10.106.42.129 - - [13/May/2023:00:47:52 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.42.129 - - [13/May/2023:00:48:02 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.42.129 - - [13/May/2023:00:48:02 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.42.129 - - [13/May/2023:00:48:12 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.43.6 - - [13/May/2023:00:48:16 +0000] "GET /am/json/metrics/prometheus HTTP/1.1" 200 356594 31ms 10.106.42.129 - - [13/May/2023:00:48:22 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.42.129 - - [13/May/2023:00:48:32 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.42.129 - - [13/May/2023:00:48:32 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.42.129 - - [13/May/2023:00:48:42 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.43.6 - - [13/May/2023:00:48:46 +0000] "GET /am/json/metrics/prometheus HTTP/1.1" 200 356576 41ms 10.106.42.129 - - [13/May/2023:00:48:52 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 3ms 10.106.41.135 - - [13/May/2023:00:49:01 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 5ms 10.106.42.129 - - [13/May/2023:00:49:02 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 2ms 10.106.42.129 - - [13/May/2023:00:49:02 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.42.129 - - [13/May/2023:00:49:12 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.43.6 - - [13/May/2023:00:49:16 +0000] "GET /am/json/metrics/prometheus HTTP/1.1" 200 356613 28ms 10.106.42.129 - - [13/May/2023:00:49:22 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms 10.106.42.129 - - [13/May/2023:00:49:32 +0000] "GET /am/json/health/live HTTP/1.1" 200 - 3ms 10.106.42.129 - - [13/May/2023:00:49:32 +0000] "GET /am/json/health/ready HTTP/1.1" 200 - 4ms