--Task-- name: CheckReplication_ds-idrepo enabled: True class_name: CheckReplicationTask source_name: ds-idrepo source_namespace: >default< target_name: ds-idrepo target_namespace: >default< start: 0 stop: None timeout: not yet initialized for this Task as timeout was not explicitly defined, see updates bellow loop: False interval: None dependencies: [] wait_for: [] options: {} group_name: None Current dir: /mnt/disks/data/xslou/lodestar-fork/pyrock ________________________________________________________________________________ [21/Feb/2023 01:20:41] CheckReplication_ds-idrepo pre : N/A ________________________________________________________________________________ max_time: 59m 50s sleep_interval: 10 [21/Feb/2023 01:20:41] - INFO: Timeout for this Task has changed to 2h (7200 seconds) [21/Feb/2023 01:20:41] - INFO: timeout was set to Task default (2x global_duration or 300 seconds (whatever is higher)), because it was unset ________________________________________________________________________________ [21/Feb/2023 01:20:41] CheckReplication_ds-idrepo step1 : Check number of entries ________________________________________________________________________________ loop 1 : SUCCESS: Pods "ds-idrepo-0" and "ds-idrepo-1" are in sync (both have 1000022 entries) loop 1 : ERROR : Pod "ds-idrepo-0" has 1000022 entries but pod "ds-idrepo-2" has 1000021 entries (should have same number of entries) loop 1 : ERROR : Pod "ds-idrepo-1" has 1000022 entries but pod "ds-idrepo-2" has 1000021 entries (should have same number of entries) loop 1 : Sleep 10 and retry loop 2 : SUCCESS: Pods "ds-idrepo-0" and "ds-idrepo-1" are in sync (both have 1000021 entries) loop 2 : SUCCESS: Pods "ds-idrepo-0" and "ds-idrepo-2" are in sync (both have 1000021 entries) loop 2 : SUCCESS: Pods "ds-idrepo-1" and "ds-idrepo-2" are in sync (both have 1000021 entries) [check_number_of_entries] SUCCESS : replication is synchronised after 15s ________________________________________________________________________________ [21/Feb/2023 01:21:02] CheckReplication_ds-idrepo step2 : Monitor replication attributes in cn=monitor ________________________________________________________________________________ loop 1 : SUCCESS: ds-idrepo-0 : ds-mon-current-delay = 0 loop 1 : SUCCESS: ds-idrepo-0 : ds-mon-updates-inbound-queue = 0 loop 1 : SUCCESS: ds-idrepo-1 : ds-mon-current-delay = 0 loop 1 : SUCCESS: ds-idrepo-1 : ds-mon-updates-inbound-queue = 0 loop 1 : SUCCESS: ds-idrepo-2 : ds-mon-current-delay = 0 loop 1 : SUCCESS: ds-idrepo-2 : ds-mon-updates-inbound-queue = 0 [check_monitoring_replication] final status : { "ds-idrepo-0": { "status": true, "ds-mon-current-delay": 0, "ds-mon-updates-inbound-queue": 0 }, "ds-idrepo-1": { "status": true, "ds-mon-current-delay": 0, "ds-mon-updates-inbound-queue": 0 }, "ds-idrepo-2": { "status": true, "ds-mon-current-delay": 0, "ds-mon-updates-inbound-queue": 0 } } [check_monitoring_replication] SUCCESS : replication data in cn=monitor is synchronised after 0s ________________________________________________________________________________ [21/Feb/2023 01:21:08] CheckReplication_ds-idrepo step3 : Temporary step for PERF-1831 : Dump cn=monitor replication attributes ________________________________________________________________________________ ---------------- Pod ds-idrepo-0 ---------------- [run_command]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "cn=replicas,cn=replication,cn=monitor" "objectclass=*" [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- [print_head_tail]: Print head (80 lines) and tail (80 lines) of input string (172 lines): ----- output ----- dn: cn=replicas,cn=replication,cn=monitor objectclass: top objectclass: ds-monitor objectclass: ds-monitor-branch cn: replicas dn: ds-mon-domain-name=dc=openidm\,dc=forgerock\,dc=io,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.52.12:8989 ds-mon-connected-to-server-id: ds-idrepo-0 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 555879 ds-mon-domain-name: dc=openidm,dc=forgerock,dc=io ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":0,"total":0.000,"mean_rate":0.000,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.000,"mean":0.000,"min":0.000,"max":0.000,"stddev":0.000,"p50":0.000,"p75":0.000,"p95":0.000,"p98":0.000,"p99":0.000,"p999":0.000,"p9999":0.000,"p99999":0.000} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 2000885 ds-mon-server-id: ds-idrepo-0 ds-mon-server-state: 010b0186718f6c16003d2daeds-idrepo-0 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004357.087Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [3, 1, 1, 0, 2, 0, 0, 1, 0, 0, 1, 0, 2, 0, 0, 2] dn: ds-mon-domain-name=ou=am-config,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.52.12:8989 ds-mon-connected-to-server-id: ds-idrepo-0 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 2338230 ds-mon-domain-name: ou=am-config ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":13,"total":54.758,"mean_rate":0.006,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.001,"mean":4.211,"min":1.597,"max":10.551,"stddev":2.027,"p50":3.867,"p75":4.489,"p95":10.551,"p98":10.551,"p99":10.551,"p999":10.551,"p9999":10.551,"p99999":10.551} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 2 ds-mon-server-id: ds-idrepo-0 ds-mon-server-state: 010b0186716d6e2200000022ds-idrepo-0 ds-mon-server-state: 010b0186716ee288000000a9ds-idrepo-2 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004357.363Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [3, 1, 1, 0, 2, 0, 0, 1, 0, 0, 1, 0, 2, 0, 0, 2] dn: cn=remote replicas,ds-mon-domain-name=ou=am-config,cn=replicas,cn=replication,cn=monitor objectclass: top objectclass: ds-monitor objectclass: ds-monitor-branch cn: remote replicas dn: ds-mon-server-id=ds-idrepo-2,cn=remote replicas,ds-mon-domain-name=ou=am-config,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-remote-replica ds-mon-domain-name: ou=am-config ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 [...] ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":0,"total":0.000,"mean_rate":0.000,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.000,"mean":0.000,"min":0.000,"max":0.000,"stddev":0.000,"p50":0.000,"p75":0.000,"p95":0.000,"p98":0.000,"p99":0.000,"p999":0.000,"p9999":0.000,"p99999":0.000} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 2000005 ds-mon-server-id: ds-idrepo-0 ds-mon-server-state: 010b0186718f0e38003d2d30ds-idrepo-0 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004357.424Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [3, 1, 1, 0, 2, 0, 0, 1, 0, 0, 1, 0, 2, 0, 0, 2] dn: ds-mon-domain-name=ou=tokens,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.52.12:8989 ds-mon-connected-to-server-id: ds-idrepo-0 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 69125 ds-mon-domain-name: ou=tokens ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":0,"total":0.000,"mean_rate":0.000,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.000,"mean":0.000,"min":0.000,"max":0.000,"stddev":0.000,"p50":0.000,"p75":0.000,"p95":0.000,"p98":0.000,"p99":0.000,"p999":0.000,"p9999":0.000,"p99999":0.000} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 2 ds-mon-server-id: ds-idrepo-0 ds-mon-server-state: 010b0186716d63f700000011ds-idrepo-0 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004357.475Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [3, 1, 1, 0, 2, 0, 0, 1, 0, 0, 1, 0, 2, 0, 0, 2] dn: ds-mon-domain-name=uid=monitor,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.52.12:8989 ds-mon-connected-to-server-id: ds-idrepo-0 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 48 ds-mon-domain-name: uid=monitor ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":0,"total":0.000,"mean_rate":0.000,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.000,"mean":0.000,"min":0.000,"max":0.000,"stddev":0.000,"p50":0.000,"p75":0.000,"p95":0.000,"p98":0.000,"p99":0.000,"p999":0.000,"p9999":0.000,"p99999":0.000} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 0 ds-mon-server-id: ds-idrepo-0 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004357.312Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [3, 1, 1, 0, 2, 0, 0, 1, 0, 0, 1, 0, 2, 0, 0, 2] --- stderr --- ---------------- Pod ds-idrepo-1 ---------------- [run_command]: kubectl --namespace=xlou exec ds-idrepo-1 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "cn=replicas,cn=replication,cn=monitor" "objectclass=*" [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- [print_head_tail]: Print head (80 lines) and tail (80 lines) of input string (230 lines): ----- output ----- dn: cn=replicas,cn=replication,cn=monitor objectclass: top objectclass: ds-monitor objectclass: ds-monitor-branch cn: replicas dn: ds-mon-domain-name=dc=openidm\,dc=forgerock\,dc=io,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.54.12:8989 ds-mon-connected-to-server-id: ds-idrepo-1 ds-mon-current-receive-window: 2145482901 ds-mon-domain-generation-id: 555879 ds-mon-domain-name: dc=openidm,dc=forgerock,dc=io ds-mon-entries-awaiting-updates-count: 1 ds-mon-lost-connections: 0 ds-mon-receive-delay: 197 ds-mon-replay-delay: 1000 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":2000886,"total":937579.475,"mean_rate":910.221,"m1_rate":0.660,"m5_rate":260.888,"m15_rate":659.357,"mean":0.469,"min":0.266,"max":74.449,"stddev":0.377,"p50":0.438,"p75":0.498,"p95":0.618,"p98":0.692,"p99":0.778,"p999":3.326,"p9999":20.972,"p99999":40.108} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 0 ds-mon-server-id: ds-idrepo-1 ds-mon-server-state: 010b0186718f6c16003d2daeds-idrepo-0 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004433.541Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 1 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [250124, 250350, 250934, 251005, 250246, 250342, 250003, 250259, 250432, 248966, 250640, 248902, 250493, 249452, 249074, 249684] dn: cn=remote replicas,ds-mon-domain-name=dc=openidm\,dc=forgerock\,dc=io,cn=replicas,cn=replication,cn=monitor objectclass: top objectclass: ds-monitor objectclass: ds-monitor-branch cn: remote replicas dn: ds-mon-server-id=ds-idrepo-0,cn=remote replicas,ds-mon-domain-name=dc=openidm\,dc=forgerock\,dc=io,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-remote-replica ds-mon-domain-name: dc=openidm,dc=forgerock,dc=io ds-mon-receive-delay: 197 ds-mon-replay-delay: 1000 ds-mon-replayed-updates: {"count":2000886,"total":937579.475,"mean_rate":929.499,"m1_rate":0.658,"m5_rate":260.375,"m15_rate":659.252,"mean":0.469,"min":0.266,"max":74.449,"stddev":0.377,"p50":0.438,"p75":0.498,"p95":0.618,"p98":0.692,"p99":0.778,"p999":3.326,"p9999":20.972,"p99999":40.108} ds-mon-server-id: ds-idrepo-0 dn: ds-mon-domain-name=ou=am-config,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.54.12:8989 ds-mon-connected-to-server-id: ds-idrepo-1 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 2338230 ds-mon-domain-name: ou=am-config ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":15,"total":72.195,"mean_rate":0.007,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.036,"mean":4.810,"min":1.786,"max":16.646,"stddev":3.656,"p50":3.555,"p75":4.653,"p95":16.646,"p98":16.646,"p99":16.646,"p999":16.646,"p9999":16.646,"p99999":16.646} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 0 ds-mon-server-id: ds-idrepo-1 ds-mon-server-state: 010b0186716d6e2200000022ds-idrepo-0 ds-mon-server-state: 010b0186716ee288000000a9ds-idrepo-2 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004433.892Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 [...] ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-updates: {"count":2000005,"total":1845452.591,"mean_rate":909.981,"m1_rate":0.236,"m5_rate":260.138,"m15_rate":658.737,"mean":0.923,"min":0.262,"max":278.921,"stddev":2.725,"p50":0.795,"p75":1.073,"p95":1.753,"p98":2.408,"p99":3.244,"p999":23.724,"p9999":130.023,"p99999":255.853} ds-mon-server-id: ds-idrepo-0 dn: ds-mon-domain-name=ou=tokens,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.54.12:8989 ds-mon-connected-to-server-id: ds-idrepo-1 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 69125 ds-mon-domain-name: ou=tokens ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":2,"total":4.601,"mean_rate":0.001,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.035,"mean":2.294,"min":2.114,"max":2.474,"stddev":0.172,"p50":2.130,"p75":2.474,"p95":2.474,"p98":2.474,"p99":2.474,"p999":2.474,"p9999":2.474,"p99999":2.474} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 0 ds-mon-server-id: ds-idrepo-1 ds-mon-server-state: 010b0186716d63f700000011ds-idrepo-0 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004434.095Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [250124, 250350, 250934, 251005, 250246, 250342, 250003, 250259, 250432, 248966, 250640, 248902, 250493, 249452, 249074, 249684] dn: cn=remote replicas,ds-mon-domain-name=ou=tokens,cn=replicas,cn=replication,cn=monitor objectclass: top objectclass: ds-monitor objectclass: ds-monitor-branch cn: remote replicas dn: ds-mon-server-id=ds-idrepo-0,cn=remote replicas,ds-mon-domain-name=ou=tokens,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-remote-replica ds-mon-domain-name: ou=tokens ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-updates: {"count":2,"total":4.601,"mean_rate":0.001,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.035,"mean":2.294,"min":2.114,"max":2.474,"stddev":0.172,"p50":2.130,"p75":2.474,"p95":2.474,"p98":2.474,"p99":2.474,"p999":2.474,"p9999":2.474,"p99999":2.474} ds-mon-server-id: ds-idrepo-0 dn: ds-mon-domain-name=uid=monitor,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.54.12:8989 ds-mon-connected-to-server-id: ds-idrepo-1 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 48 ds-mon-domain-name: uid=monitor ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":0,"total":0.000,"mean_rate":0.000,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.000,"mean":0.000,"min":0.000,"max":0.000,"stddev":0.000,"p50":0.000,"p75":0.000,"p95":0.000,"p98":0.000,"p99":0.000,"p999":0.000,"p9999":0.000,"p99999":0.000} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 0 ds-mon-server-id: ds-idrepo-1 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004433.803Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [250124, 250350, 250934, 251005, 250246, 250342, 250003, 250259, 250432, 248966, 250640, 248902, 250493, 249452, 249074, 249684] --- stderr --- ---------------- Pod ds-idrepo-2 ---------------- [run_command]: kubectl --namespace=xlou exec ds-idrepo-2 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "cn=replicas,cn=replication,cn=monitor" "objectclass=*" [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- [print_head_tail]: Print head (80 lines) and tail (80 lines) of input string (220 lines): ----- output ----- dn: cn=replicas,cn=replication,cn=monitor objectclass: top objectclass: ds-monitor objectclass: ds-monitor-branch cn: replicas dn: ds-mon-domain-name=dc=openidm\,dc=forgerock\,dc=io,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.53.138:8989 ds-mon-connected-to-server-id: ds-idrepo-2 ds-mon-current-receive-window: 2145482901 ds-mon-domain-generation-id: 555879 ds-mon-domain-name: dc=openidm,dc=forgerock,dc=io ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":2000886,"total":917052.256,"mean_rate":923.370,"m1_rate":0.648,"m5_rate":258.329,"m15_rate":657.197,"mean":0.458,"min":0.264,"max":180.355,"stddev":0.528,"p50":0.426,"p75":0.483,"p95":0.610,"p98":0.692,"p99":0.778,"p999":2.851,"p9999":21.365,"p99999":59.245} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 0 ds-mon-server-id: ds-idrepo-2 ds-mon-server-state: 010b0186718f6ffe003d2db5ds-idrepo-0 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004506.447Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [250121, 250348, 250934, 251005, 250244, 250342, 250002, 250257, 250432, 248966, 250638, 248902, 250491, 249452, 249074, 249682] dn: cn=remote replicas,ds-mon-domain-name=dc=openidm\,dc=forgerock\,dc=io,cn=replicas,cn=replication,cn=monitor objectclass: top objectclass: ds-monitor objectclass: ds-monitor-branch cn: remote replicas dn: ds-mon-server-id=ds-idrepo-0,cn=remote replicas,ds-mon-domain-name=dc=openidm\,dc=forgerock\,dc=io,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-remote-replica ds-mon-domain-name: dc=openidm,dc=forgerock,dc=io ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-updates: {"count":2000886,"total":917052.256,"mean_rate":928.787,"m1_rate":0.658,"m5_rate":260.376,"m15_rate":659.252,"mean":0.458,"min":0.264,"max":180.355,"stddev":0.528,"p50":0.426,"p75":0.483,"p95":0.610,"p98":0.692,"p99":0.778,"p999":2.851,"p9999":21.365,"p99999":59.245} ds-mon-server-id: ds-idrepo-0 dn: ds-mon-domain-name=ou=am-config,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.53.138:8989 ds-mon-connected-to-server-id: ds-idrepo-2 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 2338230 ds-mon-domain-name: ou=am-config ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":2,"total":18.837,"mean_rate":0.001,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.036,"mean":9.413,"min":4.751,"max":14.090,"stddev":4.645,"p50":4.784,"p75":14.090,"p95":14.090,"p98":14.090,"p99":14.090,"p999":14.090,"p9999":14.090,"p99999":14.090} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 13 ds-mon-server-id: ds-idrepo-2 ds-mon-server-state: 010b0186716ee288000000a9ds-idrepo-2 ds-mon-server-state: 010b0186716d6e2200000022ds-idrepo-0 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004506.963Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 [...] ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-updates: {"count":2000005,"total":1902799.628,"mean_rate":923.149,"m1_rate":0.225,"m5_rate":257.537,"m15_rate":656.540,"mean":0.951,"min":0.262,"max":287.310,"stddev":3.284,"p50":0.795,"p75":1.073,"p95":1.794,"p98":2.507,"p99":3.588,"p999":29.360,"p9999":171.966,"p99999":249.561} ds-mon-server-id: ds-idrepo-0 dn: ds-mon-domain-name=ou=tokens,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.53.138:8989 ds-mon-connected-to-server-id: ds-idrepo-2 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 69125 ds-mon-domain-name: ou=tokens ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":2,"total":4.098,"mean_rate":0.001,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.036,"mean":2.050,"min":1.630,"max":2.474,"stddev":0.416,"p50":1.638,"p75":2.474,"p95":2.474,"p98":2.474,"p99":2.474,"p999":2.474,"p9999":2.474,"p99999":2.474} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 0 ds-mon-server-id: ds-idrepo-2 ds-mon-server-state: 010b0186716d63f700000011ds-idrepo-0 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004507.109Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [250121, 250348, 250934, 251005, 250244, 250342, 250002, 250257, 250432, 248966, 250638, 248902, 250491, 249452, 249074, 249682] dn: cn=remote replicas,ds-mon-domain-name=ou=tokens,cn=replicas,cn=replication,cn=monitor objectclass: top objectclass: ds-monitor objectclass: ds-monitor-branch cn: remote replicas dn: ds-mon-server-id=ds-idrepo-0,cn=remote replicas,ds-mon-domain-name=ou=tokens,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-remote-replica ds-mon-domain-name: ou=tokens ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-updates: {"count":2,"total":4.098,"mean_rate":0.001,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.036,"mean":2.050,"min":1.630,"max":2.474,"stddev":0.416,"p50":1.638,"p75":2.474,"p95":2.474,"p98":2.474,"p99":2.474,"p999":2.474,"p9999":2.474,"p99999":2.474} ds-mon-server-id: ds-idrepo-0 dn: ds-mon-domain-name=uid=monitor,cn=replicas,cn=replication,cn=monitor objectClass: top objectClass: ds-monitor objectClass: ds-monitor-replica ds-mon-connected-to-server-hostport: 10.56.53.138:8989 ds-mon-connected-to-server-id: ds-idrepo-2 ds-mon-current-receive-window: 2147483647 ds-mon-domain-generation-id: 48 ds-mon-domain-name: uid=monitor ds-mon-entries-awaiting-updates-count: 0 ds-mon-lost-connections: 0 ds-mon-receive-delay: 0 ds-mon-replay-delay: 0 ds-mon-replayed-internal-updates: 0 ds-mon-replayed-updates: {"count":0,"total":0.000,"mean_rate":0.000,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.000,"mean":0.000,"min":0.000,"max":0.000,"stddev":0.000,"p50":0.000,"p75":0.000,"p95":0.000,"p98":0.000,"p99":0.000,"p999":0.000,"p9999":0.000,"p99999":0.000} ds-mon-replayed-updates-conflicts-resolved: 0 ds-mon-replayed-updates-conflicts-unresolved: 0 ds-mon-sent-updates: 0 ds-mon-server-id: ds-idrepo-2 ds-mon-ssl-encryption: true ds-mon-status: Normal ds-mon-status-last-changed: 20230221004506.751Z ds-mon-total-update-entry-count: 0 ds-mon-total-update-entry-left: 0 ds-mon-updates-already-in-progress: 0 ds-mon-updates-inbound-queue: 0 ds-mon-updates-outbound-queue: 0 ds-mon-updates-totals-per-replay-thread: [250121, 250348, 250934, 251005, 250244, 250342, 250002, 250257, 250432, 248966, 250638, 248902, 250491, 249452, 249074, 249682] --- stderr --- ________________________________________________________________________________ [21/Feb/2023 01:21:13] CheckReplication_ds-idrepo step4 : Temporary step for PERF-1831 : checks logs ________________________________________________________________________________ ---------------- Pod ds-idrepo-0 ---------------- [loop_until]: kubectl --namespace=xlou logs ds-idrepo-0 [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- Initializing "data/changelogDb" from Docker image Initializing "data/import-tmp" from Docker image Upgrading configuration and data... * OpenDJ data has already been upgraded to version 7.3.0.b492db3c2465a45eac7e7cbb8af094f4e03404cb Rebuilding degraded indexes for base DN "ou=tokens"... Rebuilding degraded indexes for base DN "ou=identities"... Rebuilding degraded indexes for base DN "ou=am-config"... Rebuilding degraded indexes for base DN "dc=openidm,dc=forgerock,dc=io"... Server configured with: Group ID : default Server ID : ds-idrepo-0 Advertised listen address : ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local Bootstrap replication server(s) : ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8989,ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8989 {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:55.451Z","category":"CORE","severity":"NOTICE","msgId":134,"message":"ForgeRock Directory Services 7.3.0-20230217104355-b492db3c2465a45eac7e7cbb8af094f4e03404cb (build 20230217105922, revision number b492db3c2465a45eac7e7cbb8af094f4e03404cb) starting up"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:55.451Z","category":"JVM","severity":"NOTICE","msgId":21,"message":"Installation Directory: /opt/opendj"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:55.451Z","category":"JVM","severity":"NOTICE","msgId":23,"message":"Instance Directory: /opt/opendj"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:55.451Z","category":"JVM","severity":"NOTICE","msgId":17,"message":"JVM Information: 17.0.6+10-LTS by Azul Systems, Inc., 64-bit architecture, 11274289152 bytes heap size"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:55.451Z","category":"JVM","severity":"NOTICE","msgId":20,"message":"JVM Host: ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local default/ds-idrepo-0, running Linux 5.10.147+ amd64, unknown physical memory size, number of processors available 16"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:55.451Z","category":"JVM","severity":"NOTICE","msgId":19,"message":"JVM Arguments: \"-XX:+UseG1GC\", \"-XX:MaxRAMPercentage=75\", \"-XX:MaxTenuringThreshold=1\", \"-Djava.security.egd=file:/dev/urandom\", \"-Dorg.opends.server.scriptName=start-ds\""} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:56.052Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend amCts containing 5 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:56.253Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend idmRepo containing 42 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:56.659Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend cfgStore containing 115 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:56.659Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend amIdentityStore containing 21 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.059Z","category":"SYNC","severity":"NOTICE","msgId":204,"message":"Replication server RS(ds-idrepo-0) started listening for new connections on address 0.0.0.0 port 8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.059Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 42 entries and skipped 0 in 0 seconds (average rate 2333.3/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.259Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from directory server DS(ds-idrepo-0) for domain \"dc=openidm,dc=forgerock,dc=io\" at /10.56.52.12:38328"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.259Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-0) has connected to replication server RS(ds-idrepo-0) for domain \"dc=openidm,dc=forgerock,dc=io\" at 10.56.52.12:8989 with generation ID 555879"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.259Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from directory server DS(ds-idrepo-0) for domain \"uid=monitor\" at /10.56.52.12:38356"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.259Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-0) has connected to replication server RS(ds-idrepo-0) for domain \"uid=monitor\" at 10.56.52.12:8989 with generation ID 48"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.259Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 115 entries and skipped 0 in 0 seconds (average rate 7666.7/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.259Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from directory server DS(ds-idrepo-0) for domain \"ou=am-config\" at /10.56.52.12:38366"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.259Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-0) has connected to replication server RS(ds-idrepo-0) for domain \"ou=am-config\" at 10.56.52.12:8989 with generation ID 2338230"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.259Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 21 entries and skipped 0 in 0 seconds (average rate 10500.0/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from directory server DS(ds-idrepo-0) for domain \"ou=identities\" at /10.56.52.12:38392"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-0) has connected to replication server RS(ds-idrepo-0) for domain \"ou=identities\" at 10.56.52.12:8989 with generation ID 253127"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 5 entries and skipped 0 in 0 seconds (average rate 5000.0/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from directory server DS(ds-idrepo-0) for domain \"ou=tokens\" at /10.56.52.12:38404"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-0) has connected to replication server RS(ds-idrepo-0) for domain \"ou=tokens\" at 10.56.52.12:8989 with generation ID 69125"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"CORE","severity":"NOTICE","msgId":818,"message":"Registered 0 static groups, 0 dynamic groups and 0 virtual static groups. The static group cache is using 0 bytes of memory"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on Administration Connector 0.0.0.0:4444"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on LDAP 0.0.0.0:1389"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on HTTP 0.0.0.0:8080"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on HTTPS 0.0.0.0:8443"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on LDAPS 0.0.0.0:1636"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"CORE","severity":"NOTICE","msgId":135,"message":"The Directory Server has started successfully"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:43:57.460Z","category":"CORE","severity":"NOTICE","msgId":139,"message":"The Directory Server has sent an alert notification generated by class org.opends.server.core.DirectoryServer (alert type org.opends.server.DirectoryServerStarted, alert ID org.opends.messages.core-135): The Directory Server has started successfully"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3","timestamp":"2023-02-21T00:43:58.496Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2","client":{"ip":"10.56.50.206","port":37960},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":28,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-25","timestamp":"2023-02-21T00:44:03.985Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-24","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":96,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.695Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from replication server RS(ds-idrepo-1) for domain \"dc=openidm,dc=forgerock,dc=io\" at /10.56.54.12:55038"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.695Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from replication server RS(ds-idrepo-1) for domain \"uid=monitor\" at /10.56.54.12:55060"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.895Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from replication server RS(ds-idrepo-1) for domain \"ou=am-config\" at /10.56.54.12:55076"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-29","timestamp":"2023-02-21T00:44:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-28","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":51,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.895Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from replication server RS(ds-idrepo-1) for domain \"ou=identities\" at /10.56.54.12:55082"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.095Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from replication server RS(ds-idrepo-1) for domain \"ou=tokens\" at /10.56.54.12:55098"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-33","timestamp":"2023-02-21T00:45:02.279Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-32","client":{"ip":"10.56.51.60","port":36636},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-37","timestamp":"2023-02-21T00:45:02.308Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-36","client":{"ip":"10.56.51.60","port":36652},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-41","timestamp":"2023-02-21T00:45:02.322Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-40","client":{"ip":"10.56.51.60","port":36666},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-45","timestamp":"2023-02-21T00:45:02.335Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-44","client":{"ip":"10.56.51.60","port":36676},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-49","timestamp":"2023-02-21T00:45:02.829Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-48","client":{"ip":"10.56.48.181","port":42912},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-53","timestamp":"2023-02-21T00:45:02.857Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-52","client":{"ip":"10.56.48.181","port":42918},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-57","timestamp":"2023-02-21T00:45:02.871Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-56","client":{"ip":"10.56.48.181","port":42926},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-61","timestamp":"2023-02-21T00:45:02.883Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-60","client":{"ip":"10.56.48.181","port":42934},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-65","timestamp":"2023-02-21T00:45:03.320Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-64","client":{"ip":"10.56.49.63","port":36420},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-69","timestamp":"2023-02-21T00:45:03.349Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-68","client":{"ip":"10.56.49.63","port":36432},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-73","timestamp":"2023-02-21T00:45:03.362Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-72","client":{"ip":"10.56.49.63","port":36438},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-77","timestamp":"2023-02-21T00:45:03.374Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-76","client":{"ip":"10.56.49.63","port":36440},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/alive","headers":{"accept":["*/*"],"host":["ds-idrepo-0.ds-idrepo:8080"],"user-agent":["curl/7.74.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-81","timestamp":"2023-02-21T00:45:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-80","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.520Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-0) has accepted a connection from replication server RS(ds-idrepo-2) for domain \"dc=openidm,dc=forgerock,dc=io\" at /10.56.53.138:36184"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.721Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-0) has connected to replication server RS(ds-idrepo-2) for domain \"ou=am-config\" at ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local/10.56.53.138:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.721Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-0) has connected to replication server RS(ds-idrepo-2) for domain \"ou=tokens\" at ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local/10.56.53.138:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.721Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-0) has connected to replication server RS(ds-idrepo-2) for domain \"uid=monitor\" at ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local/10.56.53.138:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.721Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-0) has connected to replication server RS(ds-idrepo-2) for domain \"ou=identities\" at ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local/10.56.53.138:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.921Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-260","timestamp":"2023-02-21T00:45:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-258","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-407","timestamp":"2023-02-21T00:46:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-406","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-554","timestamp":"2023-02-21T00:46:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-553","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:46:50.996Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-1' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-708","timestamp":"2023-02-21T00:47:03.985Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-707","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:47:16.013Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-2' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-867","timestamp":"2023-02-21T00:47:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-866","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1032","timestamp":"2023-02-21T00:48:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1031","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1210","timestamp":"2023-02-21T00:48:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1209","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":36,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1387","timestamp":"2023-02-21T00:49:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1386","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1563","timestamp":"2023-02-21T00:49:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1562","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":47,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1739","timestamp":"2023-02-21T00:50:03.985Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1738","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:50:06.922Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":51,"msgId":4,"dn":"cn=config","scope":"base","filter":"(objectClass=*)","attrs":["1.1"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1876","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.769Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1878","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":51,"msgId":5,"dn":"cn=config","scope":"base","filter":"(objectClass=*)","attrs":["objectclass"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1879","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.772Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1881","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":51,"msgId":6,"dn":"cn=config","scope":"base","filter":"(objectClass=*)","attrs":["ds-cfg-time-limit","ds-cfg-default-password-policy","ds-cfg-etime-resolution","ds-cfg-allowed-task","ds-cfg-invalid-attribute-syntax-behavior","ds-cfg-max-psearches","ds-cfg-group-id","ds-cfg-check-schema","ds-cfg-add-missing-rdn-attributes","ds-cfg-max-candidate-set-size","ds-cfg-max-internal-buffer-size","ds-cfg-restricted-client","ds-cfg-single-structural-objectclass-behavior","ds-cfg-size-limit","ds-cfg-trust-transaction-ids","ds-cfg-proxy-protocol-enabled","ds-cfg-subordinate-base-dn","ds-cfg-denied-client","ds-cfg-bind-with-dn-requires-password","ds-cfg-advertised-listen-address","ds-cfg-allowed-client","ds-cfg-server-id","ds-cfg-max-allowed-client-connections","ds-cfg-listen-address","ds-cfg-writability-mode","ds-cfg-save-config-on-successful-startup","ds-cfg-proxy-protocol-allowed-client","ds-cfg-disabled-privilege","ds-cfg-return-bind-error-messages","ds-cfg-je-backend-shared-cache-enabled","ds-cfg-unauthenticated-requests-policy","ds-cfg-idle-time-limit","ds-cfg-proxied-authorization-identity-mapper","ds-cfg-restricted-client-connection-limit","ds-cfg-group-id-failover-order","ds-cfg-notify-abandoned-operations","ds-cfg-allow-attribute-name-exceptions"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1882","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.777Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1884","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":51,"msgId":8,"dn":"cn=Default Password Policy,cn=Password Policies,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["1.1"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1886","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.797Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1888","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":51,"msgId":9,"dn":"cn=Default Password Policy,cn=Password Policies,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["objectclass"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1889","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.799Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1891","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":51,"msgId":10,"dn":"cn=Default Password Policy,cn=Password Policies,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["ds-cfg-java-class","ds-cfg-min-password-age","ds-cfg-require-change-by-time","ds-cfg-password-validator","ds-cfg-password-generator","ds-cfg-password-history-count","ds-cfg-default-password-storage-scheme","ds-cfg-allow-expired-password-changes","ds-cfg-lockout-duration","ds-cfg-allow-pre-encoded-passwords","ds-cfg-password-change-requires-current-password","ds-cfg-allow-multiple-password-values","ds-cfg-max-password-reset-age","ds-cfg-skip-validation-for-administrators","ds-cfg-last-login-time-attribute","ds-cfg-require-secure-password-changes","ds-cfg-require-secure-authentication","ds-cfg-state-update-failure-policy","ds-cfg-grace-login-count","ds-cfg-lockout-failure-expiration-interval","ds-cfg-password-expiration-warning-interval","ds-cfg-allow-user-password-changes","ds-cfg-force-change-on-reset","ds-cfg-lockout-failure-count","ds-cfg-password-history-duration","ds-cfg-expire-passwords-without-warning","ds-cfg-max-password-age","ds-cfg-last-login-time-format","ds-cfg-previous-last-login-time-format","ds-cfg-deprecated-password-storage-scheme","ds-cfg-account-status-notification-handler","ds-cfg-idle-lockout-interval","ds-cfg-password-attribute","ds-cfg-force-change-on-add"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1892","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.802Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1894","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":51,"msgId":12,"dn":"cn=Exact Match,cn=Identity Mappers,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["1.1"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1896","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.809Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1898","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":51,"msgId":13,"dn":"cn=Exact Match,cn=Identity Mappers,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["objectclass"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1899","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.811Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1901","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":51,"msgId":14,"dn":"cn=Exact Match,cn=Identity Mappers,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["ds-cfg-java-class","ds-cfg-match-base-dn","ds-cfg-match-attribute","ds-cfg-enabled"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1902","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.813Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1904","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":42198},"server":{"ip":"10.56.52.12","port":4444},"request":{"protocol":"LDAPS","operation":"MODIFY","connId":51,"msgId":15,"dn":"cn=config"},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1905","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":36,"elapsedTimeUnits":"MILLISECONDS"},"userId":"uid=admin","timestamp":"2023-02-21T00:50:25.854Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1907","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.52.12","port":49924},"server":{"ip":"10.56.52.12","port":1389},"request":{"protocol":"LDAP","operation":"SEARCH","connId":52,"msgId":3,"dn":"cn=Filtering Criteria,cn=Console LDAP Access Logger,cn=Loggers,cn=config","scope":"sub","filter":"(objectClass=*)","attrs":["ALL"]},"transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1926","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":7},"userId":"uid=admin","timestamp":"2023-02-21T00:50:31.724Z","_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1928","source":"audit","topic":"ldap-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1938","timestamp":"2023-02-21T00:50:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1937","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-2029","timestamp":"2023-02-21T00:51:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2028","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-2153","timestamp":"2023-02-21T00:51:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2152","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":51,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-2265","timestamp":"2023-02-21T00:52:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2264","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":53,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:52:15.999Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-2' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-2394","timestamp":"2023-02-21T00:52:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2393","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":54,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:52:36.013Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-1' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-2508","timestamp":"2023-02-21T00:53:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2507","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":52,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-2620","timestamp":"2023-02-21T00:53:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2619","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":50,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-2732","timestamp":"2023-02-21T00:54:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2731","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":51,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-2844","timestamp":"2023-02-21T00:54:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2843","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":50,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-2956","timestamp":"2023-02-21T00:55:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-2955","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":51,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3069","timestamp":"2023-02-21T00:55:33.985Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-3068","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":53,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3181","timestamp":"2023-02-21T00:56:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-3180","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3294","timestamp":"2023-02-21T00:56:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-3293","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3407","timestamp":"2023-02-21T00:57:03.983Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-3406","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":52,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3523","timestamp":"2023-02-21T00:57:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-3522","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":52,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3636","timestamp":"2023-02-21T00:58:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-3635","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3752","timestamp":"2023-02-21T00:58:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-3751","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3865","timestamp":"2023-02-21T00:59:03.985Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-3864","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":50,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-3978","timestamp":"2023-02-21T00:59:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-3977","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":50,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-4090","timestamp":"2023-02-21T01:00:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-4089","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":53,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-4204","timestamp":"2023-02-21T01:00:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-4203","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-4316","timestamp":"2023-02-21T01:01:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-4315","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-4430","timestamp":"2023-02-21T01:01:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-4429","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-4542","timestamp":"2023-02-21T01:02:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-4541","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-4656","timestamp":"2023-02-21T01:02:33.985Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-4655","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:02:35.925Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-1' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-4768","timestamp":"2023-02-21T01:03:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-4767","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":47,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-4881","timestamp":"2023-02-21T01:03:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-4880","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-4994","timestamp":"2023-02-21T01:04:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-4993","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-5109","timestamp":"2023-02-21T01:04:33.983Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-5108","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":72,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:04:35.008Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-2' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-5222","timestamp":"2023-02-21T01:05:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-5221","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-5336","timestamp":"2023-02-21T01:05:33.983Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-5335","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-5449","timestamp":"2023-02-21T01:06:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-5448","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-5562","timestamp":"2023-02-21T01:06:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-5561","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":54,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-5674","timestamp":"2023-02-21T01:07:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-5673","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-5786","timestamp":"2023-02-21T01:07:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-5785","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":52,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-5898","timestamp":"2023-02-21T01:08:03.983Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-5897","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":47,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-6012","timestamp":"2023-02-21T01:08:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-6011","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":51,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-6126","timestamp":"2023-02-21T01:09:03.985Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-6125","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":52,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-6239","timestamp":"2023-02-21T01:09:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-6238","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":50,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:09:34.928Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-2' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-6351","timestamp":"2023-02-21T01:10:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-6350","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":54,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-6466","timestamp":"2023-02-21T01:10:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-6465","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-6580","timestamp":"2023-02-21T01:11:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-6579","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-6693","timestamp":"2023-02-21T01:11:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-6692","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":53,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-6808","timestamp":"2023-02-21T01:12:03.985Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-6807","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-6926","timestamp":"2023-02-21T01:12:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-6925","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":36,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:12:36.881Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-1' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-7047","timestamp":"2023-02-21T01:13:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-7046","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":36,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-7199","timestamp":"2023-02-21T01:13:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-7198","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-7320","timestamp":"2023-02-21T01:14:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-7319","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-7442","timestamp":"2023-02-21T01:14:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-7441","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-7563","timestamp":"2023-02-21T01:15:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-7562","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-7686","timestamp":"2023-02-21T01:15:33.983Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-7685","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-7807","timestamp":"2023-02-21T01:16:03.983Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-7806","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-7928","timestamp":"2023-02-21T01:16:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-7927","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-8049","timestamp":"2023-02-21T01:17:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-8048","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-8170","timestamp":"2023-02-21T01:17:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-8169","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-8291","timestamp":"2023-02-21T01:18:03.983Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-8290","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":33,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-8412","timestamp":"2023-02-21T01:18:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-8411","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":33,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-8533","timestamp":"2023-02-21T01:19:03.989Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-8532","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":33,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-8654","timestamp":"2023-02-21T01:19:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-8653","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-8775","timestamp":"2023-02-21T01:20:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-8774","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":33,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-8896","timestamp":"2023-02-21T01:20:33.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-8895","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":33,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"29ab12fd-79db-4040-a53f-963a87e7c899-1069532","timestamp":"2023-02-21T01:21:03.984Z","eventName":"DJ-HTTP","transactionId":"29ab12fd-79db-4040-a53f-963a87e7c899-1064670","client":{"ip":"10.56.48.23","port":42428},"server":{"ip":"10.56.52.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.52.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":63,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} --- stderr --- - Looking for string "heart" - Looking for string "publish" ---------------- Pod ds-idrepo-1 ---------------- [loop_until]: kubectl --namespace=xlou logs ds-idrepo-1 [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- Initializing "data/changelogDb" from Docker image Initializing "data/import-tmp" from Docker image Upgrading configuration and data... * OpenDJ data has already been upgraded to version 7.3.0.b492db3c2465a45eac7e7cbb8af094f4e03404cb Rebuilding degraded indexes for base DN "ou=tokens"... Rebuilding degraded indexes for base DN "ou=identities"... Rebuilding degraded indexes for base DN "ou=am-config"... Rebuilding degraded indexes for base DN "dc=openidm,dc=forgerock,dc=io"... Server configured with: Group ID : default Server ID : ds-idrepo-1 Advertised listen address : ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local Bootstrap replication server(s) : ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8989,ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8989 {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:32.029Z","category":"CORE","severity":"NOTICE","msgId":134,"message":"ForgeRock Directory Services 7.3.0-20230217104355-b492db3c2465a45eac7e7cbb8af094f4e03404cb (build 20230217105922, revision number b492db3c2465a45eac7e7cbb8af094f4e03404cb) starting up"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:32.029Z","category":"JVM","severity":"NOTICE","msgId":21,"message":"Installation Directory: /opt/opendj"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:32.029Z","category":"JVM","severity":"NOTICE","msgId":23,"message":"Instance Directory: /opt/opendj"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:32.029Z","category":"JVM","severity":"NOTICE","msgId":17,"message":"JVM Information: 17.0.6+10-LTS by Azul Systems, Inc., 64-bit architecture, 11274289152 bytes heap size"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:32.029Z","category":"JVM","severity":"NOTICE","msgId":20,"message":"JVM Host: ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local default/ds-idrepo-1, running Linux 5.10.147+ amd64, unknown physical memory size, number of processors available 16"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:32.029Z","category":"JVM","severity":"NOTICE","msgId":19,"message":"JVM Arguments: \"-XX:+UseG1GC\", \"-XX:MaxRAMPercentage=75\", \"-XX:MaxTenuringThreshold=1\", \"-Djava.security.egd=file:/dev/urandom\", \"-Dorg.opends.server.scriptName=start-ds\""} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:32.629Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend amCts containing 5 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:32.833Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend idmRepo containing 42 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.034Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend cfgStore containing 115 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.234Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend amIdentityStore containing 21 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.434Z","category":"SYNC","severity":"NOTICE","msgId":204,"message":"Replication server RS(ds-idrepo-1) started listening for new connections on address 0.0.0.0 port 8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.434Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 42 entries and skipped 0 in 0 seconds (average rate 2333.3/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.634Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-0) for domain \"dc=openidm,dc=forgerock,dc=io\" at ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local/10.56.52.12:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.634Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-1) has accepted a connection from directory server DS(ds-idrepo-1) for domain \"dc=openidm,dc=forgerock,dc=io\" at /10.56.54.12:34524"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.634Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-1) for domain \"dc=openidm,dc=forgerock,dc=io\" at 10.56.54.12:8989 with generation ID 555879"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.835Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-0) for domain \"uid=monitor\" at ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local/10.56.52.12:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.835Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-1) has accepted a connection from directory server DS(ds-idrepo-1) for domain \"uid=monitor\" at /10.56.54.12:34540"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.835Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-1) for domain \"uid=monitor\" at 10.56.54.12:8989 with generation ID 48"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.835Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 115 entries and skipped 0 in 0 seconds (average rate 6388.9/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.835Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-0) for domain \"ou=am-config\" at ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local/10.56.52.12:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.835Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-1) has accepted a connection from directory server DS(ds-idrepo-1) for domain \"ou=am-config\" at /10.56.54.12:34550"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.835Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-1) for domain \"ou=am-config\" at 10.56.54.12:8989 with generation ID 2338230"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:33.835Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 21 entries and skipped 0 in 0 seconds (average rate 5250.0/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-0) for domain \"ou=identities\" at ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local/10.56.52.12:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-1) has accepted a connection from directory server DS(ds-idrepo-1) for domain \"ou=identities\" at /10.56.54.12:34574"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-1) for domain \"ou=identities\" at 10.56.54.12:8989 with generation ID 253127"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 5 entries and skipped 0 in 0 seconds (average rate 5000.0/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-0) for domain \"ou=tokens\" at ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local/10.56.52.12:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-1) has accepted a connection from directory server DS(ds-idrepo-1) for domain \"ou=tokens\" at /10.56.54.12:34594"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-1) for domain \"ou=tokens\" at 10.56.54.12:8989 with generation ID 69125"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"CORE","severity":"NOTICE","msgId":818,"message":"Registered 0 static groups, 0 dynamic groups and 0 virtual static groups. The static group cache is using 0 bytes of memory"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on Administration Connector 0.0.0.0:4444"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.035Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on LDAP 0.0.0.0:1389"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.235Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on HTTP 0.0.0.0:8080"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.235Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on HTTPS 0.0.0.0:8443"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.235Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on LDAPS 0.0.0.0:1636"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.235Z","category":"CORE","severity":"NOTICE","msgId":135,"message":"The Directory Server has started successfully"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:44:34.235Z","category":"CORE","severity":"NOTICE","msgId":139,"message":"The Directory Server has sent an alert notification generated by class org.opends.server.core.DirectoryServer (alert type org.opends.server.DirectoryServerStarted, alert ID org.opends.messages.core-135): The Directory Server has started successfully"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3","timestamp":"2023-02-21T00:44:53.763Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":137,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.664Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-1) has accepted a connection from replication server RS(ds-idrepo-2) for domain \"dc=openidm,dc=forgerock,dc=io\" at /10.56.53.138:33994"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.864Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-2) for domain \"ou=am-config\" at ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local/10.56.53.138:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.864Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-2) for domain \"ou=tokens\" at ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local/10.56.53.138:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.864Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-2) for domain \"uid=monitor\" at ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local/10.56.53.138:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.864Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-1) has connected to replication server RS(ds-idrepo-2) for domain \"ou=identities\" at ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local/10.56.53.138:8989"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-7","timestamp":"2023-02-21T00:45:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-6","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":55,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-63","timestamp":"2023-02-21T00:45:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-62","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-113","timestamp":"2023-02-21T00:46:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-112","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:46:30.323Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-2' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:46:50.335Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-1' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-163","timestamp":"2023-02-21T00:46:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-162","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-231","timestamp":"2023-02-21T00:47:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-230","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-296","timestamp":"2023-02-21T00:47:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-295","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-374","timestamp":"2023-02-21T00:48:23.727Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-373","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":39,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-454","timestamp":"2023-02-21T00:48:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-453","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-534","timestamp":"2023-02-21T00:49:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-533","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-614","timestamp":"2023-02-21T00:49:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-613","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":39,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-694","timestamp":"2023-02-21T00:50:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-693","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":38,"msgId":4,"dn":"cn=config","scope":"base","filter":"(objectClass=*)","attrs":["1.1"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-719","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.842Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-721","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":38,"msgId":5,"dn":"cn=config","scope":"base","filter":"(objectClass=*)","attrs":["objectclass"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-722","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.844Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-724","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":38,"msgId":6,"dn":"cn=config","scope":"base","filter":"(objectClass=*)","attrs":["ds-cfg-time-limit","ds-cfg-default-password-policy","ds-cfg-etime-resolution","ds-cfg-allowed-task","ds-cfg-invalid-attribute-syntax-behavior","ds-cfg-max-psearches","ds-cfg-group-id","ds-cfg-check-schema","ds-cfg-add-missing-rdn-attributes","ds-cfg-max-candidate-set-size","ds-cfg-max-internal-buffer-size","ds-cfg-restricted-client","ds-cfg-single-structural-objectclass-behavior","ds-cfg-size-limit","ds-cfg-trust-transaction-ids","ds-cfg-proxy-protocol-enabled","ds-cfg-subordinate-base-dn","ds-cfg-denied-client","ds-cfg-bind-with-dn-requires-password","ds-cfg-advertised-listen-address","ds-cfg-allowed-client","ds-cfg-server-id","ds-cfg-max-allowed-client-connections","ds-cfg-listen-address","ds-cfg-writability-mode","ds-cfg-save-config-on-successful-startup","ds-cfg-proxy-protocol-allowed-client","ds-cfg-disabled-privilege","ds-cfg-return-bind-error-messages","ds-cfg-je-backend-shared-cache-enabled","ds-cfg-unauthenticated-requests-policy","ds-cfg-idle-time-limit","ds-cfg-proxied-authorization-identity-mapper","ds-cfg-restricted-client-connection-limit","ds-cfg-group-id-failover-order","ds-cfg-notify-abandoned-operations","ds-cfg-allow-attribute-name-exceptions"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-725","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.848Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-727","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":38,"msgId":8,"dn":"cn=Default Password Policy,cn=Password Policies,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["1.1"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-729","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.868Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-731","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":38,"msgId":9,"dn":"cn=Default Password Policy,cn=Password Policies,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["objectclass"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-732","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.870Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-734","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":38,"msgId":10,"dn":"cn=Default Password Policy,cn=Password Policies,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["ds-cfg-java-class","ds-cfg-min-password-age","ds-cfg-require-change-by-time","ds-cfg-password-validator","ds-cfg-password-generator","ds-cfg-password-history-count","ds-cfg-default-password-storage-scheme","ds-cfg-allow-expired-password-changes","ds-cfg-lockout-duration","ds-cfg-allow-pre-encoded-passwords","ds-cfg-password-change-requires-current-password","ds-cfg-allow-multiple-password-values","ds-cfg-max-password-reset-age","ds-cfg-skip-validation-for-administrators","ds-cfg-last-login-time-attribute","ds-cfg-require-secure-password-changes","ds-cfg-require-secure-authentication","ds-cfg-state-update-failure-policy","ds-cfg-grace-login-count","ds-cfg-lockout-failure-expiration-interval","ds-cfg-password-expiration-warning-interval","ds-cfg-allow-user-password-changes","ds-cfg-force-change-on-reset","ds-cfg-lockout-failure-count","ds-cfg-password-history-duration","ds-cfg-expire-passwords-without-warning","ds-cfg-max-password-age","ds-cfg-last-login-time-format","ds-cfg-previous-last-login-time-format","ds-cfg-deprecated-password-storage-scheme","ds-cfg-account-status-notification-handler","ds-cfg-idle-lockout-interval","ds-cfg-password-attribute","ds-cfg-force-change-on-add"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-735","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.873Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-737","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":38,"msgId":12,"dn":"cn=Exact Match,cn=Identity Mappers,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["1.1"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-739","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.880Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-741","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":38,"msgId":13,"dn":"cn=Exact Match,cn=Identity Mappers,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["objectclass"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-742","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.882Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-744","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":38,"msgId":14,"dn":"cn=Exact Match,cn=Identity Mappers,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["ds-cfg-java-class","ds-cfg-match-base-dn","ds-cfg-match-attribute","ds-cfg-enabled"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-745","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.884Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-747","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58210},"server":{"ip":"10.56.54.12","port":4444},"request":{"protocol":"LDAPS","operation":"MODIFY","connId":38,"msgId":15,"dn":"cn=config"},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-748","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"userId":"uid=admin","timestamp":"2023-02-21T00:50:27.923Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-750","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.54.12","port":58816},"server":{"ip":"10.56.54.12","port":1389},"request":{"protocol":"LDAP","operation":"SEARCH","connId":39,"msgId":3,"dn":"cn=Filtering Criteria,cn=Console LDAP Access Logger,cn=Loggers,cn=config","scope":"sub","filter":"(objectClass=*)","attrs":["ALL"]},"transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-757","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":2,"elapsedTimeUnits":"MILLISECONDS","nentries":7},"userId":"uid=admin","timestamp":"2023-02-21T00:50:33.345Z","_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-759","source":"audit","topic":"ldap-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-816","timestamp":"2023-02-21T00:50:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-815","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-896","timestamp":"2023-02-21T00:51:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-895","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-976","timestamp":"2023-02-21T00:51:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-975","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":51,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:52:05.328Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1056","timestamp":"2023-02-21T00:52:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1055","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1136","timestamp":"2023-02-21T00:52:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1135","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":47,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1216","timestamp":"2023-02-21T00:53:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1215","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1296","timestamp":"2023-02-21T00:53:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1295","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1376","timestamp":"2023-02-21T00:54:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1375","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":46,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1456","timestamp":"2023-02-21T00:54:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1455","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1536","timestamp":"2023-02-21T00:55:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1535","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1616","timestamp":"2023-02-21T00:55:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1615","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1696","timestamp":"2023-02-21T00:56:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1695","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1776","timestamp":"2023-02-21T00:56:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1775","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:57:05.357Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1856","timestamp":"2023-02-21T00:57:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1855","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":46,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1936","timestamp":"2023-02-21T00:57:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-1935","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2016","timestamp":"2023-02-21T00:58:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2015","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2096","timestamp":"2023-02-21T00:58:53.727Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2095","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":46,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2176","timestamp":"2023-02-21T00:59:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2175","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:59:46.333Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-2' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2256","timestamp":"2023-02-21T00:59:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2255","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":65,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2336","timestamp":"2023-02-21T01:00:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2335","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2416","timestamp":"2023-02-21T01:00:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2415","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":46,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2496","timestamp":"2023-02-21T01:01:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2495","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2576","timestamp":"2023-02-21T01:01:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2575","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2656","timestamp":"2023-02-21T01:02:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2655","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2736","timestamp":"2023-02-21T01:02:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2735","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":51,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2816","timestamp":"2023-02-21T01:03:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2815","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":51,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2896","timestamp":"2023-02-21T01:03:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2895","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2976","timestamp":"2023-02-21T01:04:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-2975","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3056","timestamp":"2023-02-21T01:04:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3055","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3136","timestamp":"2023-02-21T01:05:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3135","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3217","timestamp":"2023-02-21T01:05:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3216","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3297","timestamp":"2023-02-21T01:06:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3296","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":39,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3377","timestamp":"2023-02-21T01:06:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3376","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:07:05.349Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3457","timestamp":"2023-02-21T01:07:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3456","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3537","timestamp":"2023-02-21T01:07:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3536","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3618","timestamp":"2023-02-21T01:08:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3617","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3698","timestamp":"2023-02-21T01:08:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3697","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3778","timestamp":"2023-02-21T01:09:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3777","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3858","timestamp":"2023-02-21T01:09:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3857","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3938","timestamp":"2023-02-21T01:10:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-3937","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4018","timestamp":"2023-02-21T01:10:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4017","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4098","timestamp":"2023-02-21T01:11:23.727Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4097","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4178","timestamp":"2023-02-21T01:11:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4177","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4258","timestamp":"2023-02-21T01:12:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4257","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4338","timestamp":"2023-02-21T01:12:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4337","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:13:15.335Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-2' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4418","timestamp":"2023-02-21T01:13:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4417","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4498","timestamp":"2023-02-21T01:13:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4497","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4578","timestamp":"2023-02-21T01:14:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4577","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4658","timestamp":"2023-02-21T01:14:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4657","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4738","timestamp":"2023-02-21T01:15:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4737","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4818","timestamp":"2023-02-21T01:15:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4817","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4898","timestamp":"2023-02-21T01:16:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4897","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4978","timestamp":"2023-02-21T01:16:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-4977","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5058","timestamp":"2023-02-21T01:17:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5057","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5138","timestamp":"2023-02-21T01:17:53.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5137","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":33,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:18:06.488Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5218","timestamp":"2023-02-21T01:18:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5217","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5298","timestamp":"2023-02-21T01:18:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5297","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":33,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5378","timestamp":"2023-02-21T01:19:23.726Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5377","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5458","timestamp":"2023-02-21T01:19:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5457","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5538","timestamp":"2023-02-21T01:20:23.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5537","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5622","timestamp":"2023-02-21T01:20:53.725Z","eventName":"DJ-HTTP","transactionId":"4d394fe8-9e31-49ca-b59c-b4eb82180ab7-5621","client":{"ip":"10.56.48.23","port":58584},"server":{"ip":"10.56.54.12","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.54.12:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} --- stderr --- - Looking for string "heart" - Looking for string "publish" ---------------- Pod ds-idrepo-2 ---------------- [loop_until]: kubectl --namespace=xlou logs ds-idrepo-2 [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- Initializing "data/changelogDb" from Docker image Initializing "data/import-tmp" from Docker image Upgrading configuration and data... * OpenDJ data has already been upgraded to version 7.3.0.b492db3c2465a45eac7e7cbb8af094f4e03404cb Rebuilding degraded indexes for base DN "ou=tokens"... Rebuilding degraded indexes for base DN "ou=identities"... Rebuilding degraded indexes for base DN "ou=am-config"... Rebuilding degraded indexes for base DN "dc=openidm,dc=forgerock,dc=io"... Server configured with: Group ID : default Server ID : ds-idrepo-2 Advertised listen address : ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local Bootstrap replication server(s) : ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local:8989,ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local:8989 {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:05.042Z","category":"CORE","severity":"NOTICE","msgId":134,"message":"ForgeRock Directory Services 7.3.0-20230217104355-b492db3c2465a45eac7e7cbb8af094f4e03404cb (build 20230217105922, revision number b492db3c2465a45eac7e7cbb8af094f4e03404cb) starting up"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:05.042Z","category":"JVM","severity":"NOTICE","msgId":21,"message":"Installation Directory: /opt/opendj"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:05.042Z","category":"JVM","severity":"NOTICE","msgId":23,"message":"Instance Directory: /opt/opendj"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:05.042Z","category":"JVM","severity":"NOTICE","msgId":17,"message":"JVM Information: 17.0.6+10-LTS by Azul Systems, Inc., 64-bit architecture, 11274289152 bytes heap size"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:05.042Z","category":"JVM","severity":"NOTICE","msgId":20,"message":"JVM Host: ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local default/ds-idrepo-2, running Linux 5.10.147+ amd64, unknown physical memory size, number of processors available 16"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:05.042Z","category":"JVM","severity":"NOTICE","msgId":19,"message":"JVM Arguments: \"-XX:+UseG1GC\", \"-XX:MaxRAMPercentage=75\", \"-XX:MaxTenuringThreshold=1\", \"-Djava.security.egd=file:/dev/urandom\", \"-Dorg.opends.server.scriptName=start-ds\""} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:05.645Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend amCts containing 5 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:05.645Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend idmRepo containing 42 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.047Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend cfgStore containing 115 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.047Z","category":"BACKEND","severity":"NOTICE","msgId":513,"message":"The database backend amIdentityStore containing 21 entries has started"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.248Z","category":"SYNC","severity":"NOTICE","msgId":204,"message":"Replication server RS(ds-idrepo-2) started listening for new connections on address 0.0.0.0 port 8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.449Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 42 entries and skipped 0 in 0 seconds (average rate 2470.6/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.650Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-2) has connected to replication server RS(ds-idrepo-0) for domain \"dc=openidm,dc=forgerock,dc=io\" at ds-idrepo-0.ds-idrepo.xlou.svc.cluster.local/10.56.52.12:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.650Z","category":"SYNC","severity":"INFORMATION","msgId":205,"message":"Replication server RS(ds-idrepo-2) has connected to replication server RS(ds-idrepo-1) for domain \"dc=openidm,dc=forgerock,dc=io\" at ds-idrepo-1.ds-idrepo.xlou.svc.cluster.local/10.56.54.12:8989"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.650Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from directory server DS(ds-idrepo-2) for domain \"dc=openidm,dc=forgerock,dc=io\" at /10.56.53.138:46300"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.650Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-2) has connected to replication server RS(ds-idrepo-2) for domain \"dc=openidm,dc=forgerock,dc=io\" at 10.56.53.138:8989 with generation ID 555879"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.650Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from replication server RS(ds-idrepo-0) for domain \"ou=am-config\" at /10.56.52.12:41246"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.650Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from replication server RS(ds-idrepo-0) for domain \"ou=tokens\" at /10.56.52.12:41250"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.650Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from replication server RS(ds-idrepo-0) for domain \"uid=monitor\" at /10.56.52.12:41256"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.650Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from replication server RS(ds-idrepo-0) for domain \"ou=identities\" at /10.56.52.12:41264"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from replication server RS(ds-idrepo-1) for domain \"ou=am-config\" at /10.56.54.12:42408"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from replication server RS(ds-idrepo-1) for domain \"ou=tokens\" at /10.56.54.12:42418"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from replication server RS(ds-idrepo-1) for domain \"uid=monitor\" at /10.56.54.12:42420"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"SYNC","severity":"INFORMATION","msgId":206,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from replication server RS(ds-idrepo-1) for domain \"ou=identities\" at /10.56.54.12:42430"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from directory server DS(ds-idrepo-2) for domain \"uid=monitor\" at /10.56.53.138:46310"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-2) has connected to replication server RS(ds-idrepo-2) for domain \"uid=monitor\" at 10.56.53.138:8989 with generation ID 48"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 115 entries and skipped 0 in 0 seconds (average rate 8214.3/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from directory server DS(ds-idrepo-2) for domain \"ou=am-config\" at /10.56.53.138:46322"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-2) has connected to replication server RS(ds-idrepo-2) for domain \"ou=am-config\" at 10.56.53.138:8989 with generation ID 2338230"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:06.851Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 21 entries and skipped 0 in 0 seconds (average rate 10500.0/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from directory server DS(ds-idrepo-2) for domain \"ou=identities\" at /10.56.53.138:46332"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-2) has connected to replication server RS(ds-idrepo-2) for domain \"ou=identities\" at 10.56.53.138:8989 with generation ID 253127"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"BACKEND","severity":"NOTICE","msgId":515,"message":"Exported 5 entries and skipped 0 in 0 seconds (average rate 5000.0/sec)"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"SYNC","severity":"INFORMATION","msgId":207,"message":"Replication server RS(ds-idrepo-2) has accepted a connection from directory server DS(ds-idrepo-2) for domain \"ou=tokens\" at /10.56.53.138:46340"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"SYNC","severity":"NOTICE","msgId":62,"message":"Directory server DS(ds-idrepo-2) has connected to replication server RS(ds-idrepo-2) for domain \"ou=tokens\" at 10.56.53.138:8989 with generation ID 69125"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"CORE","severity":"NOTICE","msgId":818,"message":"Registered 0 static groups, 0 dynamic groups and 0 virtual static groups. The static group cache is using 0 bytes of memory"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on Administration Connector 0.0.0.0:4444"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on LDAP 0.0.0.0:1389"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on HTTP 0.0.0.0:8080"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on HTTPS 0.0.0.0:8443"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.052Z","category":"PROTOCOL","severity":"NOTICE","msgId":276,"message":"Started listening for new connections on LDAPS 0.0.0.0:1636"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.253Z","category":"CORE","severity":"NOTICE","msgId":135,"message":"The Directory Server has started successfully"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:45:07.253Z","category":"CORE","severity":"NOTICE","msgId":139,"message":"The Directory Server has sent an alert notification generated by class org.opends.server.core.DirectoryServer (alert type org.opends.server.DirectoryServerStarted, alert ID org.opends.messages.core-135): The Directory Server has started successfully"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3","timestamp":"2023-02-21T00:45:11.115Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":129,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-235","timestamp":"2023-02-21T00:45:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-234","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":56,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-330","timestamp":"2023-02-21T00:46:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-329","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:46:20.406Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-407","timestamp":"2023-02-21T00:46:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-406","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-500","timestamp":"2023-02-21T00:47:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-499","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-597","timestamp":"2023-02-21T00:47:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-596","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-717","timestamp":"2023-02-21T00:48:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-716","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-835","timestamp":"2023-02-21T00:48:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-834","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":39,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-956","timestamp":"2023-02-21T00:49:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-955","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1081","timestamp":"2023-02-21T00:49:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1080","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1206","timestamp":"2023-02-21T00:50:11.081Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1205","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":102,"msgId":4,"dn":"cn=config","scope":"base","filter":"(objectClass=*)","attrs":["1.1"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1289","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:29.970Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1291","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":102,"msgId":5,"dn":"cn=config","scope":"base","filter":"(objectClass=*)","attrs":["objectclass"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1292","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:29.973Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1294","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":102,"msgId":6,"dn":"cn=config","scope":"base","filter":"(objectClass=*)","attrs":["ds-cfg-time-limit","ds-cfg-default-password-policy","ds-cfg-etime-resolution","ds-cfg-allowed-task","ds-cfg-invalid-attribute-syntax-behavior","ds-cfg-max-psearches","ds-cfg-group-id","ds-cfg-check-schema","ds-cfg-add-missing-rdn-attributes","ds-cfg-max-candidate-set-size","ds-cfg-max-internal-buffer-size","ds-cfg-restricted-client","ds-cfg-single-structural-objectclass-behavior","ds-cfg-size-limit","ds-cfg-trust-transaction-ids","ds-cfg-proxy-protocol-enabled","ds-cfg-subordinate-base-dn","ds-cfg-denied-client","ds-cfg-bind-with-dn-requires-password","ds-cfg-advertised-listen-address","ds-cfg-allowed-client","ds-cfg-server-id","ds-cfg-max-allowed-client-connections","ds-cfg-listen-address","ds-cfg-writability-mode","ds-cfg-save-config-on-successful-startup","ds-cfg-proxy-protocol-allowed-client","ds-cfg-disabled-privilege","ds-cfg-return-bind-error-messages","ds-cfg-je-backend-shared-cache-enabled","ds-cfg-unauthenticated-requests-policy","ds-cfg-idle-time-limit","ds-cfg-proxied-authorization-identity-mapper","ds-cfg-restricted-client-connection-limit","ds-cfg-group-id-failover-order","ds-cfg-notify-abandoned-operations","ds-cfg-allow-attribute-name-exceptions"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1295","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:29.976Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1297","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":102,"msgId":8,"dn":"cn=Default Password Policy,cn=Password Policies,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["1.1"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1299","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:29.995Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1301","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":102,"msgId":9,"dn":"cn=Default Password Policy,cn=Password Policies,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["objectclass"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1302","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:29.997Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1304","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":102,"msgId":10,"dn":"cn=Default Password Policy,cn=Password Policies,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["ds-cfg-java-class","ds-cfg-min-password-age","ds-cfg-require-change-by-time","ds-cfg-password-validator","ds-cfg-password-generator","ds-cfg-password-history-count","ds-cfg-default-password-storage-scheme","ds-cfg-allow-expired-password-changes","ds-cfg-lockout-duration","ds-cfg-allow-pre-encoded-passwords","ds-cfg-password-change-requires-current-password","ds-cfg-allow-multiple-password-values","ds-cfg-max-password-reset-age","ds-cfg-skip-validation-for-administrators","ds-cfg-last-login-time-attribute","ds-cfg-require-secure-password-changes","ds-cfg-require-secure-authentication","ds-cfg-state-update-failure-policy","ds-cfg-grace-login-count","ds-cfg-lockout-failure-expiration-interval","ds-cfg-password-expiration-warning-interval","ds-cfg-allow-user-password-changes","ds-cfg-force-change-on-reset","ds-cfg-lockout-failure-count","ds-cfg-password-history-duration","ds-cfg-expire-passwords-without-warning","ds-cfg-max-password-age","ds-cfg-last-login-time-format","ds-cfg-previous-last-login-time-format","ds-cfg-deprecated-password-storage-scheme","ds-cfg-account-status-notification-handler","ds-cfg-idle-lockout-interval","ds-cfg-password-attribute","ds-cfg-force-change-on-add"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1305","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:29.999Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1307","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":102,"msgId":12,"dn":"cn=Exact Match,cn=Identity Mappers,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["1.1"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1309","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:30.005Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1311","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":102,"msgId":13,"dn":"cn=Exact Match,cn=Identity Mappers,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["objectclass"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1312","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:30.006Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1314","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"SEARCH","connId":102,"msgId":14,"dn":"cn=Exact Match,cn=Identity Mappers,cn=config","scope":"base","filter":"(objectClass=*)","attrs":["ds-cfg-java-class","ds-cfg-match-base-dn","ds-cfg-match-attribute","ds-cfg-enabled"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1315","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":0,"elapsedTimeUnits":"MILLISECONDS","nentries":1},"userId":"uid=admin","timestamp":"2023-02-21T00:50:30.008Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1317","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":46816},"server":{"ip":"10.56.53.138","port":4444},"request":{"protocol":"LDAPS","operation":"MODIFY","connId":102,"msgId":15,"dn":"cn=config"},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1318","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"userId":"uid=admin","timestamp":"2023-02-21T00:50:30.046Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1320","source":"audit","topic":"ldap-access","level":"INFO"} {"eventName":"DJ-LDAP","client":{"ip":"10.56.53.138","port":45220},"server":{"ip":"10.56.53.138","port":1389},"request":{"protocol":"LDAP","operation":"SEARCH","connId":103,"msgId":3,"dn":"cn=Filtering Criteria,cn=Console LDAP Access Logger,cn=Loggers,cn=config","scope":"sub","filter":"(objectClass=*)","attrs":["ALL"]},"transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1334","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","nentries":7},"userId":"uid=admin","timestamp":"2023-02-21T00:50:34.945Z","_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1336","source":"audit","topic":"ldap-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1371","timestamp":"2023-02-21T00:50:41.079Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1370","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1486","timestamp":"2023-02-21T00:51:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1485","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T00:51:20.252Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1626","timestamp":"2023-02-21T00:51:41.081Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1625","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":49,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1740","timestamp":"2023-02-21T00:52:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1739","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":50,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1858","timestamp":"2023-02-21T00:52:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1857","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":51,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1972","timestamp":"2023-02-21T00:53:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-1971","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":53,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2090","timestamp":"2023-02-21T00:53:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2089","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":47,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2204","timestamp":"2023-02-21T00:54:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2203","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":46,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2323","timestamp":"2023-02-21T00:54:41.082Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2322","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":48,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2439","timestamp":"2023-02-21T00:55:11.079Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2438","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":46,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2559","timestamp":"2023-02-21T00:55:41.079Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2558","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":47,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2678","timestamp":"2023-02-21T00:56:11.079Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2677","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2802","timestamp":"2023-02-21T00:56:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2801","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2926","timestamp":"2023-02-21T00:57:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-2925","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3050","timestamp":"2023-02-21T00:57:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3049","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3173","timestamp":"2023-02-21T00:58:11.081Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3172","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3296","timestamp":"2023-02-21T00:58:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3295","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3420","timestamp":"2023-02-21T00:59:11.081Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3419","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":46,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3545","timestamp":"2023-02-21T00:59:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3544","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":55,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3669","timestamp":"2023-02-21T01:00:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3668","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3794","timestamp":"2023-02-21T01:00:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3793","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3918","timestamp":"2023-02-21T01:01:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-3917","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:01:20.299Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4043","timestamp":"2023-02-21T01:01:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4042","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":46,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4168","timestamp":"2023-02-21T01:02:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4167","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4293","timestamp":"2023-02-21T01:02:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4292","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":38,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4418","timestamp":"2023-02-21T01:03:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4417","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":36,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4543","timestamp":"2023-02-21T01:03:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4542","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4668","timestamp":"2023-02-21T01:04:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4667","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":44,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4794","timestamp":"2023-02-21T01:04:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4793","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":39,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4920","timestamp":"2023-02-21T01:05:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-4919","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5045","timestamp":"2023-02-21T01:05:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5044","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":56,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5170","timestamp":"2023-02-21T01:06:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5169","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":67,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5295","timestamp":"2023-02-21T01:06:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5294","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5420","timestamp":"2023-02-21T01:07:11.081Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5419","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":46,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5545","timestamp":"2023-02-21T01:07:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5544","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5670","timestamp":"2023-02-21T01:08:11.082Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5669","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5795","timestamp":"2023-02-21T01:08:41.084Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5794","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":43,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5920","timestamp":"2023-02-21T01:09:11.079Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-5919","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":45,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6045","timestamp":"2023-02-21T01:09:41.081Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6044","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":40,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6170","timestamp":"2023-02-21T01:10:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6169","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":41,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6295","timestamp":"2023-02-21T01:10:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6294","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:10:42.424Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-2' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6420","timestamp":"2023-02-21T01:11:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6419","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":37,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:11:20.241Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-0' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6545","timestamp":"2023-02-21T01:11:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6544","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":39,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6670","timestamp":"2023-02-21T01:12:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6669","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":42,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6795","timestamp":"2023-02-21T01:12:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6794","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6920","timestamp":"2023-02-21T01:13:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-6919","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7045","timestamp":"2023-02-21T01:13:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7044","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7170","timestamp":"2023-02-21T01:14:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7169","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7295","timestamp":"2023-02-21T01:14:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7294","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7420","timestamp":"2023-02-21T01:15:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7419","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7545","timestamp":"2023-02-21T01:15:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7544","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"eventName":"DJ-SERVER","timestamp":"2023-02-21T01:15:45.378Z","category":"SYNC","severity":"WARNING","msgId":383,"message":"The change number indexing cannot advance because replica 'ds-idrepo-2' has not sent any messages for a while. Verify whether the replica is still running or is connected to a replication server"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7670","timestamp":"2023-02-21T01:16:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7669","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7795","timestamp":"2023-02-21T01:16:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7794","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7920","timestamp":"2023-02-21T01:17:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-7919","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8045","timestamp":"2023-02-21T01:17:41.079Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8044","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8170","timestamp":"2023-02-21T01:18:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8169","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8295","timestamp":"2023-02-21T01:18:41.079Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8294","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8420","timestamp":"2023-02-21T01:19:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8419","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8545","timestamp":"2023-02-21T01:19:41.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8544","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":35,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8670","timestamp":"2023-02-21T01:20:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8669","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":34,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8795","timestamp":"2023-02-21T01:20:41.079Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8794","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":59,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} {"_id":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8932","timestamp":"2023-02-21T01:21:11.080Z","eventName":"DJ-HTTP","transactionId":"cc2fc0da-5e97-4eea-bee5-a8151489b84a-8931","client":{"ip":"10.56.48.23","port":34360},"server":{"ip":"10.56.53.138","port":8080},"http":{"request":{"secure":false,"method":"GET","path":"http://ds-idrepo-2.ds-idrepo.xlou.svc.cluster.local:8080/metrics/prometheus","headers":{"accept":["application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1"],"host":["10.56.53.138:8080"],"user-agent":["Prometheus/2.41.0"]}}},"response":{"status":"SUCCESSFUL","statusCode":"200","elapsedTime":36,"elapsedTimeUnits":"MILLISECONDS"},"source":"audit","topic":"http-access","level":"INFO"} --- stderr --- - Looking for string "heart" - Looking for string "publish" ________________________________________________________________________________ [21/Feb/2023 01:21:15] CheckReplication_ds-idrepo step5 : Compare ds-sync-state for suffixes across all replicas ________________________________________________________________________________ [run_command]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b " " -s base "objectclass=*" namingContexts [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: namingContexts: uid=admin namingContexts: ou=tokens namingContexts: uid=monitor namingContexts: dc=openidm,dc=forgerock,dc=io namingContexts: ou=am-config namingContexts: ou=identities namingContexts: uid=proxy --- stderr --- => Using namingContexts: {'uid=proxy', 'ou=identities', 'ou=tokens'} [run_command]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "uid=proxy" -s base "objectclass=*" ds-sync-state [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: uid=proxy --- stderr --- => Skipping suffix "uid=proxy" as "ds-sync-state" not found in stdout [run_command]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "ou=identities" -s base "objectclass=*" ds-sync-state [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ou=identities ds-sync-state: 010b0186718f0e38003d2d30ds-idrepo-0 --- stderr --- [run_command]: kubectl --namespace=xlou exec ds-idrepo-1 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "ou=identities" -s base "objectclass=*" ds-sync-state [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ou=identities ds-sync-state: 010b0186718f0e38003d2d30ds-idrepo-0 --- stderr --- [run_command]: kubectl --namespace=xlou exec ds-idrepo-2 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "ou=identities" -s base "objectclass=*" ds-sync-state [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ou=identities ds-sync-state: 010b0186718f0e38003d2d30ds-idrepo-0 --- stderr --- [run_command]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "ou=tokens" -s base "objectclass=*" ds-sync-state [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ou=tokens ds-sync-state: 010b0186716d63f700000011ds-idrepo-0 --- stderr --- [run_command]: kubectl --namespace=xlou exec ds-idrepo-1 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "ou=tokens" -s base "objectclass=*" ds-sync-state [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ou=tokens ds-sync-state: 010b0186716d63f700000011ds-idrepo-0 --- stderr --- [run_command]: kubectl --namespace=xlou exec ds-idrepo-2 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "ou=tokens" -s base "objectclass=*" ds-sync-state [run_command]: OK (rc = 0 - expected to be in [0]) --- stdout --- dn: ou=tokens ds-sync-state: 010b0186716d63f700000011ds-idrepo-0 --- stderr --- => loop 1 out of 30 : repl_state = {'ou=identities': {'ds-idrepo-0': ['010b0186718f0e38003d2d30ds-idrepo-0'], 'ds-idrepo-1': ['010b0186718f0e38003d2d30ds-idrepo-0'], 'ds-idrepo-2': ['010b0186718f0e38003d2d30ds-idrepo-0']}, 'ou=tokens': {'ds-idrepo-0': ['010b0186716d63f700000011ds-idrepo-0'], 'ds-idrepo-1': ['010b0186716d63f700000011ds-idrepo-0'], 'ds-idrepo-2': ['010b0186716d63f700000011ds-idrepo-0']}} => ds-sync-state match for "ou=identities" => ds-sync-state match for "ou=tokens" => {'ou=tokens', 'uid=proxy', 'ou=identities'} expunged. set() remain SUCCESS: replication seems to be synchronised ________________________________________________________________________________ [21/Feb/2023 01:21:31] CheckReplication_ds-idrepo step6 : Check replication is in sync by adding once and searching all ________________________________________________________________________________ [loop_until]: kubectl --namespace=xlou cp /tmp/tmpwr8udqkp -c ds ds-idrepo-0:/tmp/entries.ldif [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- --- stderr --- [loop_until]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapmodify --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -f /tmp/entries.ldif [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- # ADD operation successful for DN uid=pyrock_bufje,ou=identities --- stderr --- => Added test entry with dn "uid=pyrock_bufje,ou=identities" => loop 1 out of 5 to check "3" pods for presence of test entry "uid=pyrock_bufje,ou=identities" [run_command]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "uid=pyrock_bufje,ou=identities" -s base objectclass=* 1.1 [run_command]: OK (rc = 0 - expected to be in [0, 32]) --- stdout --- dn: uid=pyrock_bufje,ou=identities --- stderr --- => Found test entry "uid=pyrock_bufje,ou=identities" on "ds-idrepo-0" [run_command]: kubectl --namespace=xlou exec ds-idrepo-1 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "uid=pyrock_bufje,ou=identities" -s base objectclass=* 1.1 [run_command]: OK (rc = 0 - expected to be in [0, 32]) --- stdout --- dn: uid=pyrock_bufje,ou=identities --- stderr --- => Found test entry "uid=pyrock_bufje,ou=identities" on "ds-idrepo-1" [run_command]: kubectl --namespace=xlou exec ds-idrepo-2 -c ds -- ldapsearch --noPropertiesFile -p 1389 --useStartTls --trustAll -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" -b "uid=pyrock_bufje,ou=identities" -s base objectclass=* 1.1 [run_command]: OK (rc = 0 - expected to be in [0, 32]) --- stdout --- dn: uid=pyrock_bufje,ou=identities --- stderr --- => Found test entry "uid=pyrock_bufje,ou=identities" on "ds-idrepo-2" [loop_until]: kubectl --namespace=xlou exec ds-idrepo-0 -c ds -- ldapdelete -p 4444 -Z -X -D "uid=admin" -w "GSjqN2wc7hyl16OQj13R3XXWwFTWMC0K" uid=pyrock_bufje,ou=identities [loop_until]: (max_time=180, interval=5, expected_rc=[0] [loop_until]: OK (rc = 0) --- stdout --- # DELETE operation successful for DN uid=pyrock_bufje,ou=identities --- stderr --- => Deleted test entry with dn "uid=pyrock_bufje,ou=identities" SUCCESS: replication seems to be synchronised ________________________________________________________________________________ [21/Feb/2023 01:21:52] CheckReplication_ds-idrepo post : N/A ________________________________________________________________________________ SUCCESS : replication seems to be synchronised Setting result to PASS Task has been successfully stopped