Skip to content

Alternative to #5538: my/routes allowed-route manifest#5556

Merged
ramonsmits merged 1 commit into
add-mypermissions-endpointfrom
spike/my-routes-manifest
Jun 26, 2026
Merged

Alternative to #5538: my/routes allowed-route manifest#5556
ramonsmits merged 1 commit into
add-mypermissions-endpointfrom
spike/my-routes-manifest

Conversation

@ramonsmits

@ramonsmits ramonsmits commented Jun 25, 2026

Copy link
Copy Markdown
Member

Alternative to:

Instead of exposing ServiceControl's internal instance:resource:action permission vocabulary (my/permissions/all + a boolean summary), each instance exposes a single GET /api/my/routes returning the API routes the current token may call, as [{ method, urlTemplate }]. ServicePulse gates its UI on the HTTP routes it already calls, so it never has to learn or hand-maintain the permission grammar — the route is a public contract SP already depends on, so gating on it adds no new coupling.

  • Manifest is generated from the same EndpointDataSource metadata the authorization middleware enforces (no hand-listed routes; can't drift from what's enforced).
  • Per-instance (each process reports only its own routes), per-request resolution, fail-open when auth/RBAC is disabled.
  • Permissions stay internal; an anti-drift guard test asserts every [Authorize(Policy=…)] is a known permission.

Compare #5538 branch vs this branch: add-mypermissions-endpoint...spike/my-routes-manifest

@ramonsmits ramonsmits changed the base branch from add-mypermissions-endpoint to auth June 25, 2026 11:56
@ramonsmits ramonsmits requested a review from dvdstelt June 26, 2026 07:46
@ramonsmits ramonsmits changed the base branch from auth to add-mypermissions-endpoint June 26, 2026 09:13
@ramonsmits ramonsmits marked this pull request as ready for review June 26, 2026 09:13
Alternative to #5538. Instead of leaking internal permission strings to
ServicePulse, expose the concrete set of API routes the caller is allowed to
reach, so the UI gates on a stable route contract rather than coupling to the
permission catalogue.

- Project the ASP.NET EndpointDataSource into a route⇒permission table and
  resolve per-request effective permissions, filtering the manifest to the
  routes the caller can reach.
- Add the my/routes controller (served by every instance), route template
  normalization, and the manifest DTOs.
- Remove the my/permissions endpoint, MeController, PermissionsResponse and
  the root-doc permission fields.
- Add an admin role (read-all + manage config/admin-area resources, no
  message-triage write actions), sitting between reader and writer.
- Acceptance, infrastructure and approval tests for the new surface, plus an
  anti-drift policy guard.
@ramonsmits ramonsmits force-pushed the spike/my-routes-manifest branch from 39c934b to f644e7c Compare June 26, 2026 09:21
@ramonsmits ramonsmits merged commit ecd3e0e into add-mypermissions-endpoint Jun 26, 2026
33 checks passed
@ramonsmits ramonsmits deleted the spike/my-routes-manifest branch June 26, 2026 09:25
WilliamBZA added a commit that referenced this pull request Jul 1, 2026
* Add permissions endpoint

* Add endpoints to show user permissions

* Fix approvals

* Show username instead of ID

* Use IReadOnlySet instead of List

* 🐛 Update HttpApiRoutes approval for GetAllMyPermissions rename

* 🐛 Deserialize permissions response into a concrete HashSet in OIDC tests

The /api/my/permissions/all endpoint serializes PermissionsDescriptor
whose Permissions property is IReadOnlySet<string>. System.Text.Json
cannot deserialize into an interface, so the acceptance tests that read
the response back threw NotSupportedException. Introduce a shared
PermissionsResponse DTO with a concrete HashSet<string> and use it from
both When_my_permissions_are_requested and
When_role_based_authorization_is_disabled. The production API contract
(IReadOnlySet) is unchanged.

* ✨ Replace my/permissions with a my/routes allowed-route manifest (#5556)

Instead of leaking internal permission strings to
ServicePulse, expose the concrete set of API routes the caller is allowed to
reach, so the UI gates on a stable route contract rather than coupling to the
permission catalogue.

- Project the ASP.NET EndpointDataSource into a route⇒permission table and
  resolve per-request effective permissions, filtering the manifest to the
  routes the caller can reach.
- Add the my/routes controller (served by every instance), route template
  normalization, and the manifest DTOs.
- Remove the my/permissions endpoint, MeController, PermissionsResponse and
  the root-doc permission fields.
- Add an admin role (read-all + manage config/admin-area resources, no
  message-triage write actions), sitting between reader and writer.
- Acceptance, infrastructure and approval tests for the new surface, plus an
  anti-drift policy guard.

* Add my/routes to RootController

---------

Co-authored-by: Ramon Smits <ramon.smits@gmail.com>
WilliamBZA added a commit that referenced this pull request Jul 7, 2026
* Add permissions catalog with constants for access control

Add Authorize attribute to all controller APIs

* 1st pass of registering an authorization provider

* ✨ Per-IdP roles claim path via IClaimsTransformation

Adds Authentication.RolesClaim (default "realm_access.roles") and a
RolesClaimsTransformation that normalises per-IdP role claim shapes —
Keycloak's nested realm_access.roles, flat repeated claims (Entra app
roles, Keycloak with a User Realm Role mapper, Cognito groups), and
JSON-array-as-string values — into canonical "roles" claims that
PermissionVerbHandler reads unchanged.

The parsing lives in a pure static RolesClaimExtractor in Infrastructure
so it can be unit-tested without an ASP.NET host; the IClaimsTransformation
implementation in Hosting is a thin wrapper that adds idempotency via a
sentinel claim. Removes the TODO on PermissionVerbHandler that the
transformation now resolves.

* 🐛 Authorize policies require authenticated user

Without RequireAuthenticatedUser() in the policy, an unauthenticated
request reaches PermissionVerbHandler, finds no roles, and fails the
requirement — which ASP.NET classifies as FailedRequirements and turns
into a 403 Forbid. The OIDC acceptance tests expect 401 Challenge
(Should_reject_requests_without_bearer_token, ..._with_invalid/expired
/wrong_audience/wrong_issuer). Adding the auth requirement first ensures
the failure is classified as FailedAuthentication when no valid
principal is present.

* 🐛 Wire authorization in acceptance-test runners

The Primary, Audit, and Monitoring acceptance-test runners called
AddServiceControlAuthentication but not AddServiceControlAuthorization,
while production RunCommand wires both. Without the authorization
provider, ASP.NET cannot resolve the policy names emitted by the
Permissions catalogue and any request to a controller carrying
[Authorize(Policy = Permissions.X)] throws "AuthorizationPolicy named
'...' was not found" — which breaks the test hosts and times out the
audit acceptance tests that exercise authorized endpoints.

* 🐛 Accept-token tests must supply a role-bearing claim

Should_accept_requests_with_valid_bearer_token previously passed in CI
by accident: AddServiceControlAuthorization was missing from the test
runners, so the policy provider could not resolve the [Authorize] policy
name and threw, returning 500 — which the assertion "not 401 and not
403" accepted. With the policy provider now wired up, a valid token
without any role claim correctly fails the permission requirement and
returns 403. The test must therefore supply the canonical "roles" claim
with a value that maps to a role granting the endpoint's permission.

"reader" grants every *:*:view permission, which covers the endpoints
the three acceptance-test variants exercise (error:messages:view,
audit:message:view, monitoring:endpoint:view).

* ✅ Approve RolesClaim in PlatformSampleSettings snapshots

The PlatformSampleSettings approval tests in SC, Audit, and Monitoring
unit-test projects serialise the settings object and diff it against an
approved JSON snapshot. The earlier Authentication.RolesClaim addition
needs the snapshots updated to include the new property.

* Fix rebase issues

* magic number

* 🐛 Always register the permission policy provider

The recent refactor moved Authentication.RoleBasedAuthorizationEnabled
to a separate master switch (default false) and made the entire
authorization registration short-circuit when it was off. That left the
permission policy provider unregistered in every default deployment, so
ASP.NET could not resolve the policy names emitted by [Authorize(Policy
= Permissions.X)] attributes — every annotated endpoint returned 500
with "AuthorizationPolicy named '...' was not found", which is what was
timing out the audit acceptance tests at 90s each.

PermissionPolicyProvider already returns allow-all policies for every
known permission when its oidcEnabled flag is false. Registering it
unconditionally and passing RoleBasedAuthorizationEnabled to that flag
gives the right behaviour in all three combinations: RBAC off →
allow-all (controllers reachable, no permission check), RBAC on →
require auth + the permission requirement evaluated by
PermissionVerbHandler. The handler itself remains gated on
RoleBasedAuthorizationEnabled since it has nothing to evaluate when
RBAC is off.

* 🐛 OIDC enabled tests must enable RBAC explicitly

Authentication.RoleBasedAuthorizationEnabled defaults to false, so
without an explicit opt-in the policy provider returns allow-all and
unauthenticated requests reach the controller — breaking every
Should_reject_requests_* test in the three When_authentication_is_enabled
classes. Add WithRoleBasedAuthorizationEnabled() to the test
configuration helper and call it alongside WithAuthenticationEnabled()
in all three OIDC enabled fixtures.

* ✅ Refresh PlatformSampleSettings snapshots for RBAC settings

Two settings shifted in the OpenIdConnectSettings serialisation:
RolesClaim moved after the ServicePulse block and its default changed
from "realm_access.roles" to "roles", and the new
RoleBasedAuthorizationEnabled property (default false) was added. Sync
the three approved snapshots so the approval tests reflect the current
shape.

* magic number

* Make role based always on, but only filter if auth enabled

* Audit log for authorization decisions (#5520)

* ✨ Audit log for every authorization decision

PermissionVerbHandler now resolves the calling principal's subject id
(sub claim), display name (preferred_username, falling back to name then
sub), and roles, and emits a structured "allow" or "deny" entry through
the new IAuthorizationAuditLog for every verb-level check. Both
outcomes are captured — denies alone are insufficient for most
compliance use cases — and the reason embeds the matched role(s) for
allow and the candidate role(s) for deny.

AuthorizationAuditLog writes on the stable category "ServiceControl.Audit"
via a source-generated structured log method so any ILogger-compatible
sink (Seq, OTLP, file, in-memory test double, …) can collect or filter
the trail without coupling to the concrete type name. The audit log is
registered alongside the verb handler — only when OIDC is enabled and
the handler has decisions to make.

Unauthenticated requests are skipped at the top of HandleRequirementAsync
so the audit log only records identified principals; the framework
challenges with 401 via the policy's RequireAuthenticatedUser anyway.

Ported from the keycloak-rbac-poc spike (with the namespace flattened
from Infrastructure.Auth.Rbac to Infrastructure.Auth to match the real
branch) along with a RecordingLoggerProvider test helper colocated with
the unit tests.

* ✨ Configurable required subject claims for the audit log

PermissionVerbHandler now reads the subject id and subject name from
configurable claim keys (Authentication.SubjectIdClaim, default "sub";
Authentication.SubjectNameClaim, default "preferred_username") and
throws InvalidOperationException when an authenticated principal lacks
either claim or carries an empty value — both are required for the
audit log to be meaningful and a missing value indicates an IdP
misconfiguration the operator needs to fix.

The settings are passed through AddServiceControlAuthorization, which
now takes the full OpenIdConnectSettings (the existing bool-only
overload is removed; the six callers — three RunCommand entry points
and three acceptance-test runners — pass the settings object). The
MockOidcServer test helper defaults preferred_username to the subject
value so existing OIDC acceptance tests don't have to repeat the
boilerplate.

* Separate allow/deny log templates, log deny as Warning

* Add TODO for instance-specific audit categories in AuthorizationAuditLog

* IDE0055

* Refactor dependencies on settings to inject `OpenIdConnectSettings`

* ✨ Route ServiceControl.Audit category to a structured JSON log target

Add a dedicated NLog target and a final logging rule that emit the authorization
audit trail as structured JSON on the ServiceControl.Audit category, separate
from the plain-text operational log, so it can be shipped to a SIEM without the
two streams polluting each other.

Audit events are captured from Info upward (allow = Information, deny = Warning)
independent of the operational LogLevel, so lowering operational verbosity never
drops entries from the audit trail. The audit rule is registered before the
catch-all operational rules and marked Final so audit events are not duplicated
into the operational log.

Extracts BuildConfiguration from ConfigureNLog, registers the targets explicitly
so the configuration is fully formed before it is installed, and exposes
AuthorizationAuditLog.AuditCategory so the routing is unit-testable against a
single source of truth for the category name. Tests assert the routing structure
and that a real decision renders as one valid JSON object per line.

* 🐛 Align AuthorizationAuditLog tests with the Allow:/Deny: templates

The audit log message templates were changed to a capitalised "Allow:"/"Deny:"
prefix (and deny moved to Warning) when the allow/deny templates were split, but
these tests still asserted the old lowercase "allow"/"deny" substrings and an
Information level for deny — so they failed on CI (Linux-Default, Windows-Default).

Update the assertions to match the current output: "Allow:"/"Deny:" and deny at
Warning level.

* ✨ Emit the authorization audit event as an Elastic Common Schema (ECS) document

AuthorizationAuditLog now serialises each decision as an ECS-shaped JSON document
(@timestamp, event.kind/category/type/action/outcome, user.id/name, and the
app-specific servicecontrol.* namespace) so it ingests into Elastic/Kibana — and
most SIEMs — with no custom mapping. The schema is owned in the domain class; the
NLog audit target now writes the pre-rendered document verbatim (one object per
line) instead of assembling JSON in logging configuration.

Allow/deny is carried by event.type (["allowed"]/["denied"]) and event.outcome
(success/failure); the log level still differs (Information/Warning) so sinks can
alert on denies without parsing the payload. Relaxed JSON escaping keeps the
output readable for log sinks.

Only fields available at the verb-level check are populated today; user.roles,
user.email and resource scope follow as that data reaches the decision point.

* Remove outdated TODO in `AuthorizationAuditLog` comment regarding instance-specific categories

* Apply suggestions from code review

Co-authored-by: WilliamBZA <WilliamBZA@users.noreply.github.com>

* cleanup

---------

Co-authored-by: WilliamBZA <WilliamBZA@users.noreply.github.com>

* Add service.name/version to OTLP log resource (#5536)

* ✨ Attribute exported OTLP logs to the ServiceControl instance

The OTLP log exporter was registered with a bare AddOtlpExporter() and no
OpenTelemetry resource, so exported log records had no service identity and
showed up as "unknown_service" in the backend.

LoggerUtil.Initialize(serviceName, serviceVersion) is now called once at process
startup (in each instance's Program.cs, before any logger is created) and builds a
single OpenTelemetry resource — service.name = instance name, service.version =
instance version, plus an auto-generated service.instance.id. Both the host logging
pipeline and the static bootstrap loggers (CreateStaticLogger) share that resource,
so every exported record — including early startup diagnostics — is attributed to
the instance. service.name matches the value used by the Audit metrics resource, so
logs and metrics correlate.

The resource still uses ResourceBuilder.CreateDefault(), so operators can enrich it
with deployment attributes via OTEL_SERVICE_NAME / OTEL_RESOURCE_ATTRIBUTES.

* Refactor LoggerUtil to auto-resolve service identity at startup

* As resource builder is now created based only on info from entry assembly that can be handled fully private

* 🐛 Fix IDE0055 formatting: remove double space in CreateResourcesBuilder signature

* Expose a per-instance my/routes authorization manifest (#5538)

* Add permissions endpoint

* Add endpoints to show user permissions

* Fix approvals

* Show username instead of ID

* Use IReadOnlySet instead of List

* 🐛 Update HttpApiRoutes approval for GetAllMyPermissions rename

* 🐛 Deserialize permissions response into a concrete HashSet in OIDC tests

The /api/my/permissions/all endpoint serializes PermissionsDescriptor
whose Permissions property is IReadOnlySet<string>. System.Text.Json
cannot deserialize into an interface, so the acceptance tests that read
the response back threw NotSupportedException. Introduce a shared
PermissionsResponse DTO with a concrete HashSet<string> and use it from
both When_my_permissions_are_requested and
When_role_based_authorization_is_disabled. The production API contract
(IReadOnlySet) is unchanged.

* ✨ Replace my/permissions with a my/routes allowed-route manifest (#5556)

Instead of leaking internal permission strings to
ServicePulse, expose the concrete set of API routes the caller is allowed to
reach, so the UI gates on a stable route contract rather than coupling to the
permission catalogue.

- Project the ASP.NET EndpointDataSource into a route⇒permission table and
  resolve per-request effective permissions, filtering the manifest to the
  routes the caller can reach.
- Add the my/routes controller (served by every instance), route template
  normalization, and the manifest DTOs.
- Remove the my/permissions endpoint, MeController, PermissionsResponse and
  the root-doc permission fields.
- Add an admin role (read-all + manage config/admin-area resources, no
  message-triage write actions), sitting between reader and writer.
- Acceptance, infrastructure and approval tests for the new surface, plus an
  anti-drift policy guard.

* Add my/routes to RootController

---------

Co-authored-by: Ramon Smits <ramon.smits@gmail.com>

* ✨ Add AuditUser and message-action enums

* ✨ Add MessageActionAuditLog ECS emitter

* ✨ Add ICurrentUserAccessor for audit identity resolution

* ✨ Add AuditHeaders identity stamp/read seam

* ✨ Register message-action audit services

* ✅ Guard that ServiceControl.Audit.Messages routes to the audit sink

* ✨ Audit group-retry operations

* ✨ Audit group-archive and group-unarchive operations

* ✨ Audit direct retry operations

* ✨ Audit archive, unarchive, and pending-retry operations

* ✅ Strengthen batch-audit test assertions (count/resource)

* ✨ Audit per-message entries for batch id operations

* ✅ File-scoped namespaces and per-message coverage for audit tests

* Replace wildcard role-permission expansion with explicit lists (#5571)

* ♻️ Replace wildcard role-permission expansion with explicit lists

Roles are explicit permission-constant lists built from four additive
groups: Read (16 views), ReadConfiguration (licensing/notifications/
redirects/throughput views), Operate (message triage, housekeeping
deletes, endpoints/connections manage), Configure (licensing/
notifications/redirects/throughput manage + test). Reader = Read +
ReadConfiguration, Writer = Read + Operate, Admin = everything.
All pattern parsing/expansion machinery is removed.

The sets match the include/exclude patterns of #5569 exactly: writer
holds endpoints:manage and connections:manage but has no access to the
licensing/notifications/redirects/throughput areas (not even :view),
so reader is intentionally not a subset of writer.

Guard tests break the build when a new permission constant is not
assigned to a role, enforce reader/writer ⊂ admin, and pin writer's
configuration-area exclusions.

* minimize diff

* Remove unneeded tests that test which permissions certain role have.

* Pin the my/routes manifest to one wire shape across instances (#5573)

* 🐛 Pin my/routes manifest field names to snake_case across instances

The Primary instance serializes JSON snake_case while the Monitoring instance
serializes camelCase, so the same my/routes contract emitted url_template on one
and urlTemplate on the other. A client merging both manifests would silently
drop every entry from the differently-cased instance.

Pin the field names on RouteManifestEntry with [JsonPropertyName] so every host
emits an identical { method, url_template } shape regardless of its global JSON
naming policy.

* ✅ Guard the my/routes manifest snake_case wire contract

The acceptance test deserializes the response into the C# record, which is
casing-agnostic, so it would not catch a regression of the emitted field names.
Add a serialization test asserting RouteManifestEntry emits snake_case even under
a camelCase policy (as the Monitoring instance uses), pinning the contract that
[JsonPropertyName] enforces.

* Include user roles in routes manifest (#5574)

* Include user rols in routes manifest

* Use verify instead

* ✨ Audit each retried/archived message at execution time

Emit a per-message IMessageActionAuditLog.MessageAction for every message
actually retried, archived, or unarchived, attributed to the initiating
user and correlated to the operation entry via an operation id.

- Wire the AuditHeaders stamp/read seam into the async command pipeline;
  persist the initiating user + operation id on RetryBatch and the
  Archive/Unarchive operation documents so bulk/group operations resolved
  by background schedulers (and resumed after restart) stay attributed.
- Emit per-message entries at the execution choke points (RetryProcessor
  staging, MessageArchiver batch loop, and the archive/unarchive/pending
  handlers) instead of at the API, so each message is logged exactly once
  when it is actually acted on.
- Use the ASP.NET Core request id (HttpContext.TraceIdentifier) as the
  operation id and return it as a Request-Id response header on all three
  instances (exposed via CORS) so callers can correlate.

* ♻️ Omit null-valued properties from audit ECS documents

Set JsonIgnoreCondition.WhenWritingNull on both audit logs so unset fields
(resource/count/message) are dropped rather than emitted as null. Idiomatic
ECS, and trims the high-volume per-message stream.

* ⚜️ Use file-scoped namespaces for new audit files

* Audit edit operations

* Update assertions

* Use correct permissions for roles (#5578)

* Remove some unneeded white box tests

* Use File-scoped namespaces

* Trying to fix format

* 🐛 Register IMessageActionAuditLog in all hosts that consume the input queue

The registration lived only in AddServiceControlAuthorization, which is
called by RunCommand alone. The --import-failed-errors host consumes the
same input queue with all handlers registered, so any pending recoverability
command (ArchiveMessage, UnArchiveMessages, EditAndSend, ...) failed handler
activation and was moved to the error queue.

Move the registration to the primary AddServiceControl so every host that
can activate the recoverability handlers has it, and add a startup-mode test
asserting all RecoverabilityComponent handlers can be activated from the
import host's container (via ActivatorUtilities, mirroring NServiceBus).

* 🐛 Keep audit attribution on archive operation documents across batches

ToArchiveOperation/ToUnarchiveOperation rebuilt the operation document from
the in-memory progress state, which does not carry InitiatedById/
InitiatedByName/OperationId, and MessageArchiver stored that stripped copy
over the original after every batch. A restart mid-operation then resumed
with no attribution and silently skipped the per-message audit entries of
all remaining batches — the exact scenario the persisted fields exist for.

The attribution is now passed into the mapping explicitly so no call site
can drop it, and tests snapshot the persisted document after the first
batch (the state a restart resumes from) for both archive and unarchive.

Also fixes an IDE0055 formatting error in EditHandlerAuditTests that broke
the ServiceControl.Persistence.Tests.RavenDB build.

* 🐛 Audit pending retries at staging time instead of intent time

PendingRetriesHandler emitted per-message retry entries when it resolved
the ids and then sent RetryMessagesById without the audit headers. That
recorded messages as retried that might never be staged (e.g. claimed by a
concurrent batch), and severed the attribution chain so the staged batch
carried no OperationId — the execution-time entries every other retry path
emits were never produced.

The handler no longer audits (and no longer needs the audit log); it
re-stamps the audit headers on the follow-up command so the batch carries
the attribution and RetryProcessor emits the per-message entries when the
messages are actually staged, like all other retry paths.

Also corrects the stale comments on RetryBatch.OperationId and
AuditStagedMessages claiming single/explicit-id retries leave OperationId
null — every path threads it; only legacy unstamped commands are null.

* 🐛 Keep the remote instance's Request-Id on forwarded responses

A retry forwarded to a remote instance (instance_id routing) is audited on
the remote under the remote's TraceIdentifier, and YARP copies that
Request-Id back onto the response — but the middleware's OnStarting callback
overwrote it with the local proxy's TraceIdentifier, handing the caller an
operation id that no audit entry on any instance matches. The header is now
set only when absent.

The identical middleware was pasted into all three instances; it now lives
once in ServiceControl.Hosting (UseRequestIdHeader) next to the other
cross-instance pipeline extensions, and the header name constant replaces
the magic string in the three CORS exposed-header lists.

* 🐛 Don't audit group operations that are skipped as already in progress

The group archive/unarchive/retry controllers emitted the success operation
entry before the IsOperationInProgressFor guard, so a request ignored as a
no-op (operation already running, double-click) was still recorded as a
successful operation attributed to the caller — with an operation id that
never gains any per-message entries. The audit entry is now emitted inside
the guard, next to the work it describes.

* 🐛 Record the real outcome on operation audit entries

Every controller wrote the operation entry with the implicit success
outcome before performing the send, so a transport failure returned HTTP
500 with nothing enqueued while the trail already claimed success — and
the failure outcome was dead code no production path could ever emit.

Controllers now run the action through AuditedOperation, which executes
the send and records the entry afterwards with the actual outcome
(failure + rethrow when the send throws). The stamped local SendOptions
ritual repeated at every call site is collapsed into
AuditHeaders.LocalSendOptions.

* 🐛 Audit an edit only after the edited message is dispatched

The per-message edit entry was emitted right after the failed message was
marked resolved but before the edited message was dispatched, so a
dispatch failure (transport down) left a success entry for an edit whose
message never went anywhere — repeated on every redelivery. The entry is
now emitted after the dispatch, so each audit entry corresponds to an
actual dispatch of the edited message.

* 🐛 Carry the batch scope on ArchiveMessage per-message audit entries

A batch archive fans out into one ArchiveMessage command per id, and the
handler hardcoded scope 'single' on the per-message entries while the
operation entry said 'batch' — the only path where per-message scope did
not match the originating operation (unarchive batch/range, group and
retry paths all propagate it). The command now carries the operation's
scope; the default (Single) keeps legacy in-flight commands and the
single-archive endpoint correct.

* 🐛 Deflake AllMessagesInUnArchivedGroupShouldNotExpire

The test archived group B and immediately unarchived it, but
UnarchiveDocumentManager.GetGroupDetails reads ArchivedGroupsViewIndex,
which is async — on a slow runner the index hadn't seen the archive yet,
so the unarchive logged 'No messages to unarchive' and no-opped, message B
expired along with A, and the wait for exactly one remaining message timed
out (observed on Linux-PrimaryRavenPersistence; Windows passed).

Wait for indexing between the archive and the unarchive, like the test
already does after ingestion.

* ⚡️ Skip building ECS audit documents for filtered-out categories

MessageAction/Operation built the full ECS JSON document (timestamp
formatting, anonymous object graph, reflection-based serialization) before
the source-generated log method's internal IsEnabled check — wasted work
for every message of a bulk retry/archive when the operator filters the
high-volume ServiceControl.Audit.Messages category, which the class
explicitly supports. The level check now runs first.

Also shares the single EcsJsonOptions with AuthorizationAuditLog (the PR
already had to retrofit WhenWritingNull onto one copy to keep the two
streams consistent) and replaces the per-entry scope
ToString().ToLowerInvariant() with constant names.

Tests pin the contracts these changes could break: per-outcome level
semantics under a Warning category filter, and the exact lowercase scope
mapping for every MessageActionScope value.

* ⚡️ Export the audit documents as the OTLP record body

The audit entries were logged through source-generated methods with the
ECS JSON as a '{AuditEvent}' template parameter. Over OTLP that exports
the literal '{AuditEvent}' placeholder as the record body with the
document only in an attribute — backends that map body → message (e.g.
Elastic) show the placeholder and need a pipeline rule to lift the JSON.

Both audit logs now log the pre-rendered document as a plain-string
state: the OTLP record carries the JSON exactly once, as the body, with
no attributes. The NLog audit.json line and the ILogger contract
(categories, levels, event ids 1001/1002/2001/2002) are unchanged.
Verified against an OTel collector and against audit.json on disk.

---------

Co-authored-by: williambza <william.brander@gmail.com>
Co-authored-by: WilliamBZA <WilliamBZA@users.noreply.github.com>
Co-authored-by: Dennis van der Stelt <dvdstelt@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants