Audit runbook

This page is the canonical audit procedure reference. The audit list is an investigation view. It is not an integrity proof. Integrity comes from the implemented chain verifier and, when configured, the external-anchor verifier. Do not edit or delete audit rows while any procedure below is running.

Audit

Procedure gate

Gate	Contract
Classification	Read-only
Prerequisites	An operator identity permitted to view audit data, or the direct store route and its configured store access; a protected output location.
Owner	Authority operator or incident responder.
Approval	None for listing or verification. Approval is required before changing audit keys, retention, or export destinations.
Backup point	Preserve the store backup, external audit keys, and off-cluster export before any repair or restore.
Command or check	gdsgate --config <authority-config> authority audit list --class admin-action --limit 20 for rows; gdsgate --config <authority-config> authority verify-audit for integrity.
Expected status and exit	Listing exits 0 and prints rows newest first. Verification exits 0 only when the chain and configured anchors verify; any integrity or store error is non-zero.
Stop condition	Stop destructive maintenance on a broken chain, anchor mismatch, corrupt anchor row, missing key, or unexplained gap. Preserve the original store and exports.
Recovery	Move to the off-band verification workstation, compare the last trusted export and checkpoint, and follow the chain-integrity procedure.
Rollback	No rollback for a read. Restore only from an approved backup after incident authority decides which history is authoritative.
Escalation	Escalate chain or anchor failures to security and the Authority owner; escalate an unavailable store to the database owner.

Authority appends defined security events to its hash-chained audit log, including issuance authorization decisions, node registration, administrative operations, and session activity. High-volume per-request paths do not all append an allow row; their documented request events and metrics are separate. Each chain record links to the previous record hash. Grant paths that require a durable decision record persist it before issuing access. Listing, exporting, and verification are separate operations with separate failure boundaries.

Chain integrity vs. the database

Procedure gate

Gate	Contract
Classification	Incident action
Prerequisites	A read-only copy or controlled access to the store, the original external anchor key when configured, and the last trusted export.
Owner	Security incident responder with the Authority owner.
Approval	Incident authorization before restoring, clearing, or altering any store object. Verification itself needs no change approval.
Backup point	Take or preserve an immutable store snapshot before any further write. Preserve the exact key and export files.
Command or check	gdsgate --config <authority-config> authority verify-audit; compare its result with the off-cluster export and anchor checkpoint.
Expected status and exit	Exit 0 means the loaded chain links and configured anchors cross-check. A broken link, truncated or rewritten chain, wrong key, malformed anchor row, or store read error returns non-zero.
Stop condition	Stop immediately on any verification error. A self-consistent database chain is not sufficient evidence if the external anchor or export disagrees.
Recovery	Quarantine the suspect store, verify a read-only backup with the original key, and identify the last trusted serial.
Rollback	Restore the last approved store and external material only after incident approval. Never rewrite rows to make a check pass.
Escalation	Security owns suspected tampering; the database owner owns storage corruption; the Authority owner owns service recovery.

The in-database chain catches a changed row or broken link. It cannot detect an attacker with full write access who deletes the table and creates a new chain from genesis. When [audit].anchor_key_path is configured, Authority seals each committed chain head with an HMAC key held outside the database and stores the checkpoint in audit_anchor. verify-audit first replays the chain, then cross-checks every anchor against the matching serial and key. The exported JSON-lines witness is a separate comparison source, not a replacement for the implemented verifier.

Configuring chain anchoring

Procedure gate

Gate	Contract
Classification	Planned change
Prerequisites	Authority configuration access, a protected directory for a 32-byte HMAC key, a store backup, and an approved key-custody plan.
Owner	Authority security owner.
Approval	Security and change approval before enabling, rotating, or replacing an anchor key.
Backup point	Back up the existing store, key, anchor rows, and export before changing the setting.
Command or check	gdsgate --config <authority-config> authority init-anchor --path <anchor-key-path>; then run gdsgate --config <authority-config> authority verify-audit.
Expected status and exit	init-anchor exits 0 when it creates a new key and refuses to overwrite an existing file. Verification exits 0 only when the configured key agrees with all anchors.
Stop condition	Stop if an existing anchor key is missing, the path changes for an already-anchored store, or verification fails. Do not clear audit_anchor as routine repair.
Recovery	Restore the original key at its configured path or restore the paired store and key backup. A fresh key is valid only for a fresh history after an approved incident decision.
Rollback	Restore the previous [audit] configuration and key path together; never replace a live key in place.
Escalation	Escalate key loss or suspected exposure to security; escalate startup refusal to the Authority owner.

Set [audit].anchor_key_path to the generated file. The other external integrity keys, when enabled, are separate: approval_key_path, revocation_key_path, delegation_key_path, and inventory_key_path. They are not interchangeable with the audit anchor key. Protect the key file and its parent directory using the deployment's filesystem controls. A key is held outside the database; keep an approved custody copy separate from the live Authority host.

Continuous verification

Procedure gate

Gate	Contract
Classification	Planned change
Prerequisites	A configured anchor key, a working alert destination, and an agreed verify_interval_secs value.
Owner	Authority operator and security monitoring owner.
Approval	Change approval before enabling or changing the periodic verifier.
Backup point	Preserve the prior [audit] settings and current store/export backup.
Command or check	Set [audit].verify_interval_secs to the approved positive interval, restart Authority, then watch the audit.tamper target and run authority verify-audit once off-band.
Expected status and exit	The long-running Authority stays active and the background verifier runs at the configured cadence. A startup failure exits non-zero. Any verification failure is logged at audit.tamper, and the off-band command returns non-zero.
Stop condition	Stop maintenance and traffic changes on an audit.tamper error. Do not suppress the target or increase the interval to hide failures.
Recovery	Preserve logs and the store, run off-band verification with the original key, and follow the chain-integrity procedure.
Rollback	Restore the prior interval or set it to 0 only as an approved temporary containment measure while an off-band verifier remains in use.
Escalation	Security receives every integrity failure; the Authority owner handles service configuration.

The implemented periodic verifier calls the same durable chain path as the off-band command. With an anchor key it also verifies the audit_anchor table; without one it verifies only the chain. This page does not promise a metric for chain depth or CA rotation. Use the emitted logs, the verifier exit status, and the health metrics documented in health.

Off-cluster export

Procedure gate

Gate	Contract
Classification	Planned change
Prerequisites	Protected off-cluster storage, an approved SIEM route, a store backup, and access to the external audit key only where verification requires it.
Owner	Authority operator and security logging owner.
Approval	Security approval before enabling an export path or changing its reader set.
Backup point	Preserve the current export, anchor key custody record, and store backup before changing the destination.
Command or check	Set [audit].export_path to the approved append-only JSON-lines path, restart Authority, then parse a copied line with jq -e . < <export-file>.
Expected status and exit	Each committed audit batch and its closing anchor are appended as independently parseable records. jq -e . exits 0 for a valid copied line. An export failure logs a warning but does not undo an already durable chain commit.
Stop condition	Stop export if the destination is writable by untrusted users, contains key material, leaks credentials, or loses ordering. Do not treat an export outage as permission to edit the chain.
Recovery	Protect or replace the destination, retain the gap window, and reconcile it with the store and later export batches.
Rollback	Restore the previous export path or unset it, retaining the store chain as the durable source.
Escalation	Escalate data-loss or disclosure to security and the logging owner; escalate durable grant failures to Authority.

The file lane is best effort. Durable grant decisions fail closed if their audit row cannot be persisted, while ordinary listing and sign-in records remain best effort. Never place key material beside an unencrypted export.

Audit event classes

Procedure gate

Gate	Contract
Classification	Read-only
Prerequisites	A row listing or export and the event-class vocabulary below.
Owner	Incident responder or audit analyst.
Approval	None for filtering. Approval is required before sharing rows outside the incident or audit boundary.
Backup point	Preserve the unfiltered source before filtering.
Command or check	gdsgate --config <authority-config> authority audit list --class <class> --limit <n>.
Expected status and exit	Exit 0 prints matching rows; an invalid class or inaccessible route is non-zero. A class is not an outcome.
Stop condition	Stop analysis if the source is incomplete or the row's version predates a field relied on by the query.
Recovery	Re-run against the complete store/export and account for older row schemas.
Rollback	N/A. Restore the unfiltered source if an analyst overwrote a working copy.
Escalation	Escalate classification ambiguity to the audit owner, not to policy or storage repair.

The source event classes are authentication, authorization, ca-operation, session-activity, admin-action, capability-grant, client-assertion, and resource-integrity:

Class	Examples and boundary
Authentication	auth.signin records a verified identity observation. It carries no token, factor, group list, or source address.
Authorization	policy.decision records the principal, resource, decision, and the Cedar action in cedar_action. A route miss is session data because no policy was consulted.
CA and capability	Node registration, renewal, CA operations, and JIT approvals. A refused renewal records its reason.
Administration	Policy, catalog, session, audit, recording, and CA administration. A list operation is still a read, not a proof of integrity.
Session activity	Open/close metadata and lane records for SSH, database, Kubernetes, MCP, TCP, web, and model sessions. Recording bytes are referenced by a chain-sealed digest.
Client assertion	sandbox.claim records a delegated run's own assertion and does not make it a verified fact.
Resource integrity	MCP descriptor pin observations, including baseline, changed, reverted, repinned, unsealed, ignored, and unreported outcomes.

Credential type: presented

Procedure gate

Gate	Contract
Classification	Read-only
Prerequisites	Session or decision rows and the policy context vocabulary.
Owner	Audit analyst.
Approval	None for analysis; approval is required before exporting identity-bearing rows.
Backup point	Preserve the original rows before adding analyst annotations.
Command or check	List the relevant rows and inspect presented; compare it with the session's signed credential evidence.
Expected status and exit	presented=key means possession of the certificate-bound key; presented=cookie means the browser session cookie path. Absence is not silently converted to key.
Stop condition	Stop if a query treats absence as a credential type or joins rows from before the field existed without version handling.
Recovery	Re-run with the row-version caveat and distinguish no presentation from a legacy row.
Rollback	N/A. Remove only analyst-derived copies, not source rows.
Escalation	Escalate unexplained credential attribution to security and the identity owner.

presented is written on session open and close, inner rows, and policy decisions, including denials. key means possession of the certificate-bound key. cookie means the clientless browser entry bound the session certificate to a cookie digest. A row before the field existed or a path with nothing presented is absent, not key.

Outcome values

Procedure gate

Gate	Contract
Classification	Read-only
Prerequisites	Audit rows with their outcome, action, and error context.
Owner	Audit analyst or incident responder.
Approval	None for interpretation; approval is required before using a row as a customer-facing incident statement.
Backup point	Preserve the original row set.
Command or check	Compare success, failure, allow, and deny with the actor and gateway status fields.
Expected status and exit	success means the gateway action completed; failure means it did not without a policy refusal; allow or deny records a policy decision. HTTP status is a separate field.
Stop condition	Stop if a backend's own 403 is described as a gateway policy deny or if a pre-v0.8.23 connector row is compared without its historical outcome rule.
Recovery	Re-read the action and event code, then include both the gateway outcome and backend status in the report.
Rollback	N/A. Correct an analyst report, not the immutable row.
Escalation	Escalate a policy decision mismatch to policy/security and a backend response mismatch to the connector owner.

An application, apiserver, or model provider returning 403 is a backend failure from the gateway's point of view. A gateway policy denial is deny. Older connector rows may encode some refusals as failure; preserve that historical meaning when querying across versions.

Database observations and gateway decisions

Procedure gate

Gate	Contract
Classification	Incident action
Prerequisites	The session and audit rows, database engine metadata, the gateway classifier version, and a read-only database observation path.
Owner	Database and connector incident owners.
Approval	Incident authorization before changing database permissions, classifier rules, or stored rows.
Backup point	Preserve the relevant audit rows, statement evidence, and database snapshot before remediation.
Command or check	Query the audit rows for db.write_in_read_only_session, db.write_refused, and db.write_misclassified; correlate session_uid, engine, and kind.
Expected status and exit	The query completes without changing the store. The first two outcomes are incident signals; the third is a classifier defect report.
Stop condition	Stop writes or customer traffic when a write passed a read-only gateway session or the server contradicts the gateway category.
Recovery	Preserve the statement and server command tag, contain the session, and reproduce against a controlled database.
Rollback	Roll back the customer-side change or database transaction only under the database owner's procedure. Never rewrite the audit observation.
Escalation	Escalate read-only bypasses to security and database owners; escalate classification defects to the connector owner.

The connector classifies statements before forwarding and also reads the answer stream. It records these distinct observations:

Event	Outcome	Meaning
db.write_in_read_only_session	failure	The session had no write capability but the server reported a write.
db.write_refused	deny	The server refused a write with SQLSTATE 25006; the gateway had forwarded it and the read-only database mode stopped it.
db.write_misclassified	failure	The server reported a write for a statement classified as a read in a write-capable session.

engine identifies the database engine and kind identifies the wire lane. CockroachDB is reached over the PostgreSQL wire, so those values can differ.

Reading the audit chain

Procedure gate

Gate	Contract
Classification	Read-only
Prerequisites	A route permitted to list rows, and the direct store plus anchor key for integrity verification.
Owner	Audit analyst or incident responder.
Approval	None to list or verify; approval is required before sharing output.
Backup point	Preserve the command output and source snapshot.
Command or check	gdsgate --config <authority-config> authority audit list --class admin-action --limit 20 and, separately, gdsgate --config <authority-config> authority verify-audit.
Expected status and exit	audit list prints newest-first rows and exits 0. It does not prove chain or anchor integrity. verify-audit performs that distinct proof and exits non-zero on failure.
Stop condition	Stop using list output as evidence of integrity if verification has not passed.
Recovery	Run the off-band verifier and compare the trusted export.
Rollback	N/A.
Escalation	Escalate verification failures to security and the Authority owner.

The list command opens a routed view or the direct store and prints rows. The implemented proof path is verify_durable_chain: it loads the entire chain, calls the chain verifier, and, when a key is supplied, cross-checks all audit_anchor rows. The periodic verifier calls the same path. There is no separate claim that listing rows proves depth, continuity, or anchor integrity.

Actor attribution

Procedure gate

Gate	Contract
Classification	Read-only
Prerequisites	Rows containing actor, submitter, owner, and optional claimed_operator.
Owner	Audit analyst.
Approval	None for internal analysis; approval is required before treating a claimed name as verified identity.
Backup point	Preserve the original row and export.
Command or check	Compare actor, submitter, and owner with certificate or verified-token evidence; keep claimed_operator separate.
Expected status and exit	The row remains unchanged. actor, submitter, and owner are cluster-derived; claimed_operator is explicitly unverified.
Stop condition	Stop if a claimed name is used as a Cedar principal, request context, or verified actor.
Recovery	Reissue the report with the verified fields and label the claim as unverified.
Rollback	N/A.
Escalation	Escalate identity disagreement to security and the identity provider owner.

actor identifies who performed the action, submitter who delivered the record, and owner on whose behalf a workload acts. claimed_operator is a caller-supplied label accepted only from the control-plane identity and never used as a decision principal, filter, or context field.

Authentication identity

Procedure gate

Gate	Contract
Classification	Read-only
Prerequisites	Authentication event rows and the identity provider's separate audit record when sign-in timing must be proved.
Owner	Identity or audit analyst.
Approval	None for internal analysis; approval is required before exporting identity-bearing rows.
Backup point	Preserve the cluster row and provider event id or time window.
Command or check	Count distinct actor over authentication / success rows for the time window and compare with the provider record.
Expected status and exit	The result is an analysis count, not a service exit. A token or fragment is never present in the cluster row.
Stop condition	Stop if a report treats every decision row as a sign-in or expects the cluster to prove the provider's full authentication ceremony.
Recovery	Use the provider's audit for login ceremony and the cluster row only for the first verified request here.
Rollback	N/A.
Escalation	Escalate identity-provider discrepancies to the provider owner and cluster attribution discrepancies to security.

The cluster records the first request carrying a verified resulting token, at most once per person per day on a running Authority instance. A restart or HA failover can re-arm local deduplication, so distinct actors are safer than row counts for a period. The row contains actor, issuer, write time, and optional auth_time, but no token, factor, groups, email, or source address.