Admin guide¶
This page is for the deployment administrator. It chooses a topology, enrolls Authority, Proxy, and Connector roles, publishes one resource, proves the user journey, and records recovery ownership. The binary, configuration, state-store endpoint, and any external identity service are supplied by the deployment owner. This page does not name a public download, hosted console, registry, or credential source.
The current supported release is the exact value printed by the supplied gdsgate --version. Pin that value, the target x86_64-unknown-linux-musl, the checksum/signature record, and the SBOM record in the deployment handoff. The examples use authority-1, proxy-1, connector-1, lab-postgres, lab-read, lab-denied, and alice@example.test.
Deployment shapes¶
Choose the smallest shape that proves the next operational boundary. Every shape ends with the same evidence: a client authenticates with the intended TLS trust, one resource is visible and reachable, a nearest denial is recorded, and an authorized operator can verify the audit chain.
The four supported choices are the split lab, minimal production, multi-node without HA, and HA. The split lab uses Client to proxy-1, Proxy to authority-1 for decisions, Proxy to connector-1 for data, and Connector to lab-postgres. The fixture exposes lab-read as the positive row and lab-denied as a visible refusal. Keep lab keys and data isolated from a production state store. The Quickstart owns the full lab transcript. Minimal production uses separate Authority, Proxy, and Connector processes with a persistent state store, private control-plane links, a client-facing Proxy listener, and one Connector per protected zone. Confirm backup/restore, CA rotation, node revocation, policy recovery, and an audit export before adding more nodes.
All-in-one¶
An all-in-one process is a lab or development shape. It is useful for a single-host smoke test and not a production availability boundary. Keep its state and credentials disposable. The nearest negative is running it with a production store or public listener; stop and replace the configuration rather than trying to turn the process into a multi-node cluster in place.
Multi-node¶
For multi-node operation without HA, place one Authority behind a private control route, one or more Proxies at the client edge, and Connectors beside backends. A persistent store is required. There is no Proxy session handover: draining or losing a Proxy ends its sessions, so document the drain procedure.
High availability¶
HA uses multiple Authority instances over a shared PostgreSQL-compatible store, with lease/fencing rules supplied by the deployment. It protects control-plane availability, not an in-flight Proxy session. Prove leader change, policy convergence, audit-chain continuity, and a Connector reconnect before calling the topology ready.
Installation¶
Actor: the deployment administrator on each target host. Prerequisites: the supplied binary, pinned release record, target architecture, service account, network zones, and empty identity directories. Do not fetch an artifact from a URL invented in this guide.
From a release¶
Obtain the binary and its checksum, signature, and SBOM from the deployment owner's release record. Verify all three before placing the binary on a target host:
Expected result: the version, digest, target, and release record agree. The nearest negative is a mismatch or an unsigned snapshot. Quarantine the file, do not execute it, and ask the release owner for a replacement. Cleanup removes only the rejected file and its temporary verification directory.
Configuration templates¶
Render one config per role from the source-backed configuration reference. Keep the state-store address, identity-provider issuer, CA paths, and backend credentials out of a public template. A template should identify ownership, TLS trust, listeners, and a persistent store without carrying a secret:
profile = "deployment-name"
store_url = "PERSISTENT_STORE_FROM_OWNER"
[endpoints]
authority = "AUTHORITY_ENDPOINT_FROM_OWNER"
proxy_public = "PROXY_PUBLIC_ENDPOINT_FROM_OWNER"
[telemetry]
metrics_listen = "METRICS_BIND_FROM_OWNER"
Replace each value from the deployment record before starting. Expected result: gdsgate --config "$ROLE_CONFIG" doctor --json reports no configuration or trust error. The nearest negative is an unknown key, an in-memory store for a non-lab role, or a listener in the wrong zone. Correct the template and rerun strict validation. Cleanup removes only an unstarted temporary render.
Keep role fragments separate when different owners render the deployment. Each fragment below is valid TOML; substitute the deployment value before a daemon starts.
The emergency socket belongs only on the Authority host. Leave it unset on Proxy and Connector hosts. A socket path is a recovery boundary, not a remote administration endpoint.
These limits bound relay fan-out. Confirm the capacity decision with the Proxy owner and test the refusal when a limit is reached.
The node name is an operator correlation label. It is not a role proof or a policy grant; the one-time token and enrolled identity supply those boundaries.
Use this fragment for a least-privilege workstation baseline. transparent
access and system-wide resolver changes require a separate host-capability
review.
[[connector.backends]]
resource = "tools-dev"
kind = "mcp"
addr = "BACKEND_ADDR_FROM_OWNER"
enforce_tool_policy = true
This enables Cedar decisions for each MCP tool call in addition to the static allow-list. Keep the allow-list narrow even when Cedar is enabled.
The seed path is read for first provisioning. Once the store has an active version, use the versioned pull, validate, and push workflow below.
Quorum and break-glass settings need separate owners. Verify both in a lab before relying on them during a lock-out.
Record the cascade with the policy owner. A resource-specific threshold can be narrower than this global and environment setting.
One command for a connector¶
Actor: an operator on the protected backend host. Authority creates a one-time connector token; the token is read from standard input and never placed in an argument:
umask 077
gdsgate --config "$ADMIN_CONFIG" authority create-token \
--role connector --ttl 3600 > "$CONNECTOR_TOKEN"
gdsgate join "$DOMAIN" --role connector --name connector-1 \
--serve lab-postgres=postgres@BACKEND_ADDR \
--output "$CONNECTOR_CONFIG" --state-dir "$CONNECTOR_STATE" \
< "$CONNECTOR_TOKEN"
Expected result: the connector config and transport identity are written and the token is consumed once. Start the Connector only after reviewing those files. Positive proof is authority list-nodes, authority list-offered, and the Connector health result. The nearest negative is a wrong-role or replayed token; enrollment refuses and leaves no active registry row. Recovery is a new token with the intended role. Cleanup removes the temporary token file after the operator confirms enrollment; retain only the node identity and approved config.
As a systemd unit¶
Use the supplied service-manager unit after reviewing its user, config, state directory, capabilities, and log destination. The unit must run the same version and config tested in the foreground:
Expected result: the foreground and service configs render the same role. The nearest negative is a unit that starts with an empty identity or an unapproved capability. Stop it, correct ownership, and rerun doctor; do not add broad privilege. Cleanup stops and disables the unit, then runs the role's normal identity/revocation procedure.
As a container¶
Use an OCI image digest supplied by the release owner, with a read-only root, an explicit non-root user where the role permits it, a writable state volume, and the minimum network zone. Mount the config and identity directories with owner-only permissions. Expected result is the same role-specific health and readiness as a host process. The nearest negative is a container that is healthy but cannot reach the required control route; inspect the network zone and Connector tunnel, not the backend credential. Cleanup removes the container and temporary volume only after node revocation and audit export.
Bringing up a multi-node cluster¶
This is the minimal production path. Run each command on the host named in the step, and keep the resulting IDs in the deployment record.
1. Authority and the state store¶
Actor: the Authority owner. Use a persistent SQLite store for a single Authority lab or minimal deployment. Use a shared PostgreSQL-compatible store for HA. The store must have backups, restricted credentials, encryption in transit, and a tested restore. An in-memory store is a deliberate lab-only negative because enrollment and audit history disappear on restart.
Expected result: Authority starts with its control listener and reports ready. Positive evidence is the health/readiness response plus a successful gdsgate doctor. The nearest negative is a store that opens but is not persistent or not reachable from an HA peer. Stop before enrolling nodes and fix the store ownership. Cleanup removes only the disposable lab store.
2. First-boot bootstrap¶
Run the one direct bootstrap on the Authority host or the approved control machine:
Expected result: cert.pem, key.pem, and ca.pem are written to the owner-only identity directory, or an existing usable identity is renewed. The command does not create a second identity when renewal is possible. The nearest negative is an existing identity that the cluster rejects; investigate the rejection and node registry before bootstrapping again. Positive proof is authority list-nodes showing one admin row and an audit enrollment event. Cleanup removes a failed temporary directory, never the active identity.
3. Bootstrap tokens for joining nodes¶
On the admin host, mint one token per role and route it through a protected file descriptor or standard input:
umask 077
gdsgate --config "$ADMIN_CONFIG" authority create-token --role proxy --ttl 3600 > "$PROXY_TOKEN"
gdsgate --config "$ADMIN_CONFIG" authority create-token --role connector --ttl 3600 > "$CONNECTOR_TOKEN"
Expected result: each stdout value is a one-time token for exactly one role. The nearest negative is presenting a proxy token to a Connector or reusing a spent token; both are refused without role elevation. Recovery is a fresh token with a short TTL. Cleanup deletes token files after consumption and confirms no token was copied into shell history or a process argument.
4. Register the Proxy¶
The Proxy host receives its role-specific config, token through the deployment enrollment mechanism, and transport CA trust. Start the Proxy only after checking that its public and internal listeners are in the intended zones.
Expected result: authority list-nodes contains a Proxy row and the client can complete the first fingerprinted connect. The nearest negative is a Proxy that registers but exposes its internal listener publicly. Stop it, correct the zone, and revoke the row before re-enrollment. Cleanup is node revocation and removal of the temporary token.
5. Register the Connector¶
Use the connector command from One command for a connector with a backend file when the resource needs options that compact --serve cannot express. cockroach is a Connector backend kind; the catalog advertises its PostgreSQL wire resource as postgres.
Expected result: authority list-offered reports the Connector's declaration. The nearest negative is declared=no: the Connector says it serves a resource that Authority has not catalogued. Declare the exact resource metadata before testing a client. Cleanup is to stop the Connector before removing its node or backend identity.
6. Publish the transport CA fingerprint¶
Publish the transport CA fingerprint through the deployment's trusted handoff, not a client-facing page. On the Proxy or Authority host, use the release's fingerprint command and record the output with the config version. On the client host, run Connect to the cluster with that value.
Expected result: the client refuses a changed fingerprint and accepts the intended Proxy. The nearest negative is a mismatch or an interactive prompt answered without comparison. Recovery is to stop and reconcile the two records. Cleanup is retaining the published fingerprint through its CA rotation window.
7. Publish and prove the first resource¶
On the admin host, pair the Connector's offered lab-read backend with the
catalog row and the reviewed minimal policy. On the client host, prove the same
row through a native database client:
gdsgate --config "$ADMIN_CONFIG" authority set-resource lab-read \
--kind postgres --environment dev
gdsgate --config "$ADMIN_CONFIG" authority policy push "$MINIMAL_POLICY"
gdsgate --config "$CLIENT_CONFIG" ls
gdsgate --config "$CLIENT_CONFIG" db proxy lab-read --listen 127.0.0.1:5433 &
DB_PID=$!
trap 'kill "$DB_PID" 2>/dev/null || true' EXIT
psql -h 127.0.0.1 -p 5433 -c 'select count(*) from lab_table;'
Expected result: list-offered names lab-read, the catalog row has kind
postgres, the user can view it, and the fixture query returns 3. Positive
evidence is the matching catalog, policy, connect, query, and close audit
sequence. The nearest negative is lab-denied: keep it visible but refuse its
connection before the backend. If the offered ID, catalog ID, type, or policy
entity differs, correct that pairing instead of broadening policy. Cleanup stops
the client relay; resource removal, node revocation, and backend teardown remain
separate owner actions. The Policy guide owns
the minimal rule, and the User guide owns later
client tasks.
Registration lifecycle and PKI¶
Enrollment issues a transport identity tied to an enrolled-node registry row. The row, leaf fingerprint, expiry, and revocation state are separate from the node's claimed display name. Record both the registry ID and the host owner.
What gets persisted where¶
Authority persists node registry, policy versions, catalog rows, and audit anchors in the configured store. A role persists its private key and leaf in its owner-only state directory. A client persists trust and identity cache in its own directory. Back up encrypted store data and the audit anchor under the store owner's process; never place private keys in the shared backup manifest.
What the cluster tells a registering node¶
The registration answer supplies cluster settings, trust, endpoint roles, and renewal facts. A node's --name or claimed host label is an assertion for operator correlation, not a proof of physical location and not a Cedar grant. Verify the fingerprint, role, and resource list before starting the role.
Where a node reaches the cluster to register¶
Enrollment reaches the designated enrollment listener over the configured TLS route. Runtime renewal uses the node's transport identity over the Authority or Proxy route chosen by the deployment. Do not use a client-facing address for a control-plane certificate unless the topology explicitly publishes that route.
Renewal near expiry¶
Renew before the leaf expires and keep the state directory across restarts. Positive proof is the same registry ID with a new leaf fingerprint and an audit renewal event. The nearest negative is an unknown or revoked fingerprint; renewal must fail closed. Recovery is a fresh role token after the owner confirms whether the old row should be revoked. Cleanup removes superseded temporary keys only after the new identity is verified.
Per-node revocation¶
List the row, revoke it, then check the next renewal and active session:
gdsgate --config "$ADMIN_CONFIG" authority list-nodes
gdsgate --config "$ADMIN_CONFIG" authority revoke-node NODE_ID
Expected result: renewal is refused and short-lived certificates age out. Revocation is idempotent. The nearest negative is assuming revocation instantly erases an already issued leaf; use short TTLs and terminate active sessions for prompt eviction. Delete a long-revoked row only with authority delete-node after the retention rule, never to restore access. Cleanup removes the node's state only after audit and owner approval.
Naming a node¶
Names are constrained ASCII claims used to correlate logs. They do not prove the node's DNS identity, role, or location. Keep the registry ID and fingerprint as the decisive fields. A duplicate or invalid name is refused rather than trimmed.
Reinstalled connectors, and removing a row¶
A reinstall that loses its identity is a new enrollment until the owner proves the old node is retired. Revoke the old row, wait for the retention period, and then delete it. A live row must never be deleted to make a new token work.
What revocation does not do¶
Revocation does not rewrite catalog metadata, erase audit rows, rotate every CA, or guarantee that a previously issued certificate is already expired. Pair it with short TTLs, session termination, and a network-zone check.
Per-workload revocation¶
Provision-token workloads have a registry row and an external revocation key. Use the exact workload ID from authority list-workloads:
gdsgate --config "$ADMIN_CONFIG" authority list-workloads
gdsgate --config "$ADMIN_CONFIG" authority revoke-workload WORKLOAD_ID
Expected result: a token-less renewal is refused and the short-lived current certificate expires. OIDC-bootstrapped stateless workloads do not appear in this registry; revoke their issuer or policy subject and rotate the affected credential. The nearest negative is a missing or unusable revocation key, which must fail closed. Cleanup removes the workload's local key after the retention decision.
CA rotation¶
Rotate one CA kind at a time, publish the new trust, allow the configured double-signing window, and verify every role before retiring the old signer. The command family includes rotate-host-tls-ca, rotate-spiffe-ca, and the other CA-specific Authority actions exposed by this release. Record the command, propagation window, active signer, and retirement decision. The nearest negative is a node that cannot renew against the new trust; stop retirement and restore the overlap. Cleanup removes only retired trust after all dependent clients and connectors have rotated.
Policy lifecycle¶
Policy is deny-by-default and is evaluated by the running Authority. Read the Policy guide before editing; this page owns the deployment flow, not an exhaustive action or schema reference.
Granting policy edit¶
Grant editPolicy to a dedicated group or control-plane identity with a review owner and an emergency recovery path. Verify that the active policy retains the break-glass rule before granting a second editor. The nearest negative is an editor who can read but cannot propose; keep that distinct from invalid Cedar.
Day-to-day flow¶
On the operator host:
gdsgate --config "$ADMIN_CONFIG" authority policy pull --output "$POLICY_FILE"
gdsgate --config "$ADMIN_CONFIG" authority policy validate "$POLICY_FILE"
gdsgate --config "$ADMIN_CONFIG" authority policy push "$POLICY_FILE"
Expected result: strict validation succeeds, the server assigns a new version, and Authority instances hot-reload it. Positive proof is the version in the push response, a policy audit event, and a positive/nearest-negative fixture decision. A stale base or invalid document is rejected while the previous active policy remains. Recovery is a fresh pull, merge, validate, and push; do not overwrite the store by hand. Cleanup archives the reviewed source and removes temporary copies.
Lock-out and recovery¶
Test lock-out in the lab before production. The normal gateway and certificate routes are policy-gated. The emergency route is a local, filesystem-authorized escape for the Authority host and has a deliberately small action set.
The emergency administrative socket¶
Use the emergency route only when the active policy admits neither the normal editor nor the control certificate:
Expected result: a new active version is written and the action is audited as emergency. The nearest negative is a socket permission refusal or a recovery policy that fails strict validation. Fix local ownership or the policy file; never widen the socket to make it reachable remotely. Cleanup closes the socket route and records the operator and incident ID.
The break-glass file¶
Configure the Authority-owned break-glass policy file before an incident. It is read on startup after the stored policy and must retain the break-glass rule. It is not the normal seed path and it must not be a shared writable file. Expected result is a new audited active version only when the file differs from the active version. The nearest negative is a missing file, a malformed policy, or a candidate that removes recovery; startup refuses or leaves the active policy unchanged. Cleanup removes the incident copy after audit retention.
Resource catalog lifecycle¶
The catalog resolves a resource before Cedar evaluates an action. The admin view shows rows that users may not see; gdsgate ls is filtered by view.
gdsgate --config "$ADMIN_CONFIG" authority list-offered
gdsgate --config "$ADMIN_CONFIG" authority set-resource lab-read --kind postgres --environment dev
gdsgate --config "$ADMIN_CONFIG" authority list-resources
Expected result: lab-read is declared, its kind and environment are exact, and list-resources shows the row. The nearest negative is a Connector offer with no catalog row or a client-visible row with no serving Connector. Recovery is to correct one source of truth, then repeat the positive and negative proof.
A registration states the kind of every resource it declares¶
A Connector declares each resource with the kind it serves it as, and the proxy
forwards both. Authority admits an ID only where the declared kind equals the
catalog row's kind (k8s and kubernetes are one kind), no other enrolled node
owns the row, and the declaration states exactly one kind for that ID. A
declaration carrying no kinds at all is refused whole, not narrowed: the proxy
drops the tunnel and no route is built. See
Order within a mixed fleet
for what that means during an upgrade.
The exception is a cluster whose catalog has never held a resource. It admits
what the Connector declares until the first set-resource, and that window does
not reopen when the catalog is later emptied.
authority list-offered shows what each Connector declared, including IDs the
catalog does not hold and declarations that were refused. A row whose kind is
empty came through a proxy that forwards none.
Seeding from [discovery] is an upsert, not a sync¶
Discovery declarations create or update rows. They do not delete rows omitted from a later file. Treat the catalog as an explicit lifecycle: review every set-resource, then remove retired IDs deliberately.
What removing a resource does¶
Removal hides the row from future catalog resolutions and prevents new access. It does not erase historical audit rows or prove that an already open session has ended. Terminate active sessions separately, revoke the node or workload if needed, and confirm the Connector no longer offers the ID:
gdsgate --config "$ADMIN_CONFIG" authority remove-resource lab-denied
gdsgate --config "$ADMIN_CONFIG" authority list-offered
The command is idempotent. Cleanup removes generated client names after the Resident grace period.
The first-provisioning window closes for good¶
The first direct bootstrap exists only before an admin identity is present. Once the identity and store are established, use the normal mTLS or gateway route. A seed file is not a lock-out recovery path and must not be used to replace a versioned active policy.
Just-in-time access: configuring approvers¶
JIT is a policy action and an operational workflow. Configure approvers with a different identity from the requester and record the approval ticket.
Who: Cedar approveRequest¶
Grant approveRequest to a named approver group, scoped by environment or resource tags. Self-approval must be explicitly denied. The nearest negative is an approver who can list but not approve; that is preferable to an unbounded grant.
How many: the [approvals] cascade¶
Set the narrowest resource override first, then environment, then global default. A value of zero delegates to the broader cascade. Record the chosen threshold with the resource owner and test both one-approver and insufficient- approver paths.
Day-to-day flow¶
Requester¶
The requester runs:
gdsgate --config "$CLIENT_CONFIG" request-access lab-read \
--action connect --reason ticket-identifier --ttl 900
gdsgate --config "$CLIENT_CONFIG" requests
Approvers¶
An authorized approver lists the ID, reviews reason, actor, resource, action, and TTL, then runs gdsgate approve REQUEST_ID. Positive proof is the approval event and one successful connection. The nearest negative is an expired or under-approved request, which must not change policy. Cleanup is expiry and session termination.
A worked deployment¶
Use a narrow environment rule and a resource override, then exercise the positive and negative fixture paths before rollout.
Operator-side cascade¶
Keep the operator-side approval settings in the role configuration. Do not put tokens or personal factors in the file.
Per-resource override for the strictest backend¶
Set the highest threshold on the sensitive resource, then verify that a lower threshold at the global level does not override it. Record the reviewer and expiry policy. The nearest negative is an approval that is valid for another resource or action; Cedar must keep it out.
Operational notes¶
An approval grants only the requested action and TTL. It does not grant catalog visibility, a different resource, or permanent group membership. Keep requester, approver, ticket, and resulting session IDs in the audit handoff.
Network zoning¶
Separate client-facing, control-plane, enrollment, and backend routes. The Proxy public listener must not be treated as the Authority store route. The Connector initiates its outbound tunnel; backends remain inside the protected zone.
Administering from outside the control zone¶
Choose one of the explicitly supported admin routes and record it with the command:
auto uses the configured control certificate where present and the signed-in gateway route otherwise. A machine identity and a human identity are not interchangeable in audit.
Through the gateway, as yourself¶
Use the gateway route for a human operator with an identity-provider token. Expected audit principal is the user, and network policy can match the source address. The nearest negative is a missing editPolicy or admin action. Use a narrow JIT grant or ask the owner; do not switch to a machine certificate to hide the denial.
The control-plane certificate, for machines and for an unreachable gateway¶
Use the admin identity directory for a machine control plane or a human who cannot reach the gateway. The certificate subject is the audit principal. Keep its key owner-only and rotate it like any enrolled node. The nearest negative is an expired or revoked certificate. Recovery is renewal over the configured control route or emergency procedure when policy is locked.
The emergency socket, when the policy admits neither¶
Use only on the Authority host and only for lock-out recovery. Filesystem access is the boundary; there is no remote fallback. Record the incident and close the route after recovery.
State store¶
The store contains policy versions, catalog metadata, enrolled identities, approval state, and durable audit metadata. Use a persistent store for every non-lab role, encrypted transport, least-privilege credentials, and a tested backup. HA peers share a supported PostgreSQL-compatible store and lease/fencing configuration. Do not copy a live store while it is being written unless the store owner documents a consistent snapshot.
Positive verification is a restore into an isolated lab and a successful audit chain check. The nearest negative is a restore that lacks its anchor or policy history; stop before serving clients. Cleanup destroys only the isolated restore after retention approval.
Audit export¶
Built-in stderr logs, health/metrics/readiness, optional OTLP traces, append-only audit export, and an external SIEM shipper are separate streams. Configure each with an owner, retention, transport, and redaction rule. Never infer an audit chain from a metrics counter or a process log.
Run the verifier from a separate operator workstation whose configuration opens the audit store directly and names the anchor key:
Expected result: chain plus anchor verification. The nearest negative is a missing anchor or a tampered row, which blocks the claim of intact history. Recovery is restore from a verified store/anchor backup and follow the incident procedure. Cleanup removes temporary export files after delivery confirmation.
Key management¶
A cluster held to pilot or production seals its rows under two keyrings,
control and audit, kept apart because they are kept for different lengths
of time: the audit ring seals the journal's anchors and outlives the decisions
the control ring seals. See
Configuration → [security.integrity]
for what each ring covers and
CLI → gdsgate authority keys
for the full command family this section draws on.
Rotating a keyring¶
A ring is read once, when the role that seals or signs with it starts, so moving it to a new key is three separate steps rather than one command:
gdsgate --config "$AUTHORITY_CONFIG" authority keys rotate control --prepare
# 4c5d6e7f8091a2b3c4d5e6f708192a3b is in the control keyring, prepared. It seals nothing yet.
--prepare mints a key and adds it to the ring as prepared, leaving the
active key exactly as it was. Distribute the updated ring file to every
Authority replica and restart each one; nothing seals or verifies under the
new key until an instance has been restarted onto a ring that holds it. Then:
--activate makes the prepared key the one new rows are sealed under, and
retires the key it replaces. It refuses while any Authority replica that is
still confirming has not published that it holds the prepared key, which is
what gdsgate authority keys status shows under each key as N of M replicas
hold it. A cluster that cannot take a rolling restart across every replica
between the two steps cannot rotate a keyring: there is no way to switch the
active key while some replica is still running on the old ring alone.
The retiring key goes on verifying every row it already sealed. Once nothing names it:
--retire refuses while any row still names the retiring key, printing the
count by table, and refuses while the key seals a recorded
key-loss statement naming some other key, since that
statement is read at every production start-up. A revocation receipt or a
JIT grant expires or is re-issued on its own; there is nothing to force.
--retire never succeeds against the audit keyring. Its anchor rows are
never deleted, so the count --retire waits on never reaches zero; the audit
ring rotates the same first two steps as control, and the key that sealed
older anchors stays in the ring, verifying them, indefinitely.
--kind signing in place of the default --kind hmac rotates the audit
ring's checkpoint-signing key instead of the key rows are sealed under; the
two rotate on their own schedules and --activate only ever stands down the
active key of the kind it is switching.
Under [security].profile = "production", --prepare refuses to mint a key
on this host: a production ring comes from wherever it is held, added to the
file as state = "prepared", and distributed the same way. --activate and
--retire mint nothing under any profile and work the same everywhere.
Rotation writes no row to the audit chain; gdsgate authority keys status and
the replica lines it prints are the record of what happened and when.
Keeping keyrings out of the database backup¶
A keyring holds the material every sealed row in the database is checked
against. A backup of the database next to a copy of the keyring protects
nothing: whoever holds both holds the rows and the key that seals them, and
can write rows that verify. Back up the two keyrings separately from the
database, and separately from each other for the same reason
[security.integrity] keeps
them apart.
Expected result: one block per keyring naming its fingerprints, states, and what currently stands between the source and the material, which today is nothing. Run this before a restore, not after: it opens no store, so it still answers when there is no store left to open. A database restored without the matching keyrings cannot be verified, since every sealed row names the key that sealed it and a key nothing holds cannot answer for its rows; recovery is restoring the keyrings from their own separate backup alongside the store.
The one exception is a signing key's public half, which backup-manifest
prints and which is safe to publish with the dump; see
Verifying the journal without a keyring.
Verifying the journal without a keyring¶
A reviewer who holds no keyring and no anchor key can still check that the
audit chain has not been truncated or rewritten, using only a public key. On
the Authority host, or anywhere the audit keyring is readable:
Publish checkpoint-keys.txt beside the database dump. It names every
signing key the audit ring has ever held, active or retiring, so a
checkpoint keeps verifying under it long after the key that signed it stops
being the active one; the audit keyring never retires a key outright (its
--retire step always refuses), so a later run of this
command is always a superset of an earlier one and nothing a reviewer already
has stops verifying.
The reviewer, working from a restored copy of the database or a read replica, and never from the live cluster's own credentials, runs:
gdsgate --config "$REVIEWER_CONFIG" authority verify-audit --checkpoint-keys checkpoint-keys.txt
# audit chain ok (chain + checkpoints): 12483 rows, 0 anchors, 61 checkpoints
Expected result: exit 0 and a summary naming chain + checkpoints. This
replays the hash chain and checks every signed checkpoint's signature under
the published public keys; it reports zero anchors because the anchor
cross-check needs the ring's HMAC keys, which this reviewer does not hold and
does not need. The nearest negative is a non-zero exit naming a broken link, a
checkpoint that fails to verify, or a serial range no known key answers for;
treat any of them as tampering and escalate, the same as a chain-integrity
failure run from inside the deployment.
When a keyring is lost¶
gdsgate authority keys accept-loss records that one key's material is gone,
so the rows it sealed are read as unverifiable rather than as forged:
gdsgate --config "$AUTHORITY_CONFIG" authority keys accept-loss \
--key-id 3f9c1a2b4d5e6f708192a3b4c5d6e7f8 --accepted-by anna.ivanova
The statement is itself sealed, by the ring's current active key, and
lands on the audit chain. Because of that, the ring named by the lost key's
fingerprint has to still hold an active key of its own to seal the statement
with; a ring with nothing active refuses this command with the same refusal
keys status would show for that ring.
accept-loss refuses outright under [security].profile = "production". The
path to it is stepping the cluster down from that profile first with
gdsgate authority set-profile, which is itself an event on the audit chain,
recording who changed it and from what; declare the loss, then step the
profile back up once the keyring is sound again.
What stops verifying, and what does not, depends on the row: see the table
under keys accept-loss.
A revoked node or workload keeps reading as revoked even under a lost key; an
approval, a step-up grant, or a delegation has to be requested again; an
inventory snapshot or tool pin is republished by its connector. The journal's
anchors are the one row this never resolves: audit_anchor rows are never
deleted, so verify-audit reports the affected stretch of chain as
unverifiable for as long as the journal is kept, rather than failing outright.
Losing an individual key's material, while the rest of the ring and its active
key are intact, is what accept-loss is for. Losing the ring's file itself,
with no active key left anywhere to seal the statement, is not: no command
here reconstructs a keyring from nothing. Recovery in that case is restoring
the keyring from the separate backup described in Keeping keyrings out of the
database backup; a deployment
with no such backup and no surviving copy of the ring cannot declare its old
keys lost through the normal channel, because there is nothing left in that
ring to seal the declaration with.
Hardening checklist¶
Network¶
- Keep Authority store and emergency socket off the public path.
- Allow Proxy to Authority and Connector to Authority routes only where needed.
- Keep backend credentials and listeners inside the protected zone.
- Test private-address refusal on web-egress and direct-backend network denial.
Transport¶
- Pin the transport CA fingerprint in the client handoff.
- Keep enrollment tokens on standard input with short TTLs.
- Rotate host, workload, and onward CAs with an overlap window.
- Revoke nodes and workloads before deleting their state.
Process¶
- Run each role with a dedicated user and owner-only identity directory.
- Use read-only configuration mounts and explicit writable state paths.
- Treat readiness as role-specific: a Connector can be ready before its tunnel has an active Authority connection.
- Review generated files and service capabilities before start.
State¶
- Back up the persistent store and external audit anchor separately.
- Back up the control and audit keyrings separately from the store and from each other; see Key management.
- Test restore, policy lock-out, and node revocation on a schedule.
- Keep policy versions and approval records for the declared retention period.
Policy¶
- Start deny-all, add typed view and connect, then request-aware rules.
- Preserve the break-glass rule in every candidate policy.
- Separate human, workload, and delegated-agent identities.
- Test one allowed lab-read operation and one denied lab-denied operation after every policy or catalog change.