Skip to content

Admin guide

This page is for the deployment administrator: choosing a topology, enrolling Authority, Proxy and Connector, publishing a resource, and recovery. The deployment owner supplies the binary, configuration, state store and identity provider.

The supported release is the version the supplied gdsgate --version prints. gdsgate runs on Linux x86_64 (static musl binary) and Windows x86_64. Record the platform in the deployment handoff:

Platform Artifact Source
Linux gdsgate-<version>-x86_64-unknown-linux-musl, with checksum, signature and SBOM Release publication
Windows gdsgate-<version>-x86_64-pc-windows-msvc.msi Supplied on request; not in the release publication

Examples use placeholder hosts, resources and identities: authority-1, proxy-1, connector-1, lab-postgres, lab-read, lab-denied and alice@example.test.

Deployment shapes

Shape Layout Use Page
Minimal production Separate Authority, Proxy and Connector processes, persistent store, private control links, one Connector per protected zone First production deployment One Linux host, Separate hosts
Multi-node One Authority, several Proxies, Connectors beside backends Scale without HA Separate hosts, Kubernetes
HA Authority Several Authorities on a shared PostgreSQL-compatible store, with individual caller destinations or an L4 LB Control-plane availability Authority HA
HA Proxy Separately enrolled Proxies, private peering and a common L4 entry Availability of new resource connections Proxy HA

Every shape uses the same access checks: client trust, allowed access, policy denial, backend isolation and audit integrity. Before adding nodes to minimal production, test backup and restore, CA rotation, node revocation, policy recovery and an audit export. Keep lab keys and data out of production stores.

All-in-one

The all-in-one process is for development and single-host smoke tests. Keep its state and credentials disposable. Do not run it with a production store or a public listener, and do not grow it into a cluster; deploy the roles separately.

Multi-node

Place one Authority on a private control route, Proxies at the client edge and Connectors beside backends. A persistent store is required. Proxy sessions are not handed over: draining or losing a Proxy ends its sessions. Document the drain procedure.

High availability

Choose Authority HA and Proxy HA independently of the hosting platform. Authorities share a PostgreSQL-compatible store with leader election and fencing; bootstrap admits additional instances and synchronizes keyrings. Proxies use separate identities and private peering behind their public entry. An Authority LB is optional.

Use the composition guide to select the components. Before production, test leader change, policy convergence, audit-chain continuity and Connector reconnect. In-flight Proxy sessions are not preserved.

Installation

This is the 0.11 release-candidate edition. Use the verified v0.11.0-rc.7 artifacts supplied by the deployment owner and check gdsgate --version. Select 0.10.x in the header for an installed 0.10 build. Existing deployments follow the 0.11 upgrade guide.

Install the binary the deployment owner supplies on each host. Do not download it from anywhere else.

From a release

Linux: get the binary, SHA256SUMS, SHA256SUMS.sig and SBOM from the release record. The release owner supplies the trusted signing identity or key and the signature verifier. Verify the signature on SHA256SUMS with that verifier before checking the checksums or installing the binary. A matching checksum alone does not establish the release identity.

After the signature check succeeds, use the release directory and tag from the handoff:

: "${RELEASE_DIR:?Set RELEASE_DIR to the approved release directory}"
: "${RELEASE_TAG:?Set RELEASE_TAG to the approved release tag}"
test -r "$RELEASE_DIR/SHA256SUMS"
test -r "$RELEASE_DIR/SHA256SUMS.sig"
(cd "$RELEASE_DIR" && sha256sum -c SHA256SUMS)

Every listed file must report OK. Only after both verification steps succeed, install on each Linux host from an administrative shell:

RELEASE_BINARY="$RELEASE_DIR/gdsgate-$RELEASE_TAG-x86_64-unknown-linux-musl"
test -r "$RELEASE_BINARY"
install -m 0755 "$RELEASE_BINARY" /usr/local/bin/gdsgate
gdsgate --version

The version, digest, target and release record must agree. On a missing file, signature failure, checksum failure or target mismatch, do not install or run the file. Keep the file name and verification exit status for the release owner; ask for a replacement.

Windows: install the supplied gdsgate-<version>-x86_64-pc-windows-msvc.msi from an elevated prompt, then check the version from a new prompt:

msiexec /i gdsgate-<version>-x86_64-pc-windows-msvc.msi
gdsgate --version

The installer puts gdsgate.exe in %ProgramFiles%\gdsgate, adds it to the system PATH, and installs and starts the gdsgate-client-net service. Server roles are registered separately; see As a Windows service.

Configuration templates

Write one config per role from the configuration reference. Keep store addresses, issuer URLs, CA paths and backend credentials out of shared templates.

profile = "deployment-name"
store_url = "PERSISTENT_STORE_FROM_OWNER"

[endpoints]
authority = "AUTHORITY_ENDPOINT_FROM_OWNER"
proxy_public = "PROXY_PUBLIC_ENDPOINT_FROM_OWNER"

[telemetry]
metrics_listen = "METRICS_BIND_FROM_OWNER"

Check each rendered config with gdsgate --config "$ROLE_CONFIG" doctor --json. It must report no configuration or trust error. Typical mistakes: an unknown key, an in-memory store outside a lab, a listener in the wrong zone.

Role fragments, each valid TOML:

[authority]
emergency_socket = "EMERGENCY_SOCKET_FROM_OWNER"

Set the emergency socket on Authority hosts only. It is a local recovery path; see The emergency administrative socket.

[proxy]
max_sessions = 100
max_streams_per_tunnel = 100

Limits on relay fan-out. Agree the values with the Proxy owner and test the refusal at the limit.

[enroll]
node_name = "connector-1"

The node name labels the node in logs and listings. It grants nothing; the role comes from the one-time token.

[client]
access = "basic"
resolver = "hosts"
warm_access = true

A least-privilege workstation baseline. transparent access and a system-wide resolver need a separate review of host privileges.

[[connector.backends]]
resource = "tools-dev"
kind = "mcp"
port = 8765
addr = "BACKEND_ADDR_FROM_OWNER"
enforce_tool_policy = true

Adds a Cedar decision for each MCP tool call on top of the static allow-list. Keep the allow-list narrow.

[policy]
path = "POLICY_FILE_FROM_OWNER"

The seed policy is read at first provisioning only. After that, use pull, validate and push.

[policy.editor]
min_approvers = 2
break_glass_policy_file = "BREAK_GLASS_POLICY_FROM_OWNER"

Quorum and break-glass need separate owners. Test both in a lab before relying on them in a lock-out.

[approvals]
min_approvers = 2

[approvals.per_environment]
production = 3

A per-resource threshold can override these; see the [approvals] cascade.

One command for a connector

On the protected backend host, create a one-time connector token and pass it to join on standard input:

umask 077
gdsgate --config "$ADMIN_CONFIG" authority tokens create \
  --role connector --ttl 3600 > "$CONNECTOR_TOKEN"
gdsgate join "$DOMAIN" --role connector --name connector-1 \
  --serve lab-postgres=postgres@BACKEND_ADDR \
  --output "$CONNECTOR_CONFIG" --state-dir "$CONNECTOR_STATE" \
  < "$CONNECTOR_TOKEN"

join writes the connector config and transport identity and consumes the token. Review the files, then start the Connector and check authority nodes list, authority offers list and the Connector health. A token for another role or a reused token is refused and leaves no registry row; create a new one. Delete the token file after enrollment.

As a systemd unit

Review the supplied unit (user, config, state directory, capabilities, log destination) before enabling it. The unit must run the same version and config you tested in the foreground:

gdsgate --config "$CONNECTOR_CONFIG" doctor --json

If the unit starts with an empty identity or an unexpected capability, stop it, fix ownership and rerun doctor. To remove a node, stop and disable the unit, then revoke the node.

As a Windows service

The same gdsgate.exe runs authority, proxy, connector and client-net. Put the role config at %ProgramData%\gdsgate\<role>.toml and run from an elevated prompt:

gdsgate service install connector
sc.exe start gdsgate-connector

The service runs under a virtual account and starts on sc.exe start or at boot. service install is refused from a non-elevated prompt and when the config file is missing. To remove a node, stop the service, run gdsgate service uninstall <role>, then revoke the node.

As a container

Use the image digest the release owner supplies. Run with a read-only root, a non-root user where the role allows, a writable state volume and the minimum network access. Mount config and identity directories owner-only. Health and readiness are the same as for a host process. If the container is healthy but cannot reach its control route, check the network zone and Connector tunnel. Remove the container and its volume after node revocation and audit export.

Bringing up a multi-node cluster

The minimal production path, in order. Concrete commands and files for a chosen topology: One Linux host, Separate hosts, Docker Compose, Kubernetes, Docker Swarm. Record the resulting IDs at each step.

1. Authority and the state store

Under pilot and production, store_url must name PostgreSQL (store-postgres); a persistent SQLite file is a dev-only choice. HA needs one PostgreSQL-compatible store shared by every replica. The store needs backups, restricted credentials, encrypted transport and a tested restore.

Authority starts, reports ready, and gdsgate doctor passes. If the store is not persistent or not reachable from an HA peer, fix it before enrolling nodes.

2. First-boot bootstrap

authority identities bootstrap --role admin --output-dir <dir> writes cert.pem, key.pem and ca.pem to an owner-only directory, or renews an existing usable identity; no second identity is created when renewal is possible. Check authority nodes list for one admin row and the enrollment audit event. Never delete the active identity directory.

3. Bootstrap tokens for joining nodes

authority tokens create --role <role> --ttl <seconds>, one per role. Each token is single-use and bound to one role; a token presented for another role or reused is refused. Delete token files after use and keep tokens out of shell history and process arguments.

4. Register the Proxy

Give the Proxy host its config, its token (through the enrollment mechanism) and the transport CA trust; check that the public and internal listeners are in the intended zones, then start it. authority nodes list shows the Proxy row, and a client completes the first fingerprinted connect. If the internal listener is exposed publicly, stop the Proxy, fix the zone, and revoke the row before enrolling again.

5. Register the Connector

Use One command for a connector. Use a backend file when a resource needs options --serve cannot express. cockroach is a Connector backend kind; the catalog lists it as postgres.

authority offers list shows the Connector's declaration. declared=no means the resource is not in the catalog; declare it before testing a client. Stop the Connector before removing its node identity.

6. Publish the transport CA fingerprint

Record the transport CA fingerprint with the config version and hand it to clients through a trusted channel: Certificates. On the client host, run Connect to the cluster with that value.

The client refuses a changed fingerprint. On a mismatch, stop and reconcile the two records. Keep the published fingerprint through the CA rotation window.

7. Publish and prove the first resource

Declare the resource and push the minimal policy, then query it through a native client: Policy guide, User guide. list-offered names the resource, the catalog row has the right kind, the user sees it, and the query succeeds. The audit shows catalog, policy, connect, query and close events; a denied resource stays visible and its connection is refused before the backend. If the offered ID, catalog ID, kind or policy entity differ, fix the mismatch; do not broaden policy.

Registration lifecycle and PKI

Enrollment issues a transport identity bound to a row in the enrolled-node registry. The row holds the registry ID, leaf fingerprint, expiry and revocation state; the node's display name is separate. Record the registry ID and the host owner.

What gets persisted where

Holder Stored
Authority store Node registry, policy versions, catalog rows, audit anchors
Role state directory (owner-only) The role's private key and leaf certificate
Client directory Trust anchor and identity cache

Back up the store and the audit anchor under the store owner. Private keys do not go into the shared backup.

What the cluster tells a registering node

The registration response carries cluster settings, trust, endpoint roles and renewal parameters. A node's --name is a label, not a location or a Cedar grant. Check the fingerprint, role and resource list before starting the role.

Where a node reaches the cluster to register

Enrollment goes to the enrollment listener over the configured TLS route. Renewal uses the node's transport identity over the Authority or Proxy route the deployment chooses. Use a client-facing address for control-plane certificates only when the topology publishes that route.

Renewal near expiry

Renew before the leaf expires; keep the state directory across restarts. A successful renewal keeps the registry ID, changes the leaf fingerprint, and writes an audit event. Renewal with an unknown or revoked fingerprint is refused; enroll again with a new token after deciding whether to revoke the old row. Remove superseded keys only after the new identity works.

Per-node revocation

gdsgate --config "$ADMIN_CONFIG" authority nodes list
gdsgate --config "$ADMIN_CONFIG" authority nodes revoke NODE_ID

Renewal is refused from then on; issued certificates stay valid until they expire. Revocation is idempotent. Use short TTLs and terminate active sessions to evict a node promptly. authority nodes remove removes a long-revoked row after the retention period; it never restores access. Remove the node's state only after audit and owner approval.

Naming a node

Names are restricted ASCII labels for logs. They do not identify the node's DNS name, role or location; the registry ID and fingerprint do. A duplicate or invalid name is refused. Names are not trimmed.

Reinstalled connectors, and removing a row

A reinstall that lost its identity enrolls as a new node. Revoke the old row, wait out the retention period, then delete it. Never delete a live row to make a new token work.

What revocation does not do

Revocation does not change catalog rows, delete audit rows, rotate CAs or expire issued certificates. Combine it with short TTLs, session termination and a network check.

Per-workload revocation

Provision-token workloads have a registry row and an external revocation key:

gdsgate --config "$ADMIN_CONFIG" authority workloads list
gdsgate --config "$ADMIN_CONFIG" authority workloads revoke WORKLOAD_ID

Token-less renewal is refused and the current short-lived certificate expires. A missing or unusable revocation key also refuses renewal. OIDC-bootstrapped stateless workloads are not in this registry: revoke their issuer or policy subject and rotate the credential.

CA rotation

Rotate one CA kind at a time: publish the new trust, wait out the double-signing window, and check every role before retiring the old signer. Commands include rotate-host-tls-ca, rotate-spiffe-ca and the other CA-specific Authority actions; see CLI → administration. Record the command, window, active signer and retirement. If a node cannot renew against the new trust, stop the retirement and restore the overlap.

Policy lifecycle

Policy is deny-by-default and evaluated by Authority. Actions and schema are in the Policy guide.

Granting policy edit

Grant editPolicy to a dedicated group or control-plane identity with a review owner. Check that the active policy keeps the break-glass rule before adding a second editor. An editor who can read but not propose has a missing grant; an invalid Cedar document is a separate error.

Day-to-day flow

gdsgate --config "$ADMIN_CONFIG" authority policy pull --output "$POLICY_FILE"
gdsgate --config "$ADMIN_CONFIG" authority policy validate "$POLICY_FILE"
gdsgate --config "$ADMIN_CONFIG" authority policy push "$POLICY_FILE"

Validation is strict. The push response carries the new version; Authority instances hot-reload it and write a policy audit event. A push from a stale base or with an invalid document is rejected and the active policy stays. Pull again, merge, validate and push. Do not edit the store by hand.

Lock-out and recovery

Normal routes (gateway and control certificate) are policy-gated. The emergency route is local to the Authority host, authorized by filesystem permissions, and has a fixed small set of actions. Test lock-out recovery in a lab.

The emergency administrative socket

Use it when the active policy admits neither the normal editor nor the control certificate:

gdsgate --config "$AUTHORITY_CONFIG" --route emergency authority policy push "$RECOVERY_POLICY"

Writes a new active version, audited as emergency. A socket permission error means local ownership is wrong; a validation error means the recovery policy is invalid. Never expose the socket remotely. Record the operator and incident ID.

The break-glass file

Set break_glass_policy_file before an incident. Authority reads it at startup, after the stored policy. It must keep the break-glass rule, differ from the normal seed path, and not be writable by others. When it differs from the active version, it becomes a new audited version. A missing file, a malformed policy or a candidate that drops the break-glass rule makes startup refuse or leaves the active policy unchanged.

Resource catalog lifecycle

New installations use automatic publication. Static backends and discovery targets appear under Authority-assigned namespaces. Existing clusters and Connector identities retain their publication mode. The declaration commands below describe the manual catalog path.

Authority resolves a resource in the catalog before Cedar evaluates the action. The admin listing shows all rows; gdsgate resources list shows only rows the user can view.

gdsgate --config "$ADMIN_CONFIG" authority offers list
gdsgate --config "$ADMIN_CONFIG" authority resources set lab-read --kind postgres --environment dev
gdsgate --config "$ADMIN_CONFIG" authority resources list
Symptom Cause What to do
Manual offered resource with no catalog row Not declared Declare it with authority resources set
Automatic backend with no catalog row Publication refused or expired Check node scope, lease and the Connector admission result
Visible row with no serving Connector Connector does not offer it Fix the Connector declaration

A registration states the kind of every resource it declares

A Connector declares each resource with its kind; the proxy forwards both. Authority admits an ID when:

  • the declared kind equals the catalog row's kind (k8s and kubernetes are one kind);
  • the enrolled node owns the manual row or has an explicit Registry binding;
  • the declaration states exactly one kind for that ID.

A declaration with no kinds is refused whole: the proxy drops the tunnel and no route is built. See Order within a mixed fleet for upgrades.

In the legacy manual path, a cluster whose catalog has never held a resource admits every declaration until the first authority resources set. Emptying the catalog later does not reopen this. Automatic publication requires the active node, scope and lease. See manual shared resources for several Connector nodes serving one logical ID.

authority offers list shows each Connector's declarations, including IDs not in the catalog and refused declarations. An empty kind means the declaration came through a proxy that does not forward kinds.

Seeding from [discovery] is an upsert

Discovery creates or updates rows. It does not delete rows missing from a later file. Remove retired IDs with remove-resource.

What removing a resource does

Removal blocks new access and hides the row from catalog resolution. Audit rows and open sessions stay. Terminate sessions, revoke the node or workload if needed, and check that the Connector no longer offers the ID:

gdsgate --config "$ADMIN_CONFIG" authority resources remove lab-denied
gdsgate --config "$ADMIN_CONFIG" authority offers list

The command is idempotent. Resident client names for the resource are removed after the resident grace period.

The first-provisioning window closes for good

Direct bootstrap works only until an admin identity exists. After that, use the mTLS or gateway route. The seed policy file is not a lock-out recovery path and does not replace the active version.

Just-in-time access: configuring approvers

The approver must be a different identity from the requester. Record the approval ticket.

Who: Cedar approveRequest

Grant approveRequest to a named approver group, scoped by environment or resource tags. Deny self-approval explicitly.

How many: the [approvals] cascade

The threshold is taken from the resource override, then the environment, then the global default. 0 defers to the next level. Test both a sufficient and an insufficient number of approvals.

Day-to-day flow

Requester

gdsgate --config "$CLIENT_CONFIG" requests create lab-read \
  --action connect --reason ticket-identifier --ttl 900
gdsgate --config "$CLIENT_CONFIG" requests list

Approvers

The approver reviews the reason, requester, resource, action and TTL, then runs gdsgate requests approve REQUEST_ID. The approval writes an audit event. An expired or under-approved request grants nothing and does not change policy.

A worked deployment

Set an environment rule and a resource override, then test the allowed and denied paths before rollout.

Operator-side cascade

Keep approval settings in the role configuration. No tokens or personal factors in the file.

Per-resource override for the strictest backend

Set the highest threshold on the sensitive resource and check that a lower global value does not override it. An approval for another resource or action does not apply.

Operational notes

An approval grants only the requested action for the requested TTL. It does not grant visibility, other resources or group membership. Keep requester, approver, ticket and session IDs in the audit handoff.

Network zoning

Keep client-facing, control-plane, enrollment and backend routes separate. The Proxy public listener is not the Authority store route. Connectors open their tunnels outbound; backends stay in the protected zone. Concrete traffic table and firewall for two hosts: Separate hosts.

Administering from outside the control zone

gdsgate --config "$ADMIN_CONFIG" --route auto authority nodes list

auto uses the control certificate when configured, otherwise the signed-in gateway route. The audit principal differs between the two.

Through the gateway, as yourself

For a human operator with an identity-provider token. The audit principal is the user, and network policy can match the source address. On a missing editPolicy or admin action, request a JIT grant or ask the policy owner.

The control-plane certificate, for machines and for an unreachable gateway

Uses the admin identity directory. The certificate subject is the audit principal. Keep the key owner-only and rotate it like a node identity. An expired or revoked certificate is renewed over the control route, or through the emergency procedure when policy is locked.

The emergency socket, when the policy admits neither

Authority host only, lock-out recovery only. There is no remote equivalent. Record the incident.

State store

The store holds policy versions, catalog rows, enrolled identities, approval state and audit metadata. Every non-lab role needs a persistent store with encrypted transport, least-privilege credentials and a tested backup. HA peers share a PostgreSQL-compatible store. Copy a live store only with a consistent snapshot method from the store owner.

Give gdsgate a dedicated login role that owns its database and is not a superuser. Keep the database administrator password separate from the application password. The HA container example creates these as separate roles and keeps the store on an unpublished network. For native setup, use the database commands.

If remote Authorities need the database port, allow only their addresses. Verify access from both an Authority and an untrusted workstation. Published container ports also need forwarding-policy checks; host input rules alone can leave them reachable. Keep database availability separate from process HA and verify a live policy read after store or leader changes.

Test restores in an isolated lab and run an audit chain check on the result. A restore missing its anchor or policy history must not serve clients.

Audit export

Logs (stderr), health, metrics and readiness endpoints, optional OTLP traces, the audit export and an external SIEM shipper are separate streams. Give each an owner, retention, transport and redaction rule. Metrics and process logs are not audit evidence.

Run the verifier from an operator workstation whose config opens the audit store and names the anchor key:

gdsgate --config "$AUDIT_VERIFY_CONFIG" authority audit verify

The chain and anchors verify. A missing anchor or tampered row fails the check; restore from a verified store and anchor backup and follow the incident procedure.

Delivering session recordings

With [recording].mode = "central", the connector uploads each session's cast to Authority after the session's close event is on the chain, and deletes its copy when Authority's receipt names the same digest. The local copy stays when:

  • the receipt is empty (Authority is older than the receipt field);
  • the receipt names a different digest;
  • the deletion fails.

The close event's file:// URI does not change after delivery. gdsgate authority recordings list and gdsgate doctor show delivery status.

A delivery pass runs on the connector at start-up and then every sweep_interval_secs (default 900). It sends what the upload at session close did not. 0 turns delivery off, including the start-up pass.

gdsgate doctor on the connector shows what is waiting, why the last pass failed, and whether the central store responds. After a store outage, the next scheduled pass delivers the backlog. To deliver now:

gdsgate --config connector.toml connector recordings push --dry-run   # what is queued, and why
gdsgate --config connector.toml connector recordings push             # deliver it

Exit codes and use under authority_transport = "tunneled": gdsgate connector recordings push.

A pass skips a cast without retrying in two cases:

Case Resolution
The chain has no close event for the session Resolves when the owning connector restarts; see Sessions with no close
The bytes do not match the digest in the close event Not delivered; the file changed after it was written

Sessions with no close

On a normal end, the connector finalizes the recording, writes a session.close row with the file's length and SHA-256 digest, then uploads. The central store accepts an upload only after the close row exists and checks the bytes against its digest.

A session has no close when its connector process stopped without cleanup: systemctl stop, SIGKILL, a panic or a host failure. The chain has an open row and nothing after it; the node has a .cast.partial file.

What follows, in order:

  1. The Proxy that carried the tunnel reports the sessions relayed over it. Authority writes one observation row per session and one for the node. These rows close nothing and admit no bytes. See Sessions a connector never accounted for.
  2. At its next start, the connector truncates the incomplete last line of each .cast.partial, renames the file, and drops files holding only a header.
  3. The delivery pass requests the missing close row from Authority and uploads the cast once the row is on the chain.

The recording still does not arrive when:

Condition Result
The chain has no open row (the audit batch was dropped while Authority was unreachable) No close can be written
The node never returns (reinstalled, revoked or wiped) Casts are lost with it
[recording].sweep_interval_secs = 0 No pass runs on that node
The request comes from a connector other than the one that opened the session Refused; a session.claim_refused row is written
The bytes do not match the sealed digest, or the session was opened from another node The store refuses the upload

The local copy is deleted only against a store receipt, so in all these cases the cast stays on the connector. Run gdsgate doctor on the node to see what is waiting and why.

Key management

A cluster at profile pilot or production seals rows under two keyrings, control and audit. The audit ring seals the journal's anchors and is kept longer. Coverage of each ring: Configuration → [security.integrity]. Commands: CLI → gdsgate authority keys.

Rotating a keyring

A ring is read when its role starts, so rotation takes three steps.

gdsgate --config "$AUTHORITY_CONFIG" authority keys rotate control --prepare
# 4c5d6e7f8091a2b3c4d5e6f708192a3b  is in the control keyring, prepared. It seals nothing yet.

--prepare creates a key and adds it to the ring as prepared; the active key does not change. Copy the updated ring file to every Authority replica and restart each one. Then:

gdsgate --config "$AUTHORITY_CONFIG" authority keys rotate control --activate

--activate makes the prepared key active for new rows and marks the previous key retiring. It refuses until every replica has reported holding the prepared key; gdsgate authority keys status shows N of M replicas hold it under each key. Rotation requires a rolling restart of every replica between the two steps.

The retiring key keeps verifying the rows it sealed. When no row names it:

gdsgate --config "$AUTHORITY_CONFIG" authority keys rotate control --retire

--retire refuses while any row names the retiring key (it prints the count per table), and while the key seals a recorded key-loss statement for another key. Revocation receipts and JIT grants expire or are reissued without action.

--retire always refuses on the audit ring: anchor rows are never deleted. The audit ring rotates with --prepare and --activate; older keys stay in the ring and keep verifying their anchors.

--kind signing (default --kind hmac) rotates the audit ring's checkpoint-signing key. The two kinds rotate independently; --activate affects only the kind given.

Under [security].profile = "production", --prepare refuses to create a key on the host. Add the key from its external source to the ring file as state = "prepared" and distribute it. --activate and --retire work under every profile.

Rotation writes no audit row. gdsgate authority keys status is the record.

Keeping keyrings out of the database backup

Anyone holding the database and the keyring together can write rows that verify. Back up each keyring separately from the database and from the other keyring.

gdsgate --config "$AUTHORITY_CONFIG" authority keys manifest show

Prints one block per keyring with fingerprints, states and the protection between source and material (currently none). It opens no store, so it works before a restore. A database restored without its keyrings cannot be verified; restore the keyrings from their own backup alongside the store.

A signing key's public half, printed by backup-manifest, can be published with the dump; see Verifying the journal without a keyring.

Verifying the journal without a keyring

A reviewer with only a public key can check that the audit chain was not truncated or rewritten. On the Authority host, or where the audit keyring is readable:

gdsgate --config "$AUTHORITY_CONFIG" authority audit checkpoint-keys export > checkpoint-keys.txt

Publish checkpoint-keys.txt with the database dump. It lists every signing key the audit ring has held. The audit ring never retires a key, so each later output includes all earlier keys.

The reviewer runs against a restored copy or read replica, never with the live cluster's credentials:

gdsgate --config "$REVIEWER_CONFIG" authority audit verify --checkpoint-keys checkpoint-keys.txt
# audit chain ok (chain + checkpoints): 12483 rows, 0 anchors, 61 checkpoints

Exit 0 and chain + checkpoints in the summary. The hash chain and every checkpoint signature are checked. Anchors show 0: the anchor check needs the ring's HMAC keys. A non-zero exit names a broken link, a failed checkpoint or a serial range no known key covers; treat each as tampering and escalate.

When a keyring is lost

gdsgate authority keys loss accept records that a key's material is lost. Rows it sealed then read as unverifiable, not forged:

gdsgate --config "$AUTHORITY_CONFIG" authority keys loss accept \
  --key-id 3f9c1a2b4d5e6f708192a3b4c5d6e7f8 --accepted-by anna.ivanova

The statement is sealed by the ring's current active key and written to the audit chain. A ring with no active key refuses the command with the same refusal keys status shows.

Under [security].profile = "production" the command refuses. Step the profile down with gdsgate authority profile set (an audited event), declare the loss, then step the profile back up.

Effects per row type are listed under keys accept-loss. Revoked nodes and workloads stay revoked. Approvals, step-up grants and delegations must be requested again. Inventory snapshots and tool pins are republished by their connector. audit_anchor rows are never deleted, so verify-audit reports that stretch of the chain as unverifiable for as long as the journal is kept; it does not fail.

accept-loss covers a lost key while the ring and its active key survive. If the ring file is lost with no active key left, no command rebuilds it: restore the ring from its separate backup.

Hardening checklist

Network

  • Keep the Authority store and emergency socket off the public path.
  • Allow Proxy → Authority and Connector → Authority routes only where needed.
  • Keep backend credentials and listeners inside the protected zone.
  • Test private-address refusal on web egress and the direct-backend network denial.

Transport

  • Pin the transport CA fingerprint in the client handoff.
  • Pass enrollment tokens on standard input with short TTLs.
  • Rotate host, workload and onward CAs with an overlap window.
  • Revoke nodes and workloads before deleting their state.

Process

  • Run each role under a dedicated account with an owner-only identity directory.
  • Mount configuration read-only; make writable state paths explicit.
  • Readiness is per role: a Connector can be ready before its tunnel reaches Authority.
  • Review generated files and service capabilities before start.

State

  • Back up the store and the external audit anchor separately.
  • Back up the control and audit keyrings separately from the store and from each other; see Key management.
  • Test restore, policy lock-out and node revocation on a schedule.
  • Keep policy versions and approval records for the retention period.

Policy

  • Start deny-all, add typed view and connect, then request-aware rules.
  • Keep the break-glass rule in every candidate policy.
  • Keep human, workload and delegated-agent identities separate.
  • After every policy or catalog change, test one allowed lab-read operation and one denied lab-denied operation.