HA with direct Authority failover

A public L4 LB fronts the active Proxy pool. Each Proxy reaches Authority instances by individual addresses and falls back to another ready instance. Two Authorities share PostgreSQL and run as leader/follower for writes. Authorities synchronize integrity material through native bootstrap. Profile: pilot; Connector route: tunneled.

Hosting does not define this topology. Choose the Linux binary or Docker Compose tab below. Roles can share hosts or run on separate hosts.

The image packages the same gdsgate binary and sets gdsgate as its entrypoint. docker run and docker compose run take its arguments; docker compose exec includes the executable name. setup, authority, join, config check and their flags stay the same. The platform supplies configuration mounts, secrets, persistent directories, network addresses and the process supervisor.

This composition uses Authority HA, bootstrap and synchronization and Proxy HA. The other Authority routing topology uses the same components.

Before you start

The deployment owner supplies the same store, OIDC settings and security profile to all roles, assigns individual identities and opens the selected network paths before enabling LB backends.

Input	Example	Contract
Public Proxy entry	203.0.113.10, or gate.example.com with a public certificate	Certificates
Authority and Proxy instances	A 192.0.2.10, B 192.0.2.11 in the Linux example; the same host addresses in Compose	Individual Authority and Proxy peer addresses
Shared PostgreSQL	192.0.2.30:5432, database and role gdsgate; external store in both platform examples	Store availability arranged separately
OIDC	https://idp.example.com/realms/gdsgate, public client gdsgate, group admins	Same issuer and client on all roles
Administration	Protected operator transfer, synchronized clocks, agreed test resources	Root shell for host provisioning; private state per instance

For an existing cluster use migration. Do not run initial setup prepare over its existing state.

flowchart LR
    C[Clients and Connectors] --> PLB[Public L4 LB]
    PLB --> P1[Proxy A]
    PLB --> P2[Proxy B]
    P1 <--> P2
    P1 --> A1[Authority A]
    P1 --> A2[Authority B]
    P2 --> A1
    P2 --> A2
    A1 <--> A2
    A1 --> DB[(Shared PostgreSQL)]
    A2 --> DB

Open the paths

Listener	Who reaches it	Exposure
Proxy 50061–50063; 50064 for the join relay	Public LB, clients and Connectors through it	Public entry only
Authority 50050/50051	Authorized registration/control callers and individual Authority peers	Private
Proxy peer 50065	Other Proxies	Private, individually addressed
Authority and Proxy /readyz	Their LB health checkers and private monitoring	Private; bind addresses depend on the platform
Store 5432	Authorities	Private

Use raw TCP passthrough with no TLS termination. Registration and tunnel listeners accept no PROXY headers. For original source-IP preservation on 50061, configure both the LB and Proxy as described in Proxy HA; configure its trusted source list before enabling PROXY v2.

Run the same CLI on either platform

An Authority status check illustrates the packaging boundary. The configured file path is an input; the gdsgate operation is identical:

Linux binaryDocker Compose

On Authority A:

AUTHORITY_CONFIG=/etc/gdsgate/authority.toml
gdsgate --config "$AUTHORITY_CONFIG" authority keys status

On the Docker host:

AUTHORITY_CONFIG=/etc/gdsgate/authority.toml
docker compose exec -T authority gdsgate --config "$AUTHORITY_CONFIG" authority keys status

The same applies to setup prepare, authority keys sync tokens create, authority tokens create, join, config check and key rotation. A native service starts gdsgate --config <file> authority or proxy; Compose records those arguments after the image entrypoint. Choose the platform once for provisioning and keep its persistent state across restarts.

Configure and start

Initialize the first pair once, then join the second pair through the public Proxy entry. The shared bootstrap procedure applies to both topologies. Every role retains its own private identity and state.

Linux binaryDocker Compose

Install the supplied binary at /usr/local/bin/gdsgate on both hosts. Run provisioning in a root shell. On A, install the database password in /etc/gdsgate/store.password, protected from other accounts, then run:

gdsgate setup prepare --role pair --domain 203.0.113.10 \
  --security-profile pilot \
  --store-url postgres://gdsgate@192.0.2.30:5432/gdsgate \
  --store-password-file /etc/gdsgate/store.password \
  --authority-bind 192.0.2.10 --listen 192.0.2.10:50061 \
  --oidc-issuer https://idp.example.com/realms/gdsgate \
  --oidc-client-id gdsgate --access-group admins --start
curl --fail --silent http://127.0.0.1:9090/readyz
curl --fail --silent http://127.0.0.1:9091/readyz
(umask 077; gdsgate authority tokens create --role pair > pair.token)

Record the printed cluster CA fingerprint. Enable ready Proxy A in the external public LB. Transfer only pair.token through the protected operator channel. On B:

CA_FINGERPRINT='<approved SHA-256 fingerprint>'
gdsgate join 203.0.113.10 --role pair --listen 192.0.2.11 \
  --fingerprint "$CA_FINGERPRINT" --start < pair.token
rm pair.token
curl --fail --silent http://127.0.0.1:9090/readyz
curl --fail --silent http://127.0.0.1:9091/readyz

For a DNS entry, use gate.example.com and add the public certificate and key to A's setup. B provides its certificate paths in the private local template. The cluster delivers internal transport trust separately; no CA or keyring file is copied.

For external Proxy health checks on A, set the private health address:

gdsgate setup change --proxy-config /etc/gdsgate/proxy.toml \
  --set proxy:telemetry.metrics_listen=192.0.2.10:9091
systemctl restart gdsgate-proxy
curl --fail --silent http://192.0.2.10:9091/readyz

On B, use 192.0.2.11:9091. Allow only the LB checker and monitoring. Select the Authority routing below and enable ready Proxy B in the LB.

Use two Linux Docker hosts with individual addresses 192.0.2.10 and 192.0.2.11, the same external PostgreSQL store and external public LB as the binary example. Host networking keeps the individual listener addresses unchanged. It does not place either balancer on a gdsgate host.

On each host, create /opt/gdsgate-ha and select the supplied image:

install -d -m 0755 /opt/gdsgate-ha
cd /opt/gdsgate-ha
: "${IMAGE:?Set IMAGE to the supplied matching immutable image}"
printf 'IMAGE=%s\n' "$IMAGE" > .env
install -d -m 0700 -o 10001 -g 10001 config state account

Save compose.yaml on both hosts:

name: gdsgate-ha
x-node: &node
  image: ${IMAGE}
  network_mode: host
  read_only: true
  cap_drop: [ALL]
  security_opt: [no-new-privileges:true]
  tmpfs: [/tmp]
  restart: unless-stopped
  stop_grace_period: 45s
  environment:
    GDSGATE_KEYSTORE_ROOT: /var/lib/gdsgate/operator
    GDSGATE_DRAIN_FILE: /tmp/gdsgate-draining
  volumes:
    - ./config:/etc/gdsgate:ro
    - ./state:/var/lib/gdsgate
    - ./account:/var/lib/gdsgate/operator
services:
  configure:
    <<: *node
    profiles: [setup]
    restart: "no"
    volumes:
      - ./config:/etc/gdsgate
      - ./state:/var/lib/gdsgate
      - ./account:/var/lib/gdsgate/operator
  authority:
    <<: *node
    command: [--config, /etc/gdsgate/authority.toml, authority]
  proxy:
    <<: *node
    command: [--config, /etc/gdsgate/proxy.toml, proxy]

The image runs as UID 10001. One-shot configuration uses writable mounts; runtime configurations are read-only. Preserve all three directories on each host. Do not copy state or account from A to B.

On A, install the protected database password as config/store.password, owned by 10001:10001, mode 0600. Prepare the first pair without --start:

PUBLIC_ENTRY=203.0.113.10
NODE_IP=192.0.2.10
docker compose run --rm --no-deps configure setup prepare \
  --role pair --domain "$PUBLIC_ENTRY" --security-profile pilot \
  --store-url postgres://gdsgate@192.0.2.30:5432/gdsgate \
  --store-password-file /etc/gdsgate/store.password \
  --authority-bind "$NODE_IP" --listen "$NODE_IP:50061" \
  --oidc-issuer https://idp.example.com/realms/gdsgate \
  --oidc-client-id gdsgate --access-group admins --unit-dir /tmp/units
docker compose up -d authority
curl --fail --silent http://127.0.0.1:9090/readyz
docker compose run --rm --no-deps configure \
  --config /etc/gdsgate/authority.toml authority identities bootstrap \
  --role admin --output-dir /etc/gdsgate/admin
docker compose run --rm --no-deps configure setup change \
  --authority-config /etc/gdsgate/authority.toml \
  --set authority:admin.identity_dir=/etc/gdsgate/admin
(umask 077; docker compose exec -T authority gdsgate \
  --config /etc/gdsgate/authority.toml authority tokens create --role proxy \
  > config/proxy.token)
chown 10001:10001 config/proxy.token
docker compose run --rm --no-deps configure setup change \
  --proxy-config /etc/gdsgate/proxy.toml \
  --set 'proxy:enroll.token=${file:/etc/gdsgate/proxy.token}'
docker compose up -d proxy
curl --fail --silent http://127.0.0.1:9091/readyz
docker compose run --rm --no-deps configure setup change \
  --proxy-config /etc/gdsgate/proxy.toml --set proxy:enroll.token=
rm config/proxy.token

Record the printed CA fingerprint. Enable ready Proxy A in the external LB. For DNS public TLS, install the certificate and key on both hosts and add --tls-cert /etc/gdsgate/tls/fullchain.pem --tls-key /etc/gdsgate/tls/privkey.pem to A's setup. B supplies those paths in a private config/local-pair.toml [proxy] template with public_cert_file and public_key_file.

On A, issue the pair permission:

(umask 077; docker compose exec -T authority gdsgate \
  --config /etc/gdsgate/authority.toml authority tokens create --role pair \
  > pair.token)

Transfer only pair.token through the protected operator channel. On B:

cd /opt/gdsgate-ha
CA_FINGERPRINT='<approved SHA-256 fingerprint>'
docker compose run --rm --no-deps -T configure join 203.0.113.10 \
  --role pair --listen 192.0.2.11 --fingerprint "$CA_FINGERPRINT" < pair.token
rm pair.token
docker compose up -d authority proxy
curl --fail --silent http://127.0.0.1:9090/readyz
curl --fail --silent http://127.0.0.1:9091/readyz

For DNS mode, add --local-config /etc/gdsgate/local-pair.toml to B's join. The command arguments are the native arguments; Compose supplies mounts, identity ownership and process startup. Use an individual health bind when an external LB checks readiness. Apply the selected Authority routing below, then enable ready Proxy B in the public LB.

Keep individual Authority routing

Keep the generated [proxy].authority_topology = true. The authenticated membership supplies and refreshes individual control and registration routes. No private LB, static failover list or per-Proxy edit is needed when another Authority joins. Check a fresh access decision and fresh node join after either Authority stops.

Add resources and more instances

Follow Connector installation, with two verified Proxy tunnels for this HA example. New installations publish ordinary backends automatically inside the admitted node namespace. Manual resources use explicit Registry assignments. Perform allowed and denied native operations and retain their audit evidence.

Add a third or later Authority with join --role authority, a Proxy with join --role proxy, or both with join --role pair. Issue the corresponding permission from any admitted Authority and join through the same public entry. Update external LB backends through deployment tooling after readiness; gdsgate updates its authenticated membership without distributing keys by hand.

Connect a client

On the workstation, the public entry is independent of the server platform:

CA_FINGERPRINT='<approved SHA-256 fingerprint>'
gdsgate connect 203.0.113.10 --fingerprint "$CA_FINGERPRINT"
gdsgate login
gdsgate resources list
gdsgate up
gdsgate status

connect selects the saved user context. For another cluster, use an explicit --output ~/.gdsgate/ha/client.toml; switch by path with gdsgate contexts use ~/.gdsgate/ha/client.toml. User context selection does not replace the installed Authority administrative configuration.

For approved public DNS trust, use gdsgate connect gate.example.com. The login must use the configured OIDC provider. Test a native resource connection, its recording and canonical audit, and a denied operation. For automatic resources, use the qualified ID shown by ls, such as conn-1/demo-ssh; a short alias is usable only while unambiguous.

Verify the topology

Run these checks in an acceptance window. Restore each stopped component before testing the next failure. A successful ls is not proof of recovery of leader-dependent access decisions: open a new allowed resource connection.

Check	Expected	Recovery
Authority key status on A and B	Same revision and expected holders	Restore the affected node's store access and protected state
Readiness plus policy pull and a new native resource connection	Ready nodes in the pools and working store/leader path	Inspect role logs, store access, trust and routing
Connector doctor --explain with its actual config	2 verified distinct Proxy nodes, 2 desired; admitted catalogue entry	Check LB tunnel affinity, Connector settings and startup diagnostics
Stop the Authority write leader; open another allowed native connection	New decisions recover after lease takeover	Check the remaining node's lease, direct peer path and store access
Stop Proxy A; open another connection	Public LB selects B; new access succeeds	Correct backend withdrawal and surviving tunnel routes
Public client tries private control, peer and health listeners	Network refusal	Correct exposure before proceeding
Wrong trust pin during fresh enrollment	Refusal before identity issuance	Restore the approved pin; never disable verification
Denied resource, direct backend isolation and audit verification	Results satisfy acceptance	Follow the owning access or audit recovery procedure
Stop the preferred Authority while its Proxy keeps running	Control and fresh registration use the other individual destination	Check authenticated membership, registration routes and TLS verification

Run checks and failure injection on the platform

Linux binaryDocker Compose

On each Authority:

gdsgate --config /etc/gdsgate/authority.toml authority keys status
gdsgate --config /etc/gdsgate/authority.toml doctor

Stop A with systemctl stop gdsgate-authority or systemctl stop gdsgate-proxy. Restore with systemctl start and verify readiness before returning it to a pool. On a two-host deployment, also test loss of both roles on one host during the acceptance window.

On the Docker host:

docker compose ps
docker compose exec -T authority gdsgate --config /etc/gdsgate/authority.toml authority keys status

Stop A with docker compose stop authority or docker compose stop proxy. Restore with docker compose start and verify health before the next failure. This checks process failure on one host; it does not establish host-failure availability.

Existing streams owned by a stopped Proxy end. Keep outputs, revisions, versions and the selected topology with the deployment record, without tokens or private keys.

Upgrade and drain

Withdraw one node at a time and verify new access before proceeding. Use change planning, and the upgrade guide for version-specific changes. Keyring rotation follows the same CLI procedure on either platform; replicas reload without copying ring files.

Linux binaryDocker Compose

Drain the Proxy from the external LB and wait for measured withdrawal. Restart or update one service at a time, retaining its private state:

systemctl restart gdsgate-authority
systemctl restart gdsgate-proxy

Run readiness and native access checks after each individual restart. Do not run both commands as an availability test without checking between them. Re-admit the Proxy only after it is ready.

On one host at a time, drain its Proxy from the external LB, update .env to the supplied image and recreate each role separately:

docker compose pull authority proxy
docker compose up -d --no-deps authority
curl --fail --silent http://127.0.0.1:9090/readyz
docker compose up -d --no-deps proxy

Verify private readiness and a new native connection before returning the Proxy to the pool and proceeding to the other host.

Backup

Preserve the same application data on both platforms: shared store, configuration, Authority roots and revision caches, certificates and each node's enrolled identity. Pair Authority protected-state backups with the store backup. See paired recovery.

Linux binaryDocker Compose

Back up /etc/gdsgate and each role's private state directory using the approved backup tooling. Preserve permissions and ownership. Include keyring-sync/root.key (or provider-wrapped root.enc) and snapshot.enc from every Authority, the administrative identity and each Proxy and Connector identity.

Back up each host's config, state and account directories, preserving ownership and modes. Pair Authority protected state with the database backup; retain every node's independent receiver and transport keys.

Separate hosts and other balancers

The topology depends on individually reachable peers and stable owner names, not on co-location or container service names. Substitute real private addresses consistently when moving roles to separate hosts. The store and LB require their own availability arrangements. Kubernetes or Swarm placement must preserve the same component state and routing contracts; use component platform mapping with the corresponding platform guide.

Profile and limits

Boundary	Requirement / behavior
pilot	Verified TLS, enrolled node mTLS, OIDC, shared store and integrity material
production	Profile checks and a declared generation provider for synchronized key rotation
Admission	Same ha.max_cluster_sessions on all Proxies; 1024 by default, maximum 4096
Sessions	Loss of an owning Proxy ends streams; new connections recover through surviving routes
Browser listeners	Configure their public TCP frontends and certificates separately
Recovery	Store and Authority protected state form a paired set; deleting a watermark is not rollback