Deploy Proxy in HA¶

The deployment administrator enrolls each Proxy separately, connects private Proxy peers and publishes ready instances through a common L4 entry point. Clients and Connectors can then reach different Proxies while resource opens use local tunnels or one verified peer hop.

This procedure works with one Authority, HA Authority reached individually, or HA Authority behind an L4 LB. Select the Authority destinations first. Multiple Proxies do not require Authority bootstrap when the deployment has only one Authority.

Before you start¶

Input	Check / value	Reference
Ready Authority and administrative identity	`authority keys status`; `authority list-nodes` using its configuration	Authority HA, administration
Public entry	Example IP `203.0.113.10`; raw TCP LB managed by the network administrator	Certificates
Proxy nodes	A `192.0.2.10`, B `192.0.2.11`; individual private peer `50065`	Peering fields
Role inputs	Same OIDC issuer and client, security profile and Authority trust; separate enrollment tokens and identity directories	Enrollment
Connector route	Example `tunneled`, IP-only registration relay `50064`	Join relay
Connector capacity	`connector.proxy_connection_count = 2` or greater for redundant verified tunnels; enrolled nodes support peer opens with one tunnel too	Connector configuration

Linux commands run in a root shell. Container deployments use the same configuration and gdsgate commands through their platform launcher; see the Authority LB topology, Compose tab and container guide. Keep each Proxy's identity directory durable and private. Do not copy another Proxy's identity, even when both roles run on the same host.

Configure each Proxy¶

1. Prepare each Proxy with the CLI¶

Run on each Proxy host. Set its individual addresses first:

Proxy AProxy B

NODE_IP=192.0.2.10
OTHER_IP=192.0.2.11
NODE_NAME=proxy-a

NODE_IP=192.0.2.11
OTHER_IP=192.0.2.10
NODE_NAME=proxy-b

On both hosts, set the same public entry and identity provider:

GATEWAY_ADDRESS=203.0.113.10
PROXY_BIND=0.0.0.0
OIDC_ISSUER=https://idp.example.com/realms/gdsgate
OIDC_CLIENT_ID=gdsgate
: "${NODE_IP:?Set this node address}"
: "${OTHER_IP:?Set the other node address}"
: "${NODE_NAME:?Set a unique node name}"
: "${OIDC_ISSUER:?Set the identity provider issuer}"
: "${OIDC_CLIENT_ID:?Set its client ID}"
groupadd --system --force gdsgate
install -d -o root -g gdsgate -m 0750 /etc/gdsgate

Install the cluster's public transport CA at /etc/gdsgate/transport-ca.pem through the protected operator channel. Compare its SHA-256 fingerprint with the Authority owner's value:

openssl x509 -in /etc/gdsgate/transport-ca.pem -noout -fingerprint -sha256

On a host already running Authority, use its exported CA. This is a public certificate, not a registration private key. IP-only Proxies present their own enrolled identities; they need no public_cert_file or public_key_file.

Generate the IP-only Proxy configuration:

gdsgate setup prepare \
  --role proxy --domain "$GATEWAY_ADDRESS" --security-profile pilot \
  --authority-host "$NODE_IP" --join-relay \
  --listen "$PROXY_BIND:50061" --metrics-port 9091 \
  --proxy-output /etc/gdsgate/proxy.toml \
  --state-dir /var/lib/gdsgate/proxy \
  --oidc-issuer "$OIDC_ISSUER" --oidc-client-id "$OIDC_CLIENT_ID"

gdsgate setup change --proxy-config /etc/gdsgate/proxy.toml \
  --set "proxy:endpoints.authority_failover=$OTHER_IP:50051," \
  --set "proxy:endpoints.authority_enroll_failover=$OTHER_IP:50050," \
  --set "proxy:proxy.peer_listen=$NODE_IP:50065" \
  --set "proxy:proxy.peer_addr=https://$NODE_IP:50065" \
  --set "proxy:enroll.node_name=$NODE_NAME" \
  --set proxy:enroll.transport_sni=enroll.gdsgate.internal \
  --set proxy:enroll.transport_ca=/etc/gdsgate/transport-ca.pem \
  --set 'proxy:enroll.token=${file:/etc/gdsgate/proxy.token}' \
  --set proxy:ha.max_cluster_sessions=1024 \
  --set "proxy:telemetry.metrics_listen=$NODE_IP:9091"

The trailing comma writes a one-element fallback array. The listeners bind all interfaces; the host firewall permits these backend ports only from the external LB. Private peer and health listeners use the individual node IP. peer_addr names this Proxy, not the other Proxy or the LB.

Authority advertises the public LB addresses 50061 through 50064 to joining nodes. Keep those advertisements the same on every Authority. Proxy listener binds are local inputs; do not replace Authority's advertised LB entry with a backend address. The wildcard public bind also prevents client bootstrap from redirecting a workstation to one backend.

--role proxy does not inherit OIDC from an existing Authority configuration. Pass both OIDC flags on every Proxy, including when Authority and Proxy share one host. Empty or misspelled shell variables can otherwise leave the Proxy advertising the development issuer. Use OIDC_ISSUER consistently.

For Authority behind a private LB, apply this change on each Proxy before installing its service:

AUTHORITY_LB=192.0.2.20
gdsgate setup change --proxy-config /etc/gdsgate/proxy.toml \
  --set "proxy:endpoints.authority=$AUTHORITY_LB:50051" \
  --set "proxy:endpoints.authority_enroll=$AUTHORITY_LB:50050" \
  --set proxy:endpoints.authority_failover=, \
  --set proxy:endpoints.authority_enroll_failover=, \
  --set "proxy:enroll.endpoint=https://$AUTHORITY_LB:50050"

A comma alone writes an empty array. With one Authority, keep its control and registration destinations and remove both fallback lists the same way. Complete reference files: Proxy A and Proxy B. They use the same inputs as these commands.

For a public DNS entry, keep private Authority registration on the cluster CA. Supply the Proxy's public certificate pair, then change its public entry mode:

gdsgate setup change --proxy-config /etc/gdsgate/proxy.toml \
  --tls-cert /etc/gdsgate/tls/fullchain.pem --tls-key /etc/gdsgate/tls/privkey.pem \
  --set proxy:proxy.public_join=yes --set proxy:endpoints.proxy_join=

On every Authority, remove endpoints.proxy_join, set proxy.public_join=yes and advertise the DNS entry on proxy_public, proxy_internal and proxy_ws. Restart existing roles after those configuration changes. Connector registration then uses https://gate.example.com:50061; its public TLS trust is separate from the internal cluster CA.

2. Enroll and start each instance¶

On a provisioned Authority with its administrative configuration:

umask 077
gdsgate --config /etc/gdsgate/authority.toml authority create-token --role proxy > /etc/gdsgate/proxy-a.token
gdsgate --config /etc/gdsgate/authority.toml authority create-token --role proxy > /etc/gdsgate/proxy-b.token
scp -p /etc/gdsgate/proxy-a.token ops@192.0.2.10:proxy.token
scp -p /etc/gdsgate/proxy-b.token ops@192.0.2.11:proxy.token
rm /etc/gdsgate/proxy-a.token /etc/gdsgate/proxy-b.token

Use your operator account in place of ops; its example home is /home/ops. umask 077 keeps newly created token files at 0600; it does not set the required 0755 mode of the shared /var/lib/gdsgate parent.

On each Proxy host, install its own token, then start the service:

install -m 0640 -o root -g gdsgate /home/ops/proxy.token /etc/gdsgate/proxy.token
rm /home/ops/proxy.token
gdsgate --config /etc/gdsgate/proxy.toml service install proxy
gdsgate config check /etc/gdsgate/proxy.toml --role proxy
systemctl start gdsgate-proxy

Check A at http://192.0.2.10:9091/readyz and B at http://192.0.2.11:9091/readyz: both must return HTTP 200 ready. On enrollment failure inspect journalctl -u gdsgate-proxy -n 50; check the Authority destination, TLS trust, token and state ownership before retrying.

After readiness, on each host remove the spent input and verify restart:

gdsgate setup change --proxy-config /etc/gdsgate/proxy.toml \
  --set proxy:enroll.token=
rm /etc/gdsgate/proxy.token
systemctl restart gdsgate-proxy

Readiness must recover without another token. Also pull the active policy through the public entry and open an allowed resource after Connector setup. HTTP readiness alone does not verify the shared store and leader route. Keep the enrolled identity state. The IP-only templates use cluster-issued TLS; a DNS entry additionally needs the Proxy public certificate pair.

3. Configure the public L4 LB¶

The network administrator permits LB-to-Proxy listener connections, Proxy-to-Proxy 50065, Proxy-to-Authority control and registration, and health checks. Public clients must not reach peer or health listeners.

Public listener	Backends	Purpose
`203.0.113.10:50061`	A/B `50061`	Client API
`203.0.113.10:50062`	A/B `50062`	Connector tunnels
`203.0.113.10:50063`	A/B `50063`	WebSocket fallback
`203.0.113.10:50064`	A/B `50064`	IP-only registration and renewal relay

Use raw TCP passthrough, without TLS termination. Admit each backend only when its private 9091/readyz returns HTTP 200 ready; withdraw failed or draining instances. Do not pin tunnel connections by source IP: a Connector needs tunnels to distinct Proxies.

The IP templates accept no PROXY headers. If source-IP policy requires the original address, configure the public 50061 frontend to send PROXY v2 and set proxy.public_proxy_protocol_from to the actual LB addresses. Configure both ends together. Keep tunnel, WebSocket and registration relay listeners without PROXY headers. Without preservation, source-IP checks see the LB address. Exact contract: public listener behind a front.

With DNS and a public certificate, registration can use public_join on 50061; configure the same mode and advertised renewal entry on every Authority. Certificate and entry-port choices are independent of Authority HA routing.

Connect resources and verify¶

The deployment owner registers a Connector at the selected public entry, sets connector.proxy_connection_count = 2 before start, and declares and serves an allowed and a denied resource. IP-only registration uses https://203.0.113.10:50064 with the recorded CA fingerprint. Follow the HA Connector steps using these entry values.

Check	Expected	Recovery
`authority list-nodes`	Two active Proxy rows with distinct UUIDs	Check separate tokens and identity directories
`doctor` on each Proxy	No failed trust, Authority route or state checks	Correct the named check before enabling the backend
Connector `doctor`	Desired 2 and connected 2 distinct Proxies	Check tunnel LB affinity, reachable advertised ports and backend readiness
Client login and allowed native operation through the LB	Catalog and access succeed	Check OIDC, Connector serving and private peer reachability
Stop Proxy A; open another native connection	New access succeeds through B	Check LB withdrawal and surviving Connector tunnels
Client and Connector reach different Proxies	Resource opens through a local tunnel or one verified peer hop	Check `50065`, separate enrolled peer identities and Connector connection count
Public workstation tries peer `50065` or health `9091`	Network refusal	Correct network exposure

Restore A before repeating the failure test for B. Existing streams owned by a stopped Proxy end; they are not transferred. Complete deployment acceptance, including denied access, backend isolation and audit verification.

Drain, preserve state and recover¶

Withdraw one Proxy from the LB, wait for measured LB propagation, then stop or recreate it. Preserve that instance's identity directory. Re-admit it only after readiness and a new access check. A lost identity requires revoking its old node UUID and enrolling a new identity with a new token.

Use change planning and the Compose drain example. Availability of the store, Authorities and the public LB remains a separate boundary; test the selected composition.

Diagnose deployment errors¶

Symptom	Check	Fix
`login` uses the development issuer	`[oidc]` in every Proxy and the selected client config	Set matching issuer/client on each Proxy, restart it, then refresh that client configuration with `connect --force` and log in again
Login succeeds, later RPC returns `invalid identity token`	Issuer, client audience, validation and groups on every Authority; selected client config	Align all instances and refresh the client configuration; successful login only proves the IdP accepted it
`Authority leader discovery failed`	Shared DB connection, leader lease, individual Authority peers and control fallback lists	Restore the store or the named route; verify policy pull and a new allowed native operation
Join relay port does not listen	`endpoints.proxy_join` in every Proxy and Authority advertisements	Enable the same relay mode on all roles, restart changed roles, then check the public LB frontend
Registration `UnknownIssuer`	Registration endpoint, CA and `transport_sni`	Use the cluster CA and `enroll.gdsgate.internal` for private registration; see trust checks
Service cannot change working directory	`namei -l /var/lib/gdsgate/proxy` and unit `WorkingDirectory`	Rerun `service install proxy` with this config; verify shared parent `0755` and preserve private role permissions

Runtime readiness¶

Proxy /readyz checks all configured Authority endpoints concurrently using its current transport identity. At least one must confirm write leadership with a working store and live lease. A stalled first endpoint does not hide a healthy failover. Dependency failure returns HTTP 503; recovery restores HTTP 200 without a Proxy restart. Use this endpoint for LB health checks. /healthz reports process liveness independently.