Docker Swarm on one node

Authority and Proxy as a gdsgate stack, PostgreSQL as a gdsgate-store stack, on one Swarm node from the release image. Connectors on other hosts or as a service in the same swarm. Profile: pilot.

Placeholder	Meaning
gate.example.com	The name clients and connectors dial; resolves to the node.
203.0.113.10	The node's address.
registry.example.com/gdsgate:v0.10.0	The release image. See Running the image.
https://idp.example.com/realms/gdsgate	The OIDC issuer.

Before you start

Item	How to check	Reference
Docker Engine, not yet in a swarm	docker info -f '{{.Swarm.LocalNodeState}}' prints inactive
The image on the node, or in a registry the node pulls from	docker image inspect registry.example.com/gdsgate:v0.10.0	Running the image
Ports 50061–50063 free	ss -ltn \| grep 5006 prints nothing	Connector routes
The same ports open inbound before you install. The administrator opens them; without them join and connect from other hosts fail	from outside, after start: nc -vz gate.example.com 50061	Open the ports
The name resolves to the node	getent hosts gate.example.com	Certificates
A certificate pair for gate.example.com (Let's Encrypt or your own CA)	openssl x509 -in fullchain.pem -noout -ext subjectAltName	Certificates
OIDC client gdsgate	as on the single host	[oidc]
production only: wrapped keyrings		[security.integrity]

Connector routes

Route	Ports published	Here
tunneled through the client port (recommended)	50061–50063	Used below. public_join in both files; connectors register and renew on 50061: gdsgate join --enroll-endpoint https://gate.example.com:50061.
tunneled through the join relay	50061–50064	IP-only entry and cluster-issued registration TLS; follow the changes below. Connectors use 50064, the approved CA pin and enroll.gdsgate.internal.
fronted	50061–50063	Step 6 with authority:connector.authority_transport=fronted. Connectors: --enroll-endpoint https://gate.example.com:50061 --route fronted.
direct		Not with this layout: Authority binds its address on the overlay network, which published ports do not reach.

Full reference: split deployment, routes.

For an IP-only entry, apply these changes to the DNS example:

1. Use --domain 203.0.113.10 and --join-relay on both preparations in step 5. Omit all four certificate/key flags and skip public certificate installation. Keep the Authority state-volume mount for its generated registration pair.
2. Set Proxy's enroll.transport_sni=enroll.gdsgate.internal with setup change.
3. Create a Swarm config with docker config create transport_ca_1 config/transport-ca.pem. In stack.yaml, declare transport_ca: {external: true, name: transport_ca_1} under configs, and mount it in Proxy as /etc/gdsgate/transport-ca.pem with mode 0444. Remove the tls_cert/tls_key definitions, mounts and creation commands.
4. Uncomment the Proxy's published 50064 port. Register Connectors through https://203.0.113.10:50064 using the printed cluster CA fingerprint.

The TOML snapshots below show DNS mode. Keep the files generated for your selected mode. Registration trust.

Open the ports

Open the published ports on the node before you install: 50061, 50062, 50063, and 50064 through the join relay. The node's firewall, and a network firewall in front of it such as a cloud security group, are the administrator's responsibility; docker stack deploy does not change them. While a port is closed, gdsgate join on a connector host and gdsgate connect on a workstation fail. What each port carries: Endpoints.

Firewall	Command
ufw	ufw allow 50061:50063/tcp
firewalld	firewall-cmd --permanent --add-port=50061-50063/tcp && firewall-cmd --reload
nftables, the inet filter table of /etc/nftables.conf	nft insert rule inet filter input tcp dport 50061-50063 accept, and the same rule in /etc/nftables.conf

Published ports

mode	Address the Proxy sees	TLS and server name	Here
host	the caller's	unchanged	stack.yaml. The port opens on the node that runs the task; the placement label keeps it on the node gate.example.com resolves to.
ingress (Swarm's default)	an address of the ingress network, 10.0.0.x	unchanged	Not used. The audit's src_ip and the per-source-IP limits of the registration listeners see one address for every caller.

Configure

1. The swarm and the label that pins the stateful services to this node:

   docker swarm init --advertise-addr 203.0.113.10
   docker node update --label-add gdsgate=state "$(docker info -f '{{.Swarm.NodeID}}')"
   

2. The directory, the store password and the certificate pair. The image runs as uid 10001; the files it reads belong to it:

   install -d -m 0700 /opt/gdsgate && cd /opt/gdsgate
   install -d -m 0700 secrets
   openssl rand -hex 24 | tr -d '\n' > secrets/store_password
   openssl rand -hex 24 > secrets/postgres_password
   chmod 0600 secrets/postgres_password
   install -m 0600 -o 999 -g 999 secrets/store_password secrets/postgres_store_password
   chown 10001:10001 secrets/store_password
   chmod 0600 secrets/store_password
   curl -fsS https://gate.gds.dev/docs/deploy/files/cluster-ha/postgres-init.sh -o postgres-init.sh
   chmod 0755 postgres-init.sh
   install -d -m 0700 -o 10001 -g 10001 config config/tls
   install -m 0644 -o 10001 -g 10001 fullchain.pem config/tls/fullchain.pem
   install -m 0600 -o 10001 -g 10001 privkey.pem config/tls/privkey.pem
   install -m 0600 /dev/null proxy.env
   

   A Swarm secret cannot be read back. config/ and secrets/ stay on the node as the copy for backup.

3. /opt/gdsgate/store.yaml:

   services:
     postgres:
       image: postgres:17
       environment:
         POSTGRES_USER: postgres
         POSTGRES_DB: gdsgate
         POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
       configs:
         - {source: postgres_init, target: /docker-entrypoint-initdb.d/10-gdsgate.sh, mode: 0555}
       secrets:
         - postgres_password
         - {source: postgres_store_password, target: store_password, uid: "999", mode: 0400}
       volumes:
         - pgdata:/var/lib/postgresql/data
       networks: [gdsgate]
       healthcheck:
         test: ["CMD", "pg_isready", "-h", "127.0.0.1", "-U", "gdsgate", "-d", "gdsgate"]
         interval: 5s
         retries: 20
       deploy:
         placement:
           constraints: [node.labels.gdsgate == state]
   
   configs:
     postgres_init:
       file: ./postgres-init.sh
   
   secrets:
     postgres_store_password:
       external: true
     postgres_password:
       external: true
     store_password:
       external: true
   
   networks:
     gdsgate:
       external: true
   
   volumes:
     pgdata:
   

4. The network, the store secret, the store:

   docker network create --driver overlay --attachable gdsgate
   docker secret create postgres_store_password secrets/postgres_store_password
   docker secret create postgres_password secrets/postgres_password
   docker secret create store_password secrets/store_password
   docker stack deploy --detach=false -c store.yaml gdsgate-store
   

   PostgreSQL initializes as postgres; the init script creates application role gdsgate without superuser or role-administration privileges. Its port remains private to the overlay network.

   --attachable lets the one-off container of the next step reach postgres.

5. Write both configurations with the image. Authority's run opens the store:

   docker run --rm --network gdsgate \
     -v gdsgate_authority-state:/var/lib/gdsgate \
     -v "$PWD/config:/etc/gdsgate" \
     -v "$PWD/secrets/store_password:/run/secrets/store_password:ro" \
     registry.example.com/gdsgate:v0.10.0 setup prepare \
     --domain gate.example.com --security-profile pilot \
     --role authority --authority-bind authority \
     --store-url postgres://gdsgate@postgres:5432/gdsgate \
     --store-password-file /run/secrets/store_password \
     --enroll-cert /etc/gdsgate/tls/fullchain.pem --enroll-key /etc/gdsgate/tls/privkey.pem \
     --oidc-issuer https://idp.example.com/realms/gdsgate --oidc-client-id gdsgate \
     --unit-dir /tmp/units
   docker run --rm -v "$PWD/config:/etc/gdsgate" \
     registry.example.com/gdsgate:v0.10.0 setup prepare \
     --domain gate.example.com --security-profile pilot \
     --role proxy --authority-host authority --state-dir /var/lib/gdsgate/proxy-state \
     --enroll-cert /etc/gdsgate/tls/fullchain.pem \
     --tls-cert /etc/gdsgate/tls/fullchain.pem --tls-key /etc/gdsgate/tls/privkey.pem \
     --oidc-issuer https://idp.example.com/realms/gdsgate --oidc-client-id gdsgate \
     --unit-dir /tmp/units
   

   --authority-bind authority: Authority binds the address of its own hostname, which stack.yaml sets to authority. --unit-dir /tmp/units keeps the systemd units inside the throwaway container. The Proxy dials authority, so its run writes transport_sni = "gate.example.com", the name on the certificate.

6. The audit export:

   docker run --rm -v "$PWD/config:/etc/gdsgate" \
     registry.example.com/gdsgate:v0.10.0 setup change \
     --set authority:audit.export_path=/var/lib/gdsgate/audit.jsonl
   

   Step 5 writes the route (authority_transport, public_join) and authority_enroll, where the Proxy forwards the registrations it takes on 50061.

7. The files as written, comments shortened. config/authority.toml:

   store_url           = "postgres://gdsgate@postgres:5432/gdsgate"  # the gdsgate-store stack
   store_password_file = "/run/secrets/store_password"               # the Swarm secret
   
   [security]
   profile = "pilot"
   
   [security.integrity]
   control_keyring = "/etc/gdsgate/control.keys"   # Swarm secrets, mounted here
   audit_keyring   = "/etc/gdsgate/audit.keys"
   
   [endpoints]
   authority        = "authority:50051"            # bound: the task's own address
   authority_enroll = "authority:50050"
   proxy_public     = "gate.example.com:50061"     # told to joining nodes
   proxy_internal   = "gate.example.com:50062"
   proxy_ws         = "gate.example.com:50063"
   
   [telemetry]
   metrics_listen = "127.0.0.1:9090"               # the image's health check
   
   [authority]
   enroll_cert_file = "/etc/gdsgate/tls/fullchain.pem"
   enroll_key_file  = "/etc/gdsgate/tls/privkey.pem"
   
   [policy]
   path = "/etc/gdsgate/policy.cedar"
   
   [oidc]
   issuer    = "https://idp.example.com/realms/gdsgate"
   client_id = "gdsgate"
   
   [connector]
   authority_transport = "tunneled"                # route told to connectors
   
   [proxy]
   public_join = true                              # renewal address told: proxy_public
   
   [audit]
   export_path = "/var/lib/gdsgate/audit.jsonl"    # in the authority-state volume
   

   config/proxy.toml. The three listeners bind every interface of the container (0.0.0.0:50061 to 0.0.0.0:50063); stack.yaml publishes them:

   [endpoints]
   # proxy_public, proxy_internal, proxy_ws: 0.0.0.0:50061 to 0.0.0.0:50063
   authority        = "authority:50051"
   authority_enroll = "authority:50050"            # where public_join forwards registration
   
   [proxy]
   public_cert_file = "/etc/gdsgate/tls/fullchain.pem"
   public_key_file  = "/etc/gdsgate/tls/privkey.pem"
   public_join      = true                         # registration and renewal on 50061
   
   [enroll]
   endpoint      = "https://authority:50050"
   state_dir     = "/var/lib/gdsgate/proxy-state"  # in the proxy-state volume
   transport_sni = "gate.example.com"
   
   [oidc]
   issuer    = "https://idp.example.com/realms/gdsgate"
   client_id = "gdsgate"
   
   [security]
   profile = "pilot"
   
   [telemetry]
   metrics_listen = "127.0.0.1:9090"
   

8. Check Authority's file:

   docker run --rm -v "$PWD/config:/etc/gdsgate:ro" \
     -v "$PWD/secrets/store_password:/run/secrets/store_password:ro" \
     registry.example.com/gdsgate:v0.10.0 config check /etc/gdsgate/authority.toml --role authority
   

   Expected: 0 error(s), 0 warning(s). The Proxy's file is checked once its token is in proxy.env (Start).

9. The secrets and configs stack.yaml names:

   docker secret create control_keys config/control.keys
   docker secret create audit_keys config/audit.keys
   docker secret create tls_key_1 config/tls/privkey.pem
   docker config create tls_cert_1 config/tls/fullchain.pem
   docker config create authority_toml_1 config/authority.toml
   docker config create proxy_toml_1 config/proxy.toml
   docker config create policy_cedar_1 config/policy.cedar
   

   A config or secret cannot change. The suffix is its version: Change a file.

10. /opt/gdsgate/stack.yaml:

    x-gdsgate: &gdsgate
      image: registry.example.com/gdsgate:v0.10.0   # change it to upgrade
      read_only: true
      cap_drop: [ALL]
      networks: [gdsgate]
    
    x-deploy: &deploy
      placement:
        constraints: [node.labels.gdsgate == state] # the node with the volumes and the name
      update_config:
        order: stop-first                           # host-mode ports: one task at a time
        failure_action: rollback
    
    services:
      authority:
        <<: *gdsgate
        hostname: authority                         # the name authority.toml binds
        command: ["--config", "/etc/gdsgate/authority.toml", "authority"]
        environment:
          GDSGATE_ADMIN_IDENTITY_DIR: /var/lib/gdsgate/admin   # for the admin CLI in this container
          GDSGATE_ADMIN_ENDPOINT: https://authority:50051
        configs:
          - { source: authority_toml, target: /etc/gdsgate/authority.toml, uid: "10001", mode: 0400 }
          - { source: policy_cedar, target: /etc/gdsgate/policy.cedar, uid: "10001", mode: 0400 }
          - { source: tls_cert, target: /etc/gdsgate/tls/fullchain.pem, uid: "10001", mode: 0444 }
        secrets:
          - { source: store_password, uid: "10001", mode: 0400 }
          - { source: control_keys, target: /etc/gdsgate/control.keys, uid: "10001", mode: 0400 }
          - { source: audit_keys, target: /etc/gdsgate/audit.keys, uid: "10001", mode: 0400 }
          - { source: tls_key, target: /etc/gdsgate/tls/privkey.pem, uid: "10001", mode: 0400 }
        volumes:
          - authority-state:/var/lib/gdsgate
          - { type: tmpfs, target: /tmp }
        deploy: *deploy
    
      proxy:
        <<: *gdsgate
        command: ["--config", "/etc/gdsgate/proxy.toml", "proxy"]
        env_file: [./proxy.env]                     # the one-time token, first start only
        configs:
          - { source: proxy_toml, target: /etc/gdsgate/proxy.toml, uid: "10001", mode: 0400 }
          - { source: tls_cert, target: /etc/gdsgate/tls/fullchain.pem, uid: "10001", mode: 0444 }
        secrets:
          - { source: tls_key, target: /etc/gdsgate/tls/privkey.pem, uid: "10001", mode: 0400 }
        volumes:
          - proxy-state:/var/lib/gdsgate
          - { type: tmpfs, target: /tmp }
        ports:
          - { target: 50061, published: 50061, mode: host }   # clients; registration and renewal
          - { target: 50062, published: 50062, mode: host }   # connector tunnels
          - { target: 50063, published: 50063, mode: host }   # connector tunnels, WebSocket fallback
          # - { target: 50064, published: 50064, mode: host } # join relay, with --join-relay only
        deploy: *deploy
    
    configs:
      authority_toml: { external: true, name: authority_toml_1 }
      proxy_toml:     { external: true, name: proxy_toml_1 }
      policy_cedar:   { external: true, name: policy_cedar_1 }
      tls_cert:       { external: true, name: tls_cert_1 }
    
    secrets:
      store_password: { external: true }
      control_keys:   { external: true }
      audit_keys:     { external: true }
      tls_key:        { external: true, name: tls_key_1 }
    
    networks:
      gdsgate:
        external: true
    
    volumes:
      authority-state:
      proxy-state:
    

    The image's health check requests /readyz on 127.0.0.1:9090. Swarm replaces a task that stays unhealthy, and failure_action: rollback returns an update whose tasks do not become healthy to the previous specification. docker stack deploy ignores depends_on and does not read .env.

Start

docker stack deploy -c stack.yaml gdsgate
until docker service ls --filter name=gdsgate_authority --format '{{.Replicas}}' | grep -qx 1/1; do sleep 2; done
gdsgate_admin() {
  docker exec "$(docker ps -q -f label=com.docker.swarm.service.name=gdsgate_authority)" \
    gdsgate --config /etc/gdsgate/authority.toml authority "$@"
}
gdsgate_admin bootstrap-admin --output-dir /var/lib/gdsgate/admin
printf 'GDSGATE_ENROLL_TOKEN=%s\n' "$(gdsgate_admin create-token --role proxy)" > proxy.env
docker run --rm --env-file proxy.env -v "$PWD/config:/etc/gdsgate:ro" \
  registry.example.com/gdsgate:v0.10.0 config check /etc/gdsgate/proxy.toml --role proxy
docker stack deploy --detach=false -c stack.yaml gdsgate

Until proxy.env holds a token, the proxy task exits and Swarm starts it again: this node holds no enrolled identity in docker service logs gdsgate_proxy. The second docker stack deploy gives it the token. The Proxy check expects 0 error(s), 0 warning(s).

gdsgate_admin runs an administrative command in the Authority task; define it in each new shell. The administrative identity lasts one day; the same bootstrap-admin command renews it.

Connectors

The catalogue rows and one token per connector:

gdsgate_admin set-resource demo-ssh --kind ssh --onward-auth local
gdsgate_admin set-resource app-db --kind postgres
gdsgate_admin create-token --role connector > connector.token

On another hostA service in this swarm

As root, with connector.token copied there:

gdsgate join gate.example.com --name conn-1 \
  --enroll-endpoint https://gate.example.com:50061 \
  --serve demo-ssh=ssh < connector.token
gdsgate --config /etc/gdsgate/connector.toml service install connector
systemctl start gdsgate-connector
rm connector.token

With your own CA and the root not in the connector host's store, add --enroll-ca /etc/gdsgate/root.pem. More platforms: VPS, a connector on another host.

For resources on an overlay network, here apps with a service app-db. The join runs once, in a container, and writes into a volume on this node:

docker volume create gdsgate-connector
docker run --rm -i -v gdsgate-connector:/var/lib/gdsgate \
  registry.example.com/gdsgate:v0.10.0 join gate.example.com --name swarm-conn-1 \
  --enroll-endpoint https://gate.example.com:50061 \
  --output /var/lib/gdsgate/connector.toml --state-dir /var/lib/gdsgate/connector \
  --metrics-listen 127.0.0.1:9090 \
  --serve app-db=postgres@app-db:5432 < connector.token
rm connector.token

--metrics-listen serves the image's health check. Without it Swarm replaces the task every two minutes.

/opt/gdsgate/connector.yaml:

services:
  connector:
    image: registry.example.com/gdsgate:v0.10.0
    command: ["--config", "/var/lib/gdsgate/connector.toml", "connector"]
    read_only: true
    cap_drop: [ALL]
    networks: [apps]
    volumes:
      - connector-state:/var/lib/gdsgate
      - { type: tmpfs, target: /tmp }
    deploy:
      placement:
        constraints: [node.labels.gdsgate == state]   # the node that holds the volume

networks:
  apps:
    external: true

volumes:
  connector-state:
    external: true
    name: gdsgate-connector

docker stack deploy --detach=false -c connector.yaml gdsgate-connector

Reference: gdsgate join, [connector].

Change a file

A config or secret is created once and never edited. A changed file becomes a new object with the next suffix, and the stack names it:

docker config create proxy_toml_2 config/proxy.toml
sed -i 's|name: proxy_toml_1|name: proxy_toml_2|' stack.yaml
docker stack deploy --detach=false -c stack.yaml gdsgate
docker config rm proxy_toml_1

The service restarts with the new file. A renewed certificate pair is the same with tls_cert_2 and tls_key_2: both services restart.

Upgrade and rollback

sed -i 's|registry.example.com/gdsgate:v0.10.0|registry.example.com/gdsgate:v0.10.1|' stack.yaml
docker stack deploy --detach=false -c stack.yaml gdsgate

Action	Command
One service, without the stack file	docker service update --detach=false --image registry.example.com/gdsgate:v0.10.1 gdsgate_proxy; the next docker stack deploy sets the file's tag again.
Back to the previous specification	docker service rollback --detach=false gdsgate_proxy, then gdsgate_authority.
Back by tag	The previous tag in stack.yaml, then docker stack deploy --detach=false -c stack.yaml gdsgate.

The volumes stay. The Proxy starts with the identity in proxy-state and does not use the token again. Read the upgrade guide first.

Backup

What	How
The store	docker exec "$(docker ps -q -f label=com.docker.swarm.service.name=gdsgate-store_postgres)" pg_dump -U gdsgate -Fc gdsgate > gdsgate.dump
The keyrings, the configuration, the certificate pair	/opt/gdsgate/config, kept apart from the store dump
Database administrator and application passwords	/opt/gdsgate/secrets; protected storage

A store dump without the keyrings cannot be verified. See Keeping keyrings out of the database backup.

Verify

Command	Expected	If not: cause → fix
docker service ls	gdsgate-store_postgres, gdsgate_authority, gdsgate_proxy each 1/1	docker service ps --no-trunc gdsgate_authority names the failure; see the rows below.
docker service logs gdsgate_authority	auth service serving gRPC ... enroll_mode="TLS from the configured certificate"	cannot bind `[endpoints].authority` = authority:50051: ... Name does not resolve → hostname: authority missing from the service. Permission denied (os error 13) → a config or secret without uid: "10001".
docker service logs gdsgate_authority	settings advertised to joining nodes advert=... auth tunneled at none, renew https://gate.example.com:50061	renew none → [proxy].public_join missing in authority.toml. renew https://gate.example.com:50064 without the join relay route → [endpoints].proxy_join set in authority.toml.
docker service logs gdsgate_proxy	enrolled and persisted a new transport identity, proxy serving node registration on the public listener ... enrollment=authority:50050 ... forward="TLS", proxy serving gRPC (mTLS (enrolled), public TLS from the configured certificate)	enrollment=127.0.0.1:50050 → authority_enroll missing from proxy.toml: step 5 run without --authority-host authority. this node holds no enrolled identity → no token in proxy.env, or the second docker stack deploy not run. certificate refused (...); checked for the name ... → the certificate does not carry gate.example.com, or transport_sni differs. unknown token → a new token into proxy.env, then docker stack deploy.
docker service ps --no-trunc gdsgate_proxy	one task Running	host-mode port already in use → another process holds 50061–50063 on the node. unhealthy container → the log of that task.
gdsgate_admin list-nodes	admin, proxy rows, active
On the connector host: gdsgate join ... --enroll-endpoint https://gate.example.com:50061	ok: joined gate.example.com as conn-1 · role connector	See VPS verify.
On the connector host: grep renew_endpoint /etc/gdsgate/connector.toml	renew_endpoint = "https://gate.example.com:50061"	:50064 → see the Authority row above; fix authority.toml (Change a file), then gdsgate join ... --force with a new token, and service install connector again.
docker service logs gdsgate_proxy \| grep 'tunnel registered'	connector_label=conn-1 bound=["demo-ssh"]	bound=[] persisting → the catalogue row is missing.
gdsgate_admin list-offered	demo-ssh ssh ... yes, app-db postgres ... yes
docker service ps --no-trunc gdsgate-connector_connector	one task Running	unhealthy container every two minutes → [telemetry].metrics_listen missing in the connector's file: --metrics-listen 127.0.0.1:9090.
Workstation: gdsgate connect gate.example.com	ok: wrote ~/.gdsgate/config.toml	See Certificates.
After an upgrade: docker service logs gdsgate_proxy	reusing persisted transport identity	enrolled and persisted a new transport identity → the proxy-state volume was lost; revoke the old Proxy row.

After these checks pass, complete the access checks from a client workstation.

Profile

pilot, the same matrix rows as the single host.