Kubernetes¶

Authority and Proxy in a Kubernetes cluster, from the Helm chart or the kustomize base. PostgreSQL outside the chart. Connectors outside the cluster or in it. Profile: pilot.

Placeholder	Meaning
`gate.example.com`	The name clients and connectors dial. Resolves to the external address of Service `gdsgate-proxy`.
`oci://registry.example.com/charts/gdsgate`	The Helm chart. Chart version = product version.
`registry.example.com/gdsgate:v0.10.0`	The release image. See Running the image.
`postgres://gdsgate@postgres.example.com:5432/gdsgate`	The store.
`https://idp.example.com/realms/gdsgate`	The OIDC issuer.

Before you start¶

Item	How to check	Reference
Kubernetes 1.25 or later	`kubectl version`
Helm 3.8 or later (Helm tab)	`helm version`
A LoadBalancer implementation (cloud, MetalLB, k3s ServiceLB) passing TCP through	`kubectl get svc -A` shows an `EXTERNAL-IP` on a `LoadBalancer` Service	Services
The `gdsgate-proxy` ports open inbound to the load balancer before you install. The administrator opens them; without them `join` and `connect` from outside the cluster fail	from outside, after start: `nc -vz gate.example.com 50061`	Open the ports
A default StorageClass	`kubectl get storageclass` prints `(default)`
The nodes pull the image	`docker pull registry.example.com/gdsgate:v0.10.0` from a node's network	Running the image
PostgreSQL, database and role `gdsgate`, reachable from the pods	`kubectl run psql --rm -it --restart=Never --image=postgres:17 -- psql postgres://gdsgate@postgres.example.com:5432/gdsgate -c 'select 1'`	State store
A certificate pair for `gate.example.com` (Let's Encrypt or your own CA)	`openssl x509 -in fullchain.pem -noout -ext subjectAltName`	Certificates
OIDC client `gdsgate`	as on the single host	`[oidc]`
`gdsgate` on the workstation	`gdsgate --version`	Installation
`production` only: wrapped keyrings		`[security.integrity]`

Services¶

Service	Type	Ports	Serves
`gdsgate-proxy`	`LoadBalancer`, `externalTrafficPolicy: Local`	`50061` client port, `50062` tunnels, `50063` tunnels over WebSocket; `50064` join relay when enabled	clients, connectors. TLS passthrough: the Proxy terminates TLS.
`gdsgate-authority`	`ClusterIP`	`50050` registration, `50051` control plane	the Proxy
`gdsgate-authority-headless`	headless	`50050`, `50051`	pod names of the StatefulSet

An Ingress with TLS passthrough does not serve the client port: it routes by server name, and renewal and control-plane calls arrive there under gdsgate.internal.

Open the ports¶

Open the ports of gdsgate-proxy before you install: 50061, 50062, 50063, and 50064 with the join relay. Where they are opened depends on the load balancer, and it is the administrator's responsibility; the chart and the base do not change it. While a port is closed, gdsgate join on a connector host outside the cluster and gdsgate connect on a workstation fail. What each port carries: Endpoints.

Load balancer	Where
A cloud's	the security group or firewall rule of the load balancer and the nodes
MetalLB, k3s ServiceLB	the firewall of each node that announces the address: `ufw allow 50061:50063/tcp`, `firewall-cmd --permanent --add-port=50061-50063/tcp && firewall-cmd --reload`, or `nft insert rule inet filter input tcp dport 50061-50063 accept`

gdsgate-authority stays inside the cluster; nothing outside needs 50050 or 50051.

Connector routes¶

Route	`gdsgate-proxy` ports	Helm values	kustomize files	Connector	When
`tunneled` (recommended)	`50061`, `50062`, `50063`	`route: tunneled`	as in the base	`gdsgate join --enroll-endpoint https://gate.example.com:50061`	Connectors anywhere. Registration and renewal on the client port; control-plane calls in the tunnel.
`tunneled`, join relay	`50061`–`50064`	`route: tunneled`, `joinRelay: true`	`authority.toml`: `proxy_join = "gate.example.com:50064"`; Proxy: env `GDSGATE_PROXY_JOIN_ADDR`, port `50064` in the Service	`--enroll-endpoint https://gate.example.com:50064`	A separate port for registration. Renewal goes to the relay.
`fronted`	`50061`, `50062`, `50063`	`route: fronted`	`authority.toml`: `authority_transport = "fronted"`	`--enroll-endpoint https://gate.example.com:50061 --route fronted`	Control-plane calls on the client port as well. At most `public_admin_max_connections` (default 8) connectors reach Authority at once.
`direct`		not with this layout			Authority binds the pod address and tells connectors no address.

Keys: public_join, authority_transport, proxy_join. Full reference: split deployment, routes.

Configure¶

Steps 1 to 3 are the same for both tabs.

The namespace and the store password (the file holds the password alone):

kubectl create namespace gdsgate
kubectl -n gdsgate create secret generic gdsgate-store --from-file=password=store.password

The certificate pair as Secret gdsgate-tls:

Source	Command
Files: Let's Encrypt, or a CA the image trusts	`kubectl -n gdsgate create secret tls gdsgate-tls --cert=fullchain.pem --key=privkey.pem`
Files: your own CA	`kubectl -n gdsgate create secret generic gdsgate-tls --type=kubernetes.io/tls --from-file=tls.crt=fullchain.pem --from-file=tls.key=privkey.pem --from-file=ca.crt=root.pem`
cert-manager	a `Certificate` with `secretName: gdsgate-tls` and `dnsNames: [gate.example.com]`. A CA issuer writes `ca.crt` as well.

fullchain.pem: the leaf, then each intermediate. With your own CA the Proxy verifies Authority's registration listener against ca.crt: Helm tls.caKey: ca.crt, kustomize transport_ca in proxy.toml. A renewed Secret reaches the pods within about a minute; the next connection gets the new pair (renewal).

The two keyrings, made on the workstation:

install -d -m 0700 gdsgate-keys
gdsgate authority init --profile pilot --output gdsgate-keys/authority.toml
kubectl -n gdsgate create secret generic gdsgate-keyrings \
  --from-file=gdsgate-keys/control.keys --from-file=gdsgate-keys/audit.keys

gdsgate-keys/authority.toml is not used. Keep gdsgate-keys apart from the store backups (Backup).

The configuration:

Helmkustomize

values.yaml:

domain: gate.example.com
profile: pilot
route: tunneled
store:
  url: postgres://gdsgate@postgres.example.com:5432/gdsgate
oidc:
  issuer: https://idp.example.com/realms/gdsgate
  clientId: gdsgate
tls:
  caKey: ""        # ca.crt with your own CA
policy:
  accessGroup: admins

Value	Default	Sets
`domain`	`gate.example.com`	Addresses Authority tells clients and connectors; the name the Proxy checks on the registration certificate.
`profile`	`pilot`	`[security].profile` of both roles.
`route`	`tunneled`	Connector routes.
`joinRelay`	`false`	Join relay on `50064`.
`store.url`	required	`store_url`. The password comes from Secret `gdsgate-store`.
`oidc.issuer`, `oidc.clientId`	unset, `gdsgate`	`[oidc]` of both roles.
`tls.secretName`, `tls.caKey`	`gdsgate-tls`, unset	The certificate pair; the key holding the CA.
`keyrings.secretName`	`gdsgate-keyrings`	`control.keys`, `audit.keys`.
`policy.accessGroup`	`admins`	The group the seed policy makes the cluster's owner (`--access-group`).
`workload.trustDomain`	unset	`[workload].trust_domain` of Authority; unset: workload identity off. Issuers go to `authority.extraConfig` as `[[workload.issuers]]`. A workload outside the cluster bootstraps through the join relay (`joinRelay: true`): `gdsgate machine-id --endpoint https://gate.example.com:50064`. The client port `50061` does not issue workload identities.
`image.repository`, `image.tag`	the release registry, `v<chart version>`	The image.
`authority.replicas`	`1`	Two or more: `[ha]` on; the Proxy dials `gdsgate-authority-0` and fails over to the others. Registration goes to Service `gdsgate-authority`; every replica serves registration and renewal.
`proxy.service.type`	`LoadBalancer`	`ClusterIP` behind a TCP front of your own.
`proxy.service.annotations`, `loadBalancerSourceRanges`	none	Passed to Service `gdsgate-proxy`.
`authority.extraConfig`, `proxy.extraConfig`	none	TOML appended to the file; sections the chart does not write.
`authority.persistence.size`, `proxy.persistence.size`	`1Gi`	The volume at `/var/lib/gdsgate`.

Check: helm template gdsgate oci://registry.example.com/charts/gdsgate --version 0.10.0 -f values.yaml > /dev/null prints nothing.

packaging/kustomize/base from the release source goes to gdsgate/base; the overlay is gdsgate/overlays/prod:

gdsgate/
  base/                  packaging/kustomize/base, unchanged
  overlays/prod/
    kustomization.yaml
    authority.toml
    proxy.toml

overlays/prod/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: gdsgate
resources:
  - ../../base
images:
  - name: registry.example.com/gdsgate
    newTag: v0.10.0
configMapGenerator:
  - name: gdsgate-authority
    behavior: replace
    files:
      - authority.toml
  - name: gdsgate-proxy
    behavior: replace
    files:
      - proxy.toml

overlays/prod/authority.toml. Authority binds 50050, 50051 and 9090 on every interface of the pod (GDSGATE_AUTH_ADDR, GDSGATE_AUTH_ENROLL_ADDR, GDSGATE_TELEMETRY_METRICS_LISTEN in base/authority.yaml):

store_url           = "postgres://gdsgate@postgres.example.com:5432/gdsgate"
store_password_file = "/etc/gdsgate/store.password"   # Secret gdsgate-store

[security]
profile = "pilot"

[security.integrity]
control_keyring = "/etc/gdsgate/control.keys"         # Secret gdsgate-keyrings
audit_keyring   = "/etc/gdsgate/audit.keys"

[endpoints]
proxy_public   = "gate.example.com:50061"             # told to clients and joining nodes
proxy_internal = "gate.example.com:50062"
proxy_ws       = "gate.example.com:50063"

[connector]
authority_transport = "tunneled"                     # route told to connectors

[proxy]
public_join = true                                   # connectors renew at proxy_public

[authority]
enroll_cert_file = "/etc/gdsgate/tls/fullchain.pem"  # Secret gdsgate-tls
enroll_key_file  = "/etc/gdsgate/tls/privkey.pem"

[policy]
path = "/var/lib/gdsgate/policy.cedar"               # written by the seed-policy init container

[oidc]
issuer    = "https://idp.example.com/realms/gdsgate"
client_id = "gdsgate"

# [workload]
# trust_domain = "gds.example"                       # workload identity; unset: off

[audit]
export_path = "/var/lib/gdsgate/audit.jsonl"

overlays/prod/proxy.toml. The Proxy binds 50061 to 50063 and 9090 on every interface of the pod (GDSGATE_PROXY_PUBLIC_ADDR, GDSGATE_PROXY_INTERNAL_ADDR, GDSGATE_PROXY_WS_ADDR, GDSGATE_TELEMETRY_METRICS_LISTEN in base/proxy.yaml):

[endpoints]
authority        = "gdsgate-authority:50051"
authority_enroll = "gdsgate-authority:50050"

[proxy]
public_cert_file = "/etc/gdsgate/tls/fullchain.pem"  # Secret gdsgate-tls
public_key_file  = "/etc/gdsgate/tls/privkey.pem"
public_join      = true                              # registration and renewal on the client port

[enroll]
endpoint      = "https://gdsgate-authority:50050"
transport_sni = "gate.example.com"                   # the name on the certificate
# transport_ca = "/etc/gdsgate/tls/ca.pem"           # your own CA: ca.crt of gdsgate-tls
state_dir     = "/var/lib/gdsgate/proxy-state"

[security]
profile = "pilot"

[oidc]
issuer    = "https://idp.example.com/realms/gdsgate"
client_id = "gdsgate"

The seed policy's owner group: admins in ACCESS_GROUP of the seed-policy init container; change it with a patch in the overlay.

Check: kubectl kustomize gdsgate/overlays/prod > /dev/null prints nothing.

The pods mount /etc/gdsgate read-only (the files above and the Secrets), a volume at /var/lib/gdsgate and a memory emptyDir at /tmp. They run as uid 10001, group 10001, read-only root file system, no capabilities, RuntimeDefault seccomp. Key reference: [endpoints], [enroll], [proxy], [authority].

Start¶

Install:

Helmkustomize

helm install gdsgate oci://registry.example.com/charts/gdsgate --version 0.10.0 \
  -n gdsgate -f values.yaml

kubectl apply -k gdsgate/overlays/prod

Authority, the administrative identity, and the Proxy's token:

kubectl -n gdsgate rollout status statefulset/gdsgate-authority
kubectl -n gdsgate exec -it statefulset/gdsgate-authority -- bash

In the pod:

gdsgate --config /etc/gdsgate/authority.toml authority bootstrap-admin \
  --output-dir /var/lib/gdsgate/admin
gdsgate --config /etc/gdsgate/authority.toml authority create-token --role proxy
exit

The token into a Secret, and the Proxy restarted:

kubectl -n gdsgate create secret generic gdsgate-proxy-token --from-literal=token=<token>
kubectl -n gdsgate delete pod gdsgate-proxy-0
kubectl -n gdsgate rollout status statefulset/gdsgate-proxy

Until the token Secret exists, gdsgate-proxy-0 restarts: pilot refuses a Proxy with neither a token nor an identity. The Proxy spends the token once and keeps its identity in its volume.

Point gate.example.com at the Service's address:

kubectl -n gdsgate get svc gdsgate-proxy -o jsonpath='{.status.loadBalancer.ingress[0]}'

Administration¶

Every Authority replica serves administrative commands.

Route	Commands	Recorded as	Requires
Workstation, through the gateway	`gdsgate connect gate.example.com`, `gdsgate login`, then `gdsgate authority <command>`	you, and your address	a policy permit for your group: the seed policy gives `policy.accessGroup` connector and Proxy tokens, the catalogue and revocation (`--route`)
An Authority pod, any replica	`kubectl -n gdsgate exec -it gdsgate-authority-<n> -- bash`, then `gdsgate --config /etc/gdsgate/authority.toml authority <command>`	`gdsgate-admin`	`authority bootstrap-admin --output-dir /var/lib/gdsgate/admin` once in that pod; it lasts one day and the same command renews it

A connector outside the cluster¶

The catalogue row and a token, from the workstation:

gdsgate authority set-resource demo-ssh --kind ssh --onward-auth local
gdsgate authority create-token --role connector > connector.token

On the connector host, as root, with connector.token copied there:

gdsgate join gate.example.com --name conn-1 \
  --enroll-endpoint https://gate.example.com:50061 \
  --serve demo-ssh=ssh < connector.token
gdsgate --config /etc/gdsgate/connector.toml service install connector
systemctl start gdsgate-connector
rm connector.token

With your own CA and the root not in the connector host's store, add --enroll-ca /etc/gdsgate/root.pem. Other routes: the Connector column of Connector routes. Windows: VPS, a connector on another host.

A connector in the cluster¶

A StatefulSet beside the resources it serves, registering at Service gdsgate-proxy. The catalogue row and the token as above, the token into a Secret:

gdsgate authority set-resource orders-db --kind postgres
kubectl -n gdsgate create secret generic gdsgate-connector-token \
  --from-literal=token="$(gdsgate authority create-token --role connector)"
kubectl -n gdsgate apply -f connector.yaml

connector.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: gdsgate-connector
data:
  connector.toml: |
    [endpoints]
    proxy_internal = "gdsgate-proxy:50062"
    proxy_ws       = "gdsgate-proxy:50063"

    [enroll]
    endpoint       = "https://gdsgate-proxy:50061"      # public_join on the client port
    renew_endpoint = "https://gdsgate-proxy:50061"
    transport_sni  = "gate.example.com"                 # the name on the certificate
    # transport_ca = "/etc/gdsgate/tls/ca.pem"          # your own CA
    state_dir      = "/var/lib/gdsgate/connector"
    node_name      = "conn-k8s"

    [connector]
    id                  = "conn-k8s"
    authority_transport = "tunneled"

    [[connector.backends]]
    resource = "orders-db"
    kind     = "postgres"
    addr     = "orders-db.orders.svc:5432"

    [security]
    profile = "pilot"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: gdsgate-connector
spec:
  replicas: 1
  serviceName: gdsgate-connector
  selector:
    matchLabels:
      app.kubernetes.io/component: connector
  template:
    metadata:
      labels:
        app.kubernetes.io/component: connector
    spec:
      automountServiceAccountToken: false
      enableServiceLinks: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        runAsGroup: 10001
        fsGroup: 10001
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: connector
          image: registry.example.com/gdsgate:v0.10.0
          args: ["--config", "/etc/gdsgate/connector.toml", "connector"]
          env:
            - name: GDSGATE_TELEMETRY_METRICS_LISTEN
              value: "0.0.0.0:9090"
            - name: GDSGATE_ENROLL_TOKEN
              valueFrom:
                secretKeyRef:
                  name: gdsgate-connector-token
                  key: token
                  optional: true
          readinessProbe:
            httpGet:
              path: /readyz
              port: 9090
          securityContext:
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]
          volumeMounts:
            - name: config
              mountPath: /etc/gdsgate
              readOnly: true
            - name: state
              mountPath: /var/lib/gdsgate
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: config
          projected:
            defaultMode: 0440
            sources:
              - configMap:
                  name: gdsgate-connector
              - secret:
                  name: gdsgate-tls
                  optional: true
                  items:
                    - key: ca.crt
                      path: tls/ca.pem
        - name: tmp
          emptyDir:
            medium: Memory
  volumeClaimTemplates:
    - metadata:
        name: state
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi

In another namespace, dial gdsgate-proxy.gdsgate.svc and copy ca.crt there. renew_endpoint is set in the file: a connector started from its own file renews where the file says. With route: fronted the connector file carries authority_transport = "fronted" and [endpoints].authority = "gdsgate-proxy:50061".

Upgrade¶

Helmkustomize

helm upgrade gdsgate oci://registry.example.com/charts/gdsgate --version 0.10.1 \
  -n gdsgate -f values.yaml

newTag: v0.10.1 in overlays/prod/kustomization.yaml, the base from the v0.10.1 source, then kubectl apply -k gdsgate/overlays/prod.

The pods are replaced; the volumes stay. The Proxy starts with the identity in its volume and does not read the token. Read the upgrade guide first.

Backup¶

What	How
The store	`pg_dump -Fc postgres://gdsgate@postgres.example.com:5432/gdsgate > gdsgate.dump`
The keyrings	`gdsgate-keys/control.keys`, `gdsgate-keys/audit.keys`, kept apart from the store dump; or `kubectl -n gdsgate get secret gdsgate-keyrings -o yaml`
The configuration	`values.yaml` or `overlays/prod`
The Proxy's identity	volume `state-gdsgate-proxy-0`. Lost: a new token, and the old `proxy` row revoked.

A store dump without the keyrings cannot be verified. See Keeping keyrings out of the database backup.

Verify¶

A is kubectl -n gdsgate exec statefulset/gdsgate-authority -- gdsgate --config /etc/gdsgate/authority.toml. The gdsgate authority rows run on the workstation.

Command	Expected	If not: cause → fix
`kubectl -n gdsgate get pods`	every `gdsgate-authority-<n>` and `gdsgate-proxy-0` `1/1 Running`	`CreateContainerConfigError` → a Secret from Configure is missing: `kubectl -n gdsgate describe pod <pod>` names it. `Init:CrashLoopBackOff` on Authority → `kubectl -n gdsgate logs <pod> -c seed-policy`.
`kubectl -n gdsgate logs statefulset/gdsgate-authority`	`auth service serving gRPC ... enroll_mode="TLS from the configured certificate"`	`store` errors → `store.url`, the password in `gdsgate-store`, or PostgreSQL unreachable from the pod.
`A config check /etc/gdsgate/authority.toml --role authority`	`0 error(s), 0 warning(s)`	The finding names the key.
`kubectl -n gdsgate logs gdsgate-proxy-0`	`enrolled and persisted a new transport identity`, `proxy serving node registration on the public listener`, `proxy serving gRPC (mTLS (enrolled), public TLS from the configured certificate)`	Refusal naming `proxy-internal-mtls` → the token Secret is missing: Start, step 2. `certificate refused (...); checked for the name ...` → the certificate does not carry `domain`, or your own CA without `ca.crt`. `unknown token` → a new token into `gdsgate-proxy-token`, then `kubectl -n gdsgate delete pod gdsgate-proxy-0`.
`kubectl -n gdsgate exec gdsgate-proxy-0 -- gdsgate config check /etc/gdsgate/proxy.toml --role proxy`	`0 error(s), 0 warning(s)`
`kubectl -n gdsgate get svc gdsgate-proxy`	an `EXTERNAL-IP`; ports `50061`, `50062`, `50063`	`<pending>` → no LoadBalancer implementation in the cluster.
Workstation: `gdsgate connect gate.example.com`	`ok: wrote ~/.gdsgate/config.toml · gateway <version>`	See Certificates.
`gdsgate authority list-nodes`	`admin`, `proxy` rows, `active`	`PermissionDenied` → your group has no `viewNodes` permit.
Connector host: `gdsgate join gate.example.com ... --enroll-endpoint https://gate.example.com:50061`	`ok: joined gate.example.com as conn-1 · role connector`	`certificate refused` → the connector host does not trust the CA (`--enroll-ca`). Timeout → `gate.example.com` does not resolve to the Service address, or the front drops `50061`. `Connection reset by peer` just after install → the load balancer does not forward yet; run `join` again.
`gdsgate authority list-offered`	`demo-ssh ssh ... yes`, `orders-db postgres ... yes`	Connector log: `declared a resource the cluster is not giving it` persisting → the catalogue row is missing.
`kubectl -n gdsgate logs gdsgate-proxy-0 \\| grep 'tunnel registered'`	`connector_label=conn-1 bound=["demo-ssh"]`	`bound=[]` persisting → as the row above.
`kubectl -n gdsgate logs gdsgate-connector-0`	`tunnel transport connected`	`certificate refused` → `transport_sni` or `transport_ca` in `connector.toml`.
Connector log, from one hour before its certificate expires	`renewed the transport identity`	`node identity revoked` → the row is `revoked` in `gdsgate authority list-nodes`; a new token and `join`. Connection errors → no Authority pod is `Running`.
After an upgrade: `kubectl -n gdsgate logs gdsgate-proxy-0`	`reusing persisted transport identity`	`enrolled and persisted a new transport identity` → the volume was lost; revoke the old `proxy` row.

After these checks pass, complete the access checks from a client workstation.

Profile¶

pilot, the same matrix rows as the single host. For production: keyrings in a key service (authority init --control-keyring), oidc.issuer set, profile: production.