Gateway API for Teleport K8S

If you've been reading some of my mutterings for a bit you know I run Teleport and love it. Over the past couple of years I've been running Teleport within my OpenStack cloud and while that's been great, I've wanted to push Teleport into my Kubernetes cluster, however, I've been reluctant to do it because VM was working so well. That said, with the development of Genestack we've been using the Gateway API and I found a desire to tinker, a need to learn. So I did what any self-respecting nerd would, and decided to break an otherwise stable and perfectly functioning service so that I can redeploy in a new and different way.
Background
In Kubernetes the Gateway API has been released and introduces some much needed functionality that carries it beyond where the basic Ingress provider could. Read more about the Gateway API here. Within Genestack we're using the NGINX Gateway Fabric (by default) to provide access to our API services. This system scalable and able to be used in a lot different ways across the Kubernetes cluster environment.
Teleport provides a Kubernetes deployment model which makes several assumptions about how and where systems are deployed. That said, the core functionality of Teleport is able to be serviced on any Kubernetes environment so long as the basic access features are able to satisfied. This is where the Gateway API comes in, for the purpose of my environment, Teleport will be deployed in the teleport-cluster
namespace via helm
and the Gateway API will service as our "load balancer."
In the following examples you must change your.domain.tld to the domain you wish to use within the teleport cluster.
System Overview
At the time of this writing here's the overview of my environment
- Kubernetes
v1.28.6
- NGINX Gateway Fabric
1.4.0
- The NGINX Gateway Fabric deployment was done with experimental features enabled
- Service deployment is using
apiVersion
v1alpha2
- Teleport 16.4.x
System Setup
I'm not going into the actual deployment of the gateway services or Teleport itself, both of those processes are well documented upstream.
That said, I will share my teleports helm values file because getting this all working is Kind of a nightmare and I wouldn't want anyone to have to go through this discovery adventure...
clusterName: teleport.your.domain.tld
chartMode: standalone
proxyListenerMode: multiplex
#proxyProtocol: "off"
acme: false
publicAddr:
- teleport.your.domain.tld:443
authentication:
type: github
localAuth: true
proxy:
highAvailability:
replicaCount: 1
highAvailability:
replicaCount: 1
podDisruptionBudget:
enabled: true
certManager:
enabled: true
issuerName: "letsencrypt-production"
addCommonName: true
addPublicAddrs: true
ingress:
enabled: false
log:
level: INFO
High level this deployment config is making some assumptions that cert-manger is deployed within the Kubernetes cluster and will be usingletsencrypt
to issue certificates. This is also setup to usegithub
authentication to users. Change these values to suit your needs.
Gateway Setup
This post is all about using the Gateway API for our services in conjunction with Genestack; effectively combining our Kubernetes and OpenStack clusters to serve a greater purpose. So this post is mostly covering the deployment of the routes and listeners I use within those systems to support the workload, Teleport.
Routing
The first setup is to setup our routes. The routes allow Teleport to terminate TLS within the pods on port 443 and ensure that any connection going to the proxy service are redirected to a TLS connection. To do this achieve the goals, create a simple manifest named teleport-tlsroute.yaml
and apply it to the environment.
apiVersion
is set to v1alpha2
. Teleport is running in the teleport-cluster
namespace and our gateway services named flex-gateway
are running in the nginx-gateway
namespace.---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: custom-teleport-cluster-http-gateway-route
namespace: teleport-cluster
spec:
parentRefs:
- name: flex-gateway
sectionName: teleport-cluster-http
namespace: nginx-gateway
hostnames:
- "teleport.your.domain.tld"
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TLSRoute
metadata:
name: custom-teleport-cluster-gateway-route
namespace: teleport-cluster
spec:
parentRefs:
- name: flex-gateway
sectionName: teleport-cluster-https
namespace: nginx-gateway
hostnames:
- "teleport.your.domain.tld"
rules:
- backendRefs:
- name: teleport-cluster
port: 443
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TLSRoute
metadata:
name: custom-teleport-cluster-gateway-wild-route
namespace: teleport-cluster
spec:
parentRefs:
- name: flex-gateway
sectionName: teleport-cluster-wild-https
namespace: nginx-gateway
hostnames:
- "*.teleport.your.domain.tld"
rules:
- backendRefs:
- name: teleport-cluster
port: 443
Example manifest
Certificate Issuer
Here's the certificate issuer I'm using, this is made so that the Gateway API can be used to validate http and dns certificate requests. This file is saved at teleport-issuer-gateway.yaml
.
---
apiVersion: v1
kind: Secret
metadata:
name: cloudflare-api-token-secret
namespace: teleport-cluster
type: Opaque
stringData:
api-token: SUPER_SECRETE_TOKEN_FROM_CLOUDFLARE
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: letsencrypt-production
namespace: teleport-cluster
spec:
acme:
# The ACME server URL
server: https://acme-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: your-name@your.domain.tld
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-production
# Enable the HTTP-01 challenge provider
solvers:
- http01:
gatewayHTTPRoute:
parentRefs:
- name: flex-gateway
namespace: nginx-gateway
kind: Gateway
- dns01:
cloudflare:
apiTokenSecretRef:
name: cloudflare-api-token-secret
key: api-token
Example teleport-issuer-gateway.yaml
file
Take note that this is using the CloudFlare API to validate DNS01 challenges from LetsEncrypt. If you do not use LetsEncrypt or do not use CloudFlare, be sure to adjust this file according to your environment.
To apply the manifest use a simple kubectl
command like so.
kubectl apply -f teleport-tlsroute.yaml teleport-issuer-gateway.yaml
Listener Patches
Next we patch the listeners to include the new TLS protocol and route. To run this patch create a new patch file named teleport-tls-listener.json
with the following content.
[
{
"op": "add",
"path": "/spec/listeners/-",
"value": {
"name": "teleport-cluster-http",
"port": 80,
"protocol": "HTTP",
"hostname": "teleport.your.domain.tld",
"allowedRoutes": {
"namespaces": {
"from": "All"
}
}
}
},
{
"op": "add",
"path": "/spec/listeners/-",
"value": {
"name": "teleport-cluster-https",
"port": 443,
"protocol": "TLS",
"hostname": "teleport.your.domain.tld",
"allowedRoutes": {
"namespaces": {
"from": "All"
}
},
"tls": {
"mode": "Passthrough"
}
}
},
{
"op": "add",
"path": "/spec/listeners/-",
"value": {
"name": "teleport-cluster-wild-http",
"port": 80,
"protocol": "HTTP",
"hostname": "*.teleport.your.domain.tld",
"allowedRoutes": {
"namespaces": {
"from": "All"
}
}
}
},
{
"op": "add",
"path": "/spec/listeners/-",
"value": {
"name": "teleport-cluster-wild-https",
"port": 443,
"protocol": "TLS",
"hostname": "*.teleport.your.domain.tld",
"allowedRoutes": {
"namespaces": {
"from": "All"
}
},
"tls": {
"mode": "Passthrough"
}
}
}
]
Example Patch file teleport-tls-listener.json
Apply the patch to the running gateway instance
kubectl patch -n nginx-gateway gateway flex-gateway --type=json --patch-file=teleport-tls-listener.json
Validate Functionality
Once the patch is applied run a describe on the gateway service to validate functionality.
kubectl -n nginx-gateway describe gateways.gateway.networking.k8s.io flex-gateway
The output will show you all of the listeners running, the output your looking for will be something like so.
Attached Routes: 1
Conditions:
Last Transition Time: 2024-08-24T16:43:53Z
Message: Listener is accepted
Observed Generation: 18
Reason: Accepted
Status: True
Type: Accepted
Last Transition Time: 2024-08-24T16:43:53Z
Message: Listener is programmed
Observed Generation: 18
Reason: Programmed
Status: True
Type: Programmed
Last Transition Time: 2024-08-24T16:43:53Z
Message: All references are resolved
Observed Generation: 18
Reason: ResolvedRefs
Status: True
Type: ResolvedRefs
Last Transition Time: 2024-08-24T16:43:53Z
Message: No conflicts
Observed Generation: 18
Reason: NoConflicts
Status: False
Type: Conflicted
Name: teleport-cluster-https
Supported Kinds:
Group: gateway.networking.k8s.io
Kind: TLSRoute
Example Describe for a functioning TLSRoute
Accessing Teleport
Assuming everything is working normally, you should now be able to access Teleport on the domain defined, in the example that would be your.domain.tld
.
There's absolutely no need for an load balancer (AWS or otherwise) now that we have the power of the Gateway API.

From here, simply access Teleport normally. If you're on the setup process, proceed to Step 2/2 to generate some local users.
Client Login
Because Teleport is attempting to use a Proxy, you will need to set the environment variable TELEPORT_TLS_ROUTING_CONN_UPGRADE
with a value of true
. This environment variable can be defined on the command line, or in your terminal environment RC file: .bashrc
, .zshrc
, etc... This export can also be used on the command line at auth runtime, like so:
TLSRoute
. Until then, I'm setting the option accordingly.# Running the command
TELEPORT_TLS_ROUTING_CONN_UPGRADE=true tsh login --proxy teleport.your.domain.tld
# Command Output
If browser window does not open automatically, open it by clicking on the link:
http://127.0.0.1:49398/2147e66f-5ee3-4e2b-8e2c-ddea29967373
> Profile URL: https://teleport.your.domain.tld:443
Logged in as: YOUR_SPECIAL_USER
Cluster: teleport.your.domain.tld
Roles: access
Logins: YOUR_SPECIAL_USER
Kubernetes: enabled
Valid until: 2024-08-26 05:00:04 -0500 CDT [valid for 12h0m0s]
Extensions: login-ip, permit-agent-forwarding, permit-port-forwarding, permit-pty, private-key-policy
Example environment login settings
That's is, That's all
Simply blog post on how I use the Kubernetes Gateway API to run my Teleport environment, now running on the same cluster as my OpenStack Cloud, all powered by Genestack.