Introduction
Traefik is a high performance reverse proxy and load balancer, written in golang. It’s designed to integrate with service catalogs such as Consul. Other popular reverse proxies such as NGINX could be used but you will see how Traefik works much better for this particular solution.
If you choose to pursue a different solution, you taking it upon yourself to figure out how to make it work. Do not expect that any other product to slot in as an exact replacement.
Environment
For the sake of this guide, assume there are three cluster nodes with addressing such as:
node1 - 10.0.3.101 node2 - 10.0.3.102 node3 - 10.0.3.103
Performing actions as root
The ideal security model dictates that interactively operating as root is incorrect, and that operations should run as a user, using ‘sudo’ to elevate permissions where necessary. Unfortunately, almost everything that needs to be done here will require ‘sudo’, so it will be faster to just become root and run everything as root:
sudo su -
Perform the following steps on one node until instructed otherwise
Install Traefik
First, determine what the current Traefik version is that is available. Visit the following page and note the latest specific version. Do not use ‘latest’ or ‘stable’, always pull the most recent and most specific version (at the time of this writing, it was ‘2.10.7’).
DO NOT install use Traefik 3.x which was recently released. This configuration will not work. Use the latest Traefik 2.x for the time being. This chapter will be updated in the future to support the newer Traefik version.
https://hub.docker.com/_/traefik/tags
With that information, execute the following to create the directory structure for building the new container (and replace any version reference if a newer one exists):
mkdir -p /mnt/vagabond/docker/_build/traefik/2.10.7
cd /mnt/vagabond/docker/_build/traefik/2.10.7
Create the build script (and replace any version reference if a newer one exists):
cat << EOF > build.sh
#!/bin/bash
export DOCKER_SOURCE=traefik:2.10.7
export DOCKER_REGISTRY=registry.home.digitaldann.net:8443/traefik:2.10.7
docker pull \${DOCKER_SOURCE}
docker tag \${DOCKER_SOURCE} \${DOCKER_REGISTRY}
docker image push \${DOCKER_REGISTRY}
docker image rm \${DOCKER_SOURCE}
docker image rm \${DOCKER_REGISTRY}
EOF
chmod u+x build.sh
Now publish it to the local registry (removing future dependency on the internet-based registry):
./build.sh
Create Traefik configuration
Create the directory structure for the container bind mounts:
mkdir -p /mnt/vagabond/docker/traefik/certs
Create the default Traefik configuration:
cd /mnt/vagabond/docker/traefik
cat << EOF > traefik.yml
tls:
options:
default:
minVersion: VersionTLS12
certificates:
- certFile: /etc/traefik/dynamic/certs/wildcard.pem
keyFile: /etc/traefik/dynamic/certs/wildcard.key
stores:
default:
defaultCertificate:
certFile: /etc/traefik/dynamic/certs/wildcard.pem
keyFile: /etc/traefik/dynamic/certs/wildcard.key
EOF
Copy the TLS certificates to the Traefik configuration:
cp /mnt/vagabond/ca/certs/wildcard.pem /mnt/vagabond/docker/traefik/certs/
cp /mnt/vagabond/ca/certs/wildcard.key /mnt/vagabond/docker/traefik/certs/
Create Nomad job file for Traefik
If you want to read the official documentation for the Nomad job file spec, here it is: https://developer.hashicorp.com/nomad/docs/job-specification/job
Create the directory structure for the Nomad jobs:
mkdir /mnt/vagabond/nomad
cd /mnt/vagabond/nomad
Now create the Nomad job file:
nano traefik.2.10.7.nomad
Write the following content and save the file:
job "traefik" {
datacenters = ["dc1"]
type = "service"
constraint {
operator = "distinct_hosts"
value = "true"
}
group "traefik" {
count = 3
network {
mode = "bridge"
port "http" {
static = 80
}
port "https" {
static = 443
}
port "http-ext" {
static = 81
}
port "https-ext" {
static = 444
}
}
service {
name = "dashboard"
provider = "consul"
port = "https"
tags = [
"traefik.enable=true",
"traefik.http.routers.dashboard-${attr.unique.hostname}.entrypoints=websecure",
"traefik.http.routers.dashboard-${attr.unique.hostname}.rule=Host(`dashboard.home.digitaldann.net`)",
"traefik.http.routers.dashboard-${attr.unique.hostname}.service=api@internal",
"traefik.http.routers.dashboard-${attr.unique.hostname}.tls=true"
]
}
service {
name = "traefik"
provider = "consul"
port = "http"
}
task "traefik" {
driver = "docker"
config {
image = "registry.home.digitaldann.net:8443/traefik:2.10.7"
ports = ["http", "https", "http-ext", "https-ext"]
volumes = [
"/mnt/vagabond/docker/traefik:/etc/traefik/dynamic"
]
args = [
"--providers.docker=false",
"--api.dashboard=true",
"--entrypoints.web.address=:80",
"--entrypoints.websecure.address=:443",
"--entrypoints.web-ext.address=:81",
"--entrypoints.websecure-ext.address=:444",
"--entrypoints.web-ext.http.redirections.entrypoint.to=websecure",
"--entrypoints.web-ext.http.redirections.entrypoint.scheme=https",
"--providers.file.directory=/etc/traefik/dynamic",
"--providers.consulcatalog=true",
"--providers.consulcatalog.requireConsistent=true",
"--providers.consulcatalog.cache=true",
"--providers.consulcatalog.endpoint.address=http://${attr.unique.network.ip-address}:8500",
"--providers.consulcatalog.endpoint.scheme=http",
"--serversTransport.insecureSkipVerify=true"
]
}
}
}
}
Deploy the Traefik job
This will deploy Traefik to the appropriate number of nodes (as defined by the job property ‘count’):
cd /mnt/vagabond/nomad
nomad job run traefik.2.10.7.nomad
You should see output that looks like:
==> 2023-12-21T01:47:34Z: Monitoring evaluation "7e6d80f6"
2023-12-21T01:47:34Z: Evaluation triggered by job "traefik"
2023-12-21T01:47:35Z: Evaluation within deployment: "c13539c4"
2023-12-21T01:47:35Z: Allocation "0127c059" created: node "4b37faff", group "traefik"
2023-12-21T01:47:35Z: Allocation "050c601e" created: node "32d39aab", group "traefik"
2023-12-21T01:47:35Z: Allocation "d30a6a7e" created: node "f7d98ba8", group "traefik"
2023-12-21T01:47:35Z: Evaluation status changed: "pending" -> "complete"
==> 2023-12-21T01:47:35Z: Evaluation "7e6d80f6" finished with status "complete"
==> 2023-12-21T01:47:35Z: Monitoring deployment "c13539c4"
✓ Deployment "c13539c4" successful
2023-12-21T01:47:51Z
ID = c13539c4
Job ID = traefik
Job Version = 0
Status = successful
Description = Deployment completed successfully
Deployed
Task Group Desired Placed Healthy Unhealthy Progress Deadline
traefik 3 3 3 0 2023-12-21T01:57:49Z
Validate registration with Consul
You will want to validate that the Traefik service registered with Consul. Consul has it’s own DNS service (running on port 8600), and querying <service-name>.service.consul should resolve to the addresses of the nodes running the service. In the case of Traefik, it will be all nodes. Run the following:
dig dashboard.service.consul @localhost -p 8600
You should see:
; <<>> DiG 9.18.18-0ubuntu0.22.04.1-Ubuntu <<>> dashboard.service.consul @localhost -p 8600 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16211 ;; flags: qr aa rd; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 1232 ;; QUESTION SECTION: ;dashboard.service.consul. IN A ;; ANSWER SECTION: dashboard.service.consul. 0 IN A 10.0.3.101 dashboard.service.consul. 0 IN A 10.0.3.102 dashboard.service.consul. 0 IN A 10.0.3.103
Wrapping Up
You have deployed your first Nomad job. That’s great! Once the infrastructure is setup, deploying jobs is really straight forward.
Next, we’ll look at how to add DNS to enable custom local domains with Traefik!