Install kubernetes ingress
In this blogpost I’ll show how to setup a nginx-ingress controller and how to deploy the first application that uses the ingress. Additionally I’ll show how to setup a DigitalOcean LoadBalancer that handles requests on port 80 and 443 and redirects these requests to the ingress controller.
Architecture of Ingress
As you should see in the picture there is a Kubernetes Service deployed in the kubernetes cluster that receives requests from outside the network. In our example we use the ports 30080 and 30443. This service balances the received traffic to the ingress pods. These pods are nginx servers that are configured to forward traffic by a hostname and path variable to other services. This services are the load balancers for our applications deployed in the cluster that should be exposed to the outside. They will receive the traffic from the ingress pods and forward them to the application pods.
How does the Ingress pods know how to proxy traffic to the correct services. Therefore in kubernetes a resource called ingress is defined. The entries specified in ingress define how traffic should be proxied. An example ingress looks like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
namespace: application
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: application.koudingspawn.de
http:
paths:
- path: /
backend:
serviceName: nginx-service
servicePort: 80
This ingress specifies, that traffic received with host header application.koudingspawn.de should be forwarded to the nginx-service on port 80. After creating this resource the nginx-ingress controller will check the kubernetes-api and receive a new ingress configuration. With this information he reconfigures the nginx-ingress controller to balance incomming traffic with a host header application.koudingspawn.de to the service called nginx-service. It is important to know, that the ingress resource must be specified in the same namespace as the service is deployed that should receive the incomming traffic.
How to setup nginx-ingress
Setup of nginx-ingress is very easy, we only need to run some kubectl apply commands that will configure the nginx-ingress controller. This part is a copy of the https://github.com/kubernetes/ingress-nginx/tree/master/deploy page that describes how to deploy nginx-ingress. The cluster created in the previous posts is a cluster without RBAC access, so we must use the without-rbac.yml file to create the deployment:
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/namespace.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/tcp-services-configmap.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/udp-services-configmap.yaml \
| kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/without-rbac.yaml \
| kubectl apply -f -
This should start in the namespace ingress-nginx two pods:
kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
default-http-backend-7c7764588c-5zn2f 1/1 Running 0 1m
nginx-ingress-controller-886445d9c-wxsp9 1/1 Running 0 1m
The first pod is the default backend, to which requests are forwarded that cannot be assigned. This backend has to comply to two prerequisites:
- It must return 200 on /healthz
- It must return 404 on /
You could replace the default backend by your own implementation this is very easy. The second pod that is deployed is the nginx-ingress-controller that receives the requests and proxies them to the services defined in the ingress configuration resources.
Now we need to deploy the service that exposes the nginx-ingress-controller on ports 30080 and 30443 as described already in the tutorial:
kind: Service
apiVersion: v1
metadata:
name: ingress-lb
namespace: ingress-nginx
spec:
selector:
app: ingress-nginx
type: NodePort
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
nodePort: 30080
- name: https
protocol: TCP
port: 443
targetPort: 443
nodePort: 30443
Thats all, now we can try to access it by running curl against this resources:
curl http://{ ip }:30080
This should produce the following output: default backend - 404
Setup your first application that uses ingress
This application is a simple nginx server that receives requests on port 80 and returns the default nginx page. Therefore we create a namespace called application, a service called nginx-service and a deployment called nginx-deployment:
apiVersion: v1
kind: Namespace
metadata:
name: application
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
namespace: application
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
name: nginx-service
namespace: application
spec:
selector:
app: nginx
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
Now you should see the following output after you applied the configuration.
$ kubectl get pods -n application
NAME READY STATUS RESTARTS AGE
nginx-deployment-569477d6d8-8q7kw 1/1 Running 0 1m
nginx-deployment-569477d6d8-hms7z 1/1 Running 0 1m
nginx-deployment-569477d6d8-ldf5s 1/1 Running 0 1m
$ kubectl get svc -n application -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
nginx-service ClusterIP 10.32.0.103 <none> 80/TCP 5m app=nginx
The next step is to route traffic to the nginx-service by adding the following ingress configuration:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
namespace: application
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: application.koudingspawn.de
http:
paths:
- path: /
backend:
serviceName: nginx-service
servicePort: 80
As already specified traffic that arrives with host header application.koudingspawn.de should be proxied from the nginx-ingress-controller to the nginx-service and from there to the nginx pods.
Now you should see the following output when running curl with host header application.koudingspawn.de:
$ curl http://{public_ip}:30080/ -H "Host: application.koudingspawn.de"
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
How to setup a LoadBalancer on DigitalOcean
This step is only an appendix, I tried to run the steps but my LoadBalancer doesn’t proxy traffic to the nodes. He was healthy and the droplets were also healthy but it won’t forward traffic.
We will specify in our LoadBalancer that traffic from port 80 should be forwarded to port 30080 on each of the nodes and traffic on port 443 should be forwarded to port 30443. Ports 30080 and 30443 are the ports the ingress-lb service watches for traffic on. Additionally we specify a health check on port 30080, there we use the features of the default backend to return status code 200 on /healthz. Here is the terraform script we use to generate the LoadBalancer:
resource "digitalocean_loadbalancer" "public" {
name = "worker-loadbalancer"
region = "fra1"
forwarding_rule {
entry_port = 80
entry_protocol = "tcp"
target_port = 30080
target_protocol = "tcp"
}
forwarding_rule {
entry_port = 443
entry_protocol = "tcp"
target_port = 30443
target_protocol = "tcp"
}
healthcheck {
port = 30080
protocol = "http"
path = "/healthz"
}
droplet_tag = "kube-worker"
}
Our droplets are tagged with the kube-worker tag, so new upcomming droplets will join the LoadBalancer when their healthcheck on Port 30080/healthz returns a status of 200. This will happen when the kubelet and kube-proxy running on the nodes are comming up.
The last step is to connect a domain to the LoadBalancer in our case we do this with CloudFlare:
resource "cloudflare_record" "worker_lb" {
domain = "${var.cloudflare_domain}"
name = "application"
type = "A"
value = "${digitalocean_loadbalancer.public.ip}"
}
