Deploying AWS Load Balancer Controller and Ingress on AWS EKS

What is AWS EKS?

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed Kubernetes service that makes it easy for you to run Kubernetes on AWS and on-premises. Amazon EKS is certified Kubernetes conformant, so existing applications that run on upstream Kubernetes are compatible with Amazon EKS.

Amazon EKS automatically manages the availability and scalability of the Kubernetes control plane nodes that are responsible scheduling containers, managing the availability of applications, storing cluster data, and other key tasks.

AWS Load Balancer Controller

AWS Load Balancer Controller is a controller to help manage Elastic Load Balancers for a Kubernetes cluster.

It satisfies Kubernetes Ingress resources by provisioning Application Load Balancers .

It satisfies Kubernetes Service resources by provisioning Network Load Balancers .

This project was formerly known as “AWS ALB Ingress Controller”, we rebranded it to be “AWS Load Balancer Controller”.

Here i am deploying an Ingress (AWS Application Load Balancer) with demo container apps deployed onto AWS EKS to access AWS RDS Database residing in private subnet.

AWS Load Balancer Controller manages AWS Elastic Load Balancers for a Kubernetes cluster, hence we could use for Path-Based Routing

• eg. https://test.cloudrgb.com/adminer -> route to adminer pod
• eg. https://test.cloudrgb.com/usermgmt -> route to usermgmt pod

The AWS architecture components includes the following:

AWS EKS (Cluster)
AWS IAM
AWS RDS(MySQL)
Amazon Route53
Amazon CloudFront
AWS ELB (Application Load Balancer)
AWS NAT Gateway
AWS Certificate Manager
AWS Elastic Container Registry (ECR)
AWS VPC, Subnet , Route Table

I am using Kubectl for most of the deployment , hence here are some of the common ‘jargon’ to know when deploying k8s:

• Kubernetes Secrets – A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in a container image . Using a Secret means that you don’t need to include confidential data in your application code.

• Kubernetes Deployment – A Kubernetes Deployment is used to tell Kubernetes how to create or modify instances of the pods that hold a containerized application.

• Kubernetes Service – A Kubernetes service is a logical abstraction for a deployed group of pods in a cluster (which all perform the same function). Since pods are ephemeral, a service enables a group of pods, which provide specific functions (web services, image processing, etc.) to be assigned a name and unique IP address (clusterIP).

• Kubernetes Ingress – Kubernetes Ingress is an API object that provides routing rules to manage external users’ access to the services in a Kubernetes cluster, typically via HTTPS/HTTP. With Ingress, you can easily set up rules for routing traffic without creating a bunch of Load Balancers or exposing each service on the node

This shows a process flow of how kubernetes deployment consumed the Docker image created from a Dockerfile which is pushed into a repository (AWS Elastic Container Registry)

Prerequisites

• [Platform] – Deploy 1 x VPC with at least 2 x Availability Zones with NAT Gateways residing in each Public Subnet. This will allow Private Subnets to route internet traffic out

• [K8s Cluster] – Deploy an AWS EKS Cluster using managed Node-Group (You can deploy the whole EKS infrastructure with terraform or manual console deployment)

• [Container app] – Demo image container ready for deployment. (We will be deploying adminer as a container together with another demo container app to access AWS RDS which resides in a private subnet). All the image in this walkthrough will be from Docker Hub.

Step1: Deploy AWS Load Balancer Controller

After your AWS EKS cluster has been created, follow this AWS guide to deploy AWS Load Balancer Controller :

https://docs.aws.amazon.com/eks/latest/userguide/aws-load-balancer-controller.html

Step2: Create AWS RDS database

Deploy AWS RDS in the private subnet (do remember to edit the DB subnet groups to associate only private subnets prior to creating AWS RDS)

Step3: Deploy Kubernetes manifest

Services (SQL,NodePort), Deployment Pods & Secrets.

(For app container images, we will be using the existing images from Docker hub)

Save the following manifest below as eg. kube-manifest.yaml

#SQL external name Service

apiVersion:
kind: Service
metadata:
  name: mysql
spec:
  type: ExternalName
  externalName: <AWS RDS Endpoint URL>
---

#Stacksimplogy microservices

apiVersion: apps/v
kind: Deployment
metadata:
  name: usermgmt-microservice
  #labels:
    #app: usermgmt-restapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: usermgmt-restapp
  template:
    metadata:
      labels:
        app: usermgmt-restapp
    spec:
      initContainers:
        - name: init-db
          image: busybox:1.31
          command: ['sh', '-c', 'echo -e "Checking for the availability of MySQL Server deployment"; while ! nc -z mysql 3306; do sleep 1; printf "-"; done; echo -e "  >> MySQL DB Server has started";']
      containers:
        - name: usermgmt-restapp
          image: stacksimplify/kube-usermanagement-microservice:1.0.0
          ports:
            - containerPort: 8095
          env:
            - name: DB_HOSTNAME
              value: "mysql"
            - name: DB_PORT
              value: "3306"
            - name: DB_NAME
              value: "ebdb"
            - name: DB_USERNAME
              value: "admin"
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-db-password
                  key: db-password
          livenessProbe:
            exec:
              command:
                - /bin/sh
                - -c
                - nc -z localhost 8095
            initialDelaySeconds: 60
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /usermgmt/health-status
              port: 8095
            initialDelaySeconds: 60
            periodSeconds: 10
---

#Secret Pod

apiVersion: v1
kind: Secret
metadata:
  name: mysql-db-password
type: Opaque
data:
  db-password: <RDS Password>
---
#NodePort Services
apiVersion: v
kind: Service
metadata:
  name: usermgmt-restapp-nodeport-service
  labels:
    app: usermgmt-restapp
spec:
  type: NodePort
  selector:
    app: usermgmt-restapp
  ports:
    - port: 8095
      targetPort: 8095
---

#Deploy Ingress

apiVersion: networking.k8s.io/v
kind: Ingress
metadata:
  name: ingress-usermgmt-restapp-service
  labels:
    app: usermgmt-restapp
  annotations:
    # Ingress Core Settings
    kubernetes.io/ingress.class: "alb"
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: instance

    # Health Check Settings
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/healthcheck-port: traffic-port
    alb.ingress.kubernetes.io/healthcheck-path: /usermgmt/health-status
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: '15'
    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: '5'
    alb.ingress.kubernetes.io/success-codes: '200'
    alb.ingress.kubernetes.io/healthy-threshold-count: '2'
    alb.ingress.kubernetes.io/unhealthy-threshold-count: '2'
spec:
  rules:
    - http:
        paths:
          - path: /usermgmt
            pathType: Prefix
            backend:
              service:
                name: usermgmt-restapp-nodeport-service
                port:
                  number: 8095
          - path: /adminer
            pathType: Prefix
            backend:
              service:
                name: adminer-service-nodeport
                port:
                  number: 80
---

#Adminer - Deployment Pods

apiVersion: apps/v
kind: Deployment
metadata:
  name: adminer
spec:
  selector:
    matchLabels:
      app: adminer
  template:
    metadata:
      labels:
        app: adminer
    spec:
      containers:
      - name: adminer
        image: adminer:4.8.1
        ports:
          - containerPort: 8080
---

#Adminer - Nodeport Service

apiVersion: v1
kind: Service
metadata:
  name: adminer-service-nodeport
spec:
  type: NodePort
  selector:
    app: adminer
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP

Finally , deploy the manifest:

kubectl apply -f  kube-manifest.yaml

Once the manifest is deployed, you will an AWS Application Load Balancer created in AWS:

AWS Application Load Balancer will route traffic based on Prefix http:// / to AWS EKS Target Groups of Pods of exposed Service Name labelled in the deployment manifest:

Step4: Access to Ingress ( AWS Application Load Balancer ) DNS

After ingress deployment manifest is deployed, you can access to container pods via AWS Application Load Balancer’s DNS name

You can now test the Adminer pods to access AWS RDS which resides in private subnet. For application accessing AWS RDS you will need an container application (frontend) and AWS RDS(backend):

Step5: Configure AWS Route53 to route traffic to Ingress ( AWS Application Load Balancer )

1. Go you AWS Route53 > Select hosted zone
2. Click on the domain name (eg. test.cloudrgb.com )
3. Create A (Alias) record

• Record Type : A – Route traffic to an IPv4 address and some ….
• Route Traffic to : Alias to Application and Classic Load Balancer …. / Asia Pacific (Singapore) [ap-southeast-1] / [ Select your Application Load Balancer ]

Now you will be able to access the AWS EKS pods via your domain which routes traffics to your AWS Application Load Balancer into your AWS EKS !

Add-ons :

• AWS CloudFront Origin to AWS Application Load Balancer with SSL configured using AWS Certificate Manager .

• CICD for AWS ECR (Pipeline to package your custom container) includes container scanning capability.

• AWS App Mesh (sidecar) for Kubernetes – a service mesh which provides application-level networking & deployment capability (blue/green, canary deployment for application upgrade)

• AWS EKS Monitoring using AWS Managed Prometheus (AMP) + AWS Managed Grafana

Lastly, Let’s keep learning, thats where life journey begins… !

Credits/References:

• https://www.stacksimplify.com
• Amazon Web Services forums / articles