As we saw in the last article, both the long list of
API Object types and each
API Object spec's corresponding complexity, learning
Kubernetes is a non-trivial exercise. Fortunately, for our purposes, we can narrow down the list of types we should be familiar with to these eight:
Namespaces,
Pods,
ReplicaSets,
Deployments,
ConfigMaps,
Persistent Volumes,
Services, and
Secrets.
We can list the current set of Namespace with the following command:
In general, the one-container-per-Pod model is the most common deployment. However, Pods can contain more than one container. This approach is recommended to supported grouping containers with tightly-coupled dependencies that need to share resources. It is also common to group containers within the same Pod when each container shares identical lifecycles.
When thinking about Pods, it is important to remember that Pods are the environment in which containers run- not the containers themselves. For this reason, Kubernetes replicates Pods, not containers, when scaling the application.
You will rarely need to create a Pod directly. Usually, Deployment or Job resources create them for you. However, if you need to create one manually, we can use the following command:
If you have a simple docker-compose.yaml file, Kompose can probably handle it. For complex docker-compose.yaml files, let Kompose translate what it can, and then you can focus on the parts that didn't translate correctly. Either way, it will likely be less work than creating all the manifest files manually. Kompose is ridiculously simple to run. Run it, and if you don't like what it generates, you can always manually create your manifest.
As mentioned earlier, Kompose is easy to install and run. If you have Snap package manager installed you can just call:
As you can see, Kompose did alot of work for us. Most of the manifests generated are Deployments and Services. Here we see an example of the AccountHistoryService Deployment Manifest.
We can see in this deployment that Kompose has created the deployment with the container image (thinkmicroservices/account-history-service:latest), set the container's env variables, and set the replicas to 1.
Now we expose the AccountHistoryService with the a Service manifest.
The Service manifest creates a ClusterIP type service that exposes two ports, 5010 for the service and is mapped to the container's port 8080, and port 5019 for the management port and is mapped to the container's port 8089.
While Kompose did a pretty good job in the translation, it did miss a few things. This may be the fault of Kompose, but is more likely due an ambiguous configuration in the docker-compose.yaml. In any case, the following list contains the deficiencies in the manifests generated by Kompose, and the manual interventions needed to remediate them. We can clearly see that the amount of manual work needed was far less than starting from scratch.
The entire set of Kubernetes manifests are available in the ThinkMicrosystems Kubernete GitHub repository. Included in the repository are start.sh and stop.sh scripts to deploy and undeploy the application to Kubernetes.
Namespaces
A Namespace defines a resource's scope. Namespaces allow a single physical cluster to be divided into multiple virtual clusters with related resources scoped within the same Namespaces. This approach enables Kubernetes to attach individualized authorization and policies to each virtual cluster. When provisioning a cluster, Kubernetes will create the default namespace to hold resources lacking an explicitly declared Namespace.We can list the current set of Namespace with the following command:
microk8s kubectl get namespaces
NAME STATUS AGE default Active 7hWe can create a new Namespace from the command line:
microk8s kubectl create namespace thinkmicroservices
NAME STATUS AGE default Active 7h thinkmicroservices Active 2sAfter a Namespace has been created it can also be deleted.
microk8s kubectl delete namespace thinkmicroservices
namespace "thinkmicroservices" deletedWe can also create a Namespace using a manifest file. Here we create the thinkmicroservices-namespace.yaml file:
apiVersion: v1
kind: Namespace
metadata:
name: thinkmicroservices
microk8s kubectl apply -f ./thinkmicroservices-namespace.yaml
namespace/thinkmicroservices createdNow if we check our Namespaces again it appears in the listing.
microk8s kubectl get namespaces
NAME STATUS AGE default Active 7h thinkmicroservices Active 5s
Pods
A Pod acts as an abstraction layer over a group of one (or more) containers. Each Pod allows each of its containers to share a common spec, storage, and networking resources. Every container within a Pod is co-located, co-scheduled, and run in a shared context.In general, the one-container-per-Pod model is the most common deployment. However, Pods can contain more than one container. This approach is recommended to supported grouping containers with tightly-coupled dependencies that need to share resources. It is also common to group containers within the same Pod when each container shares identical lifecycles.
When thinking about Pods, it is important to remember that Pods are the environment in which containers run- not the containers themselves. For this reason, Kubernetes replicates Pods, not containers, when scaling the application.
You will rarely need to create a Pod directly. Usually, Deployment or Job resources create them for you. However, if you need to create one manually, we can use the following command:
microk8s kubectl run <name of pod> --image=<name of the image from registry>The simplest way to delete a Pod is to issue the command:
microk8s kubectl delete pod <name of pod>>Another, safer way, requires a few more steps. First we need to find out which node the pod is running on :
microk8s kubectl get nodes microk8s kubectl get pods -o wide | grep <nodename>Now we want to mark this node as unschedulable to ensure no new pods are scheduled during the removal operation.
kubectl cordon <nodename>Now delete the Pod as before:
microk8s kubectl delete pod <name of pos>And now uncordon the node.
microk8s kubectl uncordon <nodename>We can also create the Pod with a manifest file. Here is a simple example of a Pod manifest file.
apiVersion: v1
kind: Pod
metadata:
name: Tomcat
spec:
containers:
- name: Tomcat
image: tomcat: 8.0
ports:
containerPort: 8080
imagePullPolicy: Always
ReplicaSets
A ReplicaSet is responsible for guaranteeing that a stable set of pod replicas is maintained and is used to provide horizontal scaling for pods. Each ReplicaSet spec contains a replicas field to identify how many instances it should guarantee and a selector field to identify the type of pods to acquire. In general, the ReplicaSet is rarely managed directly. Instead, it is usually managed through a Deployment object. Below is a ReplicateSet manifest example.
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: authentication
labels:
app: guestbook
tier: authentication
spec:
# of replicas
replicas: 3
selector:
matchLabels:
tier: authentication
template:
metadata:
labels:
tier: authentication
spec:
containers:
- name: authentication-service
image: thinkmicroservices/authentication-service:latest
We can create the
ReplicaSet from the above manifest file using:
microk8s kubectl create –f example-replicaset.yaml
Deployments
A Deployment object instructs Kubernetes to create and update application instances. It describes the application's desired target state and delegates how to achieve that state to the Kubernetes scheduler. After an application's instances have been created, the Kubernetes DeploymentController monitors each instance to ensure the target state is maintained. Below is a simple example of a Deployment manifest that will run three replicas of the notification-service.
apiVersion: apps/v1
kind: Deployment
metadata:
name: notification-deployment
labels:
app: notification
spec:
replicas: 3
strategy:
type: RollingUpdate
maxSurge: 2
maxUnavailable: 1
selector:
matchLabels:
app: notification
template:
metadata:
labels:
app: notification
spec:
containers:
- name: notification
image: thinkmicroservices/notification-service:latest
ports:
- containerPort: 8080
We invoke the
Deployment manifest using the
kubectl apply command.
microk8s kubectl apply –f example-deployment.yamlTo delete a Deployment we can invoke the kubectl command.
microk8s kubectl delete –f example-deployment.yamlWe can also use Deployments to handle Rolling Updates to provide zero downtime. Rolling updates can be accomplished by setting the strategy.type field to the value RollingUpdate. By setting the maxSurge value to two in the example above, up to five containers (original three replicas + the two declared by maxSurge) can run during the update. Setting maxUnavailable to one requires at least two of the three pods to be available at all times during the update. We can check the progress of the rolling update using the kubectl rollout command.
microk8s kubectl apply –f example-deployment.yaml
microk8s kubectl rollout history deployment
ConfigMaps
ConfigMaps are dictionaries of configuration settings that provide a mechanism to externalize container configuration. This approach allows for application portability. To create a configMapmicrok8s kubectl configmap <map-name> <data-source>Here the map-name represents the name assigned to the Kubernetes ConfigMap object, and data-source is a file, directory, or literal values that constitute the map. We can also create a configMap with a manifest file.
kind: ConfigMap
apiVersion: v1
metadata:
name: config-example
namespace: default
data:
config.property.1: value1
config.property.2: value2
In this example, the
ConfigMap file includes a
data section containing the list of configuration properties. It is important to remember to create any
ConfigMaps before any
Pods reference it; otherwise, the
Pod wont start.
Persistent Volumes
Persistent Volumes provide non-volatile storage within the Kubernetes cluster. The contents of the storage persist beyond container, pod, or node restarts. Below is an example of a PersistentVolume manifest file.
apiVersion: v1
kind: PersistentVolume
metadata:
name: mongodb-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data"
Pods access
Persistent Volumes through a
Persistent Volume Claim. A
Pod registers a
Persistent Volume Claim to a
Persistent Volume. When granted, each container within the
Pod mounts the volume to its filesystems. Below is an example of a
PersistentVolumeClaim manifest.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mongodb-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
To configure a
Pod to access the
PersistentVolume, we add the
volumes field and value references to the spec, as well as adding a
volumeMount to the container.
apiVersion: v1
kind: Pod
metadata:
name: mongodb-pv-pod
spec:
volumes:
- name: mongodb-pv-storage
persistentVolumeClaim:
claimName: mongodb-pv-claim
containers:
- name: mongodb-pv-container
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: mongodb-pv-storage
Services
The term service has been overloaded throughout this series of articles. We have used it to refer to both microservices and the business logic within Spring applications. As we adopt Kubernetes, it gains yet another definition. In Kubernetes, a Service refers to the mechanism by which we expose an application running on a Pod. Services are both an abstraction that defines a logical collection of pods as well as its access policy. Kubernetes supports four types of Services:- ClusterIP- exposes the service on a cluster-internal IP address. The service is only accessible from within the cluster on this address. All Services default to this type unless otherwise defined.
- NodePort- exposes the service on a static port on each node in the cluster. It is reachable through the <node-ip-address>:<node-port>.
- LoadBalancer- exposes the service outside of a cloud provider's cluster.
- ExternalName- maps the Service to a DNS CNAME record using the Service's externalName field.
kind: Service
apiVersion: v1
metadata:
name: api-gateway-service
spec:
type: ClusterIP
selector:
app: api-gateway
ports:
name: "portname"
port: 8080
targetPort: 80
This manifest will expose the
Pod identified by the name
api-gateway as the
Service
api-gateway-service. It will map the cluster-internal IP address to the
Pod on port
8080 and route the incoming traffic to port
80.
Secrets
Secrets are similar to ConfigMaps. Both are intended to hold configuration information; however, Secrets are intended for sensitive data like password, tokens, and keys. Kubernetes treats the storage of Secrets differently than ConfigMaps. It writes Secrets to a tmpfs that appears as a temporary file system but is, in fact stored in volatile memory. When the node terminates, the secret ceases to exist. This approach prevents Secrets from living being the life of a node. Below is an example of a Secret manifest file.
kind: Secret
apiVersion: v1
metadata:
name: jwt-shared-secret
type: Opaque
data:
key: 7h1nkm1cr053rv1c35
We see in the following example how a
Pod can access the
secret by mounting it as volume.
kind: Pod
apiVersion: v1
metadata:
name: authentication-service-with-secret
spec:
volumes:
- name: authentication-service-with-secret-volume
secret:
secretName: jwt-shared-secret
containers:
- name: auth-secret
image: thinkmicroservices/authentication-service
volumeMounts:
- name: authentication-service-with-secret-volume
mountPath: /etc/secret
The container now can access the
Secret via the
/etc/secret directory mapped to the container.
Converting the docker-compose.yaml
Now that we have a substantially better understanding of Kubernetes API Objects, we are ready to begin translating our docker-compose.yaml file to Kubernetes manifests files. We could manually write our Kubernetes manifests files, but the size of the full reference implementation's docker-compose.yaml file will make that process tedious and error-prone. Instead, we will introduce a translation tool to do most of the heavy-lifting for us.Kubernetes Kompose
Kompose is a tool that translates Docker Compose files into a corresponding set of Kubernetes resource manifest files. Unfortunately, Kompose isn't a perfect solution. Since Docker Compose lacks a 1:1 correspondence to Kubernetes features, the output product of Kompose may not be a 100% solution (for a better understanding of exactly which Docker Compose features can be directly translated, refer to the Kompose conversion matrix). You may ask yourself, "If Kompose doesn't provide a 100% solution, why use it?".If you have a simple docker-compose.yaml file, Kompose can probably handle it. For complex docker-compose.yaml files, let Kompose translate what it can, and then you can focus on the parts that didn't translate correctly. Either way, it will likely be less work than creating all the manifest files manually. Kompose is ridiculously simple to run. Run it, and if you don't like what it generates, you can always manually create your manifest.
As mentioned earlier, Kompose is easy to install and run. If you have Snap package manager installed you can just call:
sudo snap install komposeIf you dont want to use Snap you can do the following for Linux:
curl -L https://github.com/kubernetes/kompose/releases/download/v1.19.0/kompose-linux-amd64 -o kompose
chmod +x kompose
sudo mv ./kompose /usr/local/bin/komposeFor macOS:
curl -L https://github.com/kubernetes/kompose/releases/download/v1.21.0/kompose-darwin-amd64 -o kompose
chmod +x kompose
sudo mv ./kompose /usr/local/bin/komposeVerify that it is installed correctly by checking its version:
kompose version
Now we can give it a shot. Create a new directory and copy the docker-compose.yaml file into it. Then call the Kompose command.kompose version
kompose convert -f docker-compose.yamlThe output should be similar to this:
INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination WARN Volume mount on the host "/home/thinkmicroservices/kompose-example/datasource.yml" isn't supported - ignoring path on the host INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination WARN Volume mount on the host "/home/thinkmicroservices/kompose-example/prometheus.yml" isn't supported - ignoring path on the host WARN Volume mount on the host "/home/thinkmicroservices/kompose-example/rules.yml" isn't supported - ignoring path on the host INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Network spring_ri_network is detected at Source, shall be converted to equivalent NetworkPolicy at Destination INFO Kubernetes file "account-history-service-service.yaml" created INFO Kubernetes file "account-profile-service-service.yaml" created INFO Kubernetes file "administration-service-service.yaml" created INFO Kubernetes file "api-gateway-service-service.yaml" created INFO Kubernetes file "auth-service-service.yaml" created INFO Kubernetes file "config-service-service.yaml" created INFO Kubernetes file "content-service-service.yaml" created INFO Kubernetes file "discovery-service-service.yaml" created INFO Kubernetes file "elasticsearch-service.yaml" created INFO Kubernetes file "email-outbound-service-service.yaml" created INFO Kubernetes file "feature-service-service.yaml" created INFO Kubernetes file "fluentd-service.yaml" created INFO Kubernetes file "grafana-service.yaml" created INFO Kubernetes file "kibana-service.yaml" created INFO Kubernetes file "mongo-db-service-service.yaml" created INFO Kubernetes file "mongo-express-service.yaml" created INFO Kubernetes file "notification-service-service.yaml" created INFO Kubernetes file "peer-signaling-service-service.yaml" created INFO Kubernetes file "postgresadmin-service.yaml" created INFO Kubernetes file "postgresdb-service.yaml" created INFO Kubernetes file "prometheus-service.yaml" created INFO Kubernetes file "rabbitmq-service.yaml" created INFO Kubernetes file "sms-outbound-service-service.yaml" created INFO Kubernetes file "telemetry-service-service.yaml" created INFO Kubernetes file "account-history-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "account-profile-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "account-profile-started-deployment.yaml" created INFO Kubernetes file "administration-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "api-gateway-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "auth-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "config-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "config-started-deployment.yaml" created INFO Kubernetes file "content-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "discovery-service-pod.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "discovery-started-deployment.yaml" created INFO Kubernetes file "elasticsearch-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "email-outbound-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "feature-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "fluentd-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "grafana-deployment.yaml" created INFO Kubernetes file "grafana-claim0-persistentvolumeclaim.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "kibana-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "mongo-db-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "mongo-express-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "notification-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "peer-signaling-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "postgresadmin-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "postgresdb-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "prometheus-deployment.yaml" created INFO Kubernetes file "prometheus-claim0-persistentvolumeclaim.yaml" created INFO Kubernetes file "prometheus-claim1-persistentvolumeclaim.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "rabbitmq-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "sms-outbound-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created INFO Kubernetes file "telemetry-service-deployment.yaml" created INFO Kubernetes file "spring_ri_network-networkpolicy.yaml" created
As you can see, Kompose did alot of work for us. Most of the manifests generated are Deployments and Services. Here we see an example of the AccountHistoryService Deployment Manifest.
loading...We can see in this deployment that Kompose has created the deployment with the container image (thinkmicroservices/account-history-service:latest), set the container's env variables, and set the replicas to 1.
Now we expose the AccountHistoryService with the a Service manifest.
loading...The Service manifest creates a ClusterIP type service that exposes two ports, 5010 for the service and is mapped to the container's port 8080, and port 5019 for the management port and is mapped to the container's port 8089.
While Kompose did a pretty good job in the translation, it did miss a few things. This may be the fault of Kompose, but is more likely due an ambiguous configuration in the docker-compose.yaml. In any case, the following list contains the deficiencies in the manifests generated by Kompose, and the manual interventions needed to remediate them. We can clearly see that the amount of manual work needed was far less than starting from scratch.
- Instead of creating a DiscoveryService Deployment manifest, it created a Pod manifest. We remedy this by manually creating our DiscoveryService
- Services host names will have a suffix added to them. With our Spring-based DiscoveryService, this leads to unresolved host issues. We can address this by addding the eureka.instance.preferIpAddress=true environment variable to each service deployment manifest that registers with the discovery-service.
- Our Prometheus instance starts, but Kompose didnt know to create a ConfigMap manifest for the prometheus.yaml configuration file. To solve this, we manually create a ConfigMap.
microk8s kubectl configmap prom-configmap --from-file prometheus.yaml
We can the generate the manifest file from the newly created ConfigMap using the following kubectl command.
microk8s kubectl describe configmap prom-configmap > prom-configmap.yaml
We can then use the prom-configmap.yaml to configure our Prometheus instance.
The entire set of Kubernetes manifests are available in the ThinkMicrosystems Kubernete GitHub repository. Included in the repository are start.sh and stop.sh scripts to deploy and undeploy the application to Kubernetes.
Twitter
Facebook
Reddit
LinkedIn
Email