What other way to celebrate 10k subscribers than to deploy 10k Kubernetes pods

SUNDAY, MAY 24, 2020 12:00 PM    

According to one of the youtube video’s comments, this is probably the nerdiest way to celebrate a 10k subscribers milestone. Love the genuineness of this guy and he is really knowledgeable based on both the video and also his blog post.



This video is about the use of kubernetes and as a way to celebrate his 10k subscriber milestone, he decided to create 10,000 pods on his kubernetes cluster. What I love about his way of celebration is the learning journey he was on as an attempt to do such a large scale deployment. He was trying out EC2 from Amazon Web Service. It was trying out k3s cluster deployment with the type and quantity of nodes he was using. It was just pure learning along with the struggles to make sense of the deployment. Really love it, and wish to call out on his heart to learn! KEEP IT UP! Learning so much from you (:




What is Kubernetes again?


Kubernetes in short is a container orchastrator. Where do you see this icon? That is right, on a container ship. Those that are really huge and is capable to load thousands of big metal containers to ship out exports.


Here is a brief video on what is kubernetes:





My personal venture with Kubernetes


I personally use kubernetes and learn how to create a k3s cluster in just about any on-premise environment. Thankful to my colleague and a guide that I’ve already memorised by now, I am almost always able to create a cluster along with all the services required within an our. Other than one time, when I was ‘destroyed’ by a permission denied error when I was starting the cluster. After months of trying, yes I meant it. Months of trying. I finally figured out that generally Linux /var/lib/ directory has a different volume mounted as compared to /mnt_app/ directory. Usually if you want to deploy your own applications, any mounting required will go under /mnt_app/ directory. Perhaps it is a linux newbie mistake, but hey, in every failure is a lesson ready to be learnt. That will definitely be a lesson I will hold tightly for the rest of my development years.


Personally, as I start to learn more about kubernetes and the variety of commands the Ctl provides, I really am getting myself a wider exposure to the world of networking and system administrators. Here’s a cheatsheet that I found useful for quick reference. But of course, you definitely have to first know what are the use cases in which you have to use these different commands. So in this article, I will be covering the top 5 most commonly used commands that I’ve used in kubectl. Just a little space for me to consolidate my learning. These commands are indicated by the frequency of its use. Not by the order of the analogy I am about to use of an empty table with rubik’s cube. Please note for simplicity sake, there are a lot more granularity such as deployment, statefulset, replicaset, api server, nodes, that are intentionally omitted for a simple non-technical individual to peek into this complicated concept of kubernetes.


Kubectl cheatsheet


1. Kubectl get pod


Image kubernetes to be like an empty table. Now this table you divide it into 6 equal parts by drawing a line. Each of these small squares are called a namespace. In each namespace, you put a 3x3 rubik’s cube and that is called a pod.
kubectl get pod -A

Will get you all the pods on the empty table. You will run this command when you want to get


The reference for this command is found here


2. Kubectl describe pod


Now that we know what is a pod, aka a 3x3 rubik’s cube on the table. Next up, we will describe it. What are its current properties.


kubectl describe pod <pod-name>

You want to run this when you wish to debug a pod. That is what I would usually do. Oh! It is not running, let’s describe the pod to get some of its properties and see what error did it indicate for its failure to run.


The reference for this command is found here


3. Kubectl apply -f update-manifest.yaml


Now with your empty table, you want to add stuff into. You have to instruct the butler that is in-charge of your table, and the conversation goes like this:


“Hi sir, what may I do for you?” asked the butler.


“Oh here’s my list of requests I want to have to make”. Proceeding to pass him a single list of the cubes I want to put onto the table.


“I would like to add in 2 rubik’s cubes in the space called “server”. 3 rubik’s cube in this space called “cert-manager”. Oh and help me add in a new space on the table I would like to name as “Joshua”, then add in 1 cube for me there. Thank you."


This is a simplified illustration of kubernetes and what you can do to update your cluster through the api server which is the butler serving my table.


You want to use this command when you want to add your application or services into the cluster.


The reference for this command is found here.


4. Kubectl get deployment


Okay, now things gets a bit more sci-fi and you need to use bit more imagination in our analogy. When we mentioned that we are adding these rubik’s cubes onto the table, what we are actually doing is to put a little ‘person’ who will create that rubik’s cube for you. I tell butler I want one rubik’s cube in Joshua’s namespace. Butler proceeds to put one mini person in Joshua’s space on the table, and tells him: “create 1x rubric cube and make sure it is running correctly, tell me if it’s wrong so I can tell my customer about it”.


This mini-person is actually called a deployment. It is in charge of creating one pod and if that pod is down or crashed due to an error, it will attempt to start it up again, always ensuring that there is a desired quantity of 1 pod running as indicated in the manifest. It is amazing how a process manager such as PM2 can now be added into your cluster to make sure your services are running while you sleep soundly at night.


The reference for this command is found here


5. Kubectl logs pod-name -f


Now you want to see the logs that are printing out by your services, you can now take one of these cubes, and run this command:


kubectl logs <pod-name>

And it will print out the logs of your service. You want to do this when you are checking your logs and making sure all services and requests are running okay. And if there is an error, what error could that be.


The reference for this command is found here




More cool kubernetes commands to include into your daily use


Beyond this point gets a whole lot more technically intense. Do not proceed if you are currently already lost with all the technical terms at this point of time. Now I think the next 4 coolest commands that I’ve learnt from this kubectl cheat sheet


1. Kubectl rollout status deployment/frontend -w


Firstly, the command “rollout” sounds really cool. This command allows you to follow through on a deployment of a resource and ensure that the deployment status reaches running, which indicates that your service is running without error (though there are exceptions of course).


2. Kubectl cordon my-node, kubectl drain my-node, uncordon my-node.


kubectl cordon my-node        # Mark my-node as unschedulable
kubectl drain my-node         # Drain my-node in preparation for maintenance
kubectl uncordon my-node      # Mark my-node as schedulable
kubectl top node my-node      # Show metrics for a given node

These commands are really cool and super useful to debug or to decommission a node that is currently in used. Drain allows you to indicate to the kubernetes api-server not to use this node anymore, and any pods that are running on this node should be rescheduled to other nodes. So this ensures that the services are up and running while removed from the server that it is currently running on.


The reference for this command is found here


3. Kubectl api-resources -o name


This allows you to list out al the api-resources that you currently have in your cluster. this gives a simple overview to what kind of resources does your cluster have. I kinda use this only when i want check if the custom resource that I’ve added into the cluster has been removed entirely.


The reference for this command is found here


4. Kubectl verbosity level


This is intensely cool. I never knew that we are allowed to indicate the verbosity of our outputs from the cluster. This is so amazing and definitely useful when you want to debug your cluster. Oh here it indicates that --v=4 is to debug level verbosity.




You want to version control your configurations


I wish to add that which some of these commands are really useful, they should be avoided if possible.

Kubectl patch, kubectl edit, kubectl annotate, kubectl scale, 

In the world of deployment, there is a concept called infrastructure as code. Under this concept is a sub-concept called configuration management. Configuration management ensures that all the services and deployments you run on the cluster can be replicated and since there are recorded down in files that can be pushed into a version control repository, if anything happens, you can just tear down the entire deployment and just reapply them again because all the configurations needed to reach your desired deployment state are checked into a version control repository.


In view of this configuration that we want to maintain and kept as our source of truth, we do not want any changes done to the cluster that are not included into the configuration and checked into version control. the commands indicated above allows us to change the configuration of the cluster, and creates a deviation from the configuration code if one forgets to update the configuration file. So I would often like to avoid these commands if possible. Instead, simply add that new configuration into your configuration file, and kubectl apply it again. Then now you have that configuration documented down and updated to that source of truth.