Kubernetes Learning Path

Today Kubernetes is present in all major clouds (Google cloud, AWS, Azure, etc…) as a fully managed service, serverless at this point and It is getting extensible day by day so that new components can be added on top of it without any hassle. At present Kubernetes has become the talk of the town and learning Kubernetes is important for techies and new aspirants. The intent of this article is to provide a Kubernetes learning path for beginners who are eager to learn about it.

Since Kubernetes is growing very rapidly, online tutorials, books are getting easily outdated, but most of the information is available on the official website. But information is scattered and takes good time and effort to find the needed information. So I have made an effort to collate all the important topics with links that provide the learning path for Beginners to get started with Kubernetes. This is not the final list. If you need to learn more, please explore the official website.

For your better understanding, I have divided the concepts into 8 phases and you can target each phase for a week or two and learn from Kubernetes basics to advanced. This learning path will help you to learn Kubernetes in a decent amount of time.

Pre-requisites to learn Kubernetes

This Kubernetes learning path is intended for beginners. Basically you should have a good understanding of command lines, Linux, YAML.

Phase 1: Kubernetes Basics

The goal of this Phase is to learn 

1. What is Kubernetes and its features?
2. Understand basic concepts and usage of Kubernetes 
3. Setup Kubernetes environment. 

Learning path 

Before learning any technology, it is important to understand the background and significance of its appearance.  In terms of Kubernetes basics, cover below topics

1. What is Kubernetes and its features? 
2. How Kubernetes works?
3. Common Kubernetes Scenarios?
4. Understand the basics of Containers?
5. What problem does Kubernetes solve? 
6. Are there any other similar systems available in the market today? 
7. Explore what are the services offered by Kubernetes managed service providers. 
8. Explore case studies in Using Kubernetes on an Enterprise Scale.

Once you are comfortable with the Kubernetes terminology, it is best recommended to set up a local environment using minikube and start deploying test applications to get proficient with Kubernetes resources and concepts such a Pod, Node, Service, Labels and Selectors, Annotations, Config,  Secrets, Deployment, Namespace and moreover Understanding Kubernetes Objects

Note: You can always search the reference materials and online courses in your own way. 

Recommended official document

1. Official Tutorial: Learn Kubernetes Basics
2. Official Guestbook example: Guestbook Example

My Two Cents

If you are starting as a beginner to learn Kubernetes, please use Kubernetes as a black box till you get acquainted with basic concepts. Don’t try to explore too much information in phase 1. You will easily get confused with a lot of information in the reference materials.

Phase 2: Basic architecture of Kubernetes

The goal of this Phase is to learn 

1. The basic Architecture of Kubernetes.
2. The basic process of Kubernetes container scheduling

Learning path 

In the second phase, start learning with the basic architecture of Kubernetes. Understand how the components fit together and set up Kubernetes manually which is recommended to understand more about how components are working with each other. Below topics should be covered at least to have a fair understanding of Kubernetes architecture components.

• Master & Node
  • Understand what is the relationship between master and node and how they communicate with each other. Moreover, understand what is master and what is Node.
• Master component
  • API Server. How does k8 receive the request, and how does it return the response to the client.
  • Etcd. Understand what is the major role of Etcd.
  • Controller Manager. Understand the principle of the controller, and know roughly what types of controllers Kubernetes contains by default.
  • Scheduler. Understand the role of the scheduler in the entire scheduling process.
• Node components
  • Kubelet. Know how Kubelet accepts scheduling requests and starts the container.
  • Kube-proxy. Understand the role of Kube-proxy and its capabilities.
  • Container Runtime. Understand what Container Runtimes are available with Kubernetes.
• Core Addons and Plugins
  • DNS. DNS provides support for cluster service discovery. Kubernetes v1.13 starts to use CoreDNS by default. Check the version details of CoreDNS shipped with k8 here.
  • Network Plugin. The Kubernetes multi-node environment needs to deploy a network plug-in to enable the network connectivity of the container and remove the resources when the container is deleted. By default,  K8 allows the usage of kubenet, a non-CNI network plugin.
  • There are several third-party CNI plugins out there. Among the most famous ones are:
    • Flannel
    • Calico
    • Cilium
    • Weave Net

Note: In this phase, you have to get a good understanding of Kubernetes components and interaction between the components and how the containers are managed and provide services to the outside world. 

Recommended official document

1. Kubernetes Components

My Two Cents

Don’t memorize all components of Kubernetes in this phase; To understand k8 components more clearly, deploy a Kubernetes cluster from 0 (scratch) to get a deep understanding of each component. If you want to speed up your learning process, you can also try to use tools such as kubeadm to deploy the cluster.  

References

Refer to the following link for a simple explanation of Kubernetes architecture components.

Phase 3: Kubernetes core concepts

The goal of this Phase is to learn 

1. Understand Kubernetes core concepts

Learning Path

Kubernetes core concepts – In brief you should develop a good understanding of pods, nodes, node pools, control plane, namespaces, daemon sets, deployments, YAML files, and more. Also, it is equally important to get a clear picture of how Kubernetes schedules pods into nodes and which algorithm (Bin Packing)  it uses to achieve hardware capacity utilization. 

The below are the key directions that will deepen your understanding of its various technical points. In this phase 3, you have to concentrate only on the core capabilities of Kubernetes listed below and other functions please learn based on your ability. 

Workloads

• Node 
• Cluster
• Pod 
• Namespace   

Scheduling

• Scheduler
• Taints and Tolerations
• Pod/Node Affinity & Anti-affinity
• Resource Bin Packing 
• Pod Disruption Budget
• Pod Priority and Preemption

Pod Controllers

• Deployment 
• StatefulSet
• DaemonSet 
• HorizontalPodAutoscaler 
• PodDisruptionBudget 
• Job 
• CronJob 
• Pod Lifecycle & Healthcheck & RestartPolicy
• Pod Init-Container

Configuration

• ConfigMaps
• Secrets

Storage

• Volumes
• Persistent Volumes
• Storage Classes

Services / Networking

• Ingress 
• Ingress Controllers
• Service 
• Network Policy 
• Load balancing
• EndpointSlices
•  Iptables
• The principle of DNS service, and the CoreDNS scheme

Security – RBAC 

• ServiceAccount 
• Role 
• ClusterRole 
• RoleBinding 
• ClusterRoleBinding
• AuthN , AuthZ & Admission Control
• Pod/Container SecurityContext
• Pod Security Policy (PSP)

My Two cents

In this phase, the difficulty index is a little high, because you should have a good hold on core concepts including  POD, node, schedulers, DaemonSet, StatefulSet, StorageClass, NetworkPolicy, security, PSP, and so on.  Moreover, you should have a deeper understanding of how these resources are working under one umbrella.

Phase 4: Understand Kubernetes Api and subsystems of Kubernetes

The goal of this Phase is to learn 

1. Understand Kubernetes API structure
2. How to satisfy non-functional requirements using Kubernetes.
3. Deepen the understanding of various subsystems of Kubernetes.

Learning Path

In this phase, you have to understand the API structure, NFR, and subsystems of Kubernetes. The core concept of Kubernetes is an API-driven system. It has a scheduling session system that has a master to keep track of the resources, scheduling (Containers), State of the cluster. So Kubernetes API allows you to query and manipulate the state of the objects in Kubernetes. The core of the k8 control plane is the API server. So you have to understand the API completely in the below areas

1. How the Kubernetes API controls the version
2. How Kubernetes APIs are grouped
3. Representation method and design concept of Kubernetes objects
4. Access control of Kubernetes API

Next, Though your project requirements will drive you around serverless or managed solutions (GKE, AKS, EKS, third party providers ) for running your workloads on either cloud or on-premise. But you have to understand deeply on how to satisfy non-functional requirements like Performance, Scalability, Availability, Reliability, Maintainability, and security using Kubernetes, and Deepen the understanding of various subsystems of Kubernetes.

 Below is a brief of important items needed to be understood on NFR

1. Performance 
2. Scalability
3. Availability
4. Reliability
5. Maintainability
6. Security

Here is a brief list of important subsystems of Kubernetes:

1. Authentication & Authorization.
2. Node
3. Scheduling
4. Storage
5. Network
6. Command-line CLI
7. MultiCluster
8. K8 across Cloud Service Providers
9. Auto Scaling
10. Monitoring 

My Two cents

The last phase has exposure to only Kubernetes basics. In this phase learning Kubernetes API + NFR + Subsystems of Kubernetes will give you a clearer understanding of Kubernetes concepts and API structure.

Note: You can also learn about the Kubernetes API group by going through the Kubernetes API code repository. The following link will clearly explain the overall specification of API. Next, You can browse the Kubernetes/Community code repository to learn about various interest groups (SIG), “SIG” is the abbreviation of Special Interest Group. For example, sig-apps is responsible for deciding whether to introduce new APIs or existing ones that can be upgraded, etc.

Phase 5: Deep understanding of Kubernetes Architecture component capabilities

The goal of this Phase is to learn 

1. Deepen the understanding of Kubernetes architecture component capabilities
2. Learn more Kubernetes cluster-level functions

Learning Path

After you understand Kubernetes core concepts and learn enough API resources on how to use them, let us understand more about Kubernetes master & node architecture and the components that they run. In fact, each component of Kubernetes has strong configuration capability and we can deep dive into some of the most used functions of Kubernetes around each component of Kubernetes.

I would recommended to use the Kubernetes Command Line Reference to understand the configuration of these components:

1. Kube-api-server
2. Etcd
3. kube-scheduler
4. kube-controller-manager
5. kube-proxy
6. kubelet
7. Kubectl
8. Container Runtime.

I have listed some of the functions consolidated for Master and Node for your reference.

Master

• Auditing
  • Kubernetes Auditing provides a dynamically configurable audit function
  • API Server with the corresponding –audit- parameters prefix
• All Admission Controllers
  • List all the admission controllers supported by Kubernetes, and each admission is associated with a specific function of Kubernetes
  • Corresponding to the API Server –enable-admission-plugins parameter, the parameter comment lists all the default admission controllers

Other notable parameters include:

1. Controller-Manager –node-eviction-rate
2. Controller-Manager –pod-eviction-timeout
3. Controller-Manager –terminated-pod-gc-threshold
4. Controller-Manager –pv-recycler-minimum-timeout-*
5. API-Server –max-requests-inflight,–min-request-timeout
6. API-Server –watch-cache,–watch-cache-sizes

Node

• Image GC
  • Clean up the disk space occupied by the container image
  • Kubelet with the corresponding –image-gc- parameters of the prefix, and the –minimum-image-ttl-duration parameters
• Resource Reserve
  • Reserve certain resources for system resources to ensure the stability of nodes
  • Corresponding to Kubelet –kube-reserved and –kube-reserved-cgroup other parameters
• CPU Manager
  • Provide more CPU management capabilities, such as static CPU affinity
  • –cpu-manager-* Parameters corresponding to the Kubelet prefix
• Storage Limit
  • Avoid excessive mounting of data volumes on nodes.
  • Change limits by setting value KUBE_MAX_PD_VOLS env variable.

Other notable parameters include:

1. Kubeproxy –nodeport-addresses
2. Kubelet –host-*
3. Kubelet –fail-swap-on
4. Kubelet –cgroups-per-qos
5. Kubelet –max-pods,–pods-per-core
6. Kubelet –resolv-conf

My Two Cents

Kubernetes provides FeatureGate to control different feature switches. Based on the feature stages (Alpha, Beta, GA) you can learn the new features of Kubernetes. Note in GA the feature is always enabled; you cannot disable it. In addition, in order to facilitate developers and configuration management, Kubernetes have moved all Component Configurations to the following GitHub code repository, which I have listed for your reference.

1. https://github.com/kubernetes/kubectl
2. https://github.com/kubernetes/kubelet
3. https://github.com/kubernetes/kube-proxy
4. https://github.com/kubernetes/kube-controller-manager
5. https://github.com/kubernetes/kube-scheduler

Phase 6: Extending Kubernetes 

The goal of this Phase is to learn 

1. Learn about the extension mechanism of Kubernetes
2. Write a sample Kubernetes controller.

Learning Path

In this phase, we can begin to understand the various extension mechanisms of Kubernetes. Kubernetes core concepts, architecture design, and Api are the major cornerstones, but here extension methods are meant to be an architectural extension and not a functional extension. For example, Pod supports various Probes to perform health checks, including customization. Here I do not classify it as an extension mechanism.

API resource extension 

1. CustomResourceDefinition: Customize the Kubernetes API
2. API Aggregation: Aggregation of multiple API Servers, suitable for a large amount of API customization

An important way to learn API resource extension is to create an extension resource or write your own controller. I would strongly recommend writing a controller by yourself and understanding it thoroughly. Here are a few common tools to develop your own Kubernetes APIs

1. kubebuilder : the official API extension project from Kubernetes
2. sample-controller : A sample from the Kubernetes official
3. operator-sdk : an operator library from Red Hat

API access extension 

1. Authentication checks whether the user is a valid user of the API and if it is valid, establishes its user identity.
2. Authorization checks whether users have permission to read, write and execute the requested operation.
3. Admission control  Once the request is authenticated and authorized, the admission controller intercepts and performs a variety of additional configurable checks on the request to enforce cluster policies.

Kubernetes API access extension is mainly achieved through Webhooks (Authentication webhook, Authorization webhook,  Admission webhook ). The external services for authentication and authentication are registered when the API Server is started and cannot be added later. This will cause too much security risk when the external authentication and authentication services are dynamically added.

Scheduler expansion capability

1. Extension interface (Scheduler Extender): similar to Webhook, call external services to make scheduling decisions
2. Multiple schedulers: Support multiple schedulers to schedule different jobs in Kubernetes
3. Scheduler framework:  pluggable architecture for the k8 scheduler. Users can perform “code-level” customization without forking Kubernetes Scheduler code.

The scheduler extender can be a good option to use in many scenarios, but note that it does have some limitations like extender is limited to certain phases, such as “Filter” and “Prioritize”, meaning they cannot be called in the beginning or middle of any phase and so on.

Network expansion capability

1. Network plug-in CNI: Using the CNI plug-in, we can choose any network scheme we need
2. Ingress Controller: Ingress defines a set of API interfaces, we can choose to implement it randomly.
3. NetworkPolicy Controller: Same as above, optional implementations include Calico, Cilium, etc.
4. DNS controller: Kubernetes defines a set of DNS specifications, we can choose to implement it randomly

The network plug-in CNI is a container network standard. Kubernetes provides good support. Commonly used plug-ins include flannel, Calico, and so on. Regarding Ingress, NetworkPolicy, DNS, you should understand thoroughly. It is essentially a set of APIs defined by Kubernetes. The underlying implementation is pluggable, and users can have their own choices.

Storage expansion 

1. FlexVolume: A dynamic docking storage solution provided by Kubernetes, which supports user-defined storage backends.
2. Storage plug-in CSI: Using the CSI plug-in, you can choose any storage solution you need.

FlexVolume is a solution that comes with Kubernetes to connect to external storage. Users can write a small amount of code to add a custom storage backend, which is suitable for simple scenarios. The storage plug-in CSI is a container network standard and it provides a complete set of SDK solutions to facilitate third-party implementation. Currently, it supports more than 100 csi which can be used with Kubernetes.

Runtime scalability

Runtime interface CRI: Using the CRI plug-in, you can choose any runtime solution we need.

The runtime interface CRI is a solution proposed by Kubernetes to support multiple runtimes. At any runtime, you only need to implement CRI-related interfaces to connect to Kubernetes. Some of the common CRI with Kubernetes (Linux) are containerd, CRI-O, Docker

Resource expansion capability

1. Extended Resource: Support for adding custom resources through Kubernetes native API
2. Device Plugin: Use Device Plugin to connect to any system hardware resources we need.

The core of the Device Plugin is to automatically access various special hardware such as Nvidia, Infiniband, FPGA, etc. Currently, the most used Device Plugin is mainly Nvidia’s GPU device plugin.

Monitor capabilities

Custom monitoring: support the use of custom monitoring components such as Prometheus to provide monitoring indicators

Custom monitoring includes Resource metrics (Metrics Server), Custom Metrics (Prometheus Adapter), and External Metrics (Keda). 

Cloud provider scalability

Cloud Controller Manager: supports pluggable cloud service providers

The goal of cloud scalability is to enable various cloud providers to access their services without changing the Kubernetes source code. Each cloud provider has its own project.

Command-line plugin

Kubectl Plugin: The kubectl plugin supports the extension of kubectl subcommands, which can provide nearly native usage in combination with API extension capabilities.

My Two cents

I would recommend implementing a sample end-to-end Kubernetes controller, which will give an idea of all the expansion capabilities, I would recommend establishing an extensive understanding first and then go deep into a certain capability when needed.

Phase 7 : Kubernetes Hello world

The goal of this Phase is to learn 

1. Create a simple Kubernetes Hello world

Learning Path

In this phase, we will take a simple scenario and try to implement all the basic concepts to gain confidence with Kubernetes. From my experience, any software I am learning, I used to start with the “Hello World” example to get started on.

Scenario: Let us take a company named TechBlost, which is developing a website that supports selling Grocery Products online and their minimum Requirement is to use the Kubernetes Microservices-based solutions. You use the Kubernetes latest Version as the framework to build the end-to-end TechBlost suite. 

1. Create your Node.js application 
2. Write a small Service Application Code in Node.js with a file named productService.js and push the code into GitHub. You can run this simple command to see your “Kubernetes Hello World!” message at your localhost (“http://localhost:8080”).  Example- node productService.js
3. Package your application in a Docker container image.
4. All services should have respective Application Config in the same GitHub location as code and it should be mounted in their corresponding pods with the following location /config/appconfig
5. All services should have respective Environment Config in the same GitHub location as code and it should be mounted in their corresponding pods with the following location /config/envconfig
6. All services should have respective Secrets Config in the same GitHub location as code and it should be mounted in their corresponding pods with the following location /config/secconfig and it should be encrypted as base64.
7. All the service application should be Independently Deployable Service and Deployments
8. All the service should be Autoscalabale based on a CPU Threshold of 30%
9. All Configs should be externalized
10. All Secrets should be externalized
11. You should use Rhel latest version as the base image
12. Each service/Deployment should be deployed on a unique namespace.
13. The application should be limited to use the CPU and Memory as per the above requirement.
14. All Deployments should be done via Kubectl and YAML Configs.
15. All Services/Deployments should have labels like Name, Author, Version.

My Two Cents

I know this requirement will be a little vague for a beginner. But until you are not trying you will not get an idea on how Kubernetes works. 

Phase 8: Kubernetes Resources and News

The goal of this Phase is to learn 

1. Follow Kubernetes resources, community.
2. Follow CNCF community development

Learning Path

I believe you have learned a lot of Kubernetes concepts so far, but it is not only the most important part, you have to always stay up to date on Kubernetes news. In this follow-up on the Kubernetes new releases and follow up on the community to keep you updated on Kubernetes.

Also, follow up on the CNCF community development for several important projects which are available in the Graduated / Incubation stage.

Projects Graduated

1. Envoy: is a high-performance open-source edge and service proxy that makes the network transparent to applications.
2. Helm: helps you manage Kubernetes with package management capabilities to facilitate users to quickly define, install and upgrade various services.
3. Prometheus: is an open-source system for monitoring and alerting and it is one of the most important monitoring components in the cloud-native environment.
4. Open policy Agent: is an open-source, policy engine that unifies the policy implementation across the IT environments, more especially in cloud-native applications. 

Above, I have only listed a very small number of important projects, and there are so many projects around Kubernetes and explore it based on your interest. Next, we will list some reliable resources to keep you updated.

1. Kubernetes Blog:  Official Kubernetes blog
2. CNCF Blog: CNCF blog
3. Kubernetes StackOverflow: Kubernetes  QUESTIONS and Answers for DEVELOPER specific
4. Kubernetes Forum: Kubernetes Community forums
5. Kubernetes Slack channel: Kubernetes Slack Channel
6. Kubernetes Twitter: Kubernetes news in Twitter
7. Kubernetes Enhancement: Focus on the discussion of new features
8. Kubernetes Community: Focus on community organization
9. CNCF TOC: Follow the progress of CNCF, discuss various new projects, etc.
10. Awesome Kubernetes: The Kubernetes project’s immovable series
11. Kube Weekly: The weekly newsletter for all things Kubernetes and beyond.
12. Kube List: Curated weekly Kubernetes information reconciliation loop.
13. Kubernetes Glossary: Comprehensive, standardized list of Kubernetes terminology
14. Kubernetes Resources List

My Two cents

There are a lot of moving parts in the Kubernetes and its ecosystem is evolving day by day with more layers added like (Service Mesh, … ) to the list. My advice to all learning aspirants is to hear and read a lot about real-world implementation, production experience, Kubernetes patterns, and more importantly failure stories to reduce the unknown unknowns of running Kubernetes workloads in your production cluster.

Conclusion

Learning technology is a routine ability that you have to follow religiously. In terms of Kubernetes, you need to follow all the 8 stages to become proficient in Kubernetes.  My advice would be to involve yourself in the practical session of each and every concept for better understanding which will help to not lose track. Keep in mind, your major goal is to

1. Deeply understand the Kubernetes architecture and core capabilities.
2. More Practical sessions.
3. How to locate the capabilities we need in the official documentation and stay updated with the latest news about Kubernetes.

All the best. I welcome your thoughts on this article.