Kubernetes & Docker Networking Principles (II)

In the previous article, Docker’s network implementation was introduced and discussed. For Docker network, its biggest limitation is the gap in the cross-host container communication scheme, while Kubernetes, as a container orchestration platform suitable for large-scale distributed clusters, mainly solves the following problems at the network implementation level. inter-container communication. Pod-to-Pod communication. Pod-to-Service communication. Intra-cluster and inter-cluster communication. This blog post focuses on Kubernetes inter-container communication and inter-Pod communication, followed by a separate article on Pod-Service communication, which is related to kube-proxy working principle and service mechanism.

Ceph Cookbook

Concept OSD: the program responsible for operating the hard disk, one hard disk one OSD MON: manage cluster status, more important, can run one on each of multiple nodes MGR: monitoring cluster status RGW(optional): provides object storage API MDS(optional): provides CephFS Ways to use Ceph for storage. librados: library radosgw: Object Storage HTTP API rbd: block storage cephfs: file system Authentication Ceph client authentication requires a username + key. By default, the username is client.

An in-depth comparison of Python concurrency schemes

Preface This article is an in-depth comparison of Python concurrency scenarios and their advantages and disadvantages, mainly introducing the asyncio solution. Note: The code in this article requires Python 3.10 and above to run properly. Python Concurrency and Parallelism Schemes There are three concurrency and parallelism schemes in the Python world, as follows: multi-threading multiprocessing asynchronous IO (asyncio) Note: The difference between concurrency and parallelism will not be mentioned first, and will be better explained with examples at the end, and concurrent.

NoCopy in Golang

There is no native way to disable copying in Go. So if you have a structure that you want the user to not be able to copy, but only pointer pass to ensure global uniqueness, you can do that by defining a structure called noCopy and implementing the sync.Locker interface. 1 2 3 4 5 6 7 8 9 10 // noCopy may be embedded into structs which must not be copied // after the first use.

How to view hardware and system information in Linux

Linux often needs to check the system resources or hardware resources, and there are many command lines corresponding to it. In this article, we will summarize and list the common query methods based on our experience for reference. The following commands use Ubuntu 1804 and CentOS7 as the test platform. 1. System resources 1.1. Linux distributions and kernels The uname -r command outputs the distribution and kernel information. Most distributions basically come with this command, but the output may not contain the full name of the distribution.

Kubernetes & Docker Networking Principles (I)

When you are developing and maintaining Kubernetes, the most common concepts you come across are the networking concepts of Docker and Kubernetes. Especially for Kubernetes, there are various IPs, ports, and sometimes confusion. Therefore, it is necessary to learn the underlying network implementation of Docker and Kubernetes. In this article, we will first analyze and introduce the network implementation of Docker. Docker Networking Basics Docker’s networking implementation mainly makes use of Linux networking related technologies such as Network Namespace, Veth device pairs, bridges, iptables, routing.

Linux Netfilter/iptables

The Linux network stack is very efficient and at the same time complex. If we want to do something with the data we care about during the processing of the data, how can we do it?Linux provides a mechanism to implement custom packet processing for the user. There is a set of callback function hooks in the Linux network stack, through which hooks can be attached to the Linux network stack to perform some operations on the packet during processing, such as filtering, modifying, dropping, etc.

client-go Getting Started

client-go is the programmatic interactive client library used by the Kubernetes project. The core source code related to resource interaction is separated out as a project, client-go. That is to say, the Kubernetes used now is integrated with client-go, so the coding quality of this library should be assured. client-go is a programmatic interactive client library that allows you to add, delete, modify, and query resource objects in a kubernetes cluster by writing Go code.

What Is Cloud Native

Exploration of the definition of cloud-native The concept of “cloud native” is mentioned, but how many people really understand this concept? Whenever I browse some communities (not least some professional technical communities) and see some friends discussing cloud-native related topics, I sometimes feel that there is still a little bit of a problem with their understanding of cloud-native. Many people directly think that applications deployed in the cloud are called cloud-native applications.

Istio Sidecar's interception mechanism for traffic

The basic process of flowing traffic through a host Inbound traffic passes through the NIC and enters the host’s network stack. the protocol stack checks the messages against pre-customized network rules (iptables/netfilter). after the stack rules check, compliant Inbound traffic enters from kernel space to user space and enters the process that specifies the listening port. the user process in the user state receives the network traffic message for processing and returns the processed result to the network protocol stack in kernel space via user space.

Istio Sidecar injection mechanism

Service Mesh and Sidecar Concepts Before understanding the injection mechanism of Sidecar, it’s important to clarify the what and why questions. First, what is a Service Mesh? Service Mesh, or translated as “Service Mesh”, is a configurable low-latency infrastructure layer designed to handle a large amount of network-based inter-process communication between application services through APIs (Application Programming Interfaces). The Service Grid ensures fast, reliable and secure communication between infrastructure services of containerized transient presence applications.

A Brief Analysis of Concurrency Models: Shared Memory/Actor/CSP

In Golang programming, when it comes to concurrency problems, there are usually two solutions. Adopt the shared memory model and use sync.Mutex / sync.RWMutex etc. to add locks and set critical zones to solve the data concurrent access problem. Adopt the message communication model and use channel for inter-goroutine communication to avoid memory sharing to solve the problem. The official recommendation is to use the second option, so what is

Permission model (RBAC/ABAC)

I recently researched the permission model, and after reading AWS IAM, I feel that AWS IAM is very well designed. In my personal opinion, RBAC is still not enough for some scenarios, mainly because the control granularity is not enough. For example, I want to control a role can only operate the resources of a certain cluster, RBAC can not express, but ABAC can express, but ABAC is much more complex, AWS IAM is ABAC, but ease of use is very good.

How to optimize docker image size

Recently, after taking over a new project, I streamlined the original 1.6GB image to over 600MB. This article documents some of the lessons learned during the optimization process. Theory and Rationale Image is essentially a compressed package consisting of a image layer and a runtime configuration file. Building a image is the process of generating a image layer and a configuration file by running the RUN, COPY and ADD commands in Dockerfile.

Using Setup and Teardown in Golang's Tests

When writing tests, we all need to start external services such as Redis or Postgres, and we need to initialize the database connection before starting the test, or prepare the test data, and close the database connection after the test is finished, and remove unnecessary test data or files. In Golang, developers don’t need to rely on third-party packages, they can do this very easily with the built-in TestMain. Here’s how to do it and how to use it.

Kubernetes API Server Authentication and Authorization Mechanism

kube-apiserver is the gateway component of kubernetes and is the only entry point for kubernetes cluster resource operations, so some processes such as authentication and authorization are obviously implemented based on this component. kubernetes clusters basically perform all operations through the apiserver component, which provides an API in the form of HTTP RESTful for clients inside and outside the cluster to call. kubernetes provides three steps of security for accessing the API: authentication, authorization, and admission control, which are verified when users request apiserver using kubectl, client-go, or the REST API.

Kubernetes' RBAC mechanism

When I was doing PaaS platform development, I was involved in tenant privilege management, and considering that Kubernetes provides RBAC (role-based access control) mechanism by default, I thought how to make good use of RBAC of Kubernetes to achieve it. However, when I started to learn this knowledge, I encountered some problems, such as Role and ClusterRole, Role Binding and ClusterRoleBinding, and many concepts are rather vague. As we learn

How kubernetes Controller Manager works

The three most important components of the kubernetes master node are kube-apiserver, kube-controller-manager, and kube-scheduler, which are responsible for resource access portal, cluster state management, and resource scheduling of the kubernetes cluster respectively. This article focuses on the kube-controller-manager component and analyzes how it and its core component informer effectively manage cluster state. Overview of how Controller Manager & Controller works We all know that managing resources in kubernetes is relatively simple, usually by writing a YAML manifest, which can be solved directly with the kubectl command.

CNI mechanism and how Flannel works

CNI, whose full name is Container Network Interface, is the API interface for container networks. The direction of kubernetes networking is to integrate different networking solutions by way of plugins, and CNI is the result of this effort. CNI focuses only on solving container network connectivity and resource release when containers are destroyed, providing a set of frameworks so that CNI can support a large number of different networking models and is easy to implement.

Understanding the Golang Context Mechanism

When using some of Golang’s frameworks, such as Gin, the Handler method of each request always needs to pass in a context object, and then a lot of request data, such as request parameters, path variables, etc. can be read out from it, in fact, in the process of using this has generally understood what the context is, but for some of the details including the specific use of the lack of understanding, so this article on the golang inside the concept of context for a brief discussion.