How to optimize docker image size

Recently, after taking over a new project, I streamlined the original 1.6GB image to over 600MB. This article documents some of the lessons learned during the optimization process. Theory and Rationale Image is essentially a compressed package consisting of a image layer and a runtime configuration file. Building a image is the process of generating a image layer and a configuration file by running the RUN, COPY and ADD commands in Dockerfile.

Using Setup and Teardown in Golang's Tests

When writing tests, we all need to start external services such as Redis or Postgres, and we need to initialize the database connection before starting the test, or prepare the test data, and close the database connection after the test is finished, and remove unnecessary test data or files. In Golang, developers don’t need to rely on third-party packages, they can do this very easily with the built-in TestMain. Here’s how to do it and how to use it.

Kubernetes API Server Authentication and Authorization Mechanism

kube-apiserver is the gateway component of kubernetes and is the only entry point for kubernetes cluster resource operations, so some processes such as authentication and authorization are obviously implemented based on this component. kubernetes clusters basically perform all operations through the apiserver component, which provides an API in the form of HTTP RESTful for clients inside and outside the cluster to call. kubernetes provides three steps of security for accessing the API: authentication, authorization, and admission control, which are verified when users request apiserver using kubectl, client-go, or the REST API.

Kubernetes' RBAC mechanism

When I was doing PaaS platform development, I was involved in tenant privilege management, and considering that Kubernetes provides RBAC (role-based access control) mechanism by default, I thought how to make good use of RBAC of Kubernetes to achieve it. However, when I started to learn this knowledge, I encountered some problems, such as Role and ClusterRole, Role Binding and ClusterRoleBinding, and many concepts are rather vague. As we learn

How kubernetes Controller Manager works

The three most important components of the kubernetes master node are kube-apiserver, kube-controller-manager, and kube-scheduler, which are responsible for resource access portal, cluster state management, and resource scheduling of the kubernetes cluster respectively. This article focuses on the kube-controller-manager component and analyzes how it and its core component informer effectively manage cluster state. Overview of how Controller Manager & Controller works We all know that managing resources in kubernetes is relatively simple, usually by writing a YAML manifest, which can be solved directly with the kubectl command.

CNI mechanism and how Flannel works

CNI, whose full name is Container Network Interface, is the API interface for container networks. The direction of kubernetes networking is to integrate different networking solutions by way of plugins, and CNI is the result of this effort. CNI focuses only on solving container network connectivity and resource release when containers are destroyed, providing a set of frameworks so that CNI can support a large number of different networking models and is easy to implement.

Understanding the Golang Context Mechanism

When using some of Golang’s frameworks, such as Gin, the Handler method of each request always needs to pass in a context object, and then a lot of request data, such as request parameters, path variables, etc. can be read out from it, in fact, in the process of using this has generally understood what the context is, but for some of the details including the specific use of the lack of understanding, so this article on the golang inside the concept of context for a brief discussion.

Vmagent Manual

I. What is vmagent The following is the original official documentation. vmagent is a tiny but powerful agent that helps us collect metrics from different sources and store them in vm or other prometheus-compatible storage systems that support the remote_write protocol. Features of vmagent Supports as a replacement for prometheus for grabbing data from, for example, node_exporter Can read data from Kafka. See these documents. Data can be written to Kafka.

Python append() function and deep and shallow copies

I have a problem I wrote the following code, which looks something like the following. In the end, all the elements in the list are exactly the same. 1 2 3 4 5 6 7 8 9 10 >>> students_list = [] >>> student = {} >>> student["name"] = "zhangsan" >>> student["age"] = 18 >>> students_list.append(student) >>> student["name"] = "zhaosi" >>> student["age"] = 25 >>> students_list.append(student) >>> print(students_list) [{'name': 'zhaosi', 'age': 25}, {'name': 'zhaosi', 'age': 25}] Reason: The append() method just stores the address of the dictionary into the list, and the way to assign a value to a key is to modify the address.

Troubleshoot client-go informer cache invalidation issue

Background Elastic Cloud online services have long been plagued by cache inconsistency. The occurrence of cache inconsistency is usually accompanied by the upgrade or restart of the kube-apiserver. When cache inconsistency occurs, the user side can perceive it more obviously, and the problem can cause online service failure when it is serious. The common failures are as follows. Platform data inconsistency: Pod status is normal at one time and abnormal

Apple's business site blocks Firefox browser

According to Ghacks, users will see a “Your browser is not supported” message when opening Apple’s business website business.apple.com in the Mozilla Firefox browser. The message is displayed on all the latest versions of Firefox, including Firefox Stable, Firefox ESR and Firefox Nightly. As you can see from the image, business.apple does block Firefox and recommends that users use Safari, Microsoft Edge or Chrome (most Choromium-based browsers can be viewed normally after testing).

.NET 7 adds Rate Limiting feature to easily limit the number of requests

.NET 7 has a built-in Rate Limiting feature, which limits the number of requests that can access a resource. For example, a database can safely handle 1000 requests per minute, and it’s not sure if more will crash. This is where you can put a rate limiter in your application that only allows 1000 requests per minute and starts rejecting requests after that number is reached. This is a way to protect resources and prevent the application from crashing in case of high browsing.

Vim shortcuts for efficient programming

The following commands are some of the shortcuts I commonly use. I use vscode as my editor and have the vim plugin installed. Cursor movement hjkl: left down up right gj and gk: move to the next or previous physical line, when a line appears linewrap, using j and k does not move to the line as seen visually {: jumps to the previous line at the beginning of the first paragraph found up }: jumps to the next line at the end of the first paragraph sought down (: jumps to the line at the beginning of the first paragraph of the upward search ): jumps to the line at the end of the first paragraph of the downward search w: the beginning of the next word b: the beginning of the previous word e: the end of the next word ge: end of the previous word gg: jump to the first line G: skip to the last line <number> + gg: jump to the number line :<number>: jumps to the number line %: match the corresponding parentheses and jump * and #: match the word the current cursor is on, move the cursor to the next or previous one f, F, t and T, in-line cursor jumping f: the current line jumps to the specified character after the cursor, press ; to jump to the next specified character F: the current line jumps to the specified character before the cursor, press ; to jump to the next specified character t: the current line jumps to a character before the specified character after the cursor, press ; to jump to a character before the next specified character T: the current line jumps to one character after the specified character before the cursor, press ``;` to jump to one character after the next specified character Generally, I only use f and F.

ESLint Tutorials

This article introduces some basic concepts of ESLint, and how to configure ESLint in webpack and vscode. Why do I need ESLint? JavaScript is different from other languages in that it does not have a pre-compilation checking process. If there is an error in your code, the error will only be exposed after the program is run. This exposure of errors greatly affects the efficiency of development. On the other hand, because JavaScript is a dynamic language, if a team is developing without following a certain specification to write code, everyone adds code according to their own style.

Git Submodules

The Git submodule feature allows us to add another repository to a repository as a subrepository of the current repository. Git submodules are simply references to a state of a repository at a given time, i.e. a reference to a commit. Introduction When developing a complex project, it is often necessary to rely on some external code packages, which facilitates code management and saves us the effort and time of creating tools repeatedly.

Troubleshooting docker exec failures

This article focuses on learning about. The direct relationship between kubelet, docker-shim, dockerd, containerd, containerd-shim, and runc How to troubleshoot: How to connect containers using docker, containerd-ctr, docker-runc runc workflow Problem Description Today, in the process of checking the system problem, I found that the system log keeps printing the docker exception log. 1 2 3 May 12 09:08:40 HOSTNAME dockerd[4085]: time="2021-05-12T09:08:40.642410594+08:00" level=error msg="stream copy error: reading from a closed

K8s choose cgroupfs or systemd?

What is a cgroup Cgroups are a Linux kernel feature that limits, audits, and isolates the resource usage (CPU, memory, disk I/O, network, etc.) of a group of processes. cgroups (Control Groups) are a mechanism provided by the linux kernel to consolidate (or segregate) a set of system tasks and their subtasks into different groups according to resource hierarchy on demand, thus providing a unified framework for system resource management. Simply put, cgroups can limit and record the physical resources used by task groups.

The Webb space telescope that captured the universe, with its flight software written in C++ and a built-in custom JS interpreter

NASA yesterday released the first full set of full-color images of the deep space from the James Webb Space Telescope (JWST). According to the report, JWST is estimated to cost up to $10 billion in total. It was lifted off by an Ariane 5 rocket on Dec. 25 last year. Once in space, JWST will need to travel for a period of time before entering its intended orbit, and will remain in orbit thereafter.

Centos8 using grubby to modify kernel boot parameters

grubby is a command line tool for updating and displaying information about the configuration files of various architecture-specific boot programs. It is mainly designed for scripts that install a new kernel and need to find information about the current boot environment, as well as to modify various information parameters for booting the kernel. This article describes how to use the grubby tool to adjust the kernel boot parameters and boot order in centos8.

Add Google Authenticator to the sshd service

This article explains how to add Google Authenticator to your centos 7 system to enable double authentication for ssh to improve the security of ssh services when logging in with a password. 1. Install Google Authenticator on Linux First we need to enable the epel repository 1 yum install epel-release Install the relevant dependency libraries. 1 yum install pam-devel Install google authenticator. 1 2 3 4 # View version information yum list google-authenticator # install yum install google-authenticator After the installation is complete, initialize the configuration by running the following command directly to start the initial configuration.