Use Wireshark to analyze TCP throughput bottlenecks.

When debugging network quality, we generally focus on two factors: latency and throughput (bandwidth). Latency is easier to verify with a single ping or mtr. This article shares a way to debug throughput. Scenarios where throughput is important are generally the so-called Long Fat Networks (LFN, rfc7323). For example, downloading large files. The throughput does not reach the upper limit of the network and can be affected by 3 main factors.

kubernetes Network Model

In this paper, we will explore the network models in Kubernetes, as well as analyze various network models. Underlay Network Model What is Underlay Network Underlay Network as the name suggests is the physical network topology that the network equipment infrastructure such as switches, routers, DWDM are linked into using the network media that is responsible for the transmission of packets between networks. Source https://community.cisco.com/t5/data-center-switches/understanding-underlay-and-overlay-networks/td-p/4295870 The underlay network can be either layer 2 or layer 3; a typical example of a layer 2 underlay network is Ethernet Ethernet, and a typical example of a layer 3 being an underlay network is the Internet Internet.

Talking about distributed consensus algorithms and data consistency

One of the most important abstractions in distributed systems is consensus: all the nodes agree on a certain proposal. One or more nodes in a distributed system can propose certain values, and a consensus algorithm will decide the final value, while the core idea of consensus is that the decision is unanimous and once decided, it cannot be changed. This article summarizes some common consensus algorithms and theories in the distributed domain, hoping to gain a more comprehensive knowledge.

How to play Dapr without Kubernetes?

Dapr is designed as an enterprise-class microservices programming platform for developers that is independent of specific technology platforms and can run “anywhere”. Dapr itself does not provide an “infrastructure”, but rather uses its own extensions to adapt to specific deployment environments. In its current state, a true native Dapr application can only be deployed in a K8S environment if you wish to take it into production. While Dapr also provides support for Hashicorp Consul, there does not appear to be a stable version available.

Snapshot in Postgres

I recently wanted to learn something about the postgres ecosystem, and I didn’t quite understand its MVCC mechanism before, so I came back to try to understand it again. Here we ignore the concurrency control and cleanup part of MVCC and just look at the Snapshot part first. Tuple Postgres doesn’t have the MySQL kind of UNDO log, the multi-version data (Tuple) is stored directly in the tablespace with meta information to distinguish the versions.

GitOps Tool Argo CD Hands-On Tutorial

Argo CD is a continuous deployment tool for Kubernetes that follows the declarative GitOps philosophy. argo CD automatically synchronizes and deploys applications when Git repositories change. Argo CD follows the GitOps model, using Git repositories as the true source for defining the required application state, and Argo CD supports multiple Kubernetes manifests. kustomize helm charts ksonnet applications jsonnet files Plain directory of YAML/json manifests Any custom config management tool configured

CPU Cache Coherence and Memory Barrier

On modern CPUs (most of them), all memory accesses need to go through layers of cache, and understanding the CPU cache update coherency issues can be of great help in designing and debugging our programs. This article will introduce the CPU cache system and how to use memory barriers for cache synchronization. The memory hierarchy of early computer systems had only three levels: CPU registers, DRAM main memory, and disk storage.

Principle and Implementation of LSM-Tree and LevelDB

LSM-Tree is a data structure for write many read few application scenarios, which is adopted by powerful NoSQL databases such as Hbase and RocksDB as the underlying file organization method. In this paper, we will introduce the design idea of LSM-Tree, and analyze how LevelDB using LSM-Tree is implemented and optimized for performance. Before understanding LSM-Tree, the storage systems I have studied, such as MySQL and etcd, are all oriented

Talk about the principle and optimization of std::uniform_int_distribution

Normally, the Pseudo Random Number Generator generates integers in the interval [0, 2^N). Each integer in this interval has an equal chance of being generated. However, the random numbers we need usually belong to a smaller interval (e.g., the dice simulator needs random numbers in the interval [1, 6]), so we must use std::uniform_int_distribution to map [0, 2^N) to the interval [min, max]. This article follows Daniel Lemire’s paper Fast Random Integer Generation in an Interval and introduces several different mapping methods in order.

Developing Kubernetes Operators with Go: Basic Architecture

Image credit from “Kubernetes Operators Explained A few years ago, I called Kubernetes the de facto standard for service orchestration and container scheduling, and today K8s is the unchallenged “kingpin” of the space. However, while Kubernetes has evolved to become very complex today, the original data model, application model and scaling approach of Kubernetes is still valid. And application models and scaling methods like Operator are becoming increasingly popular with developers and operators.

Virtual memory in Linux systems

Modern operating systems use virtual memory for memory management, and this article summarizes the principles of virtual memory and some application scenarios. The main memory of a computer system is composed of M contiguous byte arrays, each of which has a unique Physical Address (PA). Under these physical conditions, the most natural way for the CPU to access memory is to use physical addressing, as shown in the figure below, where the CPU executes a load instruction to read the data at physical address 5.

Linux Virtual File System

A file system is a mechanism for organizing data and metadata on a storage device, and with such a broad definition, implementations vary greatly from file system to file system, including ext4, NFS, /proc, etc. Linux uses a layered architecture that separates the user interface layer, the file system implementation, and the drivers for the storage device, and is thus compatible with different file systems. The Virtual File System (VFS) is a software layer in the Linux kernel that provides a standard, abstract set of file operations in the kernel, allows different file systems to coexist, and provides a unified file system interface to user space programs.

Google C++ Style Excerpt Notes

Why this excerpt? Because when I was learning C++ code, I found that its functional advantages were too powerful and that there were no strong restrictions on writing code, and C++ has actually carried a lot of old-world atmosphere after decades of iteration, which has led to different habits of people from different backgrounds writing C++ code in different times, so I wanted to learn as much as possible from how good teams write C++ code, so I wrote this one excerpt.

Reverse mapping of anonymous pages in Linux

Linux provides the memory map feature, which implements the mapping of physical memory pages into the process address space for efficient data manipulation or transfer. The kernel uses three important data structures struct vm_area_struct , struct anon_vma and struct anon_vma_chain to handle this feature, so it is important to understand these three data structures, and this article tries to clarify the origin and connection between them. vma The struct vm_area_struct is often referred to as vma in kernel code, so This structure is referred to by vma in the following.

Introduction to Browser Fingerprinting Technology

In daily life, biometric technology is already standard in most smartphones, and most phones are equipped with face recognition and fingerprint recognition, and the current fingerprint recognition technology is already very mature. But what we want to talk about today is not the fingerprint recognition in biometric technology, but the browser fingerprint. Many people love and hate this technology, why is this? Then we’ll take a deeper look at browser fingerprinting today.

eBPF, a Linux kernel monitoring technology

BPF is a very flexible and efficient virtual machine-like component in the Linux kernel, capable of safely executing bytecode at many kernel hook points. This article briefly organizes the technical principles and application scenarios of eBPF. eBPF Origins BPF is called Berkeley Packet Filter, as the name suggests, BPF was originally used for packet filtering and was used in the tcpdump command, for example running a filter rule like tcpdump tcp and dst port 443 would copy packets with protocol tcp and destination port 443 to the user state.

Google B-Tree

B-Tree and its variants data structures are widely used in storage systems, database systems, mainly as indexes for dynamic random access data scenarios. google/btree (github.com) is a pure in-memory B-Tree implementation written in Go, and its source code is analyzed in this paper. Overview B-Tree structure Organization and maintenance of large ordered indices This paper presents the B-Tree data structure. The query of B-Tree starts from the root node and performs a dichotomous lookup of the ordered data within the node and ends the query if it hits, otherwise it goes to the child node query up to the leaf node.

Radix Tree and Gin Implementation

Overview Trie Trie, also known as dictionary tree and prefix tree, is a multinomial tree structure, whose core idea is space for time, using the common prefix of strings to reduce unnecessary string comparisons to improve query efficiency. The figure below. It can be seen that two keywords with a common prefix have the same path in the prefix part of Trie, so Trie trees are also called prefix trees.

Talking about distributed lock implementation

In distributed systems and microservice architectures, different systems or different nodes of the same system share the same or a set of resources, then access to these resources often requires mutual exclusion to prevent interference with each other and ensure consistency. Distributed locks provide mutually exclusive access to shared resources in a distributed environment, and businesses rely on distributed locks to pursue efficiency gains or rely on distributed locks to pursue absolute mutual exclusivity of access.

JavaScript ESM is great, but it might not be so great right now

Preface As many front-end developers probably know, more than a year since sindresorhus published the esm only manifesto last year, many projects have started to move to esm only, i.e., esm only and not cjs, in order to force the whole ecosystem to migrate to esm only faster. Some popular projects already do this thousands of npm packages maintained by sindresorhus node-fetch remark series more. They claim: you can still