Why does the database lose data

Database management systems are an important part of software today. Open source databases such as MySQL, PostgreSQL, and commercial Oracle are everywhere, and almost all services rely on database management systems to store data. It sounds like a matter of course that a database will not lose data, and persistence capability should be the most basic guarantee of a database, but in this complex world it is very difficult to guarantee that data will not be lost.

The principle of Go language system monitoring implementation

Many systems have daemons that monitor the running state of the system in the background and respond to unexpected situations as they arise. The system monitor is an important part of the Go language runtime, checking the Go language runtime at regular intervals to ensure that the program has not entered an abnormal state. This section describes the design and implementation of the Go system monitor, including its startup, execution process, and main responsibilities.

Why is the latency of live streaming high

Advances in communication technology have contributed to the rise of video-on-demand and live streaming services, and advances in 4G and 5G network technology have made streaming technology increasingly important, but network technology does not solve the problem of high latency in live streaming, and this paper will not describe the impact of the network on live streaming services, but will analyze a common phenomenon in live streaming - the perceptible Significant network latency.

Why HTTPS requires 7 handshakes and 9x latency

HTTP (Hypertext Transfer Protocol) has become the most commonly used application layer protocol on the Internet, however, it is only a network protocol for transmitting hypertext and does not provide any security guarantees, using plaintext to transmit packets over the Internet makes eavesdropping and man-in-the-middle attacks possible. Transmitting passwords over HTTP is actually similar to running around naked on the Internet. Netscape designed the HTTPS protocol in 1994 to secure data transfers using Secure Sockets Layer (SSL), and with the development of Transport Layer Security (TLS), we now use TLS instead of the deprecated SSL protocol, although the term SSL certificate is still used.

Principle of memory allocator implementation in Go language

The data and variables in a program are allocated to the virtual memory where the program resides, which contains two important areas: the Stack and the Heap. Most of the arguments, return values, and local variables of function calls are allocated to the stack, which is managed by the compiler; different programming languages use different methods to manage the memory in the heap area. The objects in the heap are

Why TCP protocol has sticky packet problem

The TCP/IP protocol cluster establishes a conceptual model for communication protocols in the Internet, and the two main protocols in the cluster are TCP and IP. TCP in the TCP/IP protocol cluster guarantees the reliability and order of data segments, and with a reliable transport layer protocol, application layer protocols can directly use TCP to transfer data without the need to care about Loss and duplication of data segments The IP protocol solves the routing and transmission of packets, and the upper layer TCP protocol is no longer concerned with routing and addressing.

Why does the TCP protocol have a TIME_WAIT state?

The TCP protocol contains 11 different states, and the TCP connection transitions states based on the messages sent or received. The state machine shown below illustrates all possible transitions, including not only the state transition process under normal conditions, but also the state transition under abnormal conditions. Both parties communicating using the TCP protocol will trigger the TIME_WAIT state when closing the connection. The operation of closing the connection actually tells the other party to the communication that it has no data to send, but it still maintains the ability to receive data from the other party, a common process of closing the connection is as follows.

Thoughts on k8s CPU limit and throttling

You should be careful about setting the CPU limit for loads in k8s. A value that is too small can introduce extra, meaningless latency into your program, and a value that is too large can introduce too large a blast radius and weaken the overall stability of the cluster. request and limit One of the major benefits of k8s is resource isolation. By setting the request and limit of the load, we can easily allow different programs to coexist on the right node.

go-faker: a mock data generator

go-faker go-faker can generate all kinds of mock data, which is good for testing scenarios. For example, when you need a fake user record in order to complete some kind of registration process unit test, you can use. 1 2 > docker run -it --rm hedzr/faker internet {"Internet":{"DomainName":"ziemann.net","DomainSuffix":"com","DomainWord":"green","Email":"garry@hintz.biz","FreeEmail":"christian@yahoo.com","IpV4Address":"48.107.223.211","IpV6Address":"bc8e:e770:c962:4004:8045:86e2:2658:bfc7","MacAddress":"52:6b:0d:24:7d:67","Password":"Za9pR1hfV7yJXw3Mhf9I5EvL1uvo0M","SafeEmail":"sheridan.gibson@example.net","Slug":"adipisci-inventore","Url":"http://roobroberts.name/americo.lindgren","UserName":"lenny.trantow"}} This gives you a user record with a limited but almost sufficient number of fields. Installation Using the docker run will work

The principle of Go language garbage collector implementation

In almost all modern programming languages, the garbage collector is a complex system that requires a lot of effort in order to reclaim the deprecated memory without affecting the user program. Java’s garbage collection mechanism is a good example. Java 8 contains four garbage collectors, linear, concurrent, parallel marker removal, and G1. It takes a lot of effort to understand how they work and the details of their implementation. In

Stack memory and escape analysis in Go language

The memory of an application is generally divided into heap and stack areas, and the program can actively request memory space from the heap area during runtime. This memory is allocated by the memory allocator and reclaimed by the garbage collector. We have analyzed the process of requesting and releasing heap memory in detail in the last two sections, and this section will introduce the management of Go language stack

Interconversion between Protobuf and JSON

Protobuf does not provide a JSON interconversion method by default, although the Protobuf object itself has a toString() method, but it is not in JSON format, but rather in the form of 1 2 3 4 5 6 7 8 9 10 11 12 13 14 age: 57 name: "urooP" sex: MALE grade { key: 1 value { score: 2.589357441994722 rank: 32 rank: 32 } parent} parent { relation: "father"

Principles of the Go language HTTP standard library implementation

The Hypertext Transfer Protocol (HTTP) is today’s most widely used application layer protocol, drafted by Tim Berners-Lee at CERN in 1989, and has become the core of data transfer on the Internet. Over the past few years, HTTP/2 and HTTP/3 have also updated the existing protocol to provide more secure and faster transfers. The existing protocols have been updated to provide more secure and faster transfers. Most programming languages implement HTTP/1.

The principle of JSON implementation in Go language

JSON (JavaScript Object Notation), a lightweight data interchange format1, holds almost a majority of the market share today. Although it lacks serialization and deserialization performance compared to more compact data interchange formats, JSON offers good readability and ease of use, and it is a very good choice for using JSON as a serialization format when extreme mechanical performance is not sought. Design Principles Almost all modern programming languages incorporate functions for handling JSON directly into the standard library, and the Go language is no exception.

Dynamic libraries and plug-in system for Go language

Developers familiar with the Go language are generally very familiar with the principles of Goroutine and Channel, including how to design applications based on the CSP model, but the Go language’s plug-in system is a module that few people know about. With the plug-in system, we can load dynamic libraries at runtime to implement some of the more interesting features. Design Principles The plug-in system of Go language is implemented based on C dynamic libraries, so it also inherits the advantages and disadvantages of C dynamic libraries.

Why Linux needs virtual memory

CPU and Main memory are scarce resources in an operating system. All processes running in the current operating system share the CPU and memory resources in the system, and the operating system uses CPU scheduler to allocate CPU time and introduces virtual memory system to manage physical memory. Before we can answer the need for virtual memory, we need to understand what virtual memory is in an operating system and what role it plays in the operating system.

Mockito Source Code Analysis (2) - Mock and Spy

Mock Up first throw a timing diagram, later when looking at the source code can be combined with this diagram, easier to understand. I didn’t use annotations for debugging purposes Whether you use Mockito.mock() or PowerMockito.mock(), it will call the MockitoCore.mock method (note here that the MockitoCore object is a Static property, so there is only one globally). Once inside the MockitoCore#mock method, it does three main things, as shown below.

Mockito Source Code Analysis (1) - Basic Concepts

As a qualified development engineer, writing a good UT (Unit Test) is a necessary skill, and there are many UT tools on the market. I have chosen the most widely used combination of Mockito + Powermock to analyze its source code, and I hope it will bring you a rewarding experience. Please note that this series will not introduce you to basic usage. Before starting the series, let’s first agree on the version.

Agrona - High performance data structure tool library

Agrona is a Java toolkit developed by real-logic that provides a number of high-performance data structures and tooling methods, mainly including. Buffers - Thread safe direct and atomic buffers for working with on and off heap memory with memory ordering semantics. Lists - Array backed lists of int/long primitives to avoid boxing. Maps - Open addressing and linear probing with int/long primitive keys to object reference values. Maps - Open addressing and linear probing with int/long primitive keys to int/long values.

Why MySQL's self-incrementing primary keys are not monotonic or continuous

When we work with relational databases, the primary key is a concept that cannot be avoided. The primary key acts as an identifier for a record, and we can locate a unique record in a table by its identifier. In a relational database, we choose the smallest subset of multiple fields in a record as the unique identifier for that record in the table. Depending on the definition of a primary key in a relational database, we can choose either a single column as the primary key or multiple columns as the primary key, but the primary key must exist and be unique throughout the record.