Elastic APM is a tool for application performance monitoring on Elastic Stack that allows us to monitor application performance in real time by collecting incoming requests, database queries, cache calls, and more. This makes it easier for us to quickly pinpoint performance issues. Elastic APM is OpenTracing compatible, so we can use a large number of existing libraries to track application performance. For example, we can trace a request in a distributed environment (microservice architecture) and easily find possible potential performance bottlenecks.

In this section we will install and configure Filebeat to collect log data from a Kubernetes cluster and send it to ElasticSearch, Filebeat is a lightweight log collection agent that can also be configured with specific modules to parse and visualize the log format of applications (e.g., databases, Nginx, etc.). Similar to Metricbeat, Filebeat requires a configuration file to set the link information to ElasticSearch, the connection to Kibana, and the way logs are collected and parsed.

In this article, we will use Metricbeat to monitor Kubernetes clusters, as we have already installed and configured the ElasticSearch cluster in the previous article. Metricbeat is a lightweight collector on the server that is used to collect monitoring metrics for hosts and services on a regular basis. This is the first part of our build of Kubernetes full-stack monitoring. Metribeat collects system metrics by default, but also includes a

In this series of articles, we will learn how to use the Elastic technology stack to build a monitoring environment for Kubernetes. The goal of observability is to provide an operations tool for production environments to detect when a service is unavailable (e.g., if the service is down, has errors, or is slow to respond) and to keep some troubleshooting information to help us pinpoint the problem. In summary, there

Some time ago, a project was about to go live, and we needed to pressure test the core interface; since our interface is gRPC protocol, we found that there are not as many pressure testing tools as HTTP. Finally I found the tool ghz, which is also very full-featured. Afterwards I wondered why there are so few tools for gRPC piezos, what are the difficulties? In order to verify this problem, I am going to write a tool myself.

When data crawling, we often encounter program exceptions due to network problems, and the initial approach is just to record the error content and post-process the error content again. Some better exception retry methods or mechanisms are compiled here. Initial version : 1 2 3 4 5 6 7 8 9 10 11 def crawl_page(url): pass def log_error(url): pass url = "" try: crawl_page(url) except: log_error(url) Improved version (increased number of retries):

The Requests library is used to make standard HTTP requests in Python. It abstracts the complexity behind the request into a nice, simple API so you can focus on interacting with the service and using the data in your application. Requests POST/GET parameters Commonly used parameters are listed in the following table. Requests Return object Response common methods Common properties and methods of Response class. 1 2 3 4 5

Beautiful Soup is a Python library that can extract data from HTML or XML files. Simply put, it can parse HTML tag files into a tree structure and then easily get the corresponding attributes of the specified tags. This feature is similar to lxml. Beautiful Soup installation Beautiful Soup 3 is currently out of development and it is recommended to use Beautiful Soup 4 in your current projects, installed by

In the data crawl will often use regular expressions, if not familiar with Python’s re module, it is easy to be confused by the various methods inside, today we will review the Python re module. Before learning the Python module, let’s see what the official description documentation says Implementation. 1 2 import re help(re) Helpful information. 1 2 3 4 5 6 7 8 9 10 11 12 13 14

GeoPandas Introduction GeoPandas is an open source project that aims to make it easier to work with geospatial data under Python. GeoPandas extends the pandas data type to allow spatial operations on geometric types. Geometric operations are performed by shapely. GeoPandas further relies on fiona for file access and descartes, matplotlib for plotting. GeoPandas follows the data types of pandas, so there are also two data types in GeoPandas. GeoSeries

Introduction to Mapbox Founded in 2010 by Eric Gunderson, Mapbox has grown rapidly and has become a leader in the mapping renaissance. focused on providing custom base map blocks for map and application developers, Mapbox has positioned itself as the leading software company for web maps and mobile applications. In addition to the base map style toolset already widely used by program developers and cartographers, they also offer mapping tools written in Python and JavaScript.

Basemap Introduction Basemap is a toolkit under the Python visualization library Matplotlib. its main function is to draw 2D maps, which are important for visualizing spatial data. basemap itself does not do any plotting, but provides the ability to transform coordinates into one of 25 different map projections. Matplotlib can also be used to plot contours, images, vectors, lines, or points in transformed coordinates. basemap includes the GSSH coastline dataset,

Yu-Chin Juan, the author of FFM, has open-sourced the C++ version of the code libffm on GitHub. Since the daily data processing is in a Python environment, expect to find a Python version of FFM. Related projects on Github There are many on Github, such as this one: A Python wrapper for LibFFM. Installation of libffm in Windows+Anaconda environment Installation of libffm-python package The project is installed on Windows as

Introduction to systemd-analyze systemd-analyze is a tool that comes with Linux to analyze system boot performance. Commands available for systemd-analyze. systemd-analyze [OPTIONS…] [time] systemd-analyze [OPTIONS…] blame systemd-analyze [OPTIONS…] critical-chain [UNIT…] * systemd-analyze [OPTIONS…] critical-chain [UNIT… systemd-analyze [OPTIONS…] plot [> file.svg] systemd-analyze [OPTIONS…] dot [PATTERN…] [> file.dot] systemd-analyze [OPTIONS…] dump systemd-analyze [OPTIONS…] set-log-level LEVEL systemd-analyze [OPTIONS…] set-log-target TARGET systemd-analyze [OPTIONS…] get-log-level systemd-analyze [OPTIONS…] get-log-target systemd-analyze [OPTIONS…] syscall-filter [SET…] systemd-analyze [OPTIONS…] verify [FILES…] The systemd-analyze command means.

crond is a daemon process used under linux to periodically perform certain tasks or wait for certain events to be processed, similar to the scheduled tasks under windows, when the installation of the operating system is complete, this service tool will be installed by default and will automatically start the crond process, the crond process will periodically check every minute if there is a task to be performed, and if there is a task to be performed, the task will be automatically executed.

CentOS 7.0 has replaced the original firewall iptables with FirewallD, which supports IPv4, IPv6 firewall settings and Ethernet bridging, and has both runtime and permanent configuration options, and is called a dynamically managed firewall, meaning that changes can be applied without restarting the entire firewall. firewalld is installed by default, if not, execute yum install firewalld firewalld-config to install it, where firewalld-config is a GUI tool. netfilter iptables, firewalld, and

In the previous article on how to use GitLab CI for CI/CD in a Kubernetes cluster, we basically used the Docker On Docker model for building images, because the Kubernetes cluster uses a container runtime like Docker, so we could mount the host docker.sock file from the host to the container to build the image, but recently we changed the container runtime to Containerd after using Kubernetes version 1.22.X, so

As Kubernetes releases iterate, many Helm Chart packages simply cannot keep up with the updates, resulting in many Helm Chart packages being incompatible when using newer versions of Kubernetes, so it is necessary to consider compatibility with different versions of Kubernetes when developing Helm Chart packages. The core of achieving compatibility with different versions is to make use of the built-in object Capabilities provided by the Helm Chart template, which provides information about Kubernetes cluster support features, including the following features.

Because Java code is very easy to decompile, Java code obfuscation is needed to protect Java source code well. This article describes how to use ProGuard’s plugin proguard-maven-plugin to obfuscate Java programs. How ProGuard works ProGuard can rename classes, fields and methods by removing useless code and using short, meaningless names. This allows you to compress, optimize and obfuscate your code. The result is a smaller compiled file that is

To date, 180 billion Arm-based processors have been included in cell phones, IoT sensors and other devices. Today, this technology has evolved to support hyperscale data centers and cloud computing. Arm-based processors are attractive to cloud customers because of their linear scalability, predictable performance, and ultra-high core density and affordability. With Arm-based cloud computing, customers can run existing loads more cost-effectively and build new applications with superior economics and performance.