Talking about operation and maintenance with building blocks, such IT people are very knowledgeable

Introduction: The log service SLS provides data collection, processing, analysis, alarm visualization and delivery functions, supports AIOps, big data analysis, operation services, big data security and other scenarios, and can adapt various operation and maintenance scenarios by building blocks to assist enterprises in IT decisions. Recently, the log service SLS has added two important functions, which help to further improve the digital capabilities of research and development, operation and maintenance and other scenarios.

The assembly of building blocks is a meticulous work. Using different building blocks for combination and transformation, children may use building blocks to build tall buildings, cars and roads, while IT people choose to build small building blocks to explain the observable universe.

The well known log service SLS is not only a "log storage", but also a one-stop intelligent operation and maintenance scheme and an observable platform. It provides data collection, processing, analysis, alarm visualization and delivery functions, supports AIOps, big data analysis, operation services, big data security and other scenarios, and can adapt various operation and maintenance scenarios in a building block way to assist enterprises in IT decision-making.

Recently, the log service SLS has added two important functions, which help to further improve the digital capabilities of research and development, operation and maintenance and other scenarios.

1、 Five levels of log service architecture

According to relevant reports, the application of observability enables enterprises to use their data characteristics to gain competitive advantages. If it can be planned and successfully implemented in the strategy, the observability application will become the most powerful source of data-driven decision-making.

From the perspective of architecture, the log service SLS, as a one-stop cloud native observable analysis platform, can be divided into five levels by function. The bottom layer is data acquisition and pipeline, and the upper layer is a unified observation storage platform; Then, flexible high-performance computing engine and intelligent Ops platform tools; The top layer is a variety of out of the box functions.

1. Data acquisition and pipeline

It is recognized in the industry that Log, Metric and Trace are the three pillars of full observation. By building a unified observation system, the operation and maintenance personnel can understand the system operation status "in advance", quickly locate the fault "in the process", and analyze the root cause "afterwards", so that they can advance and retreat in a timely manner without panic.

SLS, as an intermediate product of Alibaba, Ant and other logs, can serve as a traffic pipeline and provide large-scale, low-cost, real-time platform services for log/metric/trace and other data.

2. Observable storage platform

For different types of data, SLS provides a unified storage capacity. The LogHub supports the connection with various real-time calculations and services. It can not only provide complete progress monitoring, alarm and other functions, but also implement customized consumption according to SDK/API. In addition, SLS has a strong data processing capability, which is widely applicable to data normalization, enrichment, distribution, aggregation, indexing and other scenarios.

In order to reduce the cost of long-term storage for users, SLS provides the cold storage function. The storage cost of cold storage data is very low, and the conversion of cold and hot data will not incur costs.

3. High performance computing engine

SLS has many Internet and enterprise customers. Their business systems will generate a large amount of embedded log data in real time. These data often need to be saved for a long time, and there is a need for offline or real-time computing.

In order to better serve these customers, SLS provides index based high-performance query/analysis capabilities, and 1 billion level data can be returned in seconds. In terms of syntax, SLS is fully compatible with the SQL92 standard, and seamlessly interfaces with mainstream (Flink, Spark, Storm, etc.) stream computing products.

4. Intelligent Ops platform tool

SLS supports complete AIOps capabilities, and can connect all observable data into one platform, and then build a cloud native observable platform by combining high-performance data query engine and association analysis capabilities.

Users can conveniently and efficiently build ITOps, SecOps and FinOps upper applications based on this platform. Combined with intelligent alarm and response center, AI based data analysis and anomaly inspection module, the nerve center of enterprise IT system is completed.

5. Out of the box applications

SLS has the features of full trusteeship, operation and maintenance free, and out of the box use. It provides DSL layout capabilities, built-in 200+functions, built-in operators, and can quickly realize secondary development.

Cloud Lens reuses SLS's storage analysis platform, and can uniformly collect access logs, usage data, monitoring indicators and other observable data across accounts and regions, and then create a unified scenario based insight market to help users achieve the observability of cloud products at a low threshold.

2、 Log service adds query type specifications and Scan scanning mode

The existing Standard Logstore of SLS contains the complete set of SLS functions. Users can query and analyze data with high performance on the Standard Logstore, which can be used to adapt to various business scenarios. However, in fact, some users' business scenarios do not rely on analysis capabilities, but rely more on queries to solve problems.

The philosopher Lao Tzu once said that the main road is simple. Similarly, products also need to make "subtraction" according to circumstances. Therefore, SLS has introduced a new Query Logstore. According to statistics, the traffic cost of the query type index is reduced by about 72% compared with the standard type, and the comprehensive cost is expected to be reduced by 30%. It supports high-performance queries and does not support analysis and statistics. It is suitable for debugging, problem diagnosis, audit traceability and other scenarios that focus on query rather than analysis (or no analysis).

Under the same project, standard and query type Logstores can be maintained at the same time. Users can flexibly combine them according to different business requirements. The SLS Query specification also supports hot and cold storage tiering, which is compatible with the query function and further reduces costs.

In addition to the query specification, SLS also achieved a major breakthrough in the computing engine this year. The index query/analysis mode is a schema on write mode, which pays more attention to stability and efficiency. The corresponding index needs to be established in advance. Fields without an index cannot be used for query analysis. It is applicable to log fields with clear business requirements. However, for the part with relatively vague requirements, users are often caught in the dilemma of whether to focus on building indexes in advance of business requirements or balancing costs.

The schema on read mode introduced by SLS does not require users to build indexes in advance. It can implement lightweight (performance/scale limited) query and analysis functions without pre built indexes, and the corresponding cost will be greatly reduced. In the same log data, through the flexible combination of the two computing modes, SLS has successfully become a "master of water", taking into account both business needs and costs.

Conclusion: Get up early and fly together with the greedy black. The total vacation is to be determined. That is human flesh operation and maintenance. In the era of intelligent operation and maintenance, what the operation and maintenance personnel need is to observe the operation and maintenance of literature and energy, and analyze the retrieval of military energy. With the log service SLS that can flexibly assemble the underlying capabilities, the operation and maintenance personnel can further free themselves from heavy transactions.