Can the Log Be Changed? SLS Logstore Adds Native Support for Log Updates and Deletes

Introduction

We are the Alibaba Cloud SLS team. SLS is Alibaba Cloud’s one-stop log and observability platform, responsible for the ingestion, storage, querying, analysis, and shipping of massive volumes of enterprise log data every day. For the past decade, logs written to LogStore have been immutable. Recently, however, we introduced native update and delete capabilities for existing data. In this article, we would like to share the story behind this decision and the engineering trade-offs we navigated along the way.

I. Foreword

"Can logs be modified?"

Over the past decade, the answer from Simple Log Service (SLS) has been very clear: they cannot be modified, and they should not be modified.

Because the underlying layer of a Logstore is an append-only design: what is written becomes history, which is also the core advantage of a log system in terms of performance, stability, and audit credibility.

Then why do we add the capabilities of modification and deletion to it this time?

To answer this question, we first review the development history of logs over the years, and you can see that the uses of logs have been continuously expanded.

Phase 1: Logs Are for Viewing

The earliest logs are just for humans to view. If a service encounters a fault, you can log on to the machine to run tail -f, run grep for a few keywords, and see which line reports an error. In the generation of centralized log systems, the task of "viewing logs" has been moved to the browser, but the essence remains unchanged: the lifecycle of logs is still very simple, which includes writing, viewing, and then being cleaned up after the logs expire.

This usage mode brings a very clear design premise to the log system: append-only, no modification.

The write path does not require edit locks or version snaps. The storage layout is naturally organized by time. Downstream consumers only need to record the offset to recover the consumption progress. The high throughput, low cost, and stability of the log system are largely built on this set of rules.

Phase 2: Logs Are for Analysis

Later, logs begin to be structured, indexed, and queried and analyzed.

SLS also continuously complements capabilities such as SQL, Structured Process Language (SPL), aggregate functions, machine learning operators, alerting, dashboards, delivery, and consumption at this stage. Logs are no longer just texts that are opened only when you troubleshoot faults, but gradually become data sources in operational systems.

Risk control systems subscribe to behavioral logs for real-time rule matching. Monitoring systems compute P99 and QPS metrics based on gateway logs. Security teams perform association analysis in audit logs. BI teams feed transaction logs to reports after aggregation.

Logs have changed from being "read by humans" to being "read by programs", and continuously enter the links of analysis, alerting, reports, and automated consumption.

However, this stage still doesn't challenge the append-only foundation. The write path remains a simple append operation—what changes is the addition of stronger indexing, computation, and consumption capabilities layered on top. SLS's goal is to make queries faster, analytics more comprehensive, and subscriptions smoother—not to modify data that has already been written.

Phase 3: "Logs" Carry More Business Scenarios

As logs carry more and more business semantics, requirements such as correction, backfill, overwrite, and cleanup, which rarely appeared in log systems in the past, also begin to become common.

Typical scenarios include:

After the metering and billing details are generated by billing cycle, some records need to be corrected at the end of the month because of discounts, refunds, or account adjustments.
For user features or content tags produced by AI pipelines, historical data needs to be recalculated in batches and old results need to be overwritten after the model is upgraded.
After the risk control system publishes a new version of the scoring rule, it is found that the old rule evaluates the threat levels of some users too low, and batch correction is required.
Information such as user feedback, manual annotations, and quality scores in LLM or Agent applications usually arrives with a delay after the request ends, and needs to be backfilled to the original records.
After collection configurations are faulty or test traffic mistakenly enters the production environment, targeted cleanup is required based on conditions.

In the past, when users encountered these requirements, they usually needed to handle them by using soft deletion and rewrites, supplementing variable fields in external KV or databases, or appending new records and then letting downstream components merge them on their own. The solution can work, but the link is longer, the maintenance cost is higher, and the consistency is more difficult to guarantee.

If a Logstore can natively support partial updates and deletes, this type of scenarios can be completed in a closed loop within the Logstore, without the need to detour to external systems.

II. Design Tradeoffs

If you look closely at the preceding scenarios, the data entity still has typical log features: it is continuously generated, has a large write volume, is naturally organized by time, and is usually semi-structured. At the same time, it requires queries, analysis, alerting, and reports, and also needs to be consumed and delivered by downstream components.

It is exactly these features that allow the Logstore to gradually form today's capability combination: elastic capacity, flexible schema, high-throughput append writes, low-cost storage, strong query and analysis, and a complete consumption and deliver ecosystem.

Now there are requirements for partial updates, but the core features remain unchanged: they are still mainly massive appends, supplemented by a small number of corrections.

Therefore, a Logstore natively supports update and delete operations. The core design principle is as follows:

Append writing remains the main path, and modification and deletion are supplementary paths for existing data.

Specifically, this is reflected in the following design decisions.

Modification Capability Needs to Be Explicitly Enabled

Only when you set enableModify=true in the Logstore properties, the modification and deletion capabilities are enabled (they cannot be disabled after they are enabled).

For data that does not need to be modified, such as access logs, audit logs, and trace details, you can continue to use the append-only model.

Raw Data Ingestion Requests Remain Append-Only

Even after enableModify=true is set, new data is still ingested via direct append writes. This means ingestion requests won't deduplicate based on a business primary key or automatically overwrite existing records. To modify or delete ingested data, you must explicitly issue update or delete requests using __rowid__ or specific query conditions.

This design ensures that the normal ingestion path still maintains high throughput, low cost, and stable consumption semantics.

Accordingly, real-time LogHub consumption and delivery jobs still operate on the raw ingestion stream—they have no awareness of any subsequent updates or deletions.

Modifications and Deletions Are Synchronously Visible

To ensure high performance for normal ingestions, indexes are built in near real-time. There may be a very short delay between log ingestion and queryability. Because normal ingestions follow append-only semantics, this does not affect the ingestion order.

The modification and deletion operations added this time take effect synchronously. That is, after the API returns a success, you can see the corresponding effects in subsequent query and analysis.

This is particularly important for continuous partial updates. After the previous update returns a success, the subsequent update can ensure that it continues to take effect based on the results of the previous update. This avoids the semantic ambiguity of whether the subsequent update sees the previous update.

Query Experience Remains Unchanged

After enableModify=true is set, query and analysis are still completed through Search, SQL, and SPL, and the usage remains unchanged.

The modified or deleted data will reflect the corresponding results in subsequent query and analysis. For businesses that use Logstores to retrieve data, analyze data, generate reports, and manage the backend, the original query links do not need to be refactored because of the update and delete capabilities.

III. Capability Forms

After enableModify=true is set, Logstores provide modification and deletion capabilities in two dimensions: operating on a single record at a fixed point based on __rowid__, and performing batch operations on a batch of records based on query conditions.

Operation	Invocation method	Analogy	Scenarios
Modify specified rows	Specify `__rowid__` + modify fields	`UPDATE WHERE __rowid__ = ?`	Single backfill and targeted repair
Modify by query statement	Specify time range + `query` + modify fields	`UPDATE WHERE condition`	Batch correction and field backfill
Delete specified rows	Specify `__rowid__`	`DELETE WHERE __rowid__ = ?`	Single cleanup
Delete by query statement	Specify time range + `query`	`DELETE WHERE condition`	Batch cleanup of dirty data

Rowid: Row-Level Addressing Identity

After enableModify=true is set, each log is assigned a stable internal __rowid__ identity rowid. The user obtains the __rowid__ through Search, SQL, or SPL queries, and then uses it to precisely specify which record to modify or delete.

Note that __rowid__ is an internal identity used for row addressing in Logstores. It is not a business primary key or a primary key in a database. It is only used to locate existing records, and cannot be used for "insert by ID" or "overwrite by ID".

Therefore, operations based on __rowid__ are more suitable for scenarios where you query first and then modify. For example, a batch of records is retrieved from the management backend, the user selects one of them, and then precisely writes it back based on __rowid__.

Operations Based on Query Conditions: Batch Correction Using Business Fields

If you want to operate based on business fields such as request_id, batch_id, order_id, and trace_id, you can use the conditional update or delete method.

The caller only needs to specify the time range and query expression, and the records that hit the range will be updated or deleted at one time. This method does not require querying __rowid__ first. As long as you can clearly describe "which records need to be modified" using a query statement, you can directly initiate the operation.

Note that query-based operations only work if the conditions match data that's already been indexed. Because indexing happens asynchronously after ingestion, you'll usually need to wait until the data is queryable before running updates or deletes—so this approach doesn't work well for "modify right after write" scenarios.

Also, each query-based update or delete can affect up to 10,000 rows at most. We recommend breaking large jobs into smaller batches based on business dimensions—like billing cycle, batch, user, instance, or time window—to keep the scope of each operation manageable.

IV. Typical Scenarios

Overall, business scenarios suitable for using update/delete on a Logstore usually have the following features:

Data has typical log features. Data is continuously generated, large in size, and mainly written by append. It requires retrieval and analysis. At the same time, there are relatively low-frequency requirements for modification, backfill, or cleanup.
There is a natural time recency between modification and writing. For example, user feedback arrives with a delay, models are recomputed by version, and billing cycles are adjusted at the end of the month.
The range of each modification is relatively controllable. The hit range is naturally limited by request_id, batch, or billing cycle.

You can look at several typical scenarios below.

Risk Control and Marketing: Purge of History Labels After Rule Upgrades

Consider a risk control scenario as an example: each transaction written to a Logstore may carry a threat level computed by the model at that time. When the risk control team releases a new version of the scoring rules, they may find that the old rule underestimated the risk for certain transactions. In that case, they need to apply the new rule's logic to correct a batch of historical labels.

For instance, you could update a merchant's risk_level from medium to high for the past 7 days, ensuring that downstream review queues, threat dashboards, and reports immediately reflect the updated standard.

client.update_logs(
    project="risk",
    logstore="transactions",
    from_time=seven_days_ago,
    to_time=now,
    query='merchant_id: "M-2049" and risk_level: "medium"',
    log_item={
        "risk_level": "high",
        "risk_model_version": "v3.2",
        "re_evaluated_at": now,
    },
)

Similar scenarios also include marketing channel attribution adjustments, experiment grouping strategy fixes, report standard field purges, and content tags or user persona recomputations. Their similarity is: the rule is upgraded, and the history results also need to be purged accordingly.

Metering and Billing: Detail Correction After Billing Cycle Settlement

Metering and billing details are continuously generated by billing cycle and written to a Logatore for aggregation and reconciliation. During settlement at the end of the month, some records may need to be corrected because of discount write-offs, refunds, customer negotiations, or manual adjustments.

These types of scenarios are naturally suitable for update/delete: adjustments usually occur after the billing cycle ends, and there is a clear time recency between writing and modification. The correction range can also be delineated by fields such as billing cycle, user identifier (ID), and instance ID, with clear borders.

For example, you can delineate a batch of records by billing_period="2026-05" and instance_id="inst_8821", and only update fields such as amount, adjust_reason, and adjusted_at.

The adjustment result is directly reflected on the original details, and downstream reports, aggregated bills, and audit queries automatically see the corrected data.

LLM/Agent: Field Backfill for Delayed Feedback

A single request of an LLM/Agent application generates a large amount of procedure data. Among them, execution details, such as each step and each tool call, are suitable to remain append-only. However, information that arrives with a delay after the request ends is very suitable to be backfilled to the original request record.

This type of information includes user upvotes and downvotes, operational badcase annotations, offline quality scores, and manual review conclusions.

The invocation method is similar to the previous one. You can locate the original request record by request_id + time range, and backfill the feedback field. In this way, trace details still aggregate execution details by trace_id, and feedback analysis and quality assessment can directly query the fields on the request record.

O&M/Management Interface: Precise Modification of Records of Specified Rows

In some scenarios, modifications are triggered by humans on the interface—customer service changes the order status from "Pending" to "Contacted" in the ticketing system, the data administrator views the collected annotation data and corrects a fault label, or operations add review comments to an LLM/Agent request in the badcase management backend.

The typical flow of this type of operation is "query first, modify later": the interface queries a batch of records through a query, and these records will all carry a unique __rowid__. After the user selects one of them to modify, it is precisely located and written back by __rowid__ when it is saved.

# After the user selects a record to modify, it is precisely written back by __rowid__
client.update_logs(
    project="ops",
    logstore="work-orders",
    rowid="1|1048576|63",   # The __rowid__ retrieved from the query result
    log_item={
        "status": "contacted",
        "handler": "alice",
        "handled_at": now,
    },
)

This method is suitable for scenarios such as single manual correction, review write-back, and backend management operations. Compared with redefining conditions based on business fields, __rowid__ can precisely locate the record that the user sees on the interface.

V. Final Thoughts

Let's return to the original question: can logs be modified?

If we're talking about traditional O&M logs—records of what the system did and when errors occurred—the answer remains no. Immutability is precisely what gives them their value.

However, a Logstore today handles far more than that. Increasingly, business data is generated continuously in log form. This data needs the elasticity, retrieval, analysis, and consumption/delivery capabilities that a Logstore provides—while also giving rise to backfill, correction, and cleanup needs within business workflows.

For this type of data, native update/delete support offers a direct solution: no need to work around it through external systems, and no need to sacrifice the Logstore's inherent strengths.

If your requirements match the features described earlier, you can set enableModify=true for your Logstore to unlock more scenario possibilities.

For specific activation methods, API parameters, restriction checklists, and billing instructions, you can refer to the official document Modify and delete log data.

Community

Can the Log Be Changed? SLS Logstore Adds Native Support for Log Updates and Deletes

Introduction

I. Foreword

Phase 1: Logs Are for Viewing

Phase 2: Logs Are for Analysis

Phase 3: "Logs" Carry More Business Scenarios

II. Design Tradeoffs

Modification Capability Needs to Be Explicitly Enabled

Raw Data Ingestion Requests Remain Append-Only

Modifications and Deletions Are Synchronously Visible

Query Experience Remains Unchanged

III. Capability Forms

Rowid: Row-Level Addressing Identity

Operations Based on Query Conditions: Batch Correction Using Business Fields

IV. Typical Scenarios

Risk Control and Marketing: Purge of History Labels After Rule Upgrades

Metering and Billing: Detail Correction After Billing Cycle Settlement

LLM/Agent: Field Backfill for Delayed Feedback

O&M/Management Interface: Precise Modification of Records of Specified Rows

V. Final Thoughts

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

Simple Log Service

CloudMonitor

Application Real-Time Monitoring Service

Data Lake Storage Solution