On June 26, OpenAI previewed GPT-5.6 Sol. According to the official introduction, the model's capabilities continue to advance towards longer-term coding, tool coordination, and safety tasks; at the same time, OpenAI also placed stronger protection, monitoring, and staged releases in the same announcement. This juxtaposition is very telling: the stronger the Coding Agent's capabilities, the workspace it accesses is no longer just an editor, but an execution path composed of code repositories, model context, tool invocations, external authorization, and internal enterprise data.
Recent discussions surrounding OpenAI Codex have pushed this boundary issue to the forefront. The discussion on GitHub at openai/codex#2847 is about how sensitive paths like .env, .pem, .aws/, and .ssh/ should be excluded more deterministically. BeyondTrust disclosed command injection issues related to Codex branch names, with risks including GitHub User Access Token exposure. DARKNAVY / Security Inner Reference has also issued warnings regarding unauthorized execution, malicious repositories, and local development environment exposure. These materials do not point to the same vulnerability, nor should they be simply packaged as "Agents are dangerous." Specific vulnerabilities will be patched, single-point mechanisms will be strengthened, and the more difficult problem for enterprises to handle in daily operations lies at another layer: when model input, model reply, tool invocation, tool results, and application context appear simultaneously in an Agent's task, how does the security system determine which risk should truly enter the high-risk queue.
Agent auditing is not about alerting at every stir of the wind, but about making the risks that actually enter the high-risk queue more accurate. Accuracy does not mean reporting less, nor does it mean letting uncertain risks slide; accuracy means that a single-point hit can be traced back to the context to be explained, and the risk event can show which boundary it crossed, which objects it affected, where the evidence is, and what the next step in handling should be.
The prerequisite for accurate judgment is not to write more aggressive rules first, but to have sufficiently complete behavioral facts first. LoongSuite Pilot first solves the "fact foundation" problem: uniformly collecting sessions, rounds, model requests, model replies, tool invocations, and tool results across different Coding Agents and Agent applications. Without a stable foundation of facts, no matter how sensitive the detection rules are, they can only guess based on local fragments. This is also where Agent auditing differs from ordinary log alerts. A credential leak does not necessarily occur in only a single line of text. A credential may first appear in a tool result and then be pieced into the next round of model input; it may also be generated by a model reply and subsequently enter task results, ticket comments, or downstream logs. Looking only at individual fields, it is easy to mistake local suspicion for high risk; looking only at the final result, you may miss how it came to be.
Therefore, what AgentLoop Audit processes is not a set of isolated logs, but a behavioral chain that can be replayed. Sessions, model inputs/outputs, tool invocations, and tool results are first collected as facts; subsequent risk judgments then place these facts back into the same task, the same application, and the same risk object to be examined. See it all first, then judge accurately.
The core methodology of AgentLoop Audit is such a pipeline: behavioral facts are first uniformly collected; rules hit on local facts, forming low-fidelity signals; the system then contacts the context to make semantic determinations, elevating parts with more sufficient evidence and clearer boundaries to high-fidelity risk events; finally, the events are mapped to objects that can be located and governed.
Low-fidelity signals are not useless. The appearance of a suspected AK in model input, a suspected AK in model reply, or suspected sensitive fragments in tool parameters are all worth recording. The issue is that they are still just hits on local facts. If every hit directly becomes high risk, security colleagues will see a pile of red dots that "might have issues," rather than truly actionable risks.
High-fidelity events need to go a step further. Take Alibaba Cloud credentials as an example, seeing a single LTA... fragment only shows that a suspected AK leak rule was hit; if the same credential appears in the model output and external upload targets, the nature of this risk changes. It is no longer just a "suspected AK leak," but a specific credential "confirmed exposed" in the Agent's behavioral chain.
Contextual semantic determination is not about reducing alerts, but about making the high-risk queue more credible. Low-fidelity signals can be abundant, and facts can be as complete as possible, but events that are truly pushed to the high-risk position must be able to explain why they deserve priority treatment.
Security operations do not lack lists. What is truly lacking is prioritization. If all risks are sorted backward by occurrence time, a repeatedly leaked AK, a credential that has just spread to multiple applications, and an isolated low-confidence hit will be mixed in the same table. Users see "many risks," but it is difficult to determine which one to handle first today.
The AgentLoop Audit overview page answers this operational question first. In the current screenshot, "Risk with Largest Impact" points to "Data Leak: Secret / Alibaba Cloud Credentials" and shows that it has already affected 6 applications; "Fastest Worsening Risk" points to the growth of "Secret / Authorization Header". Such a first screen is not to create a sense of tension, but to elevate risks from a chronological list to a work queue: which type of risk has the largest impact, which type of risk is getting worse, and which type of issue is worth entering investigation first.
Similarly, when an Alibaba Cloud credential appears, different locations mean different risk boundaries, and different handling actions are required. The data leak page breaks down "Secret / Alibaba Cloud Credentials" by leak method. In the screenshot, there are 38 sensitive data submissions to the model, and 43 model replies leaking sensitive data. This distribution is crucial: submission to the model shows that the credential entered the model input context; model reply leak shows that the credential has reached the output side, potentially continuing into chat windows, task results, or downstream logs.
This is part of the contextual semantic determination. Looking at "AK hit" in isolation only tells you that there is a sensitive fragment; placing it within boundaries like "model input" and "model reply" allows security colleagues to know whether to check context splicing, output filtering, log persistence, or to first rotate credentials that have already been exposed to the output side. Risk types not only make alert metrics clearer but also allow handling actions to quickly find the right direction.
Finding "Alibaba Cloud credential leak" is not enough. One AK leaking multiple times is not the same as multiple AKs leaking in a dispersed manner. The risk details in the screenshot are first clustered by "Risk Type + Application". After filtering "Model Reply Leak," you can see the impact of similar risks across different applications: for example, under loongsuite-pilot-qoder there are 3 specific risks, 19 risk events, and 4 sessions; other applications also have their respective risk numbers and session numbers. This layer first answers "which type of risk is more concentrated in which application."
Unfolding another layer, the system aggregates the specific leaked credential values. The screenshot shows multiple masked LTA... credentials, each followed by its corresponding risk count, session count, and latest occurrence time. This layer answers another question: is this the same AK repeatedly exposed, or multiple AKs individually exposed.
The former often points to a specific credential that needs to be rotated and its source traced; the latter is more likely to indicate systematic issues with context splicing, output filtering, or application usage. Giving only individual events forces users to judge among duplicates themselves; by first aggregating to risk types, applications, and specific credentials, the system organizes the investigation entry points in advance.
The risk details page solves the question of "where is the evidence." In the screenshot, after a user opens a certain Alibaba Cloud credential leak, they can see the application, severity, discovery ID, evidence summary, hit fields, and associated events. The session view on the right jumps directly to the risk event and highlights the AK in the model output. That is, the system doesn't just tell you "the model reply leaked a secret," but presents the hit field, the hit value, the occurring session, and the specific text location all together.
This is very important for high-fidelity judgment. Security colleagues do not need to manually search again through an entire segment of prompt, response, or tool result, nor do they need to guess which Agent interaction this risk is related to. The details page pulls the raw evidence behind low-fidelity hits back into context, turning "hits" into reviewable events.
Many times, security investigations do not start from sessions, but from objects.
When an AK has been confirmed leaked, the next step is naturally to ask: where else does it appear, which application is it associated with, which user did it come from, which hosts or containers does it affect, and is it continuously related to a certain tool invocation. The entity investigation page turns objects like Secret, PII, Application, Host/Container, User, Source IP, and Tool into investigation entry points, allowing users to reverse search the impact scope from a risk object.
The Secret entity list in the screenshot shows that a certain LTA... credential is associated with model input exposure and model output exposure, with corresponding high-risk event counts and session counts. After entering the relationship graph of this AK, you can see that it is associated with the loongsuite-pilot-qoder application, qoder:exec tool, a certain host/container, user, and source IP.
This step advances risks from "a certain alert" to "governable objects." If only one session and one application are affected, handling can be more focused; if the same AK is associated with multiple applications, hosts, users, or tools, the scope of investigation must be expanded to check credential sources, usage methods, and context handling strategies. The entity perspective provides not more charts, but entry points closer to governance actions.
Reducing false positives does not mean there are no false negatives, and rules cannot be written all at once. What AgentLoop Audit currently needs to establish firmly is first running through this entire pipeline of fact collection, rule hits, contextual semantic determination, and evidence localization. In this pipeline, Pilot is responsible for keeping facts like sessions, model inputs/outputs, tool invocations, and tool results from being scattered everywhere; risk auditing is responsible for putting local hits back into context to judge whether they have crossed input, output, or other key boundaries; entity investigation is responsible for mapping confirmed risk objects into impact scopes, so that governance actions do not just stop at "having looked at a detail page."
Agent auditing should not mark all suspicious points as high risk at every stir of the wind, nor should it hide uncertain risks in order to report less. A truly useful system must allow the underlying layers to retain enough low-fidelity signals while making events in the high-risk queue more believable, reviewable, and actionable. Fishing out real risks from massive "noise" relies not on turning down the volume, but on ensuring every high-risk alert has context, evidence, and an impact scope.
STAROps RUM Inspection in Practice: Spotting Experience Degradation Early
760 posts | 60 followers
FollowAlibaba Cloud Native Community - May 25, 2026
Alibaba Cloud Native Community - June 25, 2026
Alibaba Cloud Native Community - May 26, 2026
Alibaba Cloud Native Community - May 7, 2026
Alibaba Cloud Native Community - April 20, 2026
Alibaba Cloud Native Community - June 10, 2026
760 posts | 60 followers
Follow
Token Plan
Build more, spend less. One plan, every modality.
Learn More
Alibaba Cloud Model Studio
A one-stop generative AI platform to build intelligent applications that understand your business, based on Qwen model series such as Qwen-Max and other popular models
Learn More
CloudMonitor
Automate performance monitoring of all your web resources and applications in real-time
Learn More
AgentLoop
An Agent self-evolution platform designed to make your Agents progressively smarter and more attuned to users.
Learn MoreMore Posts by Alibaba Cloud Native Community
