Change
ce78629
ce786296dbda395512e93f5e9eaa5a27561e9bdf · commit on GitHub
aws-blog-feed: changed (580345 bytes, HTTP 200)
raw/aws-blog-feed/response.xml modified
- Source
- aws-blog-feed
- Lines added
- +3,704
- Lines removed
- -3,854
- Stored bytes at this commit
- 580,345
- Timestamp
- observed
- Raw artifact at this commit
- raw/aws-blog-feed/response.xml
Recorded headers
| observed_at | 2026-09-15T04:55:20.130Z |
|---|---|
| origin_date | null |
| status | 200 |
| final URL | https://aws.amazon.com/blogs/machine-learning/feed/ |
| etag | null |
| last-modified | Mon, 14 Sep 2026 21:22:50 GMT |
| date | Tue, 15 Sep 2026 04:55:20 GMT |
| age | null |
| cache-control | null |
| cf-cache-status | null |
| content-encoding | null |
| content-length | null |
@
@@ -5,7 +5,7 @@ <atom:link href="https://aws.amazon.com/blogs/machine-learning/feed/" rel="self" type="application/rss+xml"/> <link>https://aws.amazon.com/blogs/machine-learning/</link> <description>Official Machine Learning Blog of Amazon Web Services</description>-
<lastBuildDate>Fri, 11 Sep 2026 22:28:31 +0000</lastBuildDate>+
<lastBuildDate>Mon, 14 Sep 2026 21:22:45 +0000</lastBuildDate> <language>en-US</language> <sy:updatePeriod> hourly </sy:updatePeriod>@
@@ -13,501 +13,791 @@ 1 </sy:updateFrequency> <item>-
<title>Monitoring production agent lifecycle with AWS DevOps Agent and AgentCore Evaluations</title>-
<link>https://aws.amazon.com/blogs/machine-learning/monitoring-production-agent-lifecycle-with-aws-devops-agent-and-agentcore-evaluations/</link>+
<title>Abnormal AI: Amazon Bedrock AgentCore for agentic email security at scale</title>+
<link>https://aws.amazon.com/blogs/machine-learning/abnormal-ai-amazon-bedrock-agentcore-for-agentic-email-security-at-scale/</link> -
<dc:creator><![CDATA[Meghana Ashok]]></dc:creator>-
<pubDate>Fri, 11 Sep 2026 18:26:38 +0000</pubDate>+
<dc:creator><![CDATA[Aswin Vasudevan]]></dc:creator>+
<pubDate>Mon, 14 Sep 2026 21:22:45 +0000</pubDate> <category><![CDATA[Advanced (300)]]></category> <category><![CDATA[Amazon Bedrock AgentCore]]></category>-
<category><![CDATA[Best Practices]]></category>-
<category><![CDATA[DevOps]]></category>-
<guid isPermaLink="false">9da22a53a45cf214e154c67bafd0a4bd7eefa26b</guid>+
<category><![CDATA[Customer Solutions]]></category>+
<guid isPermaLink="false">5765f76e7a987692146e6fe2b1eb9962dcd91bac</guid>-
<description>Multi-agent systems fail in ways traditional monitoring misses. This post presents a dual-layer approach to monitoring production agents: Amazon Bedrock AgentCore Evaluations for continuous quality scoring and AWS DevOps Agent for autonomous infrastructure investigation, shown on a…-
<content:encoded><p>Multi-agent systems in production experience issues in ways that traditional monitoring misses. For example, the agent can’t invoke its foundation model (FM) and returns an empty response. This could be because of a missing AWS Identity and Access Management (IAM)…-
<p>Infrastructure monitoring and agent effectiveness monitoring require different approaches. <a href="https://aws.amazon.com/cloudwatch/" target="_blank" rel="noopener">Amazon CloudWatch</a> metrics show whether systems executed correctly, but not whether agents helped users accom…-
<p>These problems compound in multi-agent systems where a single user request triggers a supervisor agent that routes work to multiple specialists, each with their own tools and model invocations. There’s typically no fixed execution graph to instrument, failures can occur at multiple handoff …-
<p>We built a production airline reservation system with four specialized agents that combine <a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/evaluations.html" target="_blank" rel="noopener">Amazon Bedrock AgentCore Evaluations</a> for continuous agent qualit…-
<h2 id="key-technologies">Key technologies</h2> -
<p>The system uses several AWS services:</p> +
<description>Learn how Abnormal AI deployed Amazon Bedrock AgentCore Code Interpreter as an ephemeral compute scratch pad for the agents behind its real-time email threat detection at billion-message scale, plus the sandbox design decisions and practical lessons for builders deploying Code Inte…+
<content:encoded><p>AI agents now run in production at a scale of billions of operations a day, and a recurring architectural pattern has surfaced: agents need a compute scratch pad. Not only for coding tasks, but for data aggregation, analysis, verification, and any workflow where s…+
<p>Abnormal AI, a behavioral security service that protects more than 25 percent of the Fortune 500, has deployed Amazon Bedrock AgentCore Code Interpreter, a capability of Amazon Bedrock AgentCore. Abnormal AI uses it for the agents that support its real-time inline email threat detection. Th…+
<p>This work is part of how Abnormal AI builds software. Today, 80 percent of their code changes are built using an agent in some way, and 40 percent are built end-to-end by a background agent (fully AI built, not AI assisted). Their use of AgentCore Code Interpreter for threat detection refle…+
<p>In this post, we share how Abnormal AI architected these systems, the design decisions behind their sandbox approach, and practical lessons for builders deploying Code Interpreter at scale.</p> +
<h2 id="what-is-amazon-bedrock-agentcore-code-interpreter">What is Amazon Bedrock AgentCore Code Interpreter?</h2> +
<p><a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/code-interpreter-tool.html" target="_blank" rel="noopener">Amazon Bedrock AgentCore Code Interpreter</a> provides a fully managed, serverless runtime for agents to execute code dynamically. Key characteristic…<ul> -
<li><a href="https://docs.aws.amazon.com/bedrock/" target="_blank" rel="noopener">Amazon Bedrock</a> provides API access to foundation models from leading AI companies including Anthropic, Meta, Mistral, and Amazon. In our airline reservation system built on AgentCore runtime, Ama…-
<li><a href="https://aws.amazon.com/bedrock/agentcore/" target="_blank" rel="noopener">AgentCore runtime</a> handles agent orchestration and manages interaction lifecycles, with built-in observability through OpenTelemetry instrumentation.</li> -
<li>With the <a href="https://github.com/awslabs/fullstack-solution-template-for-agentcore" target="_blank" rel="noopener">Fullstack AgentCore Solution Template (FAST)</a>, teams can quickly deploy a secured React frontend connected to an AgentCore backend.</li> -
<li><a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/evaluations.html" target="_blank" rel="noopener">AgentCore Evaluations</a> is a quality assessment framework integrated into AgentCore runtime. It continuously scores agent interactions using <a href="ht…-
<li><a href="https://aws.amazon.com/devops-guru/" target="_blank" rel="noopener">AWS DevOps Agent</a> is an autonomous investigation tool that acts as an on-call engineer for your infrastructure. When incidents occur, it automatically analyzes Amazon CloudWatch logs, traces failur…-
<li><a href="https://strandsagents.com/" target="_blank" rel="noopener">Strands Agents</a> is an open source SDK for building AI agents with a model-driven approach, supporting multi-agent collaboration patterns including Swarm, Graph, and Agents-as-Tools.</li> -
<li><a href="https://opentelemetry.io/" target="_blank" rel="noopener">OpenTelemetry</a> is an open source observability framework that provides standardized instrumentation for collecting traces, metrics, and logs. AgentCore runtime uses OpenTelemetry to emit monitoring data to A…-
<li><a href="https://strandsagents.com/docs/user-guide/concepts/multi-agent/swarm/" target="_blank" rel="noopener">The Swarm Pattern</a> is a multi-agent orchestration approach where a supervisor agent dynamically routes work to specialized agents based on the task at hand. Unlike…+
<li>Ephemeral MicroVM sessions: configurable time-to-live from 15 minutes (default) up to 8 hours for long-running tasks.</li> +
<li>Security: sessions run in secure sandboxes with full separation at the host operating system level, designed to help prevent inadvertent disclosure between sessions.</li> +
<li>Flexible networking: configurable for sandbox virtual private cloud (VPC) mode, or public internet access.</li> +
<li>File handling: up to 100 MB through the API directly, or connect to Amazon Simple Storage Service (Amazon S3) for larger datasets.</li> +
<li>Preloaded runtimes: Python and <code>Node.js</code> environments with common visualization, statistics, and data processing libraries.</li> +
<li>Built-in observability: logs sent to Amazon CloudWatch and AWS CloudTrail for monitoring.</li> </ul> -
<h2 id="dual-layer-monitoring-quality-and-infrastructure">Dual-layer monitoring: quality and infrastructure</h2> -
<p>The monitoring architecture answers two questions: Is the agent working well? Is the infrastructure healthy?</p> -
<p><strong>Amazon Bedrock AgentCore Evaluations: Continuous quality monitoring</strong></p> -
<p>AgentCore Evaluations scores live agent interactions on helpfulness, correctness, and goal completion. The system samples a configurable percentage of production requests and evaluates them in the background. Every score comes with reasoning that explains why that score was assigned based o…-
<p>When quality metrics drop, it runs pattern analysis across recent low-scoring sessions to identify common failure modes. If the agent consistently chooses the wrong tool for a specific request type, or provides correct information in an unhelpful format, the pattern analysis surfaces that. …-
<p><strong>AWS DevOps Agent: Autonomous infrastructure investigation</strong></p> -
<p>AWS DevOps Agent monitors system health across metrics, logs, and error patterns. When something breaks, the agent investigates on its own. It pulls relevant Amazon CloudWatch logs, builds a topology graph of affected resources, correlates errors across services (IAM, Amazon Bedrock, your a…-
<p><strong>How the layers work together</strong></p> -
<p>AgentCore Evaluations replaces intuition with quantitative quality metrics, so teams can measure the actual impact of changes. AWS DevOps Agent automates much of the investigation that previously required manual war rooms by autonomously investigating infrastructure incidents the moment the…-
<h2 id="the-airline-reservation-system-why-its-hard-to-monitor">The airline reservation system: why it’s hard to monitor</h2> -
<p>To demonstrate this architecture end to end, we built an airline reservation system that handles complex, dynamic queries: multi-city bookings, loyalty program benefit application, and company travel policy compliance, all within a single conversational turn. This use case demands exactly t…-
<p>Consider a request like this one: <em>“Book me from Seattle to Boston on March 15th, then Boston to Miami on March 18th. Use my companion certificate for the second leg and make sure both flights comply with my company’s travel policy. I’m Gold status so apply any eligible upgrades.”&…-
<p>Handling this in a single conversational turn requires searching two separate routes and retrieving loyalty status and certificates from a different data source while the flight search runs. The bookings must then be sequenced in the right order, because the companion certificate can’t be a…-
<p>We built this with four specialized agents using the <strong>Swarm pattern</strong> (Figure 1). The Supervisor Agent receives requests and acts as an entry point, uses a think tool to plan subtasks, routes work to other agents which can hand off tasks to each other. The Flight A…-
<p>In a Swarm, agents share working memory and hand off work to one another dynamically. Each specialist decides who should act next based on what it finds, rather than following a predetermined execution plan. The supervisor is only the entry point. After a request is in flight, control passe…-
<p>Failures can occur at any handoff point, and because execution paths change based on runtime decisions, the failure path changes each time too. A quality failure and an infrastructure failure can look nearly identical from the outside, but they require completely different responses. From t…-
<div style="width: 516px" class="wp-caption alignnone">-
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/08/31/ML-20434-1-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/08/31/ML-20434-1-1.png" alt="Diagram of the Swar…-
<p class="wp-caption-text">Figure 1: Swarm multi-agent pattern</p>+
<p>Critically, Code Interpreter is exposed as an API. This means it doesn’t dictate the agent’s workflow. Instead, it provides a box where the agent can run commands, upload files, and retrieve results. For teams with existing agent infrastructure, this plug-and-play design makes the integrati…+
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/11/ML-21301-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/11/ML-21301-1.png" alt="Agent invoking the Amaz…+
<p class="wp-caption-text">Figure 1: Amazon Bedrock AgentCore Code Interpreter architecture, where the agent invokes the Code Interpreter API to provision an ephemeral MicroVM sandbox session for code execution, file input and output, and result retrieval</p></div> -
<h2 id="dual-monitoring-architecture">Dual monitoring architecture</h2> -
<p>The following diagram shows how these components connect.</p> -
<div style="width: 696px" class="wp-caption alignnone">-
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/08/31/ML-20434-2-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/08/31/ML-20434-2-1.jpg" alt="Full system archite…-
<p class="wp-caption-text">Figure 2: Full system architecture showing the React frontend, AgentCore runtime, Amazon CloudWatch, and AWS DevOps Agent</p>+
<h2 id="why-agents-need-a-compute-scratch-pad">Why agents need a compute scratch pad</h2> +
<p>Large language models (LLMs) excel at reasoning and semantic coherence, but many real-world operations don’t map to semantic reasoning:</p> +
<ul> +
<li>Basic math and counting: “How many phishing email did we detect in the past hour?” requires computation, not language generation.</li> +
<li>Data processing and visualization: transforming raw data into charts, PDFs, or structured reports.</li> +
<li>Code verification: running unit tests, linting, and integration tests to validate agent-generated outputs.</li> +
</ul> +
<p>By pairing a large language model with Code Interpreter, you enhance the agent’s capabilities beyond what reasoning alone can achieve.</p> +
<blockquote>+
<p><em>“Pretty much any agent, whether it’s writing code or not, needs a code interpreter sandbox that allows it to actually crunch data and come to answers.”</em></p>+
</blockquote> +
<p>— Shrivu Shankar, VP of AI Strategy, Abnormal AI</p> +
<h2 id="abnormal-ais-architecture-three-tiered-detection-at-billion-message-scale">Abnormal AI’s architecture: Three-tiered detection at billion-message scale</h2> +
<p>Abnormal AI processes billions of email messages through a three-tiered detection architecture, as shown in Figure 2.</p> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/11/ML-21301-2.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/11/ML-21301-2.png" alt="Three-tiered email dete…+
<p class="wp-caption-text">Figure 2: Abnormal AI’s three-tiered email detection pipeline, with Tier 1 (heuristics, billions/day), Tier 2 (machine learning models, millions/day), and Tier 3 (inline agents with Code Interpreter, tens of thousands/day), where each tier handles progressively hard…</div> -
<p>We built a React frontend hosted on <a href="https://aws.amazon.com/amplify/" target="_blank" rel="noopener">AWS Amplify</a> that connects through Amazon Bedrock AgentCore Identity, a capability of Amazon Bedrock AgentCore, to Amazon Bedrock AgentCore runtime, where the four-age…-
<p>Monitoring data originates from a single source: the Amazon Bedrock AgentCore runtime hosting the four-agent swarm. Amazon Bedrock AgentCore Observability, a capability of Amazon Bedrock AgentCore, instruments the runtime directly, capturing traces and metrics in OpenTelemetry format and fo…-
<p>The second monitoring layer connects to this same backend. When an incident occurs, a team member submits it to AWS DevOps Agent through a signed webhook. The agent pulls Amazon CloudWatch logs and metrics, investigates autonomously, and returns findings and remediation steps without requir…-
<h3 id="getting-started">Getting started</h3> -
<p><strong>Open source</strong>: We built this system on FAST and the complete source code for this system including CDK infrastructure, evaluation dashboard, and AWS DevOps Agent integration is available in the <a href="https://github.com/aws-samples/sample-FAST-applications/tr…-
<p>To use Amazon Bedrock AgentCore Evaluations, you need:</p> +
<h3 id="tier-1-high-volume-lightweight-classification-billionsday">Tier 1 — high-volume lightweight classification (billions/day)</h3> +
<p>Small models, heuristic rules, and lightweight classifiers (logistic regressions) handle the largest volume of traffic. At this scale, it’s both cost-prohibitive to run larger models and unnecessary. Most messages can be classified without deep analysis.</p> +
<h3 id="tier-2-medium-models-for-uncertain-cases-millionsday">Tier 2 — medium models for uncertain cases (millions/day)</h3> +
<p>Messages that Tier 1 is unconfident about flow into deep learning and machine learning (ML) models that perform more behavioral signal analysis.</p> +
<h3 id="tier-3-inline-agents-with-code-interpreter-tens-of-thousandsday">Tier 3 — inline agents with Code Interpreter (tens of thousands/day)</h3> +
<p>The hardest cases, which would typically require a human analyst to evaluate, are processed by inline agents. These agents receive the threat intelligence data and use a sandbox to analyze it, writing scripts dynamically. They then evaluate how it fits into the overall behavioral model and …+
<h2 id="the-analyst-agent-batch-intelligence">The analyst agent — batch intelligence</h2> +
<p>Beyond the real-time classification pipeline shown in Figure 2, Abnormal deploys an <em>analyst agent</em> that operates in batch mode (Figure 3):</p> <ul> -
<li>AgentCore CLI (<a class="uri" href="https://github.com/aws/agentcore-cli" target="_blank" rel="noopener">https://github.com/aws/agentcore-cli</a>)</li> -
<li>AWS credentials with bedrock-agentcore and Amazon CloudWatch permissions.</li> -
<li>The bedrock-agentcore Python SDK (Boto3 client)</li> +
<li>Ingests misclassifications and tuning signals from its detection pipeline.</li> +
<li>Identifies patterns and trends across large message sets.</li> +
<li>Autonomously writes draft candidate heuristics for Tier 1, operating on Abnormal AI’s own detection-pipeline features and signals.</li> +
<li>Improved models for Tier 2.</li> +
<li>Runs on the scale of approximately 100 batch jobs per week.</li> </ul> -
<h3 id="layer-1-amazon-bedrock-agentcore-evaluations-is-the-agent-working-well">Layer 1: Amazon Bedrock AgentCore Evaluations, is the agent working well?</h3> -
<p>Silent quality issues in production agents impact team efficiency, user trust, and business outcomes. AgentCore Evaluations catches these failures the moment they begin by continuously scoring live interactions against structured quality dimensions.</p> -
<div style="width: 640px;" class="wp-video">-
<video class="wp-video-shortcode" id="video-138259-1" width="640" height="360" preload="metadata" controls="controls">-
<source type="video/mp4" src="https://d2908q01vomqb2.cloudfront.net/artifacts/DBSBlogs/ML-20434/blogpost_eval_video.mp4?_=1">-
</video>+
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/11/ML-21301-3.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/11/ML-21301-3.png" alt="Analyst agent feedback …+
<p class="wp-caption-text">Figure 3: The analyst agent feedback loop, where the batch agent ingests misclassifications from the real-time pipeline, analyzes patterns using Code Interpreter sessions, and feeds improved heuristics and models back into Tier 1 and Tier 2</p></div> -
<p><em>Video 1: Extracting evaluation metrics for sessions, traces, and spans; viewing metrics on the dashboard to understand agent performance; performing on-demand evaluation by exploring session traces and spans; running the AI engine to identify patterns across low-scoring sessions; …-
<p>Amazon Bedrock AgentCore provides 16 built-in evaluators organized by what they measure and when they run. There are 13 LLM-as-a-Judge evaluators to score interactions with detailed explanations, so teams can verify judgments and understand exactly why an interaction received a particular r…+
<p>These batch jobs can run for more than 30 minutes with Code Interpreter sessions maintained throughout. They can also span day-long operations where the agent uses Code Interpreter intermittently. For example, it runs a session, trains a model externally, then re-invokes Code Interpreter to…+
<h2 id="security-zero-trust-sandbox-design">Security: Zero-trust sandbox design</h2> +
<p>Abnormal chose the sandbox (no egress) configuration for Code Interpreter driven by two considerations:</p> +
<ol type="1"> +
<li>Reproducibility – Because the sandbox has no external network access, nothing outside Abnormal AI’s control can influence the agent’s behavior during that session. The environment is designed to be fully deterministic.</li> +
<li>Data exfiltration prevention – threat intelligence data enters the sandbox for analysis. Even if the agent becomes malicious through prompt injection or stochastic behavior, it is designed to prevent the exfiltration of that data to the internet.</li> +
</ol> +
<p>Additional security practices:</p> +
<ul> +
<li>Controlled data ingestion: intentional about what types of data enter Code Interpreter and what write actions are permitted.</li> +
<li>Subprocessor alignment: Code Interpreter operates under the existing AWS subprocessor relationship and reduces compliance overhead.</li> +
<li>Network isolation layering: sandbox isolation on top of their existing network-isolated harness provides defense in depth.</li> +
</ul> +
<h2 id="lessons-learned-and-best-practices">Lessons learned and best practices</h2> +
<p>Several practices emerged from running Code Interpreter in production at Abnormal AI.</p> +
<h3 id="give-the-agent-what-it-wants">1. Give the agent what it wants</h3> +
<p>Agents perform better with a lightweight, general harness rather than rigid step-by-step workflows. Provide high-level principles for solving a problem and let the agent use its intelligence to determine the approach.</p> +
<h3 id="every-agent-needs-a-scratch-pad">2. Every agent needs a scratch pad</h3> +
<p>Code Interpreter isn’t only for coding agents. Security agents analyzing email benefit from compute scratch pads for data aggregation, pattern analysis, and verification.</p> +
<h3 id="use-programmatic-verifiers-as-guardrails">3. Use programmatic verifiers as guardrails</h3> +
<p>Agents deliver higher quality outputs when they have programmatic verification tools. Unit tests, integration tests, and linting allow the agent to self-test within the sandbox before delivering final results.</p> +
<h3 id="use-file-systems-as-recovery-points-for-long-running-tasks">4. Use file systems as recovery points for long-running tasks</h3> +
<p>For operations exceeding the Code Interpreter session time (for example, model training), use the file system as a checkpoint. Run Code Interpreter for computation, persist state to files, perform long-running operations externally, then re-invoke Code Interpreter to process results. The an…+
<h2 id="conclusion">Conclusion</h2> +
<p>Abnormal AI’s implementation demonstrates a key insight for production agent systems: Code Interpreter is not merely a coding tool. It’s fundamental infrastructure that agents use to reason computationally. By combining the managed, secure sandbox of AgentCore Code Interpreter with their ow…+
<ul> +
<li>Zero-trust security posture through sandbox isolation helping prevent data exfiltration.</li> +
<li>Billion-message scale by reserving agent compute for the hardest cases (Figure 2).</li> +
</ul> +
<p>Whether you’re building security agents or a system where agents need to crunch data and verify their own outputs, the pattern is clear. Give your agents a scratch pad and trust their evaluations more than you trust their assertions.</p> +
<h2 id="next-steps">Next steps</h2> +
<ul> +
<li>Get started with Amazon Bedrock AgentCore Code Interpreter.</li> +
<li>Explore AgentCore capabilities and other tools (Gateway, Memory, Runtime, Identity).</li> +
<li>Learn more about Abnormal AI at abnormalsecurity.com.</li> +
</ul> +
<p>Abnormal AI is an AWS customer. The views and opinions expressed in this post are those of the customer and don’t necessarily reflect the views of Amazon Web Services.</p> +
<hr style="width: 100%"> +
<h2>About the authors</h2> +
<footer> +
<div class="blog-author-box" style="padding-top: 2.0em"> +
<div class="blog-author-image" style="margin-right: 1.0em">+
<img loading="lazy" class="alignnone size-full" src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/11/ML-21301-5.jpg" alt="Aswin Vasudevan" width="100" height="133">+
</div> +
<h3 class="lb-h4">Aswin Vasudevan</h3> +
<p style="overflow: hidden">Aswin is a Senior Solutions Architect for Security, ISV at AWS. He is a big fan of generative AI and serverless architecture and enjoys collaborating and working with customers to build solutions that drive business value.</p> +
</div> +
<div class="blog-author-box" style="padding-top: 2.0em"> +
<div class="blog-author-image" style="margin-right: 1.0em">+
<img loading="lazy" class="alignnone wp-image-139393 size-thumbnail" src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/feliplp-headshot-2.jpg" alt="" width="100" height="122">+
</div> +
<h3 class="lb-h4">Felipe Lopez</h3> +
<p style="overflow: hidden">Felipe is a Principal AI/ML Specialist Solutions Architect at AWS. Prior to joining AWS, Felipe worked with GE Digital and SLB, where he focused on modeling and optimization products for industrial applications.</p> +
</div> +
<div class="blog-author-box" style="padding-top: 2.0em"> +
<div class="blog-author-image" style="margin-right: 1.0em">+
<img loading="lazy" class="alignnone size-full" src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/11/ML-21301-6.jpg" alt="Shrivu Shankar" width="100" height="133">+
</div> +
<h3 class="lb-h4">Shrivu Shankar</h3> +
<p style="overflow: hidden">Shrivu is VP of AI Strategy at Abnormal AI, where he leads AI transformation and advances practical applications of agentic systems in cybersecurity and software development. A machine-learning engineer by background, he specializes in building AI-native workflows…+
</div> +
</footer></content:encoded>+
+
+
+
</item>+
<item>+
<title>Manage end-user OAuth consent for AI agents with Amazon Bedrock AgentCore</title>+
<link>https://aws.amazon.com/blogs/machine-learning/manage-end-user-oauth-consent-for-ai-agents-with-amazon-bedrock-agentcore/</link>+
+
<dc:creator><![CDATA[Swara Gandhi]]></dc:creator>+
<pubDate>Mon, 14 Sep 2026 20:35:45 +0000</pubDate>+
<category><![CDATA[Amazon Bedrock AgentCore]]></category>+
<category><![CDATA[Announcements]]></category>+
<category><![CDATA[Intermediate (200)]]></category>+
<guid isPermaLink="false">495d8f6ad0d4243b65c44fc7dcf8019b7a6ecbef</guid>+
+
<description>Amazon Bedrock AgentCore Identity now offers a Consent portal, a managed web experience and session binding endpoint for AgentCore Gateway. This post walks through provisioning a portal, configuring GitHub and Slack 3LO targets, and the end-user consent flow, and shows how to revie…+
<content:encoded><p>AI agents often need to access services such as GitHub and Slack on a user’s behalf. Before an agent can act, the user must authenticate with the provider and explicitly approve the requested access. The application must then securely associate the resulting OAuth…+
<p>Previously, customers using the AgentCore Identity (a capability of Amazon Bedrock AgentCore) three-legged OAuth (3LO) flow (also known as OAuth 2.0 authorization code flow) had to build and host their own session binding infrastructure. This included presenting the authorization URL, hosti…+
<p>AgentCore Identity now offers a Consent portal, a managed web experience and session binding endpoint for AgentCore Gateway, a capability of Amazon Bedrock AgentCore. You create a portal for a gateway and share its URL with your users. Users authenticate with your organization’s identity pr…+
<p>This capability is particularly useful for agents accessed through IDE and Model Context Protocol (MCP) clients such as Kiro, Claude Code, Cursor, and Visual Studio Code. Users can grant consent before invoking a tool, and subsequent tool calls can use the token already stored for that user…+
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-1.png" alt="Consent portal architec…+
<p class="wp-caption-text">Figure 1: The Consent portal authenticates the user with the corporate IdP, uses its IAM execution role to discover configured gateway targets, presents provider connections, completes session binding, and stores per-user tokens in the AgentCore Identity token vault…+
</div> +
<h2 id="example-scenario-give-a-development-assistant-access-to-github">Example scenario: Give a development assistant access to GitHub</h2> +
<p>Consider a company named Example Corp that provides its developers with an AI coding assistant through an AgentCore Gateway. The assistant has two targets:</p> +
<ul> +
<li>A GitHub target that can list repositories and create issues.</li> +
<li>A Slack target that can list public channels and post messages.</li> +
</ul> +
<p>Example Corp uses its corporate IdP to authenticate employees. The administrator wants each GitHub and Slack OAuth grant to remain associated with the employee who approved it. Developers can connect either provider independently and return to their IDE without repeated prompts.</p> +
<p>The walkthrough follows these two roles:</p> +
<ul> +
<li><strong>Administrator:</strong> Configures the corporate IdP, GitHub and Slack gateway targets, execution role, and Consent portal, and then sends the portal URL to developers.</li> +
<li><strong>End user:</strong> Opens the URL, signs in with the corporate IdP, connects GitHub when needed, and can grant Slack access separately.</li> +
</ul> +
<h2 id="prerequisites">Prerequisites</h2> +
<p>Before starting the walkthrough, Example Corp needs:</p> +
<ul> +
<li>An <a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/agentcore-get-started-cli.html" target="_blank" rel="noopener">AWS account with access to Amazon Bedrock AgentCore</a>.</li> +
<li>An <a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway-quick-start.html" target="_blank" rel="noopener">AgentCore Gateway</a> configured with <a href="https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway-inbound-auth.html" target=…+
<li>An IDE or MCP client configured to connect to the same AgentCore Gateway that will be attached to the Consent portal.</li> +
<li>Administrative access to the corporate IdP.</li> +
<li>A registered <a href="https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/creating-an-oauth-app" target="_blank" rel="noopener">GitHub OAuth App</a> and <a href="https://docs.slack.dev/authentication/installing-with-oauth/" target="_blank" rel="noopener">Slack…+
<li>Permission to register the AgentCore Identity callback URL in each provider application.</li> +
</ul> +
<h2 id="getting-started">Getting started</h2> +
<p>The following steps show what the Example Corp administrator configures and what a developer experiences after receiving the portal URL.</p> +
<h4 id="steps-for-administrator">Steps for administrator</h4> +
<p>The administrator completes Steps 1–6 to configure the identity provider, gateway targets, execution role, and Consent portal.</p> +
<h4 id="step-1-prepare-the-corporate-idp-and-gateway-connections">Step 1: Prepare the corporate IdP and gateway connections</h4> +
<p><strong>Administrator IAM policy</strong></p> +
<p>Attach this policy to the administrator identity that performs Steps 1–3. Replace the account ID.</p> +
<div class="hide-language"> +
<pre><code class="language-json">{+
"Version": "2012-10-17",+
"Statement": [+
{+
"Sid": "ManageConsentPortalResources",+
"Effect": "Allow",+
"Action": [+
"bedrock-agentcore:CreateConsentPortal",+
"bedrock-agentcore:GetConsentPortal",+
"bedrock-agentcore:ListConsentPortals",+
"bedrock-agentcore:CreateOauth2CredentialProvider",+
"bedrock-agentcore:GetOauth2CredentialProvider",+
"bedrock-agentcore:ListOauth2CredentialProviders",+
"bedrock-agentcore:GetGateway",+
"bedrock-agentcore:ListGateways",+
"bedrock-agentcore:CreateGatewayTarget",+
"bedrock-agentcore:GetGatewayTarget",+
"bedrock-agentcore:ListGatewayTargets",+
"bedrock-agentcore:UpdateGatewayTarget"+
],+
"Resource": "*"+
},+
{+
"Sid": "CreateAndPassExecutionRole",+
"Effect": "Allow",+
"Action": [+
"iam:CreateRole",+
"iam:GetRole",+
"iam:PutRolePolicy",+
"iam:GetRolePolicy",+
"iam:PassRole"+
],+
"Resource": "arn:aws:iam::111122223333:role/service-role/AmazonBedrockAgentCoreConsentPortal*"+
}+
]+
}</code></pre> +
</div> +
<p>The AWS Identity and Access Management (IAM) statement covers the <strong>Create default role</strong> option on the console, which creates a service role named <code>AmazonBedrockAgentCoreConsentPortalDefaultServiceRole-&lt;suffix&gt;</code>. If you supply y…+
<h4 id="step-2-configure-the-corporate-idp-application">Step 2: Configure the corporate IdP application</h4> +
<ol type="1"> +
<li>In the corporate IdP, create an OpenID Connect (OIDC) web application for the Consent portal.</li> +
<li>Enable the authorization code grant and generate a client ID and client secret.</li> +
<li>Configure the login scopes, at minimum <code>openid</code>.</li> +
<li>Record the OpenID Connect (OIDC) discovery URL. The portal uses the authorization endpoint, token endpoint, and signing keys from this document.</li> +
<li>Add a temporary callback URL. You replace it in Step 6, after the portal URL exists.</li> +
</ol> +
<p>The IdP must issue a JSON Web Token (JWT) access token that the portal can validate. For example, with Okta, use a custom authorization server with an access policy that permits the application and the authorization code grant. With Auth0, configure an audience when needed so the IdP return…+
<h4 id="step-3-create-the-idp-credential-provider">Step 3: Create the IdP credential provider</h4> +
<p>The portal reads the IdP client ID and client secret from an OAuth2 credential provider.</p> +
<ol type="1"> +
<li>Open the Amazon Bedrock AgentCore console.</li> +
<li>Under <strong>Build</strong>, choose <strong>Identity</strong>.</li> +
<li>In <strong>Outbound Auth</strong>, choose <strong>Add Outbound Auth</strong>, and then choose <strong>Add OAuth client</strong>.</li> +
<li>Enter a name, such as <code>gateway-demo-idp</code>.</li> +
<li>Enter the client ID and client secret from the corporate IdP application, and provide the OIDC discovery configuration.</li> +
<li>Choose <strong>Add OAuth client</strong>.</li> +
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-5.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-5.png" alt="AgentCore Identity page…+
<p class="wp-caption-text">Figure 2: AgentCore Identity uses one credential provider for portal sign-in and separate outbound providers for GitHub and Slack</p>+
</div> +
<h4 id="step-4-verify-outbound-prerequisites">Step 4: Verify outbound prerequisites</h4> +
<p>Before creating the Consent portal, confirm:</p> +
<ul> +
<li>The GitHub and Slack OAuth applications are registered, and their client secrets are stored in AWS Secrets Manager.</li> +
<li>AgentCore Identity has separate outbound OAuth credential providers for GitHub and Slack, and each generated callback URL is registered with the matching provider application.</li> +
<li>The GitHub and Slack gateway targets use the authorization code grant, request only the required scopes, and have a <strong>Ready</strong> status.</li> +
<li>The Consent portal execution role selected in Step 5 can read any customer-managed secrets referenced by the outbound credential providers.</li> +
</ul> +
<p>After the portal URL is assigned, configure each target’s default return URL in Step 6.</p> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-6.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-6.png" alt="Gateway with independen…+
<p class="wp-caption-text">Figure 3: The gateway exposes independent GitHub and Slack targets, each associated with its own outbound OAuth provider</p>+
</div> +
<h4 id="step-5-create-the-consent-portal">Step 5: Create the Consent portal</h4> +
<p>On the <strong>Identity</strong> page, the <strong>Consent portals</strong> section lists the portals in the account and AWS Region.</p> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-2.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-2.png" alt="Consent portals section…+
<p class="wp-caption-text">Figure 4: The Consent portals section on the AgentCore Identity page. Create a portal only when a gateway uses 3LO that requires user consent</p>+
</div> +
<ol type="1"> +
<li>In <strong>Consent portals</strong>, choose <strong>Create portal</strong>.</li> +
<li>Under <strong>Consent portal details</strong>, for <strong>Name</strong>, enter a name such as <code>consent-portal-heqk0</code>. Names accept 1–50 characters, using letters, numbers, hyphens, and underscores.</li> +
<li>Optionally enter a <strong>Description</strong> of up to 512 characters.</li> +
<li>For <strong>Gateway</strong>, select the development assistant’s gateway. The gateway name is visible to end users in the Consent portal, so choose a clear, recognizable name. One Consent portal is allowed per gateway, and the gateway can’t be changed after creation.</li>…+
<li>Under <strong>IdP credential configurations</strong>, for <strong>IdP Credential Provider</strong>, select the OAuth2 credential provider created in Step 3.</li> +
<li>Under <strong>Scopes</strong>, keep the required <code>openid</code> scope. Additional scopes are optional. Choose <strong>Add scope</strong> only when your IdP or application requires them.</li> +
<li>For <strong>Audience – optional</strong>, keep <strong>None</strong> unless your gateway specifies audiences. The value is validated against the audiences configured on the AgentCore Gateway.</li> +
<li>Expand <strong>Permissions</strong>. For <strong>IAM permissions</strong>, choose <strong>Create default role</strong> to have the console create a service role with the required permissions, or choose <strong>Use another role</strong> to se…+
<li>Choose <strong>Create portal</strong>.</li> +
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-3.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-3.png" alt="Consent portal detail p…+
<p class="wp-caption-text">Figure 5: While the status is Creating, the portal ARN and execution role are visible but the URL isn’t assigned yet</p>+
</div> +
<p>When provisioning finishes, the status changes to <strong>Active</strong>, the <strong>Consent portal URL</strong> appears, and a <strong>Launch Consent portal</strong> button opens it in a new tab. The URL follows the pattern <code>https://&lt;…+
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-4.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/09/14/ML-21874-4.png" alt="Consent portal detail p…+
<p class="wp-caption-text">Figure 6: After the portal becomes Active, the console shows the assigned URL that you share with end users</p>+
</div> +
<h4 id="step-6-register-callback-urls-and-send-the-portal-url">Step 6: Register callback URLs and send the portal URL</h4> +
<ol type="1"> +
<li>Copy the <strong>consent portal URL</strong> from the portal details page.</li> +
<li>In the corporate IdP application, for example, Amazon Cognito, Okta, or Auth0, replace the temporary callback with <code>&lt;portal-url&gt;/callback</code>. Don’t add a trailing slash. This isn’t the GitHub or Slack application callback. Those applications use the uniq…+
<li>For each 3LO gateway target, set the default return URL to <code>&lt;portal-url&gt;/connect/callback</code>.</li> +
<li>Confirm that each outbound provider application contains the AgentCore Identity callback URL returned when its OAuth credential provider was created.</li> +
<li>Test the portal URL in a browser.</li> +
<li>Send the portal URL to the development team through an approved communication channel.</li> +
</ol> +
<h4 id="callback-url-reference">Callback URL reference</h4> <table border="1px" width="100%" cellpadding="10px"> <tbody> <tr> -
<td><strong>Evaluator</strong></td> -
<td><strong>Definition</strong></td> -
<td><strong>Evaluation Level</strong></td> -
</tr> -
<tr> -
<td>Goal Success Rate</td> -
<td>Assesses whether the AI assistant successfully completed the user goals within a conversation session by analyzing the entire conversation end-to-end.</td> -
<td>Session</td> -
</tr> -
<tr> -
<td>Coherence</td> -
<td>Assesses the logical consistency and cohesion of a response, checking for self-contradictions, logic gaps, and soundness of reasoning without evaluating factual accuracy.</td> -
<td>Trace</td> -
</tr> -
<tr> -
<td>Conciseness</td> -
<td>Measures how efficiently the assistant communicates, assessing whether responses provide necessary information using minimal words without unnecessary elaboration.</td> -
<td>Trace</td> -
</tr> -
<tr> -
<td>Correctness</td> -
<td>Assesses the factual accuracy of a response to a given task, focusing on whether the content and solution are accurate regardless of style or presentation.</td> -
<td>Trace</td> -
</tr> -
<tr> -
<td>Faithfulness</td> -
<td>Assesses whether a response remains consistent with the conversation history, identifying conflicts between the current response and previous interactions within the same conversation.</td> -
<td>Trace</td> Diff display stops at 400 lines. The line counts above are from the whole diff. 63 lines shown here cut at 300 characters. The raw artifact at this commit is linked above.