Change
50705ce
50705ceb80e71e902aae54e3246d16b3c97bfe86 · commit on GitHub
aws-blog-feed: changed (699635 bytes, HTTP 200)
raw/aws-blog-feed/response.xml modified
- Source
- aws-blog-feed
- Lines added
- +1,173
- Lines removed
- -1,076
- Stored bytes at this commit
- 699,635
- Timestamp
- observed
- Raw artifact at this commit
- raw/aws-blog-feed/response.xml
Recorded headers
| observed_at | 2026-10-27T05:05:08.284Z |
|---|---|
| origin_date | null |
| status | 200 |
| final URL | https://aws.amazon.com/blogs/machine-learning/feed/ |
| etag | null |
| last-modified | Mon, 26 Oct 2026 23:03:18 GMT |
| date | Tue, 27 Oct 2026 05:05:08 GMT |
| age | null |
| cache-control | null |
| cf-cache-status | null |
| content-encoding | null |
| content-length | null |
@
@@ -5,7 +5,7 @@ <atom:link href="https://aws.amazon.com/blogs/machine-learning/feed/" rel="self" type="application/rss+xml"/> <link>https://aws.amazon.com/blogs/machine-learning/</link> <description>Official Machine Learning Blog of Amazon Web Services</description>-
<lastBuildDate>Fri, 23 Oct 2026 17:48:54 +0000</lastBuildDate>+
<lastBuildDate>Mon, 26 Oct 2026 17:56:58 +0000</lastBuildDate> <language>en-US</language> <sy:updatePeriod> hourly </sy:updatePeriod>@
@@ -13,130 +13,1218 @@ 1 </sy:updateFrequency> <item>-
<title>Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide</title>-
<link>https://aws.amazon.com/blogs/machine-learning/multi-turn-reinforcement-learning-on-amazon-sagemaker-ai-a-hands-on-guide/</link>+
<title>Fleet-wide computer vision lifecycle at the edge with SageMaker SDK v3</title>+
<link>https://aws.amazon.com/blogs/machine-learning/fleet-wide-computer-vision-lifecycle-at-the-edge-with-sagemaker-sdk-v3/</link> -
<dc:creator><![CDATA[Daniel Quang]]></dc:creator>-
<pubDate>Fri, 23 Oct 2026 17:48:54 +0000</pubDate>-
<category><![CDATA[Amazon SageMaker AI]]></category>-
<category><![CDATA[Expert (400)]]></category>+
<dc:creator><![CDATA[Kiowa Jackson]]></dc:creator>+
<pubDate>Mon, 26 Oct 2026 17:56:58 +0000</pubDate>+
<category><![CDATA[Advanced (300)]]></category>+
<category><![CDATA[Amazon SageMaker]]></category> <category><![CDATA[Technical How-to]]></category>-
<guid isPermaLink="false">d4a0021627e124e1cbf32bd2a1426e1abfda6b33</guid>+
<guid isPermaLink="false">24d8d5b36e78caf18670ef6d83892be19ec25356</guid>-
<description>Multi-Turn Reinforcement Learning (MTRL) on Amazon SageMaker AI trains models on the outcome of a whole multi-step task rather than labeled steps. This hands-on guide walks the full pipeline using Wordle: deploying the agent and reward on Amazon Bedrock AgentCore, preparing the dat…-
<content:encoded><p>Most fine-tuning teaches a model to produce a better answer to a single prompt. Agentic tasks are different. An agent that files an expense report or triages a support ticket takes a sequence of actions, and each step depends on what came before. There’s rarely on…-
<p>Multi-Turn Reinforcement Learning (MTRL) on Amazon SageMaker AI trains models on exactly that signal. SageMaker AI sends prompts to an agent you host on Amazon Bedrock AgentCore, or on your own infrastructure, such as Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Compute Cl…-
<p>This post uses <a href="https://en.wikipedia.org/wiki/Wordle" target="_blank" rel="noopener">Wordle</a>, a game in which a player has six tries to guess a hidden five-letter word, as the example. After each guess, every letter turns green if it is in the right position, yellow i…-
<p>In our run, MTRL took gpt-oss-20b from a 61 to an 84 percent solve rate on held-out words at its default reasoning effort, and from 15 to 69 percent at the low effort it trained at, for about $339 in training tokens. In this post, we walk through the pipeline end-to-end: deploying the agent…+
<description>Fleet-wide computer vision needs a model per camera, not one model for the whole fleet. This post presents a working Amazon SageMaker SDK v3 reference architecture that trains, quality-gates, compiles per-camera models with SageMaker Neo, and versions thousands of them, with an AWS…+
<content:encoded><p>Fleet-wide computer vision model lifecycle management is a problem that shows up the moment a quality inspection proof of concept succeeds. One camera and one model work fine. Then the plant asks for the same result on 40 lines, each with several cameras, and ever…+
<p>This post updates the pattern introduced in the 2023 series <a href="https://aws.amazon.com/blogs/machine-learning/build-an-end-to-end-mlops-pipeline-for-visual-quality-inspection-at-the-edge-part-1/" target="_blank" rel="noopener">Build an end-to-end MLOps pipeline for visual quality…+
<p>The complete sample is available in the <a href="https://github.com/aws-samples/sample-sagemaker-fleet-cv-pipeline" target="_blank" rel="noopener">aws-samples GitHub repository</a>.</p> <h2 id="solution-overview">Solution overview</h2> -
<p>The system has two halves. SageMaker AI owns the training job and the policy model. AgentCore hosts the agent, the Wordle environment, and the reward function. The following diagram shows how they connect.</p> +
<p>The following diagram shows the sample’s fleet-wide model lifecycle. An orchestration layer queues re-training requests and limits concurrent work. A per-camera training pipeline prepares data, trains and evaluates an anomaly detector and a classifier, and produces versioned model artifacts…<div style="width: 810px" class="wp-caption alignnone">-
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/12/ML-22014-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/12/ML-22014-1.png" alt="SageMaker AI training j…-
<p class="wp-caption-text">Figure 1: MTRL on SageMaker AI, where the training job invokes the AgentCore-hosted agent once per rollout to sample the policy model, play the game, and report a reward</p>+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20483-1-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20483-1-1.png" alt="Solution architectu…+
<p class="wp-caption-text">Figure 1: Solution architecture</p></div> -
<p>For every rollout, the MTRL service invokes the AgentCore runtime, a capability of Amazon Bedrock AgentCore, with one training row, which holds one secret word. A Strands agent in the runtime calls the policy model through the SageMaker Job Runtime’s OpenAI-compatible endpoint. The model pl…-
<p>Three design points matter for what follows:</p> +
<p>The training half is a single SageMaker Pipeline, parameterized by camera, that runs 13 steps per execution. Three entry steps run in parallel: Preprocess, Fetch Latest Anomaly, and Fetch Latest Classifier. The pipeline preprocesses that camera’s images and fetches the camera’s most recent …+
<p>The orchestration half paces the fleet. Re-training requests land on an Amazon Simple Queue Service (Amazon SQS) FIFO queue, and an AWS Lambda function starts a Step Functions execution per request. Each execution acquires a slot through a DynamoDB semaphore before starting its pipeline, so…+
<p>The output of the system is versioned, quality-gated, per-camera model artifacts. The Model Registry holds the portable ONNX, and S3 holds the Neo-compiled build of that model, which is a device-optimized shared library loaded by the DLR runtime rather than ONNX. Both are ready for whatever…+
<h2 id="prerequisites">Prerequisites</h2> +
<p>To deploy the sample you need the following:</p> <ul> -
<li><strong>The agent and reward live in one place.</strong> A single Python file deployed to AgentCore holds the environment, the tool, and the scoring function.</li> -
<li><strong>The dataset carries only the secret word.</strong> The word lists bundled with the agent are built from the open-source <a href="http://wordlist.aspell.net/" target="_blank" rel="noopener">SCOWL</a> word lists and are used only for scoring.</li> -
<li><strong>The reward is outcome-dominant.</strong> A solve pays 1.05–1.55 depending on speed, a loss pays -0.5, and never making a valid guess pays -1.5. A small shaping term rewards good strategy within each tier.</li> +
<li>An AWS account with permissions to create AWS Identity and Access Management (IAM) roles.</li> +
<li>An Amazon Simple Storage Service (Amazon S3) bucket.</li> +
<li>An Amazon Elastic Container Registry (Amazon ECR) repository.</li> +
<li>The SageMaker AI, AWS Step Functions, AWS Lambda, Amazon Simple Queue Service (Amazon SQS), Amazon Simple Notification Service (Amazon SNS), Amazon EventBridge, and Amazon DynamoDB resources in the stacks.</li> +
<li>Python 3.10, 3.11, or 3.12, Docker.</li> +
<li>The AWS CDK CLI version 2.1131.0 or later.</li> </ul> -
<h3 id="supported-models-and-regions">Supported models and AWS Regions</h3> -
<p>MTRL is serverless: you pick a base model and the service provisions the training and sampling infrastructure. At the time of writing, it supports the following base models, each trained with a LoRA adapter:</p> +
<p>Previous CLI versions fail with a cloud assembly schema mismatch against recent versions of <code>aws-cdk-lib</code>. No service quota increase is needed. The orchestration layer admits at most five pipeline executions at a time by default, which keeps concurrent SageMaker train…+
<p>If you deploy to an AWS Region other than your CLI default, set all three of <code>AWS_REGION</code>, <code>AWS_DEFAULT_REGION</code>, and <code>CDK_DEFAULT_REGION</code> on the deploy command itself rather than in your shell profile, for example, <cod…+
<h2 id="defining-the-training-pipeline-with-sdk-v3">Defining the training pipeline with SDK v3</h2> +
<p>The code in this section is excerpted from the CDK application in the repository rather than a standalone script, and it runs during cdk deploy, when the stack synthesizes the pipeline definition. Environment setup and the deploy commands come later, under Deploying and running the sample. …+
<div class="hide-language"> +
<pre><code class="language-python">from sagemaker.train import ModelTrainer+
from sagemaker.train.configs import InputData, Compute, OutputDataConfig+
from sagemaker.core.training.configs import MetricDefinition+
from sagemaker.serve.model_builder import ModelBuilder+
from sagemaker.mlops.workflow.pipeline import Pipeline+
from sagemaker.mlops.workflow.steps import ProcessingStep, TrainingStep+
from sagemaker.mlops.workflow.model_step import ModelStep+
from sagemaker.mlops.workflow.condition_step import ConditionStep+
from sagemaker.core.workflow.conditions import ConditionGreaterThanOrEqualTo+
from sagemaker.core.workflow.functions import JsonGet</code></pre> +
</div> +
<p>Installing <code>sagemaker&gt;=3.0.0,&lt;4</code> pulls in <code>sagemaker-core</code>, <code>sagemaker-train</code>, <code>sagemaker-serve</code>, and <code>sagemaker-mlops</code> as separate packages. Training configurati…+
<div class="hide-language"> +
<pre><code class="language-python">anomalib_trainer = ModelTrainer(+
training_image=custom_training_image_uri,+
compute=Compute(+
instance_type=training_instance_type.to_string(),+
instance_count=1,+
),+
base_job_name="cv-pipeline-anomalib-train",+
sagemaker_session=pipeline_session,+
role=role_arn,+
hyperparameters={+
"program": "train_anomalib.py",+
"model_name": "autoencoder",+
"image_size": "256",+
"num_epochs": "3",+
"batch_size": "16",+
"learning_rate": "0.001",+
},+
input_data_config=anomalib_input_data,+
output_data_config=OutputDataConfig(+
s3_output_path=run_prefix("anomalib-training")+
),+
).with_metric_definitions(+
[+
MetricDefinition(+
name="image_auroc",+
regex=r"validation:image_auroc=([0-9]*\.?[0-9]+)",+
),+
]+
)</code></pre> +
</div> +
<p><code>anomalib_input_data</code> wires four channels (<code>train</code>, <code>validation</code>, <code>test</code>, <code>pretrained_model</code>) from upstream step properties, and <code>run_prefix</code> scopes ever…+
<p>Two v3 behaviors matter when the pipeline lives inside a CDK application, and neither appears in the changelog as a breaking change. First, SDK v3 validates the pipeline’s execution role against live IAM while building the pipeline definition. This validation happens at synthesis time rathe…+
<p>Second, the v3 role validator simulates ECR pull permissions without a resource ARN, so an execution role scoped to a single ECR repository fails validation even though it would work at runtime. The sample’s role carries a broader ECR grant for this reason.</p> +
<p>Lastly, one v3 habit the sample uses: constructing the pipeline with <code>PipelineSession(default_bucket=your_bucket)</code> routes SDK-managed uploads, such as step code, into the pipeline’s own bucket, so every artifact lives in one place and the execution role’s permissions …+
<h2 id="quality-gates-that-match-how-defect-data-behaves">Quality gates that match how defect data behaves</h2> +
<p>Every trained model is evaluated before registration, and the evaluation runs against the exported ONNX artifact rather than the in-memory PyTorch model. That distinction matters: the artifact you gate is the artifact you ship, so an export problem can never slip a broken model past a healt…+
<p>The anomaly branch gates on AUROC of at least 0.8. The classifier branch gates on two metrics at once, and the choice of metrics is shaped by the defining property of defect data: heavy class imbalance. In this sample roughly 90 percent of images are good, which is realistic for a productio…+
<div class="hide-language"> +
<pre><code class="language-python">step_check_classifier = ConditionStep(+
name="CheckClassifierQuality",+
conditions=[+
ConditionGreaterThanOrEqualTo(+
left=JsonGet(+
step_name=step_eval_classifier.name,+
property_file="evaluation",+
json_path="balanced_accuracy",+
),+
right=min_accuracy,+
),+
ConditionGreaterThanOrEqualTo(+
left=JsonGet(+
step_name=step_eval_classifier.name,+
property_file="evaluation",+
json_path="defect_recall",+
),+
right=min_defect_recall,+
),+
],+
if_steps=[step_compile_classifier, step_register_classifier],+
else_steps=[step_fail_classifier],+
depends_on=[step_eval_classifier],+
)</code></pre> +
</div> +
<p>A <code>ConditionStep</code> ANDs its conditions, so both must pass. The thresholds are pipeline parameters, adjustable per execution without redeploying. If a gate fails, a <code>FailStep</code> marks the execution failed and that branch’s model never reaches the re…+
<p>The training code addresses the same imbalance from the other side. The classifier is a fine-tuned <code>resnet18</code> with ImageNet weights and a two-class head. At per-camera dataset sizes, fine-tuning pretrained weights is what keeps the model from collapsing onto the major…+
<h2 id="compiling-for-the-edge-with-sagemaker-neo-inside-the-pipeline">Compiling for the edge with SageMaker Neo, inside the pipeline</h2> +
<p>Notice the gate’s <code>if_steps</code> in the preceding code: past the gate, compilation runs first and registration depends on it. Both training scripts export ONNX with dynamic batch axes. The quality gates score those exported artifacts under ONNX Runtime, and then a <cod…+
<p>The compile step encapsulates three error-prone details. Neo requires the input tarball to contain only the model file, but training jobs emit <code>model.tar.gz</code> with the checkpoint and metrics alongside the ONNX model. The step repackages an ONNX-only tarball first. Neo …+
<p>The target is a pipeline parameter: <code>NeoTargetDevice</code> selects a named device such as <code>jetson_xavier</code> or <code>rasp4b</code>, and its default compiles for the <code>NeoTargetOs</code>/<code>NeoTargetArch</code>…+
<h2 id="per-camera-models-in-the-model-registry">Per-camera models in the Model Registry</h2> +
<p>Each branch registers into its own model package group, and every package carries the camera ID and its evaluation metrics. From the sample, with the metrics wiring elided:</p> +
<div class="hide-language"> +
<pre><code class="language-python">classifier_model_builder = ModelBuilder(+
s3_model_data_url=classifier_model_data,+
image_uri=custom_training_image_uri,+
sagemaker_session=pipeline_session,+
role_arn=role_arn,+
)+
step_register_classifier = ModelStep(+
name="RegisterClassifierModel",+
depends_on=[step_compile_classifier],+
step_args=classifier_model_builder.register(+
model_package_group_name="sm1k-classifier-model-group",+
approval_status="Approved",+
model_metrics=classifier_metrics,+
customer_metadata_properties={"CameraId": camera_id},+
),+
)</code></pre> +
</div> +
<p>Camera identity in <code>customer_metadata_properties</code> is what turns two shared package groups into a fleet catalog. Packages from all cameras live in the same groups, and per-camera history is recovered by filtering on the metadata. Each branch has a fetch step that pages…+
<p>Note what these registry entries are and are not: they catalog versioned, quality-gated artifacts, but their container reference is the training image, which serves no inference endpoints. They exist for a fleet workflow where a downstream system pulls artifacts and ships them to devices, n…+
<h2 id="orchestrating-fleet-training-requests">Orchestrating fleet training requests</h2> +
<p>Every AWS account has service quotas on concurrent SageMaker training jobs, and each pipeline execution here launches several jobs, so fleet re-training needs pacing. Something has to keep a burst of 500 requests from becoming 500 simultaneous pipeline launches. The sample uses a semaphore,…+
<div class="hide-language"> +
<pre><code class="language-python">acquire_semaphore = tasks.DynamoUpdateItem(+
self, "AcquireSemaphore",+
table=semaphore_table,+
key={"LockName": tasks.DynamoAttributeValue.from_string("pipeline-concurrency")},+
update_expression="SET ActiveCount = if_not_exists(ActiveCount, :zero) + :one",+
condition_expression="attribute_not_exists(ActiveCount) OR ActiveCount &lt; :limit",+
)</code></pre> +
</div> +
<p>If all slots are taken, the execution waits and retries. When its pipeline finishes, it decrements the counter to release the slot. One detail is worth carrying into your own designs. The DynamoDB condition expression enforces the cap in a single atomic operation, rather than the state mach…+
<p>The cap of five is a stack parameter, set low so the sample deploys and runs in any standard account without requesting quota increases. Adjust it to whatever your account’s training-job quotas support. Past a single account’s quotas, scaling fleet training is its own topic. The standard le…+
<h2 id="deploying-and-running-the-sample">Deploying and running the sample</h2> +
<p>The walkthrough in the repository README is the authoritative version. The following short form starts from a clean checkout and sets up the environment before the first deploy:</p> +
<div class="hide-language"> +
<pre><code class="language-bash">git clone https://github.com/aws-samples/sample-sagemaker-fleet-cv-pipeline+
cd sample-sagemaker-fleet-cv-pipeline &amp;&amp; python3 -m venv .venv &amp;&amp; source .venv/bin/activate+
pip install -r requirements.txt+
cdk bootstrap # once per account/region+
cdk deploy SageMakerCVPipelineStack -c bootstrap=true # roles, bucket, ECR+
cdk deploy --all # pipeline + orchestration+
./scripts/push_ecr_image.sh # build and push training image</code></pre> +
</div> +
<p>By default the pipeline generates synthetic production line images, so you can validate the whole system end to end with no dataset and no data approvals. A full run executes all 13 steps, including both Neo compilations, in about 15 minutes and costs roughly $0.07 in billed <code>ml.…+
<p>To exercise the pipeline on real images, the repository includes a downloader for the <a href="https://github.com/Shenggan/BCCD_Dataset" target="_blank" rel="noopener">BCCD dataset</a>. The dataset is a collection of 364 annotated blood smear microscopy images originally open so…+
<h2 id="cleaning-up">Cleaning up</h2> +
<p>Model packages are created at pipeline runtime, so AWS CloudFormation cannot delete them. Remove them with the AWS CLI first, then destroy the stacks with <code>cdk destroy --all -c bootstrap=true --force</code>. The <a href="https://github.com/aws-samples/sample-sagemaker-fl…+
<h2 id="conclusion">Conclusion</h2> +
<p>In this post, I showed a working SageMaker SDK v3 reference architecture for the fleet-scale computer vision lifecycle. The architecture includes a dual-branch training pipeline where every model is quality-gated on its exported ONNX artifact, compiled with SageMaker Neo, and only then regi…+
<p>The sample deploys in minutes and a full validation run costs about seven cents of compute, so a low-cost next step is to clone the <a href="https://github.com/aws-samples/sample-sagemaker-fleet-cv-pipeline" target="_blank" rel="noopener">repository</a> and run it. From there, p…+
<p>If you want help adapting this pattern to your production environment, reach out to your AWS account team.</p> +
<p>&nbsp;</p> +
<p style="clear: both"></p> +
<hr style="width: 100%"> +
<h2>About the author</h2> +
<footer> +
<div class="blog-author-box" style="padding-top: 2.0em"> +
<div class="blog-author-image" style="margin-right: 1.0em">+
<img loading="lazy" class="alignnone size-full" src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20483-2-1.jpg" alt="Kiowa Jackson" width="100" height="133">+
</div> +
<h3 class="lb-h4">Kiowa Jackson</h3> +
<p style="overflow: hidden">Kiowa is a Senior Machine Learning Engineer at AWS ProServe. He specializes in visual intelligence systems that combine computer vision with agentic AI, and he brings these to production in the enterprise industrial sector.</p> +
</div> +
</footer></content:encoded>+
+
+
+
</item>+
<item>+
<title>Groundhog Day for AI agents: Prompt caching patterns for using OpenAI models on Amazon Bedrock</title>+
<link>https://aws.amazon.com/blogs/machine-learning/groundhog-day-for-ai-agents-prompt-caching-patterns-for-using-openai-models-on-amazon-bedrock/</link>+
+
<dc:creator><![CDATA[Claudio Mazzoni]]></dc:creator>+
<pubDate>Mon, 26 Oct 2026 17:27:20 +0000</pubDate>+
<category><![CDATA[Amazon Bedrock]]></category>+
<category><![CDATA[Best Practices]]></category>+
<category><![CDATA[Expert (400)]]></category>+
<guid isPermaLink="false">de80c503415138bb46de169401d016e12857bb15</guid>+
+
<description>Every time an agent takes a step, it re-reads its entire context before acting. Here is how prompt caching for OpenAI models on Amazon Bedrock lets your agent remember, cutting input cost and latency without resending the same tools, instructions, and history every turn.</descripti…+
<content:encoded><p><em>Every time you spin off your agent, it has to re-read its whole life story before performing an action. Here’s how to make it remember without having to feed it the same context again.</em></p> +
<p>In the movie <em>Groundhog Day</em>, Bill Murray’s character re-lives the same February 2 every day. He wakes up to the same song playing on the radio, in the same inn, and the same small town, again and again for what some movie fans estimate to be 34 years.</p> +
<p>Nothing he accomplished the day before carries over unless he works it out again from scratch. AI agents live in a version of Groundhog Day. Every time an agent takes a step (calls a tool, reads the result, decides what to do next), it has to send the model its entire story so far to mainta…+
<p>Prompt caching breaks the cycle. The service stores your input prompt (instructions, tools, and content) and reuses it on the next request, at a fraction of the input price.</p> +
<p>In this post, I walk you through an agent’s life, from its first turn to when it gets new tools and spawns helpers. Along the way, I show you how prompt caching keeps costs down and keeps agents responsive and consistent.</p> +
<h2 id="what-an-agent-sends-to-its-llm">What an agent sends to its LLM</h2> +
<p>Before talking about caching, it helps to look at what a typical agent’s first invocation to the large language model (LLM) contains:</p> +
<ul> +
<li><strong>Tool list</strong> – The definitions of every function the agent can call. Changes when you ship a new release.</li> +
<li><strong>Instructions</strong> – The developer prompt: rules, persona, policies, examples. Also changes per release.</li> +
<li><strong>History</strong> – Everything that happened in this session so far: the task, the model’s tool calls, and the tool results (which grows every turn).</li> +
<li><strong>New turn</strong> – The latest tool result or user message (which might be different every time).</li> +
</ul> +
<p><strong>What an agent sends to the model, turn after turn if you don’t set up caching:</strong></p> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/23/ML-22246-1-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/23/ML-22246-1-1.png" alt="Five growing agent …+
<p class="wp-caption-text">Figure 1: What an agent sends to the model on every turn</p>+
</div> +
<h2 id="how-prompt-caching-works-under-the-hood">How prompt caching works under the hood</h2> +
<p>When a model reads a prompt, it first does a write called <em>prefill</em>. During prefill, it processes every input token and builds its key and value vectors (also known as key-value, or KV), which form the model’s working state. This happens before the model writes its first …+
<p>Prompt caching saves that state, which contains the system prompt, tools, content, history, and so on. When a later request starts with the same input tokens (that is, the same text), the prompt caching service loads the saved state instead of computing it again. It then processes only what…+
<p><strong>Turn 1 compared to turn 2</strong></p> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/23/ML-22246-2-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/23/ML-22246-2-1.png" alt="Turn 1 writes the t…+
<p class="wp-caption-text">Figure 2: Turn 1 saves the prompt prefix up to the breakpoint, and turn 2 reuses it and pays full price only for the new question</p>+
</div> +
<p><strong>Implicit or explicit, and why do we need both?</strong></p> +
<p>For agentic workloads, implicit caching lets Bedrock choose what to cache as the conversation grows. With explicit caching, you can mark shared content, such as tools and instructions, for reuse across sessions and when spinning up helper agents. You don’t always need both forms of caching,…+
<p>New sessions can also reuse that shared starting point while each conversation builds on its own cached history. In this post, we start with the first turn, then show how to carry cached content through later turns, new tools, and across subagents.</p> +
<h2 id="hello-cached-world">Hello cached world</h2> +
<p>The first win is to cache the static part of the prompt: the tools and instructions that stay the same across requests. These examples use your existing Amazon Bedrock client, <code>TOOLS</code>, and <code>task</code>. Replace the shortened <code>INSTRUCTIONS&l…+
<p>When caching static content, use explicit mode: choose where the saved prefix ends.</p> +
<div class="hide-language"> +
<pre><code class="language-python">KEY = "support-agent:v7:gpt-6-astra"+
INSTRUCTIONS = "Your full support policies and examples..."+
+
# Rendered context: [provider context][TOOLS][INSTRUCTIONS] | [task]+
# Cache starts at the beginning; the marker below chooses its end.+
resp = client.responses.create(+
model="us.openai.gpt-6-astra",+
store=False, # Response storage; prompt caching still works.+
include=["reasoning.encrypted_content"],+
prompt_cache_key=KEY,+
prompt_cache_options={"mode": "explicit", "ttl": "30m"},+
tools=TOOLS, # Rendered before instructions; keep order stable.+
input=[+
{+
"role": "developer",+
"content": [{+
"type": "input_text",+
"text": INSTRUCTIONS,+
# Save through this whole block, including TOOLS.+
"prompt_cache_breakpoint": {"mode": "explicit"},+
}],+
},+
# User message is processed but not cached.+
{"role": "user", "content": task},+
],+
)</code></pre> +
</div> +
<p>The <code>prompt_cache_key</code> tells the service which requests are meant to share a cache. We come back to how to use it when we get to reading the cache in new sessions. In one of our tests using explicit caching, the roughly 4,000-token tools-and-instructions prefix wrote …+
<p>Keep in mind that prompt caching works only when you pass exactly the same text. Make sure that the agent’s instructions and tool lists are serialized the same way every time. Model Context Protocol (MCP) servers and other forms of tool delivery need to be indexed to make sure they stay con…+
<p>As the content grows and varies (different tools and content), use implicit mode to let the service follow the conversation. Here, the first request has no explicit marker. Its latest eligible message is the user’s task, so the automatic breakpoint comes after that task and covers the tools…+
<div class="hide-language"> +
<pre><code class="language-python">def ask_with_implicit_cache(history):+
return client.responses.create(+
model="us.openai.gpt-6-astra",+
store=False,+
include=["reasoning.encrypted_content"],+
prompt_cache_key=KEY,+
prompt_cache_options={"mode": "implicit", "ttl": "30m"},+
tools=TOOLS,+
input=history, # Everything gets cached+
)+
+
history = [+
{"role": "developer", "content": INSTRUCTIONS},+
{"role": "user", "content": task},+
]+
# Automatic endpoint: the end of the user message.+
resp = ask_with_implicit_cache(history)</code></pre> +
</div> +
<p>This is useful when the next turn continues the same task. A different task doesn’t match the prefix through the old task, and the implicit write doesn’t necessarily create a separate entry for only the tools and instructions. Use an explicit marker there when you need that shorter shared p…+
<p>Use both when the agent needs a shared starting point and a growing history. Start with the marked developer block in the following code example and call the function <code>ask_with_implicit_cache</code>. The request-level mode stays implicit. The block-level marker adds an expl…+
<div class="hide-language"> +
<pre><code class="language-python">history = [+
{+
"role": "developer",+
"content": [{+
"type": "input_text",+
"text": INSTRUCTIONS,+
"prompt_cache_breakpoint": {"mode": "explicit"}, +
}],+
},+
{"role": "user", "content": task},+
]+
# Still implicit mode, now with an explicit startup save point in the history.+
resp = ask_with_implicit_cache(history)</code></pre> +
</div> +
<h2 id="caching-conversation-history-and-tool-results">Caching conversation history and tool results</h2> +
<p>The model requests tools by name and supplies their arguments. Your application must execute them. <code>execute_function</code> is a helper you build to route those requests to the matching Python functions and return their results. The following loop uses this helper to run ea…+
<div class="hide-language"> +
<pre><code class="language-python">import json+
+
# Map tool names to their Python implementations.+
TOOL_HANDLERS = {+
"get_owner": get_owner,+
# Register your other initial tools here.+
}+
+
def execute_function(name, arguments):+
"""Run the requested tool and return its result."""+
handler = TOOL_HANDLERS.get(name)+
if handler is None:+
return {"error": f"Unknown tool: {name}"}+
return handler(**arguments)+
+
while True:+
# Preserve all output items, including encrypted reasoning.+
history.extend(resp.output)+
calls = [x for x in resp.output if x.type == "function_call"]+
if not calls:+
break+
for call in calls:+
result = execute_function(call.name, json.loads(call.arguments))+
history.append({+
"type": "function_call_output",+
"call_id": call.call_id,+
"output": json.dumps(result),+
})+
# The last result in this group supplies the automatic endpoint.+
resp = ask_with_implicit_cache(history)</code></pre> +
</div> +
<p>For follow-ups, append a new user message and call the helper again. Keep earlier messages unchanged so their cached endpoints remain available.</p> +
<p><strong>Rolling breakpoints: Each turn reads everything before it</strong></p> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/23/ML-22246-3-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/23/ML-22246-3-1.png" alt="Rolling breakpoints…+
<p class="wp-caption-text">Figure 3: Rolling breakpoints, where each turn reads everything before it and writes only its own tool result</p>+
</div> +
<p>For a separate explicit-only run, recreate the two-message history from the combined Hello cached world example. This loop wraps tool results in <code>input_text</code> blocks and marks the final result in each batch. Earlier markers stay in <code>history</code>. See…+
<div class="hide-language"> +
<pre><code class="language-python">import json+
+
while True:+
resp = client.responses.create(+
model="us.openai.gpt-6-astra", store=False,+
include=["reasoning.encrypted_content"],+
tools=TOOLS, prompt_cache_key=KEY, input=history,+
prompt_cache_options={"mode": "explicit", "ttl": "30m"},+
)+
d = resp.usage.input_tokens_details+
print("read", d.cached_tokens, "write", d.cache_write_tokens)+
history.extend(resp.output) # Includes encrypted reasoning.+
calls = [x for x in resp.output if x.type == "function_call"]+
if not calls:+
break+
batch = []+
for call in calls:+
result = execute_function(call.name, json.loads(call.arguments))+
batch.append({+
"type": "function_call_output", "call_id": call.call_id,+
"output": [{"type": "input_text", "text": json.dumps(result)}],Diff display stops at 400 lines. The line counts above are from the whole diff. 55 lines shown here cut at 300 characters. The raw artifact at this commit is linked above.