Change
1fa3259
1fa325975d9ac21eff9e57acc046a4bfc4f11e79 · commit on GitHub
aws-blog-feed: changed (734357 bytes, HTTP 200)
raw/aws-blog-feed/response.xml modified
- Source
- aws-blog-feed
- Lines added
- +5,282
- Lines removed
- -4,249
- Stored bytes at this commit
- 734,357
- Timestamp
- observed
- Raw artifact at this commit
- raw/aws-blog-feed/response.xml
Recorded headers
| observed_at | 2026-10-21T04:47:20.607Z |
|---|---|
| origin_date | null |
| status | 200 |
| final URL | https://aws.amazon.com/blogs/machine-learning/feed/ |
| etag | null |
| last-modified | Wed, 21 Oct 2026 01:22:10 GMT |
| date | Wed, 21 Oct 2026 04:47:20 GMT |
| age | null |
| cache-control | null |
| cf-cache-status | null |
| content-encoding | null |
| content-length | null |
@
@@ -5,7 +5,7 @@ <atom:link href="https://aws.amazon.com/blogs/machine-learning/feed/" rel="self" type="application/rss+xml"/> <link>https://aws.amazon.com/blogs/machine-learning/</link> <description>Official Machine Learning Blog of Amazon Web Services</description>-
<lastBuildDate>Mon, 19 Oct 2026 23:32:19 +0000</lastBuildDate>+
<lastBuildDate>Tue, 20 Oct 2026 22:35:31 +0000</lastBuildDate> <language>en-US</language> <sy:updatePeriod> hourly </sy:updatePeriod>@
@@ -13,4515 +13,5548 @@ 1 </sy:updateFrequency> <item>-
<title>Structured outputs for Anthropic Claude models on Amazon Bedrock</title>-
<link>https://aws.amazon.com/blogs/machine-learning/structured-outputs-for-anthropic-claude-models-on-amazon-bedrock/</link>+
<title>Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch</title>+
<link>https://aws.amazon.com/blogs/machine-learning/beyond-the-kv-cache-training-and-serving-i14s-laplace-3b-linear-attention-model-on-aws-trainium2-with-native-pytorch/</link> -
<dc:creator><![CDATA[Anshika Saxena]]></dc:creator>-
<pubDate>Mon, 19 Oct 2026 23:32:19 +0000</pubDate>-
<category><![CDATA[Amazon Bedrock]]></category>-
<category><![CDATA[Announcements]]></category>-
<category><![CDATA[Intermediate (200)]]></category>-
<guid isPermaLink="false">a1099e0a38a56e4f447d2dfcf2f10f2a8cdb2b50</guid>-
-
<description>Amazon Bedrock now supports structured outputs for all Anthropic Claude models on both the bedrock-runtime and bedrock-mantle endpoints. Receive schema-compliant JSON that conforms to a defined structure, without custom parsing, retry logic, or post-processing, using JSON Schema ou…-
<content:encoded><p>Amazon Bedrock now supports structured outputs for all Anthropic Claude models, on both the bedrock-runtime and bedrock-mantle endpoints, where the model is available.</p>-
<p>Whether you call Claude through Boto3 and the bedrock-runtime APIs or through the Messages API and the bedrock-mantle endpoint, you can now receive schema-compliant JSON responses that conform to a defined structure. You no longer need custom parsing, retry logic, or post-processing.</p&…-
<h2 id="practical-applications">Practical applications</h2>-
<p>With structured outputs for Claude on Amazon Bedrock, you can automate reliable, schema-compliant tasks across industries. Examples include extracting every required field from financial earnings reports and compliance documents, parsing clinical notes into Electronic Health Records (EHR), …-
<h2 id="what-changes-with-structured-outputs-on-amazon-bedrock">What changes with structured outputs on Amazon Bedrock</h2>-
<p>Structured outputs use constrained decoding to enforce your JSON schema during token generation, not after the fact. The model doesn’t produce tokens that would violate your schema. This changes output formatting from probabilistic to deterministic.</p>-
<p>Two complementary mechanisms are now available on both Amazon Bedrock endpoints:</p>-
<table border="1px" width="100%" cellpadding="10px">-
<tbody>-
<tr>-
<td><strong>Feature</strong></td>-
<td><strong>Purpose</strong></td>-
<td><strong>Use case</strong></td>-
</tr>-
<tr>-
<td>JSON Schema output format</td>-
<td>Control the model’s response structure</td>-
<td>Data extraction, report generation, API responses</td>-
</tr>-
<tr>-
<td>Strict tool use</td>-
<td>Validate tool parameters against schemas</td>-
<td>Agentic workflows, function calling, multi-step automation</td>-
</tr>-
</tbody>-
</table>-
<h2 id="what-this-means-for-you">What this means for you</h2>-
<p>Each response is a valid JSON, so you no longer encounter <code>JSON.parse()</code> errors or parsing exceptions. Responses are also type safe: field types are enforced, and required fields are always present. Because outputs always conform to your schema, you don’t need retries…-
<h2 id="supported-models-and-availability">Supported models and availability</h2>-
<p>Structured outputs works with <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html" target="_blank" rel="noopener">batch inference</a>, so you can process large volumes of requests asynchronously while still receiving schema-compliant outputs. On th…-
<h2 id="how-it-works">How it works</h2>-
<p>When you make a request with a JSON schema, here’s what happens behind the scenes:</p>-
<ol type="1">-
<li><strong>Schema validation –</strong>&nbsp;Amazon Bedrock validates your JSON schema against the supported JSON Schema Draft 2020-12 subset.</li>-
<li><strong>Grammar compilation –</strong>&nbsp;For new schemas, Amazon Bedrock compiles a grammar artifact (the first request might take slightly longer).</li>-
<li><strong>Caching –</strong>&nbsp;Compiled grammars are cached for 24 hours, making subsequent requests faster.</li>-
<li><strong>Constrained generation –</strong>&nbsp;The model generates tokens that produce valid JSON matching your schema.</li>-
</ol>-
<h2 id="performance-considerations">Performance considerations</h2>-
<p>For performance, the first request with a new schema might take longer because Amazon Bedrock compiles a grammar artifact before generation begins. After that initial compilation, subsequent requests using the same schema run with minimal overhead because the compiled grammar is cached. Thi…-
<h2 id="getting-started-structured-outputs-with-the-invokemodel-api-bedrock-runtime">Getting started: Structured outputs with the InvokeModel API (bedrock-runtime)</h2>-
<p>The InvokeModel API takes the Anthropic-native request body, so the schema goes in <code>output_config.format</code> as a JSON object rather than a JSON string. The following example shows how to extract the structured lead data from a customer email using boto3 with an Anthropi…-
<p>Python</p>-
<div class="hide-language">-
<pre><code class="language-python">import boto3-
import json-
-
REGION = "us-east-1" # Choose your preferred region-
bedrock_runtime = boto3.client(service_name="bedrock-runtime", region_name=REGION)-
-
# Define the JSON schema for structured output-
schema = {-
"type": "object",-
"properties": {-
"name": {"type": "string", "description": "Customer name"},-
"email": {"type": "string", "description": "Customer email address"},-
"plan_interest": {"type": "string", "description": "Product plan of interest"},-
"demo_requested": {"type": "boolean", "description": "Whether a demo was requested"}-
},-
"required": ["name", "email", "plan_interest", "demo_requested"],-
"additionalProperties": False-
}-
-
# Make a request with structured outputs via the InvokeModel API-
# (reusing the schema defined for the Converse example)-
response = bedrock_runtime.invoke_model(-
modelId="global.anthropic.claude-sonnet-5-5",-
body=json.dumps({-
"anthropic_version": "bedrock-2023-05-31",-
"max_tokens": 512,-
"messages": [-
{"role": "user", "content": (-
"Paulo Santos ([email redacted]) is interested in our Enterprise plan "-
"and wants to schedule a demo for next Tuesday at 2pm."-
)}-
],-
"output_config": {"format": {"type": "json_schema", "schema": schema}}-
})-
)-
-
body = json.loads(response["body"].read())+
<dc:creator><![CDATA[Kevin Shabahang]]></dc:creator>+
<pubDate>Tue, 20 Oct 2026 21:14:10 +0000</pubDate>+
<category><![CDATA[Advanced (300)]]></category>+
<category><![CDATA[AWS Trainium]]></category>+
<category><![CDATA[Customer Solutions]]></category>+
<guid isPermaLink="false">733fa955e4db373fd2ec88938f4d9a6aa8ad0a49</guid>-
# Check the stop reason before parsing-
if body["stop_reason"] == "end_turn":-
result_text = next(b["text"] for b in body["content"] if b["type"] == "text")-
print(json.dumps(json.loads(result_text), indent=2))-
elif body["stop_reason"] == "max_tokens":-
print("Response truncated: increase max_tokens and retry.")</code></pre>+
<description>Learn how i14 trained and served Laplace 3B, a linear attention model, on AWS Trainium2 using native PyTorch. Its recurrent state stays fixed at 29.5 MB per sequence regardless of context length, avoiding the growing key-value cache of transformer models and keeping serving memory …+
<content:encoded><p>Most large language models (LLMs) use transformer architectures with softmax attention. During inference, their key-value (KV) cache grows with the conversation or document context. This can reduce concurrency, increase serving cost, and force agentic applications…+
<p><a href="https://i14.ai" target="_blank" rel="noopener">i14</a> designed Laplace 3B as a linear attention alternative. Its recurrent state stays at 29.5 MB per sequence, independent of context length. Working with AWS, i14 used <a href="https://awsdocs-neuron.readthedocs-host…+
<p>On one trn2.48xlarge instance, Laplace 3B trained at about 120,000 tokens per second. The model served 11,544 output tokens per second across 1,280 concurrent sequences. This post explains the model design, the changes i14 made for Trainium2, and the training and inference results.</p>…+
<h2 id="about-i14">About i14</h2> +
<p>i14 is an AI research and technology company that develops model architectures, components, and runtimes. Its work draws on AI, computational cognitive science, and research into how biological systems learn over time. i14 applies these ideas to models that can retain long-range context wit…+
<h2 id="why-linear-attention-models">Why linear attention models?</h2> +
<p>Transformer softmax attention compares tokens across the context. During training, this computation grows quadratically with sequence length. During inference, the KV cache grows as the model processes more tokens.</p> +
<p>Linear attention reorganizes the computation around a recurrent state that is updated for each token. Laplace 3B uses GLA to compress prior context into a fixed-size matrix state. This changes training attention from O(<em>T</em>2) to O(<em>T</em>) and keeps inferenc…+
<table border="1px" width="100%" cellpadding="10px"> +
<tbody> +
<tr> +
<td><strong>Consideration</strong></td> +
<td><strong>Transformer softmax attention</strong></td> +
<td><strong>Laplace 3B with GLA</strong></td> +
</tr> +
<tr> +
<td>Training attention</td> +
<td>Builds pairwise token scores with O(<em>T</em>2) scaling</td> +
<td>Updates chunked recurrent state with O(<em>T</em>) scaling</td> +
</tr> +
<tr> +
<td>Inference memory</td> +
<td>KV cache grows with context</td> +
<td>Recurrent state remains fixed in size</td> +
</tr> +
<tr> +
<td>Longer contexts</td> +
<td>Each sequence consumes more accelerator memory</td> +
<td>Context length does not increase recurrent-state allocation</td> +
</tr> +
<tr> +
<td>Implementation</td> +
<td>Uses widely optimized attention kernels</td> +
<td>Requires architecture-specific kernels and compiler tuning</td> +
</tr> +
</tbody> +
</table> +
<p>Linear attention compresses history rather than retaining every key and value. You must therefore evaluate retrieval quality for your workloads. The fixed state can benefit long-running agents, large-document analysis, and applications that need persistent memory with predictable serving re…+
<h2 id="how-i14-adapted-laplace-3b-for-aws-trainium2">How i14 adapted Laplace 3B for AWS Trainium2</h2> +
<p>The original implementation of Laplace 3B used GPU-optimized Triton kernels. The AWS Generative AI Innovation Center (GenAIIC) team first rewrote the GLA kernel with standard PyTorch operators for Trainium2. After comparing both approaches, the team chose the beta version of <a href="htt…+
<p>The implementation required three changes:</p> +
<ol type="1"> +
<li><strong>Using native PyTorch for the GLA attention kernel.</strong> The team replaced the GPU-specific fused_chunk_simple_gla Triton kernel with standard PyTorch operators, using 16 chunks of 128 tokens to match the 128 partitions of the NeuronCore on-chip buffer.</li> +
<li><strong>Redesigning how Muon communicates.</strong> Two all_to_all collectives in MuonFSDP2 cut per-rank traffic by 24 times and raised throughput by 1.63 times.</li> +
<li><strong>Compiling the sequential backward pass.</strong> The team statically unrolled the 16 chunk steps of the GLA recurrence to fit the graph-size and on-chip memory limits of the compiler.</li> +
</ol> +
<p>The engineering challenges section describes each change in detail.</p> +
<h2 id="why-aws-trainium-accelerators">Why AWS Trainium accelerators?</h2> +
<p><a href="https://aws.amazon.com/ai/machine-learning/trainium/" target="_blank" rel="noopener">AWS Trainium</a> is a family of AWS machine learning (ML) accelerators designed for model training and inference. A trn2.48xlarge instance combines 16 Trainium2 chips, 64 logical Neuron…+
<p>Three properties of Trainium2 shaped the implementation: software-managed on-chip memory (Challenge 1), a fast chip-to-chip interconnect (Challenge 2), and a static compiler (Challenge 3). The compiler traces the GLA recurrence as a fixed graph.</p> +
<p><strong>Software-managed on-chip memory.</strong> Each NeuronCore has a 28 MiB State Buffer (SBUF) with 128 partitions and a 2 MiB partial-sum buffer (PSUM). The PSUM is wired to a 128×128 systolic Tensor Engine, alongside separate vector and scalar units. The SBUF is not a cach…+
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/14/ML-21817-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/14/ML-21817-1.png" alt="NeuronCore-v3 internal …+
<p class="wp-caption-text">Figure 1: NeuronCore-v3 internal architecture. Software-managed SBUF (28 MiB, 128 partitions), Tensor Engine (128×128 systolic array), Vector Engine, Scalar Engine, and GPSIMD cores.</p>+
</div> +
<h2 id="i14-and-the-laplace-3b-architecture">i14 and the Laplace 3B architecture</h2> +
<p>Laplace 3B is the 24-layer i14 decoder, where every layer replaces softmax attention with Simple GLA. Figure 2 follows one layer from the input <span class="math inline"><strong>x</strong><sub><em>t</em></sub></span> at the top to the output &…+
<p><strong>Normalization and convolution.</strong> Each token’s residual-stream vector <span class="math inline"><strong>x</strong><sub><em>t</em></sub> ∈ ℝ<sup><em>D</em></sup></span> (<span class="math inl…+
<p><strong>QKVG projection.</strong> The post-conv output goes through one fused matrix <span class="math inline"><strong>W</strong><sub><em>q</em><em>k</em><em>v</em><em>g</em></sub> ∈ ℝ<sup>9, 216…+
<div style="width: 510px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/14/ML-21817-2.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/14/ML-21817-2.png" alt="Laplace decoder layer a…+
<p class="wp-caption-text">Figure 2: Laplace decoder layer architecture. Green = fixed (no params), gray = learned, yellow = recurrent state, blue = projection.</p>+
</div> +
<p><strong>GLA mixer.</strong> Each head <span class="math inline"><em>h</em></span> keeps a matrix state <span class="math inline"><strong>S</strong><sub><em>t</em></sub><sup>(<em>h</em>)</sup&g…+
<div style="width: 250px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/01-gla-recurrent-state-update.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/01-gla-recurrent-state-up…+
<p class="wp-caption-text">GLA recurrent-state update equation</p></div>-
<p>JSON</p>-
<div class="hide-language">-
<pre><code class="language-json">{-
"name": "Paulo Santos",-
"email": "[email redacted]",-
"plan_interest": "Enterprise",-
"demo_requested": true-
}</code></pre>+
<br> Here +
<span class="math inline"><em>λ</em><sub><em>h</em></sub></span> is the fixed per-head decay, and +
<span class="math inline"><strong>k</strong><sub><em>t</em></sub><strong>v</strong><sub><em>t</em></sub><sup>⊤</sup></span> is the rank-1 outer product that writes the current key-value pair into the stat…+
<br> +
<div style="width: 195px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/02-gla-state-readout.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/02-gla-state-readout.png" alt="GLA…+
<p class="wp-caption-text">GLA state-readout equation</p></div>-
<p>The response conforms to your schema. No additional validation is required.</p>-
<p>For additional API usage examples, including the Messages API, see the accompanying <a href="https://github.com/aws-samples/anthropic-on-aws/blob/main/notebooks/structured_outputs/structured-outputs-on-bedrock.ipynb" target="_blank" rel="noopener">sample notebook</a>.</p>-
<h2 id="strict-tool-use-for-agentic-workflows">Strict tool use for agentic workflows</h2>-
<p><a href="https://docs.aws.amazon.com/bedrock/latest/userguide/endpoints.html" target="_blank" rel="noopener">Amazon Bedrock Mantle</a> is designed for agentic applications with server-side tools, background inference, and stateful conversations. With strict tool use on the bedro…-
<div class="hide-language">-
<pre><code class="language-python">Import json-
from anthropic import AnthropicBedrockMantle-
-
# Reusing the client created in the Messages API example-
client = AnthropicBedrockMantle(aws_region="us-east-1")-
-
message = client.messages.create(-
model="anthropic.claude-sonnet-5",-
max_tokens=512,-
messages=[{"role": "user", "content": "What's the weather like in San Francisco?"}],-
tools=[-
{-
"name": "get_weather",-
"description": "Get the current weather for a specified location",-
"strict": True,-
"input_schema": {-
"type": "object",-
"properties": {-
"location": {-
"type": "string",-
"description": "The city and state, e.g., San Francisco, CA"-
},-
"unit": {-
"type": "string",-
"enum": ["celsius", "fahrenheit"],-
"description": "Temperature unit"-
}-
},-
"required": ["location", "unit"],-
"additionalProperties": False-
}-
}-
]-
)-
-
#Tool inputs conform to the schema-
for block in message.content:-
if block.type == "tool_use":-
print(f"Tool: {block.name}")-
print(f"Input: {json.dumps(block.input, indent=2)}")</code></pre>+
<br> Next, a self-augmentation step adds a gated copy of the value to each head’s output:+
<br> +
<div style="width: 215px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/03-gla-self-augmentation.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/03-gla-self-augmentation.png" …+
<p class="wp-caption-text">GLA self-augmentation equation</p></div>-
<p>Setting <code>strict: true</code> constrains the model’s tool input to match your schema exactly. The location field is always a string, and the unit field is always either celsius or fahrenheit. The model can’t add fields outside your schema, so the tool input contains only wha…-
<p>This is especially valuable for agentic use cases on Amazon Bedrock Mantle, where models orchestrate multi-step workflows with server-side tools like web search and code interpreter.</p>-
<h2 id="top-use-cases-for-claude-structured-outputs">Top use cases for Claude structured outputs</h2>-
<p>Structured outputs reduce error-handling complexity the most in three common patterns:</p>-
<ul>-
<li><strong>Data extraction:</strong> Extract fields from invoices, contracts, email, and transcripts into JSON that always matches your schema, ready to load into databases or customer relationship management (CRM) systems.</li>-
<li><strong>Reliable agent tool calls:</strong> Validate tool arguments against their schemas at generation time, so multi-step agent workflows run without failures from malformed input.</li>-
<li><strong>Classification at scale:</strong> Classify tickets, leads, or content by intent, priority, and sentiment for routing, moderation, and evaluation.</li>-
</ul>-
<p>Claude Haiku 5.5 reliably handles quick and repetitive workloads such as summaries, compactions, database queries, and classification requests. The following example shows how to classify and route incoming data using the Invoke API and Haiku 5.5:</p>-
<div class="hide-language">-
<pre><code class="language-python">import boto3-
import json-
-
REGION = "us-east-1" # Choose your preferred region-
bedrock_runtime = boto3.client(service_name="bedrock-runtime", region_name=REGION)-
-
# Classification schema: enums restrict output to a fixed set of labels-
schema = {-
"type": "object",-
"properties": {-
"category": {-
"type": "string",-
"enum": ["billing", "technical", "account", "sales", "other"],-
"description": "Primary ticket category"-
},-
"priority": {-
"type": "string",-
"enum": ["low", "medium", "high", "urgent"],-
"description": "Urgency based on business impact"-
},-
"sentiment": {-
"type": "string",-
"enum": ["positive", "neutral", "negative"],-
"description": "Customer sentiment"-
},-
"escalate": {"type": "boolean", "description": "Whether a human should review immediately"},-
"reason": {"type": "string", "description": "One-sentence justification for the classification"}-
},-
"required": ["category", "priority", "sentiment", "escalate", "reason"],-
"additionalProperties": False-
}-
-
ticket = (-
"Our production API calls have been failing with 500 errors since this morning. "-
"We're losing orders every minute. This is the second outage this month!"-
)-
-
# Make a request with structured outputs via the InvokeModel API-
response = bedrock_runtime.invoke_model(-
modelId="global.anthropic.claude-haiku-5-5",-
body=json.dumps({-
"anthropic_version": "bedrock-2023-05-31",-
"max_tokens": 512,-
"system": "You are a support ticket classifier. Classify the ticket accurately.",-
"messages": [-
{"role": "user", "content": f"Classify this support ticket:\n\n{ticket}"}-
],-
"output_config": {"format": {"type": "json_schema", "schema": schema}}-
})-
)-
-
body = json.loads(response["body"].read())-
-
# Check the stop reason before parsing-
if body["stop_reason"] == "end_turn":-
result_text = next(b["text"] for b in body["content"] if b["type"] == "text")-
result = json.loads(result_text)-
print(json.dumps(result, indent=2))-
# Route the ticket based on the guaranteed-valid labels-
if result["escalate"]:-
print(f"Escalating to on-call: {result['category']} / {result['priority']}")-
elif body["stop_reason"] == "max_tokens":-
print("Response truncated: increase max_tokens and retry.")</code></pre>+
<br> Here +
<span class="math inline"><em>σ</em></span> is the sigmoid and +
<span class="math inline">⊙</span> is the element-wise product. The gate is the query dotted with a learned per-channel rescaling of the key, where +
<span class="math inline"><strong>a</strong></span> is the learned scale (aug_balance). At initialization, +
<span class="math inline"><strong>a</strong> = 0</span>, which gives a half-strength skip of the value. The per-head outputs pass through a Head RMSNorm within each decay tier, are concatenated, gated by +
<span class="math inline"><em>S</em><em>i</em><em>L</em><em>U</em>(<strong>g</strong><sub><em>t</em></sub>)</span>, and projected by +
<span class="math inline"><strong>W</strong><sub><em>o</em><em>u</em><em>t</em></sub> ∈ ℝ<sup><em>D</em> × <em>D</em></sup></span> back to the residual stream.+
<p></p> +
<p><strong>Decay schedule.</strong> The per-head decay rate <span class="math inline"><em>λ</em><sub><em>h</em></sub></span> controls how fast a head forgets past tokens. It is fixed at initialization (not learned) and set by a geomet…+
<div style="width: 210px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/04-gla-decay-schedule.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/04-gla-decay-schedule.png" alt="G…+
<p class="wp-caption-text">GLA decay schedule equations</p></div>-
<h2 id="requirements-and-best-practices">Requirements and best practices</h2>-
<p>To use structured outputs effectively on Amazon Bedrock, follow these guidelines:</p>-
<ol type="1">-
<li><strong>Set additionalProperties: false on all objects</strong>: This is required for structured outputs to work. Without it, your schema won’t be accepted.</li>-
<li><strong>Use descriptive field names and descriptions</strong>: Models use property names and descriptions to understand what data to extract. Clear names like <code>customer_email</code> outperform generic names like <code>field1</code>.</li>-
<li><strong>Use enum for constrained values</strong>: When a field has a limited set of valid values, use <code>enum</code> to constrain options.</li>-
<li><strong>Start basic, then add complexity</strong>: Begin with the minimum required fields and add complexity incrementally. Basic schemas compile faster, are easier to debug, and keep your request payload lean because the schema counts toward the Converse API payload size.<…-
<li><strong>Reuse schemas to benefit from caching</strong>: Structure your application to reuse schemas across requests. The 24-hour grammar cache improves performance for repeated queries.</li>-
<li><strong>Check the stop reason in every response</strong>: Two scenarios can produce non-conforming responses: refusals (when the model declines for safety reasons) and token limits (when <code>max_tokens</code> is reached before completing). Handle both cases in yo…-
<li><strong>Test with realistic data before deployment</strong>: Validate your schemas against production-representative inputs. Edge cases in real data often reveal schema design issues.</li>-
</ol>-
<p>For additional best practices and schema design recommendations, see <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html" target="_blank" rel="noopener">Structured outputs</a> in the Amazon Bedrock User Guide.</p>-
<h2 id="supported-json-schema-features">Supported JSON Schema features</h2>-
<ol type="1">-
<li>All basic types: <code>object</code>, <code>array</code>, <code>string</code>, <code>integer</code>, <code>number</code>, <code>boolean</code>, <code>null</code>.</li>-
<li><code>enum</code> (strings, numbers, booleans, or nulls only).</li>-
<li><code>const</code>, <code>anyOf</code>, <code>allOf</code> (with limitations).</li>-
<li><code>$ref</code>, <code>$def</code>, and <code>definitions</code> (internal references only).</li>-
<li>String formats: <code>date-time</code>, <code>time</code>, <code>date</code>, <code>duration</code>, <code>email</code>, <code>hostname</code>, <code>uri</code>, <code>ipv4</code>, <code>…-
<li>Array <code>minItems</code> (only values 0 and 1).</li>-
</ol>-
<h3 id="not-supported">Not supported</h3>-
<p>These keywords are accepted by the API but are not enforced during generation. A request that includes them returns a successful response, and the model can still return values that violate them. Validate these constraints in your application rather than relying on the schema to enforce the…-
<ol type="1">-
<li>Recursive schemas.</li>-
<li>External <code>$ref</code> references.</li>-
<li>Numerical constraints (<code>minimum</code>, <code>maximum</code>, <code>multipleOf</code>).</li>-
<li>String constraints (<code>minLength</code>, <code>maxLength</code>).</li>-
<li><code>additionalProperties</code> set to anything other than false.</li>-
</ol>-
<h2 id="conclusion">Conclusion</h2>-
<p>Structured outputs for Anthropic Claude models on Amazon Bedrock provides the reliability that production AI applications require. It is available now on both the bedrock-runtime and bedrock-mantle endpoints. By combining JSON Schema output format and strict tool use, you can build data pip…-
<p>Structured outputs for Claude on Amazon Bedrock is now generally available. To learn more, see the <a href="https://docs.aws.amazon.com/bedrock/" target="_blank" rel="noopener">Amazon Bedrock documentation</a> and explore our <a href="https://github.com/aws-samples/anthropic-…-
<p>What workflows could validated, schema-compliant JSON improve in your organization? Share your feedback and questions in the comments.</p>-
<p style="clear: both"></p>-
<hr style="width: 100%">-
<h2>About the authors</h2>-
<footer>-
<div class="blog-author-box" style="padding-top: 2.0em">-
<div class="blog-author-image" style="margin-right: 1.0em">-
<img class="alignnone size-full" src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-22211-1.jpg" alt="Anshika Saxena" width="100" height="133">-
</div>-
<h3 class="lb-h4">Anshika Saxena</h3>-
<p style="overflow: hidden">Anshika is a Generative AI Data Scientist at AWS, focused on Amazon Bedrock. She helps enterprise customers accelerate their generative AI and agentic AI adoption by guiding them through building and implementing solutions on Amazon Bedrock. She also works with IS…-
</div>-
<div class="blog-author-box" style="padding-top: 2.0em">+
<br> +
<strong>Decay schedule’s reach.</strong> For this model, +
<span class="math inline"><em>T</em><sub>min</sub> = 1</span> and +
<span class="math inline"><em>T</em><sub>max</sub> = 2, 048</span>. Head 0 therefore has +
<span class="math inline"><em>λ</em><sub>0</sub> = 0.5</span> (half-life of one token), and head 15 has +
<span class="math inline"><em>λ</em><sub>15</sub> ≈ 0.9997</span> (half-life of 2,048 tokens). Heads are grouped into tiers, each sharing a Head RMSNorm.+
<p></p> +
<p><strong>Channel mixer (FFN).</strong> After the attention block and its residual, a second LayerNorm feeds a GLU (SiLU) FFN with inner dimension 8,192. Its three matrices are two up-projections <span class="math inline"><strong>W</strong><sub>1</sub>…+
<p>The state <span class="math inline"><strong>S</strong><sup>(<em>h</em>)</sup></span> is fixed at <span class="math inline"><em>d</em><sub><em>k</em></sub> × <em>d</em><sub><em>v…+
<table border="1px" width="100%" cellpadding="10px"> +
<tbody> Diff display stops at 400 lines. The line counts above are from the whole diff. 40 lines shown here cut at 300 characters. 2 email addresss masked in this display. See the About page on personal data. The raw artifact at this commit is linked above.