Change
5d2f620
5d2f620ebf4f1e53c1736e2c481fc455cb205a36 · commit on GitHub
aws-blog-feed: changed (752935 bytes, HTTP 200)
raw/aws-blog-feed/response.xml modified
- Source
- aws-blog-feed
- Lines added
- +5,249
- Lines removed
- -4,631
- Stored bytes at this commit
- 752,935
- Timestamp
- observed
- Raw artifact at this commit
- raw/aws-blog-feed/response.xml
Recorded headers
| observed_at | 2026-10-22T04:45:33.581Z |
|---|---|
| origin_date | null |
| status | 200 |
| final URL | https://aws.amazon.com/blogs/machine-learning/feed/ |
| etag | null |
| last-modified | Wed, 21 Oct 2026 23:51:25 GMT |
| date | Thu, 22 Oct 2026 04:45:33 GMT |
| age | null |
| cache-control | null |
| cf-cache-status | null |
| content-encoding | null |
| content-length | null |
@
@@ -5,7 +5,7 @@ <atom:link href="https://aws.amazon.com/blogs/machine-learning/feed/" rel="self" type="application/rss+xml"/> <link>https://aws.amazon.com/blogs/machine-learning/</link> <description>Official Machine Learning Blog of Amazon Web Services</description>-
<lastBuildDate>Tue, 20 Oct 2026 22:35:31 +0000</lastBuildDate>+
<lastBuildDate>Wed, 21 Oct 2026 23:38:38 +0000</lastBuildDate> <language>en-US</language> <sy:updatePeriod> hourly </sy:updatePeriod>@
@@ -13,276 +13,670 @@ 1 </sy:updateFrequency> <item>-
<title>Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch</title>-
<link>https://aws.amazon.com/blogs/machine-learning/beyond-the-kv-cache-training-and-serving-i14s-laplace-3b-linear-attention-model-on-aws-trainium2-with-native-pytorch/</link>+
<title>Securing Quick access from corporate networks with AWS PrivateLink</title>+
<link>https://aws.amazon.com/blogs/machine-learning/securing-quick-access-from-corporate-networks-with-aws-privatelink/</link> -
<dc:creator><![CDATA[Kevin Shabahang]]></dc:creator>-
<pubDate>Tue, 20 Oct 2026 21:14:10 +0000</pubDate>+
<dc:creator><![CDATA[Rahul Kulkarni]]></dc:creator>+
<pubDate>Wed, 21 Oct 2026 23:38:38 +0000</pubDate> <category><![CDATA[Advanced (300)]]></category>-
<category><![CDATA[AWS Trainium]]></category>-
<category><![CDATA[Customer Solutions]]></category>-
<guid isPermaLink="false">733fa955e4db373fd2ec88938f4d9a6aa8ad0a49</guid>+
<category><![CDATA[Amazon Quick Suite]]></category>+
<category><![CDATA[AWS PrivateLink]]></category>+
<category><![CDATA[Technical How-to]]></category>+
<guid isPermaLink="false">34e5ea1686b4ddadf422c6b8da002f20b620869c</guid>-
<description>Learn how i14 trained and served Laplace 3B, a linear attention model, on AWS Trainium2 using native PyTorch. Its recurrent state stays fixed at 29.5 MB per sequence regardless of context length, avoiding the growing key-value cache of transformer models and keeping serving memory …-
<content:encoded><p>Most large language models (LLMs) use transformer architectures with softmax attention. During inference, their key-value (KV) cache grows with the conversation or document context. This can reduce concurrency, increase serving cost, and force agentic applications…-
<p><a href="https://i14.ai" target="_blank" rel="noopener">i14</a> designed Laplace 3B as a linear attention alternative. Its recurrent state stays at 29.5 MB per sequence, independent of context length. Working with AWS, i14 used <a href="https://awsdocs-neuron.readthedocs-host…-
<p>On one trn2.48xlarge instance, Laplace 3B trained at about 120,000 tokens per second. The model served 11,544 output tokens per second across 1,280 concurrent sequences. This post explains the model design, the changes i14 made for Trainium2, and the training and inference results.</p>…-
<h2 id="about-i14">About i14</h2> -
<p>i14 is an AI research and technology company that develops model architectures, components, and runtimes. Its work draws on AI, computational cognitive science, and research into how biological systems learn over time. i14 applies these ideas to models that can retain long-range context wit…-
<h2 id="why-linear-attention-models">Why linear attention models?</h2> -
<p>Transformer softmax attention compares tokens across the context. During training, this computation grows quadratically with sequence length. During inference, the KV cache grows as the model processes more tokens.</p> -
<p>Linear attention reorganizes the computation around a recurrent state that is updated for each token. Laplace 3B uses GLA to compress prior context into a fixed-size matrix state. This changes training attention from O(<em>T</em>2) to O(<em>T</em>) and keeps inferenc…+
<description>Learn how to connect on-premises data sources to Amazon Quick privately with AWS PrivateLink, keeping analytics traffic off the public internet. This walkthrough covers VPC endpoints for user and API access, VPC connections for data sources, DNS and endpoint policies, identity fede…+
<content:encoded><p>When enterprise data sits behind a corporate firewall or in a private on-premises data center, there’s no direct network path from AWS, even though the AI agents and dashboards in Amazon Quick can query virtually any data source. Administrators must make sure thei…+
<p>For many enterprises, routing traffic over the public internet introduces unacceptable risk. Regulatory frameworks such as HIPAA, FedRAMP, and PCI-DSS often mandate that sensitive data remain within controlled network boundaries. Even where compliance isn’t the primary driver, exposing inte…+
<p>This walkthrough shows you how to connect on-premises data sources to Amazon Quick privately, without traffic traversing the public internet. It assumes you already have AWS Direct Connect or an AWS Site-to-Site VPN linking your on-premises environment to your virtual private cloud (VPC). F…+
<p>By the end, you will have a network architecture that keeps traffic between Quick and your on-premises data sources off the public internet, reducing the attack surface, providing consistent low-latency connectivity, and giving you a clear audit trail for compliance.</p> +
<h2 id="amazon-quick-and-establishing-private-network-connectivity">Amazon Quick and establishing private network connectivity</h2> +
<p>Amazon Quick is an AI-powered workspace that brings search, analytics, research, and automation into a single conversational experience. Understanding which components reach your private network, and how, determines the architecture in the sections that follow.</p> +
<p>Data access in Amazon Quick falls into two paths. <strong>User-facing traffic</strong> covers anyone opening the Quick console, interacting with a dashboard, making an API call, or using Quick Desktop. All this routes through VPC endpoints (see VPC endpoints for user and API acc…+
<p><strong>Data-fetching traffic</strong> is when Quick itself reaches into your network to pull data. This is handled by VPC connections (see VPC connections for data source access), and it’s where most of the architectural decisions lie. The features and integrations that initiat…+
<ul> +
<li><strong>Amazon Quick Sight</strong> queries on-premises databases (PostgreSQL, MySQL, SQL Server, Oracle, and others) directly through ENIs placed in your VPC. This is the heaviest consumer of VPC connections.</li> +
<li><strong>Amazon Quick Flows and Amazon Quick Automate</strong> are separate automation features. Flows handle repeatable tasks. Automate handles complex, multi-system orchestration with human-in-the-loop. Both invoke action connectors, including MCP servers hosted privately in …+
<li><strong>Quick Agents</strong> underpin all conversational interactions and can invoke action connectors directly. When a custom agent calls a privately hosted MCP server or internal API, traffic flows through the VPC connection.</li> +
<li><strong>Amazon Quick Index</strong> builds a unified search layer across connected sources such as Confluence, SharePoint, and Amazon Simple Storage Service (Amazon S3). For data inside your VPC or on premises, knowledge base VPC connections are currently supported only for Am…+
<li><strong>Amazon Quick Research</strong> accesses enterprise sources behind the firewall alongside public internet data for its citation-backed reports, using the VPC connection when reaching private sources.</li> +
<li><strong>Apps in Quick</strong> build interactive web applications on top of structured data and action connectors, inheriting the same VPC connectivity as Quick Sight and Flows.</li> +
<li><strong>Quick Spaces</strong> organize content from the preceding features but don’t initiate their own data connections.</li> +
</ul> +
<p>These two traffic paths require two distinct private connectivity mechanisms that work together:</p> +
<ul> +
<li><strong>VPC endpoints (AWS PrivateLink):</strong> Handle user-facing traffic such as accessing the Quick website, making API calls, and administrative access. These keep user traffic off the public internet.</li> +
<li><strong>VPC connections:</strong> Handle data source traffic. Amazon Quick reaches into your VPC (and by extension, your on-premises network) to fetch data from databases, Amazon Redshift knowledge base sources, and internal APIs. These are configured within the Quick console.…+
</ul> +
<h2 id="configuring-private-connectivity">Configuring private connectivity</h2> +
<p>This section walks through the four building blocks in order: VPC endpoints for user and API access, VPC connections for data source access, VPC endpoints for AWS data services, and outbound access to required external domains.</p> +
<h3 id="prerequisites">Prerequisites</h3> +
<p>Before configuring private connectivity for Quick, make sure the following are in place:</p> +
<ul> +
<li>An active AWS account with an Amazon Quick Enterprise subscription (VPC connections require Enterprise edition).</li> +
<li>An existing VPC in the AWS Region where Quick is deployed.</li> +
<li>A supported AWS Region. Quick website VPC endpoints are not available in the AWS China Regions or the AWS GovCloud (US) Regions.</li> +
<li>An AWS Direct Connect connection or Site-to-Site VPN established between your on-premises network and your VPC. This is the private path that carries traffic from your data center into AWS.</li> +
<li>AWS Identity and Access Management (IAM) permissions to create and manage VPC endpoints, security groups, and Quick administration. To create VPC connections, the administrator’s IAM identity also needs <code>quicksight:CreateVPCConnection</code>, <code>quicksight:Descri…+
<li>Access to your organization’s DNS management (Route 53 private hosted zones or on-premises DNS forwarders), as private DNS resolution routes traffic through VPC endpoints rather than public addresses.</li> +
</ul> +
<h3 id="vpc-endpoints-for-user-and-api-access-aws-privatelink">VPC endpoints for user and API access (AWS PrivateLink)</h3> +
<p>VPC endpoints powered by PrivateLink handle all user-facing traffic to Amazon Quick, including the website, API calls, and embedded content. They prevent users from needing to traverse the public internet to access the Amazon Quick console.</p> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-1-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-1-1.jpg" alt="Architecture diagra…+
<p class="wp-caption-text">Figure 1: Private access to Amazon Quick from a corporate data center through a VPC endpoint</p>+
</div> +
<p>As shown in Figure 1: (1) a client on the corporate network sends a request to Quick; (2) the request travels over AWS Direct Connect, through the Direct Connect gateway, into your VPC; (3) the Quick VPC endpoint evaluates the endpoint policy and allows or denies the request; (4) allowed re…+
<h4 id="planning-your-vpc-endpoint-architecture">Planning your VPC endpoint architecture</h4> +
<p>When planning your endpoint architecture, begin by identifying the AWS Region where your Quick account is deployed, as VPC endpoints must be in the same Region. Choose subnets across multiple Availability Zones for endpoint ENI placement, and make sure those subnets have routes to your on-p…+
<p>Design security groups that restrict traffic on the endpoint ENIs to only required ports and source ranges. If you want all traffic routed privately (including the calls Amazon Quick makes to Amazon S3, Amazon Redshift, and Amazon Athena), plan VPC endpoints for those services as well (see …+
<h4 id="creating-vpc-endpoints-for-quick">Creating VPC endpoints for Quick</h4> +
<p>Quick requires two interface VPC endpoints for full private access:</p> +
<ul> +
<li><strong>Website access endpoint</strong> – Service name <code>com.amazonaws.&lt;region&gt;.quicksight-website</code>. This handles the bulk of user traffic to the Quick console. Private DNS names for the Quick website differ from the public URL, so you must…+
<li><strong>API access endpoint</strong> – Service name <code>com.amazonaws.&lt;region&gt;.quicksight</code>. This handles programmatic API calls. When private DNS is enabled, existing API calls to <code>quicksight.&lt;region&gt;.amazonaws.com<…+
</ul> +
<blockquote>+
<p><strong>Note:</strong> Not every Quick API is available through the API VPC endpoint. The following 13 APIs are not supported: the action connector APIs (CreateActionConnector, DeleteActionConnector, DescribeActionConnector, DescribeActionConnectorPermissions, ListActionConnect…+
</blockquote> +
<p>If administrators sign in to Amazon Quick as IAM users through the VPC, also create the AWS Management Console endpoints:</p> +
<ul> +
<li><code>com.amazonaws.&lt;region&gt;.console</code>.</li> +
<li><code>com.amazonaws.&lt;region&gt;.signin</code>.</li> +
</ul> +
<p>To create the website endpoint in the Amazon Virtual Private Cloud (Amazon VPC) console:</p> +
<ol type="1"> +
<li>Open the Amazon VPC console. In the navigation pane, under <strong>PrivateLink and Lattice</strong>, choose <strong>Endpoints</strong>, then choose <strong>Create endpoint</strong>.</li> +
<li>Enter a <strong>Name tag</strong> (for example, my-quicksight-vpc-endpoint) and, for <strong>Type</strong>, choose <strong>AWS services</strong>.</li> +
<li>Under <strong>Services</strong>, search for <code>com.amazonaws.&lt;region&gt;.quicksight</code> and select <code>com.amazonaws.&lt;region&gt;.quicksight-website</code> (Figure 2).</li> +
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-2-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-2-1.jpg" alt="Amazon VPC console …+
<p class="wp-caption-text">Figure 2: Naming the endpoint and selecting the Quick website endpoint service</p>+
</div> +
<ol start="4" type="1"> +
<li>Under <strong>Network settings</strong>, select your VPC. Under <strong>Additional settings</strong>, keep <strong>Enable private DNS name</strong> selected.</li> +
<li>Select subnets in at least two Availability Zones, then select the security group for the endpoint ENIs (Figure 3).</li> +
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-3-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-3-1.jpg" alt="Amazon VPC console …+
<p class="wp-caption-text">Figure 3: Selecting the VPC, private DNS, subnets, and security group for the endpoint</p>+
</div> +
<ol start="6" type="1"> +
<li>Choose a <strong>Policy</strong> (see the following Setting up VPC endpoint policies section), add tags, and choose <strong>Create endpoint</strong> (Figure 4).</li> +
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-4-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-4-1.jpg" alt="Amazon VPC console …+
<p class="wp-caption-text">Figure 4: Choosing the endpoint policy and creating the endpoint</p>+
</div> +
<ol start="7" type="1"> +
<li>Repeat these steps for the API endpoint, <code>com.amazonaws.&lt;region&gt;.quicksight</code>.</li> +
</ol> +
<h4 id="configuring-dns-resolution">Configuring DNS resolution</h4> +
<p>For the website endpoint, create an A record for <code>&lt;region&gt;.quicksight.aws.amazon.com</code> and point it to the VPC endpoint DNS name. This way, when users on your corporate network resolve the Quick URL, traffic routes through the private endpoint rather than…+
<p>A Route 53 private hosted zone only answers queries from inside the VPCs associated with it. Because your users connect from the corporate network, configure your on-premises DNS servers to forward queries for <code>&lt;region&gt;.quicksight.aws.amazon.com</code> to a Ro…+
<p>To create the record in Route 53:</p> +
<ol type="1"> +
<li>In the Route 53 console, choose <strong>Hosted zones</strong>, then <strong>Create hosted zone</strong>.</li> +
<li>For <strong>Domain name</strong>, enter <code>&lt;region&gt;.quicksight.aws.amazon.com</code> (for example, <code>us-east-1.quicksight.aws.amazon.com</code>). Scoping the zone to your Region avoids overriding other Quick host names. Choose <s…+
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-5-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-5-1.jpg" alt="Amazon Route 53 con…+
<p class="wp-caption-text">Figure 5: Creating a private hosted zone in Amazon Route 53</p>+
</div> +
<ol start="3" type="1"> +
<li>In the new hosted zone, choose <strong>Create record</strong> (Figure 6).</li> +
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-6-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-6-1.jpg" alt="Amazon Route 53 con…+
<p class="wp-caption-text">Figure 6: Creating a record in the new private hosted zone</p>+
</div> +
<ol start="4" type="1"> +
<li>Leave <strong>Record name</strong> blank, choose record type <strong>A</strong>, turn on <strong>Alias</strong>, choose <strong>Alias to VPC endpoint</strong>, select your Region and the website endpoint’s DNS name, and choose <strong>…+
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-7-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-7-1.jpg" alt="Amazon Route 53 con…+
<p class="wp-caption-text">Figure 7: Creating an alias A record that points to the Quick website VPC endpoint</p>+
</div> +
<h4 id="setting-up-vpc-endpoint-policies">Setting up VPC endpoint policies</h4> +
<p>VPC endpoint policies restrict which accounts or organizations can use the endpoint. For Quick, policies can be scoped by AWS account ID or Organization ID. To change a policy, open the endpoint in the Amazon VPC console, choose the <strong>Policy</strong> tab, and choose <st…+
<blockquote>+
<p><strong>Important limitation:</strong> For the Quick website endpoint, the <code>Principal</code>, <code>Action</code>, and <code>Resource</code> fields must all be set to <code>*</code>. Conditions might only be specified against…+
</blockquote> +
<p><strong>Website endpoint policy example:</strong></p> +
<div class="hide-language"> +
<pre><code class="language-json">{+
"Version": "2012-10-17",+
"Statement": [+
{+
"Effect": "Allow",+
"Principal": "*",+
"Action": "*",+
"Resource": "*",+
"Condition": {+
"StringEquals": {+
"aws:PrincipalAccount": ["012345678901"]+
}+
}+
}+
]+
}</code></pre> +
</div> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-8-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-8-1.jpg" alt="Amazon VPC console …+
<p class="wp-caption-text">Figure 8: Opening the endpoint policy from the VPC endpoint details page</p>+
</div> +
<p><strong>API endpoint policy example (restricted to specific actions):</strong></p> +
<div class="hide-language"> +
<pre><code class="language-json">{+
"Version": "2012-10-17",+
"Statement": [+
{+
"Effect": "Allow",+
"Principal": "*",+
"Action": [+
"quicksight:DescribeUser",+
"quicksight:ListUsers"+
],+
"Resource": "*",+
"Condition": {+
"StringEquals": {+
"aws:PrincipalAccount": ["012345678901"]+
}+
}+
}+
]+
}</code></pre> +
</div> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-9-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-9-1.jpg" alt="Amazon VPC console …+
<p class="wp-caption-text">Figure 9: Editing a custom VPC endpoint policy</p>+
</div> +
<h3 id="vpc-connections-for-data-source-access">VPC connections for data source access</h3> +
<p>While VPC endpoints handle user-facing traffic, VPC connections handle the server-side path, meaning Amazon Quick reaches into your VPC to access databases, Amazon Redshift knowledge base sources, and internal APIs.</p> +
<h4 id="how-vpc-connections-work">How VPC connections work</h4> +
<p>Amazon Quick Enterprise edition is fully integrated with Amazon VPC. When you create a VPC connection in Amazon Quick, you are adding elastic network interfaces (ENIs) into your VPC subnets. These ENIs allow Quick to exchange network traffic with resources within your VPC and, by extension,…+
<p>All standard VPC security controls apply to this traffic. Route tables, network ACLs, subnets, and security groups govern traffic to and from Quick in the same way they govern traffic between any other instances in your VPC. This means you can connect to:</p> +
<ul> +
<li>Private databases not exposed to the public internet.</li> +
<li>On-premises data sources reachable through Direct Connect or VPN.</li> +
<li>Internal APIs accessible by IP address within your network.</li> +
</ul> +
<p>Create a dedicated security group (for example, <code>QuickSight-VPC</code>) for the VPC connection ENIs. On each data source’s security group, allow inbound traffic on the database port from that security group ID.</p> +
<blockquote>+
<p><strong>Note for VPC connections created before April 27, 2023:</strong> The security group attached to the Quick network interface is not stateful. You must add an inbound rule that explicitly allows return traffic from your data source’s security group on all ports (0–65535),…+
</blockquote> +
<h4 id="creating-the-execution-role">Creating the execution role</h4> +
<p>Amazon Quick doesn’t create the VPC connection execution role for you. Before you add the connection, create a dedicated IAM role (recommended), or update an existing role that already trusts Quick, such as <code>aws-quicksight-service-role-v0</code> used in the screenshots in t…+
<ul> +
<li>A permissions policy that grants the Amazon Elastic Compute Cloud (Amazon EC2) actions Quick uses to manage its ENIs (Figure 10).</li> +
<li>A trust policy that allows <code>quicksight.amazonaws.com</code> to assume the role. Only roles with this trust policy appear in the <strong>Execution role</strong> list in the Quick console.</li> +
</ul> +
<p><strong>Permissions policy:</strong></p> +
<div class="hide-language"> +
<pre><code class="language-json">{+
"Version": "2012-10-17",+
"Statement": [+
{+
"Effect": "Allow",+
"Action": [+
"ec2:CreateNetworkInterface",+
"ec2:DeleteNetworkInterface",+
"ec2:ModifyNetworkInterfaceAttribute",+
"ec2:DescribeSubnets",+
"ec2:DescribeSecurityGroups"+
],+
"Resource": "*"+
}+
]+
}</code></pre> +
</div> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-10-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-10-1.jpg" alt="IAM console addin…+
<p class="wp-caption-text">Figure 10: Adding the Amazon EC2 permissions required by the VPC connection execution role</p>+
</div> +
<p><strong>Trust policy:</strong></p> +
<div class="hide-language"> +
<pre><code class="language-json">{+
"Version": "2012-10-17",+
"Statement": [+
{+
"Effect": "Allow",+
"Principal": { "Service": "quicksight.amazonaws.com" },+
"Action": "sts:AssumeRole"+
}+
]+
}</code></pre> +
</div> +
<h4 id="configuring-a-vpc-connection">Configuring a VPC connection</h4> +
<p>To create a VPC connection in the Quick console:</p> +
<ol type="1"> +
<li>Sign in to Amazon Quick as an administrator. Choose your profile icon in the upper-right, then choose <strong>Manage account</strong> (Figure 11). If you don’t see this option, you’re not a Quick administrator.</li> +
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-11-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-11-1.jpg" alt="Amazon Quick prof…+
<p class="wp-caption-text">Figure 11: Opening Manage account from the Quick profile menu</p>+
</div> +
<ol start="2" type="1"> +
<li>In the navigation pane, under <strong>Security</strong>, choose <strong>Manage VPC connections</strong> (Figure 12), then choose <strong>Add VPC connection</strong>.</li> +
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-12-1.jpg" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-12-1.jpg" alt="Amazon Quick Mana…+
<p class="wp-caption-text">Figure 12: Manage VPC connections under Security on the Manage account page</p>+
</div> +
<ol start="3" type="1"> +
<li>Provide the following (Figure 13): +
<ol type="a"> +
<li>A descriptive <strong>VPC connection name</strong>.</li> +
<li>The <strong>VPC ID</strong> of the target VPC. This can’t be changed later.</li> +
<li>The <strong>Execution role</strong> you created in the previous step.</li> +
<li>At least two <strong>subnets</strong> in different Availability Zones.</li> +
<li>At least one <strong>security group</strong> ID.</li> +
<li>(Optional) If your database host name must be resolved through private DNS, enter the IP addresses of your Route 53 Resolver inbound endpoints under <strong>DNS resolver endpoints</strong>.</li> +
</ol> </li> +
<li>Choose <strong>ADD</strong>.</li> +
</ol> +
<div style="width: 810px" class="wp-caption alignnone">+
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-13-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/ML-20213-13-1.png" alt="Amazon Quick cons…+
<p class="wp-caption-text">Figure 13: Adding a VPC connection in the Quick console</p>+
</div> +
<p>The connection status shows <strong>UNAVAILABLE</strong> until Quick finishes provisioning the ENIs, then changes to <strong>AVAILABLE</strong>. A status of <strong>PARTIALLY AVAILABLE</strong> means one of the ENIs is unavailable, but the connection can …+
<h3 id="configuring-data-source-vpc-endpoints">Configuring data source VPC endpoints</h3> +
<p>VPC endpoints (PrivateLink, used here for AWS service APIs) and VPC connections (Quick ENIs placed in your VPC; see VPC connections for data source access) are different mechanisms. Quick accesses other AWS services (Amazon S3, Amazon Redshift, Amazon Athena, AWS Secrets Manager) when retri…<table border="1px" width="100%" cellpadding="10px"> <tbody> <tr> -
<td><strong>Consideration</strong></td> -
<td><strong>Transformer softmax attention</strong></td> -
<td><strong>Laplace 3B with GLA</strong></td> +
<td><strong>Service</strong></td> +
<td><strong>Endpoint type</strong></td> +
<td><strong>Service name</strong></td> </tr> <tr> -
<td>Training attention</td> -
<td>Builds pairwise token scores with O(<em>T</em>2) scaling</td> -
<td>Updates chunked recurrent state with O(<em>T</em>) scaling</td> +
<td>Amazon S3</td> +
<td>Gateway</td> +
<td><code>com.amazonaws.&lt;region&gt;.s3</code></td> </tr> <tr> -
<td>Inference memory</td> -
<td>KV cache grows with context</td> -
<td>Recurrent state remains fixed in size</td> +
<td>Amazon Redshift</td> +
<td>Interface</td> +
<td><code>com.amazonaws.&lt;region&gt;.redshift</code></td> </tr> <tr> -
<td>Longer contexts</td> -
<td>Each sequence consumes more accelerator memory</td> -
<td>Context length does not increase recurrent-state allocation</td> +
<td>Amazon Athena*</td> +
<td>Interface</td> +
<td><code>com.amazonaws.&lt;region&gt;.athena</code></td> </tr> <tr> -
<td>Implementation</td> -
<td>Uses widely optimized attention kernels</td> -
<td>Requires architecture-specific kernels and compiler tuning</td> +
<td>AWS Secrets Manager</td> +
<td>Interface</td> +
<td><code>com.amazonaws.&lt;region&gt;.secretsmanager</code></td> +
</tr> +
<tr> +
<td>AWS Glue</td> +
<td>Interface</td> +
<td><code>com.amazonaws.&lt;region&gt;.glue</code></td> </tr> </tbody> </table> -
<p>Linear attention compresses history rather than retaining every key and value. You must therefore evaluate retrieval quality for your workloads. The fixed state can benefit long-running agents, large-document analysis, and applications that need persistent memory with predictable serving re…-
<h2 id="how-i14-adapted-laplace-3b-for-aws-trainium2">How i14 adapted Laplace 3B for AWS Trainium2</h2> -
<p>The original implementation of Laplace 3B used GPU-optimized Triton kernels. The AWS Generative AI Innovation Center (GenAIIC) team first rewrote the GLA kernel with standard PyTorch operators for Trainium2. After comparing both approaches, the team chose the beta version of <a href="htt…-
<p>The implementation required three changes:</p> -
<ol type="1"> -
<li><strong>Using native PyTorch for the GLA attention kernel.</strong> The team replaced the GPU-specific fused_chunk_simple_gla Triton kernel with standard PyTorch operators, using 16 chunks of 128 tokens to match the 128 partitions of the NeuronCore on-chip buffer.</li> -
<li><strong>Redesigning how Muon communicates.</strong> Two all_to_all collectives in MuonFSDP2 cut per-rank traffic by 24 times and raised throughput by 1.63 times.</li> -
<li><strong>Compiling the sequential backward pass.</strong> The team statically unrolled the 16 chunk steps of the GLA recurrence to fit the graph-size and on-chip memory limits of the compiler.</li> -
</ol> -
<p>The engineering challenges section describes each change in detail.</p> -
<h2 id="why-aws-trainium-accelerators">Why AWS Trainium accelerators?</h2> -
<p><a href="https://aws.amazon.com/ai/machine-learning/trainium/" target="_blank" rel="noopener">AWS Trainium</a> is a family of AWS machine learning (ML) accelerators designed for model training and inference. A trn2.48xlarge instance combines 16 Trainium2 chips, 64 logical Neuron…-
<p>Three properties of Trainium2 shaped the implementation: software-managed on-chip memory (Challenge 1), a fast chip-to-chip interconnect (Challenge 2), and a static compiler (Challenge 3). The compiler traces the GLA recurrence as a fixed graph.</p> -
<p><strong>Software-managed on-chip memory.</strong> Each NeuronCore has a 28 MiB State Buffer (SBUF) with 128 partitions and a 2 MiB partial-sum buffer (PSUM). The PSUM is wired to a 128×128 systolic Tensor Engine, alongside separate vector and scalar units. The SBUF is not a cach…-
<div style="width: 810px" class="wp-caption alignnone">-
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/14/ML-21817-1.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/14/ML-21817-1.png" alt="NeuronCore-v3 internal …-
<p class="wp-caption-text">Figure 1: NeuronCore-v3 internal architecture. Software-managed SBUF (28 MiB, 128 partitions), Tensor Engine (128×128 systolic array), Vector Engine, Scalar Engine, and GPSIMD cores.</p>-
</div> -
<h2 id="i14-and-the-laplace-3b-architecture">i14 and the Laplace 3B architecture</h2> -
<p>Laplace 3B is the 24-layer i14 decoder, where every layer replaces softmax attention with Simple GLA. Figure 2 follows one layer from the input <span class="math inline"><strong>x</strong><sub><em>t</em></sub></span> at the top to the output &…-
<p><strong>Normalization and convolution.</strong> Each token’s residual-stream vector <span class="math inline"><strong>x</strong><sub><em>t</em></sub> ∈ ℝ<sup><em>D</em></sup></span> (<span class="math inl…-
<p><strong>QKVG projection.</strong> The post-conv output goes through one fused matrix <span class="math inline"><strong>W</strong><sub><em>q</em><em>k</em><em>v</em><em>g</em></sub> ∈ ℝ<sup>9, 216…-
<div style="width: 510px" class="wp-caption alignnone">-
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/14/ML-21817-2.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/14/ML-21817-2.png" alt="Laplace decoder layer a…-
<p class="wp-caption-text">Figure 2: Laplace decoder layer architecture. Green = fixed (no params), gray = learned, yellow = recurrent state, blue = projection.</p>-
</div> -
<p><strong>GLA mixer.</strong> Each head <span class="math inline"><em>h</em></span> keeps a matrix state <span class="math inline"><strong>S</strong><sub><em>t</em></sub><sup>(<em>h</em>)</sup&g…-
<div style="width: 250px" class="wp-caption alignnone">-
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/01-gla-recurrent-state-update.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/01-gla-recurrent-state-up…-
<p class="wp-caption-text">GLA recurrent-state update equation</p>-
</div>-
<br> Here -
<span class="math inline"><em>λ</em><sub><em>h</em></sub></span> is the fixed per-head decay, and -
<span class="math inline"><strong>k</strong><sub><em>t</em></sub><strong>v</strong><sub><em>t</em></sub><sup>⊤</sup></span> is the rank-1 outer product that writes the current key-value pair into the stat…-
<br> -
<div style="width: 195px" class="wp-caption alignnone">-
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/02-gla-state-readout.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/02-gla-state-readout.png" alt="GLA…-
<p class="wp-caption-text">GLA state-readout equation</p>-
</div>-
<br> Next, a self-augmentation step adds a gated copy of the value to each head’s output:-
<br> -
<div style="width: 215px" class="wp-caption alignnone">-
<a href="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/03-gla-self-augmentation.png" target="_blank" rel="noopener"><img src="https://d2908q01vomqb2.cloudfront.net/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59/2026/10/19/03-gla-self-augmentation.png" …-
<p class="wp-caption-text">GLA self-augmentation equation</p>-
</div>-
<br> Here -
<span class="math inline"><em>σ</em></span> is the sigmoid and -
<span class="math inline">⊙</span> is the element-wise product. The gate is the query dotted with a learned per-channel rescaling of the key, where -
<span class="math inline"><strong>a</strong></span> is the learned scale (aug_balance). At initialization, -
<span class="math inline"><strong>a</strong> = 0</span>, which gives a half-strength skip of the value. The per-head outputs pass through a Head RMSNorm within each decay tier, are concatenated, gated by -
<span class="math inline"><em>S</em><em>i</em><em>L</em><em>U</em>(<strong>g</strong><sub><em>t</em></sub>)</span>, and projected by -
<span class="math inline"><strong>W</strong><sub><em>o</em><em>u</em><em>t</em></sub> ∈ ℝ<sup><em>D</em> × <em>D</em></sup></span> back to the residual stream.-
<p></p> Diff display stops at 400 lines. The line counts above are from the whole diff. 60 lines shown here cut at 300 characters. The raw artifact at this commit is linked above.