Change
1a48fb3
1a48fb35cdda8dde12d5f2523c3bfbf9cb744884 · commit on GitHub
pytorch-blog-feed: changed (234767 bytes, HTTP 200)
raw/pytorch-blog-feed/response.xml modified
- Source
- pytorch-blog-feed
- Lines added
- +334
- Lines removed
- -217
- Stored bytes at this commit
- 234,767
- Timestamp
- origin
- Raw artifact at this commit
- raw/pytorch-blog-feed/response.xml
Recorded headers
| observed_at | 2026-10-16T04:32:14.205Z |
|---|---|
| origin_date | 2026-10-16T03:17:20.000Z |
| status | 200 |
| final URL | https://pytorch.org/blog/feed/ |
| etag | "39fe634f1ec1a84751ba3125e4eb3f65" |
| last-modified | Thu, 15 Oct 2026 21:23:18 GMT |
| date | Fri, 16 Oct 2026 04:32:14 GMT |
| age | 4494 |
| cache-control | public, max-age=60, s-maxage=43200, stale-while-revalidate=86400, stale-if-error=604800 |
| cf-cache-status | null |
| content-encoding | null |
| content-length | 234767 |
@
@@ -12,7 +12,7 @@ <atom:link href="https://pytorch.org/blog/feed/" rel="self" type="application/rss+xml" /> <link>https://pytorch.org/blog/</link> <description></description>-
<lastBuildDate>Tue, 13 Oct 2026 14:04:23 +0000</lastBuildDate>+
<lastBuildDate>Thu, 15 Oct 2026 19:18:47 +0000</lastBuildDate> <language>en-US</language> <sy:updatePeriod> hourly </sy:updatePeriod>@
@@ -28,6 +28,337 @@ <height>32</height></image> <item>+
<title>Muse Spark Is Now an Open-Weights Model</title>+
<link>https://pytorch.org/blog/muse-spark-open/</link>+
+
<dc:creator><![CDATA[PyTorch Team at Meta]]></dc:creator>+
<pubDate>Thu, 15 Oct 2026 17:55:52 +0000</pubDate>+
<category><![CDATA[Blog]]></category>+
<guid isPermaLink="false">https://pytorch.org/?p=188144</guid>+
+
<description><![CDATA[<p>Today, Meta is releasing the weights for Muse Spark Open, the most capable US open-weights model to date. It’s available today on Hugging Face under the Muse Model License. Muse...</p>+
<p>The post <a href="https://pytorch.org/blog/muse-spark-open/">Muse Spark Is Now an Open-Weights Model</a> appeared first on <a href="https://pytorch.org">PyTorch</a>.</p>+
]]></description>+
<content:encoded><![CDATA[<p><span style="font-weight: 400;">Today, Meta is releasing the weights for </span><a href="https://research.meta.ai/blog/muse-spark-open"><span style="font-weight: 400;">Muse Spark Open</span></a><span style="font-weight: 400;">, the most capable US open-weights …+
<p><span style="font-weight: 400;">Building on our </span><a href="https://research.meta.ai/blog/introducing-muse-code-and-muse-spark-1-2"><span style="font-weight: 400;">August release</span></a><span style="font-weight: 400;"> of Muse Spark, this open weights version has additional safety post-tra…+
<p><span style="font-weight: 400;">Meta loves open source, and as the creators of PyTorch, we’re committed to giving back to ensure the entire ecosystem fully supports Muse Spark Open. First, we’ve integrated </span><a href="https://pytorch.org/blog/helion/"><span style="font-weight: 400;">Helion</s…+
<h2><span style="font-weight: 400;">Muse Spark Open</span></h2>+
<p><span style="font-weight: 400;">Muse Spark Open is a coding-optimized open-weight model. Trained specifically for agentic coding workflows, it delivers higher first-attempt accuracy, reliable tool calling, and deep codebase understanding across the repos developers actually work in. You can read …+
<h3><span style="font-weight: 400;">Enhanced Coding Capabilities</span></h3>+
<p><span style="font-weight: 400;">Strong at code generation, debugging, and codebase understanding. Trained on real developer workflows — multi-file refactors, debugging sessions, and long-running tasks — so the model keeps up with how you actually work. 1M context window means it handles your enti…+
<h3><span style="font-weight: 400;">Multimodal Understanding</span></h3>+
<p><span style="font-weight: 400;">Muse Spark Open has audio and audio-video perception — alongside image and document understanding. As a reasoning model, it processes image, audio, and text inputs — natively and reliably.</span></p>+
<h3><span style="font-weight: 400;">Agentic</span></h3>+
<p><span style="font-weight: 400;">Muse Spark Open is built from the ground up as an agentic model. It maps out steps, acts, observes, and self-corrects over long horizons — not just single-shot tool calls. It’s trained to browse the web (including multimodal browsing), and use an agentic harness to…+
<h2><span style="font-weight: 400;">Why Helion kernels?</span></h2>+
<p><span style="font-weight: 400;">Helion is a PyTorch native kernel authoring language designed for writing high-performance kernels across heterogeneous hardware platforms. Following early adoption from the open-source ecosystem, Helion became a part of the </span><a href="https://pytorch.org/blog…+
</span></p>+
<p><span style="font-weight: 400;">In addition, we also launched Helion megakernels for Muse Spark Open in SGLang. Written entirely in Helion, these kernels fuse multiple operations into a single persistent kernel to improve low latency inference, and can likewise be autotuned for different shapes a…+
<p><span style="font-weight: 400;">Helion kernels can be further autotuned by users for broader hardware platforms and workloads – including different accelerator architectures (NVIDIA, AMD, Google TPU, etc.), quantization formats, tensor shapes, and parallelism strategies – without maintaining sepa…+
<h2><span style="font-weight: 400;">Post-training with TorchTitan</span></h2>+
<p><span style="font-weight: 400;">TorchTitan is a PyTorch native platform designed for rapid experimentation and large-scale training of generative AI models. Starting today, TorchTitan adds a model definition for Meta’s Muse Spark Open , so developers can tune it to their needs. </span></p>+
<p><span style="font-weight: 400;">TorchTitan features native support for multi-turn SFT and LoRA out-of-the-box, as well as reinforcement learning capabilities (TitanRL) that allow users to fine-tune Muse Spark Open.</span></p>+
<h2><span style="font-weight: 400;">Get Started</span></h2>+
<p><span style="font-weight: 400;">We’re sharing this work because we want everyone to have access to the best AI models. Muse Spark Open ships with custom, auto-tunable Helion GPU kernels in SGLang, delivering 1.03–3.95× kernel-level speedups on NVIDIA and AMD GPUs and a 3% end-to-end speedup on AM…+
<p><span style="font-weight: 400;">Start Building Today: <a href="https://huggingface.co/meta-models/Muse-Spark-open-816B-A42B">https://huggingface.co/meta-models/Muse-Spark-open-816B-A42B</a></span></p>+
<p><i><span style="font-weight: 400;">Muse Spark is the full size model from our model family. We previously released Muse Glimmer, a small dense model. Start using it today, with </span></i><a href="https://pytorch.org/blog/fast-ondevice-agentic-ai-with-executorch/"><i><span style="font-weight: 400…+
<p>The post <a href="https://pytorch.org/blog/muse-spark-open/">Muse Spark Is Now an Open-Weights Model</a> appeared first on <a href="https://pytorch.org">PyTorch</a>.</p>+
]]></content:encoded>+
+
+
+
</item>+
<item>+
<title>Bringing ExecuTorch to Arm Cortex-M CPUs</title>+
<link>https://pytorch.org/blog/bringing-executorch-to-arm-cortex-m-cpus/</link>+
+
<dc:creator><![CDATA[RJ Ascani (Meta), Adrian Lundell (Arm), Siddartha Pothapragada (Meta), Digant Desai (Meta), Zingo Andersen (Arm), Måns Nilsson (Arm), Erik Lundell (Arm)]]></dc:creator>+
<pubDate>Thu, 15 Oct 2026 14:40:35 +0000</pubDate>+
<category><![CDATA[Blog]]></category>+
<guid isPermaLink="false">https://pytorch.org/?p=188033</guid>+
+
<description><![CDATA[<p>TL;DR Deploying PyTorch models on microcontrollers has historically required converting to other framework formats, maintaining separate toolchains, and debugging cross-framework numerical mismatches. ExecuTorch eliminates this conversion step entirely. The Cortex-M...<…+
<p>The post <a href="https://pytorch.org/blog/bringing-executorch-to-arm-cortex-m-cpus/">Bringing ExecuTorch to Arm Cortex-M CPUs</a> appeared first on <a href="https://pytorch.org">PyTorch</a>.</p>+
]]></description>+
<content:encoded><![CDATA[<h2>TL;DR</h2>+
<ul>+
<li>Deploying PyTorch models on microcontrollers has historically required converting to other framework formats, maintaining separate toolchains, and debugging cross-framework numerical mismatches. <strong>ExecuTorch eliminates this conversion step entirely.</strong></li>+
<li>The Cortex-M backend enables PyTorch models to run directly on Arm Cortex-M CPUs with Arm CMSIS-NN acceleration. It supports <strong>all capability tiers</strong> (Scalar, DSP, and MVE/Helium) from a single workflow.</li>+
<li>We benchmark four of the five models from the <strong>MLPerf Tiny suite</strong> on the Alif Ensemble E8. Each model runs in 21.5 ms or less and uses under 90 KiB of RAM on a single Cortex-M55 core.</li>+
<li>For heterogeneous SoCs, the Cortex-M backend also provides the <strong>CPU execution path alongside Ethos-U NPU</strong> delegation.</li>+
<li><strong>Minimal binary footprint.</strong> The ExecuTorch core runtime is under 20 KiB of flash, and operator selective build links only the operators your model needs, so total code size scales with the model rather than the framework.</li>+
</ul>+
<h2>Background</h2>+
<p>Microcontrollers power billions of always-on devices: wearables, industrial sensors, voice interfaces, and smart home appliances. These devices operate under extreme constraints: often less than 1 MB of SRAM, single-digit megabytes of flash, and power budgets measured in milliwatts. Despite these…+
<p>Until now, deploying ML on Cortex-M meant leaving the PyTorch ecosystem. Developers had to convert models to alternative formats, maintain separate toolchains for training and deployment, and debug numerical mismatches introduced during conversion. For teams that train in PyTorch, which accounts …+
<p><a href="https://pytorch.org/blog/introducing-executorch-1-0/">ExecuTorch</a> was <a href="https://arxiv.org/abs/2605.08195">designed</a> to eliminate this gap. Arm MCU support has been available through the Ethos-U NPU, and the Cortex-M backend extends it to Cortex-M CPUs, so a model can run on …+
<h2>The Cortex-M Backend</h2>+
<p><!-- IMAGE PLACEHOLDER: Image 1 – Cortex-M backend diagram. Download from the Google Doc and re-upload via the WordPress Media Library. --></p>+
<p><a href="https://pytorch.org/wp-content/uploads/2026/10/cortex-m-lowering-flow.svg"><img decoding="async" class="alignnone size-full wp-image-188039" src="https://pytorch.org/wp-content/uploads/2026/10/cortex-m-lowering-flow.svg" alt="" /></a></p>+
<p>The ExecuTorch Cortex-M backend replaces standard PyTorch operators with <strong>CMSIS-NN-backed custom operators</strong> through ahead-of-time compiler passes. At inference time, these operators call directly into Arm’s <a href="https://arm-software.github.io/CMSIS-NN/latest/">CMSIS-NN li…+
<p>Key properties:</p>+
<ul>+
<li><strong>CMSIS-NN accelerated.</strong> Quantized operations (linear, conv, add, mul, softmax, activations, etc.) are replaced with optimized implementations that call CMSIS-NN functions and additional optimized Cortex-M implementations directly.</li>+
<li><strong>Target-aware compilation.</strong> The CPU variant (M0 through M85) and ISA tier are specified at compile time, so the correct CMSIS-NN backend is selected, eliminating runtime dispatch overhead.</li>+
<li><strong>Purpose-built quantizer.</strong> Produces INT8 quantized graphs matching the data layout requirements of CMSIS-NN kernels, using PyTorch’s PT2E quantization framework.</li>+
<li><strong>Single compiler toolchain.</strong> The Cortex-M lowering path operates directly on the ExecuTorch edge dialect, keeping the compilation pipeline simple and debuggable.</li>+
</ul>+
<h3>Cortex-M Capability Tiers</h3>+
<p>The backend targets the full Cortex-M spectrum:</p>+
<table>+
<thead>+
<tr>+
<th>Tier</th>+
<th>Cores</th>+
<th>Key Feature</th>+
<th>Use Case</th>+
</tr>+
</thead>+
<tbody>+
<tr>+
<td><strong>Scalar</strong></td>+
<td>M0, M0+, M3, M23</td>+
<td>Standard ARM instructions</td>+
<td>Ultra-low-cost, minimal silicon</td>+
</tr>+
<tr>+
<td><strong>DSP</strong></td>+
<td>M4, M7, M33, M35P</td>+
<td>SIMD multiply-accumulate</td>+
<td>Mid-range MCUs, signal processing</td>+
</tr>+
<tr>+
<td><strong>MVE (aka Helium)</strong></td>+
<td>M55, M85</td>+
<td>128-bit vector ops, beat-based execution</td>+
<td><strong>High-performance MCU ML inference</strong></td>+
</tr>+
</tbody>+
</table>+
<p><a href="https://www.arm.com/technologies/helium">Arm Helium technology</a> is where the biggest performance gains are. CMSIS-NN leverages Helium’s beat-based execution model to pipeline data movement and compute, providing significant throughput gains for the depthwise and pointwise convol…+
<h2>ExecuTorch is Embedded Friendly</h2>+
<p>Unlike mobile or cloud environments that must accommodate varied hardware at runtime, microcontroller deployments target a fixed hardware profile. Embedded developers possess precise knowledge of their deployment target, including the exact CPU core, available ISA extensions, SRAM allocation, and…+
<p>ExecuTorch’s ahead-of-time stack is designed to exploit this. ExecuTorch makes target-specific decisions at compile time: graph-level optimizations, memory planning, weight packing, and kernel selection are all target-aware at compile time.</p>+
<p>Concretely, for Cortex-M this means:</p>+
<ul>+
<li><strong>Graph-level compute elimination.</strong> Redundant dtype conversions, quantize/dequantize pairs, and memory format transposes are removed during lowering, so only necessary compute reaches the device.</li>+
<li><strong>Quantization parameter folding.</strong> Scale and zero-point arithmetic is precomputed into integer multiplier/shift form at export, so quantized kernel inference stays entirely in fixed-point.</li>+
<li><strong>AOT weight layout transposition.</strong> Convolution weights are transposed from PyTorch’s native layout (OIHW) to CMSIS-NN’s expected format (OHWI) at export time. The permute cost is paid once during compilation and never at inference.</li>+
<li><strong>Static memory planning across heterogeneous memories.</strong> All tensor and scratch buffers are allocated and laid out ahead of time, with support for spanning multiple memory regions (e.g., TCM and SRAM). Developers can also customize placement to put latency-critical buffers in fast …+
<li><strong>Target-specific operator replacement.</strong> The CortexMPassManager selects CMSIS-NN kernel variants matched to your core’s ISA tier (Scalar, DSP, or MVE) at compile time, not at runtime.</li>+
</ul>+
<h3>Developer Workflow</h3>+
<pre><code class="language-python">import torch
+
from torchao.quantization.pt2e.quantize_pt2e import convert_pt2e, prepare_pt2e
+
+
from executorch.backends.cortex_m.edge_compile_config import cortex_m_edge_compile_config
+
from executorch.backends.cortex_m.passes.cortex_m_pass_manager import CortexMPassManager
+
from executorch.backends.cortex_m.quantizer.quantizer import CortexMQuantizer
+
from executorch.backends.cortex_m.target_config import CortexM, CortexMTargetConfig
+
from executorch.exir import to_edge_transform_and_lower
+
+
# 1. Export
+
model = MyModel().eval()
+
example_inputs = (torch.randn(1, 1, 49, 10),)
+
exported = torch.export.export(model, example_inputs)
+
+
# 2. Quantize and calibrate
+
quantizer = CortexMQuantizer(use_explicit_layout=True)
+
prepared = prepare_pt2e(exported.module(), quantizer)
+
for sample in calibration_data:
+
prepared(sample)
+
quantized = convert_pt2e(prepared)
+
+
# 3. Re-export the quantized model
+
quantized_exported = torch.export.export(quantized, example_inputs)
+
+
# 4. Lower to Edge and apply the Cortex-M passes
+
pass_manager = CortexMPassManager(
+
target_config=CortexMTargetConfig(cpu=CortexM.M55),
+
use_explicit_layout=True,
+
)
+
edge = to_edge_transform_and_lower(
+
quantized_exported,
+
compile_config=cortex_m_edge_compile_config(),
+
transform_passes=pass_manager,
+
)
+
+
# 5. Serialize for integration into firmware
+
et_program = edge.to_executorch()
+
with open("model.pte", "wb") as f:
+
f.write(et_program.buffer)
+
</code></pre>+
<p>The resulting .pte binary is linked into firmware alongside the ExecuTorch runtime and CMSIS-NN library, then flashed to the device. At runtime, inference executes the precompiled graph with minimal runtime overhead and no dynamic memory allocation for tensors.</p>+
<h2>Using the Cortex-M Backend Alongside Ethos-U</h2>+
<p>Many Cortex-M MCUs pair the CPU with an Arm Ethos-U NPU such as the U55, U65, or U85. ExecuTorch’s <code>EthosUPartitioner</code> delegates supported operations to the NPU via the Vela compiler, and operations left outside the partition run on the Cortex-M CPU within the same graph.</p>+
<p>The Cortex-M passes run after Ethos-U partitioning and reuse the quantization parameters already in the graph. This allows a model quantized with <code>EthosUQuantizer</code> to use CMSIS-NN for CPU operations whose quantization, layout and shape constraints meet the Cortex-M backend requirements…+
<p>The practical benefit is that NPU operator coverage is not an all-or-nothing gate. A model with a handful of undelegated operations can ship today on mixed CPU/NPU execution, and move more of the graph onto the NPU as coverage grows.</p>+
<h2>MLPerf Tiny Benchmark Suite</h2>+
<p>We measured inference latency and memory usage across four of the five <a href="https://mlcommons.org/working-groups/benchmarks/tiny/">MLPerf Tiny Inference</a> benchmark models on the <a href="https://alifsemi.com/ensemble-e8-series/">Alif Ensemble E8</a>. These benchmark models represent a set …+
<p>All four models run in under 90 KiB of RAM, three of them in under 60 KiB, and the anomaly detection autoencoder in 5.2 KiB. Code footprint stays between 15 and 37 KiB because operator selective build links only the operators each model uses.</p>+
<table>+
<thead>+
<tr>+
<th rowspan="2">Model</th>+
<th rowspan="2">Task</th>+
<th rowspan="2">Latency (ms)</th>+
<th rowspan="2">Inferences per second</th>+
<th rowspan="2">Peak RAM (KiB)</th>+
<th colspan="3">Flash (KiB)</th>+
</tr>+
<tr>+
<th>Model</th>+
<th>Code *</th>+
<th>Total</th>+
</tr>+
</thead>+
<tbody>+
<tr>+
<td><strong>DS-CNN</strong></td>+
<td>Keyword Spotting</td>+
<td style="text-align: right;">5.825</td>+
<td style="text-align: right;">172</td>+
<td style="text-align: right;">26.00</td>+
<td>37.67</td>+
<td>36.03</td>+
<td><strong>73.70</strong></td>+
</tr>+
<tr>+
<td><strong>MobileNet v1 0.25</strong></td>+
<td>Visual Wake Words</td>+
<td style="text-align: right;">21.450</td>+
<td style="text-align: right;">47</td>+
<td style="text-align: right;">85.15</td>+
<td>253.07</td>+
<td>36.03</td>+
<td><strong>289.10</strong></td>+
</tr>+
<tr>+
<td><strong>ResNet-8</strong></td>+
<td>Image Classification</td>+
<td style="text-align: right;">15.290</td>+
<td style="text-align: right;">65</td>+
<td style="text-align: right;">55.14</td>+
<td>89.77</td>+
<td>29.82</td>+
<td><strong>119.59</strong></td>+
</tr>+
<tr>+
<td><strong>Autoencoder</strong></td>+
<td>Anomaly Detection</td>+
<td style="text-align: right;">1.208</td>+
<td style="text-align: right;">828</td>+
<td style="text-align: right;">5.20</td>+
<td>272.03</td>+
<td>14.97</td>+
<td><strong>287.00</strong></td>+
</tr>+
</tbody>+
</table>+
<p><em>* Code compiled with GCC 15.2.1 includes ExecuTorch core runtime (-Oz) and kernels (-O3) with logging disabled.</em></p>+
<h2>Demo: Real-time Person Detector</h2>+
<p><!-- IMAGE PLACEHOLDER: Image 2 – Person detector demo on Renesas EK-RA8D1. Download from the Google Doc and re-upload via the WordPress Media Library. --></p>+
<p><img fetchpriority="high" decoding="async" class="wp-image-188134 size-full aligncenter" src="https://pytorch.org/wp-content/uploads/2026/10/IMG_5203.gif" alt="" width="480" height="854" /></p>+
<p>To show what the Cortex-M backend enables in practice, we deployed a µYOLO person detection model on a Renesas EK-RA8D1 board, powered by the Arm Cortex-M85. The deployment fits comfortably within MCU-class memory constraints:</p>+
<ul>+
<li><strong>Task:</strong> Real-time person detection</li>+
<li><strong>Model:</strong> µYOLO, pruned depthwise separable CNN backbone with linear predictor head</li>+
<li><strong>Model size:</strong> 452 KiB (INT8 .pte file)</li>+
<li><strong>Total flash:</strong> 640 KiB</li>+
<li><strong>Peak RAM:</strong> 254 KiB</li>+
<li><strong>Inference latency:</strong> 24 ms per frame inference @ 480 MHz</li>+
<li><strong>End-to-end throughput:</strong> 12.7 FPS</li>+
<li><strong>Input:</strong> 128×128 RGB camera frame</li>+
</ul>+
<p>This example of resource-constrained, always-on vision on the edge supports applications such as robotics sensing, occupancy monitoring, safety systems, smart cameras, and presence-triggered devices.</p>+
<p>Just as importantly, the example demonstrates the development workflow enabled by ExecuTorch. The model definition, training code, export flow, and embedded deployment live within the same PyTorch-based ecosystem. Intermediate representations remain executable, making it possible to compare model…+
<p>The model definition, training and export flow are available in the <a href="https://github.com/pytorch/executorch/tree/main/examples/arm/person_detection_example_cortex_m">ExecuTorch repository</a>. To facilitate an easy bring-up of the demo without the need for a physical target, the in-tree ap…+
<h2>Getting Started</h2>+
<p>The Cortex-M backend is available today in the ExecuTorch repository:</p>+
<table>+
<thead>+
<tr>+
<th>Resource</th>+
<th>Description</th>+
</tr>+
</thead>+
<tbody>+
<tr>+
<td><a href="https://docs.pytorch.org/executorch/main/backends/arm-cortex-m/arm-cortex-m-overview.html">Cortex-M Backend Documentation</a></td>+
<td>Supported operators, target configuration, and build setup</td>+
</tr>+
<tr>+
<td><a href="https://github.com/pytorch/executorch/tree/main/examples/arm">Arm Backend Examples</a></td>+
<td>Full example directory with setup scripts</td>+
</tr>+
<tr>+
<td><a href="https://github.com/Arm-Examples/cmsis-executorch">CMSIS-ExecuTorch Project Template</a></td>+
<td>Docker build + Keil Studio + AVH simulation</td>+
</tr>+
<tr>+
<td><a href="https://github.com/Arm-Examples/cmsis-zephyr-executorch">Zephyr RTOS Integration</a></td>+
<td>ExecuTorch on Zephyr with CMSIS Toolbox</td>+
</tr>+
<tr>+
<td><a href="https://github.com/pytorch/executorch/tree/main/examples/raspberry_pi/pico2">Raspberry Pi Pico 2 Example</a></td>+
<td>Cortex-M33, MNIST digit classification</td>+
</tr>+
</tbody>+
</table>+
<h3>Platforms</h3>+
<table>+
<thead>+
<tr>+
<th>Platform</th>+
<th>CPU</th>+
<th>Available via</th>+
</tr>+
</thead>+
<tbody>+
<tr>+
<td><strong>Arm Corstone-300 FVP</strong></td>+
<td>Cortex-M55 + Ethos-U55</td>+
<td>In-tree examples, exercised in CI</td>+
</tr>+
<tr>+
<td><strong>Arm Corstone-320 FVP</strong></td>+
<td>Cortex-M85 + Ethos-U85</td>+
<td>In-tree examples, exercised in CI</td>+
</tr>+
<tr>+
<td><strong>Alif Ensemble E8</strong></td>+
<td>Cortex-M55 (MVE)</td>+
<td><a href="https://github.com/alifsemi/alif_ml-embedded-evaluation-kit">Alif ML Embedded Evaluation Kit</a></td>+
</tr>+
<tr>+
<td><strong>Raspberry Pi Pico 2</strong></td>+
<td>Cortex-M33 (DSP)</td>+
<td>In-tree example</td>+
</tr>+
<tr>+
<td><strong>Renesas EK-RA8D1</strong></td>+
<td>Cortex-M85</td>+
<td>Out-of-tree board port; in-tree demo runs on Corstone FVP</td>+
</tr>+
</tbody>+
</table>+
<h2>What’s Next</h2>+
<ul>+
<li><strong>Expanded mixed-precision support.</strong> Extending A8W4 and A16W8 mixed-precision support across the full operator set.</li>+
<li><strong>Floating-point kernels.</strong> CMSIS-NN 8.0.0 introduced fp32 and fp16 MVE kernels. Exposing them via the Cortex-M backend enables non-fully quantized models to use the same lowering path without separate deployment flows.</li>+
<li><strong>Broader operator coverage.</strong> Adding support for attention operators, layer normalization, and other popular architecture patterns beyond traditional CNNs, along with CMSIS-DSP operators such as FFT so feature extraction can run as part of the graph.</li>+
<li><strong>More end-to-end examples.</strong> Voice activity detection, gesture recognition, predictive maintenance, and streaming wake word, the fifth MLPerf Tiny benchmark, which needs test infrastructure for multi-frame inference.</li>+
<li><strong>Additional hardware validation.</strong> Expanding the set of validated platforms and development kits.</li>+
</ul>+
<p>We’re excited about what this unlocks for the embedded ML community. If you’re deploying ML at the edge, whether on a $1 sensor node or a Cortex-M55 running voice commands, ExecuTorch now provides a direct path from PyTorch training to on-device inference.</p>+
<p>Learn more at <a href="https://docs.pytorch.org/executorch/main/index.html">ExecuTorch Documentation</a> | <a href="https://github.com/pytorch/executorch/">ExecuTorch repo</a> | <a href="https://github.com/pytorch/executorch/tree/main/backends/cortex_m">Cortex-M Backend</a> | <a href="https://dis…+
<p>The post <a href="https://pytorch.org/blog/bringing-executorch-to-arm-cortex-m-cpus/">Bringing ExecuTorch to Arm Cortex-M CPUs</a> appeared first on <a href="https://pytorch.org">PyTorch</a>.</p>+
]]></content:encoded>+
+
+
+
</item>+
<item> <title>Extending Inductor for the IBM Spyre Accelerator</title> <link>https://pytorch.org/blog/extending-inductor-for-the-ibm-spyre-accelerator/</link> @
@@ -160,7 +491,7 @@<h2>Introduction</h2><p>Agentic workloads have changed the shape of the traffic an inference server sees. A coding session opens with a large prefill, often tens of thousands of tokens of system prompt, tool definitions, and user-specific guidance, and every turn after that appends to a context the session resends in fu…<p><!-- IMAGE PLACEHOLDER: Figure 1 (image1) - download from the Doc and upload via WP media library --></p>-
<p><em><img fetchpriority="high" decoding="async" class="alignnone wp-image-172935 size-full" src="https://pytorch.org/wp-content/uploads/2026/10/Agentic-Model-Chain-GIF-2-1.gif" alt="" width="1280" height="720" /></em></p>+
<p><em><img decoding="async" class="alignnone wp-image-172935 size-full" src="https://pytorch.org/wp-content/uploads/2026/10/Agentic-Model-Chain-GIF-2-1.gif" alt="" width="1280" height="720" /></em></p><p><em>Figure 1: Compares the alternating user and model turns of a standard chatbot with an agentic workflow that includes tool calls and tool responses. In the agentic workflow, a single user request can trigger multiple inference calls, with subsequent steps depending on intermediate results.</em…<p>Most open-source serving stacks are still built for the model on top of the figure. They route, admit, and cache each request on its own, with no notion of the harness or the session that ties it to previous turns.</p><p>Serving this shape optimally is something we have been building towards. In our previous post, <a href="https://developer.nvidia.com/blog/full-stack-optimizations-for-agentic-inference-with-nvidia-dynamo/">Full-Stack Optimizations for Agentic Inference</a>, we shared work across three layers of t…@
@@ -1096,222 +1427,8 @@ python scripts/autotune_helion_kernels.py \ -
</item>-
<item>-
<title>Optimizing Jagged Flash Attention with TLX: The Road Toward SOTA FA4 on Blackwell</title>-
<link>https://pytorch.org/blog/optimizing-jagged-flash-attention-with-tlx-the-road-toward-sota-fa4-on-blackwell/</link>-
-
<dc:creator><![CDATA[Han Xu, Jacky Zhou, Jackie (Jiaqi) Xu, Hongtao Yu, Peng Chen (Dev Infra), Darren Liu, Dev (Devashish) Shankar, Max Leung, Nick Riasanovsky, Hao Yan, Manman Ren, Yuanwei (Kevin) Fang]]></dc:creator>-
<pubDate>Thu, 01 Oct 2026 22:26:57 +0000</pubDate>-
<category><![CDATA[Blog]]></category>-
<guid isPermaLink="false">https://pytorch.org/?p=171511</guid>-
-
<description><![CDATA[<p>TL;DR In this blog post, we present our work on Jagged Flash Attention (JFA) — the attention kernel behind Meta’s Generative Ads Model (GEM) — on NVIDIA Blackwell (B200), built...</p>-
<p>The post <a href="https://pytorch.org/blog/optimizing-jagged-flash-attention-with-tlx-the-road-toward-sota-fa4-on-blackwell/">Optimizing Jagged Flash Attention with TLX: The Road Toward SOTA FA4 on Blackwell</a> appeared first on <a href="https://pytorch.org">PyTorch</a>.</p>-
]]></description>-
<content:encoded><![CDATA[<h2><span style="font-weight: 400;">TL;DR</span></h2>-
<p><span style="font-weight: 400;">In this blog post, we present our work on Jagged Flash Attention (JFA) — the attention kernel behind Meta’s Generative Ads Model (GEM) — on NVIDIA Blackwell (B200), built with TLX (Triton Low-level Extensions), which add explicit, hardware-aware control on to…-
<p><span style="font-weight: 400;">Attention is the single slowest kernel in GEM, and reaching peak Blackwell performance has traditionally required hand-written CuteDSL or CUDA — slow to develop and hard to extend to new variants. We show that TLX closes this gap on both fronts. On development effi…-
<p><span style="font-weight: 400;">We organize the work in two parts — the structural changes the TLX rewrite makes possible, and the optimizations we layer on top — all benchmarked in bfloat16 on B200 against FA4, the current state of the art on Blackwell.</span></p>-
<p><span style="font-weight: 400;">Code available at: </span><a href="https://github.com/facebookresearch/ads_model_kernel_library/tree/main/tlx_jfa"><span style="font-weight: 400;">https://github.com/facebookresearch/ads_model_kernel_library/tree/main/tlx_jfa</span></a></p>-
<h2><span style="font-weight: 400;">Introduction</span></h2>-
<p><span style="font-weight: 400;">Meta’s ads models, including the Generative Ads Model (GEM) [2] and the Kunlun architecture [5], run attention over jagged (variable-length or ragged) user sequences. As described in our </span><a href="https://engineering.fb.com/2026/08/03/ml-applications/tr…-
<p><img decoding="async" class="aligncenter wp-image-171512 size-full" src="https://pytorch.org/wp-content/uploads/2026/09/1.png" alt="" width="1376" height="768" srcset="https://pytorch.org/wp-content/uploads/2026/09/1.png 1376w, https://pytorch.org/wp-content/uploads/2026/09/1-300x167.png 300w, ht…-
<p><span style="font-weight: 400;">Hitting peak throughput on Blackwell means keeping the tensor cores continuously fed. Plain Triton leaves most of these decisions to the compiler; TLX [1] exposes them as first-class primitives (explicit SMEM/TMEM allocation, async_task warp specialization, barrier…-
<h2><span style="font-weight: 400;">1. The Challenge: Performant and Extensible Attention on Blackwell</span></h2>-
<p><span style="font-weight: 400;">Optimizing attention on Blackwell means solving two problems at once. The first is performance: attention is the single slowest kernel in GEM, and it reaches peak throughput only when the global-memory loads, softmax, and matmuls are tightly overlapped. The second …-
<p><b>Why a compiler-scheduled baseline falls short.</b><span style="font-weight: 400;"> Our starting point is a mature Triton-based JFA kernel — algorithmically correct, but it leaves all on-chip data movement and scheduling to the compiler: no control over shared-memory allocation or pipeline dept…-
<p><b>The production case, broadcast-Q, further shapes the design:</b><span style="font-weight: 400;"> a single dense Q is broadcast across every sequence in the batch, so its gradient </span><b>dQ must be summed across the entire batch</b><span style="font-weight: 400;"> — turning the dQ epilogue i…-
<h2><span style="font-weight: 400;">2. Optimizing Jagged Flash Attention with TLX</span></h2>-
<p><span style="font-weight: 400;">Our work falls into two categories. </span><b>Structural changes</b><span style="font-weight: 400;"> (§2.1) are the ground-up reorganization of the kernel that TLX makes possible — they change how work and memory are laid out across the CTA’s warps. </span><b…-
<h3><span style="font-weight: 400;">2.1 Structural changes</span></h3>-
<p><span style="font-weight: 400;">Warp specialization, explicit shared-/tensor-memory management, and barrier pipelining are standard ingredients of any high-performance Blackwell attention kernel, so we keep this brief — what matters is that TLX lets us express them in high-level Triton code rathe…-
<p><img decoding="async" class="aligncenter wp-image-171513 size-full" src="https://pytorch.org/wp-content/uploads/2026/09/2.png" alt="" width="1620" height="900" srcset="https://pytorch.org/wp-content/uploads/2026/09/2.png 1620w, https://pytorch.org/wp-content/uploads/2026/09/2-300x167.png 300w, ht…-
<p><span style="font-weight: 400;">The same explicit control extends to memory and scheduling: we hand-allocate every on-chip buffer and choose its pipeline depth (e.g. triple-buffered K/V so the load warp runs ahead), alias TMEM buffers with non-overlapping lifetimes (QK scores, P, and the softmax …-
<h3><span style="font-weight: 400;">2.2 Optimizations</span></h3>-
<p><span style="font-weight: 400;">With the warp-specialized, persistent structure in place, the remaining work is to eliminate the stalls and wasted work that keep the tensor cores from staying busy. The following optimizations are largely independent and each targets a specific bottleneck.</span><…-
<p><span style="font-weight: 400;">We found them with a consistent methodology: NVIDIA Nsight Compute (NCU) to read hardware counters such as tensor-core/TMEM pipeline utilization, SM-busy, and register spills (which show up as local-memory traffic); the ptxas dump logs to confirm spills; and Triton…-
<p><span style="font-weight: 400;">At a glance, the optimizations below are:</span></p>-
<ul>-
<li style="font-weight: 400;" aria-level="1"><b>Scheduling jagged tiles across SMs</b><span style="font-weight: 400;"> — software load balancing plus Cluster Launch Control keep every SM busy despite the order-of-magnitude length skew of jagged inputs (forward and backward).</span></li>-
<li style="font-weight: 400;" aria-level="1"><b>Multi-stage dQ staging</b><span style="font-weight: 400;"> — a double-buffered SMEM staging pipeline hides the heavily contended broadcast-Q dQ reduce-add, the #1 backward bottleneck.</span></li>-
<li style="font-weight: 400;" aria-level="1"><b>Early tensor-memory release</b><span style="font-weight: 400;"> — freeing the dQ tensor-memory buffer before its final stores lets the MMA warp start the next tile’s dQ matmul sooner.</span></li>Diff display stops at 400 lines. The line counts above are from the whole diff. 41 lines shown here cut at 300 characters. The raw artifact at this commit is linked above.