Change
9880267
9880267022263cfbd8537419f2ac9ecbe8f238d9 · commit on GitHub
pytorch-blog-feed: changed (295568 bytes, HTTP 200)
raw/pytorch-blog-feed/response.xml modified
- Source
- pytorch-blog-feed
- Lines added
- +73
- Lines removed
- -454
- Stored bytes at this commit
- 295,568
- Timestamp
- origin
- Raw artifact at this commit
- raw/pytorch-blog-feed/response.xml
Recorded headers
| observed_at | 2026-11-03T06:47:08.598Z |
|---|---|
| origin_date | 2026-11-03T06:45:16.000Z |
| status | 200 |
| final URL | https://pytorch.org/blog/feed/ |
| etag | "b59fdaaf1044aaa1f9d4ba286c80d294" |
| last-modified | Mon, 02 Nov 2026 17:45:32 GMT |
| date | Tue, 03 Nov 2026 06:47:08 GMT |
| age | 112 |
| cache-control | public, max-age=60, s-maxage=43200, stale-while-revalidate=86400, stale-if-error=604800 |
| cf-cache-status | null |
| content-encoding | null |
| content-length | 295568 |
@
@@ -12,13 +12,13 @@ <atom:link href="https://pytorch.org/blog/feed/" rel="self" type="application/rss+xml" /> <link>https://pytorch.org/blog/</link> <description></description>-
<lastBuildDate>Wed, 28 Oct 2026 17:44:20 +0000</lastBuildDate>+
<lastBuildDate>Mon, 02 Nov 2026 17:13:33 +0000</lastBuildDate> <language>en-US</language> <sy:updatePeriod> hourly </sy:updatePeriod> <sy:updateFrequency> 1 </sy:updateFrequency>-
<generator>https://wordpress.org/?v=7.1.3</generator>+
<generator>https://wordpress.org/?v=7.1.4</generator><image> <url>https://pytorch.org/wp-content/uploads/2024/10/cropped-favicon-32x32.webp</url>@
@@ -28,6 +28,75 @@ <height>32</height></image> <item>+
<title>Announcing the Special Interest Group (SIG) for Ray on Accelerators</title>+
<link>https://pytorch.org/blog/announcing-the-special-interest-group-sig-for-ray-on-accelerators/</link>+
+
<dc:creator><![CDATA[Zhe Zhang, Mengjin Yan, Ryan O’Leary, Richard Liaw]]></dc:creator>+
<pubDate>Mon, 02 Nov 2026 17:45:32 +0000</pubDate>+
<category><![CDATA[Blog]]></category>+
<guid isPermaLink="false">https://pytorch.org/?p=208672</guid>+
+
<description><![CDATA[<p>Today, we are announcing the Special Interest Group (SIG) for Ray on hardware accelerators. When Ray was created in UC Berkeley over 10 years ago, the computing landscape looked very...</p>+
<p>The post <a href="https://pytorch.org/blog/announcing-the-special-interest-group-sig-for-ray-on-accelerators/">Announcing the Special Interest Group (SIG) for Ray on Accelerators</a> appeared first on <a href="https://pytorch.org">PyTorch</a>.</p>+
]]></description>+
<content:encoded><![CDATA[<p>Today, we are announcing the Special Interest Group (SIG) for Ray on hardware accelerators.</p>+
<p>When Ray was created in UC Berkeley over 10 years ago, the computing landscape looked very different. At the time, “distributed computing” referred to CPU clusters running frameworks like Apache Hadoop and Apache Spark. GPUs were only just moving from graphics and scientific computing into machin…+
<p>Today, <em>accelerated computing</em> is front and center in the development and deployment of AI, and accelerators and their capabilities have become more specialized and diverse. For example:</p>+
<ul>+
<li>NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs support flexible workloads spanning AI training, inference, graphics, and video</li>+
<li>NVIDIA HGX B200 and the full-stack platform provide substantially greater memory capacity to support larger models, and NVIDIA GB200 NVL72 connects 72 NVIDIA Blackwell GPUs for communication-intensive, large-scale training and inference.</li>+
<li>AMD MI355X also provides substantial memory capacity for large models</li>+
<li>Google Ironwood TPUs target massive-scale AI workloads</li>+
<li>NVIDIA Groq 3 LPUs specialize in low-latency inference.</li>+
</ul>+
<p>The Ray community has been actively working on keeping up with this shift. Today, Ray <a href="https://docs.ray.io/en/latest/ray-core/scheduling/accelerators.html">supports more than a dozen kinds of hardware accelerators</a>, from NVIDIA and AMD GPUs to TPUs, Trainium, Gaudi and others. We belie…+
<p>The Accelerator Special Interest Group aims to bring together hardware vendors, Ray contributors, and application developers to define and advance a shared, vendor-neutral roadmap for accelerator support in Ray. This public interest group will be responsible for enabling Ray workloads to continue…+
<p>Our initial roadmap focuses on 3 core areas: <strong>Development and Testing Infrastructure, Hardware Awareness Capabilities, and Networking/Transport</strong>, outlined in the following sections.</p>+
<h2>Development and Testing Infra</h2>+
<p>Reliable accelerator support requires dependable access to hardware for development, testing, and debugging. Maintainers and contributors need to validate changes on the devices they affect, reproduce failures, and catch regressions before they reach users. As agentic software engineering acceler…+
<p>Our initial priorities are:</p>+
<ul>+
<li><strong>Shared hardware access</strong>. Establish a documented process for bringing cloud and vendor-hosted accelerators into Ray’s development and testing infrastructure, including access for contributors to reproduce and debug failures.</li>+
<li><strong>Reproducible development and test environments</strong>. Provide maintained environments with explicit driver, runtime, and dependency versions, so contributors can reproduce CI results and validate changes consistently across providers.</li>+
<li><strong>Broader CI coverage</strong>. Extend existing pipelines with targeted checks for pull requests and broader nightly and release testing. Coverage should include distributed workloads, failure recovery, and performance regressions, alongside basic device functionality.</li>+
<li><strong>Sustainable operation and ownership</strong>. Define responsibilities for maintaining environments, investigating failures, and keeping integrations working across releases. Make test results and infrastructure health visible, and manage capacity and costs so the infrastructure can grow …+
</ul>+
<h2>Hardware-aware Scheduling</h2>+
<p>For applications to get the most out of an accelerator, Ray first has to know what hardware each node has and how it is structured. As the accelerator ecosystem evolves, accelerator nodes are no longer just a count of identical devices. Workload performance depends on the interconnectivity of dev…+
<p>Our goal is to give Ray an accurate and consistent picture of this structure across vendors and platforms. With that picture, Ray’s scheduler can place tasks and actors on the right hardware, and users can see what their cluster is made of and how it is connected.</p>+
<p>Our initial priorities are:</p>+
<ul>+
<li><strong>Common hardware interfaces.</strong> Define vendor-neutral interfaces for labeling hardware, representing its architecture, and describing its specifications, so applications and libraries can target capabilities instead of writing vendor-specific code. This includes a consistent way to …+
<li><strong>Hardened discovery.</strong> Strengthen Ray’s discovery mechanism, with clear ownership for each accelerator’s integration, and back it with the CI coverage described in the previous section. Discovery should capture the topology and architecture of the hardware, not just the type and nu…+
<li><strong>Platform and cloud integration.</strong> Work with cloud providers and deployment platforms so that discovery reflects the underlying structure wherever a cluster runs. Some of this information, such as which rack or slice a node belongs to, is only available from the platform, not from …+
<li><strong>Hardware visibility.</strong> Provide ways to view the accelerators in a cluster and how they are connected, alongside accelerator metrics on the Ray dashboard. This lets users confirm that Ray sees their hardware correctly and understand where their work was placed. It also gives contri…+
<li><strong>Clear user guides.</strong> Document how Ray discovers and represents hardware, and how to use that information when placing work. Good guides help both application developers and the coding agents that increasingly write Ray code for them.</li>+
</ul>+
<h2>Networking and I/O Transport</h2>+
<p>Moving data between accelerators is often just as important as running computation on the accelerator itself. Modern AI workloads, like large scale reinforcement learning, require moving large model weight tensors across devices and nodes from training to inference workers. Historically, Ray’s ob…+
<p>To avoid that bottleneck, <strong>Ray Direct Transport (RDT)</strong> was introduced to enable actors to pass references to tensors through Ray while transferring the actual data directly between devices and using high performance transports like UCX and RDMA. Our goal is to make high-speed, dire…+
<p>Our initial priorities are:</p>+
<ul>+
<li><strong>Broad transport integrations</strong>. Support a wider range of accelerators, frameworks, and interconnects in RDT so Ray can select the fastest path between Actors, whether over intra-slice TPU ICI, scale-up NVIDIA NVLink, or scale-out GPUDirect RDMA.</li>+
<li><strong>Transport lifecycle and recovery</strong>. Ensure each hardware transport cleanly handles initialization, teardown, and error recovery on real devices, so RDT works reliably as actors scale or restart in dynamic workloads like reinforcement learning and disaggregated serving.</li>+
<li><strong>Validated recipes and observability</strong>. Enable cloud and hardware providers to publish validated deployment configurations, benchmark high-throughput workloads, and surface clear diagnostics and transport metrics.</li>+
</ul>+
<h2>Scope</h2>+
<p>While core scheduling and placement logic remain in Ray Core, SIG Accelerator is responsible for defining the plug-in interfaces, CI testing, and maturity tiers for accelerators in Ray. Hardware vendors and cloud providers contribute and maintain their own device plugins, transports, and day-zero…+
<h2>Get Involved</h2>+
<p>Whether you are building a new accelerator integration, running large-scale training or inference on Ray, or providing cloud and CI infrastructure, we invite you to get involved!</p>+
<ul>+
<li>Join the conversation on the <a href="https://www.ray.io/join-slack">Ray Slack</a>.</li>+
<li>Attend our community meetings to discuss proposals, review roadmap priorities, and share workload feedback.</li>+
<li>Check out open accelerator related <a href="https://github.com/ray-project/ray/issues">issues</a> and PRs on Github and contribute upstream.</li>+
</ul>+
<p>The post <a href="https://pytorch.org/blog/announcing-the-special-interest-group-sig-for-ray-on-accelerators/">Announcing the Special Interest Group (SIG) for Ray on Accelerators</a> appeared first on <a href="https://pytorch.org">PyTorch</a>.</p>+
]]></content:encoded>+
+
+
+
</item>+
<item> <title>PyTorch 2.15 Release Blog</title> <link>https://pytorch.org/blog/pytorch-2-15-release-blog/</link> @
@@ -440,7 +509,7 @@ in the Agentic Era: Bringing Native PyTorch to Cloud TPUs: Bill Jia (Google Clou <description><![CDATA[<p>The post <a href="https://pytorch.org/blog/meet-the-2026-pytorch-foundation-ambassadors/">Meet the 2026 PyTorch Foundation Ambassadors</a> appeared first on <a href="https://pytorch.org">PyTorch</a>.</p>]]></description> <content:encoded><![CDATA[-
<div id="fws_6ae23cadc32df" data-column-margin="default" data-midnight="dark" class="wpb_row vc_row-fluid vc_row top-level" style="padding-top: 0px; padding-bottom: 0px; "><div class="row-bg-wrap" data-bg-animation="none" data-bg-animation-delay="" data-bg-overlay="false"><div class="inner-wrap…+
<div id="fws_6ae8cc49a6f9e" data-column-margin="default" data-midnight="dark" class="wpb_row vc_row-fluid vc_row top-level" style="padding-top: 0px; padding-bottom: 0px; "><div class="row-bg-wrap" data-bg-animation="none" data-bg-animation-delay="" data-bg-overlay="false"><div class="inner-wrap… <div class="vc_col-sm-12 wpb_column column_container vc_column_container col no-extra-padding" data-padding-pos="all" data-has-bg-color="false" data-bg-color="" data-bg-opacity="1" data-animation="" data-delay="0" > <div class="vc_column-inner" > <div class="wpb_wrapper">@
@@ -1972,458 +2041,8 @@ Invoke directly on PyTorch tensors (Eager & <code>torch.compile</code>)</td> -
</item>-
<item>-
<title>From KTIR to SpyreCode: An MLIR Dataflow Backend for IBM Spyre</title>-
<link>https://pytorch.org/blog/from-ktir-to-spyrecode-an-mlir-dataflow-backend-for-ibm-spyre/</link>-
-
<dc:creator><![CDATA[Swagath Venkataramani, Bardia Mahjour, Prasanth Chatarasi, Alberto Mannari, Viji Srinivasan, and the IBM Spyre Team]]></dc:creator>-
<pubDate>Tue, 20 Oct 2026 16:00:15 +0000</pubDate>-
<category><![CDATA[Blog]]></category>-
<guid isPermaLink="false">https://pytorch.org/?p=188742</guid>-
-
<description><![CDATA[<p>TL;DR Torch-spyre’s Inductor decides how each kernel’s work is divided across Spyre’s cores and manages tensor residency between on-chip scratchpad and off-chip memory, and the backend compiler turns those decisions...</p>-
<p>The post <a href="https://pytorch.org/blog/from-ktir-to-spyrecode-an-mlir-dataflow-backend-for-ibm-spyre/">From KTIR to SpyreCode: An MLIR Dataflow Backend for IBM Spyre</a> appeared first on <a href="https://pytorch.org">PyTorch</a>.</p>-
]]></description>-
<content:encoded><![CDATA[<h3><strong>TL;DR</strong></h3>-
<p>Torch-spyre’s Inductor decides how each kernel’s work is divided across Spyre’s cores and manages tensor residency between on-chip scratchpad and off-chip memory, and the backend compiler turns those decisions into something the hardware can actually run. The contract between them is represented …-
<p>This post is the middle chapter of three. <a href="https://pytorch.org/blog/extending-inductor-for-the-ibm-spyre-accelerator/"><em>Extending Inductor for the IBM Spyre Accelerator</em></a> described how torch-spyre’s Inductor extensions settle layout, tiling, work division and scratchpad placemen…-
<p><!-- IMAGE PLACEHOLDER: spyre-software-stack.png - upload via Media Library. Suggested alt text: "Where the backend compiler sits in the Spyre software stack" --></p>-
<p><em><a href="https://pytorch.org/wp-content/uploads/2026/10/stack-position.svg"><img decoding="async" class="alignnone size-full wp-image-188751" src="https://pytorch.org/wp-content/uploads/2026/10/stack-position.svg" alt="" /></a>Where the backend compiler sits in the Spyre software stack</em></…-
<p>Three ideas run through the design.</p>-
<ul>-
<li><strong>The frontend decides the parallel decomposition, and the backend makes it run.</strong> KTIR is designed to carry the mapping Inductor has already chosen (which core computes what, over which data, out of which memory) so that the backend never has to rediscover it. The backend’s job is …-
<li><strong>Scheduling means building a pipeline.</strong> Spyre is a dataflow machine with decoupled access and execution, so loads, stores and compute run on separate engines communicating via FIFOs. The scheduler starts from the simplest legal pipeline and refines it one transformation at a time,…-
<li><strong>The boundary is an IR, not a library call.</strong> KTIR, the scheduler’s ScheduleIR and DataflowIR are all ordinary MLIR built on upstream dialects (<code>linalg</code>, <code>scf</code>, <code>arith</code>, <code>memref</code>, <code>tensor</code>), and the scheduler and its dialects a…-
</ul>-
<h2>KTIR: the contract between Inductor and the backend</h2>-
<h3>Why a new interface</h3>-
<p>One of the key requirements for efficient backend scheduling is a contract between torch-spyre’s Inductor and the backend compiler with the capability and flexibility to capture the complete data-parallel view of the accelerator, in which every compute tile can see every memory and a single logic…-
<p>KTIR (<a href="https://github.com/torch-spyre/RFCs/blob/main/0682-KtirSpec/0682-KtirSpecRFC.md">RFC 0682</a>, implemented in <a href="https://github.com/torch-spyre/ktir-mlir-frontend"><code>torch-spyre/ktir-mlir-frontend</code></a>) is a tile-based, block-structured IR built on open-source MLIR.…-
<h3>A data-parallel abstraction of the accelerator</h3>-
<p>KTIR is written against a deliberately simple picture of the hardware: a set of cores, each with a compute engine and its own on-chip scratchpad, joined by an on-chip interconnect that also reaches one or more off-chip memories. The property that matters most is that <strong>every compute tile ha…-
<p><!-- IMAGE PLACEHOLDER: ktir-hardware-abstraction.png - upload via Media Library. Suggested alt text: "Hardware abstraction of a multi-core accelerator in KTIR: cores with compute engines and scratchpads on an interconnect with off-chip memory" --></p>-
<p><em><img decoding="async" class="alignnone wp-image-188762 size-full" src="https://pytorch.org/wp-content/uploads/2026/10/ktir-hw-abstraction.png" alt="" width="545" height="382" srcset="https://pytorch.org/wp-content/uploads/2026/10/ktir-hw-abstraction.png 545w, https://pytorch.org/wp-content/up…-
<p><em>Hardware abstraction of a multi-core accelerator embodied in KTIR (from RFC 0682)</em></p>-
<p>That picture matches Spyre’s shape closely: 32 cores on a ring, each with 2 MB of scratchpad, backed by off-chip device memory. Data moves between device memory and the scratchpads in <em>sticks</em>, fixed 128-byte chunks that hold 64 fp16 values. Intentionally, the picture leaves out details su…-
<p>KTIR is also compositional. Arithmetic, reductions and control flow come straight from upstream MLIR dialects (<code>arith</code>, <code>math</code>, <code>linalg</code>, <code>scf</code>, <code>tensor</code>, <code>memref</code>), and a dedicated dialect, <code>ktdp</code>, adds the two things t…-
<table>-
<thead>-
<tr>-
<th><code>ktdp</code> op</th>-
<th>What it expresses</th>-
</tr>-
</thead>-
<tbody>-
<tr>-
<td><code>get_compute_tile_id</code></td>-
<td>Which compute tile this is; 1-D or multi-dimensional, over the <code>grid</code> attribute on the kernel</td>-
</tr>-
<tr>-
<td><code>construct_memory_view</code></td>-
<td>Interpret a region at some (possibly symbolic) address in a named memory space as a strided tensor; no allocation, no data movement</td>-
</tr>-
<tr>-
<td><code>construct_distributed_memory_view</code></td>-
<td>Compose several memory views, each covering part of a global coordinate space, into one logical tensor</td>-
</tr>-
<tr>-
<td><code>construct_access_tile</code></td>-
<td>A structured set of coordinates into a view: an integer set for the domain and an affine map for the traversal order, so tiles can be rectangular, strided, skewed or triangular</td>-
</tr>-
<tr>-
<td><code>construct_indirect_access_tile</code></td>-
<td>The same, with some dimensions indexed through another tensor, e.g. paged KV-cache gathers</td>-
</tr>-
<tr>-
<td><code>load</code> / <code>store</code></td>-
<td>The only data movement: consume an access tile and produce or write a data tile</td>-
</tr>-
<tr>-
<td><code>inter_tile_produce</code> / <code>inter_tile_reduce</code></td>-
<td>Communication between compute tiles, e.g. an all-reduce of partial sums</td>-
</tr>-
</tbody>-
</table>-
<p>Here is a complete kernel, an elementwise add over <code>96×64</code> fp16 tensors divided among 32 compute tiles of three rows each.</p>-
<pre><code class="language-mlir">#tile = affine_set<(d0, d1) : (d0 >= 0, -d0 + 2 >= 0, d1 >= 0, -d1 + 63 >= 0)> // 3x64
-
#full = affine_set<(d0, d1) : (d0 >= 0, -d0 + 95 >= 0, d1 >= 0, -d1 + 63 >= 0)>
-
#id = affine_map<(d0, d1) -> (d0, d1)>
-
-
// The tensors' start addresses arrive as arguments: PyTorch's allocator owns them.
-
func.func @add(%A: index, %B: index, %C: index) attributes {grid = [32]} {
-
%c0 = arith.constant 0 : index
-
%c3 = arith.constant 3 : index
-
%id = ktdp.get_compute_tile_id : index
-
%row0 = arith.muli %id, %c3 : index // this tile's 3-row slab
-
-
%A_view = ktdp.construct_memory_view %A, sizes: [96, 64], strides: [64, 1] {
-
coordinate_set = #full, memory_space = #ktdp.memory_space<global>
-
} : memref<96x64xf16>
-
// ... %B_view and %C_view alike
-
-
%A_t = ktdp.construct_access_tile %A_view[%row0, %c0] {
-
access_tile_set = #tile, access_tile_order = #id
-
} : memref<96x64xf16> -> !ktdp.access_tile<3x64xindex>
-
// ... %B_t and %C_t alike
-
-
%a = ktdp.load %A_t : !ktdp.access_tile<3x64xindex> -> tensor<3x64xf16>
-
%b = ktdp.load %B_t : !ktdp.access_tile<3x64xindex> -> tensor<3x64xf16>
-
%e = tensor.empty() : tensor<3x64xf16>
-
%c = linalg.add ins(%a, %b : tensor<3x64xf16>, tensor<3x64xf16>)
-
outs(%e : tensor<3x64xf16>) -> tensor<3x64xf16>
-
ktdp.store %c, %C_t : tensor<3x64xf16>, !ktdp.access_tile<3x64xindex>
-
return
-
}
-
</code></pre>-
<p>The kernel keeps four concerns apart. Views say how memory should be interpreted, access tiles say which coordinates a given tile touches, loads and stores are the only operations that move data, and <code>linalg</code> does the arithmetic. Notice also that the work division is already settled: <…-
<h3>Spyre intrinsics: the <code>spyreop</code> dialect</h3>-
<p>Upstream <code>arith</code> and <code>math</code> cover most of the arithmetic a kernel needs, but Spyre’s compute units also offer operations that no generic dialect has a name for. A kernel that wants one of those should be able to ask for it directly, instead of hoping the backend recognizes s…-
<p>Intrinsics appear wherever any other scalar computation would, which is inside the body of a <code>linalg.generic</code>:</p>-
<pre><code class="language-mlir">%y = linalg.generic {indexing_maps = [#in, #out],
-
iterator_types = ["parallel", "parallel", "parallel", "parallel"]}
-
ins(%x : tensor<6x64x64xf16>) outs(%init : tensor<1x6x64x64xf16>) {
-
^bb0(%a: f16, %out: f16):
-
%g = spyreop.gelu %a : f16
-
linalg.yield %g : f16
-
} -> tensor<1x6x64x64xf16>
-
</code></pre>-
<p>Because they are just operations in a <code>linalg</code> body, intrinsics fuse with the surrounding elementwise work like anything else, and the scheduler carries them through as compute until it is ready to lower them. How each intrinsic expands into the hardware’s own instruction sequence is n…-
<h3>How KTIR captures Inductor’s work mapping</h3>-
<p>The Inductor post walked through a series of decisions torch-spyre makes before any code is generated. Each of them has a direct counterpart in KTIR:</p>-
<table>-
<thead>-
<tr>-
<th>Decision in torch-spyre’s Inductor passes</th>-
<th>How it appears in KTIR</th>-
</tr>-
</thead>-
<tbody>-
<tr>-
<td>Work division: the per-op core split, e.g. a matmul’s 16 × 2 grid</td>-
<td>The kernel’s <code>grid</code> attribute, plus affine arithmetic on <code>ktdp.get_compute_tile_id</code> that picks each tile’s slice</td>-
</tr>-
<tr>-
<td>Stick-granularity layout: which dimension is the stick, and the tiled order of the others</td>-
<td>Strides of <code>construct_memory_view</code>, and the <code>access_tile_order</code> of each access tile</td>-
</tr>-
<tr>-
<td>Coarse tiling and working-set reduction</td>-
<td><code>scf.for</code> loops around loads, compute and stores, which run the chain tile by tile</td>-
</tr>-
<tr>-
<td>Scratchpad residency chosen by the scratchpad allocator</td>-
<td>A memory view in <code>#ktdp.memory_space<ct_local, ct_id = k></code> at the offset the allocator assigned</td>-
</tr>-
<tr>-
<td>A tensor split across cores so each slice fits in scratchpad</td>-
<td><code>construct_distributed_memory_view</code> over per-core <code>ct_local</code> views</td>-
</tr>-
<tr>-
<td>Core-to-core relayout (<code>lx_relayout</code>)</td>-
<td>An access tile over one distributed view, stored into a view with a different distribution</td>-
</tr>-
<tr>-
<td>Reduction split across cores (partial sums of one output tile)</td>-
<td><code>inter_tile_produce</code> / <code>inter_tile_reduce</code></td>-
</tr>-
<tr>-
<td>Paged tensors and gathers</td>-
<td><code>construct_indirect_access_tile</code></td>-
</tr>-
<tr>-
<td>Device-specific activations and normalization steps</td>-
<td><code>spyreop</code> intrinsics inside <code>linalg.generic</code> bodies</td>-
</tr>-
<tr>-
<td>Tensor addresses owned by PyTorch’s allocator</td>-
<td>Kernel arguments. They stay symbolic all the way to SpyreCode and are substituted by the runtime at launch</td>-
</tr>-
<tr>-
<td>Tensor shapes known only at runtime, within a bound</td>-
<td><code>!ktdp.runtime_arg<index, granularity = G, upperbound = U></code>, kept symbolic and substituted at launch in the same way</td>-
</tr>-
</tbody>-
</table>-
<p>The last two rows are where this post meets the runtime post. Because KTIR allows a start address or a size to be a value rather than a constant, every layer beneath it can keep that value symbolic until the runtime is in a position to supply it.</p>-
<h3>Tensors pinned in scratchpad, distributed across cores</h3>-
<p>What most sets KTIR apart from GPU-oriented tile IRs is the way it treats on-chip memory. A tile language like Triton describes each program instance loading blocks of a tensor from global memory, computing on them, and storing the results back. How those blocks are staged through shared memory, …-
<p>Each memory space names a physical memory: <code>#ktdp.memory_space<global></code> is device memory, and <code>#ktdp.memory_space<ct_local, ct_id = k></code> is the scratchpad belonging to compute tile <code>k</code>. A view’s <code>coordinate_set</code> records which part of a larger…-
<pre><code class="language-mlir">// X[128, 1024] f16, left in scratchpad by an earlier op of the kernel:
-
// 32 rows on each of four cores, at the offset the scratchpad allocator chose.
-
#rows0 = affine_set<(d0, d1) : (d0 >= 0, -d0 + 31 >= 0, d1 >= 0, -d1 + 1023 >= 0)>
-
#rows1 = affine_set<(d0, d1) : (d0 - 32 >= 0, -d0 + 63 >= 0, d1 >= 0, -d1 + 1023 >= 0)>
-
// ... #rows2, #rows3
-
-
%X0 = ktdp.construct_memory_view %lx_off, sizes: [32, 1024], strides: [1024, 1] {
-
coordinate_set = #rows0, memory_space = #ktdp.memory_space<ct_local, ct_id = 0>
-
} : memref<32x1024xf16>
-
%X1 = ktdp.construct_memory_view %lx_off, sizes: [32, 1024], strides: [1024, 1] {
-
coordinate_set = #rows1, memory_space = #ktdp.memory_space<ct_local, ct_id = 1>
-
} : memref<32x1024xf16>
-
// ... %X2, %X3
-
%X = ktdp.construct_distributed_memory_view (%X0, %X1, %X2, %X3
-
: memref<32x1024xf16>, memref<32x1024xf16>, memref<32x1024xf16>, memref<32x1024xf16>)
-
: memref<128x1024xf16>
-
-
// Compute tile 1 reads rows 48-79: half from its own scratchpad, half from core 2's.
-
#rows32 = affine_set<(d0, d1) : (d0 >= 0, -d0 + 31 >= 0, d1 >= 0, -d1 + 1023 >= 0)>
-
%id = ktdp.get_compute_tile_id : index
-
%row = affine.apply affine_map<(d0) -> (d0 * 32 + 16)>(%id)
-
%t = ktdp.construct_access_tile %X[%row, %c0] {
-
access_tile_set = #rows32, access_tile_order = #id
-
} : memref<128x1024xf16> -> !ktdp.access_tile<32x1024xindex>
-
%x = ktdp.load %t : !ktdp.access_tile<32x1024xindex> -> tensor<32x1024xf16>
-
</code></pre>-
<p><!-- IMAGE PLACEHOLDER: distributed-memory-view.png - upload via Media Library. Suggested alt text: "One logical tensor pinned across four cores' scratchpads with a distributed memory view" --></p>-
<p> </p>-
<p><em><a href="https://pytorch.org/wp-content/uploads/2026/10/ktir-distributed-view.svg"><img decoding="async" class="alignnone size-full wp-image-188754" src="https://pytorch.org/wp-content/uploads/2026/10/ktir-distributed-view.svg" alt="" /></a>A distributed memory view: one logical tensor, pinne…-
<p>This one construct does quite a lot of work. First, it lets intermediate tensors stay on-chip from one operation to the next: a producer can store into a distributed <code>ct_local</code> view and a consumer can load from it without either of them ever mentioning device memory, which is precisely…-
<h2>The backend compiler at a glance</h2>-
<p>The backend compiler is a single MLIR pipeline in two halves. The open-source <strong>Dataflow Scheduler</strong> lowers KTIR through <strong>ScheduleIR</strong> to <strong>DataflowIR</strong>, and three device-specific stages (symbolic SpyreBinary generation, metadata generation for runtime symb…-
<p><!-- IMAGE PLACEHOLDER: spyre-backend-compiler.png - upload via Media Library. Suggested alt text: "The Spyre backend compiler pipeline from KTIR to SpyreCode" --></p>-
<p><em><a href="https://pytorch.org/wp-content/uploads/2026/10/backend-pipeline.svg"><img decoding="async" class="alignnone size-full wp-image-188755" src="https://pytorch.org/wp-content/uploads/2026/10/backend-pipeline.svg" alt="" /></a>The Spyre backend compiler: KTIR in, SpyreCode out</em></p>-
<p>The rest of the post follows the figure from left to right, spending most of its time on the scheduler and then covering the three stages below it more briefly.</p>-
<h2>The Dataflow Scheduler: KTIR → ScheduleIR → DataflowIR</h2>-
<p>A dataflow accelerator is a distributed collection of compute engines and scratchpad memories connected by wide datapaths with FIFOs. It follows a <em>decoupled access-execute</em> design, in which moving data (DMA, scratchpad loads and stores) and computing on it happen on separate engines that …-
<p>The <a href="https://github.com/torch-spyre/dataflow-scheduler">Dataflow Scheduler</a> does this as an MLIR pass pipeline, and a handful of principles shape how it’s built. It is driven by an architecture description rather than hard-coded for one chip, which we look at more closely below. Every …-
<p>The passes fall into three groups, each with a fixed IR contract at its boundaries, so passes can be added or reordered within a group without disturbing the others.</p>-
<p><!-- IMAGE PLACEHOLDER: dataflow-scheduler-pass-groups.png - upload via Media Library. Suggested alt text: "Dataflow Scheduler pass groups and their IR contracts" --></p>-
<p><em><a href="https://pytorch.org/wp-content/uploads/2026/10/scheduler-pass-groups.svg"><img decoding="async" class="alignnone size-full wp-image-188756" src="https://pytorch.org/wp-content/uploads/2026/10/scheduler-pass-groups.svg" alt="" /></a>Dataflow Scheduler: three pass groups, one IR contra…-
<p>At four named <em>checkpoints</em> (<code>pre_mapping</code>, <code>pre_scheduling</code>, <code>post_scheduling</code> and <code>post_lowering</code>) the scheduler applies rewrite patterns written in PDL. Those patterns live in the architecture description rather than in the scheduler, which me…-
<p>The scheduler’s intermediate form, <strong>ScheduleIR</strong>, is the <code>ktdf</code> dialect used alongside <code>linalg</code>, <code>scf</code> and <code>memref</code>. Its vocabulary is small enough to list in full:</p>-
<table>-
<thead>-
<tr>-
<th><code>ktdf</code> construct</th>-
<th>Meaning</th>-
</tr>-
</thead>-
<tbody>-
<tr>-
<td><code>ktdf.pipeline [modulo(size: N)]</code></td>-
<td>A chain of stages that overlap across iterations; <code>modulo</code> rotates through N copies of its resources (N-buffering)</td>-
</tr>-
<tr>-
<td><code>ktdf.stage depends_in(...) depends_out(...)</code></td>-
<td>One step of a pipeline, eventually bound to one kind of execution unit (<code>applicable_units</code>)</td>-
</tr>-
<tr>-
<td><code>ktdf.private</code></td>-
<td>Resources private to one pipeline: FIFO slots, scratchpad buffers, tokens</td>-
</tr>-
<tr>-
<td><code>!ktdf.token</code></td>-
<td>A pure ordering value between stages, with no runtime cost of its own</td>-
</tr>-
<tr>-
<td><code>!ktdf.fifo.slot<SRC -> DST, 64xf16></code></td>-
<td>A slot in a hardware FIFO between two units</td>-
</tr>-
<tr>-
<td><code>ktdf.data_transfer</code> / <code>ktdf.ind_data_transfer</code></td>-
<td>Explicit movement between memories and FIFOs, direct or through an indirect-address buffer</td>-
</tr>-
<tr>-
<td><code>ktdf.read_from_fifo</code> / <code>ktdf.write_to_fifo</code></td>-
<td>How a compute stage consumes and produces FIFO data</td>-
</tr>-
<tr>-
<td><code>ktdf.parallel ... distribute(num_instances = N)</code></td>-
<td>Split an iteration space into contiguous blocks over N identical hardware instances</td>-
</tr>-
<tr>-
<td><code>ktdf.tiling.reserve_size</code> / <code>derive_size</code> / <code>linearize_index</code></td>-
<td>Placeholders for tile sizes chosen later, the current tile’s extent, and the original index</td>-
</tr>-
</tbody>-
</table>-
<h3>The architecture description</h3>-
<p>Everything the scheduler knows about the hardware comes from a single MLIR file written in the <code>ktdf_arch</code> dialect. That file describes the device as a graph. Its nodes are memories, with their kinds and sizes, and execution units, with the features the scheduler cares about: SIMD lane…-
<pre><code class="language-mlir">#L1 = { kind = "L1", size = 1048576 } // 1 MB scratchpad
-
#SFU = { kind = "SFU", ktdf_arch.features = {
-
ktdf_arch.feature.compute,
-
ktdf_arch.feature.simd = { lanes = #ktdf_arch.map<f16 = 64, f32 = 32, i32 = 32> } } }
-
#L1LU_CORE_FIFO = { ktdf_arch.features = { ktdf_arch.feature.queue = { depth = 16, ordered } } }
-
-
ktdf_arch.device @sample_device {
-
%ddr = memory #DDR
-
group { kind = "core" } share(%ddr) {
-
%l1 = memory #L1
-
%mnilu = group { kind = "MNILU_Block" } share() { ... } -> exec_unit // DMA load unit
-
%mnisu = group { kind = "MNISU_Block" } share() { ... } -> exec_unit // DMA store unit
-
datapath #DDR_MNILU %ddr to %mnilu : memory, exec_unit
-
datapath #MNILU_L1 %mnilu to %l1 : exec_unit, memory
-
%sfu = group { kind = "SFU_Block" } share() { ... } -> exec_unit // compute unit
-
%l1lu = exec_unit #L1LU // scratchpad load unit
-
%l1su = exec_unit #L1SU // scratchpad store unit
-
datapath #L1LU_CORE_FIFO %l1 to %l1lu : memory, exec_unit
-
datapath %l1lu to %sfu : exec_unit, exec_unit
-
...
-
}
-
patterns ["pre_scheduling"] { ... } // device-specific rewrites, in PDL
-
patterns ["post_scheduling"] { ... }
-
}
-
</code></pre>-
<p>A kernel refers to its target with a one-line declaration that imports this file and maps KTIR’s memory spaces onto the device’s memories, for example <code>global</code> onto <code>DDR</code> and <code>ct_local</code> onto <code>L1</code>. From then on, nearly every decision in the pipeline read…-
<p>Because all of that lives in the description rather than in the passes, retargeting the scheduler means writing a new description, not changing the scheduler. Spyre has its own description, the scheduler’s tests run against the sample device above, and the same passes run unchanged on both.</p>Diff display stops at 400 lines. The line counts above are from the whole diff. 41 lines shown here cut at 300 characters. The raw artifact at this commit is linked above.