Document Index¶

Get Started¶

Document	Content
Get Started	Build and install DTorch, then verify distributed inference of the diffusion models (SD3 / FLUX)
How To Build	Build and development environment setup
Testing Guide	Python unit tests, C++ unit tests (gtest), application tests (Flux/SD3)

Document	Content
User Guide	Getting started with DTorch: first understand the core DTensor concepts, then learn to write distributed programs with the Python API
Distributed Tensor Overview	An introduction to DTensor's DeviceMesh and Placements concepts
Python API Overview	Tensor creation, operator calls, native DTensor support, Placements inference, Redistribute, single-device distributed simulation, async value retrieval
Module Parallel	Linear's tp_dim / tp_shard_type, the ColumnParallelLinear / RowParallelLinear subclasses, DP/TP/CP/PP usage at the Module level
Llama Parallel Example	Using the Llama model as an example, showing the complete DP + TP + PP + CP implementation (layer-to-stage mapping, RoPE and CP, parallel strategy testing)
Diffusion Model Inference Application	Directory structure, DTorch API adaptation changes, ExecuteConfig, distribution, Cache, quantization, operator fusion

Document	Content
Project Overview	An overview of the layered architecture and directory structure
Key Concepts	Detailed explanations of the three core design concepts (Single-Client Single-Controller Multi-Worker, DTensor, Eager Graph Architecture)
Design Decisions	Key design decisions: why DTensor + Single-Controller, how the scheduling overhead is addressed, the LibTorch backend, etc.
Single-Controller Architecture	The Single-Client Single-Controller Multi-Worker architecture in detail, compared with Multi-Controller
Distributed Tensor	DTensor's DeviceMesh and Placements mechanisms, with complete code examples

Document	Content
Architecture Overview	The four-layer architecture (graph representation → Kernel runtime → GraphExecutor → collective communication components)
Layer 1 · Graph Representation	Operand / Operator / LogicalGraph
Layer 2 · Kernel Runtime	Blob / Kernel / KernelStream / OperatorAssignInfo
Layer 3 · GraphExecutor	Single-machine multi-thread (GraphConstructor → EagerGraphExecutor → PerDeviceThreadNodeRunner → NaiveRunner)
Layer 3 · Cluster Mode	Single-machine multi-process (RemoteRunnerPublisher, RemoteRunnerInProcess, PerDeviceProcessNodeRunner, SubProcess)
Layer 4 · Collective Communication	ThreadGroup / TensorStore (Memory/File/Network three backends)
Async Tensor Value Retrieval	GetTensorOp and the Promise/Future mechanism: `to_torch_async()` / TensorFuture
Python Kernel	Calling Python code in C++ Kernels (GIL management, CUDA Stream protection, type conversion)
Serialization	Operator serialization and deserialization: the Boost.Serialization system, OperatorSerializationPack, cross-process transfer
Process Heartbeat	gRPC bidirectional heartbeat, MainProcessHeartBeat / WorkerProcessHeartBeat, process failure detection and graceful shutdown
ZMQ Communication	The PUB-SUB + PUSH-PULL dual channels, message protocol and ordering guarantees
Stream Race Condition	CUDA Stream synchronization mechanisms and cross-stream race conditions

Document	Content
Operator System Overview	An overview of the Operator system
Operator Base Class and Derived Classes	The base class structure and the derived class hierarchy (standard / fused_compile / system)
Template Method Pattern	The algorithm skeleton and all overridable virtual functions explained
PlacementSignature	The distributed Placements mapping rules (Builder API, typical implementation patterns, the matching flow)
Operator Mapping Table	The three-layer mapping table: Python ↔ C++ API ↔ C++ core operators
How To Add an Operator	The five-step flow: comments → core operator → Kernel → API → tests
OperatorCost Estimation	FLOPs / bandwidth estimation, the base data for roofline analysis

Document	Content
Debugging Output Mismatch	Locating the root cause of output mismatches between DTorch and PyTorch models, from coarse to fine
Precision Alignment	Known scenarios of DTorch vs PyTorch output mismatch and the workarounds
Performance Optimization	Analysis and optimization of performance problems such as CUDA Kernel Launch