Distributed Inference & Parallelism

Scale-out inference projects around communication primitives, process groups, parallelism strategies, placement, and bandwidth-aware serving.

Collective Communication all-reduce, all-gather, reduce-scatter, all-to-all, broadcast, process groups, and async communication

Tensor Parallelism Sharding attention and MLP layers across GPUs for large-model inference

Pipeline Parallelism Layer partitioning, microbatching, stage scheduling, and cross-node serving

Expert Parallelism MoE expert placement, token routing, all-to-all dispatch, and load balancing

Context Parallelism Long-context sequence sharding for attention and KV-cache-heavy workloads

Data / Replica Parallelism Serving replicas, request routing, throughput scaling, and failover

Topology-Aware Serving Placement, communication overlap, bandwidth bottlenecks, and multi-node profiling