Projects

Selected LLM inference, GPU kernel, PyTorch compiler, attention, and edge deployment projects with measured performance and correctness results.

LLM inference runtimes, GPU kernels, compiler performance analysis, and edge deployment. Featured results include their workload and baseline so each measurement has context.

Featured Systems Work

GPU Kernel Engineering GPU Kernel Optimization for LLM Inference FP32 RMSNorm and GEMV/small-M GEMM in CUDA C++ and Triton on RTX 5050. Residual-add/RMSNorm fusion reduced seven eager PyTorch launches to one at 256 × 4096; passed 80/80 backend-shape checks against PyTorch. 15.3–16.3× vs. earlier M=16 Triton projections · CUDA Graph medians

Inference Runtime nano-vLLM: Single-GPU LLM Runtime Token-budget continuous batching, chunked prefill, paged KV tables, and prefix caching with OpenAI-style serving. On 12 mixed-length Qwen3-0.6B requests, output throughput rose from 58 to 223 tokens/s versus single-sequence scheduling. 3.84× throughput · Prefix reuse: median TTFT 139 → 34 ms

Compiler & Performance Analysis PyTorch Compiler & LLM Inference FX capture, torch.cond, fusion analysis, and correctness-gated benchmarks with isolated caches and fresh-process profiling on RTX 5050. Investigated both pointwise speedups and a compiled MLP latency regression. Qwen2.5-0.5B FP32: 8-step decode 97.9 → 66.8 ms · CUDA Graphs off

Attention Kernels Triton FlashAttention-2: Forward Attention FA2-style forward attention with online softmax, FP16 MHA/GQA, and causal tile pruning. Passed 60/60 CUDA cases against PyTorch SDPA with maximum absolute output error 2.44 × 10⁻⁴. 1.15× vs. eager SDPA · Causal B=1, H=16, N=511, D=64 · RTX 5050

Edge AI · LPCVC 2026 MobileCLIP2 on Qualcomm QNN Static PyTorch → ONNX → QNN deployment of MobileCLIP2-S2, with FastViT reparameterization and preprocessing corrections. Validated all 56 image and 211 text device outputs at unchanged R@10 of 0.897470. Image/text P90: 14.51/3.99 ms · S22 Family · 794/794 profiled ops on NPU

Project Areas

Performance GPU Kernels CUDA C++, Triton, and CuTe DSL kernels for deep learning operators and memory-bound workloads. CUDA C++ Triton CuTe DSL

Serving Runtime A single-GPU LLM runtime with continuous batching, chunked prefill, and paged KV caching; further exploration of serving engines. vLLM SGLang TensorRT

Frameworks DL Framework Building a small PyTorch-style framework in Python and C++ with tensors, autograd, modules, and kernels. Python C++ Autograd

Compilation DL Compilers Measured torch.compile behavior, FX graph capture, fusion, and graph breaks, alongside MLIR, XLA, and TVM explorations. MLIR XLA torch.compile

Measurement Profiling Nsight Systems, Nsight Compute, PyTorch Profiler, memory traces, and reproducible bottleneck analysis. Nsight PyTorch Perfetto

Efficiency Inference Optimization Measured batching, prefix-cache reuse, quantized loading, and compiler tradeoffs for latency, throughput, and memory efficiency. KV Cache Spec Decode Batching

Scale-Out Systems Distributed Inference & Parallelism Collective communication and model-parallel execution for multi-GPU and multi-node inference. Collectives TP / PP / EP Multi-node

Architectures Transformers Transformer implementations in NumPy and PyTorch, including attention variants and efficient architectures. Attention PyTorch NumPy

Generative Models Diffusion Denoising models, schedulers, U-Nets, latent diffusion, and efficient sampling experiments. DDPM U-Net Sampling

Learning Systems Reinforcement Learning Policy gradients, actor-critic methods, value learning, and RL systems experiments. PPO DQN Actor-Critic

Foundations DNN from Scratch Core neural network building blocks from scratch, from ANN and CNN layers to training loops. NumPy Backprop Training

Deployment Edge AI & Deployment MobileCLIP2 deployment on Qualcomm QNN, graph export, precision tradeoffs, and on-device accuracy and latency validation. ONNX QNN NPU