SPIMOE accepted by ICCAD 2026: hybrid sparsity for reasoning MoE inference on heterogeneous PIM architectures

👏 Paper title: SPIMOE: Exploiting Hybrid Sparsity for Reasoning MoE Inference on Heterogeneous PIM Architectures.

SPIMOE targets efficient long-chain reasoning inference for Mixture-of-Experts (MoE) large language models. As reasoning sequences become longer, the growing KV cache shifts the inference bottleneck toward attention, while sparse and irregular expert activation causes load imbalance and inefficient hardware utilization.

At the algorithm level, SPIMOE introduces adaptive expert routing and block-sparse attention with physical KV-cache eviction. Adaptive routing prunes low-contribution experts according to reasoning phase and layer depth while protecting reasoning-critical cognitive experts. Block-sparse attention selects only important sink, local, and high-score KV blocks, and physically evicts obsolete cache entries at semantic reasoning boundaries to reduce both attention computation and KV-cache capacity requirements.

At the architecture level, SPIMOE adopts Attention-FFN Disaggregation on a heterogeneous PIM platform. Attention is executed on SRAM-PIM for low-latency, access-intensive processing, while MoE FFN computation is mapped to HBM-PIM to exploit high internal bandwidth and bank-level parallelism. Static expert mapping and dynamic sub-batch scheduling further improve load balance and overlap attention and MoE execution across the two subsystems.

Experimental results show that SPIMOE achieves up to 8.35x speedup over an NVIDIA A100 GPU and 3.33x speedup over the prior PIM-based MoE accelerator PIMoE, while preserving reasoning accuracy comparable to full-precision baselines. The results demonstrate the potential of jointly combining reasoning-aware sparsity with heterogeneous PIM design for scalable sparse LLM inference.