3

TAP: Efficient Derivation of Tensor Parallel Plans for Large Neural Networks
We present a framework that drastically speeds up the process of deriving the tensor parallel schedule for large neural networks.
Ziji Shi, Le Jiang, Jie Zhang, Xianyan Jia, Yong Li, Chencan Wu, Jialin Li, Wei Lin
PDF Cite Poster
TAP: Efficient Derivation of Tensor Parallel Plans for Large Neural Networks