Distributed Deep Learning
Distributed Deep Learning
Russell J. Hewett
computational scientist
Publications
Parallel Primitives for Domain Decomposition in Neural Networks
We develop a suite of parallel primitives for high-order tensors in distributed deep learning.
DistDL: Distributed Deep Learning for PyTorch
DistDL is a Python package for model parallel deep learning in PyTorch.
A Linear Algebraic Approach to Model Parallelism in Deep Learning
We present a linear algebraic model for distributed deep learning.
Talks
A Linear-algebraic Approach to Distributed Deep Learning
Training deep neural networks (DNNs) in large-cluster computing environments is increasingly necessary, as networks grow in size and …