Geo-distributed training works on Akash
Akash Mainnet 6 Livestream (Akash Network)
“Yes, it is possible to distribute the training across different GPUs. In fact there is a proposal on Akash discussions talking about how they plan to use about 24,000 A100 GPU hours to train across a distributed cluster… so yes, it is possible to train on geographically distributed clusters on Akash.” — 01:20:48
Context: Answering a chat question on multi-GPU training; Greg caveats it depends on batch sizes, parallelization strategy, gradient compression, and inter-cluster latency.