WildfireDetection/apex/examples/simple/distributed

distributed_data_parallel.py and run.sh show an example using Amp with apex.parallel.DistributedDataParallel or torch.nn.parallel.DistributedDataParallel and the Pytorch multiprocess launcher script, torch.distributed.launch. The use of Amp with DistributedDataParallel does not need to change from ordinary single-process use. The only gotcha is that wrapping your model with DistributedDataParallel must come after the call to amp.initialize. Test via

bash run.sh

This is intended purely as an instructional example, not a performance showcase.