Note: Most of the strategies introduced in the single GPU section (such as mixed precision training or gradient accumulation) and multi-GPU section are generic and apply to training models in general so make sure to have a look at it before diving into this section.