# Accelerate FSDP training || RuntimeError : Forward oder differ across ranks

**URL:** <https://discuss.huggingface.co/t/accelerate-fsdp-training-runtimeerror-forward-oder-differ-across-ranks/66290>\
**Category:** 🤗Accelerate\
**Created:** [December 19, 2023, 6:51am UTC](https://discuss.huggingface.co/t/accelerate-fsdp-training-runtimeerror-forward-oder-differ-across-ranks/66290 "2023-12-19T06:51:29Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![SRINATH](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/srinath/32/21429_2.png) [@SRINATH](https://discuss.huggingface.co/u/SRINATH)\
**Post date:** [December 19, 2023, 6:51am UTC](https://discuss.huggingface.co/t/accelerate-fsdp-training-runtimeerror-forward-oder-differ-across-ranks/66290/1 "2023-12-19T06:51:29Z")

</div>

While training gpt2-large model with wikitext dataset using accelerate fsdp configuration , below error is being seen

```auto
RuntimeError: Forward order differs across ranks: rank 0 is all-gathering 1 parameters while rank 3 is all-gathering -241381119 parameters

```