Set CTX Length to 2048
#19
by michaelfeil - opened
No description provided.
This model has an 8k context length. This change appears incorrect.
michaelfeil changed pull request status to closed
right, we need to also truncate the context length of the abs pos embeddings vector for bert.