[Bug] Model Breaking Dynamic Cache
#69
by kylesayrs - opened
The line max_cache_length = past_key_values.get_max_length() will be incompatible with the next release of transformers. The line needs to be changed to use past_key_values.get_max_cache_shape()
kylesayrs changed discussion title from [Bug] Dynamic Cache to [Bug] Model Breaking Dynamic Cache