Temperature and top_p values are swapped in the example code:
#2
by sszymczyk - opened
At least comparing to the values recommended in the model card. We have:
top_p = 0.95
top_k = 50
min_p = 0.0
temperature = 0.8
But in the code there is:
response = client.chat.completions.create(
model=model_name,
messages=messages,
max_tokens=32768,
temperature=0.95,
top_p=0.8,
extra_body=dict(spaces_between_special_tokens=False),
tools=tools)
Also I see that in generation_config.json there's another set of values:
"top_p": 0.8,
"top_k": 20,
"temperature": 0.7,
Thanks for your reminder. We have fixed this issue.
RangiLyu changed discussion status to closed