Instructions to use CohereLabs/c4ai-command-r-plus with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use CohereLabs/c4ai-command-r-plus with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="CohereLabs/c4ai-command-r-plus")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("CohereLabs/c4ai-command-r-plus")
model = AutoModelForCausalLM.from_pretrained("CohereLabs/c4ai-command-r-plus")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use CohereLabs/c4ai-command-r-plus with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "CohereLabs/c4ai-command-r-plus"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CohereLabs/c4ai-command-r-plus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/CohereLabs/c4ai-command-r-plus

SGLang

How to use CohereLabs/c4ai-command-r-plus with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "CohereLabs/c4ai-command-r-plus" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CohereLabs/c4ai-command-r-plus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "CohereLabs/c4ai-command-r-plus" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "CohereLabs/c4ai-command-r-plus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use CohereLabs/c4ai-command-r-plus with Docker Model Runner:
```
docker model run hf.co/CohereLabs/c4ai-command-r-plus
```

This is the greatest AI chat model yet

#30

by JJJJJPSYCHIC - opened Apr 13, 2024

Discussion

JJJJJPSYCHIC

Apr 13, 2024

And it's not even close. This is the one to keep.

jukofyork

Apr 15, 2024

•

edited Apr 15, 2024

Yeah, it's pretty dam good at story writing too!

I nearly dismissed it as its 1-shot stories are pretty terrible, but it does a good job if asked to self-critique and refine...

Goliath-120B on the other hand wrote good 1-shot Grimdark stories but would refuse to listen and just wanted to carry on writing more and more.

VERY impressed!

(it does suck at coding though)

TheAGames10

Apr 16, 2024

I have started getting a "model overloaded" as of today, and I have no clue why it says this. I thought it meant too many people were using it, but it feels that message would come up before today as well, so it's not that...

I tried to ask itself why it gave the message, but it still gave the same message. I was able to say test and it worked, but then it gave the message after I tried another response.

ewre324

Apr 19, 2024

Extremely Intelligent model, adept at logic, answer "which is heavier 1 kg feather or 2 kg feather?" correctly, while Llama 3 failed

lenML

May 2, 2024

I strongly agree, but what puzzles me is that this model’s MMLU score is lower than that of Llama3 70B, which is clearly different from my tests (chat testing). This model’s capabilities far exceed those of Llama3, and I even believe that this model surpasses Claude3 sonet in abilities.

Aillian

May 2, 2024

Anyone tested this model as an agent?

scapking

May 9, 2024

(编写代码能力，不如OpenAi Gpt3.5，推理能力也不如Gpt3.5，如果GPT-3.5编写代码能力都不如，我想说这个怎么和GPT-4相比？)

我认为他需要重新训练，比如（回香炉重新训练）

alexrs changed discussion status to closed Jun 12, 2025

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment