Text Generation
Transformers
PyTorch
Safetensors
English
mistral
LLMs
Intel
Eval Results (legacy)
text-generation-inference
Instructions to use INC4AI/neural-chat-7b-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use INC4AI/neural-chat-7b-v3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="INC4AI/neural-chat-7b-v3")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("INC4AI/neural-chat-7b-v3") model = AutoModelForCausalLM.from_pretrained("INC4AI/neural-chat-7b-v3", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use INC4AI/neural-chat-7b-v3 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "INC4AI/neural-chat-7b-v3" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "INC4AI/neural-chat-7b-v3", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/INC4AI/neural-chat-7b-v3
- SGLang
How to use INC4AI/neural-chat-7b-v3 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "INC4AI/neural-chat-7b-v3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "INC4AI/neural-chat-7b-v3", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "INC4AI/neural-chat-7b-v3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "INC4AI/neural-chat-7b-v3", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use INC4AI/neural-chat-7b-v3 with Docker Model Runner:
docker model run hf.co/INC4AI/neural-chat-7b-v3
Commit History
-
Adding `safetensors` variant of this model (#9)
Verified signature- Adding `safetensors` variant of this model (6177ad5025422b9bdcc281cb4e5dac839605d3e8) Co-authored-by: Safetensors convertbot <SFconvertbot@users.noreply.huggingface.co>7f6ebc1
-
Update README.md
Verified signaturefc67927
-
Update README.md
Verified signatureb6e07e7 -
Update README.md
Verified signaturee1c02f9 -
Update README.md
Verified signatureUse bf16 compute typea03a0fe
-
Update README.md (#8)
Verified signature- Update README.md (8a54954a16b014512fc09a74dcb9a15be01b29ad) Co-authored-by: Benjamin Consolvo <bconsolvo@users.noreply.huggingface.co>959607f
-
add license.
lvkaokaocc14d25
-
Update README.md
31af3c3
-
add prompt template.
lvkaokao735bc4c -
Merge branch 'main' of https://huggingface.co/Intel/neural-chat-7b-v3 into main
lvkaokaob268508 -
add prompt template.
lvkaokao2382cb8 -
update hyper-parameters
lvkaokao7cc4991
-
Update README.md
4c89b7a -
Merge branch 'main' of https://huggingface.co/Intel/neural-chat-7b-v3 into main
lvkaokao830e8d3 -
update blog.
lvkaokaof6e1e39
-
Update README.md
af71dde -
update train args.
lvkaokao7da01a4 -
update metric from llm leaderboard.
lvkaokao050b53d -
update metric from llm leaderboard.
lvkaokao8131293 -
update metric from llm leaderboard.
lvkaokao70b518b
-
update model.
lvkaokao7a05c8a
-
Update README.md
972bfe2 -
Update README.md
bea6297 -
Merge branch 'main' of https://huggingface.co/Intel/neural-chat-7b-v3 into main
lvkaokao0f706cb -
update doc.
lvkaokaocb7ae0b -
Update README.md
27688f3 -
update doc.
lvkaokao7e24c55 -
update doc.
lvkaokao09710bd -
upload
lvkaokao001cba8 -
add doc.
lvkaokaoc467002 -
initial commit
7638119