llama-3.1下载摆设

下载

huggingface
详情页填写申请后等待审核

点击 头像->setting->access token 创建token

配置环境变量

下载模型

1. pip install -U huggingface_hub

复制代码

1. huggingface-cli download --resume-download meta-llama/Meta-Llama-3.1-8B-Instruct --local-dir E:\codes\model\meta-llama\Meta-Llama-3.1-8B-Instruct --local-dir-use-symlinks False --resume-download --token xxxxx  

复制代码

对于Linux系统

1. export HF_ENDPOINT=https://hf-mirror.com

复制代码

1. huggingface-cli download --resume-download meta-llama/Meta-Llama-3.1-8B-Instruct --local-dir /home/model/meta-llama/Meta-Llama-3.1-8B-Instruct --local-dir-use-symlinks False --resume-download --token xxxxx  

复制代码

使用wget一次下载单个文件

1. wget --header "Authorization: Bearer 你的token" https://hf-mirror.com/meta-llama/Meta-Llama-3.1-8B/resolve/main/model-00003-of-00004.safetensors

复制代码

摆设

环境python3.10

1. pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu121

复制代码

1. pip install transformers==4.43.2 numpy==1.26.4 bitsandbytes==0.43.3 accelerate==0.33.0 -i https://pypi.tuna.tsinghua.edu.cn/simple

复制代码

1. from transformers import pipeline
2. import torch
4. model_id = r"E:\codes\model\meta-llama\Meta-Llama-3.1-8B-Instruct"
6. # pipeline = pipeline(
7. #     "text-generation",
8. #     model=model_id,
9. #     model_kwargs={"torch_dtype": torch.bfloat16},
10. #     device_map="auto",
11. # )
12. pipeline = pipeline(
13.     "text-generation",
14.     model=model_id,
15.     model_kwargs={
16.         "torch_dtype": torch.bfloat16,
17.         "quantization_config": {"load_in_4bit": True}
18.     },
19. )
22. messages = [
23.     {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
24.     {"role": "user", "content": "Who are you?"},
25. ]
27. outputs = pipeline(
28.     messages,
29.     max_new_tokens=256,
30. )
31. print(outputs[0]["generated_text"][-1])

复制代码

免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。