Linux情况下部署阿里通义千问开源AI大模型-基于厚德云
算力情况设置参考：Linux情况下部署百度百川AI大模型-基于厚德云-CSDN博客 服务器情况
显存：24 GB
CPU：16 核
内存：100 GB
总存储: 160 GB
软件情况
git
python 3.10.13
pytorch 2.4.0
算力平台
厚德云
步调一：下载Qwen模型
mkdir /workspace/
cd /workspace/
gitclone https://github.moeyy.xyz/https://github.com/QwenLM/Qwen 步调二：下载依赖
cd Qwen
pip install -r requirements.txt
pip install -r requirements_web_demo.txt
pip install bitsandbytes==0.41.1
pip install accelerate==0.25.0
pip install streamlit
sudo apt update
sudo apt install git-lfs 步调三：下载模型库
mkdir Qwen
cd Qwen
git lfs clone https://www.modelscope.cn/ccyh123/Qwen-7B-Chat.git 步调四：启动并应用大模型
cd /workspace/Qwen/
python cli_demo.py https://i-blog.csdnimg.cn/direct/05a2c9f0d1c0409981372ba760d43497.png
https://i-blog.csdnimg.cn/direct/7aabe5c9581e4839a1282a0cebdc171f.png
步调五：网页版启动
cd /workspace/Qwen/
streamlit run web_demo.py https://i-blog.csdnimg.cn/direct/a2409236d2d44abab2f231f444284158.png
https://i-blog.csdnimg.cn/direct/f958490790ec4bc7bfa68401a597ef58.png
https://i-blog.csdnimg.cn/direct/a452b8787f3f4d458ad607a218b951e9.png
步调六：使用Qwen大模型
Streamlithttps://csdnimg.cn/release/blog_editor_html/release2.3.6/ckeditor/plugins/CsdnLink/icons/icon-default.png?t=N7T8http://211.93.18.69:64123/
或
Streamlithttps://csdnimg.cn/release/blog_editor_html/release2.3.6/ckeditor/plugins/CsdnLink/icons/icon-default.png?t=N7T8http://zhy1.dc.houdeyun.cn:64123/
FAQ
1、OutOfMemoryError: CUDA out of memory. Tried to allocate 64.00 MiB. GPU 0 has a total capacity of 23.65 GiB of which 15.81 MiB is free. Including non-PyTorch memory, this process has 23.63 GiB memory in use. Of the allocated memory 23.25 GiB is allocated by PyTorch, and 328.50 KiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
解决方案：
显存24G内存不足，升级GPU
2、OSError: Cannot find empty port in range: 8000-8000. You can specify a different port by setting the GRADIO_SERVER_PORT environment variable or passing the `server_port` parameter to `launch()`.
IMPORTANT: You are using gradio version 3.41.2, however version 4.29.0 is available, please upgrade.
解决方案
修改/workspace/Qwen/web_demo.py文件的第31行，端标语改成别的，比如8010
3、 下载 https://github.com/QwenLM/Qwen 时间让输入用户名和密码
缘故原由：仓库不存在，请查看 https://github.com/QwenLM/Qwen仓库地址是否存在
解决方案：
如果不存在，可以查找别的仓库源码，以下为gitee上查找方式
Gitee Search
https://i-blog.csdnimg.cn/direct/a3d0628f28154fd4b153de213cbe3d3a.png
git clone https://gitee.com/qzl66/Qwen
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。
页:
[1]