llama-index调用qwen大模子实现RAG

配景

llama-
index在实现RAG方案的时间多是用的llama等英文大模子，对于国内的诸多模子案例较少，本次将利用qwen大模子实现llama-
index的RAG方案。
情况设置

（1）pip包

llama
index须要预装许多包，这里先把我乐成的案例内里的pip包设置发出来，在requirements.txt内里。

2. absl-py==1.4.0
3. accelerate==0.27.2
4. aiohttp==3.9.3
5. aiosignal==1.3.1
6. aliyun-python-sdk-core==2.13.36
7. aliyun-python-sdk-kms==2.16.1
8. annotated-types==0.6.0
9. anyio==3.7.1
10. apphub @ file:///environment/apps/apphub/dist/apphub-1.0.0.tar.gz#sha256=260f99c0de4c575b19ab913aa134877e9efd81b820b97511fc8379674643c253
11. argon2-cffi==21.3.0
12. argon2-cffi-bindings==21.2.0
13. asgiref==3.7.2
14. asttokens==2.2.1
15. astunparse==1.6.3
16. async-timeout==4.0.3
17. attrs==23.1.0
18. Babel==2.12.1
19. backcall==0.2.0
20. backoff==2.2.1
21. bcrypt==4.1.2
22. beautifulsoup4==4.12.3
23. bleach==6.0.0
24. boltons @ file:///croot/boltons_1677628692245/work
25. brotlipy==0.7.0
26. bs4==0.0.2
27. build==1.1.1
28. cachetools==5.3.1
29. certifi @ file:///croot/certifi_1690232220950/work/certifi
30. cffi @ file:///croot/cffi_1670423208954/work
31. chardet==3.0.4
32. charset-normalizer @ file:///tmp/build/80754af9/charset-normalizer_1630003229654/work
33. chroma-hnswlib==0.7.3
34. chromadb==0.4.24
35. click==7.1.2
36. cmake==3.25.0
37. coloredlogs==15.0.1
38. comm==0.1.4
39. conda @ file:///croot/conda_1690494963117/work
40. conda-content-trust @ file:///tmp/abs_5952f1c8-355c-4855-ad2e-538535021ba5h26t22e5/croots/recipe/conda-content-trust_1658126371814/work
41. conda-libmamba-solver @ file:///croot/conda-libmamba-solver_1685032319139/work/src
42. conda-package-handling @ file:///croot/conda-package-handling_1685024767917/work
43. conda_package_streaming @ file:///croot/conda-package-streaming_1685019673878/work
44. contourpy==1.2.0
45. crcmod==1.7
46. cryptography @ file:///croot/cryptography_1686613057838/work
47. cycler==0.12.1
48. dataclasses-json==0.6.4
49. debugpy==1.6.7
50. decorator==5.1.1
51. defusedxml==0.7.1
52. Deprecated==1.2.14
53. dirtyjson==1.0.8
54. distro==1.9.0
55. ecdsa==0.18.0
56. exceptiongroup==1.1.2
57. executing==1.2.0
58. fastapi==0.104.1
59. fastjsonschema==2.18.0
60. featurize==0.0.24
61. filelock==3.9.0
62. flatbuffers==23.5.26
63. fonttools==4.44.0
64. frozenlist==1.4.1
65. fsspec==2024.2.0
66. gast==0.4.0
67. google-auth==2.22.0
68. google-auth-oauthlib==1.0.0
69. google-pasta==0.2.0
70. googleapis-common-protos==1.62.0
71. greenlet==3.0.3
72. grpcio==1.62.0
73. gunicorn==21.2.0
74. h11==0.14.0
75. h5py==3.9.0
76. httpcore==0.17.3
77. httptools==0.6.1
78. httpx==0.24.1
79. huggingface-hub==0.20.3
80. humanfriendly==10.0
81. idna==2.10
82. imageio==2.32.0
83. importlib-metadata==6.11.0
84. importlib_resources==6.1.3
85. ipykernel==6.25.0
86. ipython==8.14.0
87. ipython-genutils==0.2.0
88. ipywidgets==8.1.2
89. jedi==0.19.0
90. Jinja2==3.1.2
91. jmespath==0.10.0
92. joblib==1.3.2
93. json5==0.9.14
94. jsonpatch @ file:///tmp/build/80754af9/jsonpatch_1615747632069/work
95. jsonpointer==2.1
96. jsonschema==4.18.6
97. jsonschema-specifications==2023.7.1
98. jupyter-server==1.24.0
99. jupyter_client==8.3.0
100. jupyter_core==5.3.1
101. jupyterlab==3.2.9
102. jupyterlab-pygments==0.2.2
103. jupyterlab_server==2.24.0
104. jupyterlab_widgets==3.0.10
105. keras==2.13.1
106. kiwisolver==1.4.5
107. kubernetes==29.0.0
108. lazy_loader==0.3
109. libclang==16.0.6
110. libmambapy @ file:///croot/mamba-split_1685993156657/work/libmambapy
111. lit==15.0.7
112. llama-
113. index==0.10.17
114. llama-
115. index-agent-openai==0.1.5
116. llama-
117. index-cli==0.1.8
118. llama-
119. index-core==0.10.17
120. llama-
121. index-embeddings-huggingface==0.1.4
122. llama-
123. index-embeddings-openai==0.1.6
124. llama-
125. index-indices-managed-llama-cloud==0.1.3
126. llama-
127. index-legacy==0.9.48
128. llama-
129. index-llms-huggingface==0.1.3
130. llama-
131. index-llms-openai==0.1.7
132. llama-
133. index-multi-modal-llms-openai==0.1.4
134. llama-
135. index-program-openai==0.1.4
136. llama-
137. index-question-gen-openai==0.1.3
138. llama-
139. index-readers-file==0.1.8
140. llama-
141. index-readers-llama-parse==0.1.3
142. llama-
143. index-vector-stores-chroma==0.1.5
144. llama-parse==0.3.8
145. llama
146. index-py-client==0.1.13
147. Markdown==3.4.4
148. MarkupSafe==2.1.2
149. marshmallow==3.21.1
150. matplotlib==3.8.1
151. matplotlib-inline==0.1.6
152. mistune==3.0.1
153. mmh3==4.1.0
154. monotonic==1.6
155. mpmath==1.2.1
156. multidict==6.0.4
157. mypy-extensions==1.0.0
158. nbclassic==0.2.8
159. nbclient==0.8.0
160. nbconvert==7.7.3
161. nbformat==5.9.2
162. nest-asyncio==1.6.0
163. networkx==3.0
164. nltk==3.8.1
165. notebook==6.4.12
166. numpy==1.24.1
167. nvidia-cublas-cu12==12.1.3.1
168. nvidia-cuda-cupti-cu12==12.1.105
169. nvidia-cuda-nvrtc-cu12==12.1.105
170. nvidia-cuda-runtime-cu12==12.1.105
171. nvidia-cudnn-cu12==8.9.2.26
172. nvidia-cufft-cu12==11.0.2.54
173. nvidia-curand-cu12==10.3.2.106
174. nvidia-cusolver-cu12==11.4.5.107
175. nvidia-cusparse-cu12==12.1.0.106
176. nvidia-nccl-cu12==2.19.3
177. nvidia-nvjitlink-cu12==12.4.99
178. nvidia-nvtx-cu12==12.1.105
179. oauthlib==3.2.2
180. onnxruntime==1.17.1
181. openai==1.13.3
182. opencv-python==4.8.1.78
183. opentelemetry-api==1.23.0
184. opentelemetry-exporter-otlp-proto-common==1.23.0
185. opentelemetry-exporter-otlp-proto-grpc==1.23.0
186. opentelemetry-instrumentation==0.44b0
187. opentelemetry-instrumentation-asgi==0.44b0
188. opentelemetry-instrumentation-fastapi==0.44b0
189. opentelemetry-proto==1.23.0
190. opentelemetry-sdk==1.23.0
191. opentelemetry-semantic-conventions==0.44b0
192. opentelemetry-util-http==0.44b0
193. opt-einsum==3.3.0
194. orjson==3.9.15
195. oss2==2.18.1
196. overrides==7.7.0
197. packaging @ file:///croot/packaging_1678965309396/work
198. pandas==2.1.2
199. pandocfilters==1.5.0
200. parso==0.8.3
201. pexpect==4.8.0
202. pickleshare==0.7.5
203. Pillow==9.3.0
204. platformdirs==3.10.0
205. pluggy @ file:///tmp/build/80754af9/pluggy_1648024709248/work
206. posthog==3.5.0
207. prometheus-client==0.17.1
208. prompt-toolkit==3.0.39
209. protobuf==4.23.4
210. psutil==5.9.5
211. ptyprocess==0.7.0
212. pulsar-client==3.4.0
213. pure-eval==0.2.2
214. pyasn1==0.5.0
215. pyasn1-modules==0.3.0
216. pycosat @ file:///croot/pycosat_1666805502580/work
217. pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work
218. pycryptodome==3.18.0
219. pydantic==2.4.2
220. pydantic_core==2.10.1
221. Pygments==2.15.1
222. PyMuPDF==1.23.26
223. PyMuPDFb==1.23.22
224. pyOpenSSL @ file:///croot/pyopenssl_1677607685877/work
225. pyparsing==3.1.1
226. pypdf==4.1.0
227. PyPika==0.48.9
228. pyproject_hooks==1.0.0
229. PySocks @ file:///home/builder/ci_310/pysocks_1640793678128/work
230. python-dateutil==2.8.2
231. python-dotenv==1.0.0
232. pytz==2023.3.post1
233. PyYAML==6.0.1
234. pyzmq==25.1.0
235. referencing==0.30.0
236. regex==2023.12.25
237. requests==2.31.0
238. requests-oauthlib==1.3.1
239. rpds-py==0.9.2
240. rsa==4.9
241. ruamel.yaml @ file:///croot/ruamel.yaml_1666304550667/work
242. ruamel.yaml.clib @ file:///croot/ruamel.yaml.clib_1666302247304/work
243. safetensors==0.4.2
244. scikit-image==0.22.0
245. scikit-learn==1.3.2
246. scipy==1.11.3
247. seaborn==0.13.0
248. Send2Trash==1.8.2
249. six @ file:///tmp/build/80754af9/six_1644875935023/work
250. sniffio==1.3.0
251. socksio==1.0.0
252. soupsieve==2.4.1
253. SQLAlchemy==2.0.28
254. sshpubkeys==3.3.1
255. stack-data==0.6.2
256. starlette==0.27.0
257. sympy==1.11.1
258. tabulate==0.8.7
259. tenacity==8.2.3
260. tensorboard==2.13.0
261. tensorboard-data-server==0.7.1
262. tensorflow==2.13.0
263. tensorflow-estimator==2.13.0
264. tensorflow-io-gcs-filesystem==0.33.0
265. termcolor==2.3.0
266. terminado==0.17.1
267. threadpoolctl==3.2.0
268. tifffile==2023.9.26
269. tiktoken==0.6.0
270. tinycss2==1.2.1
271. tokenizers==0.15.2
272. tomli==2.0.1
273. toolz @ file:///croot/toolz_1667464077321/work
274. torch==2.2.1
275. torchaudio==2.0.2+cu118
276. torchvision==0.15.2+cu118
277. tornado==6.3.2
278. tqdm==4.66.2
279. traitlets==5.9.0
280. transformers==4.38.2
281. triton==2.2.0
282. typer==0.9.0
283. typing-inspect==0.9.0
284. typing_extensions==4.8.0
285. tzdata==2023.3
286. urllib3==1.25.11
287. uvicorn==0.23.2
288. uvloop==0.19.0
289. watchfiles==0.21.0
290. wcwidth==0.2.5
291. webencodings==0.5.1
292. websocket-client==1.2.1
293. websockets==12.0
294. Werkzeug==2.3.6
295. widgetsnbextension==4.0.10
296. workspace @ file:///home/featurize/work/workspace/dist/workspace-0.1.0.tar.gz#sha256=b292beb3599f79d3791771eff9dc422cc37c58c1fc8daadeafbf025a2e7ea986
297. wrapt==1.15.0
298. yarl==1.9.2
299. zipp==3.17.0
300. zstandard @ file:///croot/zstandard_1677013143055/work

复制代码

（2）python 情况

（3）安装下令

2. !pip install llama-
3. index
4. !pip install llama-
5. index-llms-huggingface
6. !pip install llama-
7. index-embeddings-huggingface
8. !pip install llama-
9. index ipywidgets
10. !pip install torch
11. !git clone https://www.modelscope.cn/AI-ModelScope/bge-small-zh-v1.5.git
12. !git clone https://www.modelscope.cn/qwen/Qwen1.5-4B-Chat.git

复制代码

（4）目次布局

代码 

（1）加载模子

2. import torch
3. from llama_
4. index.llms.huggingface import HuggingFaceLLM
5. from llama_
6. index.core import PromptTemplate
7. import os
8. os.environ['KMP_DUPLICATE_LIB_OK']='True'
9. # Model names (make sure you have access on HF)
10. LLAMA2_7B = "/home/featurize/Qwen1.5-4B-Chat"
11. # LLAMA2_7B_CHAT = "meta-llama/Llama-2-7b-chat-hf"
12. # LLAMA2_13B = "meta-llama/Llama-2-13b-hf"
13. LLAMA2_13B_CHAT = "/home/featurize/Qwen1.5-4B-Chat"
14. # LLAMA2_70B = "meta-llama/Llama-2-70b-hf"
15. # LLAMA2_70B_CHAT = "meta-llama/Llama-2-70b-chat-hf"
17. selected_model = LLAMA2_13B_CHAT
19. SYSTEM_PROMPT = """You are an AI assistant that answers questions in a friendly manner, based on the given source documents. Here are some rules you always follow:
20. - Generate human readable output, avoid creating output with gibberish text.
21. - Generate only the requested output, don't include any other language before or after the requested output.
22. - Never say thank you, that you are happy to help, that you are an AI agent, etc. Just answer directly.
23. - Generate professional language typically used in business documents in North America.
24. - Never generate offensive or foul language.
25. """
27. query_wrapper_prompt = PromptTemplate(
28.     "[INST]<<SYS>>\n" + SYSTEM_PROMPT + "<</SYS>>\n\n{query_str}[/INST] "
29. )
31. llm = HuggingFaceLLM(context_window=4096,
32.     max_new_tokens=2048,
33.     generate_kwargs={"temperature": 0.0, "do_sample": False},
34.     query_wrapper_prompt=query_wrapper_prompt,
35.     tokenizer_name=selected_model,
36.     model_name=selected_model,
37.     device_map="auto"
38. )

复制代码

（2）加载词嵌入向量

2. from llama_
3. index.embeddings.huggingface import HuggingFaceEmbedding
5. embed_model = HuggingFaceEmbedding(model_name="/home/featurize/bge-small-zh-v1.5")

复制代码

2. from llama_
3. index.core import Settings
5. Settings.llm = llm
6. Settings.embed_model = embed_model

复制代码

2. from llama_
3. index.core import SimpleDirectoryReader
5. # load documents
6. documents = SimpleDirectoryReader("./data/").load_data()

复制代码

2. from llama_
3. index.core import VectorStoreIndex
5. index = VectorStoreIndex.from_documents(documents)

复制代码

 

2. index

复制代码

 

1. # set Logging to DEBUG for more detailed outputsquery_engine =
2. index.as_query_engine()

复制代码

2. response = query_engine.query("小额贷款咋规定的?")
3. print(response)

复制代码

 
知识库 

llama
index实现RAG中很关键的一环就是知识库，知识库告急是各种范例的文档，这里给的文档是一个pdf文件，文件内容如下。

 总结

从上面的代码可以看出，我们利用qwen和bge-zh模子可以实现当地下载模子的RAG方案，知识库内里的内容也可以实现中文问答，这非常有利于我们举行私有化摆设方案，从而扩展我们的功能。

免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！qidao123.com:ToB企服之家，中国第一个企服评测及软件市场,开放入驻,技术点评得现金