【条记】Ubuntu中Llama3中文微调，并加载微调后的模型：中文微调数据集介绍 ...

实践：about ollama


安装

1. curl -fsSL https://ollama.com/install.sh | sh

复制代码

部署

1. ollama create example -f Modelfile

复制代码

运行

1. ollama run example

复制代码

停止(ollama加载的大模型将会停止占用显存，此时ollama属于失联状态，部署和运行操作失效，会报错：
Error: could not connect to ollama app, is it running?
必要启动后，才可以进行部署和运行操作)

1. systemctl stop ollama.service

复制代码

停止后启动（启动后，可以接着使用ollama 部署和运行大模型）

1. systemctl start ollama.service

复制代码

Modelfile contents:

1. FROM /home/wangbin/Desktop/Llama3/dir-unsloth.F16.gguf
4. PARAMETER stop "<|im_start|>"
5. PARAMETER stop "<|im_end|>"
7. TEMPLATE """
8. <|im_start|>system
9. {{ .System }}<|im_end|>
10. <|im_start|>user
11. {{ .Prompt }}<|im_end|>
12. <|im_start|>assistant
13. """
18. PARAMETER temperature 0.8
19. PARAMETER num_ctx 8192
21. PARAMETER stop "<|system|>"
22. PARAMETER stop "<|user|>"
23. PARAMETER stop "<|assistant|>"
27. SYSTEM """You are a helpful, smart, kind, and efficient AI assistant.Your name is Aila. You always fulfill the user's requests to the best of your ability."""

复制代码

ollama 参数：

1. (unsloth_env) wangbin@wangbin-LEGION-REN9000K-34IRZ:~/Desktop/Llama3$ ollama
2. Usage:
3.   ollama [flags]
4.   ollama [command]
6. Available Commands:
7.   serve       Start ollama
8.   create      Create a model from a Modelfile
9.   show        Show information for a model
10.   run         Run a model
11.   pull        Pull a model from a registry
12.   push        Push a model to a registry
13.   list        List models
14.   ps          List running models
15.   cp          Copy a model
16.   rm          Remove a model
17.   help        Help about any command
19. Flags:
20.   -h, --help      help for ollama
21.   -v, --version   Show version information

复制代码

卸载

1. 1.Stop the Ollama Service
2.     First things first, we need to stop the Ollama service from running. This ensures a smooth uninstallation process. Open your terminal and enter the following command:
4. sudo systemctl stop ollama
6. This command halts the Ollama service.
11. 2.Disable the Ollama Service
12.     Now that the service is stopped, we need to disable it so that it doesn’t start up again upon system reboot. Enter the following command:
14. sudo systemctl disable ollama
16. This ensures that Ollama won’t automatically start up in the future.
21. 3.Remove the Service File
22.     We need to tidy up by removing the service file associated with Ollama. Enter the following command:
24. sudo rm /etc/systemd/system/ollama.service
26. This deletes the service file from your system.
31. 4.Delete the Ollama Binary
32.     Next up, we’ll remove the Ollama binary itself. Enter the following command:
34. sudo rm $(which ollama)
36. This command removes the binary from your bin directory.
41. 5.Remove Downloaded Models and Ollama User
42.     Lastly, we’ll clean up any remaining bits and pieces. Enter the following commands one by one:
44. sudo rm -r /usr/share/ollama
46. sudo userdel ollama sudo groupdel ollama
48. These commands delete any downloaded models and remove the Ollama user and group from your system.

复制代码

正文：

清洗PDF:

1. 清洗PDF
2. import PyPDF2
3. import re
5. def clean_extracted_text(text):
6.     """Clean and preprocess extracted text."""
7.     # Remove chapter titles and sections
8.     text = re.sub(r'^(Introduction|Chapter \d+:|What is|Examples:|Chapter \d+)', '', text, flags=re.MULTILINE)
9.     text = re.sub(r'ctitious', 'fictitious', text)
10.     text = re.sub(r'ISBN[- ]13: \d{13}', '', text)
11.     text = re.sub(r'ISBN[- ]10: \d{10}', '', text)
12.     text = re.sub(r'Library of Congress Control Number : \d+', '', text)
13.     text = re.sub(r'(\.|\?|\!)(\S)', r'\1 \2', text)  # Ensure space after punctuation
14.     text = re.sub(r'All rights reserved|Copyright \d{4}', '', text)
15.     text = re.sub(r'\n\s*\n', '\n', text)
16.     text = re.sub(r'[^\x00-\x7F]+', ' ', text)
17.     text = re.sub(r'\s{2,}', ' ', text)
19.     # Remove all newlines and replace newlines only after periods
20.     text = text.replace('\n', ' ')
21.     text = re.sub(r'(\.)(\s)', r'\1\n', text)
23.     return text
25. def extract_text_from_pdf(pdf_path):
26.     """Extract text from a PDF file."""
27.     with open(pdf_path, 'rb') as file:
28.         reader = PyPDF2.PdfReader(file)
29.         text = ''
30.         for page in reader.pages:
31.             if page.extract_text():
32.                 text += page.extract_text() + ' '  # Append text of each page
33.     return text
35. def main():
36.     pdf_path = '/Users/charlesqin/Documents/The Art of Asking ChatGPT.pdf'  # Path to your PDF file
37.     extracted_text = extract_text_from_pdf(pdf_path)
38.     cleaned_text = clean_extracted_text(extracted_text)
40.     # Output the cleaned text to a file
41.     with open('cleaned_text_output.txt', 'w', encoding='utf-8') as file:
42.         file.write(cleaned_text)
44. if __name__ == '__main__':
45.     main()

复制代码

微调代码：

1. from unsloth import FastLanguageModel
2. import torch
4. from trl import SFTTrainer
5. from transformers import TrainingArguments
10. max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!
11. dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
12. load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.
14. # 4bit pre quantized models we support for 4x faster downloading + no OOMs.
15. fourbit_models = [
16.     "unsloth/mistral-7b-bnb-4bit",
17.     "unsloth/mistral-7b-instruct-v0.2-bnb-4bit",
18.     "unsloth/llama-2-7b-bnb-4bit",
19.     "unsloth/gemma-7b-bnb-4bit",
20.     "unsloth/gemma-7b-it-bnb-4bit", # Instruct version of Gemma 7b
21.     "unsloth/gemma-2b-bnb-4bit",
22.     "unsloth/gemma-2b-it-bnb-4bit", # Instruct version of Gemma 2b
23.     "unsloth/llama-3-8b-bnb-4bit", # [NEW] 15 Trillion token Llama-3
24. ] # More models at https://huggingface.co/unsloth
26. model, tokenizer = FastLanguageModel.from_pretrained(
27.     model_name = "unsloth/llama-3-8b-bnb-4bit",
28.     max_seq_length = max_seq_length,
29.     dtype = dtype,
30.     load_in_4bit = load_in_4bit,
31.     # token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
32. )
34. model = FastLanguageModel.get_peft_model(
35.     model,
36.     r = 16, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128
37.     target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
38.                       "gate_proj", "up_proj", "down_proj",],
39.     lora_alpha = 16,
40.     lora_dropout = 0, # Supports any, but = 0 is optimized
41.     bias = "none",    # Supports any, but = "none" is optimized
42.     # [NEW] "unsloth" uses 30% less VRAM, fits 2x larger batch sizes!
43.     use_gradient_checkpointing = "unsloth", # True or "unsloth" for very long context
44.     random_state = 3407,
45.     use_rslora = False,  # We support rank stabilized LoRA
46.     loftq_config = None, # And LoftQ
47. )
49. alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
51. ### Instruction:
52. {}
54. ### Input:
55. {}
57. ### Response:
58. {}"""
60. EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN
61. def formatting_prompts_func(examples):
62.     instructions = examples["instruction"]
63.     inputs       = examples["input"]
64.     outputs      = examples["output"]
65.     texts = []
66.     for instruction, input, output in zip(instructions, inputs, outputs):
67.         # Must add EOS_TOKEN, otherwise your generation will go on forever!
68.         text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN
69.         texts.append(text)
70.     return { "text" : texts, }
71. pass
73. from datasets import load_dataset
75. file_path = "/home/Ubuntu/alpaca_gpt4_data_zh.json"
78. dataset = load_dataset("json", data_files={"train": file_path}, split="train")
80. dataset = dataset.map(formatting_prompts_func, batched = True,)
85. trainer = SFTTrainer(
86.     model = model,
87.     tokenizer = tokenizer,
88.     train_dataset = dataset,
89.     dataset_text_field = "text",
90.     max_seq_length = max_seq_length,
91.     dataset_num_proc = 2,
92.     packing = False, # Can make training 5x faster for short sequences.
93.     args = TrainingArguments(
94.         per_device_train_batch_size = 2,
95.         gradient_accumulation_steps = 4,
96.         warmup_steps = 5,
97.         max_steps = 60,
98.         learning_rate = 2e-4,
99.         fp16 = not torch.cuda.is_bf16_supported(),
100.         bf16 = torch.cuda.is_bf16_supported(),
101.         logging_steps = 1,
102.         optim = "adamw_8bit",
103.         weight_decay = 0.01,
104.         lr_scheduler_type = "linear",
105.         seed = 3407,
106.         output_dir = "outputs",
107.     ),
108. )
110. trainer_stats = trainer.train()
112. model.save_pretrained_gguf("dir", tokenizer, quantization_method = "q4_k_m")
113. model.save_pretrained_gguf("dir", tokenizer, quantization_method = "q8_0")
114. model.save_pretrained_gguf("dir", tokenizer, quantization_method = "f16")

复制代码

Ollama:

LM Studio:

我们使用经过Fine Tuning以后的Llama3大模型，询问它题目：

然后我们使用没有经过Fine Tuning的Llama3,还是用刚才的题目询问它：

Reference link：https://www.youtube.com/watch?v=oxTVzGwKeoU


免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。