ToB企服应用市场:ToB评测及商务社交产业平台

标题: 开源模型应用落地-qwen2-7b-instruct-LoRA微调&归并-ms-swift-单机单卡-V10 [打印本页]
作者: 美丽的神话    时间: 2024-9-21 08:22
标题: 开源模型应用落地-qwen2-7b-instruct-LoRA微调&归并-ms-swift-单机单卡-V10
一、前言

    本篇文章将利用ms-swift去归并微调后的模型权重，通过阅读本文，您将能够更好地把握这些关键技术，理解其中的关键技术要点，并应用于自己的项目中。
二、术语先容

2.1. LoRA微调

    LoRA (Low-Rank Adaptation) 用于微调大型语言模型 (LLM)。  是一种有效的自适应计谋，它不会引入额外的推理延迟，并在保持模型质量的同时显着减少下游使命的可训练参数数目。
2.2.参数高效微调(PEFT) 

    仅微调少量 (额外) 模型参数，同时冻结预训练 LLM 的大部分参数，从而大大降低了盘算和存储本钱。
2.3.Qwen2-7B-Instruct

    是通义千问 Qwen2 系列中的一个指令微调模型。它在 Qwen2-7B 的底子上举行了指令微调，以进步模型在特定使命上的性能。
    Qwen2-7B-Instruct 具有以下特点：

2.4.模型归并

    指的是将多个模型的权重或参数整合到一个新的模型中，形成一个更强大的模型。
    模型归并的用途：

三、前置条件

 3.1. 底子情况及前置条件

     1. 操作体系：centos7
     2. NVIDIA Tesla V100 32GB   CUDA Version: 12.2 

     3. 提前下载好Qwen2-7B-Instruct模型
          通过以下两个所在举行下载，优先保举魔搭
          hugging face：https://huggingface.co/Qwen/Qwen2-7B-Instruct/tree/main
         

          modelscope：git clone https://www.modelscope.cn/qwen/Qwen2-7B-Instruct.git

按需选择SDK大概Git方式下载

利用git方式下载示例（大概利用git-lfs）：

3.2.Anaconda安装

      参见“开源模型应用落地-qwen-7b-chat与vllm实现推理加快的正确姿势（一）”
3.3.安装依靠

    通过命令行安装：
  1. conda create --name swift  python=3.10
  2. conda activate swift
  3. conda env remove -n swift
  4. pip install 'ms-swift[all]' -U -i https://pypi.tuna.tsinghua.edu.cn/simple
复制代码
    通过源码安装：
  1. git clone https://github.com/modelscope/swift.git
  2. cd swift
  3. pip install -e '.[llm]' -i https://pypi.tuna.tsinghua.edu.cn/simple
复制代码
3.4.完成模型微调

参见：开源模型应用落地-qwen2-7b-instruct-LoRA微调-ms-swift-单机单卡-V100（十二）
日志输出情况：
  1. Train: 100%|██████████| 873/873 [09:34<00:00,  1.69it/s]{'eval_loss': nan, 'eval_acc': 0.02320291, 'eval_runtime': 1.6477, 'eval_samples_per_second': 4.855, 'eval_steps_per_second': 4.855, 'epoch': 0.92, 'global_step/max_steps': '800/873', 'percentage': '91.64%', 'elapsed_time': '8m 47s', 'remaining_time': '48s'}
  3. Val: 100%|██████████| 8/8 [00:01<00:00,  5.65it/s]9it/s]
  4. [INFO:swift] Saving model checkpoint to /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v1-20240830-151000/checkpoint-873
  5. Train: 100%|██████████| 873/873 [09:36<00:00,  1.51it/s]
  6. [INFO:swift] last_model_checkpoint: /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v1-20240830-151000/checkpoint-873
  7. [INFO:swift] best_model_checkpoint: /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v1-20240830-151000/checkpoint-100
  8. [INFO:swift] images_dir: /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v1-20240830-151000/images
  9. [INFO:swift] End time of running main: 2024-08-30 15:20:25.615625
  10. {'eval_loss': nan, 'eval_acc': 0.02320291, 'eval_runtime': 1.6682, 'eval_samples_per_second': 4.796, 'eval_steps_per_second': 4.796, 'epoch': 1.0, 'global_step/max_steps': '873/873', 'percentage': '100.00%', 'elapsed_time': '9m 36s', 'remaining_time': '0s'}
  11. {'train_runtime': 576.7666, 'train_samples_per_second': 1.514, 'train_steps_per_second': 1.514, 'train_loss': 0.0, 'epoch': 1.0, 'global_step/max_steps': '873/873', 'percentage': '100.00%', 'elapsed_time': '9m 36s', 'remaining_time': '0s'}
复制代码
生成的模型权重：


四、技术实现

4.1.推理时归并

在推理时, 归并LoRA权重并保存
启动归并：
  1. conda activate swift
  2. swift infer --ckpt_dir /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873 --load_dataset_config true --merge_lora true --infer_backend vllm --max_model_len 8192
复制代码
若配置启用vllm，则必要安装vllm依靠
  1. pip install vllm -i https://pypi.tuna.tsinghua.edu.cn/simple
复制代码
可配置参数：
  1. InferArguments(model_type='qwen2-7b-instruct', model_id_or_path='/data/model/qwen2-7b-instruct', model_revision='master', sft_type='lora', template_type='qwen', infer_backend='vllm', ckpt_dir='/data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v1-20240901-141800/checkpoint-873', result_dir=None, load_args_from_ckpt_dir=True, load_dataset_config=True, eval_human=False, seed=42, dtype='bf16', model_kwargs=None, dataset=['qwen_zh_demo'], val_dataset=[], dataset_seed=42, dataset_test_ratio=0.01, show_dataset_sample=-1, save_result=True, system='You are a helpful assistant.', tools_prompt='react_en', max_length=None, truncation_strategy='delete', check_dataset_strategy='none', model_name=[None, None], model_author=[None, None], quant_method=None, quantization_bit=0, hqq_axis=0, hqq_dynamic_config_path=None, bnb_4bit_comp_dtype='bf16', bnb_4bit_quant_type='nf4', bnb_4bit_use_double_quant=True, bnb_4bit_quant_storage=None, max_new_tokens=2048, do_sample=True, temperature=0.3, top_k=20, top_p=0.7, repetition_penalty=1.0, num_beams=1, stop_words=[], rope_scaling=None, use_flash_attn=None, ignore_args_error=False, stream=True, merge_lora=True, merge_device_map='cpu', save_safetensors=True, overwrite_generation_config=True, verbose=None, local_repo_path=None, custom_register_path=None, custom_dataset_info='/data/service/swift/data/custom_dataset_info.json', device_map_config_path=None, device_max_memory=[], hub_token=None, gpu_memory_utilization=0.9, tensor_parallel_size=1, max_num_seqs=256, max_model_len=8192, disable_custom_all_reduce=True, enforce_eager=False, vllm_enable_lora=False, vllm_max_lora_rank=16, lora_modules=[], tp=1, cache_max_entry_count=0.8, quant_policy=0, vision_batch_size=1, self_cognition_sample=0, train_dataset_sample=-1, val_dataset_sample=None, safe_serialization=None, model_cache_dir=None, merge_lora_and_save=None, custom_train_dataset_path=[], custom_val_dataset_path=[], vllm_lora_modules=None)
复制代码
归并结果：
  1. INFO 08-30 17:16:03 model_runner.py:879] Starting to load model /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873-merged...
  2. INFO 08-30 17:16:03 selector.py:217] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
  3. INFO 08-30 17:16:03 selector.py:116] Using XFormers backend.
  4. Loading safetensors checkpoint shards:   0% Completed | 0/4 [00:00<?, ?it/s]
  5. Loading safetensors checkpoint shards:  25% Completed | 1/4 [01:07<03:23, 67.95s/it]
  6. Loading safetensors checkpoint shards:  50% Completed | 2/4 [01:21<01:12, 36.12s/it]
  7. Loading safetensors checkpoint shards:  75% Completed | 3/4 [01:22<00:20, 20.10s/it]
  8. Loading safetensors checkpoint shards: 100% Completed | 4/4 [01:23<00:00, 12.29s/it]
  9. Loading safetensors checkpoint shards: 100% Completed | 4/4 [01:23<00:00, 20.78s/it]
  11. INFO 08-30 17:17:27 model_runner.py:890] Loading model weights took 14.2487 GB
  12. INFO 08-30 17:17:29 gpu_executor.py:121] # GPU blocks: 13857, # CPU blocks: 4681
  13. INFO 08-30 17:17:32 model_runner.py:1181] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
  14. INFO 08-30 17:17:32 model_runner.py:1185] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing `gpu_memory_utilization` or enforcing eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage.
  15. INFO 08-30 17:17:59 model_runner.py:1300] Graph capturing finished in 27 secs.
  16. [INFO:swift] generation_config: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.3, top_p=0.7, top_k=20, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=2048, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=False, spaces_between_special_tokens=True, truncate_prompt_tokens=None)
  17. [INFO:swift] system: You are a helpful assistant.
  18. Map: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 881/881 [00:00<00:00, 9162.50 examples/s]
  19. [INFO:swift] val_dataset: Dataset({
  20.     features: ['query', 'response'],
  21.     num_rows: 8
  22. })
  23. [INFO:swift] Setting args.verbose: True
  24. [INFO:swift] save_result_path: /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873-merged/infer_result/20240830-171759.jsonl
  25. [INFO:swift] End time of running main: 2024-08-30 17:20:33.961291
  26. [rank0]:[W830 17:20:34.309521566 ProcessGroupNCCL.cpp:1168] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present,  but this warning has only been added since PyTorch 2.4 (function operator())
复制代码
归并后的文件：

4.2.单独归并

启动归并：
  1. conda activate swift 
  2. swift export --ckpt_dir /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873 --merge_lora true
复制代码
可配置参数：
  1. ExportArguments(model_type='qwen2-7b-instruct', model_id_or_path='/data/model/qwen2-7b-instruct', model_revision='master', sft_type='lora', template_type='qwen', infer_backend='vllm', ckpt_dir='/data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873', result_dir=None, load_args_from_ckpt_dir=True, load_dataset_config=False, eval_human=True, seed=42, dtype='fp16', model_kwargs=None, dataset=[], val_dataset=[], dataset_seed=42, dataset_test_ratio=0.01, show_dataset_sample=-1, save_result=True, system='You are a helpful assistant.', tools_prompt='react_en', max_length=None, truncation_strategy='delete', check_dataset_strategy='none', model_name=[None, None], model_author=[None, None], quant_method='awq', quantization_bit=0, hqq_axis=0, hqq_dynamic_config_path=None, bnb_4bit_comp_dtype='fp16', bnb_4bit_quant_type='nf4', bnb_4bit_use_double_quant=True, bnb_4bit_quant_storage=None, max_new_tokens=2048, do_sample=True, temperature=0.3, top_k=20, top_p=0.7, repetition_penalty=1.0, num_beams=1, stop_words=[], rope_scaling=None, use_flash_attn=None, ignore_args_error=False, stream=True, merge_lora=True, merge_device_map='auto', save_safetensors=True, overwrite_generation_config=True, verbose=None, local_repo_path=None, custom_register_path=None, custom_dataset_info='/data/service/swift/data/custom_dataset_info.json', device_map_config_path=None, device_max_memory=[], hub_token=None, gpu_memory_utilization=0.9, tensor_parallel_size=1, max_num_seqs=256, max_model_len=None, disable_custom_all_reduce=True, enforce_eager=False, vllm_enable_lora=False, vllm_max_lora_rank=16, lora_modules=[], tp=1, cache_max_entry_count=0.8, quant_policy=0, vision_batch_size=1, self_cognition_sample=0, train_dataset_sample=-1, val_dataset_sample=None, safe_serialization=None, model_cache_dir=None, merge_lora_and_save=None, custom_train_dataset_path=[], custom_val_dataset_path=[], vllm_lora_modules=None, to_peft_format=False, to_ollama=False, ollama_output_dir=None, gguf_file=None, quant_bits=0, quant_n_samples=256, quant_seqlen=2048, quant_device_map='cpu', quant_output_dir=None, quant_batch_size=1, push_to_hub=False, hub_model_id=None, hub_private_repo=False, commit_message='update files', to_megatron=False, to_hf=False, megatron_output_dir=None, hf_output_dir=None, pp=1)
复制代码
归并结果：
  1. run sh: `/usr/local/miniconda3/envs/swift/bin/python /usr/local/miniconda3/envs/swift/lib/python3.10/site-packages/swift/cli/export.py --ckpt_dir /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873 --merge_lora true`
  2. [INFO:swift] Successfully registered `/usr/local/miniconda3/envs/swift/lib/python3.10/site-packages/swift/llm/data/dataset_info.json`
  3. [INFO:swift] No LMDeploy installed, if you are using LMDeploy, you will get `ImportError: cannot import name 'prepare_lmdeploy_engine_template' from 'swift.llm'`
  4. [INFO:swift] Start time of running main: 2024-08-30 22:20:48.465926
  5. [INFO:swift] ckpt_dir: /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873
  6. [INFO:swift] Successfully registered `/data/service/swift/data/custom_dataset_info.json`
  7. [INFO:swift] Setting model_info['revision']: master
  8. [INFO:swift] Setting self.eval_human: True
  9. [INFO:swift] Setting overwrite_generation_config: True
  10. [INFO:swift] args: ExportArguments(model_type='qwen2-7b-instruct', model_id_or_path='/data/model/qwen2-7b-instruct', model_revision='master', sft_type='lora', template_type='qwen', infer_backend='vllm', ckpt_dir='/data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873', result_dir=None, load_args_from_ckpt_dir=True, load_dataset_config=False, eval_human=True, seed=42, dtype='fp16', model_kwargs=None, dataset=[], val_dataset=[], dataset_seed=42, dataset_test_ratio=0.01, show_dataset_sample=-1, save_result=True, system='You are a helpful assistant.', tools_prompt='react_en', max_length=None, truncation_strategy='delete', check_dataset_strategy='none', model_name=[None, None], model_author=[None, None], quant_method='awq', quantization_bit=0, hqq_axis=0, hqq_dynamic_config_path=None, bnb_4bit_comp_dtype='fp16', bnb_4bit_quant_type='nf4', bnb_4bit_use_double_quant=True, bnb_4bit_quant_storage=None, max_new_tokens=2048, do_sample=True, temperature=0.3, top_k=20, top_p=0.7, repetition_penalty=1.0, num_beams=1, stop_words=[], rope_scaling=None, use_flash_attn=None, ignore_args_error=False, stream=True, merge_lora=True, merge_device_map='auto', save_safetensors=True, overwrite_generation_config=True, verbose=None, local_repo_path=None, custom_register_path=None, custom_dataset_info='/data/service/swift/data/custom_dataset_info.json', device_map_config_path=None, device_max_memory=[], hub_token=None, gpu_memory_utilization=0.9, tensor_parallel_size=1, max_num_seqs=256, max_model_len=None, disable_custom_all_reduce=True, enforce_eager=False, vllm_enable_lora=False, vllm_max_lora_rank=16, lora_modules=[], tp=1, cache_max_entry_count=0.8, quant_policy=0, vision_batch_size=1, self_cognition_sample=0, train_dataset_sample=-1, val_dataset_sample=None, safe_serialization=None, model_cache_dir=None, merge_lora_and_save=None, custom_train_dataset_path=[], custom_val_dataset_path=[], vllm_lora_modules=None, to_peft_format=False, to_ollama=False, ollama_output_dir=None, gguf_file=None, quant_bits=0, quant_n_samples=256, quant_seqlen=2048, quant_device_map='cpu', quant_output_dir=None, quant_batch_size=1, push_to_hub=False, hub_model_id=None, hub_private_repo=False, commit_message='update files', to_megatron=False, to_hf=False, megatron_output_dir=None, hf_output_dir=None, pp=1)
  11. [INFO:swift] Global seed set to 42
  12. [INFO:swift] replace_if_exists: False
  13. [INFO:swift] merged_lora_path: `/data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873-merged`
  14. [INFO:swift] merge_device_map: auto
  15. [INFO:swift] device_count: 1
  16. [INFO:swift] Loading the model using model_dir: /data/model/qwen2-7b-instruct
  17. [INFO:swift] model_kwargs: {'low_cpu_mem_usage': True, 'device_map': 'auto'}
  18. Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [02:05<00:00, 31.46s/it]
  19. [INFO:swift] model.max_model_len: 32768
  20. [INFO:swift] generation_config: GenerationConfig {
  21.   "do_sample": true,
  22.   "eos_token_id": 151645,
  23.   "max_new_tokens": 2048,
  24.   "pad_token_id": 151643,
  25.   "temperature": 0.3,
  26.   "top_k": 20,
  27.   "top_p": 0.7
  28. }
  30. [INFO:swift] PeftModelForCausalLM: 7619.0572M Params (0.0000M Trainable [0.0000%]), 234.8828M Buffers.
  31. [INFO:swift] system: You are a helpful assistant.
  32. [INFO:swift] Merge LoRA...
  33. [INFO:swift] Saving merged weights...
  34. [INFO:swift] Successfully merged LoRA and saved in /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873-merged.
  35. [INFO:swift] Setting args.sft_type: 'full'
  36. [INFO:swift] Setting args.ckpt_dir: /data/model/sft/qwen2-7b-instruct-sft/qwen2-7b-instruct/v0-20240830-152615/checkpoint-873-merged
  37. [INFO:swift] End time of running main: 2024-08-30 22:24:23.811071
复制代码
归并后的文件：


免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！更多信息从访问主页：qidao123.com:ToB企服之家，中国第一个企服评测及商务社交产业平台。



欢迎光临 ToB企服应用市场:ToB评测及商务社交产业平台 (https://dis.qidao123.com/) Powered by Discuz! X3.4