【小沐学AI】Python实现语音辨认（whisper+HuggingFace）

1、简介

1.1 whisper

https://arxiv.org/pdf/2212.04356
https://github.com/openai/whisper
Whisper 是一种通用语音辨认模子。它是在各种音频的大型数据集上训练的，也是一个多任务模子，可以实验多语言语音辨认、语音翻译和语言辨认。

Transformer 序列到序列模子针对各种语音处理处罚任务举行训练，包罗多语言语音辨认、语音翻译、口语辨认和语音运动检测。这些任务共同体现为解码器要推测的一系列标记，从而允许单个模子代替传统语音处理处罚管道的很多阶段。多任务训练格式利用一组特别标记作为任务分析符或分类目的。
2、HuggingFace

https://www.hugging-face.org/models/
Hugging Face AI 是一个致力于呆板学习和数据科学的平台和社区，资助用户构建、摆设和训练 ML 模子。它为在实际应用步伐中演示、运行和实验 AI 提供了须要的根本办法。该平台利用户可以大概探索和利用其他人上传的模子和数据集。Hugging Face AI 通常被比作呆板学习的 GitHub，它鼓励对开发职员的工作举行公开共享和测试。

该平台以其 Transformers Python 库而闻名，该库简化了访问和训练 ML 模子的过程。该库为开发职员提供了一种有用的方法，可以将 Hugging Face 中的 ML 模子集成到他们的项目中并创建 ML 管道。它是实用于 PyTorch、TensorFlow 和 JAX 的开始辈的呆板学习。
Hugging Face 提供了两种方式来访问大模子：

• Inference API （Serverless） ：通过 API 举行推理。
  

1. import requests
2. API_URL = "https://api-inference.huggingface.co/models/meta-llama/Llama-2-7b-hf"  
3. headers = {"Authorization": "Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}  
4.   
5. def query(payload):  
6.     response = requests.post(API_URL, headers=headers, json=payload)  
7.     return response.json()  
8.   
9. output = query({  
10.     "inputs": "Can you please let us know more details about your ",  
11. })

复制代码

• 本地实验 ：利用 Hugging Face 的 pipeline 来举行高级操纵。
  

1. from transformers import pipeline
2. pipe = pipeline("text-generation", model="meta-llama/Llama-2-7b-hf")

复制代码

2.1 安装transformers

1. pip install transformers

复制代码

2.2 Pipeline 简介

Pipeline将数据预处理处罚、模子调用、效果后处理处罚三部分组装成的流水线，使我们可以大概直接输入文本便得到终极的答案。

Pipeline的创建与利用方式：

1. # 1、根据任务类型直接创建Pipeline
2. pipe = pipeline("text-classification")
4. # 2、指定任务类型，再指定模型，创建基于指定模型的Pipeline
5. pipe = pipeline("text-classification", model="uer/roberta-base-finetuned-dianping-chinese")
7. # 3、预先加载模型，再创建Pipeline
8. # 必须同时指定model和tokenizer
9. model = AutoModelForSequenceClassification.from_pretrained("uer/roberta-base-finetuned-dianping-chinese")
10. tokenizer = AutoTokenizer.from_pretrained("uer/roberta-base-finetuned-dianping-chinese")
11. pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)
13. # 4、使用GPU进行推理加速
14. pipe = pipeline("text-classification", model="uer/roberta-base-finetuned-dianping-chinese", device=0)

复制代码

2.3 Tasks 简介

检察Pipeline支持的任务范例：

1. from transformers.pipelines import SUPPORTED_TASKS
3. print(SUPPORTED_TASKS.items())

复制代码

1. for k, v in SUPPORTED_TASKS.items():
2.     print(k, v)

复制代码

2.3.1 sentiment-analysis

1. from transformers import pipeline
3. classifier = pipeline("sentiment-analysis")
4. text = classifier("I've been waiting for a HuggingFace course my whole life.")
5. print(text)
7. text = classifier([
8.     "I've been waiting for a HuggingFace course my whole life.",
9.     "I hate this so much!"
10. ])
11. print(text)

复制代码

2.3.2 zero-shot-classification

1. from transformers import pipeline
3. classifier = pipeline("zero-shot-classification")
4. text = classifier(
5.     "This is a course about the Transformers library",
6.     candidate_labels=["education", "politics", "business"],
7. )
8. print(text)

复制代码

2.3.3 text-generation

1. from transformers import pipeline
3. generator = pipeline("text-generation")
4. text = generator("In this course, we will teach you how to")
5. print(text)

复制代码

1. from transformers import pipeline
3. generator = pipeline("text-generation", model="distilgpt2")
4. text = generator(
5.     "In this course, we will teach you how to",
6.     max_length=30,
7.     num_return_sequences=2,
8. )
9. print(text)

复制代码

2.3.4 fill-mask

1. from transformers import pipeline
3. unmasker = pipeline("fill-mask")
4. text = unmasker("This course will teach you all about <mask> models.", top_k=2)
5. print(text)

复制代码

2.3.5 ner

1. from transformers import pipeline
3. ner = pipeline("ner", grouped_entities=True)
4. text = ner("My name is Sylvain and I work at Hugging Face in Brooklyn.")
5. print(text)

复制代码

2.3.6 question-answering

1. from transformers import pipeline
3. question_answerer = pipeline("question-answering")
4. text = question_answerer(
5.     question="Where do I work?",
6.     context="My name is Sylvain and I work at Hugging Face in Brooklyn"
7. )
8. print(text)

复制代码

2.3.7 summarization

1. from transformers import pipeline
3. summarizer = pipeline("summarization")
4. text = summarizer("""
5.     America has changed dramatically during recent years. Not only has the number of
6.     graduates in traditional engineering disciplines such as mechanical, civil,
7.     electrical, chemical, and aeronautical engineering declined, but in most of
8.     the premier American universities engineering curricula now concentrate on
9.     and encourage largely the study of engineering science. As a result, there
10.     are declining offerings in engineering subjects dealing with infrastructure,
11.     the environment, and related issues, and greater concentration on high
12.     technology subjects, largely supporting increasingly complex scientific
13.     developments. While the latter is important, it should not be at the expense
14.     of more traditional engineering.
16.     Rapidly developing economies such as China and India, as well as other
17.     industrial countries in Europe and Asia, continue to encourage and advance
18.     the teaching of engineering. Both China and India, respectively, graduate
19.     six and eight times as many traditional engineers as does the United States.
20.     Other industrial countries at minimum maintain their output, while America
21.     suffers an increasingly serious decline in the number of engineering graduates
22.     and a lack of well-educated engineers.
23. """)
24. print(text)

复制代码

2.3.8 translation

1. pip install sentencepiece

复制代码

1. from transformers import pipeline
3. # translator = pipeline("translation", model="Helsinki-NLP/opus-mt-fr-en")
4. translator = pipeline("translation", model="Helsinki-NLP/opus-mt-en-zh")
5. text=translator("To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator.")
6. print(text)

复制代码

利用HuggingFace的中译英模子和英译中模子。

• （1）中译英
  中译英模子的模子名称为：opus-mt-zh-en，下载网址为：https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/tree/main
  

1. from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
2. from transformers import pipeline
4. model_path = './zh-en/'  
5. #创建tokenizer
6. tokenizer = AutoTokenizer.from_pretrained(model_path)
7. #创建模型
8. model = AutoModelForSeq2SeqLM.from_pretrained(model_path)
9. #创建pipeline
10. pipeline = pipeline("translation", model=model, tokenizer=tokenizer)
13. chinese="""
14. 中国男子篮球职业联赛（Chinese Basketball Association），简称中职篮（CBA），是由中国篮球协会所主办的跨年度主客场制篮球联赛，中国最高等级的篮球联赛，其中诞生了如姚明、王治郅、易建联、朱芳雨等球星。"""
15. result = pipeline(chinese)
16. print(result[0]['translation_text'])

复制代码

• （2）英译中
  英译中模子的模子名称为opus-mt-en-zh，下载网址为：https://huggingface.co/Helsinki-NLP/opus-mt-en-zh/tree/main
  

1. from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
2. from transformers import pipeline
4. model_path = './en-zh/'  
5. #创建tokenizer
6. tokenizer = AutoTokenizer.from_pretrained(model_path)
7. #创建模型
8. model = AutoModelForSeq2SeqLM.from_pretrained(model_path)
9. #创建pipeline
10. pipeline = pipeline("translation", model=model, tokenizer=tokenizer)
13. english="""
14. The official site of the National Basketball Association. Follow the action on NBA scores, schedules, stats, news, Team and Player news.
15. """
16. result = pipeline(english)
17. print(result[0]['translation_text'])

复制代码

3、测试

pipeline() 提供了在任何语言、盘算机视觉、音频和多模态任务上利用 Hub 中的任何模子举行推理的简朴方法。纵然您对某个具体模态没有履历大概不认识模子背后的代码，您仍旧可以利用 pipeline() 举行推理！

1. from transformers import pipeline
3. # 首先创建一个 pipeline() 并指定一个推理任务：
4. generator = pipeline(task="automatic-speech-recognition")
6. # 将输入文本传递给 pipeline()：
7. text = generator("https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/mlk.flac")
8. print(text)

复制代码

• 视觉任务的 pipeline
  对于视觉任务，利用 pipeline() 险些是雷同的。指定您的任务并将图像转达给分类器。图像可以是链接或图像的本地路径。比方，下面表现的是哪个品种的猫？
  

1. from transformers import pipeline
3. vision_classifier = pipeline(model="google/vit-base-patch16-224")
4. preds = vision_classifier(
5.     images="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
6. )
7. preds = [{"score": round(pred["score"], 4), "label": pred["label"]} for pred in preds]
8. print(preds)

复制代码

• 文本任务的 pipeline
  对于自然语言处理处罚（NLP）任务，利用 pipeline() 险些是雷同的。
  

1. from transformers import pipeline
3. # 该模型是一个 `zero-shot-classification (零样本分类)` 模型。
4. # 它会对文本进行分类，您可以传入你能想到的任何标签
5. classifier = pipeline(model="facebook/bart-large-mnli")
6. text = classifier(
7.     "I have a problem with my iphone that needs to be resolved asap!!",
8.     candidate_labels=["urgent", "not urgent", "phone", "tablet", "computer"],
9. )
10. print(text)

复制代码

• 语音转笔墨
  

1. import os
2. from transformers import pipeline
3. import subprocess
4. import argparse
5. import json
7. os.environ["HF_ENDPOINT"] = "https://hf-mirror.com"
8. os.environ["CUDA_VISIBLE_DEVICES"] = "2"
9. os.environ["TF_ENABLE_ONEDNN_OPTS"] = "0"
12. def speech2text(speech_file):
13.     transcriber = pipeline(task="automatic-speech-recognition", model="openai/whisper-medium")
14.     text_dict = transcriber(speech_file)
15.     return text_dict
18. def main():
19.     # parser = argparse.ArgumentParser(description="语音转文本")
20.     # parser.add_argument("--audio","-a", type=str, help="输出音频文件路径")
22.     # args = parser.parse_args()
23.     # print(args)
25.     # text_dict = speech2text(args.audio)
26.     text_dict = speech2text("test.mp3")
27.     print("语音识别的文本是：\n" +  text_dict["text"])
28.     print("语音识别的文本是：\n"+ json.dumps(text_dict,indent=4, ensure_ascii=False))
30. if __name__=="__main__":
31.     main()

复制代码

更多AI信息如下：

2024第四届人工智能、自动化与高性能盘算国际集会(AIAHPC 2024)将于2024年7月19-21日在中国·珠海召开。
大会网站：更多集会详情
时间地点：中国珠海-中山大学珠海校区｜2024年7月19-21日
结语

如果您以为该方法或代码有一点点用处，可以给作者点个赞，或打赏杯咖啡；╮(￣▽￣)╭
如果您感觉方法或代码不咋地//(ㄒoㄒ)//，就在品评处留言，作者继续改进；o_O???
如果您须要相干功能的代码定制化开发，可以留言私信作者；(✿◡‿◡)
感谢各位大佬童鞋们的支持！( ´ ▽´ )ﾉ ( ´ ▽´)っ！！！

免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！qidao123.com:ToB企服之家，中国第一个企服评测及软件市场,开放入驻,技术点评得现金