保姆级教程:教你UniMRCP对接华为云ASR(Linux版)

打印 上一主题 下一主题

主题 777|帖子 777|积分 2331

本文分享自华为云社区《unimrcp对接华为云ASR(Linux版)》,作者: ASR-beginer。
本篇文章提供了unimrcp对接华为云ASR的保姆级教程,根据第一到四章,可从头渐渐编译+集成基于华为云ASR的unimrcp系统(授人以渔)。同时,本文第五章(直接给条鱼)提供了作者修改好的源码,直接一键编译即可。
一、安装unimrcp

1.提前安装必要的依靠项(系统级)

2.下载unimrcp

3.下载unimrcp的依靠库,并安装
进入https://www.unimrcp.org/downloads/dependencies,直接选择最新版本下载

解压后,进入unimrcp-deps-1.6.0,实行安装下令:
  1. ./build-dep-libs.sh
复制代码
4.安装unimrcp

之后可以在/usr/local/unimrcp 路径下,看到以下内容:

5.ASR测试
先启动服务端./unimrcpserver
  1. (base) [root@asr-build bin]# ./unimrcpserver
  2. 2024-03-09 10:23:39:729352 [NOTICE] UniMRCP Server [1.8.0]
  3. 2024-03-09 10:23:39:729427 [INFO]   APR [1.5.2]
  4. 2024-03-09 10:23:39:729436 [NOTICE] Create MRCP Server
  5. 2024-03-09 10:23:39:729459 [NOTICE] Open Config File [/usr/local/unimrcp/conf/unimrcpserver.xml]
  6. 2024-03-09 10:23:39:729874 [INFO]   Set Property ip:127.0.0.1
  7. 2024-03-09 10:23:39:729888 [INFO]   Register Codec [PCMU]
  8. 2024-03-09 10:23:39:729890 [INFO]   Register Codec [PCMA]
  9. 2024-03-09 10:23:39:729893 [INFO]   Register Codec [L16]
  10. 2024-03-09 10:23:39:729895 [INFO]   Register Codec [G722]
  11. 2024-03-09 10:23:39:729911 [NOTICE] Register Resource [speechsynth]
  12. 2024-03-09 10:23:39:729916 [NOTICE] Register Resource [speechrecog]
  13. 2024-03-09 10:23:39:729918 [NOTICE] Register Resource [recorder]
  14. 2024-03-09 10:23:39:729922 [NOTICE] Register Resource [speakverify]
  15. 2024-03-09 10:23:39:729925 [INFO]   Register Resource Factory
  16. 2024-03-09 10:23:39:729938 [NOTICE] Create SofiaSIP Agent [SIP-Agent-1] [1.12.11-239-g54ef3e2] sip:127.0.0.1:8060;transport=udp,tcp
  17. 2024-03-09 10:23:39:729943 [INFO]   Register Signaling Agent [SIP-Agent-1]
  18. 2024-03-09 10:23:39:729954 [NOTICE] Create RTSP Server [RTSP-Agent-1] 127.0.0.1:1554 [100] connection timeout [600 sec]
  19. 2024-03-09 10:23:39:730021 [INFO]   Register Signaling Agent [RTSP-Agent-1]
  20. 2024-03-09 10:23:39:730027 [NOTICE] Create MRCPv2 Agent [MRCPv2-Agent-1] 127.0.0.1:1544 [100]
  21. 2024-03-09 10:23:39:730047 [INFO]   Register Connection Agent [MRCPv2-Agent-1]
  22. 2024-03-09 10:23:39:730051 [NOTICE] Create Media Engine [Media-Engine-1]
  23. 2024-03-09 10:23:39:730058 [INFO]   Register Media Engine [Media-Engine-1]
  24. 2024-03-09 10:23:39:730062 [NOTICE] Create RTP Termination Factory 127.0.0.1:[5000,6000]
  25. 2024-03-09 10:23:39:730064 [INFO]   Register RTP Termination Factory [RTP-Factory-1]
  26. 2024-03-09 10:23:39:730069 [INFO]   Load Plugin [Demo-Synth-1] [/usr/local/unimrcp/plugin/demosynth.so]
  27. 2024-03-09 10:23:39:730166 [INFO]   Register MRCP Engine [Demo-Synth-1]
  28. 2024-03-09 10:23:39:730171 [INFO]   Load Plugin [Huawei-Recog-1] [/usr/local/unimrcp/plugin/huaweirecog.so]
  29. 2024-03-09 10:23:39:730223 [INFO]   Register MRCP Engine [Huawei-Recog-1]
  30. 2024-03-09 10:23:39:730226 [INFO]   Load Plugin [Demo-Verifier-1] [/usr/local/unimrcp/plugin/demoverifier.so]
  31. 2024-03-09 10:23:39:730276 [INFO]   Register MRCP Engine [Demo-Verifier-1]
  32. 2024-03-09 10:23:39:730279 [INFO]   Load Plugin [Recorder-1] [/usr/local/unimrcp/plugin/mrcprecorder.so]
  33. 2024-03-09 10:23:39:730323 [INFO]   Register MRCP Engine [Recorder-1]
  34. 2024-03-09 10:23:39:730333 [INFO]   Register RTP Settings [RTP-Settings-1]
  35. 2024-03-09 10:23:39:730338 [NOTICE] Create MRCPv2 Profile [uni2]
  36. 2024-03-09 10:23:39:730344 [INFO]   Associate Resource [speechsynth] to Engine [Demo-Synth-1] in Profile [uni2]
  37. 2024-03-09 10:23:39:730347 [INFO]   Associate Resource [speechrecog] to Engine [Huawei-Recog-1] in Profile [uni2]
  38. 2024-03-09 10:23:39:730349 [INFO]   Associate Resource [recorder] to Engine [Recorder-1] in Profile [uni2]
  39. 2024-03-09 10:23:39:730351 [INFO]   Associate Resource [speakverify] to Engine [Demo-Verifier-1] in Profile [uni2]
  40. 2024-03-09 10:23:39:730354 [INFO]   Register Profile [uni2]
  41. 2024-03-09 10:23:39:730356 [NOTICE] Create MRCPv1 Profile [uni1]
  42. 2024-03-09 10:23:39:730358 [INFO]   Associate Resource [speechsynth] to Engine [Demo-Synth-1] in Profile [uni1]
  43. 2024-03-09 10:23:39:730360 [INFO]   Associate Resource [speechrecog] to Engine [Huawei-Recog-1] in Profile [uni1]
  44. 2024-03-09 10:23:39:730362 [INFO]   Associate Resource [recorder] to Engine [Recorder-1] in Profile [uni1]
  45. 2024-03-09 10:23:39:730364 [INFO]   Associate Resource [speakverify] to Engine [Demo-Verifier-1] in Profile [uni1]
  46. 2024-03-09 10:23:39:730367 [INFO]   Register Profile [uni1]
  47. 2024-03-09 10:23:39:730370 [INFO]   Start Task [MRCP Server]
  48. >2024-03-09 10:23:39:730456 [INFO]   Open Engine [Demo-Verifier-1]
  49. 2024-03-09 10:23:39:730471 [INFO]   Start Task [Demo Verifier Engine]
  50. 2024-03-09 10:23:39:730526 [INFO]   Open Engine [Huawei-Recog-1]
  51. 2024-03-09 10:23:39:730530 [INFO]   Start Task [Huawei Recog Engine]
  52. 2024-03-09 10:23:39:730552 [INFO]   Open Engine [Recorder-1]
  53. 2024-03-09 10:23:39:730555 [INFO]   Open Engine [Demo-Synth-1]
  54. 2024-03-09 10:23:39:730557 [INFO]   Start Task [Demo Synth Engine]
  55. 2024-03-09 10:23:39:730580 [INFO]   Start Task [SIP-Agent-1]
  56. 2024-03-09 10:23:39:730599 [INFO]   Start Task [RTSP-Agent-1]
  57. 2024-03-09 10:23:39:730619 [INFO]   Start Task [MRCPv2-Agent-1]
  58. 2024-03-09 10:23:39:730642 [INFO]   Start Task [Media-Engine-1]
  59. sres: /etc/resolv.conf: unknown option                     
  60. 2024-03-09 10:23:39:731672 [NOTICE] MRCP Server Started
复制代码
再启动客户端,测试ASR

 
  1. >run recog
  2. >[1]
  3. 2024-03-09 10:24:46:654383 [NOTICE] Create MRCP Handle 0x7f902c035630 [uni2]
  4. 2024-03-09 10:24:46:654420 [INFO]   Create Channel umc-1 <new>
  5. 2024-03-09 10:24:46:654466 [INFO]   Receive App Request umc-1 <new> [2]
  6. 2024-03-09 10:24:46:654535 [INFO]   Add MRCP Handle umc-1 <new>
  7. 2024-03-09 10:24:46:654573 [NOTICE] Add Control Channel umc-1 <new@speechrecog>
  8. 2024-03-09 10:24:46:656847 [INFO]   Send Offer umc-1 <new> [c:1 a:1 v:0] to 127.0.0.1:8060
  9. 2024-03-09 10:24:46:656879 [INFO]   Local SDP umc-1 <new>
  10. v=0
  11. o=UniMRCPClient 0 0 IN IP4 127.0.0.1
  12. s=-
  13. c=IN IP4 127.0.0.1
  14. t=0 0
  15. m=application 9 TCP/MRCPv2 1
  16. a=setup:active
  17. a=connection:new
  18. a=resource:speechrecog
  19. a=cmid:1
  20. m=audio 4000 RTP/AVP 0 8 9 96 101
  21. a=rtpmap:0 PCMU/8000
  22. a=rtpmap:8 PCMA/8000
  23. a=rtpmap:9 G722/8000
  24. a=rtpmap:96 L16/8000
  25. a=rtpmap:101 telephone-event/8000
  26. a=fmtp:101 0-15
  27. a=sendonly
  28. a=ptime:20
  29. a=mid:1
  30. 2024-03-09 10:24:46:657606 [INFO]   Receive SIP Event [nua_i_state] Status 0 INVITE sent [SIP-Agent-1]
  31. 2024-03-09 10:24:46:657624 [NOTICE] SIP Call State umc-1 [calling]
  32. 2024-03-09 10:24:46:661600 [INFO]   Receive SIP Event [nua_r_invite] Status 200 OK [SIP-Agent-1]
  33. 2024-03-09 10:24:46:661658 [INFO]   Receive SIP Event [nua_i_state] Status 200 OK [SIP-Agent-1]
  34. 2024-03-09 10:24:46:661664 [NOTICE] SIP Call State umc-1 [ready]
  35. 2024-03-09 10:24:46:661668 [INFO]   Remote SDP umc-1 <new>
  36. v=0
  37. o=UniMRCPServer 8207428304472510678 6899681545498287106 IN IP4 192.168.34.32
  38. s=-
  39. c=IN IP4 127.0.0.1
  40. t=0 0
  41. m=application 1544 TCP/MRCPv2 1
  42. a=setup:passive
  43. a=connection:new
  44. a=channel:328a56e4ddbc11ee@speechrecog
  45. a=cmid:1
  46. m=audio 5000 RTP/AVP 0 101
  47. a=rtpmap:0 PCMU/8000
  48. a=rtpmap:101 telephone-event/8000
  49. a=fmtp:101 0-15
  50. a=recvonly
  51. a=ptime:20
  52. a=mid:1
  53. 2024-03-09 10:24:46:661738 [INFO]   Receive SIP Event [nua_i_active] Status 200 Call active [SIP-Agent-1]
  54. 2024-03-09 10:24:46:661756 [INFO]   Receive Answer umc-1 <new> [c:1 a:1 v:0] Status 200
  55. 2024-03-09 10:24:46:661892 [NOTICE] Established TCP/MRCPv2 Connection 127.0.0.1:59830 <-> 127.0.0.1:1544
  56. 2024-03-09 10:24:46:661906 [INFO]   Add Control Channel <328a56e4ddbc11ee@speechrecog> 127.0.0.1:59830 <-> 127.0.0.1:1544 [1]
  57. 2024-03-09 10:24:46:666844 [INFO]   Enable RTP Session 127.0.0.1:4000
  58. 2024-03-09 10:24:46:666873 [INFO]   Open RTP Transmitter 127.0.0.1:4000 -> 127.0.0.1:5000
  59. 2024-03-09 10:24:46:666878 [INFO]   Media Path umc-1 Source->[LPCM/8000/1]->Bridge->[LPCM/8000/1]->Encoder->[PCMU/8000/1]->Sink
  60. 2024-03-09 10:24:46:666900 [INFO]   Raise App Response umc-1 <328a56e4ddbc11ee> [2] SUCCESS [0]
  61. 2024-03-09 10:24:46:666975 [INFO]   Receive App MRCP Request umc-1 <328a56e4ddbc11ee>
  62. 2024-03-09 10:24:46:666979 [INFO]   Send MRCP Request umc-1 <328a56e4ddbc11ee@speechrecog> [1]
  63. 2024-03-09 10:24:46:667028 [INFO]   Send MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [442 bytes]
  64. MRCP/2.0 442 DEFINE-GRAMMAR 1
  65. Channel-Identifier: 328a56e4ddbc11ee@speechrecog
  66. Content-Type: application/srgs+xml
  67. Content-Id: request1@form-level
  68. Content-Length: 269
  69. <?xml version="1.0"?>
  70. <grammar xmlns="http://www.w3.org/2001/06/grammar" xml:lang="en-US" version="1.0" mode="voice" root="digit">
  71.   <rule id="digit">
  72.     <one-of>
  73.       <item>one</item>
  74.       <item>two</item>
  75.       <item>three</item>
  76.     </one-of>
  77.   </rule>
  78. </grammar>
  79. 2024-03-09 10:24:46:667298 [INFO]   Receive MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [112 bytes]
  80. MRCP/2.0 112 1 200 COMPLETE
  81. Channel-Identifier: 328a56e4ddbc11ee@speechrecog
  82. Completion-Cause: 000 success
  83. 2024-03-09 10:24:46:667336 [INFO]   Raise App MRCP Response umc-1 <328a56e4ddbc11ee>
  84. 2024-03-09 10:24:46:667403 [INFO]   Receive App MRCP Request umc-1 <328a56e4ddbc11ee>
  85. 2024-03-09 10:24:46:667406 [INFO]   Send MRCP Request umc-1 <328a56e4ddbc11ee@speechrecog> [2]
  86. 2024-03-09 10:24:46:667414 [INFO]   Set [/usr/local/unimrcp/data/one-8kHz.pcm] as Speech Source
  87. 2024-03-09 10:24:46:667428 [INFO]   Send MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [304 bytes]
  88. MRCP/2.0 304 RECOGNIZE 2
  89. Channel-Identifier: 328a56e4ddbc11ee@speechrecog
  90. Content-Type: text/uri-list
  91. Cancel-If-Queue: false
  92. No-Input-Timeout: 5000
  93. Recognition-Timeout: 10000
  94. Start-Input-Timers: true
  95. Confidence-Threshold: 0.5
  96. Save-Waveform: true
  97. Content-Length: 27
  98. session:request1@form-level
  99. 2024-03-09 10:24:46:667664 [INFO]   Receive MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [83 bytes]
  100. MRCP/2.0 83 2 200 IN-PROGRESS
  101. Channel-Identifier: 328a56e4ddbc11ee@speechrecog
  102. 2024-03-09 10:24:46:667686 [INFO]   Raise App MRCP Response umc-1 <328a56e4ddbc11ee>
  103. 2024-03-09 10:24:47:541082 [INFO]   Receive MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [94 bytes]
  104. MRCP/2.0 94 START-OF-INPUT 2 IN-PROGRESS
  105. Channel-Identifier: 328a56e4ddbc11ee@speechrecog
  106. 2024-03-09 10:24:47:541165 [INFO]   Raise App MRCP Event umc-1 <328a56e4ddbc11ee>
  107. 2024-03-09 10:24:48:011172 [INFO]   Receive MRCPv2 Data 127.0.0.1:59830 <-> 127.0.0.1:1544 [393 bytes]
  108. MRCP/2.0 393 RECOGNITION-COMPLETE 2 COMPLETE
  109. Channel-Identifier: 328a56e4ddbc11ee@speechrecog
  110. Completion-Cause: 000 success
  111. Content-Type: application/x-nlsml
  112. Content-Length: 208
  113. <?xml version="1.0"?>
  114. <result>
  115.   <interpretation grammar="session:request1@form-level.store" confidence="0.97">
  116.     <instance>one</instance>
  117.     <input mode="speech">one</input>
  118.   </interpretation>
  119. </result>
  120. 2024-03-09 10:24:48:011247 [INFO]   Raise App MRCP Event umc-1 <328a56e4ddbc11ee>
  121. 2024-03-09 10:24:48:011354 [INFO]   Interpretation[0].confidence: 0.97
  122. 2024-03-09 10:24:48:011363 [INFO]   Interpretation[0].grammar: session:request1@form-level.store
  123. 2024-03-09 10:24:48:011380 [INFO]   Interpretation[0].instance[0]: one
  124. 2024-03-09 10:24:48:011386 [INFO]   Interpretation[0].input: one
  125. 2024-03-09 10:24:48:011388 [INFO]   Interpretation[0].input.mode: speech
  126. 2024-03-09 10:24:48:011391 [INFO]   Interpretation[0].input.confidence: 1.00
  127. 2024-03-09 10:24:48:011412 [INFO]   Receive App Request umc-1 <328a56e4ddbc11ee> [1]
  128. 2024-03-09 10:24:48:011424 [INFO]   Terminate Session umc-1 <328a56e4ddbc11ee>
  129. 2024-03-09 10:24:48:011454 [INFO]   Remove Control Channel <328a56e4ddbc11ee@speechrecog> [0]
  130. 2024-03-09 10:24:48:011459 [INFO]   Close TCP/MRCPv2 Connection 127.0.0.1:59830 <-> 127.0.0.1:1544
  131. 2024-03-09 10:24:48:011791 [INFO]   Receive SIP Event [nua_r_bye] Status 200 OK [SIP-Agent-1]
  132. 2024-03-09 10:24:48:011805 [INFO]   Receive SIP Event [nua_i_state] Status 200 to BYE [SIP-Agent-1]
  133. 2024-03-09 10:24:48:011810 [NOTICE] SIP Call State umc-1 [terminated]
  134. 2024-03-09 10:24:48:011871 [INFO]   Session Terminated umc-1 <328a56e4ddbc11ee>
  135. 2024-03-09 10:24:48:016774 [INFO]   Close RTP Transmitter 127.0.0.1:4000 -> 127.0.0.1:5000 [s:67 o:10720]
  136. 2024-03-09 10:24:48:016791 [INFO]   Remove RTP Session 127.0.0.1:4000
  137. 2024-03-09 10:24:48:016831 [NOTICE] Destroy TCP/MRCPv2 Connection 127.0.0.1:59830 <-> 127.0.0.1:1544
  138. 2024-03-09 10:24:48:016851 [INFO]   Remove MRCP Handle umc-1 <328a56e4ddbc11ee>
  139. 2024-03-09 10:24:48:016855 [INFO]   Raise App Response umc-1 <328a56e4ddbc11ee> [1] SUCCESS [0]
  140. 2024-03-09 10:24:48:016953 [NOTICE] Destroy MRCP Handle umc-1
复制代码
第124~130行即为模拟的识别结果

二、安装华为ASR SDK

华为ASR C++(Linux版)SDK安装过程参考之前的文章:https://bbs.huaweicloud.com/blogs/392949
这里推荐一键安装,因为本SDK将所有依靠均打包在仓库中,可以在任意Linux机器上编译通过(g++ 4.8.5及以上,cmake版本至少是3.14)
三、unimrcp集成华为ASR SDK

本章节用从底到高的次序进行记录,即先修改底层代码/配置,再修改顶层代码
1.将(上个步骤 安装华为ASR SDK)编译好的SDK打包起来,并定名为huaweicloud_asr,作为备用,其代码结构如下:

2.进入unimrcp/plugins,调整代码结构,如下图所示
  1. cd umimrcp/plugins
  2. mv demo-recog huawei-recog    # 将原始的demo-recog重命名成huawei-recog
  3. mv huawei-recog/src/demo_recog_engine.c huawei-recog/src/huawei_recog_engine.cc
  4. cp -r path_to_sdk/huaweicloud_asr huawei-recog/huaweicloud_asr   # 将华为asr sdk拷贝到umimrcp/plugins/huawei-recog/huaweicloud_asr处
复制代码

3.修改huawei-recog/src/huawei_recog_engine.cc文件,具体内容如下






[img]https://alliance-communityfile-drcn.dbankcdn.com/FileServer/getFile/cmtybbs/519/984/817/2850086000519984817.20240326112923.63068072260154634157771007602934:50001231000000:280079D7E26ACD677EAB21F1E25E09B5FB3504A1AB73ECC644CAEC5960870E48419.png[/img]









 
  1. /** Process START-INPUT-TIMERS request */
  2. static apt_bool_t huawei_recog_channel_timers_start(
  3.     mrcp_engine_channel_t *channel, mrcp_message_t *request, mrcp_message_t *response)
  4. {
  5.     huawei_recog_channel_t *recog_channel = (huawei_recog_channel_t *)channel->method_obj;
  6.     recog_channel->timers_started = TRUE;
  7.     return mrcp_engine_channel_message_send(channel, response);
  8. }
  9. /** Dispatch MRCP request */
  10. static apt_bool_t huawei_recog_channel_request_dispatch(mrcp_engine_channel_t *channel, mrcp_message_t *request)
  11. {
  12.     apt_bool_t processed = FALSE;
  13.     mrcp_message_t *response = mrcp_response_create(request, request->pool);
  14.     switch (request->start_line.method_id) {
  15.         case RECOGNIZER_SET_PARAMS:
  16.             break;
  17.         case RECOGNIZER_GET_PARAMS:
  18.             break;
  19.         case RECOGNIZER_DEFINE_GRAMMAR:
  20.             break;
  21.         case RECOGNIZER_RECOGNIZE:
  22.             processed = huawei_recog_channel_recognize(channel, request, response);
  23.             break;
  24.         case RECOGNIZER_GET_RESULT:
  25.             break;
  26.         case RECOGNIZER_START_INPUT_TIMERS:
  27.             processed = huawei_recog_channel_timers_start(channel, request, response);
  28.             break;
  29.         case RECOGNIZER_STOP:
  30.             processed = huawei_recog_channel_stop(channel, request, response);
  31.             break;
  32.         default:
  33.             break;
  34.     }
  35.     if (processed == FALSE) {
  36.         /* send asynchronous response for not handled request */
  37.         mrcp_engine_channel_message_send(channel, response);
  38.     }
  39.     return TRUE;
  40. }
  41. /** Callback is called from MPF engine context to destroy any additional data associated with audio stream */
  42. static apt_bool_t huawei_recog_stream_destroy(mpf_audio_stream_t *stream)
  43. {
  44.     return TRUE;
  45. }
  46. /** Callback is called from MPF engine context to perform any action before open */
  47. static apt_bool_t huawei_recog_stream_open(mpf_audio_stream_t *stream, mpf_codec_t *codec)
  48. {
  49.     return TRUE;
  50. }
  51. /** Callback is called from MPF engine context to perform any action after close */
  52. static apt_bool_t huawei_recog_stream_close(mpf_audio_stream_t *stream)
  53. {
  54.     return TRUE;
  55. }
  56. /* Raise huawei START-OF-INPUT event */
  57. static apt_bool_t huawei_recog_start_of_input(huawei_recog_channel_t *recog_channel)
  58. {
  59.     /* create START-OF-INPUT event */
  60.     mrcp_message_t *message =
  61.         mrcp_event_create(recog_channel->recog_request, RECOGNIZER_START_OF_INPUT, recog_channel->recog_request->pool);
  62.     if (!message) {
  63.         return FALSE;
  64.     }
  65.     /* set request state */
  66.     message->start_line.request_state = MRCP_REQUEST_STATE_INPROGRESS;
  67.     /* send asynch event */
  68.     return mrcp_engine_channel_message_send(recog_channel->channel, message);
  69. }
  70. /* Load demo recognition result */
  71. static apt_bool_t huawei_recog_result_load(huawei_recog_channel_t *recog_channel, mrcp_message_t *message)
  72. {
  73.     mrcp_engine_channel_t *channel = recog_channel->channel;
  74.     const apt_dir_layout_t *dir_layout = channel->engine->dir_layout;
  75.     /* read the demo result from file  这里是从asr真实结果中读取*/
  76.     mrcp_generic_header_t *generic_header;
  77.     apt_string_assign_n(&message->body, recog_channel->call_back->Result().c_str(), recog_channel->call_back->Result().size(), message->pool);
  78.     /* get/allocate generic header */
  79.     generic_header = mrcp_generic_header_prepare(message);
  80.     if (generic_header) {
  81.         /* set content types */
  82.         apt_string_assign(&generic_header->content_type, "application/x-nlsml", message->pool);
  83.         mrcp_generic_header_property_add(message, GENERIC_HEADER_CONTENT_TYPE);
  84.     }
  85.     return TRUE;
  86. }
  87. /* Raise huawei RECOGNITION-COMPLETE event */
  88. static apt_bool_t huawei_recog_recognition_complete(
  89.     huawei_recog_channel_t *recog_channel, mrcp_recog_completion_cause_e cause)
  90. {
  91.     mrcp_recog_header_t *recog_header;
  92.     /* create RECOGNITION-COMPLETE event */
  93.     mrcp_message_t *message = mrcp_event_create(
  94.         recog_channel->recog_request, RECOGNIZER_RECOGNITION_COMPLETE, recog_channel->recog_request->pool);
  95.     if (!message) {
  96.         return FALSE;
  97.     }
  98.     /* get/allocate recognizer header */
  99.     recog_header = (mrcp_recog_header_t *)mrcp_resource_header_prepare(message);
  100.     if (recog_header) {
  101.         /* set completion cause */
  102.         recog_header->completion_cause = cause;
  103.         mrcp_resource_header_property_add(message, RECOGNIZER_HEADER_COMPLETION_CAUSE);
  104.     }
  105.     /* set request state */
  106.     message->start_line.request_state = MRCP_REQUEST_STATE_COMPLETE;
  107.     if (cause == RECOGNIZER_COMPLETION_CAUSE_SUCCESS) {
  108.         huawei_recog_result_load(recog_channel, message);
  109.     }
  110.     recog_channel->recog_request = NULL;
  111.     /* send asynch event */
  112.     return mrcp_engine_channel_message_send(recog_channel->channel, message);
  113. }
  114. /** Callback is called from MPF engine context to write/send new frame */
  115. static apt_bool_t huawei_recog_stream_write(mpf_audio_stream_t *stream, const mpf_frame_t *frame)
  116. {
  117.     huawei_recog_channel_t *recog_channel = (huawei_recog_channel_t *)stream->obj;
  118.     if (recog_channel->stop_response) {
  119.         /* send asynchronous response to STOP request */
  120.         mrcp_engine_channel_message_send(recog_channel->channel, recog_channel->stop_response);
  121.         recog_channel->stop_response = NULL;
  122.         recog_channel->recog_request = NULL;
  123.         return TRUE;
  124.     }
  125.     bool finish = false;
  126.     // 接收的数据帧转给SDK
  127.     if (recog_channel->recog_request) {
  128.         // 第622行至660行 是unimrcp提供的断句规则,这里禁用中间两个case(因为效果差),直接用华为云的断句规则
  129.         // mpf_detector_event_e det_event = mpf_activity_detector_process(recog_channel->detector, frame);
  130.         // switch (det_event) {
  131.         //     case MPF_DETECTOR_EVENT_ACTIVITY:
  132.         //         apt_log(RECOG_LOG_MARK,
  133.         //             APT_PRIO_INFO,
  134.         //             "Detected Voice Activity " APT_SIDRES_FMT,
  135.         //             MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  136.         //         huawei_recog_start_of_input(recog_channel);
  137.         //         break;
  138.         //     case MPF_DETECTOR_EVENT_INACTIVITY:
  139.         //         apt_log(RECOG_LOG_MARK,
  140.         //             APT_PRIO_INFO,
  141.         //             "Detected Voice Inactivity " APT_SIDRES_FMT,
  142.         //             MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  143.         //         if (recog_channel->rasr_client) {
  144.         //             recog_channel->rasr_client->SendEnd();
  145.         //             recog_channel->rasr_client->Close();
  146.         //         }
  147.         //         finish = true;
  148.         //         huawei_recog_recognition_complete(recog_channel, RECOGNIZER_COMPLETION_CAUSE_SUCCESS);
  149.         //         apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "get RECOGNIZER_COMPLETION_CAUSE_SUCCESS stop recog");
  150.         //         break;
  151.         //     case MPF_DETECTOR_EVENT_NOINPUT:
  152.         //         apt_log(RECOG_LOG_MARK,
  153.         //             APT_PRIO_INFO,
  154.         //             "Detected Noinput " APT_SIDRES_FMT,
  155.         //             MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  156.         //         if (recog_channel->timers_started == TRUE) {
  157.         //             if (recog_channel->rasr_client) {
  158.         //                 recog_channel->rasr_client->SendEnd();
  159.         //                 recog_channel->rasr_client->Close();
  160.         //             }
  161.         //             huawei_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT);
  162.         //         }
  163.         //         finish = true;
  164.         //         break;
  165.         //     default:
  166.         //         break;
  167.         // }
  168.         Status cur_status = recog_channel->call_back->GetStatus();
  169.         switch (cur_status) {
  170.             case Status::voice_start:
  171.                 apt_log(RECOG_LOG_MARK,
  172.                     APT_PRIO_INFO,
  173.                     "Detected Voice Activity " APT_SIDRES_FMT,
  174.                     MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  175.                 huawei_recog_start_of_input(recog_channel);
  176.                 recog_channel->call_back->SetStatus(Status::recognizing);
  177.                 break;
  178.             case Status::voice_end:
  179.                 apt_log(RECOG_LOG_MARK,
  180.                     APT_PRIO_INFO,
  181.                     "Detected Voice Inactivity " APT_SIDRES_FMT,
  182.                     MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  183.                 if (recog_channel->rasr_client) {
  184.                     recog_channel->rasr_client->SendEnd();
  185.                     recog_channel->rasr_client->Close();
  186.                 }
  187.                 finish = true;
  188.                 huawei_recog_recognition_complete(recog_channel, RECOGNIZER_COMPLETION_CAUSE_SUCCESS);
  189.                 apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "get RECOGNIZER_COMPLETION_CAUSE_SUCCESS stop recog");
  190.                 break;
  191.             case Status::exceeded_silence:
  192.                 apt_log(RECOG_LOG_MARK,
  193.                     APT_PRIO_INFO,
  194.                     "Detected Noinput " APT_SIDRES_FMT,
  195.                     MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  196.                 if (recog_channel->timers_started == TRUE) {
  197.                     if (recog_channel->rasr_client) {
  198.                         recog_channel->rasr_client->SendEnd();
  199.                         recog_channel->rasr_client->Close();
  200.                     }
  201.                     huawei_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT);
  202.                 }
  203.                 finish = true;
  204.                 break;
  205.             default:
  206.                 break;
  207.         }        
  208.         if (recog_channel->recog_request) {
  209.             if ((frame->type & MEDIA_FRAME_TYPE_EVENT) == MEDIA_FRAME_TYPE_EVENT) {
  210.                 if (frame->marker == MPF_MARKER_START_OF_EVENT) {
  211.                     apt_log(RECOG_LOG_MARK,
  212.                         APT_PRIO_INFO,
  213.                         "Detected Start of Event " APT_SIDRES_FMT " id:%d",
  214.                         MRCP_MESSAGE_SIDRES(recog_channel->recog_request),
  215.                         frame->event_frame.event_id);
  216.                 } else if (frame->marker == MPF_MARKER_END_OF_EVENT) {
  217.                     apt_log(RECOG_LOG_MARK,
  218.                         APT_PRIO_INFO,
  219.                         "Detected End of Event " APT_SIDRES_FMT " id:%d duration:%d ts",
  220.                         MRCP_MESSAGE_SIDRES(recog_channel->recog_request),
  221.                         frame->event_frame.event_id,
  222.                         frame->event_frame.duration);
  223.                 }
  224.             }
  225.         }
  226.         if (recog_channel->audio_out) {
  227.             fwrite(frame->codec_frame.buffer, 1, frame->codec_frame.size, recog_channel->audio_out);
  228.         }
  229.         if (recog_channel->rasr_client && !finish) {
  230.             recog_channel->rasr_client->SendBinary((unsigned char*)frame->codec_frame.buffer, frame->codec_frame.size);
  231.             apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "Sending [%d] bytes", frame->codec_frame.size);
  232.         }
  233.     }
  234.     return TRUE;
  235. }
  236. static apt_bool_t huawei_recog_msg_signal(
  237.     huawei_recog_msg_type_e type, mrcp_engine_channel_t *channel, mrcp_message_t *request)
  238. {
  239.     apt_bool_t status = FALSE;
  240.     huawei_recog_channel_t *huawei_channel = (huawei_recog_channel_t *)channel->method_obj;
  241.     huawei_recog_engine_t *huawei_engine = huawei_channel->huawei_engine;
  242.     apt_task_t *task = apt_consumer_task_base_get(huawei_engine->task);
  243.     apt_task_msg_t *msg = apt_task_msg_get(task);
  244.     if (msg) {
  245.         huawei_recog_msg_t *huawei_msg;
  246.         msg->type = TASK_MSG_USER;
  247.         huawei_msg = (huawei_recog_msg_t *)msg->data;
  248.         huawei_msg->type = type;
  249.         huawei_msg->channel = channel;
  250.         huawei_msg->request = request;
  251.         status = apt_task_msg_signal(task, msg);
  252.     }
  253.     return status;
  254. }
  255. static apt_bool_t huawei_recog_msg_process(apt_task_t *task, apt_task_msg_t *msg)
  256. {
  257.     huawei_recog_msg_t *huawei_msg = (huawei_recog_msg_t *)msg->data;
  258.     switch (huawei_msg->type) {
  259.         case HUAWEI_RECOG_MSG_OPEN_CHANNEL:
  260.             /* open channel and send asynch response */
  261.             mrcp_engine_channel_open_respond(huawei_msg->channel, TRUE);
  262.             apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "HUAWEI_RECOG_MSG_OPEN_CHANNEL");
  263.             break;
  264.         case HUAWEI_RECOG_MSG_CLOSE_CHANNEL: {
  265.             /* close channel, make sure there is no activity and send asynch response */
  266.             huawei_recog_channel_t *recog_channel = (huawei_recog_channel_t *)huawei_msg->channel->method_obj;
  267.             if (recog_channel->audio_out) {
  268.                 fclose(recog_channel->audio_out);
  269.                 recog_channel->audio_out = NULL;
  270.             }
  271.             apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "HUAWEI_RECOG_MSG_CLOSE_CHANNEL");
  272.             mrcp_engine_channel_close_respond(huawei_msg->channel);
  273.             break;
  274.         }
  275.         case HUAWEI_RECOG_MSG_REQUEST_PROCESS:
  276.             apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "HUAWEI_RECOG_MSG_REQUEST_PROCESS");
  277.             huawei_recog_channel_request_dispatch(huawei_msg->channel, huawei_msg->request);
  278.             break;
  279.         default:
  280.             break;
  281.     }
  282.     return TRUE;
  283. }
复制代码
4.修改asr插件的makefile.am文件(进入路径:unimrcp/plugins): vim huawei-recog/Makefile.am
 
  1. AM_CPPFLAGS                = $(UNIMRCP_PLUGIN_INCLUDES)
  2. plugin_LTLIBRARIES         = huaweirecog.la
  3. huaweirecog_la_SOURCES       = src/huawei_recog_engine.cc
  4. huaweirecog_la_LDFLAGS       = $(UNIMRCP_PLUGIN_OPTS) \
  5.                                -L$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party \
  6.                                -L$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/jsoncpp-install/lib64 \
  7.                                -L$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/gflags-install/lib    \
  8.                                -L$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/glog-install/lib64    \
  9.                                ${CMAKE_SOURCE_DIR}/third_party/openssl-install/lib                                    \
  10.                                -lhuawei_rasr -ssl -lcrypto -ljsoncpp -lgflags_nothreads -stdc++=14
  11. huaweirecog_ladir            = $(libdir)
  12. huaweirecog_la_DATA          = $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/libhuawei_rasr.so       \
  13.                                $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/openssl-install/lib/libcrypto.so  \
  14.                                $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/openssl-install/lib/libssl.so     \
  15.                                $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/jsoncpp-install/lib64/libjsoncpp.so \
  16.                                $(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/gflags-install/lib/libgflags_nothreads.so
  17. include $(top_srcdir)/build/rules/uniplugin.am
  18. UNIMRCP_PLUGIN_INCLUDES      += -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/include  \
  19.                                 -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/websocketpp-src \
  20.                                 -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/jsoncpp-install/include     \
  21.                                 -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/boost-src/                 \
  22.                                 -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/openssl-install/include    \
  23.                                 -I$(top_srcdir)/plugins/huawei-recog/huaweicloud_asr/third_party/gflags-install/include
复制代码
5.修改插件的Makefile.am(进入路径:unimrcp/plugins) vim Makefile.am
[img]https://alliance-communityfile-drcn.dbankcdn.com/FileServer/getFile/cmtybbs/519/984/817/2850086000519984817.20240326113413.53177337896674671853986772877395:50001231000000:2800D1C103FBA1AE304F6A53113D7748560EC474EE74B4A00909EDFA6329E25959C.png[/img]

6.修改最顶层的configure.ac(进入unimrcp根目录):
 
  1. dnl
  2. dnl Autoconf configuration file for UniMRCP.
  3. dnl
  4. dnl Use ./bootstrap to produce a configure script.
  5. dnl
  6. AC_PREREQ(2.59)
  7. AC_INIT([unimrcp],[1.8.0])
  8. AC_CONFIG_AUX_DIR([build])
  9. AC_CONFIG_MACRO_DIR([build/acmacros])
  10. dnl Set ac_macro_dir variable manually for autoconf 2.61 and above.
  11. ac_macro_dir="build/acmacros"
  12. AC_SUBST(ac_aux_dir)
  13. AC_SUBST(ac_macro_dir)
  14. dnl Include m4 macros for libtool 2.
  15. sinclude(build/acmacros/libtool.m4)
  16. sinclude(build/acmacros/ltoptions.m4)
  17. sinclude(build/acmacros/ltsugar.m4)
  18. sinclude(build/acmacros/ltversion.m4)
  19. sinclude(build/acmacros/lt~obsolete.m4)
  20. AC_PREFIX_DEFAULT(/usr/local/unimrcp)
  21. dnl Define the directory layout.
  22. APR_ENABLE_LAYOUT(classic, [plugindir logdir vardir])
  23. AC_SUBST(plugindir)
  24. AC_SUBST(logdir)
  25. AC_SUBST(vardir)
  26. dnl Reparse the configure arguments.
  27. APR_PARSE_ARGUMENTS
  28. dnl Generate ./config.nice to reuse ./configure command-line.
  29. APR_CONFIG_NICE(config.nice)
  30. AM_INIT_AUTOMAKE([no-define nostdinc foreign subdir-objects])
  31. dnl Enable silent build rules available since automake 1.11.
  32. m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
  33. dnl Set default language.
  34. AC_LANG_C
  35. AC_PROG_CC
  36. AC_PROG_CXX
  37. AM_PROG_CC_C_O
  38. AC_PROG_INSTALL
  39. dnl Skip detection of Fortran.
  40. m4_undefine([AC_PROG_F77])
  41. m4_defun([AC_PROG_F77],[])
  42. AC_PROG_LIBTOOL
  43. dnl Suppress warning: ar: 'u' modifier ignored since 'D' is the default
  44. AC_SUBST(AR_FLAGS, [cr])
  45. dnl Do not use autoconf generated compiler DEFS.
  46. rm confdefs.h
  47. touch confdefs.h
  48. dnl Check for C compiler vendor.
  49. AX_COMPILER_VENDOR
  50. dnl Search for pkg-config.
  51. AC_PATH_PROG(PKG_CONFIG, pkg-config, no)
  52. if test "x$PKG_CONFIG" = "xno"; then
  53.     AC_MSG_ERROR([pkg-config is missing])
  54. fi
  55. dnl Get version information.
  56. get_version="$srcdir/build/get-version.sh"
  57. version_hdr="$srcdir/build/uni_version.h"
  58. plugin_version_hdr="$srcdir/libs/mrcp-engine/include/mrcp_engine_plugin.h"
  59. UNI_DOTTED_VERSION="`$get_version all $version_hdr UNI`"
  60. UNI_LT_VERSION="-version-info `$get_version libtool $version_hdr UNI`"
  61. PLUGIN_LT_VERSION="-version-info `$get_version libtool $plugin_version_hdr PLUGIN`"
  62. AC_SUBST(UNI_DOTTED_VERSION)
  63. AC_SUBST(UNI_LT_VERSION)
  64. AC_SUBST(PLUGIN_LT_VERSION)
  65. echo "UniMRCP Version: ${UNI_DOTTED_VERSION}"
  66. dnl Check for the APR and APR-util libraries.
  67. UNIMRCP_CHECK_APR
  68. dnl Check for the Sofia-SIP library.
  69. UNIMRCP_CHECK_SOFIA
  70. dnl Enable AMR codec.
  71. AC_ARG_ENABLE(amr-codec,
  72.     [AC_HELP_STRING([--enable-amr-codec  ],[enable AMR codec])],
  73.     [enable_amr_codec="$enableval"],
  74.     [enable_amr_codec="no"])
  75. AC_MSG_NOTICE([enable AMR codec: $enable_amr_codec])
  76. if test "${enable_amr_codec}" != "no"; then
  77.     dnl Check for the OpenCORE AMR library.
  78.     UNIMRCP_CHECK_OPENCORE_AMR
  79.     dnl Check for the VO AMRWBENC library.
  80.     UNIMRCP_CHECK_VO_AMRWBENC
  81. fi
  82. AM_CONDITIONAL([UNIMRCP_AMR_CODEC],[test "${enable_amr_codec}" != "no"])
  83. dnl Enable inter-library dependencies.
  84. AC_ARG_ENABLE(interlib-deps,
  85.     [AC_HELP_STRING([--disable-interlib-deps  ],[disable inter-library dependencies (might break builds)])],
  86.     [enable_interlib_deps="$enableval"],
  87.     [enable_interlib_deps="yes"])
  88. AC_MSG_NOTICE([enable inter-library dependencies: $enable_interlib_deps])
  89. if test "${enable_interlib_deps}" == "yes"; then
  90.     link_all_deplibs=yes
  91.     link_all_deplibs_CXX=yes
  92. else
  93.     link_all_deplibs=no
  94.     link_all_deplibs_CXX=no
  95. fi
  96. dnl Enable maintainer mode.
  97. AC_ARG_ENABLE(maintainer-mode,
  98.     [AC_HELP_STRING([--enable-maintainer-mode  ],[turn on debugging and compile time warnings])],
  99.     [enable_maintainer_mode="$enableval"],
  100.     [enable_maintainer_mode="no"])
  101. AC_MSG_NOTICE([enable maintainer mode: $enable_maintainer_mode])
  102. if test "${enable_maintainer_mode}" != "no"; then
  103.     APR_ADDTO(CFLAGS,-g)
  104.     if test "x${ax_cv_c_compiler_vendor}"  =  "xgnu" ; then
  105.         APR_ADDTO(CFLAGS,-Wall -Werror)
  106.     fi
  107. fi
  108. dnl UniMRCP client library.
  109. AC_ARG_ENABLE(client-lib,
  110.     [AC_HELP_STRING([--disable-client-lib  ],[exclude unimrcpclient lib from build])],
  111.     [enable_client_lib="$enableval"],
  112.     [enable_client_lib="yes"])
  113. AM_CONDITIONAL([UNIMRCP_CLIENT_LIB],[test "${enable_client_lib}" = "yes"])
  114. dnl Sample UniMRCP client application in C.
  115. AC_ARG_ENABLE(client-app,
  116.     [AC_HELP_STRING([--disable-client-app  ],[exclude sample unimrcpclient app from build])],
  117.     [enable_client_app="$enableval"],
  118.     [enable_client_app="yes"])
  119. AM_CONDITIONAL([UNIMRCP_CLIENT_APP],[test "${enable_client_lib}" = "yes" && test "${enable_client_app}" = "yes"])
  120. dnl Sample UniMRCP client application in C++.
  121. AC_ARG_ENABLE(umc,
  122.     [AC_HELP_STRING([--disable-umc         ],[exclude sample unimrcpclient C++ app from build])],
  123.     [enable_umc="$enableval"],
  124.     [enable_umc="yes"])
  125. AM_CONDITIONAL([UMC],[test "${enable_client_lib}" = "yes" && test "${enable_umc}" = "yes"])
  126. dnl Miscellaneous ASR client library and application.
  127. AC_ARG_ENABLE(asr-client,
  128.     [AC_HELP_STRING([--disable-asr-client  ],[exclude misc ASR client lib and app from build])],
  129.     [enable_asr_client="$enableval"],
  130.     [enable_asr_client="yes"])
  131. AM_CONDITIONAL([ASR_CLIENT],[test "${enable_client_lib}" = "yes" && test "${enable_asr_client}" = "yes"])
  132. AM_CONDITIONAL([COMMON_CLIENT_DATA],[test "${enable_client_app}" = "yes" || test "${enable_umc}" = "yes" ||test "${enable_asr_client}" = "yes"])
  133. dnl UniMRCP server library.
  134. AC_ARG_ENABLE(server-lib,
  135.     [AC_HELP_STRING([--disable-server-lib  ],[exclude unimrcpserver lib from build])],
  136.     [enable_server_lib="$enableval"],
  137.     [enable_server_lib="yes"])
  138. AM_CONDITIONAL([UNIMRCP_SERVER_LIB],[test "${enable_server_lib}" = "yes"])
  139. dnl UniMRCP server application.
  140. AC_ARG_ENABLE(server-app,
  141.     [AC_HELP_STRING([--disable-server-app  ],[exclude unimrcpserver app from build])],
  142.     [enable_server_app="$enableval"],
  143.     [enable_server_app="yes"])
  144. AM_CONDITIONAL([UNIMRCP_SERVER_APP],[test "${enable_server_lib}" = "yes" && test "${enable_server_app}" = "yes"])
  145. dnl Demo synthesizer plugin.
  146. UNI_PLUGIN_ENABLED(demosynth)
  147. AM_CONDITIONAL([DEMOSYNTH_PLUGIN],[test "${enable_demosynth_plugin}" = "yes"])
  148. dnl Huawei recognizer plugin.
  149. UNI_PLUGIN_ENABLED(huaweirecog)
  150. AM_CONDITIONAL([HUAWEIRECOG_PLUGIN],[test "${enable_huaweirecog_plugin}" = "yes"])
  151. dnl Demo verifier plugin.
  152. UNI_PLUGIN_ENABLED(demoverifier)
  153. AM_CONDITIONAL([DEMOVERIFIER_PLUGIN],[test "${enable_demoverifier_plugin}" = "yes"])
  154. dnl Recorder plugin.
  155. UNI_PLUGIN_ENABLED(recorder)
  156. AM_CONDITIONAL([RECORDER_PLUGIN],[test "${enable_recorder_plugin}" = "yes"])
  157. dnl Enable test suites.
  158. AC_ARG_ENABLE(test-suites,
  159.     [AC_HELP_STRING([--enable-test-suites  ],[build test suites])],
  160.     [enable_test_suites="$enableval"],
  161.     [enable_test_suites="no"])
  162. AM_CONDITIONAL([TEST_SUITES],[test "${enable_test_suites}" != "no"])
  163. AM_CONDITIONAL(ISMAC, [test `uname -s` = Darwin])
  164. AC_CONFIG_FILES([
  165.     Makefile
  166.     libs/Makefile
  167.     libs/apr-toolkit/Makefile
  168.     libs/mpf/Makefile
  169.     libs/mrcp/Makefile
  170.     libs/mrcp-signaling/Makefile
  171.     libs/mrcpv2-transport/Makefile
  172.     libs/mrcp-engine/Makefile
  173.     libs/mrcp-server/Makefile
  174.     libs/mrcp-client/Makefile
  175.     libs/uni-rtsp/Makefile
  176.     modules/Makefile
  177.     modules/mrcp-sofiasip/Makefile
  178.     modules/mrcp-unirtsp/Makefile
  179.     plugins/Makefile
  180.     plugins/mrcp-recorder/Makefile
  181.     plugins/demo-synth/Makefile
  182.     plugins/huawei-recog/Makefile
  183.     plugins/demo-verifier/Makefile
  184.     platforms/Makefile
  185.     platforms/libunimrcp-server/Makefile
  186.     platforms/libunimrcp-client/Makefile
  187.     platforms/unimrcp-server/Makefile
  188.     platforms/unimrcp-client/Makefile
  189.     platforms/libasr-client/Makefile
  190.     platforms/asr-client/Makefile
  191.     platforms/umc/Makefile
  192.     tests/Makefile
  193.     tests/apttest/Makefile
  194.     tests/mpftest/Makefile
  195.     tests/mrcptest/Makefile
  196.     tests/rtsptest/Makefile
  197.     tests/strtablegen/Makefile
  198.     build/Makefile
  199.     build/pkgconfig/Makefile
  200.     build/pkgconfig/unimrcpclient.pc
  201.     build/pkgconfig/unimrcpserver.pc
  202.     build/pkgconfig/unimrcpplugin.pc
  203.     conf/Makefile
  204.     data/Makefile
  205.     docs/doxygen.conf
  206. ])
  207. AC_OUTPUT
  208. echo
  209. echo '****************************** REPORT ******************************'
  210. echo
  211. echo UniMRCP version............... : $UNI_DOTTED_VERSION
  212. echo
  213. echo APR version................... : $apr_version
  214. echo APR-util version.............. : $apu_version
  215. echo Sofia-SIP version............. : $sofia_version
  216. if test "${enable_amr_codec}" != "no"; then
  217. echo
  218. echo OpenCORE AMR version.......... : $opencore_amr_version
  219. echo VO AMRWBENC version........... : $vo_amrwbenc_version
  220. fi
  221. echo
  222. echo Compiler...................... : $CC
  223. echo Compiler flags................ : $CFLAGS
  224. echo Preprocessor definitions...... : $CPPFLAGS
  225. echo Linker flags.................. : $LDFLAGS
  226. echo
  227. echo UniMRCP client lib............ : $enable_client_lib
  228. echo Sample UniMRCP client app..... : $enable_client_app
  229. echo Sample UMC C++ client app..... : $enable_umc
  230. echo Misc ASR client lib and app... : $enable_asr_client
  231. echo
  232. echo UniMRCP server lib............ : $enable_server_lib
  233. echo UniMRCP server app............ : $enable_server_app
  234. echo
  235. echo Demo synthesizer plugin....... : $enable_demosynth_plugin
  236. echo Huawei recognizer plugin........ : $enable_huaweirecog_plugin
  237. echo Demo verifier plugin.......... : $enable_demoverifier_plugin
  238. echo Recorder plugin............... : $enable_recorder_plugin
  239. echo
  240. echo Installation layout........... : $layout_name
  241. echo Installation directory........ : $prefix
  242. echo
  243. echo '********************************************************************'
复制代码
7.添加asr配置文件(进入unimrcp根目录),这些都是必填项,按照备注获取相应的asr鉴权信息:vim conf/asr_info.conf

8.重新编译unimrcp,默认会安装在/usr/local/unimrcp中
  1. ./bootstrap
  2. make -j
  3. make install
复制代码
9.unimrcp代码结构展示

 
  1. /usr/local/unimrcp
  2. ├── bin                 # 可执行文件
  3. ├── conf                # 配置文件,可以设置访问IP和端口号
  4. ├── data                # 默认提供的一些音频文件
  5. ├── include             # 程序必须的头文件
  6. ├── lib                 # 依赖的库文件
  7. ├── log                 # 日志路径
  8. ├── plugin              # 插件所依赖的库文件
  9. └── var                 # 保存的音频文件
复制代码
10.运行结果展示
启动服务端 /usr/local/unimrcp/bin/unimrcpserver
 
  1. (base) [root@asr-build bin]# ./unimrcpserver
  2. 2024-03-11 17:17:14:704844 [NOTICE] UniMRCP Server [1.8.0]
  3. 2024-03-11 17:17:14:704913 [INFO]   APR [1.5.2]
  4. 2024-03-11 17:17:14:704924 [NOTICE] Create MRCP Server
  5. 2024-03-11 17:17:14:704946 [NOTICE] Open Config File [/usr/local/unimrcp/conf/unimrcpserver.xml]
  6. 2024-03-11 17:17:14:705077 [INFO]   Set Property ip:127.0.0.1
  7. 2024-03-11 17:17:14:705086 [INFO]   Register Codec [PCMU]
  8. 2024-03-11 17:17:14:705088 [INFO]   Register Codec [PCMA]
  9. 2024-03-11 17:17:14:705090 [INFO]   Register Codec [L16]
  10. 2024-03-11 17:17:14:705092 [INFO]   Register Codec [G722]
  11. 2024-03-11 17:17:14:705127 [NOTICE] Register Resource [speechsynth]
  12. 2024-03-11 17:17:14:705132 [NOTICE] Register Resource [speechrecog]
  13. 2024-03-11 17:17:14:705135 [NOTICE] Register Resource [recorder]
  14. 2024-03-11 17:17:14:705138 [NOTICE] Register Resource [speakverify]
  15. 2024-03-11 17:17:14:705141 [INFO]   Register Resource Factory
  16. 2024-03-11 17:17:14:705157 [NOTICE] Create SofiaSIP Agent [SIP-Agent-1] [1.12.11-239-g54ef3e2] sip:127.0.0.1:8060;transport=udp,tcp
  17. 2024-03-11 17:17:14:705163 [INFO]   Register Signaling Agent [SIP-Agent-1]
  18. 2024-03-11 17:17:14:705178 [NOTICE] Create RTSP Server [RTSP-Agent-1] 127.0.0.1:1554 [100] connection timeout [600 sec]
  19. 2024-03-11 17:17:14:705258 [INFO]   Register Signaling Agent [RTSP-Agent-1]
  20. 2024-03-11 17:17:14:705266 [NOTICE] Create MRCPv2 Agent [MRCPv2-Agent-1] 127.0.0.1:1544 [100]
  21. 2024-03-11 17:17:14:705290 [INFO]   Register Connection Agent [MRCPv2-Agent-1]
  22. 2024-03-11 17:17:14:705297 [NOTICE] Create Media Engine [Media-Engine-1]
  23. 2024-03-11 17:17:14:705305 [INFO]   Register Media Engine [Media-Engine-1]
  24. 2024-03-11 17:17:14:705309 [NOTICE] Create RTP Termination Factory 127.0.0.1:[5000,6000]
  25. 2024-03-11 17:17:14:705311 [INFO]   Register RTP Termination Factory [RTP-Factory-1]
  26. 2024-03-11 17:17:14:705316 [INFO]   Load Plugin [Demo-Synth-1] [/usr/local/unimrcp/plugin/demosynth.so]
  27. 2024-03-11 17:17:14:705393 [INFO]   Register MRCP Engine [Demo-Synth-1]
  28. 2024-03-11 17:17:14:705397 [INFO]   Load Plugin [Huawei-Recog-1] [/usr/local/unimrcp/plugin/huaweirecog.so]
  29. 2024-03-11 17:17:14:712580 [INFO]   Register MRCP Engine [Huawei-Recog-1]
  30. 2024-03-11 17:17:14:712599 [INFO]   Load Plugin [Demo-Verifier-1] [/usr/local/unimrcp/plugin/demoverifier.so]
  31. 2024-03-11 17:17:14:712677 [INFO]   Register MRCP Engine [Demo-Verifier-1]
  32. 2024-03-11 17:17:14:712682 [INFO]   Load Plugin [Recorder-1] [/usr/local/unimrcp/plugin/mrcprecorder.so]
  33. 2024-03-11 17:17:14:712739 [INFO]   Register MRCP Engine [Recorder-1]
  34. 2024-03-11 17:17:14:712755 [INFO]   Register RTP Settings [RTP-Settings-1]
  35. 2024-03-11 17:17:14:712764 [NOTICE] Create MRCPv2 Profile [uni2]
  36. 2024-03-11 17:17:14:712773 [INFO]   Associate Resource [speechsynth] to Engine [Demo-Synth-1] in Profile [uni2]
  37. 2024-03-11 17:17:14:712776 [INFO]   Associate Resource [speechrecog] to Engine [Huawei-Recog-1] in Profile [uni2]
  38. 2024-03-11 17:17:14:712779 [INFO]   Associate Resource [recorder] to Engine [Recorder-1] in Profile [uni2]
  39. 2024-03-11 17:17:14:712782 [INFO]   Associate Resource [speakverify] to Engine [Demo-Verifier-1] in Profile [uni2]
  40. 2024-03-11 17:17:14:712786 [INFO]   Register Profile [uni2]
  41. 2024-03-11 17:17:14:712790 [NOTICE] Create MRCPv1 Profile [uni1]
  42. 2024-03-11 17:17:14:712793 [INFO]   Associate Resource [speechsynth] to Engine [Demo-Synth-1] in Profile [uni1]
  43. 2024-03-11 17:17:14:712796 [INFO]   Associate Resource [speechrecog] to Engine [Huawei-Recog-1] in Profile [uni1]
  44. 2024-03-11 17:17:14:712799 [INFO]   Associate Resource [recorder] to Engine [Recorder-1] in Profile [uni1]
  45. 2024-03-11 17:17:14:712802 [INFO]   Associate Resource [speakverify] to Engine [Demo-Verifier-1] in Profile [uni1]
  46. 2024-03-11 17:17:14:712805 [INFO]   Register Profile [uni1]
  47. 2024-03-11 17:17:14:712810 [INFO]   Start Task [MRCP Server]
  48. >2024-03-11 17:17:14:712891 [INFO]   Open Engine [Demo-Verifier-1]
  49. 2024-03-11 17:17:14:712906 [INFO]   Start Task [Demo Verifier Engine]
  50. 2024-03-11 17:17:14:712963 [INFO]   Open Engine [Huawei-Recog-1]
  51. 2024-03-11 17:17:14:712966 [INFO]   Start Task [Huawei Recog Engine]
  52. 2024-03-11 17:17:14:712990 [INFO]   Open Engine [Recorder-1]
  53. 2024-03-11 17:17:14:712993 [INFO]   Open Engine [Demo-Synth-1]
  54. 2024-03-11 17:17:14:712995 [INFO]   Start Task [Demo Synth Engine]
  55. 2024-03-11 17:17:14:713019 [INFO]   Start Task [SIP-Agent-1]
  56. 2024-03-11 17:17:14:713037 [INFO]   Start Task [RTSP-Agent-1]
  57. 2024-03-11 17:17:14:713159 [INFO]   Start Task [MRCPv2-Agent-1]
  58. 2024-03-11 17:17:14:713313 [INFO]   Start Task [Media-Engine-1]
  59. sres: /etc/resolv.conf: unknown option                     
  60. 2024-03-11 17:17:14:714108 [NOTICE] MRCP Server Started
复制代码
启动客户端:/usr/local/unimrcp/umc,然后在交互窗口实行run recog,获取识别结果

四、unimrcp定制化改动

1. 修改xml文件(在conf文件夹中),指定访问地址和端口号。端口号我们采用默认端口,IP地址指定为本地IP

unimrcpclient.xml修改:

unimrcpserver.xml修改:

2. 利用unimrcp调整ASR断句参数

unimrcp的header里面提供了一组断句参数,来控制响应时间/句子长段(好比,说完一段话之后,等待多少毫秒,给出结果)。各参数名及其含义如下(在路径data/params_default.txt下)
  1. No-Input-Timeout: 5000       -----> 用户接到电话后超过5000ms没有声音,就结束识别 等价于华为云ASR的vad_head
  2. Recognition-Timeout: 20000    ------> 用户一直说话,最长能说多久(默认20s),结束识别。等价于华为云ASR的max_seconds
  3. Speech-Complete-Timeout: 400  -------> 用户开始说话后,停顿多久(默认400ms)就结束说话
复制代码
2.1 通过哀求参数传递断句参数

以上参数均是通过哀求体的header进行传递,参考文档:https://www.unimrcp.org/manuals/html/GoogleSRUsageManual.html
可以通过unimrcp打印的日记,查看参数设置是否生效:

2.2 逼迫指定断句参数

为了方便管理,发起对所有通话保留相同的断句参数,具体设置如下:
  1. if(recog_header) {
  2.         if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_START_INPUT_TIMERS) == TRUE) {
  3.             recog_channel->timers_started = recog_header->start_input_timers;
  4.         }
  5.         if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_NO_INPUT_TIMEOUT) == TRUE) {
  6.             mpf_activity_detector_noinput_timeout_set(recog_channel->detector,10000); // 强制指定 如果10s中无人说话,就结束
  7.         }
  8.         if(mrcp_resource_header_property_check(request,RECOGNIZER_HEADER_SPEECH_COMPLETE_TIMEOUT) == TRUE) {
  9.             mpf_activity_detector_silence_timeout_set(recog_channel->detector, 600); // 强制指定600ms
  10.         }
  11.     }
复制代码
3. 利用华为云ASR做智能短句

华为云ASR提供的sentence模式,可实用于外呼、数字人场景,通过设置vad_head/vad_tail/max_seconds参数可以智能短句,相关代码变动如下
  1.   if (recog_channel->recog_request) {
  2.         // 第622行至660行 是unimrcp提供的断句规则,这里禁用中间两个case(因为效果差),直接用华为云的断句规则
  3.         // mpf_detector_event_e det_event = mpf_activity_detector_process(recog_channel->detector, frame);
  4.         // switch (det_event) {
  5.         //     case MPF_DETECTOR_EVENT_ACTIVITY:
  6.         //         apt_log(RECOG_LOG_MARK,
  7.         //             APT_PRIO_INFO,
  8.         //             "Detected Voice Activity " APT_SIDRES_FMT,
  9.         //             MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  10.         //         huawei_recog_start_of_input(recog_channel);
  11.         //         break;
  12.         //     case MPF_DETECTOR_EVENT_INACTIVITY:
  13.         //         apt_log(RECOG_LOG_MARK,
  14.         //             APT_PRIO_INFO,
  15.         //             "Detected Voice Inactivity " APT_SIDRES_FMT,
  16.         //             MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  17.         //         if (recog_channel->rasr_client) {
  18.         //             recog_channel->rasr_client->SendEnd();
  19.         //             recog_channel->rasr_client->Close();
  20.         //         }
  21.         //         finish = true;
  22.         //         huawei_recog_recognition_complete(recog_channel, RECOGNIZER_COMPLETION_CAUSE_SUCCESS);
  23.         //         apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "get RECOGNIZER_COMPLETION_CAUSE_SUCCESS stop recog");
  24.         //         break;
  25.         //     case MPF_DETECTOR_EVENT_NOINPUT:
  26.         //         apt_log(RECOG_LOG_MARK,
  27.         //             APT_PRIO_INFO,
  28.         //             "Detected Noinput " APT_SIDRES_FMT,
  29.         //             MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  30.         //         if (recog_channel->timers_started == TRUE) {
  31.         //             if (recog_channel->rasr_client) {
  32.         //                 recog_channel->rasr_client->SendEnd();
  33.         //                 recog_channel->rasr_client->Close();
  34.         //             }
  35.         //             huawei_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT);
  36.         //         }
  37.         //         finish = true;
  38.         //         break;
  39.         //     default:
  40.         //         break;
  41.         // }
  42.         Status cur_status = recog_channel->call_back->GetStatus();
  43.         switch (cur_status) {
  44.             case Status::voice_start:
  45.                 apt_log(RECOG_LOG_MARK,
  46.                     APT_PRIO_INFO,
  47.                     "Detected Voice Activity " APT_SIDRES_FMT,
  48.                     MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  49.                 huawei_recog_start_of_input(recog_channel);
  50.                 recog_channel->call_back->SetStatus(Status::recognizing);
  51.                 break;
  52.             case Status::voice_end:
  53.                 apt_log(RECOG_LOG_MARK,
  54.                     APT_PRIO_INFO,
  55.                     "Detected Voice Inactivity " APT_SIDRES_FMT,
  56.                     MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  57.                 if (recog_channel->rasr_client) {
  58.                     recog_channel->rasr_client->SendEnd();
  59.                     recog_channel->rasr_client->Close();
  60.                 }
  61.                 finish = true;
  62.                 huawei_recog_recognition_complete(recog_channel, RECOGNIZER_COMPLETION_CAUSE_SUCCESS);
  63.                 apt_log(RECOG_LOG_MARK, APT_PRIO_INFO, "get RECOGNIZER_COMPLETION_CAUSE_SUCCESS stop recog");
  64.                 break;
  65.             case Status::exceeded_silence:
  66.                 apt_log(RECOG_LOG_MARK,
  67.                     APT_PRIO_INFO,
  68.                     "Detected Noinput " APT_SIDRES_FMT,
  69.                     MRCP_MESSAGE_SIDRES(recog_channel->recog_request));
  70.                 if (recog_channel->timers_started == TRUE) {
  71.                     if (recog_channel->rasr_client) {
  72.                         recog_channel->rasr_client->SendEnd();
  73.                         recog_channel->rasr_client->Close();
  74.                     }
  75.                     huawei_recog_recognition_complete(recog_channel,RECOGNIZER_COMPLETION_CAUSE_NO_INPUT_TIMEOUT);
  76.                 }
  77.                 finish = true;
  78.                 break;
  79.             default:
  80.                 break;
  81.         }   
复制代码
五、快速搭建unimrcp服务

作者按照前四章渐渐实行,并在GitHub和Gitee上发布了修改后的代码,大家可以直接拿来使用。
本代码仓提供两种安装方式供用户选择
方法一: 源码安装


  • 先下载安装华为云实时语音识别SDK,安装指南参考https://bbs.huaweicloud.com/blogs/392949
  • 将安装的华为云ASR SDK放入plugins/huawei_recog/huaweicloud_asr 】
  • 进入根目录/unimcrp-deps-1.6.0,安装安装unimrcp的依靠库
  1. ./build-dep-libs.sh
复制代码
4.进入根目录,安装unimrcp
  1. ./bootstrap
  2. ./configure
  3. make
  4. make install
复制代码
5.unimrcp默认安装在/usr/local/unimrcp中,使用前需要手动修改asr_info.conf中的配置项
方法二:直接使用release库
  1. wget https://gitee.com/computervisionlearner/unimrcp_with_huaweicloud_asr/releases/download/v0.0.1/unimrcp-1.8.0_unimrcp-deps-1.6.0_binary.tar.gz
  2. tar -xzvf unimrcp-1.8.0_unimrcp-deps-1.6.0_binary.tar.gz
复制代码
 
点击关注,第一时间了解华为云新鲜技术~ 
 

免责声明:如果侵犯了您的权益,请联系站长,我们会及时删除侵权内容,谢谢合作!更多信息从访问主页:qidao123.com:ToB企服之家,中国第一个企服评测及商务社交产业平台。

本帖子中包含更多资源

您需要 登录 才可以下载或查看,没有账号?立即注册

x
回复

使用道具 举报

0 个回复

倒序浏览

快速回复

您需要登录后才可以回帖 登录 or 立即注册

本版积分规则

欢乐狗

金牌会员
这个人很懒什么都没写!

标签云

快速回复 返回顶部 返回列表