Voice in Home Assistant

https://www.home-assistant.io/voice_control/
https://www.home-assistant.io/voice_control/start_assist_from_dashboard/


Make sure your assistant has a speech-to-text engine , otherwise you’ll only get keyboard input.

https://github.com/rhasspy/piper/
https://github.com/rhasspy/wyoming-piper

git clone https://github.com/rhasspy/wyoming-satellite.git

202403
I have been using Raspberry Pi Zero, 3A and 3B models with Rhasspy as voice assistant satellites, and decided now is the time to swap to HA Voice Assist.


Software Requirement

.Server  software
Raspberry Pi 3B
└── .Debain 11 OS
    ├── Local voice service
    │   ├── Whisper  #standalone service,For speech-to-text
    │   │   ├── /home/homeassistant/.homeassistant/configuration.yaml
    │   │   ├── turn_light_off_lounge.yaml
    │   │   ├── turn_light_on_bedroom.yaml
    │   │   └── Bluetooth integration  --> 
    │   ├──  Piper
    │   └── OpenWakeWord             https://www.home-assistant.io/integrations/wake_word/
    ├── wyoming integration   - > Connect whisper，piper，wakeword  external server to home assistant core with ip and port
    │   ├── Whisper  #standalone service,For speech-to-text
    │   │   ├── /home/homeassistant/.homeassistant/configuration.yaml
    │   │   ├── turn_light_off_lounge.yaml
    │   │   ├── turn_light_on_bedroom.yaml
    │   │   └── Bluetooth integration  --> 
    │   ├──  Piper
    │   └── OpenWakeWord             https://www.home-assistant.io/integrations/wake_word/
    │   └── wyoming integration         Connect whisper，piper，wakeword  to home assistant core
    │       └── Whisper     selected localhost and for port  pointed at port10200   
    ├── Assist Pipeline integration   - >  按需要组合各种插件
    │   ├── Whisper  #standalone service,For speech-to-text
    │   │   ├── /home/homeassistant/.homeassistant/configuration.yaml
    │   │   ├── turn_light_off_lounge.yaml
    │   │   ├── turn_light_on_bedroom.yaml
    │   │   └── Bluetooth integration  --> 
    │   ├──  Piper
    │   └── OpenWakeWord             https://www.home-assistant.io/integrations/wake_word/
    │   └── wyoming integration         Connect whisper，piper，wakeword  to home assistant core
    │       └── Whisper     selected localhost and for port  pointed at port10200               
    ├── ha core service  -->/etc/systemd/system/[email protected]
    ├── ssh server 
    ├── nginx with let’s 
    ├── frp 
    └── git

Make Python virtual environment + Home Assistant Core run automatically

Assist Pipeline integration
The  Assist Pipeline integration allows you to configure all components that make up a voice assistant in a single place.https://www.home-assistant.io/integrations/assist_pipeline/

Go to Settings > Voice assistants and select Add assistant.
for example
然后装了whisper paper openwakeword assist microphoe, 配置好后启用。语音唤醒正常，外放正常。
但是中文对话没有反应。把whisper和piper更换成homeassistant cloud，可以正常语音控制HA里的配件了，比方说开灯关灯。但是问她问题，比方说天气怎样？还是回复我不能理解你的问题。跟谷歌的中文语音差的挺远的。
https://bbs.hassbian.com/thread-24450-1-1.html

wyoming integration
Connect whisper，piper，wakeword  to home assistant core via  wyoming integration -> connects external voice services to Home Assistant
https://www.home-assistant.io/integrations/wyoming/

for example

On Home Assistant, install HA add-ons - Whisper (speech-to-text), Piper (text to speech), openWakeWord and Wyoming (to link them all together).



You can run Piper with Python or download a binary release:detail:https://github.com/rhasspy/piper/

pip install piper-tts

3.
Home Assistant Add-on: Whisper https://github.com/home-assistant/addons/tree/b2c5fdcaa091bf8b29ad10bfcc20ea9283f7bbc4/whisper
Home Assistant Add-on: Piper https://github.com/home-assistant/addons/tree/b2c5fdcaa091bf8b29ad10bfcc20ea9283f7bbc4/piper

Hardware Requirement

Any ESPHome device with a microphone
Android phones, tablets, and smart watches
Old school analog phones (with an adapter )

ESPHome device with a microphone
A voice assistant also needs hardware, so today we’re launching ESPHome support for Assist and; to top it off: we’re launching the World’s Most Private Voice Assistant.
https://esphome.io/components/voice_assistant

ESP32-S3-BOX or ESP32-S3-BOX-Lite

https://www.home-assistant.io/voice_control/s3_box_voice_assistant/



https://community.home-assistant.io/t/year-of-the-voice-chapter-5/655319

Hardware ESPHome

Exposing your devices to Assist


#Assist Pipeline integration

Speech to text
Intent recognition （意图识别）
Text to speech

A voice assistant has a lot of different parts:

hot word detection, 
speech to text,
 intent recognition,
 intent execution,
 text to speech.

The most important part is the

intent recognition and intent execution. 

We need to be able to understand your commands and execute them.

https://github.com/home-assistant/intents

https://www.home-assistant.io/integrations/conversation

Run a single-container speech-to-text service on Docker

Run a single-container text-to-speech service on Docker

Text to speech containers with Docker.Install and run Text to speech containers with Docker.

Piper is 10200 and whisper should be 10300 with default settings

Composing Voice Assistants

Assist Pipelines

The new Assist Pipeline integration allows you to configure all components that make up a voice assistant in a single place.
https://developers.home-assistant.io/docs/voice/pipelines/
https://www.home-assistant.io/blog/2023/04/27/year-of-the-voice-chapter-2/


Speech to text
Intent recognition （意图识别）
Text to speech

The fully local voice assistant



Installing a local Assist pipeline
https://www.home-assistant.io/voice_control/voice_remote_local_assistant

Piper  -  LOCAL TEXT-TO-SPEECH

Piper is a text-to-speech system created by Nabu Casa, designed to run locally on the Raspberry Pi 4.

https://www.home-assistant.io/blog/2023/07/20/year-of-the-voice-chapter-3/

To make quality text-to-speech running locally possible, we’ve had to create our own text-to-speech system that is optimized for running on a Raspberry Pi 4. It’s called Piper.

https://rhasspy.github.io/piper-samples/


run Piper as a standalone Docker container
https://hub.docker.com/r/rhasspy/wyoming-piper


Piper是一个由Nabu Casa创建的文本到语音转换系统，它旨在在Raspberry Pi 4上本地运行。作为一款功能强大的语音合成工具，Piper可以轻松地将任何文本转换为高质量的语音输出，无需连接互联网。这种离线解决方案不仅可以提供快速、可靠的的服务，还能保护用户隐私。

此外，Piper还具有多种语言支持和高度的可定制性。用户可以根据自己的需求调整语音的音调、语速和音量等参数，实现个性化的语音输出效果。而且，Piper还支持多种音频格式，包括MP3、WAV和FLAC等，方便用户在不同场景下使用。

这也是目前HomeAssistant官方极力推广的离线语音解决方案

Quality
Voices are trained at one of 4 "quality" levels:

x_low - 16Khz audio, 5-7M params
low - 16Khz audio, 15-20M params
medium - 22.05Khz audio, 15-20M params
high - 22.05Khz audio, 28-32M params

项目地址
• https://github.com/rhasspy/piper

• https://github.com/rhasspy/rhasspy3

https://mp.weixin.qq.com/s/9eJKemaUEF07t7CW0OPaVg


Install
http://192.168.2.117:8123/hassio/addon/core_piper/info

Whisper.- LOCAL speech-to-text

Whisper is an open source speech-to-text model created by OpenAI that runs locally. Since its release in 2022, Whisper has been improved by the open source community to run on less powerful hardware by projects such as whisper.cpp and faster-whisper. In less than a year of progress, Whisper is now capable of providing speech-to-text for dozens of languages on small servers and single-board computers!

https://www.home-assistant.io/blog/2023/04/27/year-of-the-voice-chapter-2/


Whisper - Install

http://192.168.2.117:8123/hassio/addon/core_whisper/info

Wyoming Protocol -  integration

https://www.home-assistant.io/integrations/wyoming/

The Wyoming Protocol integration was introduced in Home Assistant 2023.5,

The Wyoming integration connects external voice services to Home Assistant using a small protocol. This enables Assist to use a variety of local speech-to-text, text-to-speech, and wake-word-detection systems, such as:

Whisper 
Piper 
openWakeWord 

HassOS
Wyoming Protocol

Wyoming Protocol
是否要通过扩展: Piper配置Home Assistant以连接到Wyoming服务？

Wyoming Piper Addon

Docker Image

docker run -it -p 10200:10200 -v /path/to/local/data:/data rhasspy/wyoming-piper \
    --voice en_US-lessac-medium

https://github.com/rhasspy/wyoming-piper

--name matter-server

docker run -it --name piper  -p 10200:10200 -v /path/to/local/data:/data rhasspy/wyoming-piper \
    --voice en_US-lessac-medium