Passive BLE Monitor integration(Add Xiami motion sensor 2 (RTCGQ02LM) to home assistant) bindkey

Passive BLE Monitor integration

(局域网接入方式)  -  BLE events without Internet

https://github.com/custom-components/ble_monitor

Requirements

You are running Home Assistant version 2022.8.0 or newer.
You have the Built-in  Bluetooth  Adapter is enabled and can be accessed.
You have Access to the device in Yeelight app.
You know how to Grant permissions for Python to have root less access to the HCI interface in pve”.
You know how to Add device in Yeelight app.
You have some Supported Devices by Passive BLE Monitor integration

Please note that at least Built-in  Bluetooth  Adapter is required
Built-in  Bluetooth  Adapter is enabled and can be accessed, all available adapters are automatically discovered.

Note:

This custom component is an alternative for the standard build in mitemp_bt 483 integration and the Bluetooth LE Tracker 943 integration that are available in Home Assistant.
https://community.home-assistant.io/t/passive-ble-monitor-integration/303583




Add Xiami motion sensor 2 (RTCGQ02LM) to home assistant by BLE Monitor integration

https://github.com/custom-components/ble_monitor/pull/287

https://community.home-assistant.io/t/passive-ble-monitor-integration/303583

https://github.com/custom-components/ble_monitor/issues/253

Manual installation

$ ll /home/homeassistant/.homeassistant/custom_components
total 16
drwxr-xr-x  4 homeassistant homeassistant 4096 mai    7 21:40 ./
drwxrwxr-x 10 homeassistant homeassistant 4096 mai    8 08:30 ../
drwxrwxr-x  4 homeassistant homeassistant 4096 mai    7 21:40 ble_monitor/
drwxr-xr-x 12 homeassistant homeassistant 4096 mai    7 21:15 hacs/

首先要保证rpi bluetooth运行正常

米家BLE广播协议 - Mi Beacon
https://www.bookstack.cn/read/miio_open/ble-Mi-Beacon.md

将来可以替代passive ble monitor等第三方，就是原声支持ble了。
passive ble monitor插件作者也表示等到ha全部接盘后，他的插件就退市了。
但mesh现在不行，将来原生应该还是不行。

Home Assistant 2022.8 has (improved) support for passive BLE devices directly in Home Assistant. For each brand, a core BLE integration will be developed, such that maintanance can be divided over more people, using the latest Bluetooth packages (bleak). I'm working together with the Home Assistant devs to move sensors from BLE Monitor to Home Assistant core integrations. During the transition, BLE monitor will still be available, but it is possible that the core HA Bluetooth integrations will not work niceley parallel to BLE monitor. If it is not working together, try to enable active scan in the BLE Monitor. My advise, when all your sensors are available in Home Assistant, make the move. The aim is to have all sensors moved into Home Assistant as core integration. After the move, BLE monitor will probably be deprecated. If you want to help moving sensors from BLE monitor, feel free to help. Check out the links below.

Passive BLE Monitor要迁移进去了

useful links

使用Passive BLE Monitor集成代替蓝牙网关直接接入HA-
目的: 用于跳过所谓的蓝牙网关直接将蓝牙传感器设备接入HA, 注意,目前只支持获取蓝牙传感器数据,不支持发送控制指令(比如:开关)!
https://bbs.hassbian.com/thread-16148-1-1.html

1

Xiaomi Mijia Window Door Sensor 2


Hi I'm trying to integrate a MCCGQ02HL in

HA via ESP32

, I have already successfully integrated the LYWSD03MMC temperature sensors on the same ESP32. HA seem not to be receiving data from the MCCGQ02HL sensor.


Supported Devices - Passive BLE Monitor integration

Xiaomi (MiBeacon format)

MCCGQ02HL,  - Door/Window Sensor 2F70 (MCCGQ02HL)

RTCGQ02LM,   - Motion Sensor  (RTCGQ02LM)

绝大部分的米家蓝牙传感器都是加密的，所以我们首先就要获取蓝牙设备的密钥key

use Xiaomi Gateway 3 to obtain the device-mac-address and device-bin-key

I just followed the guide here. it's made very well, congratulations to the author. Copy and paste the "Sample configuration" code and modify it as you wish. I have removed "Battery Voltage" and renamed it in my language (Italian) and removing "$ name". Don't forget "external_components:" line otherwise it will report an error. Upload it via OTA and go. Simple and functional.
I used Xiaomi Gateway 3 to obtain the device-mac-address and device-bin-key,I think it can also be obtained in another way but I don't know how to help you in this. If you need more details just ask.

https://github.com/esphome/feature-requests/issues/1034

https://github.com/dentra/esphome-components/tree/master/components/miot_mccgq02hl

Step 0. Grant permissions for Python to have rootless access to the HCI interface

Python needs [b]root access to access the HCI interface[/b]. If Python doesn't have root access, you will get an error message in Home Assistant which ends with:

PermissionError: [Errno 1] Operation not permitted

非 py虚拟环境,保证 正常运行

First, try to set root access with

  sudo setcap 'cap_net_raw,cap_net_admin+eip' `readlink -f \`which python3\``

(homeassistant20230103) homeassistant20230103@raspberrypi:/srv/homeassistant20230103 $ sudo setcap 'cap_net_raw,cap_net_admin+eip' `which python3`

我们信任您已经从系统管理员那里了解了日常注意事项。
总结起来无外乎这三点：

    #1) 尊重别人的隐私。
    #2) 输入前要先考虑(后果和风险)。
    #3) 权力越大，责任越大。

[sudo] homeassistant20230103 的密码：

If you have multiple python versions, make sure it refers to the same version which is used by Home Assistant. If Home Assistant is using a different version, e.g. python3.6, run the following command to set the correct version (adjust it to your own version if needed).

sudo setcap 'cap_net_raw,cap_net_admin+eip' /usr/bin/python3.6

在开始使用 hcitool 工具之前，需要先进行一些配置。默认情况下， hcitool 工具需要有 root 权限，但是我们在使用中或者python程序中调用，都希望不需要使用 root 权限或者使用 sudo 来提权，我们可以通过以下配置来实现不需要 root 权限而对工具进行调用：

sudo setcap 'cap_net_raw,cap_net_admin+eip' `which hcitool`

Next, check that it is set correctly with the following command

sudo getcap `readlink -f \`which python3\``

When Home Assistant is installed in pyenv, the output of the command which python3 looks like /home/homeassistant/.pyenv/johndoe/python3 but this is not the real link to actual python3 interpreter. The actual binary is placed in a folder like /home/homeassistant/.pyenv/versions/3.9.9/bin/python3.9 (it might slightly differ depending on the installed python version). So to grant root access to python, it is required to run the setcap command with the correct path, i.e.

setcap 'cap_net_raw,cap_net_admin+eip' /home/homeassistant/.pyenv/versions/3.9.9/bin/python3.9


sudo ghttps://custom-components.github.io/ble_monitor/Installation#step-0-grant-permissions-for-python-to-have-rootless-access-to-the-hci-interfaceetcap `readlink -f \`which python3\``

https://custom-components.github.io/ble_monitor/faq#i-get-a-permissionerror-in-home-assistant-after-the-installation-or-python-upgrade


There are two ways to configure the integration and your devices (sensors),
in the User Interface (UI) or
in your YAML configuration file.

you must Choose one method of them, you can’t use both ways at the same time. 

2023-01-27 04:47:10.452 ERROR (Thread-2) [homeassistant] Error doing job: Fatal error: protocol.data_received() call failed.
Traceback (most recent call last):
  File "/usr/lib/python3.9/asyncio/selector_events.py", line 870, in _read_ready__data_received
    self._protocol.data_received(data)
  File "/srv/homeassistant20230103/lib/python3.9/site-packages/aioblescan/aioblescan.py", line 1855, in data_received
    sel

useful links

https://github.com/custom-components/ble_monitor/blob/f5977a9e04f430ca2afe18ea4e927e0884982ee2/faq.md#why-is-this-component-called-passive-and-what-does-it-mean

https://github.com/raspberrypi/firmware/issues/1150#issuecomment-602020073

https://github.com/custom-components/ble_monitor/issues/31

https://github.com/custom-components/ble_monitor/issues/827#issuecomment-1373437784

ha core 2023.5.4

step 1.

which python3

(homeassistant20230504) homeassistant20230504@raspberrypi:/srv/homeassistant20230504 $ which python3
/srv/homeassistant20230504/bin/python3

step 2.

root@raspberrypi:~# cd /srv/homeassistant20230504/bin
root@raspberrypi:/srv/homeassistant20230504/bin# ls -l python*
lrwxrwxrwx 1 homeassistant20230504 homeassistant20230504 10  6月  4 03:37 python -> python3.11
lrwxrwxrwx 1 homeassistant20230504 homeassistant20230504 10  6月  4 03:37 python3 -> python3.11
lrwxrwxrwx 1 homeassistant20230504 homeassistant20230504 25  6月  4 03:37 python3.11 -> /usr/local/bin/python3.11

step 3.

root@raspberrypi:/srv/homeassistant20230504/bin# sudo setcap 'cap_net_raw,cap_net_admin+eip' /usr/local/bin/python3.11
root@raspberrypi:/srv/homeassistant20230504/bin#




sudo setcap 'cap_net_raw,cap_net_admin+eip' /usr/local/bin/python3.11



I noticed that running which python3 inside the homeassistant venv points to /srv/homeassistant/bin/python3
https://github.com/custom-components/ble_monitor/issues/827#issuecomment-1373437784

ubuntu@rpi:/srv/homeassistant/bin$ ls -l python*
lrwxrwxrwx 1 homeassistant homeassistant  9 Dec 29  2021 python -> python3.9
lrwxrwxrwx 1 homeassistant homeassistant  9 Dec 29  2021 python3 -> python3.9
lrwxrwxrwx 1 homeassistant homeassistant  7 Jan  5 15:15 python3.10 -> python3
lrwxrwxrwx 1 homeassistant homeassistant 18 Dec 29  2021 python3.9 -> /usr/bin/python3.9

sucess for  xiaomi mijia  notion sensor 2

Success

.storage/core.config_entries

         "report_unknown": "Off",
          "devices": [
            {
              "mac": "54:EF:",
              "encryption_key": "4ad82cfd96b9869502037dce5d",
              "restore_state": "default",
              "use_median": "default",
              "consider_home": 180,
              "report_unknown": true,
              "decimals": "default",
              "reset_timer": 35,
              "track_device": true,
              "tracker_scan_interval": 20,
              "temperature_unit": "°C",
              "delete device": false,
              "uuid": "",
              "name": "54EF44C48283"
            }
          ],

sucess for  xiaomi mijia  notion sensor 2

Success

.storage/core.config_entries

  {
        "entry_id": "fc552cc3319f84a280c91fa071e4c802",
        "version": 1,
        "domain": "xiaomi_ble",
        "title": "Motion Sensor BF13 (RTCGQ02LM)",
        "data": {
          "bindkey": "ee93b8a000d5ce967305ef37a9c0ff6b"
        },
        "options": {},
        "pref_disable_new_entities": false,
        "pref_disable_polling": false,
        "source": "bluetooth",
        "unique_id": "54:EF:44:C7:BF:13",
        "disabled_by": null
      },
      {
        "entry_id": "28795f0744be1b5447331725056ef9ea",
        "version": 1,
        "domain": "xiaomi_ble",
        "title": "Door/Window Sensor 2F70 (MCCGQ02HL)",
        "data": {
          "bindkey": "82e7ca27d314087aac3d6c81a258e428"
        },
        "options": {},
        "pref_disable_new_entities": false,
        "pref_disable_polling": false,
        "source": "bluetooth",
        "unique_id": "E4:AA:EC:62:2F:70",
        "disabled_by": null

.storage/core.device_registry

[code]

  {
        "area_id": null,
        "config_entries": [
          "fc552cc3319f84a280c91fa071e4c802"
        ],
        "configuration_url": null,
        "connections": [],
        "disabled_by": null,
        "entry_type": null,
        "hw_version": null,
        "id": "7ddfe9e7c4602c86d15a3fca0bf83ba3",
        "identifiers": [
          [
            "bluetooth",
            "54:EF:44:C7:BF:13"
          ]
        ],
        "manufacturer": "Xiaomi",
        "model": "RTCGQ02LM",
        "name_by_user": null,
        "name": "Motion Sensor BF13",
        "sw_version": null,
        "via_device_id": null
      },
      {
        "area_id": null,
        "config_entries": [
          "ad7ed619f1bb339c165b406df931e204"
        ],
        "configuration_url": null,
        "connections": [],
        "disabled_by": null,
        "entry_type": null,
        "hw_version": null,
        "id": "341f3ccd075030f2b628aebb46b16d49",
        "identifiers": [
          [
            "ble_monitor",
            "511D96BD-3DE2-4C1C-B31B-0EA82716EFAE"
          ]
        ],
        "manufacturer": "Apple",
        "model": "iBeacon",
        "name_by_user": null,
        "name": "511D96BD3DE24C1CB31B0EA82716EFAE",
        "sw_version": "iBeacon",
        "via_device_id": null
      },
      {
        "area_id": null,
        "config_entries": [
          "ad7ed619f1bb339c165b406df931e204"
        ],
        "configuration_url": null,
        "connections": [],
        "disabled_by": null,
        "entry_type": null,
        "hw_version": null,
        "id": "6ee60c70328dadea0a17c04a00b0ed68",
        "identifiers": [
          [
            "ble_monitor",
            "54:EF:"
          ]
        ],
        "manufacturer": "Xiaomi",
        "model": "RTCGQ02LM",
        "name_by_user": null,
        "name": "54EF44C48283",
        "sw_version": "Xiaomi (MiBeacon V5 encrypted)",
        "via_device_id": null
      },
      {
        "area_id": null,
        "config_entries": [
          "ad7ed619f1bb339c165b406df931e204"
        ],
        "configuration_url": null,
        "connections": [],
        "disabled_by": null,
        "entry_type": null,
        "hw_version": null,
        "id": "847396db0fdd2aadfa7125975fbe71dc",
        "identifiers": [
          [
            "ble_monitor",
            "E4:AA:EC:62:2F:70"
          ]
        ],
        "manufacturer": "Xiaomi",
        "model": "MCCGQ02HL",
        "name_by_user": null,
        "name": "E4AAEC622F70",
        "sw_version": "Xiaomi (MiBeacon V5 encrypted)",
        "via_device_id": null
      },
      {
        "area_id": null,
        "config_entries": [
          "ad7ed619f1bb339c165b406df931e204"
        ],
        "configuration_url": null,
        "connections": [],
        "disabled_by": null,
        "entry_type": null,
        "hw_version": null,
        "id": "fd47e4ba76b8eca6705356265d548f49",
        "identifiers": [
          [
            "ble_monitor",
            "54:EF:44:C7:BF:13"
          ]
        ],
        "manufacturer": "Xiaomi",
        "model": "RTCGQ02LM",
        "name_by_user": null,
        "name": "54EF44C7BF13",
        "sw_version": "Xiaomi (MiBeacon V5 encrypted)",
        "via_device_id": null
      },
      {
        "area_id": null,
        "config_entries": [
          "28795f0744be1b5447331725056ef9ea"
        ],
        "configuration_url": null,
        "connections": [],
        "disabled_by": null,
        "entry_type": null,
        "hw_version": null,
        "id": "7fe546d37a11e29fb77dfd3f33ad8f3d",
        "identifiers": [
          [
            "bluetooth",
            "E4:AA:EC:62:2F:70"
          ]
        ],
        "manufacturer": "Xiaomi",
        "model": "MCCGQ02HL",
        "name_by_user": null,
        "name": "Door/Window Sensor 2

Encryption  -  Mijia BLE MiBeacon protocol and other Xiaomi BLE devices

Some devices use AES encryption to protect the sensor values they are broadcasting.

MiBeacon v2/v3 use unauthenticated AES with a 24 character hexadecimal (12 byte) key
MiBeacon v4/v5 use authenticated AES with a 32 character hexadecimal (16 byte) key
This keyhttps://www.home-assistant.io/integrations/xiaomi_ble is called the bindkey or beaconkey.


Xiaomi motion sensor 2 (RTCGQ02LM) is based on QN9020 MCU that makes use of GATT over Bluetooth Low Energy to communicate with mobile app.

The protocol allows to control some settings and get status updates. It's closed so I used BLE Monitor integration to analyze the Mi Home app  that handles encryption.

At first you  need to listen to your Xiami motion sensor 2  via BLE.
second you need to  encryption packets with bindkey

How  to use

way 1 :generate new bindkeys for devices

https://atc1441.github.io/TelinkFlasher.html

way 2; get  bindkeys   from Xiaomi Cloud

xiaomi gateway3 integration


useful links
Getting the bind key for the Xiaomi Mijia LYWSD03MMC sensor via TelinkFlasher
https://github.com/mKeRix/room-assistant/issues/277