Installing the conbee zigbee usb dongle on home assistant core

Z-wave and zigbee are two competing low-power wireless standards, so far I have used z-wave through a zwave hat on my raspberry pi.

Since zigbee seems to be gaining some traction I decided to also add a zigbee network. To do so, I bought the conbee usb dongle.

the integration with home assistant allows for some choices: ZHA, deconz or zigbee2mqtt. In my case, for no good reason I went with deconz first but could not get all sensors visible (my smart wall plug could be turned on and off but no power readings). Installing the ZHA integration then proved to be easier, no additonal daemon needed and all sensors were visible.

Probably though zigbee2mqtt is the way to go. I’ll try it if ZHA gives me issues.

Installation (for deconz)

/

As usual, on home assistant os all is done out of the box, for us home assistant core users we need to install the zigbee middleware (the gateway) ourselves. The required software is called ‘deconz’, no standard package is in the repositories for raspbian so we need to add the phoscon repository to apt. (alternatively a docker image is also available).

0. plug in the device. (it is recommended to use an usb extension cable to avoid interference, I did not yet do this).

  1. follow the instruction at phoscon for installing the driver.

(unfortunately all the gui tools are also installed you can expect a lot of dependencies)
start up the daemon:

after installation:
root@raspberrypi:/etc/mosquitto# systemctl list-unit-files | grep conz
deconz-gui.service                     disabled
deconz-homebridge-install.service      disabled
deconz-homebridge.service              disabled
deconz-init.service                    disabled
deconz-update.service                  enabled
deconz-wifi.service                    disabled
deconz.service                         disabled

enable deconz and start it:
systemctl enable deconz
systemctl start deconz

new status:
root@raspberrypi:/etc/mosquitto# systemctl list-unit-files | grep conz
deconz-gui.service                     disabled
deconz-homebridge-install.service      disabled
deconz-homebridge.service              disabled
deconz-init.service                    disabled
deconz-update.service                  enabled
deconz-wifi.service                    enabled
deconz.service                         enabled

check the log for correct startup. the default port is 80.

You can now connect your browser to http://<ip_of_your_pi>:80 and you will get the web interface.

2. through the web gui, you can configure the device:

  • you will be prompted for a password. set up the initial password.
  • if necessary, update firmware. Go to ‘menu’ then ‘gateway’.
  • in menu/gateway, click on ‘advanced’.

enable pairing mode by clicking on ‘authenticate app’.

3. link the zigbee gateway to your home assistant core: add the dconz integration.
(more info: https://www.home-assistant.io/integrations/deconz/)

  • go to devices and configure dconz
  • enter your ip, it can be 127.0.0.1 if running on the same system.
  • complete the pairing process.

installing an mqtt broker for home assistant core

What is MQTT and why do you want it ?

MQTT is an OASIS standard messaging protocol for the Internet of Things (IoT). It is designed as an extremely lightweight publish/subscribe messaging transport that is ideal for connecting remote devices with a small code footprint and minimal network bandwidth.

From https://mqtt.org:

In other words, it is a messaging protocol which is designed to be efficient for frequent message sending between different devices. A device can send messages to an mqtt topic, and your home assistant server could retrieve these messages from that topic.
Like the Installing zwave-js on home assistant core article, this component is actually installed out of the box on home assistant but not in the installation of the core version.

I keep the below info as a reference for myself, the document will be improved as I make changes.

Installation and configuration:

Install the package using apt. Like eg the zwave server, this will run as a separate daemon on your home assistant appliance (next to home assistant itself):

mosquitto/oldstable,now 1.5.7-1+deb10u1 armhf [installed]

go into /etc/mosquitto/ and add a custom configuration file to conf.d:

Set up a configuration:

# global
allow_anonymous false
password_file  /etc/mosquitto/conf.d/mosquitto_users

# default listener
port 1883

# ssl listenener
listener 8883
cafile /etc/mosquitto/certs/wf_ca.crt
certfile /etc/mosquitto/certs/mqtt.crt
keyfile /etc/mosquitto/certs/mqtt.key
#tls_version tlsv1.2

Here, I disable anonymous connections to the broker, configure a location for a user/password file and set up an ssl and a plaintext listener.

Configure a user:

mosquitto_passwd -c mosquitto_users mqttuser

(the -c is used to create the file so should be used only once. through -b you could add the password immediately on the cli).

Create some certificates for the ssl listener, you need:

  1. the certificate of your ca. this should be a private, internal ca.
  2. the certificate to be used for your mqtt broker
  3. the private key to be used for your mqtt broker.

place them into the certs folder:

ls -l certs/
total 16
-rw-r--r-- 1 root root  130 Nov 16  2019 README
-rw-r--r-- 1 root root 1859 Feb 10 15:01 mqtt.crt
-rw-r--r-- 1 root root 3244 Feb 10 15:01 mqtt.key
-rw-r--r-- 1 root root 1815 Feb 10 15:17 wf_ca.crt

You can now start the daemon.

Configuration in home assistant:

Unfortunately, not all settings can be configured through the gui (yet and at the time of writing). Notably configuring the ssl part. In your configuration.yml put:

mqtt:
  certificate: "/etc/mosquitto/certs/wf_ca.crt"
  broker: "192.168.1.16"
  port: 8883
  username: mqttuser
  password: "thisismypassword"

Now go to settings/devices and verify the mqtt integration is there.

If you click ‘configure’ on the mqtt broker you can test it:

Installing zwave-js on home assistant core

Multiple possibilities exist, we will install the barebones zwave-js server project (https://www.npmjs.com/package/@zwave-js/server). zwave-js server is a node project implementing the zwavejs library and which is ready to use. it requires a certain version of the library to function, so always install the package and it will install the correct dependencies.

There is no need to use the cli commands after installation, just provide a config file and start up the server. Then, you can use the migration wizard in HA to move the legacy zwave to the new zwave-js version OR of course set up a new installation.

For the installation we will use a version of node and npm installed as the homeassistant user through NVM (so not from packages). the installation was tested on a raspberry pi 3 which was migrated from legacy zwave to zwave-js.

below is meant for home assistant core (the ‘advanced use case’ describe on the zwave-js documentation at the home assistant site).

In short, it means:

  • install the zwave-js server and start the daemon
  • enable the zwave-js home assistant integration
  • migrate your zwave devices to zwave-js
  • correct any missing devices.

steps

prereqs:

make sure you have these installed: git-cli, curl.

Of course, make a backup of the database and config folder.

then, as user homeassistant, do:

  • install nvm (node version manager)
curl -O https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh
. install.sh
  • restart your shell to have the nvm environment variables added to your path.
  • install node.js and npm (node package manager):
nvm install node npm

You are no ready to install node package zwave-js/server.

installing :

npm i @zwave-js/server

For already initialized zwave sticks: create a config file with at least your existing security key in it, if you have one:

module.exports = {
  securityKeys: {
      S0_Legacy: Buffer.from("4xxxxxxxxxxxxxxxxyyyyyyyyyyyyyyD", "hex")
  }
};

Start up the binary (provide the correct serial device for your z-wave stick, in the below example ttyAMA0) and check the logs for errors.

Set up a systemd unit file if needed for auto-startup:

[Unit]
Description=ZWaveJS-server
After=network.service

[Service]
Type=simple
Environment=PATH=/home/homeassistant/.nvm/versions/node/v17.2.0/bin:/usr/local/pyenv/shims:/usr/local/pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
User=homeassistant
Group=zwavejs
WorkingDirectory=/home/zwavejs/zwave-js-server
ExecStart=/home/zwavejs/zwave-js-server/node_modules/.bin/zwave-server /dev/ttyAMA0 --config /home/zwavejs/zwavejs-config.js
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

You can now enable and start the zwavejs-server service and continue with https://www.home-assistant.io/integrations/zwave_js/#advanced-installation-instructions

Upgrading

Occasionally an upgrade may be needed, home assistant will show you during startup and you will see in the dashboard that the zwave-js integration is incompatible with the installed zwave-js server version. You should upgrade. Switch to the user running zwave-js, then:

root@raspberrypi:/home/zwavejs/zwave-js-server# npm outdated

Package                            Current    Wanted   Latest  Location                                       Depended by
@types/node                       16.11.10  16.11.26  17.0.21  node_modules/@types/node                       zwave-js-server
@types/ws                            8.2.0     8.5.1    8.5.1  node_modules/@types/ws                         zwave-js-server
@types/yargs                        17.0.7    17.0.8   17.0.8  node_modules/@types/yargs                      zwave-js-server
@typescript-eslint/eslint-plugin    4.33.0    4.33.0   5.12.1  node_modules/@typescript-eslint/eslint-plugin  zwave-js-server
@typescript-eslint/parser           4.33.0    4.33.0   5.12.1  node_modules/@typescript-eslint/parser         zwave-js-server
@zwave-js/server                    1.13.0    1.15.0   1.15.0  node_modules/@zwave-js/server                  zwave-js-server
eslint                              7.32.0    7.32.0   8.10.0  node_modules/eslint                            zwave-js-server
eslint-config-prettier               8.3.0     8.4.0    8.4.0  node_modules/eslint-config-prettier            zwave-js-server
husky                                4.3.8     4.3.8    7.0.4  node_modules/husky                             zwave-js-server
lint-staged                         12.1.2    12.3.4   12.3.4  node_modules/lint-staged                       zwave-js-server
prettier                             2.5.0     2.5.1    2.5.1  node_modules/prettier                          zwave-js-server
ts-node                             10.4.0    10.5.0   10.5.0  node_modules/ts-node                           zwave-js-server
typescript                           4.5.2     4.5.5    4.5.5  node_modules/typescript                        zwave-js-server
ws                                   8.3.0     8.5.0    8.5.0  node_modules/ws                                zwave-js-server
npm notice
npm notice New minor version of npm available! 8.2.0 -> 8.5.2
npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.5.2
npm notice Run npm install -g [email protected] to update!
npm notice

root@raspberrypi:/home/zwavejs/zwave-js-server# npm update @zwave-js/server

added 10 packages, removed 1 package, changed 37 packages, and audited 399 packages in 2m

83 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

Notes & caveats

  • copy your zwave key ! (it will also be shown through the migration wizard)
  • in my case, all my devices were migrated but not all entities. I just recreated the missing entities later through the GUI. (mostly this meant recreating some graphs or other widgets).
  • if you keep a backup of your homeassistant config folder/database, you can always revert.

Links

zwave-js server npm implementationhttps://www.npmjs.com/package/@zwave-js/server
zwave-js library documentationhttps://github.com/zwave-js
nvm documentationhttps://github.com/nvm-sh/nvm
installing node tools through nvmhttps://heynode.com/tutorial/install-nodejs-locally-nvm/
home assistant zwave-js docshttps://www.home-assistant.io/integrations/zwave_js/#advanced-installation-instructions

moving ip addresses to another device without downtime: scenarios

For the impatient: arpsend !

We all know the problem: you replace a firewall with another model, correctly configure everything and lose connectivity.

After a few hours, things automagically start working again. Continue reading “moving ip addresses to another device without downtime: scenarios”

Ubuntu Linux under vmware: network interfaces

I’m quite busy with VMware these days. I actually believe VMware’s dogma “the virtual machine is more robust and qualitatively better than the physical machine” is correct.

However, after installing two VM’s running Ubuntu Linux to do some failover testing, I noticed something odd about the numbering of the network interfaces. Good old eth0 was missing. Instead, there was an eth1 present…

Now I had this issue in the past, but always worked around it by just using eth1 instead of eth0. But maybe it would be better to find out about this and fix it “the right way”.

The problem is caused by the copying of the virtual machines. I have a VM template which I use as a base image. I then copy and customise this image when I need to implement new VM’s. The process goes like this: Continue reading “Ubuntu Linux under vmware: network interfaces”

Cover art resizing

One of the more heated debates involving audio tags is: do you organize your files using one directory for every album and put the cover art into a single file into that album (‘folder.jpg’) or do you embed the cover art into the tags of each and every file? Myself, I embed it into all files.

  • the cover art is always with the file, no matter where I copy it to
  • I do not have to put every album into one folder
  • what about single files? You have to embed the art then since creating a folder for one single file seems…strange.

Continue reading “Cover art resizing”

Digital audio players: iTunes

ITunes

Note: this article was originally posted on April 11th, 2007. Some of the mentioned extensions are no longer compatible with later iTunes versions.

Why?

ITunes is Apple’s audio player with versions for both Mac and PC. You can say a lot of things about it:

  • It always seems to ‘know better’
  • It is limited in supported playback formats
  • It feels a bit slow

Yet, : Continue reading “Digital audio players: iTunes”

MP3 compression settings

A lot of misconceptions exist about audio compression. Bear in mind that the whole idea behind it is to store digital audio but not have it take up too much space on your hard drive. Uncompressed audio is in wav format. You can compress it in 2 ways:

  • Lossless compression: no information is lost, compression is about 50%.
  • Lossy compression: the codec used tries to determine parts of the file you won’t be able to hear anyway and throws them away. The rest is compressed. Compression is determined by either the bitrate you want to give or the quality you want to give. Compression depends upon the bitrate used but can typically be around 80% while staying transparant.

So, no codec (mp3, ogg, aac, …) can ever be said to provide better or worse sound quality. What can be said is that different codecs need a different bitrate (and so, file size) to reach the same quality.

We’ll be compressing lossy. mp3 is an old codec and in theory newer ones such as aac should need a lower bitrate to get the same quality. However…When using the lame mp3 encoder you have an encoder which has excellent tuning. The advantage of using mp3 is that it’s universally supported on all devices. (indeed, most people think about mp3’s when talking about digital audio.) Also note that a song encoded with the same bitrate but by different encoders may have a very diffent quality for different encodings. When encoding to mp3 and if quality is the goal (instead of speed) always use lame! So, MP3 is what we’ll be using here… Continue reading “MP3 compression settings”

Tagging audio files

There are a few ways I can think of for an audio player to show you all relevant information (artist, title, release year, …) about an audio file.

  • derive it on-the-fly from the filename.
    • disadvantages:
      • only a limited amount of fields can be entered before file names become too long
      • changing a field would require you to rename the file, which will make programs no longer find the file.
      • you need to take care when naming your files, changing your filename format afterwards may be difficult.
    • advantages:
      • ummmm…not sure…
  • derive it from metadata added to the file.
    • disadvantages:
      • a lot of fields are standardized, but some are not. (album artist anyone?)
      • file becomes (only slightly) bigger
    • advantages:
      • very flexible: you can add fields whenever you want

Continue reading “Tagging audio files”