FHEM – Working with Objects

Now that we have installed the FHEM Server onto a Raspberry Pi, it is about time to do something with it. We have seen the result of the “naked install” at the end of the last post:

Picture 04 - FHEM Web PageThe web page itself is available but of course, nothing is configured into the system yet. Before we start working with FHEM, here is a brief discussion about how FHEM is organized.

It’s all about one file

In FHEM, it’s all about one file: fhem.cfg. This is the central configuration file which contains all – all! – definitions for the system. You can edit it directly or you can interact with the system using the Command Line and the configuration file will be updated in the background.

After the initial installation, the file contains some configuration already – how you edit it, is up to you. I have chosen to use WinSCP and got the file local onto my PC, then used Notepad++ to work with it.

Picture 03 - The Default Config FileEverything is an “Object”

In FHEM, everything you may want to work with, is considered “an Object”. It does not matter what exactly it is – a device, a log file, a room, a setting – everything is an object. With the default installation done, you can already see some objects in FHEM’s inventory – just select Everything on the left to see all objects that are currently defined – and their status.

Picture 04 - Default Object DefinitionsComparing the list of the fhem.cfg file, you can easily see that the definitions correspond. In other words: everything that is defined in the configuration file is accessible through the web interface and everything you configure through the web interface is saved to the configuration file. So how you work, is really entirely up to you – but editing the configuration file results in a “cleaner” an better structured file than having the system just dump the information there…

Our first “own” Object

To demonstrate how object definition works in FHEM, I have decided to use a very basic example that does not even need hardware. FHEM knows something called a “Dummy Object”. Perfect for getting to learn the system without having to waste money on actual hardware devices. Try this in the Command Line:

define mySwitch dummy

Through the Command Line, it results in an immediate creation of the Dummy Object – but only “in memory” – mind the “Save config” entry on top of the menu bar!

Picture 05 - A Dummy SwitchThis updated the config file – actually, the file received a couple more lines as it itself has been updated from an older version but the one line we are interested in is at the bottom:

Picture 06 - The updated Config FileYou can see that the object has a Name (“mySwitch”) and a Type (“dummy”). Those are the two parameters we have provided in the define command above. It also has a couple of other elements – sucha s a Number (or “Nr”) and a State (currently “???”) but these we did not define – they come from the system and are defaulting to initial values or are auto-calculated.

At the bottom, you can see that an Object has Attributes – in the screenshot above, one is defaulted (“room”) but others are possible. An Attribute is used to provide additional information to an Object. It typically describes the object in more detail.

Let’s take the Room attribute for the moment. Given that we may have a Home Automation System in mind that works in more than one room, it is logical to provide any device we might implement with a Room attribute to structure our devices better. Otherwise, we would have to remember which switch we have in which room…

Picture 07 - Configuration File UpdateThis time, I have chosen to update the fhem.cfg file directly and bring it back to my Raspberry Pi. Just copying the file back, however, does not help – FHEM does not “read” the updated configuration – you need to restart the FHEM Server.

shutdown restart

After the service has restarted, you can see the result of the configuration update right away:

Picture 08 - The Updated ConfigExamining the Dummy Switch more closely, you can now see that the Room attribute has been added – and that FHEM is using it in the menu to separate the devices by rooms.

Picture 09 - Device SpecificsNow, what do we do with that dummy switch? Usually, you would use a switch to… switch on a light? OK – but we need a “light” first:

Picture 10 - Lamp Device and Switch CommandAs you can see, the new dummy “myLight” is defined (Line 44) and placed in the Living Room (Line 45). At the same time, I have extended my existing switch “MySwitch” to include a command (can be “on” or “off”) to actually “switch” it (Line 41).

Picture 11 - The Switch is onIn the Web Interface, FHEM now allows me to switch “mySwitch” on or off (it is now “on”) and the system visualizes the state correctly. Last thing to do is to connect the switch to the light…

Another type of Object – an “Event”

Chaning the state of the switch is actually an “Event” – something that “happens”. And objects are defined in the already familiar way:

Picture 12 - The Notify EventThis is called a “Notify” Object because the eventy of the switch notifies the light to so something. Or in other words: if mySwitch enters the state of “on”, change the state of myLight to “on”. Needless to say that this would be a very unidirectional handling – there is currently no way to switch the light off again ūüėČ

Picture 13 - The Notify ObjectObviously, I could just repeat the step and create another Notify Object that is triggered when mySwitch if entering the state of “off” – but that is tedious. Instead, we are “tricking” our way:

Picture 14 - The Notify on all EventsInstead of “hardcoding” the values of the event triggered by mySwitch (“on”, “off”), we are reacting to simply every event – every time mySwitch changes state (and invokes the Notify Object) we are doing something to myLight.

The trick is using $EVENT (mind the uppercase style!) – that is a system variable that takes the value of the initiating object’s event (e.g. “on”) and by setting it on myLight, we can get around with one line of code. But that only works if the values are the same on both sides!

Not bad for a first attempt – we now can edit the configuration file, create devices, place them in rooms and have them interact based upon events. Next up is some cool stuff I always wanted to do… building my own weather station & indoor clima monitor.

Posted in Home Automation, Raspberry Pi | Leave a comment

FHEM – Updating the System

Like any other piece of software, FHEM needs updates. New devices may be supported or bugs may be fixed or new functionality may be added. So it is wise to keep the system up to date but at the same time, do so with care and the ability to roll back in case something goes wrong or requires more work to be done than you might have at the moment.

Rule #1: Always have a Backup

The first rule before any update is to have a backup of the system. For FHEM on a Raspberry, there are two ways of doing that: the probably most complete one is to backup the entire SD Card using a tool such as the USB Image Tool. That simply backs up the entire system and you can restore the environment as a whole should something go wrong.

Of course, if you have other environments or need to transfer the configuration between say a “Test System” and a “Live System”, you might want to choose a different approach.

FHEM has an integrated backup method. To use it, simply type the following in the FHEM Command line:

backup

Then hit Enter.

Picture 01 - Backup the FHEM EnvironmentTypically, the file resides on the Raspberry Pi’s File System in the location /opt/fhem/Backup.

Like the backup, the update is invoked via FHEM’s Command Line – guess what command?

update

The update loads a complete file list first so no worries if stuff starts to scoll through. Theoretically, the last message should inform you that the update has been performed, in my case, you really had to read the output as this was not the last line.

Picture 02 - Backup doneNow, it is time to restart FHEM – either by rebooting the Raspberry Pi or by issuing the following FHEM Command:

shutdown restart

And that should be it…

Posted in Allgemein | Leave a comment

Getting started with Home Automation

Home Automation is getting more and more attention – and devices become more and more available for prices that are affordable. So time for me to start looning into it. In combination with a Raspberry Pi, this is a neat way of adding some geek stuff to your home that looks impressive and might even help you save some money and/or increase your comfort level.

Using a Raspberry Pi for Home Automation

Considering “things to do” with a Raspberry Pi almost automatically sparks the idea of using it for some kind of home automation. It is small, it is cheap, it is extensible, it has a huge community and it can do the job.

I have an “old” Raspberry (a Raspberry PI Modell B Rev 2.0, 512MB model) sitting around that I can use as a test environment – that is trying stuff without messing up my actual “live” Home Automation Server. If things work, I can then transfer them to the live environment.

Now, before anything can be done, a clean install of the Raspberry Pi is required – and I have described this in a post way back when – so check it out here. At the time of writing this post, the current version is Raspbian Jessie with a Kernel Version 4.4, released on May 27, 2016. Of course, I am also applying the regular update and upgrade to be on the lastest versions of everything:

sudo apt-get update
sudo apt-get upgrade

Of course, there are a couple of other things you would want to consider for a “real” Home Automation Server but I am going to skip them for my test environment:

  • Provisioning of a fixed IP Address,
  • Changing the default user and password,
  • securing the Raspberry Pi against unauthorized access

And maybe some more that I forgot to mention… but since this is about playing with a Home Automation Server, I guess I just leave it at that and install one.

FHEM as Home Automation Server

Let get around this quickly: I have just chosen FHEM – there may be other solutions, there may be better solutions – this is the one I am going to play with.

like with any other solution, it needs to be installed first. But first, the Raspberry itself needs two more little bits sorted out.

First, we need to worry about the time and timezone. Let’s see where are are at with the Raspberry Pi – the command simply is

date

And the result is equally short:

Picture 01 - Date and Time on Raspberry PiTo reconfigure the time zone, you got multiple possibilities, one is the following:

sudo dpkg-reconfigure tzdata

You will receive a graphical interface to select the region first, then the city. Upon confirmation, you will see the updated configuration in the shell.

Picure 02 - Updated Timezone InformationFinally, we need some additional software packages on the Raspberry Pi – mainly Perl libraries:

sudo apt-get -f install && sudo apt-get install perl libdevice-serialport-perl libio-socket-ssl-perl libwww-perl libxml-simple-perl -y

With that done, we are ready to install FHEM – which does not come via the usual package manager but via a direct download from the fhem.de website:

sudo wget http://fhem.de/fhem-5.6.deb && sudo dpkg -i fhem-5.6.deb

This might take a moment – at least my Raspberry Pi is not the fastes anymore.

Picture 03 - FHEM installedThis is it – the software a such is now installed. A little bit on permissions needs to be adapted – change to the /opt director and do the following:

cd /opt
sudo chmod -R a+w fhem && sudo usermod -a -G tty pi && sudo usermod -a -G tty fhem

Now reboot the Raspberry Pi using the sudo reboot now command and then wait for it to come back. Then open any Internet Browser on your local machine and go to the Raspberry Pi’s FHEM Web Page:

Picture 04 - FHEM Web PageNote the port: it is 8083! If you see the above page, you should be all set to continue configuring the FHEM Server.

Posted in Home Automation, Raspberry Pi | Leave a comment

Using Voice Recognition Software for long Texts

Okay, this is going to be interesting. So far, I’ve been one of the many blog authors that used to type that texts. Now, with my right slightly unusable, to put it at best, I had to look for an alternative.

It is actually not the first time, that I have tried voice recognition software. In the past, however, to no satisfactory result. But plenty of time has passed since the last attempt. So yesterday I did a little bit of research on the Internet and as it seems, there is a sole provider for voice recognition software out there.

So went to their website and check if there would be a demo version. There was none. So I contacted one of their salespeople to see if there is any way for me to try out the software before buying it. Their response was that there would not be a demo version but they completely would refund the sales price if I would not be satisfied.

Needless to say that that was good enough for me. So I went ahead and purchased the software which is, for the record, Dragon Naturally Speaking 13 Premium.

The installation was seamless even though Windows 10 is not officially on the list of supported platforms. Also, I had no problem to use my existing microphones with the application. Playing around with it a little bit, I found out that there are certain limitations in the interaction with the operating system but as long as you are able to perform a few mouse clicks you should not have too much of an issue with that.

Most of my texts are either written in the web browser or in Microsoft Word. So I tried worked first. Dictating long texts ‚Äď in English or in German language ‚Äď proved not to be an issue at all. It in fact is naturally, much more so than typing. The initial voice-recognition is already quite good yet the correction feature which is also controlled by voice recognition is so powerful, that I barely find it necessary to type anything on the keyboard.

Sometimes the software does not pick up the proper word but I would be able to spell it and even train it on it. I admit that that sometimes is too tedious so I might simply type it in. Typically, it is then picking up the new word correctly. That works in Microsoft Word as well as my web browser, some applications, which are not supported natively, can be dealt with we had the built in dictation box from where the text can then be pasted into the target application.

So the next couple of days I will have enough opportunities to test the software a little bit more. But even with the limitations that I’m facing at least for me this is a big step forward.

And by the way: this whole article has been dictated, not a single word was typed.

Posted in Allgemein | Leave a comment

Using a Raspberry Pi as Torrent Seeder – Part I

OK, here is a “new” problem but I think it is one that I can overcome with my Raspberry Pi (still having the “old” version floating around, not the new model).

Here is what I am chewing on:

  • I got large amounts of data (legal data, that is!) – literally terabytes which¬†I want to share with friends but an FTP Server would block my upload capabilities almost immediately.
  • I don’t want the data to be “too public” meaning that I have no issues with people getting it from me but only “an interested community”, not “the whole world” might find it useful.
  • I want to make sure that I can provide my data without the need of a big computer being online constantly.

So I was thinking to myself: “What if I¬†distribute the data using a torrent approach?”

  • I would certainly need a private tracker (because I don’t want to to be dependent on a public one only) – but that is only Priority B (means: can come later).
  • I would need a¬†torrent client to seed my data continuously (at least for as long as not enough other seeders are established) and it needs to be able to support initial seeding (the the seeding that is done for an all new torrent where only one source is available).
  • I would need a¬†Raspberry Pi, connected to an external HDD as¬†an ideal solution – I have both floating around unused somewhere on my shelf.

Choosing the right BitTorrent Client

Option A: Deluge

If you start browsing the internet on the topic a little bit, you pretty quickly come across a BitTorrent client named Deluge Рits home page is here.

Deluge seems to be pretty cool and well¬†suited because of one feature that make a huge distinction from a “regular” BitTorrent client – quoting it’s own About page:

“It has been designed using the client server model with a daemon process that handles all the bittorrent activity. The Deluge daemon is able to run on headless machines with the user-interfaces being able to connect remotely from any platform.”

What does that mean? Well – nothing less than that your Raspberry Pi (without a monitor and a keyboard attached and only connected to a storage media) will be able to run the back-end Deluge daemon process and you would be able to connect to it (and get a user interface) from your regular PC (Windows, Mac, Linux…).

Option B: qBittorrent

Another choice of BitTorrent client is qBittorrent which you can find here. Unlike Deluge, it does not offer a Client –¬†Daemon architecture but it offers a Web User Interface which would also work.

Option C: any other BitTorrent Client and a Remote Desktop

Last but not least, you could install any other BitTorrent Client of your choice and use a Remote Desktop Connection into the Raspberry Pi to support your daily work with the Raspberry Pi.

Setting the Raspberry up (Basic Install)

But before you need to make that choice, you¬†first of all have to grab the latest version of the Raspberry’s operating system, Raspbian (which you can get here). This is also the opportunity to test the torrent idea: the download is available as direct download (ZIP file) and as torrent. I used the torrent – at a speed of finally 12 MB/s whereas the direct download “just” gave me 1 MB/s.

Next, the image must be transferred onto an SD Card – best to use the USB Image Tool which can be found here. After the operating system has been “installed” onto the SD Card, you can now put the card into the Raspberry’s SD Card Slot, connect a network cable and plug the device in.

Configuring the Raspberry Pi without Monitor and Keyboard

You now got two choices: you either have connected a monitor (or TV) and a keyboard to the Raspberry or you use a remote session to configure the box:

  • find the IP Address your Raspberry has been assigned (e.g. from your network router)
  • get yourself a tool like PuTTY (which you can download here)

Give it a minute or two to boo up – then connect – via SSH – to the Raspberry Pi. The default login user is “pi”, the default password is “raspberry”.

Image 01 - Commandline Prompt

Start the initial configuration of the Raspberry Pi using the following command:

sudo raspi-config

Image 02 - RaspiConfig

You might want to expand the file system to use the full SD Card and consider changing the password as well! Then click Finish (you can navigate with the Tab key and the Cursor keys) and reboot the device.

Making sure everything is up-to-date

Typically, as with any linux-based operating system, you want to make sure that your package information and your system is up-to-date:

sudo apt-get update
sudo apt-get upgrade

The two commands will first update your package library, then upgrade all installed software packages to the latest versions – which might take some time ūüôā

Setting up a Remote Desktop Server

The Raspberry Pi comes with a Desktop Environment installed and ready to use – your only issue: you need to be able to log in from a remote computer. Because we do not need anything “fancy”, the simple implementation of the RDP Protocol should be sufficient.

sudo apt-get install xrdp

This will install the RDP Protocol Layer – you should now be able to connect using a Remote Desktop Tool, e.g. from any Windows PC. It will usually take a few moments during which your Remote Screen will just look gray – give it some time, this is what you should see:

Image 14 - RDP Screen

A very important Choice

You now have a very important choice to make: the Raspberry Pi does not have any local storage media – which you will need to store your downloaded torrent data and to have the data available while seeding torrents. Which means: we need storage capacity – and probably not little!

You can either connect to any network storage location (e.g. a NAS you might have) which makes it easy to provide storage and to access the storage from other machines in your network – or – you can hook up an external HDD to the Raspberry, this separating the machine from your regular network as much as possible.

I have opted for a local HDD – primarily because my NAS is not supposed to hold redundant data just because my own library structure is different from my needs of a torrent box.

First of all, I need to verify that the Raspberry is ready to support NTFS-formatted harddisks Рthe following command will do that:

sudo apt-get install ntfs-3g

You might find that nothing is updated because the latest version of the support files are already installed – but better safe than sorry.

Now, plug in the external HDD (if not already done so) and examine what you got:

sudo fdisk -l

This should now – if everything has gone correctly – show you your internal storage (the SD Card) and the external one (the HDD).

Image 03 - External Drive

As you can see, I got roughly 1TB on my external disk available – and it is recognized as /dev/sda1.

To make the drives accessible easily, we need to mount them – for the Windows-guys: this is pretty much the same as assigning a drive letter in Windows. But sometimes, the Raspberry is faster – so let’s check if the device /dev/sda1 is already mounted:

cat /proc/mounts | grep /dev/sda1

My external drive is already auto-mounted – so there is nothing else for me to do.

Image 04 - Mount Information

One thing to make sure is that this is true even if you restart your Raspberry – so you could issue the

sudo reboot now

command and repeat the step above. If so, you are all fine.

Note: I am not going to install and/or setup a samba share to remotely access the drive – the data I have to share is by far to huge to transfer it via the Raspberry’s network interface. Instead, I will power down the box occasionally to get the data of by directly attaching it to my NAS. I know this is not ideal but my workaround to save copy time.

Option A: Setting up Deluge

If you have chosen to work with Deluge, setting up the software is the next thing to do. Or more correctly: the Daemon process running on the Raspberry Pi. To install deluged (the Daemon) and deluge-console (a think console interface) type:

sudo apt-get install deluged
sudo apt-get install deluge-console

Confirm the installations, then start the Daemon process.

deluged

Switch to the ~/.config/deluge directory and wait for the auth file to be created by the process Рthen stop the deluge daemon process:

cd ~/.config/deluge
ls (<- repeat until auth file is present!)
sudo pkill deluged

Edit the auth file, e.g. using nano – then add a line indicating the authorization required. Format is user:password:level.

nano ~/.config/deluge/auth

To keep things simple, we use the default user (make sure to provide the correct password if you changed it) and an access level of 10 (full administrative access to the Daemon process!)

Image 05 - Auth File Editing

Now, start the daemon process again, then start the console application:

deluged
deluge-console

If everything works fine, you will see the console application window displayed:

Image 06 - Deluge Console

If that does not work properly, type exit in the console and re-trace your steps! Otherwise, you can now configure deluge for remote access. Enter the following console commands:

config -s allow_remote True
config allow_remote
exit

This configures the deluge for remote access and closes the console application and verifies the value:

Image 07 - Remote Configuration

Now, you need to kill the daemon process and restart it again (there is no service restart command as much as I am aware for this one):

sudo pkill deluged
deluged

So much for that – now it is time to install the Deluge Client application (in my case the Windows version).

Installing the Deluge Client – Raspberry Pi

Although Deluge comes with a very nice Windows Desktop Client, you might want to install the Linux version of the Deluge Client on the Raspberry:

sudo apt-get install deluge

Why would you do this? Well Рfirst of all, now you can run (and handle) Deluge with only an RDP Connection available. Secondly, some things (like creating new Torrents) may seem to be easier directly from the Box rather than a remote machine. It does not cost you anything Рjust to be on the safe side.

Image 15 - Deluge Client and Console on the Raspberry Pi

The image above shows both, Deluge Client (on the left) and the Deluge Console (on the right) running on the Raspberry, seen through the RDP Connection.

Installing the Deluge Client – Windows

With the Raspberry now running Deluge (and deluge configured for remote connections), it is time to setup the front-end, the Deluge Client. That one, you can download here (pick the correct one for your OS). Then simply install it and run it.

The first think you need to do is to set the User Interface to non-Classic mode – so go to Preferences -> Interface and tick off the box (it is checked by default):

Image 08 - Deluge Settings

Note: Deluge is automatically recognizing the system language and adapts. Since I did not want to mess with my system, the screenshots are in German (but I will try and give you the English terms in addition)

Once you have changed the Interface Mode, you need to restart the Deluge Client. This time, you will be confronted with the Connection Manager where you will have to add a connection to your Raspberry Pi using the account you have specified when editing the auth file.

Image 09 - Add Host

Back to the Connection Manager screen, you should now see a green light next to the new host indicating that the Raspberry PI Daemon process was successfully contacted. You can now connect to the remote host.

Before we can start adding Torrents, there are some more configurations to make. But the good news is: with the Deluge Client connected, you do not have to do these through the console on the Raspberry, you can directly configure the Daemon through the Deluge Client.

Configuring the File Locations

One thing you certainly want to make sure is that Deluge is using the external HDD on the Raspberry to store the downloaded files. Go to Edit -> Preferences -> Downloads and check the current settings:

Image 10 - Download Locations

This is certainly not what you want – it would fill up the Raspberry’s SD Card in no time ūüėČ

Instead, I have prepared a few directories on the external HDD:

  • completed to host the completed downloads
  • torrent-backups for the backups of the torrent files
  • downloading for any active downloads
  • watch for the torrent files to automatically add

Which means that my directory configuration should look like this:

Image 11 - New Download Directories

IP Port Forwarding

Now, one last thing remains to be done – in the status bar, Deluge is currently reporting “No Incoming Connections!” which is caused by Deluge not being able to connect its required ports to the internet.

This is something you need to do on your network router which connects you to the Internet – it is usually called Port Forwarding.

In the Deluge Client, go to Edit -> Preferences -> Network. You will find the settings for Incoming Ports and Outgoing Ports – both currently set to “Use Random Ports”. But this only works if your network router¬†supports (and has enabled!) dynamic configuration via UPnP (which for example I have switched off!

Bottom line: if you need to manually forward the ports, tick off the “Use Random Ports” check boxes and specify the desired ports. Then forward them from your Internet Router to your Raspberry. Finally click the Test Active Port button to verify all is working as intended.

Testing the Installation

Before we look into some further configuration, we want a success story to tell, right? Torrents live from contribution Рthe more seeders there are to a torrent, the better the community is served. Therefore, I have decided to contribute to the seeding of the Raspbian OS Image I downloaded yesterday.

I simply had to add the torrent information file I got off their website the day before today, the download started immediately and was visible in the Deluge Client.

Image 12 - Torrent Download

Notably, my download speed is severely below a comparison download using ¬ĶTorrent from my regular PC – the Raspberry does not exceed about 1 MB/s, the PC soars up to 12 MB/s.

But the nice thing is: I can now close my Deluge Client on the PC Рthe Raspberry is still doing the download in the background via the Deluge Daemon process. And that one will also provide you with events and status updates as needed, e.g. for the end of my Raspbian Jessie download.

Image 13 - Daemon Process Status

So much for this post – what have we done so far?

  • We have installed the Raspberry Pi’s basic operating system, Raspbian.
  • We have added the ability to connect to the Raspberry’s Graphical Desktop via RDP.
  • We have installed¬†the Deluge Daemon Process and configured it for remote access.
  • We have installed the Deluge Console application.
  • We have installed the Deluge Client, both on the Raspberry and a Windows PC, and
  • we have started our first (public) Torrent, downloading the Raspbian image and then seeding it to the community.
Posted in Raspberry Pi | Tagged , | Leave a comment