From the life of a screen worker

Much to compensate

Anyone who knows me knows I talk a lot in while I sleep. Most of the time, I speak some terms related to computer science, sometimes whole sentences. Sometimes they actually make sense. For my interest and for my own amusement, I have recorded myself for months. It was very easy since apps like Sleep Talk Recorder offer this feature: The app is simply opened and waiting for noise. Once something is said and so the volume in the room is too high, the recording begins.

At some point I came up with the idea of providing the spoken stuff with subtitles and to publish them as a funny video. The only problem: creating such a video is very complex. With exporting the data from the iPhone, setting the texts and also the understanding of the spoken, I have to spend four to eight hours in a single video. That was a long time ok for me, because I simply worked on this video on long voyages as a passenger. Then, however, the charm caught me, to build a clean and smart solution that fits in my "smart apartment". Actually, I was annoyed anyway that I had to think on starting the app every night before I went to bed.

RaspberryPi as sleeping laboratory

Nein, das geht nicht. Haha. Thomas Kekeisen im Schlaf vor sich her geredet ( Video)
The fact that I use a Raspberry Pi with USB microphone was quickly defined, but not what software components I want to use. Actually, I wanted to use Ruby or Python, but then, after some tests with classic bashscript and "old" programs like ffmpeg, sox and ImageMagick I decided to use those linux tools. Sound processing and editing is really easy - if you know what you are doing.

Since I really wanted to automate the entire "video generation process" from the recorded language to the finished video, my program had of course also to offer a web interface for cutting the audio files. Finally I decided to use node.js as server, Jade as template engine and Bootstrap and wavesurfer.js for the frontend. At this point also a thanks to Michael Malura, he supported me with a lot of questions related to node.js.

The frontend of PiSleepTalk

"Well, that escalated quickly"

I worked hard on this idea from december 2015 to roughly april 2016. I learned a lot of the new components I decided to use and wrote a version of my idea that basically worked. Then, however, the desire had left me and I finished the project months later in september 2016 and after about 400 Commits in a version that was satisfactory for myself. In addition to clean code and a neat structure, I also tried to implement a simple installation script to also allow users with a very small linux experience to use PiSleepTalk. On a fresh Raspberry Pi, the complete system installation is completed with a 39 character long command and some time - about two to four hours.

Lauter oder lauter, du hast die Wahl! Thomas Kekeisen im Schlaf vor sich her geredet ( Video)
Besides the recording and evaluation of audio files, the system can also render videos, remove noise from the recordings and of course, the "frame-accurate" cutting of the recordings is offered via the web interface. In addition, audio files, for example, recorded with another device, can be imported for processing. Also imported zip files are unpacked automatically. And that I no longer have to think about starting the recording, it happens automatically by using data I generate every day. The recording only starts when the time is between one and six o'clock in the night. Also I have to be checked in at my apartment on Swarm. And to capture the scenario that I am at home, awake and listening to music, a recording starts only if no song has been scrobbled on over the past 30 minutes, too.

Debugging hazzle

Towards the end of the development, debugging was the biggest challenge. Because how should you check code, that runs while you are sleeping? I spent a couple of nights to find the best microphone settings since I had to "sleep over" the settings to check the results on the next morning. For the remaining problems - and you come across a lot of them when you use sox and ffmpeg - I solved by switching off some parts of the file processing in order to save the resulting file system. So I could restore the recordings again and again to restart the processing on the RaspberryPi. To check when and for what reasons my Raspberry Pi sometimes stopped or started recording, I had set up some info push messages. So I had a log on my iPhone directly in the next morning. The push messages themselves were, of course, sent silent.


The finished PiSleepTalk is open source on GitHub. In addition to the complete documentation and installation instructions, you can find some open issues in the issues area. I may implement in the future. Of course, I am very happy about contributions.

In the meantime, I have been using my installation for three months now and have produced three videos with a relatively low effort. Sometimes it happens that the SD card of my Raspberry is too full - still a unfixed problem. I suspect that by switching from a cheap USB microphone to a professional studio microphone, the recorded files have become many times larger.

Schlafgesäusel 6

YouTube 23. Oktober 2016 Open Video

Schlafgesäusel 5

YouTube 23. Oktober 2016 Open Video

Schlafgesäusel 4

YouTube 9. Oktober 2016 Open Video