The Raspberry Pi tutorials often mention the rc.local file for this purpose, or even the usage of cron. What’s missing with these methods is a way to control the lifecycle of our script after it have been first launched at startup. How then to stop or restart the service?
The common sysadmin way of doing that is to use the init system of your distribution. Typically on a Debian-like you have used init.d scripts and update-rc.d to schedule your script at boot time. That works fine, even if I can’t help Googling the magic commands.
Here the tricky thing: with the large adoption of systemd as the new init system you’ll have to learn a new way of doing that, which is a bit simpler, but still doesn’t resolve the Googling issue when you doing that each 3 months.
And with all these configuration files we still don’t have monitoring features1.
To make our programs start at boot time, but also continue running thereafter, no other thing is better that a dog that can watch and bark at our processes.
I found monit to be straightforward-enough to set up and a versatile tool that you can turn around your needs. Also it has the huge pro to work on most Unix distributions, so we don’t have to Google for a solution again and again each time we change of playground.
Once installed, you can directly configure your monitored services in /etc/monit/conf.d/
.
To ensure a process is always running based you simply specify its pid file and the start and stop scripts, you can for example create a radbox-web
configuration file in /etc/monit/conf.d/
:
Yes you need a PID file to store the PID of the process that you want to monitor. In the example I use make
to launch my program, which is pretty cool because… well, who does not know how to write a simple Makefile? In the negative case Google’s still here.
Just keep in mind to properly populate the PID file each time you start your program. Here a simple example how you can do it:
Then you can reload monit to enable the monitoring for the newly created service and check that all is running:
Which will gives you something like this great this great output.
You may need to uncomment the following lines in your /etc/monit/monitrc
configuration file to be able to speak with the monit deamon from the command line:
Monit comes with a complete set of monitoring features, so you can not only check that the process is running, but also that it is doing sensible operations.
In a web context you can check you’re web server is up and returns you something else than 500s:
For each rules we can choose to throw an alert, to restart the service or whatever action you have defined. If you run dozen of services and some critical ones alerts can be a great deal to help you go out or sleep without over-worrying… until you receive a mail from monit!
For more complex monitoring cases you can dig in the tool, which contains a good set of what is commonly needed. But in the end of the day if you really want to play with this dog you better go read the fucking manual than stay here.
1. That’s unfair for systemd, as it does have monitoring features, and even embeds a watchdog protocol named sd_notify. But the watchdog is not easy to use and is intrusive. And life’s unfair.
]]>To do better we should ensure the data will last beyond our enthusiastic-initiative and can be retrieved and exploited by others.
We’re fortunate because this is exactly what the Safecast project aims to provide: an open platform for everyone to publish their environmental data, with great navigation and visualization tools, a real-time portal and last but not least a hacker data oriented mindset.
The Safecast initiative also followed the Fukushima crisis, in an urge to collect and publish accurate and trustworthy open radiation data, openly accessible to citizens of the world1. The volunteer-driven organization has been known to provides efficient technical means, like the bGeigie Nano device, a full-pledged radiation sensor.
Compared to the Radiation Watch Pocket Geiger, the bGeigie it is not cheap2, but way more complete and accurate. With its GPS chip and real-time clock it allows effective mobile radiation measurements. Its pancake can monitor all Alpha, Beta and Gamma radiations. The device has been successfully used to collect millions of data-points.
Past the hardware, maybe the most interesting part is the whole system conceived and maintained by the Safecast community to store, share and consult the data. With the Safecast API measurements can be made available and controlled by citizens, researchers and journalists.
To made things really easier I’ve developed a Python wrapper for the Safecast API: here comes SafecastPy. The implementation was done following the Twitter API wrapper Twython library as a model.
To make it works, that’s the usual Python pip-ABC:
Then do whatever you want3. Only do please be conscientious about the data you sent to the Safecast API, as this is a voluntarily maintained database. To test things out you can use the development instance, which do not fear messy data.
As Python is a very convenient language to manipulate near any data format you’ll have no difficulty broadcasting you’re own sensor data to the Safecast API. This way your data will gain in power by contributing to a worldwide and open database.
You can also use the library to navigate through the Safecast open-data and use it on your own project.
Coming back to our Pocket Geiger playground, we can now publish its measurements to the Safecast API. That will be our way to create a bridge between these two great initiatives than are Radiation Watch and Safecast.
With the PiPocketGeiger library this is a no-brainer:
The complete code can be found here.
To test things out I myself maintain one of my Raspberry Pi logging my Pocket Geiger readings to Safecast:
All these measurements can been retrieved directly on the Safecast API website. As you can observe there still isn’t much interesting hazard in my house. Does someone has Cesium 137 to send me?
Now you can take a Raspberry Pi, get your hand on a Radiation Watch Pocket Geiger, and yeah you’ll have a Geigier counter for your house. The software produced in this series is open-sourced and documented5, so if you have basic hardware and Python skills you surely be able to get it work. No more unnoticed wild rays in your house!
I hope you’ve enjoyed this series. Please don’t miss the chance to tell me! Also if you have projects in the same vein I’ll be very happy to hear from you. I myself have ideas following on from that series6. Maybe you’ll have funnier ones?
1. See complete history here.
2. Nor it is ready to use, as it comes as a kit whose requires some work to assemble.
3. For the usage please refer to the GitHub README.
4. I already apologize for the data logging discontinuity, caused by the very too frequent outages of my internet connection. I really do need to relocate.
5. More or less. Tell me where it’s not good or clear enough.
6. Like putting all that in a box and create a software platform for everyone to easily publish background radiation measurements. It will be the people choice to publish their sensor data to internet services or open-data platforms X or Y - simply by pushing a button. Well, that’s an idea.
]]>In the first part we were logging the Pocket Geiger measurements to an SD card with an Arduino. As my title promises a Geiger counter for your house, we need to do better! Like something actually connected.
Indeed it would be great to post the radiation level online in real-time. To realize this mission we need a board that can be easily connected to internet, and that is obviously able to speak with our Pocket Geiger. Guess what? This is exactly what Raspberry Pi is great at!
Instead of connecting our Pocket Geiger to an Arduino we will wire it to the Raspberry Pi GPIO ports. We will benefit the Raspberry Pi generous package: Ethernet and USB ports, so the network access is simple ; HDMI output, so we can check on a screen what’s going on. The Raspberry Pi ARM chip is also way more powerful than the Arduino Uno ATmega328 microcontroller: it can honorably run a whole GNU/Linux distribution. With the dedicated Raspbian OS - a Debian-fork for… yes Raspberry Pi! - we will enjoy a familiar and welcoming environment.
Having the Pocket Geiger linked to the Raspberry Pi is exciting, however that doesn’t bring much in itself. Without software hardware’s of no use! We need a way to communicate with the Pocket Geiger and get back the readings to our Raspberry Pi.
As I haven’t found something that really convinced me1, I developed a Python library to communicate with the PocketGeiger: please welcome the PiPocketGeiger library.
The PiPocketGeiger library is built on top of the RPi.GPIO package. It uses edge detection interrupts to count the ionization events and periodically processes the current dose in a thread. This is roughly a Python rewrite of the Arduino library C code.
A major difference with the Arduino version is that Linux is not a hard real-time OS2, so we have no guarantee we’re effectively monitoring the edge falling in a timely manner. That said the timing of an interrupt from the Pocket Geiger3 makes it unlikely that we miss a wild Gamma ray. However I’m not competent enough on the subject to be definitive.
You can install the library using pip
. The RPi.GPIO package needs root privileges to access to the hardware, I thus recommend to install the lib root-wide:
That done you can use the lib in your Python script. You firstly initialize the lib to indicates which GPIO pins to read from. Then you can get the current background computed radiation level whenever you need:
For those that already ask for reactive applications, yes you can register callbacks that will be triggered when a gamma ray drop on the Pocket Geiger, or when there is noisy events:
To truly benefit the power of our Raspberry Pi the library provides examples to publish the data in live on various web services.
First typical application we can live-tweet the radiation level in our house, so nobody miss you’re hacking:
Radiation in my house (Laussou, France): 0.059 uSv/h +/- 0.012 -- 3.12 CPM @radiation_watch
— Yoan Tournade (@tournadey) 16 décembre 2015
Instead of simply logging to a file on your Rapsberry Pi you can log to your Google Drive account:
These examples are fine to keep trace of raw data, but not quite handy to visualize it. We can use Plotly, a great service to create and share plots:
For the next part we will make all that more open - I mean more open-data oriented. Coming soon. Hopefully. Yeah we’re pretty lazy here.
Edit March 30th, 2016: Here it is!
1. There is this C driver that seems to work fine (haven’t tested it), but I wanted a pure Python interface and a lib that can be easily distributed and installed with pip.
2. This comment apply only for the standard Raspbian distribution, has it does exist a patched version of Linux for real-time applications: the RT-Preempt Patch, that gives hard real-time capabilites to the kernel. You may also check the Xenomai or RTLinux projects, which each kind of make cohabit a hard real-time kernel with the Linux one, ensuring the whole can be preempted in a deterministic way.
3. The Radiation Watch sample code says the Radiation pulse normally keeps low for about 100 microseconds.
]]>Radiation Watch is a scientific and citizen initiative born after the Fukushima Daiishi disaster and which has been funded through Kickstarter in July 2011. It aims to provide a cheap radiation detector that anyone can use, even boars.
Initially conceived to be connected to an iPhone, Radation Watch now provides an embedded version of the Pocket Geiger. Whatever version is used, the board includes a X100-7 PIN photodiode from FirstSensor for gamma-ray detection. It is thus not a proper Geiger-Müller counter.
If you don’t remember well we usually classify radiations under three hats: alpha, beta and gamma rays. Alpha and beta radiation are charged particles, whereas gamma rays are photons of electromagnetic energy, with no charge and mass. They are all three considered ionizing radiation - which means they can alter the matter they go through -, but have very different penetrating and ionizing abilities due to their respective mass, size and nature.
Alpha and beta radiation can be stopped by an aluminum foil: if you don’t play with radioactive sources or ingest contaminated substances you hopefully won’t be exposed to significant level of alpha and beta rays, except maybe from the radon in your house. This contrasts with the very high frequency electromagnetic nature of gamma rays, giving them a super high ability to pass through matter: imagine your Wifi on steroids, with a frequency of above 10 exahertz and an energy more than 100,000,000 times greater. Yes, it can cooks your DNA like your microwave boils your Chinese noodles.
When you consider background radiation monitoring, detecting only gamma rays gives you a sensible indication of your exposition to radiation, from both natural and artificial sources. If it will fail to detect the contamination of your food or water, it will help you determine the environment radiation level and its changes over time. Cosmic rays are chasing your computer? Doubtful steam comes out of the house of your maybe-too-genius neighbor? You’ll have a chance to really know.
Speaking of ionizing radiation and its effect on human body, we use the sievert unit to quantify a dose. One sievert is bad enough to significantly increases chances of cancer (see this great dose chart). Since we don’t usually absorb high-level dose in a one shot manner, we associates the unit with a time dimension when we do background measurements: e.g. uSv/h, for the amount of ionizing radiation you will received staying at a place for one hour.
Ok, enough radioactivity blabla, place to electronic and code. The Pocket Geiger board comes with four pins: two are for the usual alimentation stuff (+V
, GND
) and the two others for the signals (SIG
, NS
). When the sensor is hit by a radiation, it will simply pull the radiation pin (SIG
) to an high voltage level for some microseconds. But as the photodiode sensor is sensible to vibrations, Radiation Watch also included an accelerometer so we can get notice of them through the noise pin (NS
) and dismiss the corresponding false-positives.
When your Pocket Geiger is wired, you’ll need an Arduino firmware to do something with it. Radiation Watch provides sample code to log measurements through the Arduino serial port. To ease integration with other software Thomas Weisbach has released a library. I’ve followed the Toumal work to make things even better and provides a library with cleaned code, reduced memory footprint and documented examples, so you can start rapidly hacking your things.
The library is available on GitHub and released under the MIT license. It comes with a handy Python script to plot the radiation dose in real-time from your serial port, or with a sample sketch for logging data on an SD card thanks to the Ethernet shield.
To use it, you need to create a Radiation Watch
object with the pin and matching IRQ numbers to which are connected your Pocket Geiger:
This object must be initialized at the Arduino setup phase with RadiationWatch::setup()
. In your program main loop you also need to run the RadiationWatch::loop()
frequently in order to compute the statistics.
Then you can get the radiation level whenever you need using RadiationWatch::uSvh()
, with the current incertitude specified by RadiationWatch::uSvhError()
. You can also register callbacks that will be called in case of radiation or vibration occurrence, with respectively RadiationWatch::registerRadiationCallback()
and RadiationWatch::registerNoiseCallback()
.
Complete examples can be found here.
Finally all this is good, but for one purpose: measuring the background radiation level. Let’s see what kind of data we can get from it.
I’ve first done background radiation measurement at the first floor of a house in Colombes, near Paris:
Surprisingly I was able to measure radiation being in a moving train. I expected the vibrations to skew entirely the results, but it wasn’t the case. The accelerometer wasn’t triggering the noise detection pin and the measured values are sensible ones:
In my family house, somewhere in the south-west countryside, we’re a little more exposed, but still way not enough to turn Hulk.
Since I don’t have any radioactive source (for the happy folks living in USA, you can freely buy and possess little ones) nor the time to go in the volcanic parts of France, I’ve gone to my local nuclear power plant, as a last attempt to measure something interesting. But outside an undamaged containment building and at a view-distance from the plant, that’s not very exciting.
You have maybe noticed the Pocket Geiger is not accurate enough to give instant results: it need at least two minutes to reduce the error incertitude and stabilize the readings. You can spot this on the graphs: the first results are highly dispersed, with a great incertitude range.
Since this device seems good for background monitoring, I’ll connect it to my Raspberry Pi so we can plot the data online in real-time. Stay tuned.
Edit March 14th, 2016: The second part is now out!
Thanks to Nicolas C. for reading drafts of this, to Thomas Weisbach for having managed the license issue and to Nicolas Michaux for giving me the idea. Also props to Grégoire Martinache to spot the typos no other can see. ;-)
]]>findpeaks
function in the Python world.
For those not familiar to digital signal processing, peak detection is as easy to understand as it sounds: this is the process of finding peaks - we also names them local maxima or local minima - in a signal. The MatLab DSP Toolbox makes this super easy with its findpeaks
function. Saying your want to search local maxima in an audio signal, for example 2000 samples of the Laurent Garnier famous track Cripsy Bacon, all you have to do is:
We can specify filtering options to the function so the peaks that do not interest us are discarded. All this is great, but we need something working in Python. In a perfect world it will give exactly the same output, so we have consistent results between our Python code and the MatLab code.
Contrary to other MatLab functions that have direct equivalents in the Numpy and Scipy scientific and processing packages, it is no easy task to get the same results from the Scipy find_peaks_cwt
function that from the MatLab findpeaks
. Even worse, the wavelet convolution approach of find_peaks_cwt
is not straightforward to work with: it adds complexity that is of no use for well-filtered and noiseless signals.
Wondering how to make our algorithms works as simply with Python that they were in MatLab, I’ve search around the web for other peak detection algorithms available in Python. Stackoverflow get me to peakdetect
, a translation of a MatLab script. As it is clearly more trivial to use that find_peaks_cwt
, it still won’t give you the same results that the MatLab findpeaks
function. The algorithm don’t find all peaks on low sampled signals or on short samples, and don’t have either a support for minimum peak height filter.
Going ahead I’ve checkout the GNU Octave project, a processing-intended language quite similar to MatLab. The Octave-Forge repository hosts a digital signal processing package with a findpeaks
function. Bingo? Well, yes and no. The function have an appealing interface, with a great filtering support. But this is not a Python project: as you’ll find ways to call your Octave distribution from your Python code (see oct2py), it surely won’t be effective at large scale and makes the requirements for your code more complex.
As I was going to code a Python adaptation of the Octave-Force findpeaks
, I finally found what I was searching: a Python native equivalent of the MatLab findpeaks
, with minimum distance and height filtering support. I even found two!
The first is the PeakUtils package by Lucas Hermann Negri which provides 1D peak detection utilities. Its indexes
function allows you to detect peaks with minimum height and distance filtering.
The second is published on a jupyter notebook and is written by Marcos Duarte. Not easy to find, but the greatest so far for what I need: the parameters for filtering are modeled after the MatLab findpeaks
ones. It comes as a single source file and only depends on Numpy, so it is no big deal to integrate. And more importantly, it will consistently get you the same results than MalLab findpeaks
!
To avoid others the same roaming I’ve put on GitHub an overview of these findings. Let me know if you got another open-source alternatives so we update the list.
As Lucas Hermann Negri pointed out on HN, the PeakUtils indexes
function was actually inspired by Marcos Duarte implementation of detect_peaks
, explaining the similar results. Moreover he notes that the PeakUtils comes with other convenient utilities, such as baseline
or interpolate
.
The interpolate
function enhances the peak resolution by fitting Gaussians or computing centroids. Taking the previous example, here how you get it work:
This time before the peak resolution, the baseline
function will be very handy in presence of drifting signals or to deal with unwanted low-frequency phenomenon: it kind of high-pass filter the signal. The PeakUtils documentation have a good example of its use.