Webcam using NLSU2 and OpenWRT

I am trying to set up a simple web camera to take some photos and videos of birds on our bird table.
I have had an edimax ip camera for a long time, but have had some trouble with using and configuring it from Linux.  So, I am having an attempt with a different approach from bits of hardware I have around.   There is a Linksys NLSLU2, which is made as a network storage device, and I just picked up a logitech usb web camera.   I decided to use OpenWRT as the operating system, because I used that on a previous project (bifferboard weather station).
This post is a few notes of where I have got to, and the problems encountered.

Operating System

• Download OpenWRT:   To start with I used a tarball of OpenWRT 'backfire' 10.03, but it problems with packages not being compatible.   Rather than understand why, I went for checking out the latest source code from the 'backfire' branch of the OpenWRT SVN repository.
• Build OpenWRT:  This is as simple as changing into the source directory and doing 'make menuconfig', selecting the target system as 'IXP4xx', then the profile as NSLU2, exiting and typing 'make'.   This creates a flash disk image as bin/ixp4xx/openwrt-nslu2-squashfs.bin.
• Put the NSLU2 into upgrade mode by powering off, pressing and holding the reset button with a paper clip then powering on.  Wait for ~10 sec until the top LED changes colour (very difficult for me to tell as I am colour blind...), then release the reset button.  The top LED now flashes two slightly different colours.
• Connect the computer with the OpenWRT image on it to the same ethernet network as the NSLU2 - Wireless connections do not work!.
• Flash the new image onto the NSLU2 using 'sudo upslug2 -i openwrt-nslu2-squashfs.bin'.
• Once the flashing is complete, the NSLU2 reboots and you should be able to telnet into it - default ip address is 192.168.1.1, which is a pain if that is the same one as your router...

Configuring OpenWRT

The basic OpenWRT build does not do much, so it needs to be configured and the extra required packages adding:

• Default IP Address - this can be set by editing package/base-files/files/etc/config/network [there is probably a better way to do this, but this works...].
• Add the drivers for the USB video cameras - using menuconfig select them in the kernel config section - make sure they are selected to be built-in [*] rather than modules [M], because otherwise they are not included in the flash drive image.
• Add some software to do something with the video source - add the mjpg-streamer and motion packages.
• Re-build OpenWRT and re-flash it.
• You should now be able to look at :8080?action=stream and see the image from the camera from mjpeg-stream

Making it Do Something Useful

• mjpg-streamer seems to work nicely and produces a jpeg stream that can be viewed on a browser.  No sound though - will have to look into how to deal with that.
• motion also seems to work - need to modify /etc/motion.conf to tell it what to do.
• I would really like to to record video clips when motion is detected, so am currently trying to see if I can build ffmpeg to link that to motion.....
• I have run out of time this weekend.  What I have done is just used motion and removed mjpeg-streamer, as motion provides a mjpeg stream too.  The basic set-up is:
• Logitec usb web cam.
• Motion records images to /tmp/cam1/// - separated into different directories to stop the numbers of files getting out of hand.
• A little php script /cgi-bin/browse.php is used to browse through the directories and view the images.
• I could not get ffmpeg to compile, so no videos for now.
• This is going to be installed at my sisters, so will have to talk her through setting up her router so I can ssh into it remotely to fix it when it does not work.....
• My version, using the same motion and php script set up, but using the edimax ip camera can be seen at http://maps.webhop.net/webcam.

OpenWRT Based Utility Meter Monitor

I have made some progress porting my MeterServ application (http://meterserv.webhop.net) to openWRT.   I have managed to work out how the openWRT makefile system works enough to add it to the build system and produce an openWRT package for the MeterServ application.
To do the improvements that I want to make to the application though, it needs more work.  In particular I don't like Perl, so want to convert the web interface part of it to Python.   Rather than go for a completely DIY web framework, I have adopted webpy (http://webpy.org) - this seems like a simplified version of django - you define URLs that you want the application to work with, then point it to a python class that does the processing associated with the user requesting a particular URL.
I have started a seperate google code project for meterserv, because a couple of people expressed an interest in it.  I have put the latest version of the software in its repository too - see http://code.google.com/p/meterserv.

Utility Meter Monitor

I have had a simple utility meter monitoring system working in my parents and my house for a few years.  It uses an NSLU2 single board computer connected to a simple USB input/output interface, as described at http://meterserv.webhop.net.
It worked fine for at least 3 years, but started to misbehave recently.  I have not been able to determine if it is a hardware or software fault - I suspect that I may have worn out the USB memory sticks that I used for the root filesystems.
We have gone for a hardware change to see if that is the problem, but I want to develop the software a bit more too.
Future developments include:

• Add support for cheap electricity meters (e.g. OWL USB energy monitors).
• Add support for temperature sensing.
• Better web interface (the current one uses perl based CGI scripts, which are awful to maintain).
• A simple (end user friendly) software upgrade process.

I want to get rid of the perl cgi scripts because I don't really speak perl, so they are hard to follow - will change them for an ajax based front end with python server side scripts.

The first stage is to get a sofware development environment working.  The original version used the nslu2-linux slugos operating system.  This has been changed quite a bit since I used it, so I decided that as I will need to do a complete re-build, I will standardise and use OpenWRT for all my little embedded system projects.

Compiling OpenWRT for the nslu2 was nice and easy - in the menuconfig just select the processor as an ixp4xxx and the sub-type as nslu2.

Edit target/linux/ixp4xxx/base_files/etc/config/network to give a default network configuration that will work on your system (the openwrt default ip address is 192.168.1.1, which clashes with my router....).

This creates a file called bin/ixp4xxx/openwrt-nslu2-squashfs.bin which can be flashed onto the nslu2 as follows:

• Put the NSLU2 into upgrade mode by switching off, then powering on while holding in the reset button for about 10 seconds.   When the status LED changes colour (at about 10 sec), release the reset button.  The status LED now flashes different colours (very subtly different colours if you are colour blind like me!).
• Use upslug2 (sudo apt-get install upslug2 on Ubuntu) to upgrade the NSLU2 using:  upslug2 -d wlan0 --image=openwrt-nslu2-squashfs.bin
• The NSLU2 will re-boot, but then take a while (a few minutes) to set up a writeable overlay file system before being accessible over the network.

This showed that I could build software for the NSLU2 using openwrt, but now I need to port meterserv to OpenWRT, which will be a separate post....

OpenWRT on Linksys NSLU2

I have had a couple of NSLU2's running a little application to monitor utility meter readings for a few years now (meterserv).   It worked fine until the autumn when my Dad's one stopped recording readings.  I have not been able to find out why - it looks ok.  It may be that the interface card has broken, or I do wonder if I have worn out the USB flash drive that I am using for the root filesystem.

In my inability to find a fault, I decided to do a clean install on a usb disk to rule out software and flash drive problems.   They had been using OpenSlug (now called SlugOsBE) from nslu2-linux, but it is now very out of date, because I have not updated them for a few years.  This means that all the links to package directories are broken etc.

Given the recent success in building OpenWRT for the bifferboard, I decided to use OpenWRT for the re-build.   These are my notes so I can do it again next time....

• In the MenuConfig system, set the target system to 'Intel IXP4xx', sub target to 'Generic' and target profile to 'Linksys NSLU2'.
• Because this is not my new router, I do not want it to use the default IP address of 192.168.1.1.  This can be changed in the 'Image Configuration' bit of the configuration menu.
• I would like to re-code some of the meterserv software in python, so I need python and pyusb installed in the flash image - these were selected from the 'Languages' section of the openwrt menu.
• Make builds a firmware image in bin/ixp4xxx called openwrt-nslu2-squashfs.bin, which is an 8MB image - there is a 16MB one too, but I am not convinced that my nslu2 has that much flash memory.
• You have to put the nslu2 into 'recovery mode'  by holding the re-set button in while you power up the machine - you have to release the reset button as soon as the status LED changes colour, after about 10 seconds.  The status LED then flashes a couple of different colours.
• Flash the new firmware image onto the NSLU2 using upslug2 (sudo apt-get install upslug2 on ubuntu) - the command line was sudo upslug2 --target 00:14:bf:64:df:be --image openwrt-nslu2-squashfs.bin.
• The machine now appears on the network with the right IP address.  Telnet into it and change the root password, then connect using SSH instead.

The only problem is that when I type 'python' I get an error "python: can't resolve symbol 'BC'".  So now I need to work out which package is missing - looks like openWRT missed a dependency, but I don't know which one!

I have had a bit more of a look at it.  There are two surprising things.  The first is that if I write a simple 'hello world' script and execute it, it works, so it is just interactive python that is broken.

The second is that doing "ldd /usr/bin/python" on the nslu2 and on the bifferboard give very similar results, except that the bifferboard one is linked against libgcc and libc, but the nslu2 one is linked against libc.

I think this is telling me something, but I am not sure what....

Personal Weather Station using BifferBoard

I have managed to re-compile openWRT to run on the bifferboard, and include python, libusb and pyusb in the main root directory.   I also included the wget utility, because I thought it would come in useful, plus a few other potential future extensions (lighttpd web server and wireless LAN support).
This fits on the bifferboard onboard flash memory, with about 1MB to spare.

I created a /home/weather directory and extracted a recent pywws tar archive into it.   I was very pleased that running TestWeatherStation.py produced a table of numbers as expected, so it looks as though the bifferboard is talking to the weather station ok.
Here it is:

Set up the initial weather database by doing

cd /home/weather 

python pywws/pywws/LogData.py -vvv /home/weather/data

This creates /home/weather/data/weather.ini and a directory /home/weather/data/raw which contains the raw data.

Run Hourly.py for the first time to process the data, and set up the weather.ini file ready for customisation:

pywws/Hourly.py /home/weather/data

Customise /home/weather/data/weather.ini to do what you want it to do - in my case update weatherunderground....

All is looking promising here - seems to send the update ok on an hourly basis.  There are two problems though:

• There is not much disk space left now I have added all of January's weather data (256kB ish)
• I am concerned that I will wreck the flash chip on the bifferboard with all this writing.

One option is to store the weather data in a ram disk and just accept that after a power-off it will need to re-initialise itself - not sure how pywws will cope with this - I will need to load some default weather.ini file from flash every time it boots, then let it update itself as best it can.

As a quick alternative I have found an old 256MB SD card and created an ext3 filesystem on it.  This detects as /dev/sda1 when plugged into the bifferboard.   I have modified /etc/rc.local to mount this as /home/weather.  This should solve my full filesystem problems.  [Note:  I used an ext3 filesystem because my openWRT build could not detect ext2 - this was a bit of a surprise because I thought you got ext2 support free with every Linux kernel...].
Then it is just a matter of a cron job to do the hourly updates - crontab -e, then add:

13 * * * *       python /home/weather/pywws/Hourly.py -v /home/weather/data >> /home/weather/Hourly.log 2>&1

If it works, you should continue to see the weather in our back garden at: http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IHARTLEP2.

Mistake 1:  If you put an SD card into the dual USB bifferboard, it disables one of the USB ports.  This is bad if it was the one the weather station was connected to....

Mistake 2:  Believe people when they say the clock on a board is no good.  The Bifferboard clock is no good.  Not only does it loose its time when powered off, but it drifts - by a few minutes an hour.  This meant that pywws got confused about when it should update weatherunderground, because it thought time had suddenly gone back to 2009....   To avoid this I compiled ntpclient as a package and downloaded it onto the bifferboard.   I now have it set to be called once on boot, then every 10 minutes by a cron job.  This should keep it somewhere close - just hope the time server owner doesn't mind - If I get a complaint I will have to set up my own time server...

Building openWRT for BifferBoard

I had a go at building openWRT for my new bifferboard computer last weekend, but had trouble with it - it looked like the kernel started, but then panicked around the time it was supposed to be starting init. I suspect it was something to do with the j2ffs file system.

I decided to try to be clever and rather than fix openWRT, start from scratch with an OpenEmbedded based build. I think this was too big a step - the main problem was that OpenEmbedded failed to build because some sources from handhelds.org are not available, so I had to learn how to alter 'recipes' to build them from sources from another location. I got ipkg-utils to build this way, but then hit more trouble with openssh. I decided it was going to take me a long time, so went back to openWRT for now to try to get something working!

This time I followed the exact instructions from the bifferboard site, including downloading the specific revision of openWRT.   This all compiled nicely without any errors, so the next challenge was to get it onto the board.

Flashing over the serial line seemed to take a very long time, so I had a go with network setup.
I downloaded the bifferboard utilities using:

svn co https://bifferboard.svn.sourceforge.net/svnroot/bifferboard

This provides a simple python based tftp server.   I had to modify it to use a fixed ip address for my host computer, because the script assumes that you are using eth0 for the network interface, and I was using wireless.
You then need to make sure that there is a file called bzImage in your working directory and start bootp_server.py - I had to do it as root because I got some permission denied errors about opening sockets, and the easiest way to fix it was to be root - there is probably a more elegant solution though.

First I created a symbolic link to the bzImage file that openWRT had produced.   Then booted the bifferboard with the serial line connected and pressed ESC to stop the boot process.   Entering the tftpflash command downloaded the bzImage from the server and flashed it to the disk - much quicker than a serial line.
The kernel started, but failed with a lot of j2ffs errors - I think this is because I had my old root filesystem in the flash memory.
For the next attempt I created a symbolic link from openwrt-rdc-jffs2-64k-bifferboard.img to bzImage.
Re-flashed it again the same way and it boots!

A minor detail was that I could not log into it - the serial console was fine and both telnetd and dropbear (the SSH server) were running, but I got 'connection refused' when I tried to connect.   It turned out that there was something called 'firewall' running.   I deleted /etc/rc.d/S45firewall and re-booted and it worked properly - I could telnet in initially without a password, set a root password, then use ssh to connect.
Success - Now I just need to sort out the weather station software to run on it - back to the cross compiler....

running make package/symlinks in the openWRT directory adds all of the available packages to the openWRT menuconfig program.
Added the ones I know I need (libusb, python, pyusb), plus a few others that might come in useful later (atheros wireless card drivers).  Then make.....and wait...

Building openWRT

I have got a little bifferboard single board computer which I intend to use to run pywws to send data from our weather station to the weather underground site on the internet.

The bifferboard comes installed with a very small linux distribution called openWRT.   It has a very small python installed, but no python USB support.

I have struggled a bit to work out how the openWRT package system works - the official wiki is a bit confused about what the current version is called (kamikaze or backfire).  It implies you can add packages by downloading them from svn, but this didn't seem to do anything.
I found a useful forum post here, which seems to be the best set of instructions.
You can do

./scripts/feeds update -a

This downloads the list of packages, but does not do anything else.

To get the buildroot system to compile it for you you need to 'install' it using:

./scripts/feeds install python 
./scripts/feeds install pyusb
You can then do "make menuconfig" and python and pyusb are shown to be compiled as packages "".

'make' actually compiles it.

I'll update this when I work out how to add these to the firmware image...