Well, I thought that the introduction of the ext3 (and higher) file systems had stopped all of the problems of disk crashes and mangled filesystems on linux.
But, after a few years of reliable operation the root filesystem on my home server (my old Fujitsu Siemens laptop) seems to have mangled.
The symptoms were....not booting - you get the 'Starting up......' message during boot then the system appears to hang. But if you listen carefully you hear that the disk is actually doing something. So I left it for a few hours.....
When I came back to it there was a 'failed to mount /dev/sda1' type error message, with an option to skip mounting or manually fix it.
I went for manually fixing it, because not mounting the root filesystem will not get me very far.
This dropped me into a single user shell.
I ran fsck and it said that an Inode has illegal blocks. I selected the option to clear them...and again said yes when it asked me again.
It then said it was restarting e2fsck from the beginning, and spent quite a few minutes checking....
Then at Pass 2 fsck said it had found a deleted or unused inode. Again I said 'y' to the 'Clear?' question....
Then lots of offers to fix things (to the extent that I just held my finger on the 'y' key...)
Then fsck announced that it was complete and I should re-boot linux. Re-ran fsck and it announced that /dev/sda1 was clean, so re-booted.....but booting is taking a suspiciously long time.....like it has been trying for 10 minutes and hasn't got past the boot up splash screen...I'm going to need a plan B...
Well, I don't know what the problem is. Tried booting off a USB memory stick, and the disk checks ok and mounts, but the boot process just hangs. I decided I had spent too long on this so it is currently installing Ubuntu 10.10 on the disk instead, so can't try any more diagnosis!
A record of me trying to fathom out how to do a few nerdy things - hopefully this will help me remember how I made things work when I have forgotten...
Tuesday, 29 March 2011
Sunday, 6 March 2011
Emacs Drupal Mode
I am trying to learn how to write Drupal modules for a new map rendering on demand web service (a bit like townguide, but distributed with a separate front end and rendering servers).
First thing is to get emacs23 working with php and drupal modes - the instructions here give me obscure errors about c-lang-def only being used in a file. After a bit of hunting on the internet I found it is a problem with Emacs V23, and a simple work around is to put the php-mode.el and drupal-mode.el files in ~/.emacs.d, then add the following to ~/.emacs:
First thing is to get emacs23 working with php and drupal modes - the instructions here give me obscure errors about c-lang-def only being used in a file. After a bit of hunting on the internet I found it is a problem with Emacs V23, and a simple work around is to put the php-mode.el and drupal-mode.el files in ~/.emacs.d, then add the following to ~/.emacs:
With that in my .emacs file, loading a .php or .module file starts php or drupal mode - success!; make sure the target directory is on your load-path(add-to-list 'load-path "~/.emacs.d")(autoload 'php-mode "php-mode" "Major mode for editing php code" t)(autoload 'drupal-mode "drupal-mode" "Major mode for editing drupal php " t)(add-to-list 'auto-mode-alist '("\\.\\(module\\|test\\|install\\|theme\\)$" . drupal-mode))(add-to-list 'auto-mode-alist '("\\.\\(php\\|inc\\)$" . php-mode))(add-to-list 'auto-mode-alist '("\\.info" . conf-windows-mode))
Thursday, 24 February 2011
Easier Map Creation with OpenStreetMap Data
I have been giving some thought to making the process of creating maps form OpenStreetMap data easier. The problem is that to use the mapnik map renderer, you need to install quite a lot of tools and set different configuration files before you can even start to create a map.
This means you have to be pretty keen to persevere enough to get to an end result.
I am thinking of creating a service to allow you to bypass the tool installing bits, and concentrate on the setting up of the map configuration, so you can get to a result quicker - see http://wiki.openstreetmap.org/wiki/User:Grahamjones#Speciality%20Maps.
Quite a few of the responses that I got to that proposal were about the lack of good documentation to get people started on the process. I have had a go at addressing this by putting together a simple overview presentation to go through the main concepts and tools that you need to generate a map using mapnik. The presentation is on slideshare.net, and should be visible below:
This means you have to be pretty keen to persevere enough to get to an end result.
I am thinking of creating a service to allow you to bypass the tool installing bits, and concentrate on the setting up of the map configuration, so you can get to a result quicker - see http://wiki.openstreetmap.org/wiki/User:Grahamjones#Speciality%20Maps.
Quite a few of the responses that I got to that proposal were about the lack of good documentation to get people started on the process. I have had a go at addressing this by putting together a simple overview presentation to go through the main concepts and tools that you need to generate a map using mapnik. The presentation is on slideshare.net, and should be visible below:
Friday, 18 February 2011
Mounting FTP Directory as a Filesystem
I just discovered a really useful trick to mount the directory structure of a remote FTP server as a directory on a linux computer. You use a program called curlftps. On Ubuntu do:
A solution is to create a .netrc file in the /root directory containing:
sudo apt-get install curlftpsThe directories on the remote ftp server are then mounted using:
curlftpfs ftp://[username]:[password]@[server address] [local directory mount point]This works fine (you do not even need to be root to do it), but the password will be present if you do 'ps -ef | grep ftp'.
A solution is to create a .netrc file in the /root directory containing:
machine [server address]
You can then add the file system to /etc/fstab by adding the following line to /etc/fstab:login [username]
password [password]
curlftpfs#[server address] [local directory mount point]You can then just dofuse allow_other,rw,user,noauto 0 0
sudo mount [local directory mount point]and you get the same effect with the server password invisible.
Sunday, 13 February 2011
OpenWRT Based Utility Meter Monitor
I have made some progress porting my MeterServ application (http://meterserv.webhop.net) to openWRT. I have managed to work out how the openWRT makefile system works enough to add it to the build system and produce an openWRT package for the MeterServ application.
To do the improvements that I want to make to the application though, it needs more work. In particular I don't like Perl, so want to convert the web interface part of it to Python. Rather than go for a completely DIY web framework, I have adopted webpy (http://webpy.org) - this seems like a simplified version of django - you define URLs that you want the application to work with, then point it to a python class that does the processing associated with the user requesting a particular URL.
I have started a seperate google code project for meterserv, because a couple of people expressed an interest in it. I have put the latest version of the software in its repository too - see http://code.google.com/p/meterserv.
To do the improvements that I want to make to the application though, it needs more work. In particular I don't like Perl, so want to convert the web interface part of it to Python. Rather than go for a completely DIY web framework, I have adopted webpy (http://webpy.org) - this seems like a simplified version of django - you define URLs that you want the application to work with, then point it to a python class that does the processing associated with the user requesting a particular URL.
I have started a seperate google code project for meterserv, because a couple of people expressed an interest in it. I have put the latest version of the software in its repository too - see http://code.google.com/p/meterserv.
Saturday, 29 January 2011
Utility Meter Monitor
I have had a simple utility meter monitoring system working in my parents and my house for a few years. It uses an NSLU2 single board computer connected to a simple USB input/output interface, as described at http://meterserv.webhop.net.
It worked fine for at least 3 years, but started to misbehave recently. I have not been able to determine if it is a hardware or software fault - I suspect that I may have worn out the USB memory sticks that I used for the root filesystems.
We have gone for a hardware change to see if that is the problem, but I want to develop the software a bit more too.
Future developments include:
It worked fine for at least 3 years, but started to misbehave recently. I have not been able to determine if it is a hardware or software fault - I suspect that I may have worn out the USB memory sticks that I used for the root filesystems.
We have gone for a hardware change to see if that is the problem, but I want to develop the software a bit more too.
Future developments include:
- Add support for cheap electricity meters (e.g. OWL USB energy monitors).
- Add support for temperature sensing.
- Better web interface (the current one uses perl based CGI scripts, which are awful to maintain).
- A simple (end user friendly) software upgrade process.
I want to get rid of the perl cgi scripts because I don't really speak perl, so they are hard to follow - will change them for an ajax based front end with python server side scripts.
The first stage is to get a sofware development environment working. The original version used the nslu2-linux slugos operating system. This has been changed quite a bit since I used it, so I decided that as I will need to do a complete re-build, I will standardise and use OpenWRT for all my little embedded system projects.
Compiling OpenWRT for the nslu2 was nice and easy - in the menuconfig just select the processor as an ixp4xxx and the sub-type as nslu2.
Edit target/linux/ixp4xxx/base_files/etc/config/network to give a default network configuration that will work on your system (the openwrt default ip address is 192.168.1.1, which clashes with my router....).
This creates a file called bin/ixp4xxx/openwrt-nslu2-squashfs.bin which can be flashed onto the nslu2 as follows:
- Put the NSLU2 into upgrade mode by switching off, then powering on while holding in the reset button for about 10 seconds. When the status LED changes colour (at about 10 sec), release the reset button. The status LED now flashes different colours (very subtly different colours if you are colour blind like me!).
- Use upslug2 (sudo apt-get install upslug2 on Ubuntu) to upgrade the NSLU2 using: upslug2 -d wlan0 --image=openwrt-nslu2-squashfs.bin
- The NSLU2 will re-boot, but then take a while (a few minutes) to set up a writeable overlay file system before being accessible over the network.
This showed that I could build software for the NSLU2 using openwrt, but now I need to port meterserv to OpenWRT, which will be a separate post....
Sunday, 23 January 2011
OpenWRT on Linksys NSLU2
I have had a couple of NSLU2's running a little application to monitor utility meter readings for a few years now (meterserv). It worked fine until the autumn when my Dad's one stopped recording readings. I have not been able to find out why - it looks ok. It may be that the interface card has broken, or I do wonder if I have worn out the USB flash drive that I am using for the root filesystem.
In my inability to find a fault, I decided to do a clean install on a usb disk to rule out software and flash drive problems. They had been using OpenSlug (now called SlugOsBE) from nslu2-linux, but it is now very out of date, because I have not updated them for a few years. This means that all the links to package directories are broken etc.
Given the recent success in building OpenWRT for the bifferboard, I decided to use OpenWRT for the re-build. These are my notes so I can do it again next time....
In my inability to find a fault, I decided to do a clean install on a usb disk to rule out software and flash drive problems. They had been using OpenSlug (now called SlugOsBE) from nslu2-linux, but it is now very out of date, because I have not updated them for a few years. This means that all the links to package directories are broken etc.
Given the recent success in building OpenWRT for the bifferboard, I decided to use OpenWRT for the re-build. These are my notes so I can do it again next time....
- In the MenuConfig system, set the target system to 'Intel IXP4xx', sub target to 'Generic' and target profile to 'Linksys NSLU2'.
- Because this is not my new router, I do not want it to use the default IP address of 192.168.1.1. This can be changed in the 'Image Configuration' bit of the configuration menu.
- I would like to re-code some of the meterserv software in python, so I need python and pyusb installed in the flash image - these were selected from the 'Languages' section of the openwrt menu.
- Make builds a firmware image in bin/ixp4xxx called openwrt-nslu2-squashfs.bin, which is an 8MB image - there is a 16MB one too, but I am not convinced that my nslu2 has that much flash memory.
- You have to put the nslu2 into 'recovery mode' by holding the re-set button in while you power up the machine - you have to release the reset button as soon as the status LED changes colour, after about 10 seconds. The status LED then flashes a couple of different colours.
- Flash the new firmware image onto the NSLU2 using upslug2 (sudo apt-get install upslug2 on ubuntu) - the command line was sudo upslug2 --target 00:14:bf:64:df:be --image openwrt-nslu2-squashfs.bin.
- The machine now appears on the network with the right IP address. Telnet into it and change the root password, then connect using SSH instead.
I have had a bit more of a look at it. There are two surprising things. The first is that if I write a simple 'hello world' script and execute it, it works, so it is just interactive python that is broken.
The second is that doing "ldd /usr/bin/python" on the nslu2 and on the bifferboard give very similar results, except that the bifferboard one is linked against libgcc and libc, but the nslu2 one is linked against libc.
I think this is telling me something, but I am not sure what....
Friday, 21 January 2011
Mapnik Map Rendering
Someone asked a question on the osm-gb mailing list about viewing coverage of waterways - canals and navigable rivers.
We don't seem to have one, so I created one on my http://maps.webhop.net web site.
The trouble with that site is that although it is running on a nice powerful computer (my old laptop), it is behind my domestic broadband service from Virgin Media. Although this service has a nice fast download speed, the upload speed is very poor ~900 kbps compared to 9600 kbps download. This makes the web site very slow from the outside world.
I have a little virtual server provided by CloudNext - it is nowhere near as powerful as my own server, and has much less disk space, but at least it has a fast internet connection.
I loaded a british isles extract of OpenStreetMap data into the database - it took a very long time - maybe getting on for 12 hours (not sure when it finished!) - my server takes about 2 hours. Performance rendering maps was not too bad though - rendering down to zoom level 14 took about 2 hours, which is respectable.
Just need to get the database updating to keep it up to date, and re-generate changed tiles now....
You can see the difference at http://maps.webhop.net/canals (my home server) compared to http://maps2.webhop.net/canals (the virtual server)
We don't seem to have one, so I created one on my http://maps.webhop.net web site.
The trouble with that site is that although it is running on a nice powerful computer (my old laptop), it is behind my domestic broadband service from Virgin Media. Although this service has a nice fast download speed, the upload speed is very poor ~900 kbps compared to 9600 kbps download. This makes the web site very slow from the outside world.
I have a little virtual server provided by CloudNext - it is nowhere near as powerful as my own server, and has much less disk space, but at least it has a fast internet connection.
I loaded a british isles extract of OpenStreetMap data into the database - it took a very long time - maybe getting on for 12 hours (not sure when it finished!) - my server takes about 2 hours. Performance rendering maps was not too bad though - rendering down to zoom level 14 took about 2 hours, which is respectable.
Just need to get the database updating to keep it up to date, and re-generate changed tiles now....
You can see the difference at http://maps.webhop.net/canals (my home server) compared to http://maps2.webhop.net/canals (the virtual server)
Sunday, 16 January 2011
Personal Weather Station using BifferBoard
I have managed to re-compile openWRT to run on the bifferboard, and include python, libusb and pyusb in the main root directory. I also included the wget utility, because I thought it would come in useful, plus a few other potential future extensions (lighttpd web server and wireless LAN support).
This fits on the bifferboard onboard flash memory, with about 1MB to spare.
I created a /home/weather directory and extracted a recent pywws tar archive into it. I was very pleased that running TestWeatherStation.py produced a table of numbers as expected, so it looks as though the bifferboard is talking to the weather station ok.
Here it is:
Set up the initial weather database by doing
This fits on the bifferboard onboard flash memory, with about 1MB to spare.
I created a /home/weather directory and extracted a recent pywws tar archive into it. I was very pleased that running TestWeatherStation.py produced a table of numbers as expected, so it looks as though the bifferboard is talking to the weather station ok.
Here it is:
Set up the initial weather database by doing
cd /home/weather
python pywws/pywws/LogData.py -vvv /home/weather/data
This creates /home/weather/data/weather.ini and a directory /home/weather/data/raw which contains the raw data.
Run Hourly.py for the first time to process the data, and set up the weather.ini file ready for customisation:
pywws/Hourly.py /home/weather/data
Customise /home/weather/data/weather.ini to do what you want it to do - in my case update weatherunderground....
All is looking promising here - seems to send the update ok on an hourly basis. There are two problems though:
- There is not much disk space left now I have added all of January's weather data (256kB ish)
- I am concerned that I will wreck the flash chip on the bifferboard with all this writing.
One option is to store the weather data in a ram disk and just accept that after a power-off it will need to re-initialise itself - not sure how pywws will cope with this - I will need to load some default weather.ini file from flash every time it boots, then let it update itself as best it can.
As a quick alternative I have found an old 256MB SD card and created an ext3 filesystem on it. This detects as /dev/sda1 when plugged into the bifferboard. I have modified /etc/rc.local to mount this as /home/weather. This should solve my full filesystem problems. [Note: I used an ext3 filesystem because my openWRT build could not detect ext2 - this was a bit of a surprise because I thought you got ext2 support free with every Linux kernel...].
Then it is just a matter of a cron job to do the hourly updates - crontab -e, then add:
Then it is just a matter of a cron job to do the hourly updates - crontab -e, then add:
13 * * * * python /home/weather/pywws/Hourly.py -v /home/weather/data >> /home/weather/Hourly.log 2>&1
If it works, you should continue to see the weather in our back garden at: http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IHARTLEP2.
Mistake 1: If you put an SD card into the dual USB bifferboard, it disables one of the USB ports. This is bad if it was the one the weather station was connected to....
Mistake 2: Believe people when they say the clock on a board is no good. The Bifferboard clock is no good. Not only does it loose its time when powered off, but it drifts - by a few minutes an hour. This meant that pywws got confused about when it should update weatherunderground, because it thought time had suddenly gone back to 2009.... To avoid this I compiled ntpclient as a package and downloaded it onto the bifferboard. I now have it set to be called once on boot, then every 10 minutes by a cron job. This should keep it somewhere close - just hope the time server owner doesn't mind - If I get a complaint I will have to set up my own time server...
Mistake 1: If you put an SD card into the dual USB bifferboard, it disables one of the USB ports. This is bad if it was the one the weather station was connected to....
Mistake 2: Believe people when they say the clock on a board is no good. The Bifferboard clock is no good. Not only does it loose its time when powered off, but it drifts - by a few minutes an hour. This meant that pywws got confused about when it should update weatherunderground, because it thought time had suddenly gone back to 2009.... To avoid this I compiled ntpclient as a package and downloaded it onto the bifferboard. I now have it set to be called once on boot, then every 10 minutes by a cron job. This should keep it somewhere close - just hope the time server owner doesn't mind - If I get a complaint I will have to set up my own time server...
Building openWRT for BifferBoard
I had a go at building openWRT for my new bifferboard computer last weekend, but had trouble with it - it looked like the kernel started, but then panicked around the time it was supposed to be starting init. I suspect it was something to do with the j2ffs file system.
I decided to try to be clever and rather than fix openWRT, start from scratch with an OpenEmbedded based build. I think this was too big a step - the main problem was that OpenEmbedded failed to build because some sources from handhelds.org are not available, so I had to learn how to alter 'recipes' to build them from sources from another location. I got ipkg-utils to build this way, but then hit more trouble with openssh. I decided it was going to take me a long time, so went back to openWRT for now to try to get something working!
This time I followed the exact instructions from the bifferboard site, including downloading the specific revision of openWRT. This all compiled nicely without any errors, so the next challenge was to get it onto the board.
Flashing over the serial line seemed to take a very long time, so I had a go with network setup.
I downloaded the bifferboard utilities using:
You then need to make sure that there is a file called bzImage in your working directory and start bootp_server.py - I had to do it as root because I got some permission denied errors about opening sockets, and the easiest way to fix it was to be root - there is probably a more elegant solution though.
First I created a symbolic link to the bzImage file that openWRT had produced. Then booted the bifferboard with the serial line connected and pressed ESC to stop the boot process. Entering the tftpflash command downloaded the bzImage from the server and flashed it to the disk - much quicker than a serial line.
The kernel started, but failed with a lot of j2ffs errors - I think this is because I had my old root filesystem in the flash memory.
For the next attempt I created a symbolic link from openwrt-rdc-jffs2-64k-bifferboard.img to bzImage.
Re-flashed it again the same way and it boots!
A minor detail was that I could not log into it - the serial console was fine and both telnetd and dropbear (the SSH server) were running, but I got 'connection refused' when I tried to connect. It turned out that there was something called 'firewall' running. I deleted /etc/rc.d/S45firewall and re-booted and it worked properly - I could telnet in initially without a password, set a root password, then use ssh to connect.
Success - Now I just need to sort out the weather station software to run on it - back to the cross compiler....
running make package/symlinks in the openWRT directory adds all of the available packages to the openWRT menuconfig program.
Added the ones I know I need (libusb, python, pyusb), plus a few others that might come in useful later (atheros wireless card drivers). Then make.....and wait...
I decided to try to be clever and rather than fix openWRT, start from scratch with an OpenEmbedded based build. I think this was too big a step - the main problem was that OpenEmbedded failed to build because some sources from handhelds.org are not available, so I had to learn how to alter 'recipes' to build them from sources from another location. I got ipkg-utils to build this way, but then hit more trouble with openssh. I decided it was going to take me a long time, so went back to openWRT for now to try to get something working!
This time I followed the exact instructions from the bifferboard site, including downloading the specific revision of openWRT. This all compiled nicely without any errors, so the next challenge was to get it onto the board.
Flashing over the serial line seemed to take a very long time, so I had a go with network setup.
I downloaded the bifferboard utilities using:
svn co https://bifferboard.svn.sourceforge.net/svnroot/bifferboardThis provides a simple python based tftp server. I had to modify it to use a fixed ip address for my host computer, because the script assumes that you are using eth0 for the network interface, and I was using wireless.
You then need to make sure that there is a file called bzImage in your working directory and start bootp_server.py - I had to do it as root because I got some permission denied errors about opening sockets, and the easiest way to fix it was to be root - there is probably a more elegant solution though.
First I created a symbolic link to the bzImage file that openWRT had produced. Then booted the bifferboard with the serial line connected and pressed ESC to stop the boot process. Entering the tftpflash command downloaded the bzImage from the server and flashed it to the disk - much quicker than a serial line.
The kernel started, but failed with a lot of j2ffs errors - I think this is because I had my old root filesystem in the flash memory.
For the next attempt I created a symbolic link from openwrt-rdc-jffs2-64k-bifferboard.img to bzImage.
Re-flashed it again the same way and it boots!
A minor detail was that I could not log into it - the serial console was fine and both telnetd and dropbear (the SSH server) were running, but I got 'connection refused' when I tried to connect. It turned out that there was something called 'firewall' running. I deleted /etc/rc.d/S45firewall and re-booted and it worked properly - I could telnet in initially without a password, set a root password, then use ssh to connect.
Success - Now I just need to sort out the weather station software to run on it - back to the cross compiler....
running make package/symlinks in the openWRT directory adds all of the available packages to the openWRT menuconfig program.
Added the ones I know I need (libusb, python, pyusb), plus a few others that might come in useful later (atheros wireless card drivers). Then make.....and wait...
Saturday, 8 January 2011
A "Hacking Embedded Linux Devices" wiki site?
I have done a bit of work trying to hack cheap consumer devices to run different software - NSLU2, mediaMVP, musicPal, edimax IP Cameras etc. I have also had a look at 'off the shelf' single board computers like bifferboard.
Whenever I start I have the difficult learning curve of trying to remember how to set up a cross compiler, cross compile libraries and link them into new software.
I also think it would be useful to have a nice list of which devices have been successfully hacked and which ones are difficult.
I am thinking of setting up a site to collate such basic information, which could then link out to the more specific project sites.
Unless anyone knows of one already?
I think it would need a wiki to store most of the information, and an email discussion group. I wonder how best to do it - there are google sites and google groups, wikispaces, pbworks.
Any suggestions on the best way to do this?
Whenever I start I have the difficult learning curve of trying to remember how to set up a cross compiler, cross compile libraries and link them into new software.
I also think it would be useful to have a nice list of which devices have been successfully hacked and which ones are difficult.
I am thinking of setting up a site to collate such basic information, which could then link out to the more specific project sites.
Unless anyone knows of one already?
I think it would need a wiki to store most of the information, and an email discussion group. I wonder how best to do it - there are google sites and google groups, wikispaces, pbworks.
Any suggestions on the best way to do this?
Building openWRT
I have got a little bifferboard single board computer which I intend to use to run pywws to send data from our weather station to the weather underground site on the internet.
The bifferboard comes installed with a very small linux distribution called openWRT. It has a very small python installed, but no python USB support.
I have struggled a bit to work out how the openWRT package system works - the official wiki is a bit confused about what the current version is called (kamikaze or backfire). It implies you can add packages by downloading them from svn, but this didn't seem to do anything.
I found a useful forum post here, which seems to be the best set of instructions.
You can do
The bifferboard comes installed with a very small linux distribution called openWRT. It has a very small python installed, but no python USB support.
I have struggled a bit to work out how the openWRT package system works - the official wiki is a bit confused about what the current version is called (kamikaze or backfire). It implies you can add packages by downloading them from svn, but this didn't seem to do anything.
I found a useful forum post here, which seems to be the best set of instructions.
You can do
./scripts/feeds update -aThis downloads the list of packages, but does not do anything else.
To get the buildroot system to compile it for you you need to 'install' it using:./scripts/feeds install python./scripts/feeds install pyusbYou can then do "make menuconfig" and python and pyusb are shown to be compiled as packages "".
'make' actually compiles it.I'll update this when I work out how to add these to the firmware image...Saturday, 1 January 2011
Hacking the Parrot-DF3120 Picture Frame
Some clever people have been working on running Linux on a parrot DF-3120 Picture Frame, as described on http://sites.google.com/site/repurposelinux/df3120.
The device sounds like a little computer with a colour display, USB connection, SD card and bluetooth wireless (I would have liked a wireless lan, but never mind!). Most significantly, Amazon are selling them for less than £10, so I thought it was worth getting one to play with. It arrived very quickly so I started to play with it yesterday.
The instructions at http://sites.google.com/site/repurposelinux/df3120 fall into two distinct parts:
The device sounds like a little computer with a colour display, USB connection, SD card and bluetooth wireless (I would have liked a wireless lan, but never mind!). Most significantly, Amazon are selling them for less than £10, so I thought it was worth getting one to play with. It arrived very quickly so I started to play with it yesterday.
The instructions at http://sites.google.com/site/repurposelinux/df3120 fall into two distinct parts:
- Produce a cross compiler for the board, and use it to compile a Linux kernel, busybox and other things needed to construct a basic root filesystem. This is all done by a single clever script (minifs). It actually took a bit of doing to get it to work on my Ubuntu 10.10 system - when it fails you have to look at the log files to see why it has failed. The most noticeable thing was that I had to install gtk-doc from a source tarball because Ubuntu does not have a package for it - everything else (bison, flex etc.) were installed from ubuntu packages.
The script produced an iso image of a root filesystem. - Install a boot loader on the device. The instructions have you download and compile the u-boot boot loader for the device, and package it into a false firmware upgrade file (.plf) file. There is some black magic required to copy the file file onto the device in the right directory etc.
The boot loader installation apparently worked. If you just switch on the device without touching anything, it boots normally. If you hold down the centre and left buttons (when viewed from the front of the device) as you power it on, the screen goes blank, which is what the instructions said would happen if u-boot tries to boot linux.
In the 'black screen' mode, connecting the device to my computer creates a /dev/ttyACM0 device, which sounds promising for this being the u-boot serial console.
I have tried using a terminal emulator program to connect to the device via ttyACM0 (picocom /dev/ttyACM0, or cu -l /dev/ttyACM0). In both cases the program connects without error, but I do not see anything on the display or in the terminal.
The copy of u-boot seems to have come from openMoko, so I tried their wiki (http://wiki.openmoko.org/wiki/U-boot). It sounds like what I am doing should have worked - tried a few baud rates, but I am getting nothing, rather than gibberish - I think I'll read through the u-boot configuration that I just installed and try to work it out.....
Friday, 31 December 2010
Inside the Edimax IC-3010WG IP Camera
I have just added my photos of the inside of the Edimax IC-3010WG IP camera to my web page: http://ic-3010wg.webhop.net.
If anyone knows what 'J1' is, I would like to know!
If anyone knows what 'J1' is, I would like to know!
Thursday, 30 December 2010
Edimax IC-3010 Firmware Modification
I just discovered the 'stats' facility in blogger.com and realised that the most read posts in my blog are the ones about the Edimax IC3010 wireless ip camera. People are also looking at my web pages about it (http://ic-3010wg.webhop.net).
This got me thinking about it again so I have had a poke around inside the case - I'll post some pictures at the weekend.
There is not much to see in side - there is a mini wireless card and a single board computer. I was hoping to see a USB port, but the nearest I can find is a four pin header that is likely to be one (or more of):
- USB Port
- Serial (RS232 ish) console
- JTAG interface
I'll have to find the main system-on-chip pin-outs to try to decide what it is.
The main thing I struggle with is getting into the firmware files - I think the choice if you want to hack a device is to either connect a serial line to the console to get access to the boot loader, or modify the firmware. Modifying the firmware sounds simplest, but you run the risk of turning the device into a brick....
That is about where I got to in the summer of 2009. I am just downloading the latest firmware and source code from edimax to try to de-code the firmware building utility. While I was doing this, I discovered that someone has already done it - http://www.suborbital.org.uk/canofworms/index.php?/archives/3-Getting-telnet-access-on-an-Edimax-IC3010-webcam.html. Well done to them!
This got me thinking about it again so I have had a poke around inside the case - I'll post some pictures at the weekend.
There is not much to see in side - there is a mini wireless card and a single board computer. I was hoping to see a USB port, but the nearest I can find is a four pin header that is likely to be one (or more of):
- USB Port
- Serial (RS232 ish) console
- JTAG interface
I'll have to find the main system-on-chip pin-outs to try to decide what it is.
The main thing I struggle with is getting into the firmware files - I think the choice if you want to hack a device is to either connect a serial line to the console to get access to the boot loader, or modify the firmware. Modifying the firmware sounds simplest, but you run the risk of turning the device into a brick....
That is about where I got to in the summer of 2009. I am just downloading the latest firmware and source code from edimax to try to de-code the firmware building utility. While I was doing this, I discovered that someone has already done it - http://www.suborbital.org.uk/canofworms/index.php?/archives/3-Getting-telnet-access-on-an-Edimax-IC3010-webcam.html. Well done to them!
Wednesday, 29 December 2010
Personal Weather Station using NSLU2
While I wait for my new bifferboard to be delivered I thought I would get the weather station working and updating to weatherunderground.com using a Linksys NSLU2 that I have available.
The NSLU2 is already configured to run Linux. There are quite a few varieties of NSLU2 linux, and I think this one was called OpenSlug when I installed it, but it now seems to be called SlugOSBE.
When I first set it up there was no python package available for it, so I had to write my utility meter monitoring program using Perl (yuk!).
Fortunately there is now a repository at http://ipkg.nslu2-linux.org/feeds/optware/slugosbe/cross/unstable/ that includes python packages.
To get python running (with USB support) on the NSLU2 I had to download the following packages from the repository, and install them with ipkg install
libusb, libdb, libstdc++,ncursesw,readline,sqlite_3, zlib, python25, py25-usb
The python distribution did not put an executable in /usr/bin, so I had to do
ln -s /opt/bin/python2.5 /usr/bin/python
With all this installed I could plug the weather station into the NSLU2 (via a USB hub in my case) and pywws just worked.
I hope the bifferboard is as easy!
Tuesday, 28 December 2010
Personal Weather Station
I have just bought my wife, Sandie a personal weather station from Maplin. It is a much better one than the simple one that broke recently - it has external humidity as well as temperature, plus wind speed and direction and rain detection. Most importantly it has a USB connection so we should be able to connect it to an internet weather network so it can appear on our iGoogle home pages.
The maplin version is apparently a re-branded WH1080PC weather station - I chose this one rather than a more modern one because people have managed to get it working with Linux - I want to use a very low power single board computer to interface it, rather than a big PC running Windows.
I have chosen a bifferboard single board computer because it is cheap and has both USB and ethernet connections (I would have liked wireless lan too, but could not find something to do that, so I got the two USB port version of bifferboard, so will have a go at adding a USB wireless network interface card to it later).
My original idea was to use some software called meteoplug because you can buy a bifferboard with that already installed, so I know it would work. However, when I looked into it, it is a subscription service that is not open source - all I really want to do is obtain data from the weather station and send it to weatherunderground, which can not be that difficult.
Fortunately there is an open source project, pywws, which is compatibly with the WH-1080PC weather station.
I thought it best to get it going on a desktop linux set-up first, then try to get it working on the single board computer.
From my Ubuntu 10.10 laptop I did:
To get rid of the requirement to run the program as root you have to tell 'udev' to give other users permission.
lsusb reports the vendor and product ID of the device (1941 and 8021 respectively), but this is actually reported as a USB missile launcher rather than a weather station.
I added a file called 05-weatherstation.rules to /etc/udev/rules.d, which contained the following line:
I also added a line to /etc/group to create the 'usb' group, and made myself a member of that group.
Re-started udev with 'sudo service udev restart', and now the test works without being root (note the colon in 'MODE:=' - it is very important!
pywws seems to work suspiciously well - creating a data directory (~/weather/data) and running
The maplin version is apparently a re-branded WH1080PC weather station - I chose this one rather than a more modern one because people have managed to get it working with Linux - I want to use a very low power single board computer to interface it, rather than a big PC running Windows.
I have chosen a bifferboard single board computer because it is cheap and has both USB and ethernet connections (I would have liked wireless lan too, but could not find something to do that, so I got the two USB port version of bifferboard, so will have a go at adding a USB wireless network interface card to it later).
My original idea was to use some software called meteoplug because you can buy a bifferboard with that already installed, so I know it would work. However, when I looked into it, it is a subscription service that is not open source - all I really want to do is obtain data from the weather station and send it to weatherunderground, which can not be that difficult.
Fortunately there is an open source project, pywws, which is compatibly with the WH-1080PC weather station.
I thought it best to get it going on a desktop linux set-up first, then try to get it working on the single board computer.
From my Ubuntu 10.10 laptop I did:
svn checkout http://pywws.googlecode.com/svn/trunk/ pywws
sudo apt-get install python-usb
cd pywws
python TestWeatherStation.pyInitially this gave an error "IOError: Claim interface failed", so I tried:
sudo python TestWeatherStation.pySuccess - a string of numbers, and no errors!
To get rid of the requirement to run the program as root you have to tell 'udev' to give other users permission.
lsusb reports the vendor and product ID of the device (1941 and 8021 respectively), but this is actually reported as a USB missile launcher rather than a weather station.
I added a file called 05-weatherstation.rules to /etc/udev/rules.d, which contained the following line:
SUBSYSTEM=="usb", ATTRS{idVendor}=="1941", ATTRS{idProduct}=="8021", MODE:="0666", GROUP:="usb"
I also added a line to /etc/group to create the 'usb' group, and made myself a member of that group.
Re-started udev with 'sudo service udev restart', and now the test works without being root (note the colon in 'MODE:=' - it is very important!
pywws seems to work suspiciously well - creating a data directory (~/weather/data) and running
pywws/LogData.py -vvv ~/weather/data/
creates a file with some convincing looking numbers in it. I have created a weather station on weatherunderground, so I think the next thing to do is install the weather station outside, then try to use pywws to send data to weatherunderground.
Ubuntu Linux on a Packard Bell OneTwo
This post is a record of me getting Ubuntu Linux (Release 10.10 - Maverick) running on our new Packard Bell OneTwo all-in-one touch screen PC. I will update it as I make progress fixing the bits that have caused problems.
Installation
Installation was pretty easy - I booted of an Ubuntu 10.10 USB memory stick, and selected manual disk partitioning.
I kept the Windows 7 partition (/dev/sda1) unchanged, and did not touch the other two windows partitions (/dev/sda2 and /dev/sda3). I deleted the blank 'Data' partition (/dev/sda4) and created a new swap partition (/dev/sda5) and ext4 root partition (/dev/sda6).
Installation went without problem and the machine now dual boots Ubuntu and Windows 7.
Note that I did have a funny happening yesterday when the machine completely refused to boot after I had been using windows. I fixed this by booting off the USB memory stick and re-installing grub as described in a previous post - This has not happened again.
Audio
Worked out of the box.
Video
Ubuntu did not detect the correct screen size initially - it used a normal aspect ratio rather than wide screen, so only the centre of the screen was used, and the touch screen calibration was way out - this is because it thinks it has two displays (I don't know where the second is - there is no VGA or s-video connector on the back).
This was fixed by going to the System->Preferences->Monitor menu and un-checking the "Same image in all monitors" option.
You can then drag the main screen (labelled "Laptop") away from the second (labelled "Unknown") and set the laptop resolution to 1600x900 resolution.
This makes the screen the correct size, but every now and then it flickers twice within less than a second - it seems that the screen backlight dims momentarily before coming back to full brightness. This seems to be associated with disk activity - I don't know what causes this - it does not do it in Windows - Any suggestions for fixing it would be appreciated!.
Touch Screen
I was impressed that the touch screen worked 'out of the box' once I had fixed the monitor resolution issue.
Update 01/01/2011 - I decided to do a clean install of Ubuntu 10.10, just in case I had mangled it installing different testing packages going through bugs on launchpad. Having done the install and updating all packages using update manager, the touch screen is working perfectly - pressing the screen is the same as moving the mouse to the location and left click - Success!!!! (I hope!) - Ok, I'll give it a couple of days to make sure it doesn't beak before I say success.
Unfortunately there are problems with it. It seems that once you have done a 'drag' using the touch screen, x-windows thinks the mouse button is still pressed - moving the real mouse changes the shape of the drag region, and worse still, the real mouse button stops working. Sometimes the keyboard stops working too (both mouse and keyboard are wireless, using a USB receiver)
Investigations so far:
provides a program called 'lsinput'. Doing
lists the touch screen as:
Installing multi-touch utilities by doing:Provides amongst other things a utility called mtview. Running that lets you draw pretty patterns on the screen using multi touch - it certainly seems capable of detecting and tracking two fingers at once.Given this I think the touch screen is basically working, and it is the interface to x-windows that is the problem.I'll have to work out how x-windows drivers work to fix this though....
I will update this section as I investigate further.
Web Cam
Not tested.
Installation
Installation was pretty easy - I booted of an Ubuntu 10.10 USB memory stick, and selected manual disk partitioning.
I kept the Windows 7 partition (/dev/sda1) unchanged, and did not touch the other two windows partitions (/dev/sda2 and /dev/sda3). I deleted the blank 'Data' partition (/dev/sda4) and created a new swap partition (/dev/sda5) and ext4 root partition (/dev/sda6).
Installation went without problem and the machine now dual boots Ubuntu and Windows 7.
Note that I did have a funny happening yesterday when the machine completely refused to boot after I had been using windows. I fixed this by booting off the USB memory stick and re-installing grub as described in a previous post - This has not happened again.
Audio
Worked out of the box.
Video
Ubuntu did not detect the correct screen size initially - it used a normal aspect ratio rather than wide screen, so only the centre of the screen was used, and the touch screen calibration was way out - this is because it thinks it has two displays (I don't know where the second is - there is no VGA or s-video connector on the back).
This was fixed by going to the System->Preferences->Monitor menu and un-checking the "Same image in all monitors" option.
You can then drag the main screen (labelled "Laptop") away from the second (labelled "Unknown") and set the laptop resolution to 1600x900 resolution.
This makes the screen the correct size, but every now and then it flickers twice within less than a second - it seems that the screen backlight dims momentarily before coming back to full brightness. This seems to be associated with disk activity - I don't know what causes this - it does not do it in Windows - Any suggestions for fixing it would be appreciated!.
Touch Screen
I was impressed that the touch screen worked 'out of the box' once I had fixed the monitor resolution issue.
Update 01/01/2011 - I decided to do a clean install of Ubuntu 10.10, just in case I had mangled it installing different testing packages going through bugs on launchpad. Having done the install and updating all packages using update manager, the touch screen is working perfectly - pressing the screen is the same as moving the mouse to the location and left click - Success!!!! (I hope!) - Ok, I'll give it a couple of days to make sure it doesn't beak before I say success.
sudo apt-get install input-utils
sudo lsinput
/dev/input/event6
bustype : BUS_USB
vendor : 0x408
product : 0x3001
version : 272
name : "PixArt Imaging Inc. Optical Touc"
phys : "usb-0000:00:1a.2-1/input0"
uniq : ""
bits ev : EV_SYN EV_KEY EV_ABS EV_MSC
sudo apt-get install utouch libutouch-grail-dev utouch-grail-tools
sudo apt-get install mtdev-tools
bzr branch lp:~utouch-team/utouch/mtview
cd mtview
make
I will update this section as I investigate further.
Web Cam
Not tested.
Monday, 27 December 2010
Trouble with Ubuntu and Windows 7 dual boot
I just had a surprising problem with our new Packard Bell OneTwo. While it had apparently worked fine after installing Ubuntu on a spare partition in the hard drive, I booted it under its original Windows 7 operating system this morning to test benjamin's program with windows.
It worked fine, but after it had been switched off it absolutely refused to boot - no error messages or anything.
Went into the bios settings to take off quite boot and quick boot, but still no errors.
It booted off the Ubuntu live CD ok, and I could mount the linux root partition.
My best guess was that the boot loader was mangled, so I re-installed grub by doing:
It worked fine, but after it had been switched off it absolutely refused to boot - no error messages or anything.
Went into the bios settings to take off quite boot and quick boot, but still no errors.
It booted off the Ubuntu live CD ok, and I could mount the linux root partition.
My best guess was that the boot loader was mangled, so I re-installed grub by doing:
sudo mount /dev/sda6 /mnt [sda6 is my root partition]
grub-install --root-directory=/mnt /dev/sdaThis worked ok with no errors, and re-booting the machine gave the familiar grub boot menu. Ubuntu booted ok...we'll see if it works after I use Windows next time - I suspect it was Windows that killed the boot loader!
Sunday, 26 December 2010
Framebuffer on Ubuntu
I would like to be able to run my benjap application in a simple framebuffer rather than using X windows.
Unfortunately the 'out of the box' ubuntu desktop does not have a framebuffer (/dev/fb0) installed by default.
It looks like you need to load the fbcon and vesafb modules (found here):
Unfortunately the 'out of the box' ubuntu desktop does not have a framebuffer (/dev/fb0) installed by default.
It looks like you need to load the fbcon and vesafb modules (found here):
sudo modprobe fbcon vesafbHaving done this /dev/fb0 appears. It appears to work as long as you run applications as root (a job for later to fix that!), but I have a problem with sound, which is that even running as root I get a "no available audio device" error when pygame.mixer.init() runs. Not sure how to fix this.....
Subscribe to:
Posts (Atom)