Simple Games for Cheap Android Tablet

My Autistic son has developed quite a liking for hand held electronic devices - he likes our daughter's Nintendo DS with an animated dog on it, and apparently plays with an iPad at school.
I am not a fan of 'i' anythings because of the difficulty developing software for them, so I thought I would get him a cheap android based tablet to try.   Just need some software for it now.

I have got a Eken M009S from Amazon - it was just under £70.   I was hoping to develop some simple games using javascript to run in the web browser, which would mean they would work on any device, but the processor on the device is not up to it - there are some nice javascript game demos at http://kevs3d.co.uk/dev/ which work very well on my laptop, but are so slow on the tablet that they are not useable.

This is a shame as it would have been nice to develop something that would work on any platform - I think I will have to remember how to write 'native' (or at least java) android programs....

Access Web Server on VirtualBox Guest

I have a Windows computer where I use Oracle VirtualBox to allow me to run Ubuntu Linux.
I just found out how to access the web server on the virtual machine from the Windows Host.
You need to use the vboxmanage command that comes with virtualbox:
First do:

vboxmanage list vms

to list the names all of the virtual machines on the computer, then do:

vboxmanage modifyvm "[virtual machine name]" --natpf1 "http,tcp,,8080,,80"

You can then point your web browser on the windows host to http://localhost:8080 to see the web server on the virtual machine.

I don't know the syntax - see the VirtualBox Manual, Chapter 6 for details.

Ubuntu Linux on a Packard Bell OneTwo - Update

The Packard Bell OneTwo touch screen computer has not been used for quite a while, so I decided to start using it rather than my daughter's PC, because it uses a lot less power and is quieter.

The main issues from when I tried it before were:

• Screen flicker every now and then
• Touch screen calibration issues.

We installed Ubuntu 11.10 on it as a clean install.  Went pretty well.  The issues were:

1. Used a normal screen shape rather than wide screen.   Solved this by going to the settings / display menu and changing to a 16:10 aspect ratio screen resolution.
2. Touch Screen Calibration:  This took a while to solve because when you pressed the screen the mouse pointer appeared at a different position.  Eventually I traced it to the display settings that I had altered - Although there is only one monitor on the computer, the software detects two displays (there must be tv-out hardware in the box, but no connector outside).  I had un-clicked the 'mirror displays' option and this had resulted in the two screens being shown side by side.  I didn't think anything of it as there is no monitor connected to the second, but it confused the touch screen calibration - switching the un-used display to 'off' solved it and the touch screen calibration is fine now.
3. Multi-Touch:  The touch screen works just like a mouse.  I think it should be able to cope with multiple touches so you can do 'pinch' gestures etc. to re-size windows.  This is not working.   I tried using the mtview utility as described in the Ubuntu wiki, and it seems like multi touch is working - you can draw with two fingers at the same time, but the window manager does not seem to be using them.

Therefore, Ubuntu 11.10 worked much better 'out of the box' than the previous version I had tried - the main missing thing is multi-touch support for the touch screen.  This is an important one to fix for me because our son is VERY short sighted, and looks very closely at the screen, often pressing his forehead against it.  Once he has done that, the touch screen does not work because it only detects one press at a time!   I am going to have to learn how x windows input devices work to debug this I think.....

Breaking out of a Frozen SSH Terminal Session

For a long time I have found that if the internet connection is broken during a ssh terminal session that the terminal hangs and the keyboard is un-responsive - CTRL-C, CTRL-D etc. do not do anything.  I have always just cursed and closed the terminal window.

I just found that ~. will break out of the frozen session, giving you access back to the terminal.   ~? gives a list of other escape commands, but I am not sure what I would use those for.

Using Android Phone as a Modem

For quite a while I had been able to use my Samsung Galaxy Apollo phone as a modem - I just plugged it into my work laptop (running Windows XP) and it appeared as a modem, which I could use to connect to the internet nicely.

Then I tried (and failed) to update my phone using Samsung Kies and it stopped working.   I remembered as part of trying to get Kies to work I found an obscure code to type on the keypad to switch the phone between 'PDA' mode and 'Modem' mode.   I have just managed to switch it back to modem mode, and this has got my modem working again!

To do this I had to type *#7284# on the phone dialer keypad.  This brought up a menu where I selected the USB option to be 'Modem'

Now plugging it into my computer shows it up as a Samsung Mobile Modem.   I have the phone number set to *99***1#, which seems to work ok.  

I disabled the internal modem in the laptop because sometimes windows tried to dial that one instead, which is not connected to anything.

Samsung Colour Laser Printer and Ubuntu Linux

Note:  You might find a more recent post more useful.

I have got sufficiently sick of my Epson inkjet printer suffering from clogged print heads, and noticed how cheap colour laser printers have become, so decided to give one a try.

I settled for a Samsung CLP-325W (w=wireless) colour laser printer.   The reason for this is that Samsung provide linux drivers for it, and I had an old Samsung laser printer a few years ago that worked nicely with linux.

To start with, the CD that came with the printer would not mount on my daughter's Ubuntu 11.04 machine - no idea why, so we downloaded the Samsung unified driver for linux from the Samsung UK support web site.

We installed the driver by running install.sh as root, plugged in the printer and it detected ok and seems to work.  We had to select the "Samsung CLP-300 Series (SPL-C)" driver, because the recommended "CLP-325 foomatic...." one only printed black and white.

So far, so good!

The fun came when I tried to set up wireless printing so I could print to it from my laptop without my daughter's computer being switched on.   The first attempt was to repeat what I had done for the other computer, and install the Samsung unified driver on my laptop.  Unfortunately it refused to detect the printer, even when it was plugged into the USB port on the laptop.   I never found out why and tried a different approach.

I plugged the printer into my ethernet network and re-started it.  I found out that to print the diagnostics you hold the 'cancel' button (with the triangle symbol on it) for a few seconds.  This printed  out a network diagnostic page that said the network type was 'dhcp' and the ip address was 192.0.0.192.   This is odd because my router, which would act as dhcp server should give out addresses 192.168.1.*, so I assumed that it was lying about using dhcp and had a static default address.
Re-configured laptop to use the 192.0.0.* subnet and tried looking for the printer - tried a web browser, ping and nmap, and no sign of it.

Had a look at my router's dhcp list, and realised that the printer had been assigned an ip address by the router, which was not the one on the network diagnostics!!!!!
Re-printed the network diagnostics and got the correct ip address - don't know what happened there.

Pointed a web browser at the printer's IP address and got a nice status web page, with a 'login' option at the top right hand corner of the screen (once I had scrolled across because I couldn't see it on my laptop screen for some reason....).   Had to search on the internet for the default login credentials - found out that it is user name -  "admin", password - "sec00000".

Once logged in, I could set up the network to use a static IP address (so I always know where it is, and connect it to my wireless network).

Disconnected the lan cable from the printer, and sure enough, I can see the printer from my laptop.  Used the Samsung printer configuration tool to set it up as a network printer on the laptop and....it works!!!

phew!

CodeIgniter / Bonfire

I have been progressing an automated way of generating maps from openstreetmap data, allowing layers to be selected, including contours and hill shading etc. (https://github.com/jones139/disrend).

I want a web front end to allow people to generate maps using this renderer.   When I did this for my townguide program I just built a very simple php based web interface.
Ideally I would like things like user authentication though, which would mean writing a lot of code to go with it.
The CodeIgniter framework goes some way to this by providing an interface to sessions etc, but there is still quite a bit of boilerplate code to write to produce the database interface.
The CodeIgniter / Bonfire application seems to solve this by providing:

1. Roles based authentication.
2. Modular structure (this was an issue I had with codeigniter - you ended up with code scattered in different places).
3. A really neat module builder function that produces a basic module with database interface for you to build on.

I am just trying to get the hang of using bonfire now - I have had a few issues with installing modules based on the module builder code - will write up how to do this once I have it working...

Building Debian Packages

I worked out how to make simple debian packages by hand as described in a previous post.
Trying to build other software into packages is still rather difficult...
I found a really good, simple tutorial on how to use the debian tools to do this more easily at:
http://www.debian.org/doc/packaging-manuals/packaging-tutorial/packaging-tutorial.pdf.

Ubuntu packages and repositories

Creating Ubuntu (debian) packages always seems a bit fiddly....At the moment I can do it, so thought I'd better write down quickly how I did it.

Packages

• Create a directory and put the files you want to install into a directory tree under it (e.g. if you create a directory called [pkgname], files that you want in /usr/local/bin go in [pkgname]/usr/local/bin.
• Create a directory called [pkgname]/DEBIAN and put a file in it called 'control'.  This can be as simple as:

Package: grahamstestdeb
Version: 001
Architecture: i386
Depends:
Maintainer: Graham Jones (grahamjones139@gmail.com)
Description: Test Debian Package

• Create a file called 'postinst' in DEBIAN.  This will be executed when the package is installed. This can be

• #!/bin/sh
• echo "postinst works!!!!"

• Create the .deb package by doing

dpkg-deb --build [pkgname]

•  Copy your new [pkgname].deb file into a new folder called repository.
• Run 'sudo dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz' in the repostitory directory.
• Upload the repository directory to your web server:

ncftpput -v -R -u maps3.org.uk -p ftp.maps3.org.uk public_html/apt/ *

• Add the following to /etc/apt/sources.lst 

deb http://apt.maps3.org.uk / 

• Run sudo apt-get update
• Run sudo apt-cache search [pkgname] - should find your package.
• Run sudo apt-get install [pkgname] - should run your postinst script.

For better instructions see: http://odzangba.wordpress.com/2006/10/13/how-to-build-local-apt-repositories/#comment-7304.

Rendering OpenStreetMap Data using Carto and Mapnik

Mapnik is often used to render OpenStreetMap (OSM) data.   The map style (line colours and sizes, icons etc.) are defined by a style file.  There is a very detailed one provided with OpenStreetMap's mapnik tools.   Unfortunately that level of detail means that the style definition is very complicated and it is hard to follow how it works.
A tool called carto is available which is a pre-processor that will take a simpler format of style file, and convert it into the mapnik file format.   I have been experimenting with trying to render OSM data using styles written for carto.   I had been hoping to use a nice graphical editor with map preview capability called tilemill, but tilemill does not work with postgresql datasources, which are necessary for OSM rendering.   Instead, I installed tilemill and used the version of carto that is included with it, but did not use the tilemill editor.

Carto Layer Definitions
Rendering data with carto needs you to do two things - define the data you want to plot on your map ('Layers'), and define how you want the data displayed ('Styles').

The datasource definitions go in a .mml file that looks something like this:

{
"_center":{"lat":36.870832154494,"lon":-113.79638671427,"zoom":5},
"_format":"png",
"srs":"+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs",
"Stylesheet":["style2.mss",
"areas2.mss",
"roads2.mss",
"labels2.mss",
"natural2.mss",
"contours2.mss",
"POIs2.mss"],

"Layer":[
  {"id":"coastline",
   "name":"coastline",
   "srs":"+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over",
   "class":"",
   "geometry":"polygon",
   "Datasource":{"file":"/home/graham/OSM/data/mapdata/world_boundaries/processed_p.shp","type":"shape"}},

 {"id":"landuse",
  "name":"landuse",
  "srs":"+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs",
  "class":"",
  "geometry":"area",
  "Datasource": {
               "type":"postgis",
      "dbname":"kefalonia",
      "user":"www",
      "password":"1234",
      "host":"localhost",
      "port":"",
      "table":"(select way,landuse,name,\"name:en\" from planet_osm_polygon where landuse is not null) as landuse"
               }
  }
]
}

The first few lines define the projection of the output map, and the stylesheets that are to be used to define how to render the data.   The bulk of the file is layer definitions.   In this example two layers are defined.  One is a shapefile containing the coastlines (the same one that is used by the normal OSM style).   The second layer extracts all areas tagged with 'landuse=xxx'.

We have not told mapnik how to plot the data though - this is done in the style files.  The ones relevant to this example are "style2.mss" and "areas2.mss".  Style2.mss contains:

@land: #e0e0c0;
@water: #C0E0F8;
@waterline: #8CE;
Map {
  background-color:@water;
}
#coastline::outline {
  line-color:@waterline;
  line-width:1.6;
}
#coastline::fill {
  polygon-fill:@land;
  polygon-gamma:0.75;
}

Here the things starting '@' are variables which allow you to define colours etc.  at the top of the file, then use them several times throughout it, making future maintenance easier.   In this example we define the background colour for the map, fill it in blue to represent water.   We then draw a line around the costline, and fill in the land with a separate colour.

The areas2.mss style is slightly more complicated:

#landuse {
   [landuse="residential"]
   {
      polygon-fill: #b0b0b0;
   }
   [landuse="industrial"]
   {
      polygon-fill: #a0a0a0;
   }
   [landuse="forest"]
   {
      polygon-fill: #a0d0a0;
   }
}

In this example the [] expressions are filters, which allows us to define different fill colours depending on the landuse tag.

These are just extracts - the actual styles I am using are getting progressively more and more complicated.   You can see the files, and a modified version of the generate_image.py script provided by OSM, which takes a carto .mml file as input in my svn repository (soon to be copied to github..)

Using these styles, I have updated my kefalonia map to give this:

and this:

These styles are nowhere near as clever as the standard OSM ones.  In particular I have not tried to devine different styles fo rdifferent zoom levels  instead I have two sets of stylesheets for two different zoom levels - I think in the end I will have there - one for a 'country level', another for a 'town level', and a third for higher zoom levels.

There are also a few issues - the line styles do not look as nice as the OSM ones - I do not know why...and there is a problem with the road names where sometimes other roads crossing over each other obliterate the labels.   THe reason is that I draw the road and the label at the same time - it will probably look better if I do the labels after all roads have been drawn.   Will hav eto work out how to do that!

Installation of Carto Mapnik Style File Builder on Ubuntu

I find mapnik style files a bit clumsy to edit - lots of repetition that makes them hard to follow.
There are (at least) two utilities available that may make style file generation simpler.  These are Cascadenik and Carto.
It appears that Carto is being more actively developed at the moment, so I thought I would look at that first. Carto also reports to be much faster than Cascadenink.
The problem is that Carto is new-fangled and uses a javascript based language called node.js.   Despite my dislike of javascript, I thought I would persevere.  This is how I got it working on my Ubuntu 10.10 system:

• Downloaded node.js version 0.4.7 from http://nodejs.org, and extracted it into /usr/local/node-v0.4.7.  Built with ./configure; make; make install.
• Installed the node package manager npm using: curl http://npmjs.org/install.sh | sh
• Installed carto dependencies:
• npm install get     [used to be node-get, but the module has changed name]
• npm install step
• npm install srs
• npm install underscore
• npm install zipfile (actually did this twice because one of the document build actions failed on the first attempt...).
• Downloaded carto using:  git clone https://github.com/mapbox/carto.git
• No compilation necessary (it is javascript).  Executing cd carto; ./bin/carto gave no nasty errors, just "no input files", which looks promising.

I just need to work out how to use it now.

I have had a quick look at TileMill, which is a web based editor for carto stylesheets - it is very impressive - you can change the stylesheet and preview the map in real time.   The only problem that I have found with tilemill is that it does not seem to work with a postgresql database as the data source, which is where my OSM data is stored.  But at least I should get the idea about the structure of the files using that tool.

Kefalonia Map

We are going to Kefalonia on our holidays this year, and I usually like to do a bit of OSM mapping when I go somewhere new, and may do some 'armchair mapping' before I go to get a head start, so I thought I would have a look at how well mapped it is.
The answer is not very - there are the main roads, and a few minor roads and footpaths, but not much else - this is a shame because I like to know where things like supermarkets and banks are.  Unfortunately there is not a lot of information available for remote mapping - very few GPS traces, and Bing Imagery does not cover the area I am interested in.
So instead I decided to make myself a map of the island showing what is there (in OSM anyway).  Because there is not a lot of detail, I want to have contours so I can tell where the mountains are, and also highlight what points of interest there are at lower zoom levels than the standard OSM style.   These are my notes on how I produced the map, so I can remember next time I need to do it.

• The map bounding box was found by looking at the (lon,lat)  readouts on http://www.informationfreeway.org to check the bottom left and top right of the island.  They are (20.33, 38.03) and (20.82,38.50).
• I downloaded the OSM data from the mapquest XAPI server using: 

wget http://open.mapquestapi.com/xapi/api/0.6/map?bbox=20.33,38.03,20.82,38.50 -O kefalonia.osm

• Imported the data into postgresql database using:

osm2pgsql -s -S default.style -d kefalonia -m kefalonia.osm

• Downloaded the SRTM elevation data covering the island from http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Eurasia/ - only needed one file (N38E020.hgt.zip), because it is a small island.
• Generated the contours and imported them into my postgresql kefalonia database as described in http://wiki.openstreetmap.org/wiki/Contours. 
• Modified the standard osm mapnik style file as described in http://wiki.openstreetmap.org/wiki/Contours, using the 'PostGIS' method.
• Generated the map image using the generate_image.py file from the OSM mapnik archive.  I initially had some trouble with no roads appearing, but solved this by re-compiling osm2pgsql from the latest sources - I do not know what the problem was.
• Having done that, I get this map image:

Kefalonia Map - Version 1 (25 April 2011)

Next I wanted to add a 1km grid to the map to make it easier to judge scale..."are we nearly there yet?" etc.
I did this by adopting the generate_graticule script from mapnik-utils to produce a grid on the google spherical mercator projection (=SRS 900913).  I must admit to not really knowing how this works, but my updated version (generate_grid.py) is in my SVN repository.   I added this to the map by creating an extra xml include file to define the grid layer and style (inc/grid.xml.inc), adding this to layers.xml.inc, and finally defining the layer in osm.xml.

The result (Version 2) is shown below:

Kefalonia Map Version 2 (26 April 2011)

I created another include file for holiday based points of interests (holiday_pois.xml.inc), added it to layers.xml.inc and again added the actual layer definition to osm.xml.
This gave the following map:

Kefalonia Map Version 3 (26 April 2011)

The final thing I needed was the English spelling adding to the Greek text labels.   I did this by adding name:en to the default.style file used by osm2pgsql and re-importing the OSM data into the database.
The challenge was updating the mapnik style file to use the name:en column - the OSM style files are very complicated with lots of SQL SELECT statements to extract the data out of the database.  I was tempted to do a global replace of 'name' with 'name:en'...but 'name' appears all over the place.
I decided the main thing I wanted with both writing styles were place names and I found the code for them in inc/layer-placenames.xml.inc.
I modified the SELECT statement from a simple 'name' to ,'"name" || ' (' || "name:en" || ')' as name'.  This gives me the greek spelling followed by the english spelling in brackets using the postgresql '||' string concatenation operator.  The result is here:

Kefalonia Map Version 4 (27 April 2011)

I think that is about the finished product - I will have to try printing it to see how it looks, then may need to adjust the image size to suit printing....then do some mapping to add some more POIs - Sacla in the south east of the island should be a lot better by the middle of June....

Problems with GoogleCode SVN

I use GoogleCode (http://code.google.com) for my little open source projects, and it usually works quite nicely.
I was just trying to set up one of my repositories in writable mode on a new computer, and "svn ci" (checkin) gave 500-Internal Server errors, which was not very helpful.
Eventually I realised that although I had done the "svn checkout" with a --username=", when I was doing the checking it was using my computer username, rather than my google one...so presumably the authentication failed.
I had to do "svn ci --username=", then it worked ok.   I don't know why!

Trouble with mysql, apache, php5 and codeIgniter

I am working on a new web interface to townguide based on php so it can be used on a low cost web hosting service (rendering will be done by a separate computer).
I am going to use the codeIgniter web framework because it seems simple, and is simiar to django, which I was just starting to get used to.....
Alas getting database access working on my laptop for development was difficult.
I have apache, php5 and mysql all working, but whenever I tried to load the database module of codeIgniter I got a 500 Internal Server error, but no errors in either the apache or codeigniter error logs.   This made debugging difficult.
After much searching on the internet I realised it was because I needed to install libapache2-mod-auth-mysql as well as php5-mysql.
Doing sudo apt-get install libapache2-mod-auth-mysql and re-starting apache made the following trivial mysql db connection work:

$conn = mysql_connect('localhost', 'www', '1234');
if($conn) {
  echo 'Finally connected!!!';
 } else {
  die('Still cannot connect' . mysql_error());
 }
?> 

A bit late now, but I think this will also have solved the problem connecting to the database using codeigniter....

Disk Crash on Linux

Well, I thought that the introduction of the ext3 (and higher) file systems had stopped all of the problems of disk crashes and mangled filesystems on linux.
But, after a few years of reliable operation the root filesystem on my home server (my old Fujitsu Siemens laptop) seems to have mangled.
The symptoms were....not booting - you get the 'Starting up......' message during boot then the system appears to hang.  But if you listen carefully you hear that the disk is actually doing something.  So I left it for a few hours.....
When I came back to it there was a 'failed to mount /dev/sda1' type error message, with an option to skip mounting or manually fix it.
I went for manually fixing it, because not mounting the root filesystem will not get me very far.
This dropped me into a single user shell.
I ran fsck and it said that an Inode has illegal blocks.  I selected the option to clear them...and again said yes when it asked me again.
It then said it was restarting e2fsck from the beginning, and spent quite a few minutes checking....

Then at Pass 2 fsck said it had found a deleted or unused inode.  Again I said 'y' to the 'Clear?' question....
Then lots of offers to fix things (to the extent that I just held my finger on the 'y' key...)

Then fsck announced that it was complete and I should re-boot linux.   Re-ran fsck and it announced that /dev/sda1 was clean, so re-booted.....but booting is taking a suspiciously long time.....like it has been trying for 10 minutes and hasn't got past the boot up splash screen...I'm going to need a plan B...

Well, I don't know what the problem is.   Tried booting off a USB memory stick, and the disk checks ok and mounts, but the boot process just hangs.   I decided I had spent too long on this so it is currently installing Ubuntu 10.10 on the disk instead, so can't try any more diagnosis!

Emacs Drupal Mode

I am trying to learn how to write Drupal modules for a new map rendering on demand web service (a bit like townguide, but distributed with a separate front end and rendering servers).

First thing is to get emacs23 working with php and drupal modes - the instructions here give me obscure errors about c-lang-def only being used in a file.   After a bit of hunting on the internet I found it is a problem with Emacs V23, and a simple work around is to put the php-mode.el and drupal-mode.el files in ~/.emacs.d, then add the following to ~/.emacs:

; make sure the target directory is on your load-path

(add-to-list 'load-path "~/.emacs.d")

(autoload 'php-mode "php-mode" "Major mode for editing php code" t)

(autoload 'drupal-mode "drupal-mode" "Major mode for editing drupal php " t)

(add-to-list 'auto-mode-alist '("\\.\\(module\\|test\\|install\\|theme\\)$" . drupal-mode))

  (add-to-list 'auto-mode-alist '("\\.\\(php\\|inc\\)$" . php-mode))

  (add-to-list 'auto-mode-alist '("\\.info" . conf-windows-mode))

With that in my .emacs file, loading a .php or .module file starts php or drupal mode - success!

Easier Map Creation with OpenStreetMap Data

I have been giving some thought to making the process of creating maps form OpenStreetMap data easier.  The problem is that to use the mapnik map renderer, you need to install quite a lot of tools and set different configuration files before you can even start to create a map.
This means you have to be pretty keen to persevere enough to get to an end result.
I am thinking of creating a service to allow you to bypass the tool installing bits, and concentrate on the setting up of the map configuration, so you can get to a result quicker - see http://wiki.openstreetmap.org/wiki/User:Grahamjones#Speciality%20Maps.

Quite a few of the responses that I got to that proposal were about the lack of good documentation to get people started on the process.  I have had a go at addressing this by putting together a simple overview presentation to go through the main concepts and tools that you need to generate a map using mapnik.   The presentation is on slideshare.net, and should be visible below:

Rendering OpenStreetMap Data using Mapnik

View more presentations from jones139

Mounting FTP Directory as a Filesystem

I just discovered a really useful trick to mount the directory structure of a remote FTP server as a directory on a linux computer.   You use a program called curlftps.   On Ubuntu do:

sudo apt-get install curlftps

The directories on the remote ftp server are then mounted using:

 curlftpfs ftp://[username]:[password]@[server address] [local directory mount point] 

This works fine (you do not even need to be root to do it), but the password will be present if you do 'ps -ef | grep ftp'.
A solution is to create a .netrc file in the /root directory containing:

machine [server address]

login [username]
password [password]

You can then add the file system to /etc/fstab by adding the following line to /etc/fstab:

curlftpfs#[server address] [local directory mount point] fuse allow_other,rw,user,noauto 0 0

You can then just do

sudo mount [local directory mount point]

and you get the same effect with the server password invisible.

OpenWRT Based Utility Meter Monitor

I have made some progress porting my MeterServ application (http://meterserv.webhop.net) to openWRT.   I have managed to work out how the openWRT makefile system works enough to add it to the build system and produce an openWRT package for the MeterServ application.
To do the improvements that I want to make to the application though, it needs more work.  In particular I don't like Perl, so want to convert the web interface part of it to Python.   Rather than go for a completely DIY web framework, I have adopted webpy (http://webpy.org) - this seems like a simplified version of django - you define URLs that you want the application to work with, then point it to a python class that does the processing associated with the user requesting a particular URL.
I have started a seperate google code project for meterserv, because a couple of people expressed an interest in it.  I have put the latest version of the software in its repository too - see http://code.google.com/p/meterserv.

Utility Meter Monitor

I have had a simple utility meter monitoring system working in my parents and my house for a few years.  It uses an NSLU2 single board computer connected to a simple USB input/output interface, as described at http://meterserv.webhop.net.
It worked fine for at least 3 years, but started to misbehave recently.  I have not been able to determine if it is a hardware or software fault - I suspect that I may have worn out the USB memory sticks that I used for the root filesystems.
We have gone for a hardware change to see if that is the problem, but I want to develop the software a bit more too.
Future developments include:

• Add support for cheap electricity meters (e.g. OWL USB energy monitors).
• Add support for temperature sensing.
• Better web interface (the current one uses perl based CGI scripts, which are awful to maintain).
• A simple (end user friendly) software upgrade process.

I want to get rid of the perl cgi scripts because I don't really speak perl, so they are hard to follow - will change them for an ajax based front end with python server side scripts.

The first stage is to get a sofware development environment working.  The original version used the nslu2-linux slugos operating system.  This has been changed quite a bit since I used it, so I decided that as I will need to do a complete re-build, I will standardise and use OpenWRT for all my little embedded system projects.

Compiling OpenWRT for the nslu2 was nice and easy - in the menuconfig just select the processor as an ixp4xxx and the sub-type as nslu2.

Edit target/linux/ixp4xxx/base_files/etc/config/network to give a default network configuration that will work on your system (the openwrt default ip address is 192.168.1.1, which clashes with my router....).

This creates a file called bin/ixp4xxx/openwrt-nslu2-squashfs.bin which can be flashed onto the nslu2 as follows:

• Put the NSLU2 into upgrade mode by switching off, then powering on while holding in the reset button for about 10 seconds.   When the status LED changes colour (at about 10 sec), release the reset button.  The status LED now flashes different colours (very subtly different colours if you are colour blind like me!).
• Use upslug2 (sudo apt-get install upslug2 on Ubuntu) to upgrade the NSLU2 using:  upslug2 -d wlan0 --image=openwrt-nslu2-squashfs.bin
• The NSLU2 will re-boot, but then take a while (a few minutes) to set up a writeable overlay file system before being accessible over the network.

This showed that I could build software for the NSLU2 using openwrt, but now I need to port meterserv to OpenWRT, which will be a separate post....

OpenWRT on Linksys NSLU2

I have had a couple of NSLU2's running a little application to monitor utility meter readings for a few years now (meterserv).   It worked fine until the autumn when my Dad's one stopped recording readings.  I have not been able to find out why - it looks ok.  It may be that the interface card has broken, or I do wonder if I have worn out the USB flash drive that I am using for the root filesystem.

In my inability to find a fault, I decided to do a clean install on a usb disk to rule out software and flash drive problems.   They had been using OpenSlug (now called SlugOsBE) from nslu2-linux, but it is now very out of date, because I have not updated them for a few years.  This means that all the links to package directories are broken etc.

Given the recent success in building OpenWRT for the bifferboard, I decided to use OpenWRT for the re-build.   These are my notes so I can do it again next time....

• In the MenuConfig system, set the target system to 'Intel IXP4xx', sub target to 'Generic' and target profile to 'Linksys NSLU2'.
• Because this is not my new router, I do not want it to use the default IP address of 192.168.1.1.  This can be changed in the 'Image Configuration' bit of the configuration menu.
• I would like to re-code some of the meterserv software in python, so I need python and pyusb installed in the flash image - these were selected from the 'Languages' section of the openwrt menu.
• Make builds a firmware image in bin/ixp4xxx called openwrt-nslu2-squashfs.bin, which is an 8MB image - there is a 16MB one too, but I am not convinced that my nslu2 has that much flash memory.
• You have to put the nslu2 into 'recovery mode'  by holding the re-set button in while you power up the machine - you have to release the reset button as soon as the status LED changes colour, after about 10 seconds.  The status LED then flashes a couple of different colours.
• Flash the new firmware image onto the NSLU2 using upslug2 (sudo apt-get install upslug2 on ubuntu) - the command line was sudo upslug2 --target 00:14:bf:64:df:be --image openwrt-nslu2-squashfs.bin.
• The machine now appears on the network with the right IP address.  Telnet into it and change the root password, then connect using SSH instead.

The only problem is that when I type 'python' I get an error "python: can't resolve symbol 'BC'".  So now I need to work out which package is missing - looks like openWRT missed a dependency, but I don't know which one!

I have had a bit more of a look at it.  There are two surprising things.  The first is that if I write a simple 'hello world' script and execute it, it works, so it is just interactive python that is broken.

The second is that doing "ldd /usr/bin/python" on the nslu2 and on the bifferboard give very similar results, except that the bifferboard one is linked against libgcc and libc, but the nslu2 one is linked against libc.

I think this is telling me something, but I am not sure what....

Mapnik Map Rendering

Someone asked a question on the osm-gb mailing list about viewing coverage of waterways - canals and navigable rivers.
We don't seem to have one, so I created one on my http://maps.webhop.net web site.
The trouble with that site is that although it is running on a nice powerful computer (my old laptop), it is behind my domestic broadband service from Virgin Media.   Although this service has a nice fast download speed, the upload speed is very poor ~900 kbps compared to 9600 kbps download.   This makes the web site very slow from the outside world.

I have a little virtual server provided by CloudNext - it is nowhere near as powerful as my own server, and has much less disk space, but at least it has a fast internet connection.

I loaded a british isles extract of OpenStreetMap data into the database - it took a very long time - maybe getting on for 12 hours (not sure when it finished!) - my server takes about 2 hours.   Performance rendering maps was not too bad though - rendering down to zoom level 14 took about 2 hours, which is respectable.

Just need to get the database updating to keep it up to date, and re-generate changed tiles now....
You can see the difference at http://maps.webhop.net/canals (my home server) compared to http://maps2.webhop.net/canals (the virtual server)

Personal Weather Station using BifferBoard

I have managed to re-compile openWRT to run on the bifferboard, and include python, libusb and pyusb in the main root directory.   I also included the wget utility, because I thought it would come in useful, plus a few other potential future extensions (lighttpd web server and wireless LAN support).
This fits on the bifferboard onboard flash memory, with about 1MB to spare.

I created a /home/weather directory and extracted a recent pywws tar archive into it.   I was very pleased that running TestWeatherStation.py produced a table of numbers as expected, so it looks as though the bifferboard is talking to the weather station ok.
Here it is:

Set up the initial weather database by doing

cd /home/weather 

python pywws/pywws/LogData.py -vvv /home/weather/data

This creates /home/weather/data/weather.ini and a directory /home/weather/data/raw which contains the raw data.

Run Hourly.py for the first time to process the data, and set up the weather.ini file ready for customisation:

pywws/Hourly.py /home/weather/data

Customise /home/weather/data/weather.ini to do what you want it to do - in my case update weatherunderground....

All is looking promising here - seems to send the update ok on an hourly basis.  There are two problems though:

• There is not much disk space left now I have added all of January's weather data (256kB ish)
• I am concerned that I will wreck the flash chip on the bifferboard with all this writing.

One option is to store the weather data in a ram disk and just accept that after a power-off it will need to re-initialise itself - not sure how pywws will cope with this - I will need to load some default weather.ini file from flash every time it boots, then let it update itself as best it can.

As a quick alternative I have found an old 256MB SD card and created an ext3 filesystem on it.  This detects as /dev/sda1 when plugged into the bifferboard.   I have modified /etc/rc.local to mount this as /home/weather.  This should solve my full filesystem problems.  [Note:  I used an ext3 filesystem because my openWRT build could not detect ext2 - this was a bit of a surprise because I thought you got ext2 support free with every Linux kernel...].
Then it is just a matter of a cron job to do the hourly updates - crontab -e, then add:

13 * * * *       python /home/weather/pywws/Hourly.py -v /home/weather/data >> /home/weather/Hourly.log 2>&1

If it works, you should continue to see the weather in our back garden at: http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IHARTLEP2.

Mistake 1:  If you put an SD card into the dual USB bifferboard, it disables one of the USB ports.  This is bad if it was the one the weather station was connected to....

Mistake 2:  Believe people when they say the clock on a board is no good.  The Bifferboard clock is no good.  Not only does it loose its time when powered off, but it drifts - by a few minutes an hour.  This meant that pywws got confused about when it should update weatherunderground, because it thought time had suddenly gone back to 2009....   To avoid this I compiled ntpclient as a package and downloaded it onto the bifferboard.   I now have it set to be called once on boot, then every 10 minutes by a cron job.  This should keep it somewhere close - just hope the time server owner doesn't mind - If I get a complaint I will have to set up my own time server...

Building openWRT for BifferBoard

I had a go at building openWRT for my new bifferboard computer last weekend, but had trouble with it - it looked like the kernel started, but then panicked around the time it was supposed to be starting init. I suspect it was something to do with the j2ffs file system.

I decided to try to be clever and rather than fix openWRT, start from scratch with an OpenEmbedded based build. I think this was too big a step - the main problem was that OpenEmbedded failed to build because some sources from handhelds.org are not available, so I had to learn how to alter 'recipes' to build them from sources from another location. I got ipkg-utils to build this way, but then hit more trouble with openssh. I decided it was going to take me a long time, so went back to openWRT for now to try to get something working!

This time I followed the exact instructions from the bifferboard site, including downloading the specific revision of openWRT.   This all compiled nicely without any errors, so the next challenge was to get it onto the board.

Flashing over the serial line seemed to take a very long time, so I had a go with network setup.
I downloaded the bifferboard utilities using:

svn co https://bifferboard.svn.sourceforge.net/svnroot/bifferboard

This provides a simple python based tftp server.   I had to modify it to use a fixed ip address for my host computer, because the script assumes that you are using eth0 for the network interface, and I was using wireless.
You then need to make sure that there is a file called bzImage in your working directory and start bootp_server.py - I had to do it as root because I got some permission denied errors about opening sockets, and the easiest way to fix it was to be root - there is probably a more elegant solution though.

First I created a symbolic link to the bzImage file that openWRT had produced.   Then booted the bifferboard with the serial line connected and pressed ESC to stop the boot process.   Entering the tftpflash command downloaded the bzImage from the server and flashed it to the disk - much quicker than a serial line.
The kernel started, but failed with a lot of j2ffs errors - I think this is because I had my old root filesystem in the flash memory.
For the next attempt I created a symbolic link from openwrt-rdc-jffs2-64k-bifferboard.img to bzImage.
Re-flashed it again the same way and it boots!

A minor detail was that I could not log into it - the serial console was fine and both telnetd and dropbear (the SSH server) were running, but I got 'connection refused' when I tried to connect.   It turned out that there was something called 'firewall' running.   I deleted /etc/rc.d/S45firewall and re-booted and it worked properly - I could telnet in initially without a password, set a root password, then use ssh to connect.
Success - Now I just need to sort out the weather station software to run on it - back to the cross compiler....

running make package/symlinks in the openWRT directory adds all of the available packages to the openWRT menuconfig program.
Added the ones I know I need (libusb, python, pyusb), plus a few others that might come in useful later (atheros wireless card drivers).  Then make.....and wait...

A "Hacking Embedded Linux Devices" wiki site?

I have done a bit of work trying to hack cheap consumer devices to run different software - NSLU2, mediaMVP, musicPal, edimax IP Cameras etc.   I have also had a look at 'off the shelf' single board computers like bifferboard.
Whenever I start I have the difficult learning curve of trying to remember how to set up a cross compiler, cross compile libraries and link them into new software.
I also think it would be useful to have a nice list of which devices have been successfully hacked and which ones are difficult.
I am thinking of setting up a site to collate such basic information, which could then link out to the more specific project sites.
Unless anyone knows of one already?

I think it would need a wiki to store most of the information, and an email discussion group.  I wonder how best to do it - there are google sites and google groups, wikispaces, pbworks.

Any suggestions on the best way to do this?

Building openWRT

I have got a little bifferboard single board computer which I intend to use to run pywws to send data from our weather station to the weather underground site on the internet.

The bifferboard comes installed with a very small linux distribution called openWRT.   It has a very small python installed, but no python USB support.

I have struggled a bit to work out how the openWRT package system works - the official wiki is a bit confused about what the current version is called (kamikaze or backfire).  It implies you can add packages by downloading them from svn, but this didn't seem to do anything.
I found a useful forum post here, which seems to be the best set of instructions.
You can do

./scripts/feeds update -a

This downloads the list of packages, but does not do anything else.

To get the buildroot system to compile it for you you need to 'install' it using:

./scripts/feeds install python 
./scripts/feeds install pyusb
You can then do "make menuconfig" and python and pyusb are shown to be compiled as packages "".

'make' actually compiles it.

I'll update this when I work out how to add these to the firmware image...

Hacking the Parrot-DF3120 Picture Frame

Some clever people have been working on running Linux on a parrot DF-3120 Picture Frame, as described on http://sites.google.com/site/repurposelinux/df3120.  
The device sounds like a little computer with a colour display, USB connection, SD card and bluetooth wireless (I would have liked a wireless lan, but never mind!).   Most significantly, Amazon are selling them for less than £10, so I thought it was worth getting one to play with.   It arrived very quickly so I started to play with it yesterday.

The instructions at http://sites.google.com/site/repurposelinux/df3120 fall into two distinct parts:

1. Produce a cross compiler for the board, and use it to compile a Linux kernel, busybox and other things needed to construct a basic root filesystem.  This is all done by a single clever script (minifs).   It actually took a bit of doing to get it to work on my Ubuntu 10.10 system - when it fails you have to look at the log files to see why it has failed.   The most noticeable thing was that I had to install gtk-doc from a source tarball because Ubuntu does not have a package for it - everything else (bison, flex etc.) were installed from ubuntu packages.  
   The script produced an iso image of a root filesystem.
2. Install a boot loader on the device.   The instructions have you download and compile the u-boot boot loader for the device, and package it into  a false firmware upgrade file (.plf) file.   There is some black magic required to copy the file file onto the device in the right directory etc.

The boot loader installation apparently worked.  If you just switch on the device without touching anything, it boots normally.  If you hold down the centre and left buttons (when viewed from the front of the device) as you power it on, the screen goes blank, which is what the instructions said would happen if u-boot tries to boot linux.

In the 'black screen' mode, connecting the device to my computer creates a /dev/ttyACM0 device, which sounds promising for this being the u-boot serial console.

I have tried using a terminal emulator program to connect to the device via ttyACM0 (picocom /dev/ttyACM0, or cu -l /dev/ttyACM0).  In both cases the program connects without error, but I do not see anything on the display or in the terminal.

The copy of u-boot seems to have come from openMoko, so I tried their wiki (http://wiki.openmoko.org/wiki/U-boot).   It sounds like what I am doing should have worked - tried a few baud rates, but I am getting nothing, rather than gibberish - I think I'll read through the u-boot configuration that I just installed and try to work it out.....