Nerdy Toad

Ubuntu 11.10 on a Lenovo G570 Laptop

2012-02-03T19:27:00.001Z

I have just bought myself a new laptop. I got a Lenovo G570 from PC World for just under £450. It seems pretty good Intel i5 processor with 4GB Ram.

Installed the 64 bit version of Ubuntu 11.10. Installation went without hitch. Initial boot hang for some reason, but switching it off and on again solved that, and it has worked fine since. Basically everything I have tried worked out of the box - screen, track pad mouse, web cam, USB. Therefore absolutely no issues - a very good advert for Lenovo and Ubuntu.

I was really surprised at how powerful it is - I just installed a simple OSM tile server on it using the instructions at http://wiki.openstreetmap.org/wiki/Ubuntu_tile_server. Loading the british isles OSM data extract took less than 2 hours. My home server (ex laptop) - an intel dual core processor and 2GB Ram running 32 bit Ubuntu takes almost 24 hours.

I am thinking I should have bought 2 of them and use one as the server - it only uses 20 Watts when running at low load, which is probably less than my existing one.....

Using Blender 2.5

2012-01-01T08:13:00.001Z

My daughter and I have been trying to learn to use Blender 2.5 to crate 3D models for use in our simple games for Benjamin. It has proved harder than I had expected! The main issue seems to be that most tutorials are written for Blender 2.4, and there are lots of changes to the user interfaces in 2.5. Plus the official documentation is still for 2.4 and documentation for 2.6 is in preparation...but not 2.5.

This is a few reminders for us for how to do things. I will add more as I work them out.

Adding an armature to pose a model:
See http://www.youtube.com/watch?v=nNTyE9SGRYw for details of linking bones using inverse kinematics. The author has a web site with a lot of additional information here.

Add a bone in Object Mode
Go to edit mode and position the bone within the model mesh.
Extrude the bone to add a joint (e.g. elbow)
In Object mode select the model mesh then shift-right click to select the bone. Do control-P to bring up the 'parenting' menu and select Armature Deform with Automatic Weights.
Set the model view to solid (or texture), and in the armature properties menu, select X-Ray so that you can see the bones.
You can now enter pose mode and move the armature to pose the model.
To do a complete skeleton you should set up the back bone to be the parents of all the other bones so you can just grab the back bone to move the model.

Simple Games for Cheap Android Tablet

2011-12-17T20:00:00.000Z

My Autistic son has developed quite a liking for hand held electronic devices - he likes our daughter's Nintendo DS with an animated dog on it, and apparently plays with an iPad at school.
I am not a fan of 'i' anythings because of the difficulty developing software for them, so I thought I would get him a cheap android based tablet to try. Just need some software for it now.

I have got a Eken M009S from Amazon - it was just under £70. I was hoping to develop some simple games using javascript to run in the web browser, which would mean they would work on any device, but the processor on the device is not up to it - there are some nice javascript game demos at http://kevs3d.co.uk/dev/ which work very well on my laptop, but are so slow on the tablet that they are not useable.

This is a shame as it would have been nice to develop something that would work on any platform - I think I will have to remember how to write 'native' (or at least java) android programs....

Access Web Server on VirtualBox Guest

2011-12-13T22:07:00.000Z

I have a Windows computer where I use Oracle VirtualBox to allow me to run Ubuntu Linux.
I just found out how to access the web server on the virtual machine from the Windows Host.
You need to use the vboxmanage command that comes with virtualbox:
First do:

vboxmanage list vms

to list the names all of the virtual machines on the computer, then do:

vboxmanage modifyvm "" --natpf1 "http,tcp,,8080,,80"

You can then point your web browser on the windows host to http://localhost:8080 to see the web server on the virtual machine.

I don't know the syntax - see the VirtualBox Manual, Chapter 6 for details.

Ubuntu Linux on a Packard Bell OneTwo - Update

2011-12-04T13:59:00.001Z

The Packard Bell OneTwo touch screen computer has not been used for quite a while, so I decided to start using it rather than my daughter's PC, because it uses a lot less power and is quieter.

The main issues from when I tried it before were:

Screen flicker every now and then
Touch screen calibration issues.

We installed Ubuntu 11.10 on it as a clean install. Went pretty well. The issues were:

Used a normal screen shape rather than wide screen. Solved this by going to the settings / display menu and changing to a 16:10 aspect ratio screen resolution.
Touch Screen Calibration: This took a while to solve because when you pressed the screen the mouse pointer appeared at a different position. Eventually I traced it to the display settings that I had altered - Although there is only one monitor on the computer, the software detects two displays (there must be tv-out hardware in the box, but no connector outside). I had un-clicked the 'mirror displays' option and this had resulted in the two screens being shown side by side. I didn't think anything of it as there is no monitor connected to the second, but it confused the touch screen calibration - switching the un-used display to 'off' solved it and the touch screen calibration is fine now.
Multi-Touch: The touch screen works just like a mouse. I think it should be able to cope with multiple touches so you can do 'pinch' gestures etc. to re-size windows. This is not working. I tried using the mtview utility as described in the Ubuntu wiki, and it seems like multi touch is working - you can draw with two fingers at the same time, but the window manager does not seem to be using them.

Therefore, Ubuntu 11.10 worked much better 'out of the box' than the previous version I had tried - the main missing thing is multi-touch support for the touch screen. This is an important one to fix for me because our son is VERY short sighted, and looks very closely at the screen, often pressing his forehead against it. Once he has done that, the touch screen does not work because it only detects one press at a time! I am going to have to learn how x windows input devices work to debug this I think.....

Breaking out of a Frozen SSH Terminal Session

2011-11-16T06:09:00.001Z

For a long time I have found that if the internet connection is broken during a ssh terminal session that the terminal hangs and the keyboard is un-responsive - CTRL-C, CTRL-D etc. do not do anything. I have always just cursed and closed the terminal window.

I just found that ~. will break out of the frozen session, giving you access back to the terminal. ~? gives a list of other escape commands, but I am not sure what I would use those for.

Using Android Phone as a Modem

2011-11-08T09:23:00.000Z

For quite a while I had been able to use my Samsung Galaxy Apollo phone as a modem - I just plugged it into my work laptop (running Windows XP) and it appeared as a modem, which I could use to connect to the internet nicely.

Then I tried (and failed) to update my phone using Samsung Kies and it stopped working. I remembered as part of trying to get Kies to work I found an obscure code to type on the keypad to switch the phone between 'PDA' mode and 'Modem' mode. I have just managed to switch it back to modem mode, and this has got my modem working again!

To do this I had to type *#7284# on the phone dialer keypad. This brought up a menu where I selected the USB option to be 'Modem'

Now plugging it into my computer shows it up as a Samsung Mobile Modem. I have the phone number set to *99***1#, which seems to work ok.

I disabled the internal modem in the laptop because sometimes windows tried to dial that one instead, which is not connected to anything.

Samsung Colour Laser Printer and Ubuntu Linux

2011-11-06T18:02:00.000Z

I have got sufficiently sick of my Epson inkjet printer suffering from clogged print heads, and noticed how cheap colour laser printers have become, so decided to give one a try.

I settled for a Samsung CLP-325W (w=wireless) colour laser printer. The reason for this is that Samsung provide linux drivers for it, and I had an old Samsung laser printer a few years ago that worked nicely with linux.

To start with, the CD that came with the printer would not mount on my daughter's Ubuntu 11.04 machine - no idea why, so we downloaded the Samsung unified driver for linux from the Samsung UK support web site.

We installed the driver by running install.sh as root, plugged in the printer and it detected ok and seems to work. We had to select the "Samsung CLP-300 Series (SPL-C)" driver, because the recommended "CLP-325 foomatic...." one only printed black and white.

So far, so good!

The fun came when I tried to set up wireless printing so I could print to it from my laptop without my daughter's computer being switched on. The first attempt was to repeat what I had done for the other computer, and install the Samsung unified driver on my laptop. Unfortunately it refused to detect the printer, even when it was plugged into the USB port on the laptop. I never found out why and tried a different approach.

I plugged the printer into my ethernet network and re-started it. I found out that to print the diagnostics you hold the 'cancel' button (with the triangle symbol on it) for a few seconds. This printed out a network diagnostic page that said the network type was 'dhcp' and the ip address was 192.0.0.192. This is odd because my router, which would act as dhcp server should give out addresses 192.168.1.*, so I assumed that it was lying about using dhcp and had a static default address.
Re-configured laptop to use the 192.0.0.* subnet and tried looking for the printer - tried a web browser, ping and nmap, and no sign of it.

Had a look at my router's dhcp list, and realised that the printer had been assigned an ip address by the router, which was not the one on the network diagnostics!!!!!
Re-printed the network diagnostics and got the correct ip address - don't know what happened there.

Pointed a web browser at the printer's IP address and got a nice status web page, with a 'login' option at the top right hand corner of the screen (once I had scrolled across because I couldn't see it on my laptop screen for some reason....). Had to search on the internet for the default login credentials - found out that it is user name - "admin", password - "sec00000".

Once logged in, I could set up the network to use a static IP address (so I always know where it is, and connect it to my wireless network).

Disconnected the lan cable from the printer, and sure enough, I can see the printer from my laptop. Used the Samsung printer configuration tool to set it up as a network printer on the laptop and....it works!!!

phew!

CodeIgniter / Bonfire

2011-10-11T06:56:00.000+01:00

I have been progressing an automated way of generating maps from openstreetmap data, allowing layers to be selected, including contours and hill shading etc. (https://github.com/jones139/disrend).

I want a web front end to allow people to generate maps using this renderer. When I did this for my townguide program I just built a very simple php based web interface.
Ideally I would like things like user authentication though, which would mean writing a lot of code to go with it.
The CodeIgniter framework goes some way to this by providing an interface to sessions etc, but there is still quite a bit of boilerplate code to write to produce the database interface.
The CodeIgniter / Bonfire application seems to solve this by providing:

Roles based authentication.
Modular structure (this was an issue I had with codeigniter - you ended up with code scattered in different places).
A really neat module builder function that produces a basic module with database interface for you to build on.

I am just trying to get the hang of using bonfire now - I have had a few issues with installing modules based on the module builder code - will write up how to do this once I have it working...

Building Debian Packages

2011-07-22T21:18:00.000+01:00

I worked out how to make simple debian packages by hand as described in a previous post.
Trying to build other software into packages is still rather difficult...
I found a really good, simple tutorial on how to use the debian tools to do this more easily at:
http://www.debian.org/doc/packaging-manuals/packaging-tutorial/packaging-tutorial.pdf.

Ubuntu packages and repositories

2011-06-17T22:34:00.001+01:00

Creating Ubuntu (debian) packages always seems a bit fiddly....At the moment I can do it, so thought I'd better write down quickly how I did it.

Packages

Create a directory and put the files you want to install into a directory tree under it (e.g. if you create a directory called [pkgname], files that you want in /usr/local/bin go in [pkgname]/usr/local/bin.
Create a directory called [pkgname]/DEBIAN and put a file in it called 'control'. This can be as simple as:

Package: grahamstestdeb
Version: 001
Architecture: i386
Depends:
Maintainer: Graham Jones (grahamjones139@gmail.com)
Description: Test Debian Package

Create a file called 'postinst' in DEBIAN. This will be executed when the package is installed. This can be

#!/bin/sh
echo "postinst works!!!!"

Create the .deb package by doing

dpkg-deb --build [pkgname]

Copy your new [pkgname].deb file into a new folder called repository.
Run 'sudo dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz' in the repostitory directory.
Upload the repository directory to your web server:

ncftpput -v -R -u maps3.org.uk -p ftp.maps3.org.uk public_html/apt/ *

Add the following to /etc/apt/sources.lst

deb http://apt.maps3.org.uk /

Run sudo apt-get update
Run sudo apt-cache search [pkgname] - should find your package.
Run sudo apt-get install [pkgname] - should run your postinst script.

For better instructions see: http://odzangba.wordpress.com/2006/10/13/how-to-build-local-apt-repositories/#comment-7304.

Rendering OpenStreetMap Data using Carto and Mapnik

2011-05-19T22:29:00.001+01:00

Mapnik is often used to render OpenStreetMap (OSM) data. The map style (line colours and sizes, icons etc.) are defined by a style file. There is a very detailed one provided with OpenStreetMap's mapnik tools. Unfortunately that level of detail means that the style definition is very complicated and it is hard to follow how it works.
A tool called carto is available which is a pre-processor that will take a simpler format of style file, and convert it into the mapnik file format. I have been experimenting with trying to render OSM data using styles written for carto. I had been hoping to use a nice graphical editor with map preview capability called tilemill, but tilemill does not work with postgresql datasources, which are necessary for OSM rendering. Instead, I installed tilemill and used the version of carto that is included with it, but did not use the tilemill editor.

Carto Layer Definitions
Rendering data with carto needs you to do two things - define the data you want to plot on your map ('Layers'), and define how you want the data displayed ('Styles').

The datasource definitions go in a .mml file that looks something like this:

{
"_center":{"lat":36.870832154494,"lon":-113.79638671427,"zoom":5},
"_format":"png",
"srs":"+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs",
"Stylesheet":["style2.mss",
"areas2.mss",
"roads2.mss",
"labels2.mss",
"natural2.mss",
"contours2.mss",
"POIs2.mss"],

"Layer":[
{"id":"coastline",
"name":"coastline",
"srs":"+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over",
"class":"",
"geometry":"polygon",
"Datasource":{"file":"/home/graham/OSM/data/mapdata/world_boundaries/processed_p.shp","type":"shape"}},

{"id":"landuse",
"name":"landuse",
"srs":"+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs",
"class":"",
"geometry":"area",
"Datasource": {
"type":"postgis",
"dbname":"kefalonia",
"user":"www",
"password":"1234",
"host":"localhost",
"port":"",
"table":"(select way,landuse,name,\"name:en\" from planet_osm_polygon where landuse is not null) as landuse"
}
}
]
}

The first few lines define the projection of the output map, and the stylesheets that are to be used to define how to render the data. The bulk of the file is layer definitions. In this example two layers are defined. One is a shapefile containing the coastlines (the same one that is used by the normal OSM style). The second layer extracts all areas tagged with 'landuse=xxx'.

We have not told mapnik how to plot the data though - this is done in the style files. The ones relevant to this example are "style2.mss" and "areas2.mss". Style2.mss contains:

@land: #e0e0c0;
@water: #C0E0F8;
@waterline: #8CE;
Map {
background-color:@water;
}
#coastline::outline {
line-color:@waterline;
line-width:1.6;
}
#coastline::fill {
polygon-fill:@land;
polygon-gamma:0.75;
}

Here the things starting '@' are variables which allow you to define colours etc. at the top of the file, then use them several times throughout it, making future maintenance easier. In this example we define the background colour for the map, fill it in blue to represent water. We then draw a line around the costline, and fill in the land with a separate colour.

The areas2.mss style is slightly more complicated:

#landuse {
[landuse="residential"]
{
polygon-fill: #b0b0b0;
}
[landuse="industrial"]
{
polygon-fill: #a0a0a0;
}
[landuse="forest"]
{
polygon-fill: #a0d0a0;
}
}

In this example the [] expressions are filters, which allows us to define different fill colours depending on the landuse tag.

These are just extracts - the actual styles I am using are getting progressively more and more complicated. You can see the files, and a modified version of the generate_image.py script provided by OSM, which takes a carto .mml file as input in my svn repository (soon to be copied to github..)

Using these styles, I have updated my kefalonia map to give this:

and this:

These styles are nowhere near as clever as the standard OSM ones. In particular I have not tried to devine different styles fo rdifferent zoom levels instead I have two sets of stylesheets for two different zoom levels - I think in the end I will have there - one for a 'country level', another for a 'town level', and a third for higher zoom levels.

There are also a few issues - the line styles do not look as nice as the OSM ones - I do not know why...and there is a problem with the road names where sometimes other roads crossing over each other obliterate the labels. THe reason is that I draw the road and the label at the same time - it will probably look better if I do the labels after all roads have been drawn. Will hav eto work out how to do that!

Installation of Carto Mapnik Style File Builder on Ubuntu

2011-04-29T20:59:00.003+01:00

I find mapnik style files a bit clumsy to edit - lots of repetition that makes them hard to follow.
There are (at least) two utilities available that may make style file generation simpler. These are Cascadenik and Carto.
It appears that Carto is being more actively developed at the moment, so I thought I would look at that first. Carto also reports to be much faster than Cascadenink.
The problem is that Carto is new-fangled and uses a javascript based language called node.js. Despite my dislike of javascript, I thought I would persevere. This is how I got it working on my Ubuntu 10.10 system:

Downloaded node.js version 0.4.7 from http://nodejs.org, and extracted it into /usr/local/node-v0.4.7. Built with ./configure; make; make install.
Installed the node package manager npm using: curl http://npmjs.org/install.sh | sh
Installed carto dependencies:

npm install get [used to be node-get, but the module has changed name]
npm install step
npm install srs
npm install underscore
npm install zipfile (actually did this twice because one of the document build actions failed on the first attempt...).

Downloaded carto using: git clone https://github.com/mapbox/carto.git
No compilation necessary (it is javascript). Executing cd carto; ./bin/carto gave no nasty errors, just "no input files", which looks promising.

I just need to work out how to use it now.

I have had a quick look at TileMill, which is a web based editor for carto stylesheets - it is very impressive - you can change the stylesheet and preview the map in real time. The only problem that I have found with tilemill is that it does not seem to work with a postgresql database as the data source, which is where my OSM data is stored. But at least I should get the idea about the structure of the files using that tool.

Kefalonia Map

2011-04-25T21:24:00.004+01:00

We are going to Kefalonia on our holidays this year, and I usually like to do a bit of OSM mapping when I go somewhere new, and may do some 'armchair mapping' before I go to get a head start, so I thought I would have a look at how well mapped it is.
The answer is not very - there are the main roads, and a few minor roads and footpaths, but not much else - this is a shame because I like to know where things like supermarkets and banks are. Unfortunately there is not a lot of information available for remote mapping - very few GPS traces, and Bing Imagery does not cover the area I am interested in.
So instead I decided to make myself a map of the island showing what is there (in OSM anyway). Because there is not a lot of detail, I want to have contours so I can tell where the mountains are, and also highlight what points of interest there are at lower zoom levels than the standard OSM style. These are my notes on how I produced the map, so I can remember next time I need to do it.

The map bounding box was found by looking at the (lon,lat) readouts on http://www.informationfreeway.org to check the bottom left and top right of the island. They are (20.33, 38.03) and (20.82,38.50).
I downloaded the OSM data from the mapquest XAPI server using:

wget http://open.mapquestapi.com/xapi/api/0.6/map?bbox=20.33,38.03,20.82,38.50 -O kefalonia.osm

Imported the data into postgresql database using:

osm2pgsql -s -S default.style -d kefalonia -m kefalonia.osm

Downloaded the SRTM elevation data covering the island from http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Eurasia/ - only needed one file (N38E020.hgt.zip), because it is a small island.
Generated the contours and imported them into my postgresql kefalonia database as described in http://wiki.openstreetmap.org/wiki/Contours.
Modified the standard osm mapnik style file as described in http://wiki.openstreetmap.org/wiki/Contours, using the 'PostGIS' method.
Generated the map image using the generate_image.py file from the OSM mapnik archive. I initially had some trouble with no roads appearing, but solved this by re-compiling osm2pgsql from the latest sources - I do not know what the problem was.
Having done that, I get this map image:

Kefalonia Map - Version 1 (25 April 2011)

Next I wanted to add a 1km grid to the map to make it easier to judge scale..."are we nearly there yet?" etc.
I did this by adopting the generate_graticule script from mapnik-utils to produce a grid on the google spherical mercator projection (=SRS 900913). I must admit to not really knowing how this works, but my updated version (generate_grid.py) is in my SVN repository. I added this to the map by creating an extra xml include file to define the grid layer and style (inc/grid.xml.inc), adding this to layers.xml.inc, and finally defining the layer in osm.xml.

The result (Version 2) is shown below:

Kefalonia Map Version 2 (26 April 2011)

I created another include file for holiday based points of interests (holiday_pois.xml.inc), added it to layers.xml.inc and again added the actual layer definition to osm.xml.
This gave the following map:

Kefalonia Map Version 3 (26 April 2011)

The final thing I needed was the English spelling adding to the Greek text labels. I did this by adding name:en to the default.style file used by osm2pgsql and re-importing the OSM data into the database.
The challenge was updating the mapnik style file to use the name:en column - the OSM style files are very complicated with lots of SQL SELECT statements to extract the data out of the database. I was tempted to do a global replace of 'name' with 'name:en'...but 'name' appears all over the place.
I decided the main thing I wanted with both writing styles were place names and I found the code for them in inc/layer-placenames.xml.inc.
I modified the SELECT statement from a simple 'name' to ,'"name" || ' (' || "name:en" || ')' as name'. This gives me the greek spelling followed by the english spelling in brackets using the postgresql '||' string concatenation operator. The result is here:

Kefalonia Map Version 4 (27 April 2011)

I think that is about the finished product - I will have to try printing it to see how it looks, then may need to adjust the image size to suit printing....then do some mapping to add some more POIs - Sacla in the south east of the island should be a lot better by the middle of June....

Problems with GoogleCode SVN

2011-04-17T20:39:00.001+01:00

I use GoogleCode (http://code.google.com) for my little open source projects, and it usually works quite nicely.
I was just trying to set up one of my repositories in writable mode on a new computer, and "svn ci" (checkin) gave 500-Internal Server errors, which was not very helpful.
Eventually I realised that although I had done the "svn checkout" with a --username=", when I was doing the checking it was using my computer username, rather than my google one...so presumably the authentication failed.
I had to do "svn ci --username=", then it worked ok. I don't know why!

Trouble with mysql, apache, php5 and codeIgniter

2011-04-03T23:35:00.000+01:00

I am working on a new web interface to townguide based on php so it can be used on a low cost web hosting service (rendering will be done by a separate computer).
I am going to use the codeIgniter web framework because it seems simple, and is simiar to django, which I was just starting to get used to.....
Alas getting database access working on my laptop for development was difficult.
I have apache, php5 and mysql all working, but whenever I tried to load the database module of codeIgniter I got a 500 Internal Server error, but no errors in either the apache or codeigniter error logs. This made debugging difficult.
After much searching on the internet I realised it was because I needed to install libapache2-mod-auth-mysql as well as php5-mysql.
Doing sudo apt-get install libapache2-mod-auth-mysql and re-starting apache made the following trivial mysql db connection work:

$conn = mysql_connect('localhost', 'www', '1234');
if($conn) {
echo 'Finally connected!!!';
} else {
die('Still cannot connect' . mysql_error());
}
?>

A bit late now, but I think this will also have solved the problem connecting to the database using codeigniter....

Disk Crash on Linux

2011-03-29T17:01:00.001+01:00

Well, I thought that the introduction of the ext3 (and higher) file systems had stopped all of the problems of disk crashes and mangled filesystems on linux.
But, after a few years of reliable operation the root filesystem on my home server (my old Fujitsu Siemens laptop) seems to have mangled.
The symptoms were....not booting - you get the 'Starting up......' message during boot then the system appears to hang. But if you listen carefully you hear that the disk is actually doing something. So I left it for a few hours.....
When I came back to it there was a 'failed to mount /dev/sda1' type error message, with an option to skip mounting or manually fix it.
I went for manually fixing it, because not mounting the root filesystem will not get me very far.
This dropped me into a single user shell.
I ran fsck and it said that an Inode has illegal blocks. I selected the option to clear them...and again said yes when it asked me again.
It then said it was restarting e2fsck from the beginning, and spent quite a few minutes checking....

Then at Pass 2 fsck said it had found a deleted or unused inode. Again I said 'y' to the 'Clear?' question....
Then lots of offers to fix things (to the extent that I just held my finger on the 'y' key...)

Then fsck announced that it was complete and I should re-boot linux. Re-ran fsck and it announced that /dev/sda1 was clean, so re-booted.....but booting is taking a suspiciously long time.....like it has been trying for 10 minutes and hasn't got past the boot up splash screen...I'm going to need a plan B...

Well, I don't know what the problem is. Tried booting off a USB memory stick, and the disk checks ok and mounts, but the boot process just hangs. I decided I had spent too long on this so it is currently installing Ubuntu 10.10 on the disk instead, so can't try any more diagnosis!

Emacs Drupal Mode

2011-03-06T19:15:00.000Z

I am trying to learn how to write Drupal modules for a new map rendering on demand web service (a bit like townguide, but distributed with a separate front end and rendering servers).

First thing is to get emacs23 working with php and drupal modes - the instructions here give me obscure errors about c-lang-def only being used in a file. After a bit of hunting on the internet I found it is a problem with Emacs V23, and a simple work around is to put the php-mode.el and drupal-mode.el files in ~/.emacs.d, then add the following to ~/.emacs:

; make sure the target directory is on your load-path
(add-to-list 'load-path "~/.emacs.d")
(autoload 'php-mode "php-mode" "Major mode for editing php code" t)
(autoload 'drupal-mode "drupal-mode" "Major mode for editing drupal php " t)
(add-to-list 'auto-mode-alist '("\\.\$module\\|test\\|install\\|theme\$$" . drupal-mode))
(add-to-list 'auto-mode-alist '("\\.\$php\\|inc\$$" . php-mode))
(add-to-list 'auto-mode-alist '("\\.info" . conf-windows-mode))

With that in my .emacs file, loading a .php or .module file starts php or drupal mode - success!

Easier Map Creation with OpenStreetMap Data

2011-02-24T16:33:00.000Z

I have been giving some thought to making the process of creating maps form OpenStreetMap data easier. The problem is that to use the mapnik map renderer, you need to install quite a lot of tools and set different configuration files before you can even start to create a map.
This means you have to be pretty keen to persevere enough to get to an end result.
I am thinking of creating a service to allow you to bypass the tool installing bits, and concentrate on the setting up of the map configuration, so you can get to a result quicker - see http://wiki.openstreetmap.org/wiki/User:Grahamjones#Speciality%20Maps.

Quite a few of the responses that I got to that proposal were about the lack of good documentation to get people started on the process. I have had a go at addressing this by putting together a simple overview presentation to go through the main concepts and tools that you need to generate a map using mapnik. The presentation is on slideshare.net, and should be visible below:

Rendering OpenStreetMap Data using Mapnik

View more presentations from jones139

Mounting FTP Directory as a Filesystem

2011-02-18T21:11:00.001Z

I just discovered a really useful trick to mount the directory structure of a remote FTP server as a directory on a linux computer. You use a program called curlftps. On Ubuntu do:

sudo apt-get install curlftps

The directories on the remote ftp server are then mounted using:

curlftpfs ftp://[username]:[password]@[server address] [local directory mount point]

This works fine (you do not even need to be root to do it), but the password will be present if you do 'ps -ef | grep ftp'.
A solution is to create a .netrc file in the /root directory containing:

machine [server address]

login [username]
password [password]

You can then add the file system to /etc/fstab by adding the following line to /etc/fstab:

curlftpfs#[server address] [local directory mount point] fuse allow_other,rw,user,noauto 0 0

You can then just do

sudo mount [local directory mount point]

and you get the same effect with the server password invisible.

OpenWRT Based Utility Meter Monitor

2011-02-13T10:38:00.000Z

I have made some progress porting my MeterServ application (http://meterserv.webhop.net) to openWRT. I have managed to work out how the openWRT makefile system works enough to add it to the build system and produce an openWRT package for the MeterServ application.
To do the improvements that I want to make to the application though, it needs more work. In particular I don't like Perl, so want to convert the web interface part of it to Python. Rather than go for a completely DIY web framework, I have adopted webpy (http://webpy.org) - this seems like a simplified version of django - you define URLs that you want the application to work with, then point it to a python class that does the processing associated with the user requesting a particular URL.
I have started a seperate google code project for meterserv, because a couple of people expressed an interest in it. I have put the latest version of the software in its repository too - see http://code.google.com/p/meterserv.

Utility Meter Monitor

2011-01-29T21:44:00.000Z

I have had a simple utility meter monitoring system working in my parents and my house for a few years. It uses an NSLU2 single board computer connected to a simple USB input/output interface, as described at http://meterserv.webhop.net.
It worked fine for at least 3 years, but started to misbehave recently. I have not been able to determine if it is a hardware or software fault - I suspect that I may have worn out the USB memory sticks that I used for the root filesystems.
We have gone for a hardware change to see if that is the problem, but I want to develop the software a bit more too.
Future developments include:

Add support for cheap electricity meters (e.g. OWL USB energy monitors).
Add support for temperature sensing.
Better web interface (the current one uses perl based CGI scripts, which are awful to maintain).
A simple (end user friendly) software upgrade process.

I want to get rid of the perl cgi scripts because I don't really speak perl, so they are hard to follow - will change them for an ajax based front end with python server side scripts.

The first stage is to get a sofware development environment working. The original version used the nslu2-linux slugos operating system. This has been changed quite a bit since I used it, so I decided that as I will need to do a complete re-build, I will standardise and use OpenWRT for all my little embedded system projects.

Compiling OpenWRT for the nslu2 was nice and easy - in the menuconfig just select the processor as an ixp4xxx and the sub-type as nslu2.

Edit target/linux/ixp4xxx/base_files/etc/config/network to give a default network configuration that will work on your system (the openwrt default ip address is 192.168.1.1, which clashes with my router....).

This creates a file called bin/ixp4xxx/openwrt-nslu2-squashfs.bin which can be flashed onto the nslu2 as follows:

Put the NSLU2 into upgrade mode by switching off, then powering on while holding in the reset button for about 10 seconds. When the status LED changes colour (at about 10 sec), release the reset button. The status LED now flashes different colours (very subtly different colours if you are colour blind like me!).
Use upslug2 (sudo apt-get install upslug2 on Ubuntu) to upgrade the NSLU2 using: upslug2 -d wlan0 --image=openwrt-nslu2-squashfs.bin
The NSLU2 will re-boot, but then take a while (a few minutes) to set up a writeable overlay file system before being accessible over the network.

This showed that I could build software for the NSLU2 using openwrt, but now I need to port meterserv to OpenWRT, which will be a separate post....

OpenWRT on Linksys NSLU2

2011-01-23T15:41:00.001Z

I have had a couple of NSLU2's running a little application to monitor utility meter readings for a few years now (meterserv). It worked fine until the autumn when my Dad's one stopped recording readings. I have not been able to find out why - it looks ok. It may be that the interface card has broken, or I do wonder if I have worn out the USB flash drive that I am using for the root filesystem.

In my inability to find a fault, I decided to do a clean install on a usb disk to rule out software and flash drive problems. They had been using OpenSlug (now called SlugOsBE) from nslu2-linux, but it is now very out of date, because I have not updated them for a few years. This means that all the links to package directories are broken etc.

Given the recent success in building OpenWRT for the bifferboard, I decided to use OpenWRT for the re-build. These are my notes so I can do it again next time....

In the MenuConfig system, set the target system to 'Intel IXP4xx', sub target to 'Generic' and target profile to 'Linksys NSLU2'.
Because this is not my new router, I do not want it to use the default IP address of 192.168.1.1. This can be changed in the 'Image Configuration' bit of the configuration menu.
I would like to re-code some of the meterserv software in python, so I need python and pyusb installed in the flash image - these were selected from the 'Languages' section of the openwrt menu.
Make builds a firmware image in bin/ixp4xxx called openwrt-nslu2-squashfs.bin, which is an 8MB image - there is a 16MB one too, but I am not convinced that my nslu2 has that much flash memory.
You have to put the nslu2 into 'recovery mode' by holding the re-set button in while you power up the machine - you have to release the reset button as soon as the status LED changes colour, after about 10 seconds. The status LED then flashes a couple of different colours.
Flash the new firmware image onto the NSLU2 using upslug2 (sudo apt-get install upslug2 on ubuntu) - the command line was sudo upslug2 --target 00:14:bf:64:df:be --image openwrt-nslu2-squashfs.bin.
The machine now appears on the network with the right IP address. Telnet into it and change the root password, then connect using SSH instead.

The only problem is that when I type 'python' I get an error "python: can't resolve symbol 'BC'". So now I need to work out which package is missing - looks like openWRT missed a dependency, but I don't know which one!

I have had a bit more of a look at it. There are two surprising things. The first is that if I write a simple 'hello world' script and execute it, it works, so it is just interactive python that is broken.

The second is that doing "ldd /usr/bin/python" on the nslu2 and on the bifferboard give very similar results, except that the bifferboard one is linked against libgcc and libc, but the nslu2 one is linked against libc.

I think this is telling me something, but I am not sure what....

Mapnik Map Rendering

2011-01-21T20:20:00.000Z

Someone asked a question on the osm-gb mailing list about viewing coverage of waterways - canals and navigable rivers.
We don't seem to have one, so I created one on my http://maps.webhop.net web site.
The trouble with that site is that although it is running on a nice powerful computer (my old laptop), it is behind my domestic broadband service from Virgin Media. Although this service has a nice fast download speed, the upload speed is very poor ~900 kbps compared to 9600 kbps download. This makes the web site very slow from the outside world.

I have a little virtual server provided by CloudNext - it is nowhere near as powerful as my own server, and has much less disk space, but at least it has a fast internet connection.

I loaded a british isles extract of OpenStreetMap data into the database - it took a very long time - maybe getting on for 12 hours (not sure when it finished!) - my server takes about 2 hours. Performance rendering maps was not too bad though - rendering down to zoom level 14 took about 2 hours, which is respectable.

Just need to get the database updating to keep it up to date, and re-generate changed tiles now....
You can see the difference at http://maps.webhop.net/canals (my home server) compared to http://maps2.webhop.net/canals (the virtual server)

Personal Weather Station using BifferBoard

2011-01-16T20:52:00.006Z

I have managed to re-compile openWRT to run on the bifferboard, and include python, libusb and pyusb in the main root directory. I also included the wget utility, because I thought it would come in useful, plus a few other potential future extensions (lighttpd web server and wireless LAN support).
This fits on the bifferboard onboard flash memory, with about 1MB to spare.

I created a /home/weather directory and extracted a recent pywws tar archive into it. I was very pleased that running TestWeatherStation.py produced a table of numbers as expected, so it looks as though the bifferboard is talking to the weather station ok.
Here it is:

Set up the initial weather database by doing

cd /home/weather

python pywws/pywws/LogData.py -vvv /home/weather/data

This creates /home/weather/data/weather.ini and a directory /home/weather/data/raw which contains the raw data.

Run Hourly.py for the first time to process the data, and set up the weather.ini file ready for customisation:

pywws/Hourly.py /home/weather/data

Customise /home/weather/data/weather.ini to do what you want it to do - in my case update weatherunderground....

All is looking promising here - seems to send the update ok on an hourly basis. There are two problems though:

There is not much disk space left now I have added all of January's weather data (256kB ish)
I am concerned that I will wreck the flash chip on the bifferboard with all this writing.

One option is to store the weather data in a ram disk and just accept that after a power-off it will need to re-initialise itself - not sure how pywws will cope with this - I will need to load some default weather.ini file from flash every time it boots, then let it update itself as best it can.

As a quick alternative I have found an old 256MB SD card and created an ext3 filesystem on it. This detects as /dev/sda1 when plugged into the bifferboard. I have modified /etc/rc.local to mount this as /home/weather. This should solve my full filesystem problems. [Note: I used an ext3 filesystem because my openWRT build could not detect ext2 - this was a bit of a surprise because I thought you got ext2 support free with every Linux kernel...].
Then it is just a matter of a cron job to do the hourly updates - crontab -e, then add:

13 * * * * python /home/weather/pywws/Hourly.py -v /home/weather/data >> /home/weather/Hourly.log 2>&1

If it works, you should continue to see the weather in our back garden at: http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IHARTLEP2.

Mistake 1: If you put an SD card into the dual USB bifferboard, it disables one of the USB ports. This is bad if it was the one the weather station was connected to....

Mistake 2: Believe people when they say the clock on a board is no good. The Bifferboard clock is no good. Not only does it loose its time when powered off, but it drifts - by a few minutes an hour. This meant that pywws got confused about when it should update weatherunderground, because it thought time had suddenly gone back to 2009.... To avoid this I compiled ntpclient as a package and downloaded it onto the bifferboard. I now have it set to be called once on boot, then every 10 minutes by a cron job. This should keep it somewhere close - just hope the time server owner doesn't mind - If I get a complaint I will have to set up my own time server...

Building openWRT for BifferBoard

2011-01-16T15:40:00.002Z

I had a go at building openWRT for my new bifferboard computer last weekend, but had trouble with it - it looked like the kernel started, but then panicked around the time it was supposed to be starting init. I suspect it was something to do with the j2ffs file system.

I decided to try to be clever and rather than fix openWRT, start from scratch with an OpenEmbedded based build. I think this was too big a step - the main problem was that OpenEmbedded failed to build because some sources from handhelds.org are not available, so I had to learn how to alter 'recipes' to build them from sources from another location. I got ipkg-utils to build this way, but then hit more trouble with openssh. I decided it was going to take me a long time, so went back to openWRT for now to try to get something working!

This time I followed the exact instructions from the bifferboard site, including downloading the specific revision of openWRT. This all compiled nicely without any errors, so the next challenge was to get it onto the board.

Flashing over the serial line seemed to take a very long time, so I had a go with network setup.
I downloaded the bifferboard utilities using:

svn co https://bifferboard.svn.sourceforge.net/svnroot/bifferboard

This provides a simple python based tftp server. I had to modify it to use a fixed ip address for my host computer, because the script assumes that you are using eth0 for the network interface, and I was using wireless.
You then need to make sure that there is a file called bzImage in your working directory and start bootp_server.py - I had to do it as root because I got some permission denied errors about opening sockets, and the easiest way to fix it was to be root - there is probably a more elegant solution though.

First I created a symbolic link to the bzImage file that openWRT had produced. Then booted the bifferboard with the serial line connected and pressed ESC to stop the boot process. Entering the tftpflash command downloaded the bzImage from the server and flashed it to the disk - much quicker than a serial line.
The kernel started, but failed with a lot of j2ffs errors - I think this is because I had my old root filesystem in the flash memory.
For the next attempt I created a symbolic link from openwrt-rdc-jffs2-64k-bifferboard.img to bzImage.
Re-flashed it again the same way and it boots!

A minor detail was that I could not log into it - the serial console was fine and both telnetd and dropbear (the SSH server) were running, but I got 'connection refused' when I tried to connect. It turned out that there was something called 'firewall' running. I deleted /etc/rc.d/S45firewall and re-booted and it worked properly - I could telnet in initially without a password, set a root password, then use ssh to connect.
Success - Now I just need to sort out the weather station software to run on it - back to the cross compiler....

running make package/symlinks in the openWRT directory adds all of the available packages to the openWRT menuconfig program.
Added the ones I know I need (libusb, python, pyusb), plus a few others that might come in useful later (atheros wireless card drivers). Then make.....and wait...

A "Hacking Embedded Linux Devices" wiki site?

2011-01-08T16:22:00.000Z

I have done a bit of work trying to hack cheap consumer devices to run different software - NSLU2, mediaMVP, musicPal, edimax IP Cameras etc. I have also had a look at 'off the shelf' single board computers like bifferboard.
Whenever I start I have the difficult learning curve of trying to remember how to set up a cross compiler, cross compile libraries and link them into new software.
I also think it would be useful to have a nice list of which devices have been successfully hacked and which ones are difficult.
I am thinking of setting up a site to collate such basic information, which could then link out to the more specific project sites.
Unless anyone knows of one already?

I think it would need a wiki to store most of the information, and an email discussion group. I wonder how best to do it - there are google sites and google groups, wikispaces, pbworks.

Any suggestions on the best way to do this?

Building openWRT

2011-01-08T15:04:00.003Z

I have got a little bifferboard single board computer which I intend to use to run pywws to send data from our weather station to the weather underground site on the internet.

The bifferboard comes installed with a very small linux distribution called openWRT. It has a very small python installed, but no python USB support.

I have struggled a bit to work out how the openWRT package system works - the official wiki is a bit confused about what the current version is called (kamikaze or backfire). It implies you can add packages by downloading them from svn, but this didn't seem to do anything.
I found a useful forum post here, which seems to be the best set of instructions.
You can do

./scripts/feeds update -a

This downloads the list of packages, but does not do anything else.

To get the buildroot system to compile it for you you need to 'install' it using:

./scripts/feeds install python 
./scripts/feeds install pyusb
You can then do "make menuconfig" and python and pyusb are shown to be compiled as packages "".

'make' actually compiles it.

I'll update this when I work out how to add these to the firmware image...

Hacking the Parrot-DF3120 Picture Frame

2011-01-01T22:37:00.000Z

Some clever people have been working on running Linux on a parrot DF-3120 Picture Frame, as described on http://sites.google.com/site/repurposelinux/df3120.
The device sounds like a little computer with a colour display, USB connection, SD card and bluetooth wireless (I would have liked a wireless lan, but never mind!). Most significantly, Amazon are selling them for less than £10, so I thought it was worth getting one to play with. It arrived very quickly so I started to play with it yesterday.

The instructions at http://sites.google.com/site/repurposelinux/df3120 fall into two distinct parts:

Produce a cross compiler for the board, and use it to compile a Linux kernel, busybox and other things needed to construct a basic root filesystem. This is all done by a single clever script (minifs). It actually took a bit of doing to get it to work on my Ubuntu 10.10 system - when it fails you have to look at the log files to see why it has failed. The most noticeable thing was that I had to install gtk-doc from a source tarball because Ubuntu does not have a package for it - everything else (bison, flex etc.) were installed from ubuntu packages.
The script produced an iso image of a root filesystem.
Install a boot loader on the device. The instructions have you download and compile the u-boot boot loader for the device, and package it into a false firmware upgrade file (.plf) file. There is some black magic required to copy the file file onto the device in the right directory etc.

The boot loader installation apparently worked. If you just switch on the device without touching anything, it boots normally. If you hold down the centre and left buttons (when viewed from the front of the device) as you power it on, the screen goes blank, which is what the instructions said would happen if u-boot tries to boot linux.

In the 'black screen' mode, connecting the device to my computer creates a /dev/ttyACM0 device, which sounds promising for this being the u-boot serial console.

I have tried using a terminal emulator program to connect to the device via ttyACM0 (picocom /dev/ttyACM0, or cu -l /dev/ttyACM0). In both cases the program connects without error, but I do not see anything on the display or in the terminal.

The copy of u-boot seems to have come from openMoko, so I tried their wiki (http://wiki.openmoko.org/wiki/U-boot). It sounds like what I am doing should have worked - tried a few baud rates, but I am getting nothing, rather than gibberish - I think I'll read through the u-boot configuration that I just installed and try to work it out.....

Inside the Edimax IC-3010WG IP Camera

2010-12-31T09:02:00.000Z

I have just added my photos of the inside of the Edimax IC-3010WG IP camera to my web page: http://ic-3010wg.webhop.net.
If anyone knows what 'J1' is, I would like to know!

Edimax IC-3010 Firmware Modification

2010-12-30T15:23:00.000Z

I just discovered the 'stats' facility in blogger.com and realised that the most read posts in my blog are the ones about the Edimax IC3010 wireless ip camera. People are also looking at my web pages about it (http://ic-3010wg.webhop.net).
This got me thinking about it again so I have had a poke around inside the case - I'll post some pictures at the weekend.
There is not much to see in side - there is a mini wireless card and a single board computer. I was hoping to see a USB port, but the nearest I can find is a four pin header that is likely to be one (or more of):
- USB Port
- Serial (RS232 ish) console
- JTAG interface

I'll have to find the main system-on-chip pin-outs to try to decide what it is.

The main thing I struggle with is getting into the firmware files - I think the choice if you want to hack a device is to either connect a serial line to the console to get access to the boot loader, or modify the firmware. Modifying the firmware sounds simplest, but you run the risk of turning the device into a brick....

That is about where I got to in the summer of 2009. I am just downloading the latest firmware and source code from edimax to try to de-code the firmware building utility. While I was doing this, I discovered that someone has already done it - http://www.suborbital.org.uk/canofworms/index.php?/archives/3-Getting-telnet-access-on-an-Edimax-IC3010-webcam.html. Well done to them!

Personal Weather Station using NSLU2

2010-12-29T15:09:00.001Z

While I wait for my new bifferboard to be delivered I thought I would get the weather station working and updating to weatherunderground.com using a Linksys NSLU2 that I have available.
The NSLU2 is already configured to run Linux. There are quite a few varieties of NSLU2 linux, and I think this one was called OpenSlug when I installed it, but it now seems to be called SlugOSBE.
When I first set it up there was no python package available for it, so I had to write my utility meter monitoring program using Perl (yuk!).
Fortunately there is now a repository at http://ipkg.nslu2-linux.org/feeds/optware/slugosbe/cross/unstable/ that includes python packages.
To get python running (with USB support) on the NSLU2 I had to download the following packages from the repository, and install them with ipkg install .ipk:
libusb, libdb, libstdc++,ncursesw,readline,sqlite_3, zlib, python25, py25-usb

The python distribution did not put an executable in /usr/bin, so I had to do
ln -s /opt/bin/python2.5 /usr/bin/python

With all this installed I could plug the weather station into the NSLU2 (via a USB hub in my case) and pywws just worked.
I hope the bifferboard is as easy!

Personal Weather Station

2010-12-28T20:49:00.003Z

I have just bought my wife, Sandie a personal weather station from Maplin. It is a much better one than the simple one that broke recently - it has external humidity as well as temperature, plus wind speed and direction and rain detection. Most importantly it has a USB connection so we should be able to connect it to an internet weather network so it can appear on our iGoogle home pages.

The maplin version is apparently a re-branded WH1080PC weather station - I chose this one rather than a more modern one because people have managed to get it working with Linux - I want to use a very low power single board computer to interface it, rather than a big PC running Windows.

I have chosen a bifferboard single board computer because it is cheap and has both USB and ethernet connections (I would have liked wireless lan too, but could not find something to do that, so I got the two USB port version of bifferboard, so will have a go at adding a USB wireless network interface card to it later).
My original idea was to use some software called meteoplug because you can buy a bifferboard with that already installed, so I know it would work. However, when I looked into it, it is a subscription service that is not open source - all I really want to do is obtain data from the weather station and send it to weatherunderground, which can not be that difficult.
Fortunately there is an open source project, pywws, which is compatibly with the WH-1080PC weather station.
I thought it best to get it going on a desktop linux set-up first, then try to get it working on the single board computer.
From my Ubuntu 10.10 laptop I did:

svn checkout http://pywws.googlecode.com/svn/trunk/ pywws

sudo apt-get install python-usb

cd pywws

python TestWeatherStation.py

Initially this gave an error "IOError: Claim interface failed", so I tried:

sudo python TestWeatherStation.py

Success - a string of numbers, and no errors!

To get rid of the requirement to run the program as root you have to tell 'udev' to give other users permission.
lsusb reports the vendor and product ID of the device (1941 and 8021 respectively), but this is actually reported as a USB missile launcher rather than a weather station.
I added a file called 05-weatherstation.rules to /etc/udev/rules.d, which contained the following line:

SUBSYSTEM=="usb", ATTRS{idVendor}=="1941", ATTRS{idProduct}=="8021", MODE:="0666", GROUP:="usb"

I also added a line to /etc/group to create the 'usb' group, and made myself a member of that group.
Re-started udev with 'sudo service udev restart', and now the test works without being root (note the colon in 'MODE:=' - it is very important!

pywws seems to work suspiciously well - creating a data directory (~/weather/data) and running

pywws/LogData.py -vvv ~/weather/data/

creates a file with some convincing looking numbers in it. I have created a weather station on weatherunderground, so I think the next thing to do is install the weather station outside, then try to use pywws to send data to weatherunderground.

Ubuntu Linux on a Packard Bell OneTwo

2010-12-28T19:20:00.004Z

This post is a record of me getting Ubuntu Linux (Release 10.10 - Maverick) running on our new Packard Bell OneTwo all-in-one touch screen PC. I will update it as I make progress fixing the bits that have caused problems.

Installation
Installation was pretty easy - I booted of an Ubuntu 10.10 USB memory stick, and selected manual disk partitioning.
I kept the Windows 7 partition (/dev/sda1) unchanged, and did not touch the other two windows partitions (/dev/sda2 and /dev/sda3). I deleted the blank 'Data' partition (/dev/sda4) and created a new swap partition (/dev/sda5) and ext4 root partition (/dev/sda6).
Installation went without problem and the machine now dual boots Ubuntu and Windows 7.

Note that I did have a funny happening yesterday when the machine completely refused to boot after I had been using windows. I fixed this by booting off the USB memory stick and re-installing grub as described in a previous post - This has not happened again.

Audio
Worked out of the box.

Video
Ubuntu did not detect the correct screen size initially - it used a normal aspect ratio rather than wide screen, so only the centre of the screen was used, and the touch screen calibration was way out - this is because it thinks it has two displays (I don't know where the second is - there is no VGA or s-video connector on the back).
This was fixed by going to the System->Preferences->Monitor menu and un-checking the "Same image in all monitors" option.
You can then drag the main screen (labelled "Laptop") away from the second (labelled "Unknown") and set the laptop resolution to 1600x900 resolution.

This makes the screen the correct size, but every now and then it flickers twice within less than a second - it seems that the screen backlight dims momentarily before coming back to full brightness. This seems to be associated with disk activity - I don't know what causes this - it does not do it in Windows - Any suggestions for fixing it would be appreciated!.

Touch Screen
I was impressed that the touch screen worked 'out of the box' once I had fixed the monitor resolution issue.
Update 01/01/2011 - I decided to do a clean install of Ubuntu 10.10, just in case I had mangled it installing different testing packages going through bugs on launchpad. Having done the install and updating all packages using update manager, the touch screen is working perfectly - pressing the screen is the same as moving the mouse to the location and left click - Success!!!! (I hope!) - Ok, I'll give it a couple of days to make sure it doesn't beak before I say success.

Unfortunately there are problems with it. It seems that once you have done a 'drag' using the touch screen, x-windows thinks the mouse button is still pressed - moving the real mouse changes the shape of the drag region, and worse still, the real mouse button stops working. Sometimes the keyboard stops working too (both mouse and keyboard are wireless, using a USB receiver)

~~Investigations so far:~~

~~sudo apt-get install input-utils~~

~~provides a program called 'lsinput'. Doing~~

~~sudo lsinput~~

~~lists the touch screen as:~~

~~/dev/input/event6~~

~~bustype : BUS_USB~~

~~vendor : 0x408~~

~~product : 0x3001~~

~~version : 272~~

~~name : "PixArt Imaging Inc. Optical Touc"~~

~~phys : "usb-0000:00:1a.2-1/input0"~~

~~uniq : ""~~

~~bits ev : EV_SYN EV_KEY EV_ABS EV_MSC~~

~~Installing multi-touch utilities by doing:~~

~~sudo apt-get install utouch libutouch-grail-dev utouch-grail-tools~~

~~sudo apt-get install mtdev-tools~~

~~bzr branch lp:~utouch-team/utouch/mtview~~

~~cd mtview~~

~~make~~

Provides amongst other things a utility called mtview. Running that lets you draw pretty patterns on the screen using multi touch - it certainly seems capable of detecting and tracking two fingers at once.

~~Given this I think the touch screen is basically working, and it is the interface to x-windows that is the problem.~~

~~I'll have to work out how x-windows drivers work to fix this though....~~

I will update this section as I investigate further.

Web Cam
Not tested.

Trouble with Ubuntu and Windows 7 dual boot

2010-12-27T15:50:00.002Z

I just had a surprising problem with our new Packard Bell OneTwo. While it had apparently worked fine after installing Ubuntu on a spare partition in the hard drive, I booted it under its original Windows 7 operating system this morning to test benjamin's program with windows.
It worked fine, but after it had been switched off it absolutely refused to boot - no error messages or anything.
Went into the bios settings to take off quite boot and quick boot, but still no errors.
It booted off the Ubuntu live CD ok, and I could mount the linux root partition.

My best guess was that the boot loader was mangled, so I re-installed grub by doing:

sudo mount /dev/sda6 /mnt [sda6 is my root partition]

grub-install --root-directory=/mnt /dev/sda

This worked ok with no errors, and re-booting the machine gave the familiar grub boot menu. Ubuntu booted ok...we'll see if it works after I use Windows next time - I suspect it was Windows that killed the boot loader!

Framebuffer on Ubuntu

2010-12-26T22:01:00.001Z

I would like to be able to run my benjap application in a simple framebuffer rather than using X windows.
Unfortunately the 'out of the box' ubuntu desktop does not have a framebuffer (/dev/fb0) installed by default.
It looks like you need to load the fbcon and vesafb modules (found here):

sudo modprobe fbcon vesafb

Having done this /dev/fb0 appears. It appears to work as long as you run applications as root (a job for later to fix that!), but I have a problem with sound, which is that even running as root I get a "no available audio device" error when pygame.mixer.init() runs. Not sure how to fix this.....

Benjap - A program for children with Autism or Learning Difficulties

2010-12-26T21:27:00.000Z

I have decided to write a program for our autistic son (who also has severe learning difficulties) with my daughter, Laura.
We have bought a Packard Bell OneTwo all-in-one PC with a touch screen to use for it.
This is because Benjamin will not use a keyboard or mouse, but I hope that the cause and effect will be more obvious using a touch screen.

The basic program is written in Python using the pygame library. The structure is of a main application with big icons to select the other applications.
The various applications are (or will be):

Stars (a bit like the start of star-trek) you can change the origin of the stars that shoot towards the edge of the screen by pressing the touch screen.
Bounce: Lots of bright coloured balls bouncing off each other and the edge of the screen (not very interactive at the moment!).
Comet: Dragging your finger around the screen leaves a multi-coloured trail like a comet.
Fireworks: Pressing the screen launches a firework that explodes colourfully.

I have started a new project page for the application at http://code.google.com/p/benjap.

More to follow....

Packard Bell OneTwo with Linux

2010-12-22T16:25:00.003Z

Our son, Benjamin is autistic and has severe learning disabilities. Persuading him to do anything other than his current 'favourite' activity is really difficult. We took him to a light room today and I was impressed that he showed signs of interacting with a white square which lit up where you touched it. Laura and I started to think about making him one. It looks quite difficult because as well as the hardware to mount all of the lights we will need a touch sensitive panel and produce some software to drive it. For this reason I started to think about going for a purely software solution instead - just use a simple computer and a touch sensitive monitor.

I have contacted a couple of suppliers to see what they can do, but the first quote for a whole panel PC (which would be neat - no wires!) was very expensive £1620 for a 32" one.

An alternative would be to go for something smaller - HP and Acer make 23" all-in-one touch screen PCs for <£800k, which is still quite a lot, but getting better.

PC World and Currys are selling a 20" touch sensitive monitor for £180 which is more like the price I was thinking of, but I will need to add a computer, which will mean wires. I do have a little netbook which I don't use anymore, which could do the job though, so I may try this and just invest in a bigger one if he seems interested.

When I went to PC World today they did not have any of the monitors, but they did have a Packard Bell OneTwo all in one PC with touch sensitive screen. This was £599 so I decided to give it a go, because at least it meant no wires.

I downloaded Ubuntu 10.10 onto a USB memory stick and installed it in the spare 'Data' partition on the drive (I actually deleted the data partition and re-created a 10GB swap partition and the rest as a ext4 filesystem to use for the ubuntu filesystem. The install went ok, and grub appears to give me the option of booting windows (I haven't tried yet).

The most noticeable thing was that the screen resolution was not very good, and it was not using the widescreen aspect ratio. I fixed this by going to the monitor settings in the ubuntu system menu and un-clicking the 'mirror' option - it seemed to think it has two monitors, one of which is low resolution. Once I had de-selected that option I could select the resolution of the main monitor (called 'laptop' in this case). Now that is done it looks better, and the touch screen appears to be working.

For more information on my experiences using Linux on the Packard Bell OneTwo, see my other blog entries, in particular http://nerdytoad.blogspot.com/2010/12/ubuntu-linux-on-packard-bell-onetwo.html.

My daughter Laura and I are working on the applications for Benjamin to use - I am doing the code and she is on graphics at the moment.

The application is at an early stage, but the outline can be downloaded using

svn co https://ntmisc.googlecode.com/svn/benApp

There will be more about this application as we work on it and try to get Benjamin interested in it.

SVN Keywords

2010-12-12T14:01:00.001Z

I just realised that my townguide files are no longer automatically updating their SVN keywords to record the revision number of the files.
It turns out that this is because the keyword properties in svn is set by the client, not the repository, and worse still you can not apply keyword properties to directories - you have to do each file individually.
I got around this by using the find command to add keywords to all of the python files by doing:

find . -name \*.py -exec svn propset svn:keywords "LastChangedDate Rev Author" '{}' \;

Of course for files that do not have keywords set you have to edit them to add $Rev$ etc. into the comments.

Townguide - progress at last!

2010-12-12T09:13:00.000Z

You can't beat some really bad weather to help make progress with my nerdy jobs.
I have got Waldemar's django front end to townguide working (http://dtownguide.webhop.net), and have managed to make a few changes, including adding fancy tab things to the main map selection form.
There are still a few things to sort out:

The area selection on the map does not align properly with the mouse pointer - not sure why - must be something to do with OpenLayers projections...
Mapnik re-sizes the output map to match the bounding box, so townguide needs to check the actual map size to make sure the grid squares are right.
There is something funny about the output resolution - need to see what it is using - the output looks much higher resolution than I asked for, so I suspect there is a sum wrong somewhere!
Add the GPX track and waypoint plugins (they are just templates at the moment).
Add an option to suppress un-named ways - the 'None' in the street index can confuse people.
Reinstate other output formats - only 'poster' is working.

Townguide - where next?

2010-11-27T17:03:00.000Z

My townguide map rendering application is about a year old now, but hasn't changed much since last winter.
Waldemar did a lot of work on it for his Google Summer of Code project where he re-hashed the townguide renderer code to make it more modular and produced a django based web front end.
I have not done very well getting this working because I do not understand django well enough to sort out a few minor bugs in it.
Instead I have taken his re-hashed renderer code and developed it further to make it even more modular - as well as 'plugins' for different PDF output formats, there are plugins for the base map and for different map overlays (a grid, custom markers, gpx tracks etc.) [well nearly anyway]. It also uses mapnik2 so can now produce nice high resolution output to make it comparable to the much prettier maposmatic output.

I have been reading a book about django for a few days now (Python Web Development with Django by Forcier, Bissex and Chun) and feel that I should be able to understand both Waldemar's code and the maposmatic version, so I will have a go at the web front end now - there seems little point in developing my clanky php version with these two much slicker front ends available.

IBM ThinkPad X60S Middle Mouse Button

2010-10-24T08:39:00.000+01:00

Every time a new release of Ubuntu linux comes out, something breaks. Fortunately over time the somethings are becoming less and less significant!
This time it was the middle mouse button on my IBM ThinkPad X60S. I had got it working following the instructions here in version 10.04 - it just needed a configuration file in /usr/lib/X11/xorg.conf.d.
But it stopped working following the upgrade to Ubuntu 10.10.
It turns out that the xorg.conf.d location has changed to /usr/share/X11/xorg.conf.d.
Putting a file named 20-thinkpad.conf in that directory with the following contents got middle button scrolling working nicely again!

Section "InputClass"
Identifier "Trackpoint Wheel Emulation"
MatchProduct "TPPS/2 IBM TrackPoint|DualPoint Stick|Synaptics Inc. Composite
TouchPad / TrackPoint"
MatchDevicePath "/dev/input/event*"
Option "EmulateWheel" "true"
Option "EmulateWheelButton" "2"
Option "Emulate3Buttons" "false"
Option "XAxisMapping" "6 7"
Option "YAxisMapping" "4 5"
EndSection

There is talk that it should work by doing:

sudo apt-get install gpointing-device-settings

But it did not work for me (at least not just like that - tried re-starting xwindows, but still no scrolling - maybe there is some configuration needed, but I like the old way of dropping a file into a configuration directory!

OS Vector Map District with PostGIS - Take 3

2010-10-03T19:10:00.002+01:00

To recap:

The entire UK's VectorMapDistrict dataset has been loaded into my postgresql/postgis database using my vmd2pgsql script, which is based around shp2pgsql.
I created some stylesheets for mapnik rendering of the data based on the mapnik tutorial, but they came out with a transparent background and black lines and text, which is not what I wanted.

The rendering problem turned out to be quite easy - it is just that the mapnik tutorial is based on Mapnik 0.7, not mapnik2. To make it work with mapnik2 you need to:

Replace the bgcolor parameter of the map element with background-color.
Remove all of the CSSParameter business in the style definitions with simple name="xyz" constructs.

It is a shame that mapnik did not complain about these issues, even in debug mode, but never mind!

Adding the different linear features works ok - you see something of an improvement from the black and white version to something resembling a map:

So now all I need to do is add the areas (water, woodland towns etc.) to make it look extra pretty, and sort out what to plot at different zoom levels so it is reasonable.

Here I hit a problem. When I tried to add the settlement_area or naturalfeature_area items, mapnik crashed with an "invalid geometry" error, after taking quite a long time thinking about it! It is something to do with polygons not being closed properly. I suspect it is rounding errors, but am not too sure. One possibility is that I should have used the shp2pgsql option to just use integers, which would have avoided rounding, or I could try to fix the database. As it took 9 hours to import, and I don't know if integers will work, I'll try to fix it.

With much internet searching I discovered that you can check the validity of each geometric feature with the postgis st_isvalid() function. Running a simple select statement on the database like:

select gid from naturalfeature_area where not st_isvalid(the_geom);

takes a long time (about an hour I think) and gives me a list of invalid geometries, after pages and pages of warnings about things crossing over themselves.

It seems that one trick that people use to force gemetries to be valid is to use the st_buffer() command with the buffer radius set to zero. I checked it worked by doing:

select st_isvalid(st_buffer(the_geom,0.0)) from naturalfeature_area where gid=2342149;

(2342149 was the first entry in my list of invalid geometries) - Success - it returns 't' for true meaning the st_buffer trick fixes it.

Now to update the database. To correct the error I had to do:

update naturalfeature_area set the_geom=st_multi(st_buffer(the_geom,0.0)) where gid=2342149;

Note that I do not really know what the st_multi thing does, but I got errors about geometry constraint violations without it - I think that st_buffer returns the simplest type of geometry it can, and st_multi forces it to be a multipolygon, but I could be wrong!.

Now all I need to do is remember enough about subqueries to write a bit of SQL that does the correction for every invalid geometry - back to that well known search engine....

Well, I have set it going using the following to try to repair the geometries:

update naturalfeature_area set the_geom=st_multi(st_buffer(the_geom,0.0)) where gid in (select gid from naturalfeature_area where not st_isvalid(the_geom));

I think I'll leave it for a few hours....

ThinkPad X60s Battery Trouble

2010-10-03T09:11:00.000+01:00

I recently bought myself a second hand IBM Thinkpad X60s because I wanted a small laptop to use on the train, which has enough power to run postgresql/mapnik etc. so I can play with maps while I am living out of hotels (better on the body than too much food and beer, but not as pleasant!).
The problem I have is that at every charge cycle the gnome power manager in Ubuntu reports that the battery is less charged than before, and then it shuts down very quickly when running off battery because it thinks it is discharged.
I tried running it to completely empty by letting it suspend on low battery, then waiting for the battery to really die. The next charge was higher than before, but the same problem has repeated.
I just used gconf-editor to set the action on low battery to 'nothing' so that it will run until the battery is properly dead. The charge history is shown below:

Basically Ubuntu thinks the battery was dead nearly an hour ago, but it is still running!

I think this must be a problem with ACPI, but don't know how to sort it - a job for a very rainy afternoon. For now I will just leave the auto-suspend switched off and try to guess when the battery is about to give up.

Using VectorMapDistrict with PostGIS

2010-10-02T22:43:00.002+01:00

Well, vmd2pgsql took about 9 hours to import the entire UK into my postgresql database...No too bad.
Now I need to do something with it, because the mapnik stylesheets that I set up to use the shapefiles will not work - need to change it to use the postgresql database instead.
The things that need changing are:

Add the layers and datasources in XML (no need to do it in python now I don't have hundreds of separate datasources).
Convert the styles to use lowercase letters for the fields ('featcode' rather than 'FEATCODE') - postgresql seems to be case sensitive, and they have gone into the database lower case, but were upper case in the shapefiles.

Sounds easy, but I decided I don't want it to look like the OSM stylesheet, which is a bit garbled - I want a nice structure to make it easy to maintain. I could have used XML entities, but that seems a bit crude - define an entity, then 'use' it to actually 'call' it. Includes seem much more suitable....But to get includes working I need mapnik2. I have put a basic structure at code.google.com/p/ntmisc/vmdmap, but still have a few issues:

I have had to define the database parameters manually because entities do not seem to be working in the include files, which is no good!
The output is transparent background with black lines (see picture below), which is not what I asked for - not sure if this is a problem with the style file, or my build of mpanik2 - I'll have to run some tests on mapnik2 to make sure it works right.
But it's too late to fix it now - job for tomorrow!

More Rendering of VectorMapDistrict Data

2010-10-01T22:06:00.001+01:00

The nice people from Ordnance Survey have sent me a complete set of VectorMapDistrict data (which is on 6 DVDs!). Surprisingly they didn't even charge me for the DVDs or postage.
Now I have all that data I thought I'd better do something with it!

First step was to get it onto a nice fast disk, so I copied it onto the hard disk of my server...which took a while...

Then I tried to use my vmdmap.py program to render it all, but mapnik bombed out with an error as it was adding the various shapefile layers. I suspect that this is because every single shapefile is in its own layer, as it is a completely separate datasource, and I think I either ran out of memory or hit some mapnik internal limit. This means I can't use vmdmap.py to render the whole country, which is a bit of a shame.

To get around this I think I need to merge them into a single datasource. I don't know enough about manipulating shapefiles to do this, so instead am making use of shp2pgsql which allows you to import a shapefile into a postgresql database. I have written another program, based on vmdmap.py called vmd2pgsql which will scan through a directory tree looking for the various shapefiles in the vectormap district dataset, and importing them into postgresql. This gives a much lower number of tables - just one per shapefile name, but each one will have a lot of data.

It is importing now, so will see how long the import takes, then how well it renders. I suppose it should render ok because I have the whole UK OSM dataset in a single database and that works, but we'll see over the weekend if it ever finishes! The code is at my google code site.

Mapnik and OS_OpenData

2010-09-03T22:15:00.003+01:00

I decided to compare the quality of OpenStreetMap mapping to that which has been released by Ordnance Survey OpenData.
I started with VectorMapDistrict, which is a dataset provided as ESRI Shapefiles. It is provided as a large number of shapefiles, so setting up a mapnik stylesheet manually would have been a pain. Instead I separated the styles into an XML file, but added the layers using a python program which scans down the directory tree to add the various shapefiles to the map. The python code is vmdmap.py and the stylesheet is styles.xml.

Then I tried Meridian2. This is much simpler - just one shapefile for each sort of feature (A-Road, river etc.). I kept the same structure with the styles defined in an XML file, and the layers in python, but it could have all been done in a single XML file this time. The python code is md2map.py and the stylesheet is styles_md2.xml.

You can see the results below.

The most notable things are that Meridian2 is much simpler geometry, but it includes more road names than Vector Map District. The OpenStreetMap rendering is fancier because I used the standard OSM style, rather than the home made ones I used for the OS Data. I do like the rocks that appear in the VectorMap District rendering though - I will have to import it into OSM!.

VectorMapDistrict

Meridian2

OpenStreetMap

Mapping - OSM and Mapnik

2010-08-08T22:20:00.000+01:00

I have been playing with mapnik quite a bit lately - I decided I needed to learn how to use it better to improve my townguide program.
I have been very impressed with Lonvia's Hiking Map, which is an overlay showing hiking routes on top of a standard OpenStreetMap map, and thought I could do something similar.

My first effort was a map showing the locations of supermarkets (handy when we go on holiday so we know where to go to stock up on provisions) - It can be seen here.
Someone has imported a lot of powerline data from Ordnance Survey into my local area, so I tried a Power Station Map. These maps all have an 'about' link showing how they work.

I have produced a similar one to show the use of Ordnance Survey OpenData in the OpenStreetMap map of the UK, which can be seen here.

More recently I have wanted to produce maps that include contours, and also allow me to plot GPX traces to show where we have been. To do this I have imported the SRTM data for the UK into my postgresql database using the technique described here. This allows me to render an Openstreetmap map with contours, which is useful for the countryside.

Adding a GPX trace over the top was a bit difficult - I wanted to do it in python to save hard coding the path to the GPX file in the mapnik style file, but I haven't managed to do that - I ended up adding a GPX layer to my mapnik style file using the mapnik ogr plugin. The result can be seen here.

The Irritating thing about Android

2010-04-25T08:17:00.000+01:00

I have recently got an Android phone so that I can write software for it without the irritating security issues you get with J2ME on 'normal' phones (boxes popping up saying 'do you want to let this program do this?' etc.).

I was very surprised that some things that most phones do 'out of the box' are a real pain in Android, that makes me think this is still a platform for gadget enthusiasts rather than end users.

These include:

* Sharing Contacts: All of the phones I have seen with bluetooth let you send contacts from one phone to another over bluetooth. Not Android - I ended up adding my contacts from my old phone to Google Mail manually. This made them appear on the Android one.
* Using the phone as a modem: My Sony Ericsson W890i just works - plug the phone into a computer and it appears as an ethernet card - dead easy (I think I had to select this as an option in the phone menu, but it was just a simple menu action. The nearest thing I have got with Android is a program called 'Proxoid' to run on the phone. You then have to use the ADB program that comes with the Android Developers SDK to forward packets to the phone, and set up your browser to use a proxy. A long way from an end user experience. I'll do a separate post on this so I can remember how to do it next time I am stranded in Spain and my T-Mobile broadband dongle won't work!

Reprojecting Ordnance Survey Raster Data

2010-04-03T21:34:00.000+01:00

I have been looking at the Ordnance Survey StreetView data that was released to the public on 1st April.
This has got me thinking about a previous project that I failed to make work - I have some old maps of my home town and wanted to be able to display them on the web and switch between them so you can see what changes from year to year - I have maps from 1870 and 1914. I want to use OpenStreetMap as the recent data. Now StreetView is available I can use that too for comparison.

The problem is projecting them onto the same coordinate system - I failed miserably last time, but my two scanned old maps were on the same scale, so you can see them at (http://maps.webhop.net/oldmaps/openlayers.html).

The maps are in Ordnance Survey projection, which has the code EPSG:27700. The downloaded OS data has the origin specified in metres northings and eastings.
You can translate this into the same projection as OpenStreetMap (EPSG:900113) using:

gdalwarp -s_srs EPSG:27700 -t_srs EPSG:900913 *.TIF nz.tif

Check the projection information with

gdalinfo nz.tif | more

Ordnance Survey Data Released

2010-04-01T20:38:00.002+01:00

Ordnance Survey have released quite a lot of data for free use (http://www.ordnancesurvey.co.uk/oswebsite/opendata/).
The most interesting sounding data (meridian) is a 2Gb shape file, so I chickened out of trying to do anything with that tonight.
Instead I downloaded the 50k Gazetteer and have put it into a postgresql database so it can be queried to look for things.
Work in progress is at http://www.maps2.webhop.net/openos.
It sort of works (just not very useful yet - you can do http://maps2.webhop.net/openos/gaz/www/doSearch.php, and it gives you a list of all of the items in the database containing the string 'hartlepool' - just need to transfer them back to the browser and plot them on a map now....

Android User Interfaces

2010-03-07T22:33:00.000Z

I am working on a colour processing program for android (more later once it works). A fundamental part of this is to create a custom view to use in the user interface - it will eventually display a colour wheel, but it is nowhere near that yet!
The problem was that I am creating a subClass of View, which draws something on the screen.
I then simply added it to my UI XML description and tried to use it.
I got an InflateException, which was a bit of a surprise.
It turns out that I made a mistake - I had overridden the simple View(Context) constructor, which would be fine if I was adding the view directly from a program, but to use it from XML you need to override another constructor, View(Context, AttributeSet).

Now I have done that it works. Sorry this makes no sense unless you are used to Android things, and if you are used to Android, it is probably obvious, but this will help me next time I forget! I found this out here.

GPX Route Editor

2010-03-07T19:53:00.000Z

I was trying to plan a family cycling trip in a rush this morning and struggling to find a map to
use (opencyclemap is no good because I like to go to un-mapped places so I can add them to the map).
I finally remembered that ages ago I wrote a little javascript application to display an Ordnance Survey map, and let you put route markers on it which you can then put onto a Garmin GPS receiver to use for navigation - I used that to record where I intended to go, so I knew where there was supposed to be a bridleway...
Anyway I thought I should add that program to my repository in case anyone else is interested. There are a few things to do to it - see the README file in the repository directory - The source code is in my ntmisc code repository under gpxedit.
You can find the program working at http://www.maps.webhop.net.

I will do a bit more work on this when I get chance. I always intended to convert it to OpenLayers rather than multimap's proprietary API, and at the moment you have to copy and paste the GPX file into a text editor to get around the irritating web browser security - need a way of getting around that - the simplest would be to add an 'echo' server side application that would allow you to download the GPX file from the server, but that feels like brute force and ignorance!

Android Development on Ubuntu

2010-02-17T22:35:00.000Z

I just got an LG GW620 android mobile phone. I want to use it for mapping (http://www.openstreetmap.org). There is an android editor called Vespucci available, but it needs some work to make the UI easier to use. I thought I would have a look at the source code to see if I can work out how to modify it.
The android developer web site says that it is best to use the Eclipse IDE, but when I tried I got all sorts of errors about un-satisfied dependencies.
Fortunately someone had posted some instructions to get it working on Ubuntu (http://ubuntuforums.org/showthread.php?p=8842292#post8842292).
Now I have followed this I have a working IDE to develop Android applications, and have got a basic "Hello World" application working. I have also downloaded and complied the Vespucci source code, and run it on the SDK emulator.
Just need to understand how Android UIs work now....

Progress Mapping Hartlepool

2010-02-14T22:36:00.001Z

Benjamin and I are making progress mapping Hartlepool for OpenStreetMap.
I have just run a query on my database to see how many kilometers we must have cycled to produce the map. I was really quite shocked that it was over 550 km! No wonder I am tired...
Progress with the map and the breakdown of the distances are shown on the Hartlepool OpenStreetMap page.

Townguide Web Service Working

2010-02-02T20:47:00.000Z

Well, I think the townguide web service is working now (http://www.townguide.webhop.net).
There is a simple queue so it only renders one job at a time, and you can view the progress of your job through the queue.
The source code and early documentation is stored at http://code.google.com/p/townguide.
The main thing to fix now is the output resolution - if I increase the resolution (in dpi) of the output map too far, mapnik changes zoom level and reduces the size of the text, so it is no easier to read - I need to understand mapnik style sheets now....

Town Guide Progress

2010-01-03T22:43:00.000Z

The good thing about so much bad weather over Christmas was that I could do some nerdy work rather than going out walking or cycling.....
The town guide Python project has therefore made good progress. The program can now generate a PDF 'poster' that can include a street index, and labelled features from the map database (pubs, shops, tourist attractions etc.). A typical example is shown at http://www.townguide.webhop.net/example1_hartlepool.pdf.

I have set up a very rudimentary web service using a PHP script to call townguide.py to generate a town guide on demand at http://www.townguide.webhop.net. It is running on a very small computer, and there is no queue, so there is no saying what will happen if more than one person tries to use it at once - that is another job!

Animation

2009-12-15T21:48:00.001Z

Laura and I have been experimenting with stop-motion animation - set up a camera on a tripod with a remote shutter release, then position a scene in front of it, and change it by a very small amount between shooting frames.
10 frames per second seems to work ok for what we are doing (and you get more video for your money, so it is a bit more rewarding!).
I am using mencoder to turn the jpeg images from the camera into a mpeg4 video using the following:


mencoder mf://*.JPG -mf fps=10 -o leaves.avi -ovc lavc -lavcopts vcodec=msmpeg4v2:vbitrate=800 -vf scale=640:480

This produces an mpeg4 encoded .avi file, scaled to 640x480 pixels.
You can then run mencoder again to add a mp3 audio stream.
The results can be seen at http://flickr.com/jones139.

Town Guide (again)

2009-11-29T20:03:00.001Z

Progress with the Town Guide program stalled a bit when I went on holiday, then was rather busy at work for a while. I am back to thinking about it now.
The current version has two output modes, HTML and PDF - both are quite crude, but work.
The PDF output uses LaTeX, which seemed like a good idea at the time, but I am struggling because LaTeX thinks it knows better than me what the output is supposed to look like - it keeps putting the tables etc. at the end, rather than where I ask for them.
This means I have two options - either do battle with LaTeX to sort it out, or abandon it and go for a more 'direct' PDF production.
I think that as the output I am producing is short and simple, I may as well go for 'direct' production. I am going to have a try with ReportLab's python library (http://www.reportlab.org/oss/rl-toolkit/guide/).
I'll go and read the instructions and see how it goes....

Free Town Guide

2009-10-04T23:06:00.001+01:00

For ages I have been intending to write a program to produce a nice guide to a town or area based on OpenStreetMap data.
I finally decided that I knew enough about postgresql and python to have a go.
The result is a program called townguide.py. The idea is that you load the openstreetmap data into a postgresql database (like you do if you want to render it using Mapnik), then produce an xml file to tell townguide.py what you want rendering. It then goes and produces the nice output for you.
Version 0.1 is working and is on SVN. At the moment it only produces html output (because that is easy), and it is not very pretty.
The next stage is to make it produce postscript to give nice printable output on a defined paper size.
An example of the html output is at http://hartlepoolmap.webhop.net.

Having looked at the output it looks like it will be quite useful from a mapping point of view - if you know the area you can review the features that are summarised and look for missing banks, supermarkets etc.

Using a Graphics Tablet in Ubuntu 9.04

2009-09-19T17:55:00.000+01:00

I have a cheap Trust graphics tablet that I used to use for drawing in Inkscape or Gimp.
It stopped working at some point (Can't remember which Ubuntu version), but I decided I needed to use it again, because drawing with a touchpad on a laptop is a pain in the thumb...
It was surprising that it didn't just work when you plug it in, because I'm sure a previous version of Ubuntu behaved like this.
I found the instructions at https://help.ubuntu.com/community/AiptekTablet.
You basically have to install the Aiptek xorg driver and put a configuration file in a directory under /etc/hal. I wondered where the X11 configuration was now that Xorg.conf is so small....

Anyway, it works now...

Progress with IP Camera Viewer

2009-09-06T22:07:00.000+01:00

The Edimax IP Camera seems to work fairly well, but the video stream seems to get interrupted every now and again, which makes the player crash - vlc just hangs, and mplayer exits.
I decided that exiting was better so I have written a simple pyGTK front end for mplayer to play the screen. It also includes a wireless network link quality monitor to help understand why it is not working - it is here as 'bentv'.

Note that bentv uses a library that I have started work on (named ntpylib for want of a better name) - so far it only contains one class, 'prefs', which deals with saving and loading of simple key/value data from XML files. It includes a dialog box that GUI programs (such as bentv.py) can use to allow the user to edit the data. prefs.py and prefs.glade need copying or linking into the bentv directory to make it work - there is no clever installer!

Embedded Linux Firmware Binary Files

2009-08-23T22:30:00.001+01:00

I have always struggled to understand the format of the binary files that store the firmware upgrades.
They tend to be a mixture of some simple text labels, a kernel image and one or more mountable filesystems.
I have written a little script that goes through a file in a brute force (one byte at a time) way and attempts to mount it as various filesystem types (script stored here).
I used the information found by this script, and the source code provided by Edimax to work out some information about the internals of the IC-3010WG IP Camera - it is written up here.

Practical Experience using the Edimax IP Camera

2009-08-22T16:15:00.001+01:00

We were away on holiday last week so I got to try using the Edimax IP camera 'for real' as a monitor to keep an eye on our disabled son - known as 'BenTV-ng' (next generation) to distinguish it from 'BenTV' which was the old analogue system I used to use.

I gave up on using ad-hoc networking and instead took my wireless access point with me. This had the added advantage of improving the range of transmission because I could arrange for the access point to be between the camera and the computer. The computer I used was a Toshiba netbook running Ubuntu Linux.

The first thing I noticed was that when I powered everything up, nothing happened. At first I thought it was because the access point needed to see a wired connection as well as a wireless one, because when I connected the computer to it using a wire things started to work. I soon realised that the range was awful - it stopped working as soon as you were outside of his room. This turned out to be my fault because I had also switched on the analogue camera, which transmitted on the same frequency as the wireless network, so the noise level was huge. Things got better when I switched off the analogue camera.

At first everything appeared to work - vlc rtsp://admin:1234@bentv/ipcam.sdp played the video stream with sound.
I soon noticed that if the network got interrupted vlc froze, giving an apparently good picture, but no movement, which was no use to me. Instead I changed to mplayer - at least if that got into trouble it crashed and closed the video playback window so you knew something was wrong.

I set up a simple shell script that loops indefinitely starting mplayer then when mplayer exits it re-boots the ip camera, waits for 30 seconds and starts mplayer again.
This approach worked pretty well - the 30 second interruptions during the re-boot were not usually too troublesome.

We did find some problems when the video stream only worked for a few seconds before mplayer exited and the re-boot sequence started again, which was no good at all. The only way I found to cure this was to power off the camera all together - a software re-boot via the web interface did not cure it. I'll set up a little test at home now to see how long it takes to get into this state, and whether it makes a difference if you use wireless or wired networking.

The code to achieve this is stored at http://code.google.com/p/ntmisc/source/browse/bentv/.

The most up to date information I have on using this camera with linux can be found at http://ic-3010wg.webhop.net.

Ad-Hoc Wireless Network Troubles

2009-07-24T21:57:00.000+01:00

Now that I have the Edimax IP Camera working I want it to connect to my Toshiba NB100 netbook computer as simply as possible - the idea is to use it when we are away on holiday so I don't want to have to set up a whole network with an access point etc. Instead I just want to set up an ad-hoc network between the computer and the IP Camera.
According to the instructions this should be easy - I tell the IP Camera to connect to an ad-hoc network called 'bentv', and then use NetworkManager in the Ubuntu desktop of the netbook to create the ad-hoc network.
This works fine on my main laptop, but fails on the netbook.
It turns out that NetworkManager is not capable of putting the wireless card into ad-hoc mode.
The work around was to have the script to start the VLC viewer shutdown network manager and set up the network manually as follows:


#!/bin/sh
/etc/init.d/NetworkManager stop
ifconfig ath1 down
wlanconfig ath1 destroy
wlanconfig ath1 create wlandev wifi0 wlanmode adhoc
iwconfig ath1 essid bentv
iwconfig ath1 channel 1
iwconfig ath1 192.168.1.35
iwconfig ath1 netmask 255.255.255.0
ifconfig ath1 up

vlc --fullscreen rtsp://192.168.1.17/ipcam.sdp

/etc/init.d/NetworkManager start

This is based on information I found here.
This seems to work, but the wireless networking is a bit iffy after the script exits - I might have to do the wlanconfig destroy bit again to put it into infrastructure mode.

More on Edimax IP Camera

2009-07-12T10:21:00.001+01:00

I think we're getting somewhere. Searching around for references to IP cameras and mplayer yielded this thread which suggested the following:

mkfifo fifo.mpeg4
wget -q http://admin:1234@192.168.1.117:4322 -Ofifo.mpeg4 &
mplayer -V -demuxer 35 fifo.mpeg4

This seems to work ok - there is a bit of a delay in the video, but probably ~0.1 sec, which is acceptable.
Progress!
Just need to get sound working now....

Well, I did a bit more hunting around in the camera web set-up and realised I had missed a section about RTSP. The default was for RTSP to be enabled on port 554 with a rtsp path of ipcam.sdp.
When I tried

mplayer rtsp://192.168.1.117:554/ipcam.sdp

I got pictures AND sound!
I had tried rtsp before, but had missed the ipcam.sdp bit. Mplayer complains that my little netbook is too slow to play the stream, so there is a bit more work to do, but it looks promising. I'll try VLC now.

The most up to date information I have on using this camera with linux can be found at http://ic-3010wg.webhop.net.

Edimax IC-3010Wg yet again

2009-07-11T22:26:00.000+01:00

Still struggling with my IP Camera. Although mplayer can sort of play the mp4 stream, it does not seem reliable and takes a long time to start to play, so I do not think it is working correctly. I think I'll try the Edimax support, but have upgraded from version 1.20 to 1.25 of the firmware first, because they are bound to ask if I am using the latest software....

Edimax IC-3010Wg again

2009-07-09T21:42:00.000+01:00

I have made a bit more progress getting my Edimax IP Camera working, and detailed what I have learned at http://ic-3010wg.webhop.net.

The main problems are still that although I can download an MPEG4 stream using wget, neither mplayer nor vlc will play it directly, which is no good.
The other issue is that sound still does not work.....

Edimax IC-3010Wg IP Camera and Linux

2009-07-07T21:45:00.001+01:00

I have just got an Edimax IC-3010Wg IP Camera. I intend to use it as a replacement for the analogue video system we use to keep an eye on our disabled son.
I was rather disappointed to find that it does not work with Firefox - you need an Internet Explorer ActiveX control to view the video images. This means it will not work on my Linux computers as supplied by the manufacturer.

What works:

The camera gets its network configuration by DHCP (despite what the quick start guide says) so by checking the DHCP deamon log you can find its IP address.
You can access its configuration screen with a web browser (default username admin, password 1234)
Receive a snapshot picture using http:///snapshot.jpg
Download a mjpeg video stream using wget http://admin:1234@192.168.1.117/mjpg/video.mjpg. (I don't know how to display this in real time though - mplayer doesn't seem to work).
Some other things are detailed here.
Someone has written a tcl application that works with a similar camera, but not this one (see here).

I think I am going to end up writing something based on the tcl application shown above to display live video with sound, which is what I need. If anyone knows an off-the shelf one, please let me know!

The most up to date information I have on using this camera with linux can be found at http://ic-3010wg.webhop.net.

Re-Programming MusicPal

2009-07-07T21:08:00.000+01:00

I have not made any progress with the MusicPal since Christmas, but someone else has been busy and made a good start on some alternative software to run on it, which he has published here http://musicpal.v3v.de/.
Well done Marco - you have got much further than I have!
Giles also made progress with sound output and left his code in a comment to one of my earlier posts here.
Maybe it would be worth starting a project on Google Code or somewhere so other people can help develop it?

A General Purpose Network Solver?

2009-06-28T21:06:00.000+01:00

I often get frustrated if I want to solve a simple fluid flow or thermal problem that is in the form of a network with branches, parallel paths etc. These tend to be a bit to difficult for a "back of the envelope" solution, which is always my preferred approach, so you end up writing code to iterate to find solution.

The easy way would be to use a commercial package like Aspen Hysis, SINDA or ESATAN, but these are very expensive for the once a year or thereabouts that I need to solve something like this. It would be nice to have a simple, low cost solver available.

I thought there would be plenty of open source ones available, but I have not been able to find a simple network solver, so I am going to resort to writing it myself. I did do one once before, but I seem to have lost the source code....It never got finished properly anyway.

I have started the project at http://code.google.com/p/openproc. There is no code worth talking of there yet, but the Wiki describes the sort of design I am thinking of. Any offers of help greatly appreciated!

More Ubuntu 9.04 Sound Problems

2009-06-09T22:34:00.000+01:00

I 'accidentally' wrecked my gnome desktop (no menus, toolabrs etc.) - I had been doing a bit of tidying up and must have uninstalled something important. I could not decide what, so re-installed the whole ubuntu-desktop package.
This came with pulseaudio again, so sound stopped working like last time.
Decided to invest an hour in sorting it. No real success searching on the web, but I did find this article - http://ubuntuforums.org/showthread.php?t=962695. The author provided a little script to update the ALSA drivers automatically, and quite a few people reported that this cured the sound for them.

I am always wary about running unknown programs as root, but the script looked safe enough - just an automated way of downloading the ALSA source packages and running 'make install' etc. so I trusted it.
It ran and it installed ALSA 1.0.20, compared to version 1.0.18 in the Ubuntu package.
To my surprise, when I re-booted the computer, sound is back!

Ubuntu 9.04 Sound Problems

2009-05-11T22:06:00.001+01:00

I upgraded my Fujitsu-Siemens laptop to Ubuntu 9.04 from 8.10 a few days ago.
Everything worked fine except for sound. Sound went from working perfectly in 8.10 to being quiet, and awfully crackly in 9.04.
If I updated the gnome settings to use OSS rather than ALSA it worked ok, except flash movies still made crackly noises (I don't think flash can use OSS).
I messed about with the gnome mixer without luck, except I did notice that changing the PCM volume did nothing (which is wrong), but changing the 'beep' volume did alter the sound volume (Along with crackles).
I don't know why the beep and PCM are the wrong way around, and don't have time to work out how sound works properly, so I went for brute force and ignorance - I had heard a suggestion that Pulseaudio might be the problem, so I replaced pulseaudio with esound, and it all seems to work now...
Therefore, I don't know what the problem was, but it was cured by getting rid of pulseaudio...

What has been happening?

2009-04-15T20:56:00.001+01:00

I've been forgetting to update this with what has been happening.

WherewasI now has a Python/GTK front end that allows you to plot your GPX traces on a OpenStreetMap map to see where you were, as well as altitude profiles etc. It could still do with a few more features, but it does most of what I want at the moment.

I have decided to make a concerted effort to finish off mapping Hartlepool for OpenStreetMap. The main problem is that am better at cycling around collecting GPX traces than I am at recording the street names. This means I end up with lots of roads with no names - no use if anyone wants to search for an address....To do this I decided to use audio mapping by recording audio clips with my mobile phone. The clips are GeoTagged using data from a little bluetooth GPS receiver. I use GPSMid on the mobile phone. Unfortunately it does not produce the GPX file, or the audio clips in the correct format for the JOSM OpenStreetMap Editor. I wrote a little python script to do the conversion so you can see the positions of the audio clips in the JOSM Editor (See the OSM GPSMid Page for details.

GPX Track Analyser

2009-02-22T22:29:00.000Z

I have realised why I never finish anything - it is far too easy to get distracted by another little job.
This time my daughter asked me how far we had climbed after a very hilly bike ride on our holidays.
I was a bit disappointed that I could not give an instant answer, as I had a track log from my Garmin Etrex GPS device, which includes elevation data.
Therefore that evening I wrote a little program in Python to analyse GPX files and provide some basic statistics on the route - this can be found at http://code.google.com/p/wherewasi.

Integrating Windows XP and Ubuntu Linux

2009-01-27T20:30:00.001Z

There are not many things that you can not do on Linux that you could on Windows, but there are a few, in particular there are some software packages that I can not manage to get running under Linux using Wine. These are:

Microsoft Office (I know OpenOffice is a good alternative, but it is different to Excel and Word that I use all the time at Work, and switching is difficult. This is especially a problem for the rest of the family that are not as used to switching between operating systems as I am).
Family Tree Maker (My Mam is used to this program, and the Open Source equivalent (can't remember its name) just wasn't as good as far as she is concerned.
Microsoft Home Publishing (My daughter uses this to make cards etc. Although you can use Inkscape or Scribus to do the same under linux, it is not as simple - maybe someone else knows an Open Source equivalent?).

When I heard about VirtualBox I thought it was too good to be true - you are supposed to be able to run Ubuntu Linux as your main system (Host), but run Windows XP as a 'Guest' in an emulator - as far as Windows is concerned it is running on its own stand-alone machine, but VirtualBox is emulating a real computer in software. This sounds like an awful job to do, and I expected everything to grind very slowly, but as my computers now have lots of memory (2GB and 4GB for my Laptop and the Kids desktop respectively), I thought it might be possible.

Installing virtualbox in Ubuntu is easy:
sudo apt-get install virtualbox-ose
Note that Ubuntu has another package "vboxgtk" which is an alternative front end to virtualbox. I struggled this - the windows installer crashed when you used the keyboard, so I just stuck to the normal virtualbox program.

You then start it by running virtualbox as a normal user. You then create a new virtual hard disk on which to install windows, which is actually a normal file in your home directory.

You create a new 'virtual machine' by associating the virtual hard disk with it, and giving it access to your host system cdrom. I put the Windows CD in the host computer CD drive, and selected 'run' to start the new virtual machine. It booted off the CD and windows installed in the new virtual hard disk.

Then the problems started. The main one was that when I tried to boot windows off the new virtual hard disk it hung when it loaded a module called agp440.sys. This was odd because it was the recovery disk for the version of WIndows XP that runs nicely on the real computer. It turned out that the problem was that the recovery disk assumes the hardware configuration for the computer - in particular the graphics card (AGP), and the VirtualBox virtual computer has a different graphics card, which caused the crash. I could not work out how to get around this, but I found another Windows XP CD, and this did a normal windows installation, where it picked the drivers for the particular hardware concerned. This then booted nicely and you get a window, which contains the WinXP desktop.

Using windows was a bit of a pain in this configuration - the desktop was very small, so it was difficult to fit application windows on it. Also windows kept grabbing the mouse and keyboard, and you have to press a secret key (the right Ctrl key) to release it back to the host operating system.

When I read a bit more about VirtualBox I found that it should work 'seamlessly' so you do not have a separate windows desktop - the Windows XP windows sit on the same desktop as the Ubunto Gnome ones. To do this you have to use the web browser in Ubuntu (the host operating system) to download the guest additions package as an iso disk image, and put it in /usr/share/virtualbox with the filename "VBoxGuestAdditions.iso". You can then install the guest additions package using the menu at the top of the windows desktop window.
Once you have installed the Guest Additions package you can set the system into seamless mode using a menu, or pressing -L. This gives you a windows start menu bar at the bottom of the screen, in addition to the usual Ubuntu ones.

There are menu options to give the guest operating system internet access - selecting the 'NAT' option seems to work, and to give it audio access (I selected the 'ALSA' option, which seems OK.

You can share files between the guest operating system and the host one by selecting a 'shared folder' on the host system - this appears as a network share in Windows.

Things appear to work now - MS Office installs and runs. Haven't tried other things yet, and need to work out how printing works, but it looks very promising.

MusicPal USB Socket

2009-01-25T22:11:00.000Z

Other people have reported that it is easy to add a USB socket to the MusicPal. As its internal memory is very limited I think this will be useful for software development, so decided to give it a try.
Sure enough, you can open the case easily by removing three screws and pulling the knobs off the front panel. There are clips on the right hand side of the front panel, so you have to start to open the case from the left (speaker) side and pull the front panel to the left a bit to release the clips. I think it would have been possible to remove the front panel without removing the warranty seal on the screws behind the speaker - just removing the middle screw is probably enough to get the front panel off.
I bought a USB socket from RS Components and it soldered easily onto the circuit board, then just had to cut the back panel blank out and it went together nicely. Now if you plug a USB memory stick into it, an extra menu item appears on the Freecom Nashville application menu, and you can play music off it. I haven't tried to disable writing to the memory stick as they did here because I think I will want to write to it from the musicpal. We'll see....
Haven't tried running software off it yet though.

Nearly Broke It

2009-01-19T19:41:00.000Z

I'm trying to get back into fiddling with the MusicPal internet radio after I got distracted by a few other things instead over Christmas.
The first thing I noticed was that I had broken the manufacturers' software - it appeared to boot up ok, and I could play files of my server, but internet radio didn't work, and neither did the clock on the device.
I tried fiddling around with the settings via the MusicPal's menu, but couldn't find anything wrong.
It turned out that I had filled up the filesystem on the device, so it did not have enough storage space to create /etc/resolve.conf or similar, so the internet DNS resolution did not work. Once I deleted my experimental files, it all started to work again....

This got me thinking that to do any serious development on this, I will need more storage space - deleting the manufacturers' software altogether is a bit of a big step given the rate I work on these things. The simplest would be to persuade it to boot over the network, just like the MediaMVP - this means I can easily change the root filesystem image, and if it doesn't work it is trivial to put hte original back. The trouble with this is that I will have to update the uboot configuration - this would be easy if I had a serial console for the device, but I don't. I think there is a real possibility of turning it into a brick if I get this wrong.....
The less intrusive alternative would be to compile a kernel with NFS support and have it nfs mount a filesystem with my software on it. This is less drastic, but I will have to make sure I can do the emergency re-flash of the firmware in case I mess it up. A job for later.
The simplest would be to put my files on a USB memory stick and use that - people seem to have added USB support to the musicpal quite easily, so I had a look inside. Sure enough, there is space for a solder connection USB socket, and a punch out panel in the back of the case. I think I will get a USB socket and try this first to see how it goes - I can't find one in anything I can dismantle at the moment, so will have to buy one.

Media MVP Slideshow

2008-12-15T22:31:00.000Z

I want to create a simple slideshow viewer for the Hauppauge Media MVP.
The user interface has to be simple, requiring no intervention other than powering up, because it is for our Autistic son and he does not press buttons. I am intending to make a really big bright, easy to press button for him to power on the unit, and it will shutdown after a set period.

Mvpmc almost does what I want, but you have to use the remote control to select an image to display before it will go into slideshow mode. I am not keen on the mvpmc code - it has lots of 'modules' to do different functions, but the code is all interwoven, so it takes a bit of un-picking. Therefore I thought I would go for something simpler. Fbi is a simple frame buffer image viewer which should do the trick. After much messing about I got it to compile, but unfortunately it complained that the MVP /dev/fb0 was not behaving like a standard linux frame buffer. I gave up on fbi at this point and decided that un-picking the mvpmc code would be easier - at least I know it is capable of displaying images.

I updated main.c and gui.c in mvpmc to add a new command line startup option --startup slideshow:file=.

Cross Compiling for MediaMVP

2008-11-30T20:28:00.001Z

I have a couple of Hauppauge Media MVP media players and I want to turn one into a customised photo view for our son - it will have to be very simple because he will not press buttons, so I want it to just start as soon as it is switched on, and provide a customised "Hello Benjamin" screen as it boots.

The mvpmc project should provide most of what I need, so I just wanted to build a cross compiler and have a play. I downloaded the binary distribution and got it running ok (needed a dhcp and tftp server on my little computer in the atic to do that), and the binary distribution of mvpmc works fine.
I downloaded the source. It says it is supposed to build its own toolchain if you do not have one, but this did not work - it ran for a long time, then complained about bits of the toolchain not being there.

I thought I was too clever for that and I would just do it myself using crosstool. This failed with obscure syntax errors, which was hard to believe.
I eventually read that building cross compilers with recent versions of gcc is troublesome, so I installed an old one instead (gcc.3.4.6 from the Ubuntu repositories). It worked ok and built a PPC-405 cross compiler for me.
Unfortunately my "hello world" program compiled ok but would not run - the MVP responded with "not found" when I executed ./hello_world. I think this is because the MVP is using uClibc, but I had just build a cross compiler with gLibc. "oh dear...".
Next tried the Embedded Linux Development Kit (ELDK). Again "hello world" compiled ok, but got the same "not found" error.

Looks like I will have to find out why the mvp toolchain build failed.

Hunting a very long way back in the build log showed that the first error (that I spotted) was when it tried to download the linux headers to compile against. It looks like the uClibc website has been rearranged and the ones that the MVP needs have been moved to a "old-releases" directory. Downloading the file and putting it in ~/downloads like the mvpmc build script does resulted in the build running apparently ok - still hasn't finished though....

OK, it's finished now, and the new toolchain compiles a "hello world" that runs ok!

Now I just have to strip out the mvpmc code down to a very simple viewer, and it should work.....sounds easy, but it's getting late now...

autoconf / automake / aclocal

2008-11-28T22:00:00.000Z

I am trying to build Music Player Daemon for the FreeCom MusicPal. I have written an output plugin that I think will drive the hardware, but now I am having trouble compiling this into the MPD source code. I have never really understood the GNU configure scripts (./configure and all that), so I have probably messed something up.
I have added new bits to configure.ac, Makefile.in etc. But now what do I do?

As far as I can tell from the simplest tutorial I can find suggests that I should just be able to edit configure.ac and Makefile.am then run autoconf, but it complains that the files are designed for automake version 1.6, and I have 1.10. The error suggests running aclocal, but this just leads to more errors....

Progress with MusicPal Sound Programming

2008-11-23T22:54:00.000Z

After a couple of hours of using strace to try to understand what Nashville was doing, and looking at the "base_audio" directory in the MusicPal kernel source directory, I have finally got it to make a noise.

Basically you have to use /dev/audio and send some ioctl's to initialise the hardware, and set up the graphic equaliser. You can then write raw data to /dev/audio just like you would with OSS. In fact, I used the OSS sinegen.c example program as the basis and just altered the ioctls. http://www.musicpal.webhop.net/audiotest.c is my sample program (it needs audio.h from the MusicPal kernel source code to compile) - it makes a continuous tone come out of the MusicPal speaker, which is progress! There are some comments in the code to show what it is doing (as far as I can work out anyway!).

MusicPal Debug Tools

2008-11-23T19:20:00.000Z

I am trying to get sound playback working in my own code on the MusicPal. I have been advised to use a utility called 'strace' which allows you to monitor what system calls an application is executing - this should allow me to see how nashville initialises and uses the sound system on the MusicPal (http://forum.freecompromo.com/viewtopic.php?p=22633#22633).

I downloaded the source code for strace from sourceforge. It uses the standard gnu ./compile system to configure and compile it.
I got it to compile for the MusicPal using:

source ~/alinux/setvars; CFLAGS=-I~/alinux/linux/include ./configure --host=arm-linux
make

It compiled ok, and won't run on my PC, which is a good indication that it has cross compiled.
The problem is that I can not transfer it to the MusicPal because the executable is about 290kB, and there is only 176kB free on the MusicPal /home directory.

Oh Dear....I think the MusicPal has quite a lot of RAM - I'll see if I can make a RAM disk - either that or I have to get NFS working on the MusicPal so I can mount a decent sized directory, or even more drastic, pull the MusicPal appart and put a USB socket in it to use a USB disk - easier in the long run, but this is supposed to be a present for my wife, so I'd better not.

Silly me - there is already a ram disk mounted as /tmp....

MusicPal Sound Programming

2008-11-22T19:41:00.000Z

The MusicPal has a few devices that look like they are to do with sound:

/dev/dsp
/dev/mixer
/dev/audio

I know very little about audio on Linux, but from a quick read I thought that these looked like OSS devices. I downloaded the simple OSS sine wave generator example singen.c. It runs ok on my laptop, and cross compiled ok, but when I tried to run it on the MusicPal I got a "/dev/dsp: No such device or address" error. This makes me think that maybe it is not OSS - I'm going to have to learn more.

A couple of hours later and I might be getting somewhere. I tried doing fuser /dev/dsp and fuser /dev/audio. These both returned nothing with the main application (nashville) sutdown. Re-starting it with /etc/init.d/mainapp start, then repeating showed that /dev/dsp was still not used, but /dev/audio was used by the 12 running versions of /bin/nashville. This has persuaded me that Freecom use /dev/audio, not /dev/dsp.

So, what is /dev/audio, and how do I use it?

The source code provided by freecom contains a directory .arch/arm/mach-mv88w8xx8/base_audio. This might be a clue. It even contains a file called audio_api.h - is this a modified version of OSS, or something unique?

Tried doing cat file.au > /dev/audio and cat file.mp3 > /dev/audio - no sound at all if the machine had just been re-booted and nashville stopped, and had to CTRL-C to get back to the shell.

Re-started nashville, played a tune, put nashville into sleep mode, then stopped it. Repeated the cat xxx process - no sound, but dmesg showed "audio released irq disabled" messages, which come from audio.c in the base_audio directory.

Re-started nashville, and now this does not make any sound either - I'm definitely doing something to its sound driver....

Re-Booted. Set nashville playing, then stopped it with /etc/init.d/mainapp stop. Then did cat /usr/share/wakeup.mp3 > /dev/audio - got awful noise out of the speaker!

This is progress - I must have to do some initialisation to get it working, then send it data in the right format - just a matter of finding out how and what now - looks like audio.c might have some answers though...

My_MusicPal

2008-11-22T18:28:00.000Z

I have added a page to the My_Musicpal Wiki on what I have found out so far:

More on Writing to MusicPal Display

2008-11-21T21:59:00.001Z

I've made some progress on writing to the MusicPal display now.
This is all written in C - haven't got as far as trying to make Python work yet.
I downloaded the Embedded Linux Development Kit and the MusicPal Linux source code from the FreeCom web site by clicking on the 'Legal Notice' link.
The source code included a directory .../alinux/linux-2.6.16.16/arch/arm/mach-mv88w8xx8/lcd_framebuffer which appears to be the framebuffer driver source code for the MusicPal LCD Display.
It included an example file "mvlcd_test.c" which did not work, but I managed to comment out the errors to get it to compile and run on the MusicPal - I transferred the executable to the MusicPal using wget on the MusicPal to download it from my web server. Most of the options to do with filling in the screen did not work, but the 'Write String' one did once I realised that you have to do 'manually refresh display' afterwards to see what you have done. It is also best to kill the MusicPal main application with:


/etc/init.d/watching stop
/etc/init.d/mainapp stop

Otherwise it overwrites the things you have written.

I have got a simple example running which shows text in three different fonts. The example program is:


#include 
#include "glcd.h"
main() {
  lcd_clear_screen();
  lcd_set_font(0);
  lcd_write_string("font0",0,0);
  lcd_set_font(1);
  lcd_write_string("font1",0,10);
  lcd_set_font(2);
  lcd_write_string("22:00",0,20);
  lcd_redraw_screen();
}

This in turn relies on a simple library that I have started - glcd.c and glcd.h:
glcd.h


/*
 * glcd.h
 * Desc:  Graham's simple interface to display text on the MusicPal LCD
 *        Display.
 * HIST: 21nov2008  GJ ORIGINAL VERSIO
 */

extern int lcd_font_number;  /* Default font - 0,1 or 2 */

int lcd_set_font(int);
int lcd_write_string(char *str,int x,int y);
int lcd_clear_screen(void);
int lcd_redraw_screen(void);
int lcd_poll_buttons(void);

glcd.c



#include 
#include 
#include "lcdwrite.h"
#include "glcd.h"

int lcd_font = 0;

int lcd_write_string(char *str, int x, int y) {
  int fbfd = 0;
  int retval = 0;
  struct disp_string_lcd disp_lcd;
  strcpy(disp_lcd.str,str);
  disp_lcd.x = x;
  disp_lcd.y = y;

  switch (lcd_font) {
  case 0:
    disp_lcd.flags = STR_FLAG_FONT_6X8;
    break;
  case 1:
    disp_lcd.flags = STR_FLAG_FONT_8X8;
    break;
  case 2:
    disp_lcd.flags = STR_FLAG_FONT_24X24;
    break;
  default:
    disp_lcd.flags = STR_FLAG_FONT_6X8;
  }

  printf("write_string: str=%s, x=%d, y=%d, flags=%d\n",
     disp_lcd.str, disp_lcd.x, disp_lcd.y, disp_lcd.flags);
  // Open the file for reading and writing
  fbfd = open("/dev/fb0", O_RDWR);
  if (!fbfd) {
    printf("Error: cannot open framebuffer device.\n");
    retval = -1;
  } else {
    if (ioctl(fbfd, MVLCD_DISP_STRING, &disp_lcd)) {
      printf("Error refreshing.\n");
      return 1;
    }
  }
  close(fbfd);
  return 0;
}


int lcd_clear_screen() {
  int fbfd = 0;
  int retval = 0;
  // Open the file for reading and writing
  fbfd = open("/dev/fb0", O_RDWR);
  if (!fbfd) {
    printf("Error: cannot open framebuffer device.\n");
    retval = -1;
  } else {
    if (ioctl(fbfd, MVLCD_CLEAR_SCREEN, 0)) {
      printf("Error refreshing.\n");
      return 1;
    }
  }
  close(fbfd);
  return 0;

}

int lcd_redraw_screen() {
  int fbfd = 0;
  int retval = 0;
  // Open the file for reading and writing
  fbfd = open("/dev/fb0", O_RDWR);
  if (!fbfd) {
    printf("Error: cannot open framebuffer device.\n");
    retval = -1;
  } else {
    if (ioctl(fbfd, MVLCD_REFRESH, 0)) {
      printf("Error refreshing.\n");
      return 1;
    }
  }
  close(fbfd);
  return 0;

}

int poll_buttons() {
  int fbfd = 0;
  int retval = 0;
  int buttons;
  // Open the file for reading and writing
  fbfd = open("/dev/fb0", O_RDWR);
  if (!fbfd) {
    printf("Error: cannot open framebuffer device.\n");
    retval = -1;
  } else {
    if (ioctl(fbfd, MVLCD_POLL_BUTTONS, &buttons)) {
      printf("Error refreshing.\n");
      return 1;
    }
  }
  close(fbfd);
  return buttons;

}


int lcd_set_font(int font) {
  if (font<0>2) {
    fprintf(stderr,"WARNING: Font Number %d is an odd number - this might not work\n", font);
  }
  lcd_font = font;
}

This doesn't do much, but I can now display text on the MusicPal display, so I am progressing towards a replacement firmware for it - just a little way to go still!

NT

Writing to the MusicPal Display

2008-11-19T20:52:00.001Z

Made a bit of progress with writing to the MusicPal display.
There is a device /dev/lcd which looked promising, but I didn't know what to do with it, but then I noticed that /dev/fb0 links to it. /dev/fb0 should be a linux framebuffer interface to the graphics hardware. Unfortunately most of the guides I could find on the internet were not about how to program the framebuffer, but how to set it up. I thought I was going to have to resort to reading /usr/include/linux/fb.h to try to understand it. I was pleased to find that someone (lesky?) has done that and published a bit of code to work with the framebuffer here.
He provided some example code to open the framebuffer device and extract information about it from the hardware as shown below:


#include 
#include 
#include 
#include 
#include 

int main()
{
        int fbfd = 0;
        struct fb_var_screeninfo vinfo;
        struct fb_fix_screeninfo finfo;
        long int screensize = 0;
        char *fbp = 0;
        int x = 0, y = 0;
        long int location = 0;

        /* Open the file for reading and writing */
        fbfd = open("/dev/fb0", O_RDWR);
        if (!fbfd) {
                printf("Error: cannot open framebuffer device.\n");
                exit(1);
        }
        printf("The framebuffer device was opened successfully.\n");

        /* Get fixed screen information */
        if (ioctl(fbfd, FBIOGET_FSCREENINFO, &finfo)) {
                printf("Error reading fixed information.\n");
                exit(2);
        }

        /* Get variable screen information */
        if (ioctl(fbfd, FBIOGET_VSCREENINFO, &vinfo)) {
                printf("Error reading variable information.\n");
                exit(3);
        }

        /* Figure out the size of the screen in bytes */
        screensize = vinfo.xres * vinfo.yres * vinfo.bits_per_pixel / 8;

    printf("xres=%d, yres=%d, bits_per_pixel=%d\n",
        vinfo.xres, vinfo.yres, vinfo.bits_per_pixel);
       
        /* Map the device to memory */
        fbp = (char *)mmap(0, screensize, PROT_READ | PROT_WRITE, MAP_SHARED,
                fbfd, 0);      
        if ((int)fbp == -1) {
      printf("Error: failed to map framebuffer device to memory.\n");
      exit(4);
        }
        printf("The framebuffer device was mapped to memory successfully.\n");

        x = 100; y = 100;       /* Where we are going to put the pixel */

        /* Figure out where in memory to put the pixel */
    location = (x+vinfo.xoffset) * (vinfo.bits_per_pixel/8) +
            (y+vinfo.yoffset) * finfo.line_length;

        *(fbp + location) = 100;    /* Some blue */
        *(fbp + location + 1) = 15;     /* A little green */
        *(fbp + location + 2) = 200;    /* A lot of red */
        *(fbp + location + 3) = 0;      /* No transparency */

        munmap(fbp, screensize);
        close(fbfd);
        return 0;
}

MusicPal Internet Radio

2008-11-16T16:05:00.000Z

I have bought a Freecom MusicPal internet radio as a present for Sandie.
It turns out that it is actually a little single board computer running linux, so I have to see what I can do with it.
Fortunately some other people have made a start as shown here. Most importantly they show that if you point your web browser at the musicpal and log in, you can run an extra CGI script as http:////admin/cgi-bin/debug.cgi. This allows you to switch the telnet server on and off in the MusicPal.
Once you have switched the telnet server on you can log into the machine as root and have a look around.
To do anything interesting with it there will be three main requirements:

Read the keyboard (well the buttons on the device anyway!)
Display things on the screen.
Play music files.

I have made a start on this. The buttons can easily be read from /dev/keypad. Typing cat /dev/keypad shows the key currently being pressed. Note that there seems to be a delay between typing cat /dev/keypad and anything starting to appear - not sure why.
The codes returned are:

Fav - 1
Menu - 2
Volume Button Press - 4
Volume Button Clockwise - V
Volume Button Anti-Clockwise - v
Select Button Press - 8
Select Button Clockwise - N
Select Button Anti-Clockwise - n
No Key Pressed - T

That's a start anyway - displaying things on the screen will be more difficult - I'll ahve to find out what sort of driver is provided - there /proc/mvlcd looks like something to do with it, but I don't know what.....

NT

Fujitsu Li2727 Sound Problems in Ubuntu 8.10

2008-11-06T22:54:00.000Z

I just realised that the sound on my Futitsu Siemens Li2727 laptop had stopped working - probably when I upgraded to Ubuntu 8.10, but not sure.
All you get when you play sound is crackling.
Dmesg gave the following:
[ 13.411005] HDA Intel 0000:00:1b.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
[ 13.411038] HDA Intel 0000:00:1b.0: setting latency timer to 64
[ 13.442811] hda_codec: Unknown model for ALC268, trying auto-probe from BIOS.
The 'Unknown model' looked like a problem, so I spent a while searching for solutions to do with this, but to no avail.

Found a bug report - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/281322, which sounded like my problem, but was solved by adjusting alsamixer.
When I used alsamixer I only saw a single channel, which I think was the PC speaker, so it didn't sound right. Looked at https://wiki.ubuntu.com/DebuggingSoundProblems which said there was another program, gnome-alsamixer. Tried this and it showed me lots of different sound card channels. The volume of the PCM channel was zero - setting this to full volume cured the problem - Solved!

Cross Browser JavaScript

2008-11-03T21:09:00.000Z

I have been having terrible trouble making my daughter's 'Countdown to Christmas' google gadget work on internet explorer - works fine on firefox, but alas quite a lot of the world use IE, and on IE you just get a blank square.

IE is not helpful with debugging - it seems to be fussier about little problems that firefox let me get away with, so there were quite a few typos (no closing '>' etc.).

One significant issue was that I was attempting to scale the background image in html using width='100%' height='100%' - firefox stretches the image to fit, but within a google gadget at any rate, internet explorer just displays a blank square - no helpful error messages or anything - useful!
This means that my very clever bit of code to put the text in the correct position on the image doesn't work...Cured by forcing the image to a specified size in pixels rather than % - just means you have to size your page to fit it....This was not pretty, so had to solve it. Went for specifying width=100%, which works, and let the height float to preserve the aspect ratio of the image - just had to add yet more code to pick the vertical position of the text correctly.

The last problem was the dates - IE said there were about 600000 days to christmas, which was not quite right - this was because getYear() returns 108 in Firefox and 2008 in IE. Using getFullYear() cured it.

Wireless Networking in Linux on Fujitsu Siemens Amilo Li2727 laptop

2008-11-02T20:38:00.002Z

I have a Fujitsu Siemens Amilo Li2727 laptop, which has a built in wireless network card.
Ubuntu Linux 8.10 detects the card as an AR242X (by typing lspci), but the madwifi driver compatibility page says it should be an AR5007EG.

The symptoms are that wireless networking does not work at all. dmesg reports:
"wifi%d: unable to attach hardware: 'Hardware revision not supported' (HAL status 13)".
No wifi devices present when doing ifconfig, iwconfig etc. There is one odd device (pan0) - I don't know what this is!

There are quite a few forum posts about this problem - they say that the madwifi driver can solve the problem, but you need to download a patch and re-compile the driver. When you download the patch the README file says the instructions are out of date, and you should use the updated madwifi driver as described in http://madwifi.org/ticket/1192. That page gives a link to the driver source code and the compilation instructions are stored in http://madwifi.org/wiki/UserDocs/FirstTimeHowTo. I followed these - they were very easy - first deleting the Ubuntu supplied copy of the madwifi driver, then compiling and installing the new one.

After re-booting I have two devices - wifi0 and ath0. iwconfig reports that ath0 has wifi extensions, so this looks promising.

Tried iwlist ath0 scanning - it reported that the network was down.
did ifconfig ath0 up. Then iwlist ath0 scanning says has no scan results, which is wrong because I am sat next to a wireless router...

Had a look at http://ubuntuforums.org/showthread.php?t=669267.
This got me looking at the Hardware Drivers manager in System/Administration - this said that there was a proprietary driver for the wireless card activated, so maybe this is intefering with things? Deactivated it..and rebooted again..

To start with, no wireless at all, but doing 'modprobe ath-pci' gave the following when I typed dmesg:

ath_hal: module license 'Proprietary' taints kernel.
[ 159.942692] AR5210, AR5211, AR5212, AR5416, RF5111, RF5112, RF2413, RF5413, RF2133, RF2425, RF2417)
[ 160.035904] ath_pci 0000:08:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[ 160.035939] ath_pci 0000:08:00.0: setting latency timer to 64
[ 160.534217] MadWifi: ath_attach: Switching rfkill capability off.
[ 160.541042] wifi0: Atheros AR2425 chip found (MAC 14.2, PHY SChip 7.0, Radio 10.2)
[ 160.559097] ath_pci: wifi0: Atheros 5424/2424: mem=0xfa000000, irq=18
[ 198.800025] ath0: no IPv6 routers present

At least there are no errors....but still iwlist reports no scan results.

Another thing I have read about is that the Amilo does not power up the wireless adapter automatically - could the radio transmitter be switched off?

The madwifi compatibility page pointed to this article:
http://madwifi.org/wiki/UserDocs/MiniPCI#FujistsuSiemensAmiloLi2727withAtherosAR2425AR5007EG802.11abgPCI-erev01
This talks about downloading the acerhk hotkey driver source and compiling it, but
I found that it is already installed in ubuntu and doing 'modprobe acerhk' ran it without error.
I pressssed Fn-F1 a few times and nothing happened (apparantly), then I realised that to make the wifi led light you have to do that in software (echo on > /proc/driver/acerhk/wirelessled). Doing this put the led on, and iwlist ath0 scanning gave me a list of access points - Success!!!

It seems that the acerhk driver has the code to switch on the wireless transmitter by doing the echo command above, but the Fn-F1 key does not work.

Now the procedure to get wireless networking working after boot is:
modprobe ath-pci
modprobe acerhk
echo on > /proc/driver/acerhk/wirelessled
ifdown ath0
ifup ath0

This needs automating!...

framework.system.filesystem

2008-10-23T22:14:00.000+01:00

Making a bit of progress now - framework.system.filesystem look like it should do what I want.
I have tried this:

function buttonClicked() {

debug.trace("ButtonClicked");
fname=framework.BrowseForFile("*|*.*");
oFs = framework.system.filesystem;
alert("oFs="+oFs);
if oFs.FileExists(fname) {
alert("file exists!");
oFile = oFs.GetFile("/home/graham/Countdown.js");
oStream = oFile.OpenAsTextStream(1,-2);
linstr = oStream.ReadLine;
alert("ofile="+ofile+" linstr="+linstr);
} else {
alert("file "+fname+" does not exist");
}
}

It looks promising - oFs is a real native object, which is encouraging, but I get an 'uncaught exception' error - still needs some digging into how it works!

gadget.storage.openText()

2008-10-22T21:56:00.000+01:00

A bit of progress with local file access.
It is nice and easy to get a file browser to open to select a file name, using framwork.BrowseForFile().
Still can't read it though. The nearest I have found is gadget.storage.openText(), which works nicely reading files stored in the gadget .gg file, but I can't get it to open other files....
Code for a simple button to demonstrate the above:

function buttonClicked() {
debug.trace("ButtonClicked");
fname=framework.BrowseForFile("*|*.*");
debug.trace("fname="+fname);

file=gadget.storage.openText("main.xml");

alert("file="+file);
}

Unfortunately code.google.com seems to be dead, or I was going to ask on the developer's forum....

Google Desktop Gadgets

2008-10-21T22:39:00.000+01:00

I've just discovered Google Desktop Gadgets. This site is for me to record things I have tried, so I can find them again.
Desktop Gadgets are simple javascript applications that run on Windows, Linux or Mac, which is neat. I use the Linux version.

A gadget is a simple zip file renamed to have a .gg extension. As a minimum it needs to contain three files, gadget.gmanifest, main.xml and main.js. .

gadget.gmanifest is a little file describing the gadget, so the Google Desktop application knows what to do with it.
main.xml defines the user interface of the gadget.
main.js is hte javascript code to provide the functionality.

When you install a gadget you get a warning saying that the gadget is being granted access to the local file system. This is good because the reason I am interested in them is that I have a web application which is a GPX route editor (http://www.maps.webhop.net) but I would like it to have access to the local file system so it can save its output file for you to put on your GPS device. I think it should be possible to port the application to work as a Gadget so it will retain its network capability with local file access.
It's just a matter of making it work.....

I am struggling to make local file access work, never mind anything more complicated!