My PiThon Project

Pardon the really terrible pun - but I'm working on a small side project on a Raspberry Pi in Python.

Ok, now that we got the Pi jokes done, here's what I'm doing:

I'm a building an station display (board) for nearby train and other public transprt stations with real-time data and more importantly potential delays. Idea is to have a brief glance at it before leaving the house, so you know whether to rush or leisurely stroll to the station.

For that I'm re-using and old PI (3B+ to be precise, not really the fastest one) with a Raspberry Touch Display 2 and a bit of Python code.

So easy do develop (in PyCharm) and test (on my PC), then "stage" it to a Raspberry OS within a virtualbox, and then deploy to a real bare metal RaspPi.

I get the data from the ÖBB online timetable (as json) and the Wiener Linien open real time data (german description here). I chose those 2 because that's really all I need in my neighborhood. ÖBB (the Austrian federal railways) timetable covers all train journeys within Austria, and as I'm living in Vienna, I only need the Vienna public transport timetable. *

For the GUI I started the tkinter but started to regret this almost immediately. It is a terrible API without proper control for tables for example - and yes, showing an online timetable needs a proper table - it's sort of in the name.

When I then first deployed this to a real Raspberry the performance was abysmal - and I'm talking about fetching the data from their soures, but presenting that data (once loaded) in a tkinter window/table.

Think of old terminals connected via a serial V.32bis modem... where you can watch the lines being popuplated - almost that bad. You could see the table being drawn... really.

So, terrible API, hardly control over table formatting, terrible performance on the target HW, lets look for a different GUI library.

PyQt to the rescue. I found the API and the constructs more familiar (from Swing, Android, even back OS/2 and Windows ;)) and it is definitely more performant.

So PyQt it is. Rewrite was done in less then an hour. No visible rendering/drawing artifacts.

Right now, the configuration of the stations is right still in the source code (yuck), but that's the next step then - one needs to fill their spare time, right?

More to follow.

---

* yes, my code is modular enough to plug-in any other transport provider as long as they have machine readable online timetable data via whatever over HTTP without logins ;)

Maps mash up: Underground vs post office

The other day, when I had to post an important letter, I was wondering if there is any post office close to an underground station on my commute, and I could not come up with one. So - me being me after all - I decided to do some in-depth analysis on this. And learn Python along the way.

1.  The Idea

Get all the underground stations and post offices as geo coordinates, and find those closest to each other.

2. Getting the underground data

Easy, the geo data of all the public transport stations (or ony the underground stations) can be found easily, since the are part of the open government data, shared by the municipality of Vienna.

3. Going postal

Getting the post office data is rather challenging, because they are seemingly not considered to be public enough.

Still, this can be done; so first get a list of all the post offices in Vienna from post.at. Best with a little python script (my first!!) to parse it.
Get all those with a ZIP code starting with '1' into a CSV file including their full street address.

Then - thanks for the hint, martin - use the Yahoo! PlaceFinder API to convert those postal addresses to geo coordinates. Get an Appid for Yahoo!, if you don't yet have one.

Important trick here: Don't place everything into the q=... query string, but separate it into postal, city, street, etc. Like this

u = yahooURL+"?appid="+yahooAppidy
u += "&postal="+po.zip
u += "&city="+po.city
u += "&street="+quote_plus(unicodedata.normalize('NFKD', po.addr).encode('ascii','ignore'))
u += "&country=Austria"

From Y! we get some fine XML back and use XPath to access the geo coordinates

doc = ElementTree(file=urlopen(u))
lat = doc.findall('.//Result/latitude')[0].text
lon = doc.findall('.//Result/longitude')[0].text

So now we have the geo coordinates of all the underground stations and all the post offices.

4. Visualize & Verify the data
Let's again use Python to create a simple KML file to load the date acquired so far into Google Maps (or Google Earth). There's a Simple KML python library to do just that. Here's are the maps (post offices, underground stations)

5. Do the Geo Math
This is the tricky part. I decided to put all the data into a (relational) database, and since I have DB2 installed on my system, it was of course DB2. I did not use any geo/spatial extension, but just put the latitude and longitude into proper types. Then I created a user defined function (UDF) to do the geo math. Actually, for this purpose this could have been simpler, since one can disregard all spherical aspects and assume the surface (of the Earth in Vienna) to be flat.
With my newly created haversine function the query then looks like this:

select haversine(p.lat,p.lon, u.lat,u.lon) as distance, p.plz,p.street,p.lat,p.lon,u.station,u.lat,u.lon
from ubahn.post p, ubahn.stationen u
where haversine(p.lat,p.lon, u.lat,u.lon) <1
order by 1 asc
fetch first 20 rows only

I'm only interested in post offices that are maximum 1km (distance<1) from an underground, and I only want the closest 20 of those (first 20 rows only)

6. Create a map
Export those data into a CSV file, and run a pyhton script that creates the KML file for this, with the post office being a point, the distance being a line, and the underground again a point. Map can be found here.

Turns out, there are more pairs than I actually thought.