Wednesday, October 31, 2012

No Jelly Beans yet

Ok, so it does not really look like October, does it ...

After the earlier report  about October, I was really ...optimistic 

IT conservations: Jelly Bean on its way: From The Verge : The Taiwan-based manufacturer also revealed that a Jelly Bean update for the HTC  One X  and  One S  will arrive in Oct...

Saturday, October 27, 2012

Maps mash up: Underground vs post office

The other day, when I had to post an important letter, I was wondering if there is any post office close to an underground station on my commute, and I could not come up with one. So - me being me after all - I decided to do some in-depth analysis on this. And learn Python along the way.

1.  The Idea
Get all the underground stations and post offices as geo coordinates, and find those closest to each other.


2. Getting the underground data
Easy, the geo data of all the public transport stations (or ony the underground stations) can be found easily, since the are part of the open government data, shared by the municipality of Vienna.

3. Going postal
Getting the post office data is rather challenging, because they are seemingly not considered to be public enough.

Still, this can be done; so first get a list of all the post offices in Vienna from post.at. Best with a little python script (my first!!) to parse it.
Get all those with a ZIP code starting with '1' into a CSV file including their full street address.

Then - thanks for the hint, martin - use the Yahoo! PlaceFinder API to convert those postal addresses to geo coordinates. Get an Appid for Yahoo!, if you don't yet have one.

Important trick here: Don't place everything into the q=... query string, but separate it into postal, city, street, etc. Like this
u = yahooURL+"?appid="+yahooAppidy
u += "&postal="+po.zip
u += "&city="+po.city
u += "&street="+quote_plus(unicodedata.normalize('NFKD', po.addr).encode('ascii','ignore'))
u += "&country=Austria"
From Y! we get some fine XML back and use XPath to access the geo coordinates

doc = ElementTree(file=urlopen(u))
lat = doc.findall('.//Result/latitude')[0].text
lon = doc.findall('.//Result/longitude')[0].text
So now we have the geo coordinates of all the underground stations and all the post offices.

4. Visualize & Verify the data
Let's again use Python to create a simple KML file to load the date acquired so far into Google Maps (or Google Earth). There's a Simple KML python library to do just that. Here's are the maps (post offices, underground stations)

5. Do the Geo Math
This is the tricky part. I decided to put all the data into a (relational) database, and since I have DB2 installed on my system, it was of course DB2. I did not use any geo/spatial extension, but just put the latitude and longitude into proper types. Then I created a user defined function (UDF) to do the geo math. Actually, for this purpose this could have been simpler, since one can disregard all spherical aspects and assume the surface (of the Earth in Vienna) to be flat.
With my newly created haversine function the query then looks like this:

select haversine(p.lat,p.lon, u.lat,u.lon) as distance, p.plz,p.street,p.lat,p.lon,u.station,u.lat,u.lon
from ubahn.post p, ubahn.stationen u
where haversine(p.lat,p.lon, u.lat,u.lon) <1
order by 1 asc
fetch first 20 rows only

I'm only interested in post offices that are maximum 1km (distance<1) from an underground, and I only want the closest 20 of those (first 20 rows only)

6. Create a map
Export those data into a CSV file, and run a pyhton script that creates the KML file for this, with the post office being a point, the distance being a line, and the underground again a point. Map can be found here.

Turns out, there are more pairs than I actually thought.

Sunday, October 21, 2012

Geo-Distance vlg Haversine as a DB2 UDF

In a hack I'm currently working on (to be published soon) I needed to calculate the distance between two locations (on the surface of our planet). Admittedly, those are close addresses (all in Vienna), so I could have assumed them all to be on a plane (not an airplane, a flat surface...), but I felt like some real math.

This is a workaround for all of you who do not have spatial extender available or installed.

To calculate the distance of two points on (a perfectly spherical earth) you apply the haversine formula.
This is the representation of it as a DB2 user defined function (UDF)


CREATE FUNCTION HAVERSINE( lat1 decimal(10,8), lon1 decimal(10,8), lat2 decimal(10,8), lon2 decimal(10,8))
RETURNS DECIMAL(15,8)
F1: BEGIN ATOMIC
declare dlat, dlon, a, d double;
set dlat = radians(lat2-lat1);
set dlon = radians(lon2-lon1);
set a = sin(dlat/2) * sin(dlat/2) + cos(radians(lat1)) * cos(radians(lat2)) * sin(dlon/2) * sin(dlon/2);
set d = 2*6367*asin(sqrt(a));
return d;
END

The two points are given in latitude and longitude (in degrees) as lat1/lon1 and lat2/lon2. It returns the distance in kilometers. If you want to change this, replace 6367 (which happens to be the Earth's radius im km in Vienna, for which I needed this) with your radius and units,e.g. 3956 if you want miles in London.
Check it simply at Wolfram Alpha with "earth radius in london".

I guess it is not perfect, but it does work well. I should probably make it DETERMINISTIC (because it is) to improve performance.
Go ahead, play around.

Monday, October 15, 2012

Something's odd here

In tablets we currently observe two business models:

The Apple model -  make money on the hardware, therefore lock users in,  and make sure that there's enough content available for iOS to do so

The Amazon model - get consumers to buy content and stuff from Amazon,  and make sure they have a device for this.

What is odd here,  is that the Apple model,  which depends on globally availability and distribution of a physical goods (the iPad) is more scalable than the distribution of purely electric or virtual goods (content) in the Amazon model, because Amazon is (still) limited by availability of said content in each country and has to negotiate it...

Sick.

Wednesday, October 10, 2012

Firefox 16 is out

... and fighting with he F1 addon (an addon to share pages via Facebook, Twitter, gmail)

Problem was that the tab-key stopped working... both in web-pages as well as in the awesome bar (to jump to the search inbox).
Google revealed bug 788050  with the following exchange:

Luckily, F1 has been replaced by the Firefox Share addon, which solves the tab-key problem, but does not have a share button/icon on the address bar... 

The worrisome part of it, is that the addon is from Mozilla Labs, and not some thirdparty script kiddie.

Tuesday, October 02, 2012

Jelly Bean on its way

From The Verge:
The Taiwan-based manufacturer also revealed that a Jelly Bean update for the HTC One X and One S will arrive in October
hell yeah...