T-Mobile's Android root runs shallow

T-Mobile's only Android device, the G1, isn't as open as you would think. The device lacks root access and the bootloader requires a signed image. Older versions of the OS have had a successful jailbreak but the most recent (R30) over the air firmware upload "fixed" that.

I jumped on the G1 bandwagon the day after it was announced hoping that it would live up to the openness Android was claiming. To be fair, the SDK does allow access to quite a bit more than the iPhone's SDK does which makes for some interesting applications not do able on the iPhone (for now). But the Google/T-Mobile team has stopped a bit short when it comes to moving from one Android device to another. There's no desktop client nor is there a backup/restore client! Think about that for a moment. When the time comes that you have the option to move to another Android device or, as in my case, have to replace your current one you're out of luck.

I searched the internet and even posted in a few google read forums (groups.google.com/groups/android-developers, forums.t-mobile.com) about how to move from one to the other. I read about the jailbreaking < R30 and the R30 hacked image if you already were jailbroken. I was told that it's T-Mobile's fault and to stop complaining. I was even told by one person that they clear their phone all the time for app testing and it only takes about 10-15 minutes and isn't a big deal. That person obviously isn't really using their phone.

T-Mobile customer support isn't much better either. They say you just have to download the applications again from the market and accept the data loss or they'll tell you about the jailbreak and how R30 shut it down and it's "impossible" to hack this new version. When you try to tell them that you want T-Mobile to provide root access or unlock the boot loader they tell you that Google writes the code and HTC makes the phone. They have nothing to do with it. This is a lack of understanding on the support staff's part. Google writes the base code, but it's modified for T-Mobile and only runs as a signed image.

Basically it seems that what's happening is that Google released some software prematurely as usual with their code but this is a very different case than the web were only the Google devoted will pick up the software before it's ready for prime time. T-Mobile pushed out the G1 as a iPhone replacement when it is not because of the build quality, the hardware itself and the premature OS. Now that they actually have real users with real problems that should have been caught long before in acceptance testing, they just want to ignore them. And they'll be able to because most people using the G1 won't know any better. There's always the Google apologists (and in some cases the Android devs themselves) that will say it's not their problem. There's no way to actually get in contact with someone who actually knows something about this level of detail about the G1 at T-Mobile. And finally, we already paid and signed a contract and are well past the 14 day return period. So the only way out is to pay more money to T-Mobile.

For anyone out there looking to develop on Android and the G1, PLEASE, PLEASE just spend the extra money on the Google Dev Phone 1 (http://code.google.com/android/dev-devices.html).

• No Contract


• No $18/$36 upgrade/activation fee (which is a whole different kind of BS)


• Full Control of your phone. Add new images. Root access.

To me this is what the G1 should have been. This is what Android was to represent.

Firebug Console for power users == REPL

There have been many, many posts through out the internet explaining why Firebug is the greatest thing ever for the web developer from the CSS editor to the AJAX transmission viewer to the javascript debugger. I'm going to throw one more thing in; javascript REPL.

REPL stands for Read, Evaluate, Print, Loop for those that don't know and basically means it's an interactive shell into your program. Most of us know about the console in Firebug but how many know about the power it gives? When you open Firebug the console tab is selected by default. You may know this as the place those console.log() messages go. At the bottom is a blank line denoted by ">>>", this is the interactive part. You can execute a line of javascript here. If you click the circle with the up arrow on the right you open the multiline console. I recommend this as the way to go. You can write multiple lines, such as functions, and execute with ctrl-enter or command-enter.

Anyway, back to that REPL. The console lets you interact with any loaded javascript on the page. You can change global variables, call functions, pretty much anything javascripty. This is where the power comes in. You can define functions meaning you can redefine functions! And it'll work when some element or function on the page calls it.

Imagine for a moment that your trying to create a new function or debug a function on your production server where you have to interact with the real data. You can't just change the code, check it in, build and deploy each time due to various issues like time and the client noticing the site going up and down. What you can do, and what I had to do recently, is open the console, paste in the fixes to the definition of the function, evaluate it, and then go about interacting with the page. It becomes a much faster solution that has a couple of benefits.

1. No redeploy until you know it works
2. Don't have to worry about anyone else getting in there and using your code while you're testing it out
3. Interaction with the real problem, with the real data causing it.

If you go about writing your javascript code with the console in mind there are some great uses for it.

• Have your logging statements controlled by a global variable. On the page it's false and not printing. When you need to see what's going on in Production or some other uncontrolled by you environment, just change that variable to true and you get your logging.
• Fire functions and events while developing so you don't have to keep reloading the page and waiting for template recompiles. I like to use this when writing something with AJAX calls while working on the backend. I create a refresh() method ,that is the initial loader on the page anyway, and call it from the console.
• Last, but not least, a personal favorite, testing out javascript expressions. We all have been there where you can't remember exactly how substring() works and you'd just like to try on the page instead of googling it and then interpreting what you find to fit your scenario. I use this all the time for things like figuring out a complex Prototype.js expression on the page to see how it works and tweet it before putting it on the page for real and going through the refresh cycle. In fact I'd go as far to say it'd be very, very hard to do without and I'm starting to wish for one in Java (aside from things like groovysh and jirb)

I hope this opens up some options and makes life easier for you developers out there. At least when working in Firefox. Firebug does have the downside of making you feel that much more helpless when you have some stupid IE bug.

Google, T-Mobile and HTC invent English

Imagine for a moment that the year is 2008 BC and Google, T-Mobile and HTC get to get together and create a new language called English. "Its wonderful", they say. "It can be used for communication during business. Its open, any country can adopt it and change it as they see fit!" yada, yada, yada.
And its true. England, the US and Australia all want in. So they turn to HTC and T-Mobile to give us a platform to introduce English to the world.
They call together members of the press to announce their darling creation and what do they bring to the stage?

A half retarded, southerner with one leg named Andy. Sure he meets all the requirements; human, vocal cords, at least one eye, some type of self awareness. But something's off, he's not quite there. He talks with a drawl. He takes a bit too long to form sentences and has a slight stutter. He really can't remember all that much. He can sing but is off key unless you provide him with some other piece of equipment. He can't dance but is open to learning. Actually, he can't do much but is always open to learning as long as someone else teaches him. He "shur' as heck ain't gonna'" learn on his own. Oh, and to top it all off T-Mobile won't let him go any further than 1Km from his house without taking his other leg off. (But they'll reattach it at the beginning of the next billing cycle)

Andy, ambassador of English to the World! (I think the World might want to stick to the romantic languages)

This is what "The big G and the pink T" gave us yesterday. Some phone from 2005 with a flashy new interface. What is it about the G1/Dream that HTC needed so much time to develop? Sure the digital compass is new, but other than that it's a smart phone from before there was an iPhone. It seems that they didn't take any ideas from the iPhone at all. Why isn't the Touch HD the android phone? When you preemptively answer the "Why HTC" question in your press conference, you know things aren't going to go over well.

I'm sure the spin machines at those camps will try to say that "Oh, no this is better than the iPhone because it's open and unrestricted.." at first. Then when they get tired of saying that, they'll get angry and start with the "Well, this was never intended to be an iPhone killer."

Another underlying cause of concern for me is that Google didn't stop this train wreck. Either they seem to really be confident and are just treating this like another Google Beta or they don't care. Someone there had to know that this was not a good phone. No one voiced any concern? And to stand on stage, even with all the phone's shortcomings, and tout it as something so great in a post iPhone world is disheartening.

And T-Mobile, unless you have another android phone coming, I really think you've got yourself a Zune to At&t's iPod. Congratulations!

EDIT 9-24-08
T-Mobile has started taking steps in the right direction. They are reducing the 1Gb cap from a hard, fast rule to a reserved right for those that abuse the system. Also, phones after the first shipment will contain a 3.5mm headphone adapter.

T-Mobile Address Manager export "Hack"

Edit: Thanks to yincrash for pointing out that they changed their namespace on this object to TMO.ABPORTAL.

My friend Candice got a 1st Gen iPhone on the cheap and needed to get her numbers from T-Mobile's new online Address Manager (TAB). It does syncing to and from a regular T-Mobile phone but the iPhone isn't a regular T-Mobile Phone. The app is new and they have a note about it not syncing changes made on the website to the phone yet and either by design or evil plotting there was no way to export the contacts.

My vocation to the rescue! They were using the prototype.js library to run the app and after some digging around I found the data store.
TMO.ABPORTAL.ABIHM.ContactManager is the object that manages the contacts' actions and the _datas property is an array of contact javascript objects. So in the firebug console I wrote some code to get the contacts and wrote a csv file to an empty div. I then took that div into Wordpad to get the lines to break correctly on Windows and did an import into Windows Address Book. The rest is history!

Here's the code I used:

var data = ["firstname,lastname,name,homeMobile,homePhone1,homePhone2,workMobile,workPhone1,workPhone2"];
TMO.ABPORTAL.ABIHM.contactManager._datas.each(function(person) {
    var a = [];
    a.push(person.firstname);
    a.push(person.lastname);
    a.push(person.name);
    a.push(person.homeMobile);
    a.push(person.homePhone1);
    a.push(person.homePhone2);
    a.push(person.workMobile);
    a.push(person.workPhone1);
    a.push(person.workPhone2);
    data.push(a.join(","));
});
$('L_footer_pane').update(data.join('\r\n'));

Playing with the Struts 2 Dispatcher

Due to the nature of our permission handling on a project at work I am in need of a way to pass URLs in to the Struts dispatcher and get back the class name and method of the action to invoke (possibly the results as well...). See we have our classes implement a "SecurityAware" interface that returns an array of [OR'd] permission names required to view a page. We also have a permission based menu system that sometimes requires different permissions than the page the menu item points to. I had run into a problem in trying to find all those needed permissions between the database and the source code which was a lot of manual work. So you can see why I'd want to just pass in the URLs from the menu and get the action, call the security method and display the needed permissions for the resulting page.

What I did was start taking a trip through the Struts 2 source and the test classes for an idea of what was going on. I started at the FilterDispatcher since that's the one class that I define and is the starting point of my (any?) Struts2 applications.

The filterDispatcher creates a Dispatcher object. This seems to be the core configuration beast of the system. This has an ActionMapper injected into it. This is where my problems begin.

How are things injected into the FilterDispatcher since I'm "creating" it in the web.xml? There's a ConfigurationManager that is created inside that loads the struts config files and I've had luck printing out parameters that I've set in my struts.xml so I think it's loading correctly. What I'm now having trouble with is the ActionMapper. I'm playing with this in jRuby and I have no idea how to deal with Java5 Generics (I guess I could Google it...) so I took the route of just injecting my created actionMapper.

actionMapper =  DefaultActionMapper.new
dispatcher.configurationManager.configuration.container.inject(actionMapper)

This does have an effect in that I can set struts.enable.SlashesInActionNames to true and then call actionMapper.slashesInActionNames? and have the result be true (it defaults to false, so there is a change).

I should mention that I'm running this code in the struts blank example app's WEB-INF/classes directory. I can create a MockServletRequest and pass in URLs that should and shouldn't work. It errors out appropriately when the URL doesn't end with ".action" and I can get it to parse the namespace portion correctly, but I'm having no luck with the Method. For example /example/Login_input.action should result in an ActionMapping with:

• Name: Login_input


• Namespace: /example


• Method: input

Or at least that's what I'm expecting, but I never get anything for the method.

Here's my code so far.

require 'java'

import 'org.apache.struts2.dispatcher.Dispatcher'
import 'org.apache.struts2.dispatcher.FilterDispatcher'
import 'org.apache.struts2.dispatcher.mapper.DefaultActionMapper'

import 'org.springframework.mock.web.MockServletContext'
import 'org.springframework.mock.web.MockFilterConfig'
import 'org.springframework.mock.web.MockHttpServletRequest'
import 'org.springframework.mock.web.MockHttpServletResponse'


filterConfig = MockFilterConfig.new
servletContext = MockServletContext.new


dispatcher = Dispatcher.new(servletContext, {})
dispatcher.init
Dispatcher.instance = dispatcher

actionMapper =  DefaultActionMapper.new
dispatcher.configurationManager.configuration.container.inject(actionMapper)


request = MockHttpServletRequest.new(servletContext, 'GET', '/example/other.action');  
response = MockHttpServletResponse.new;

actionMapping = actionMapper.getMapping(request,dispatcher.configurationManager)
puts "Name: #{actionMapping.name}"
puts "Namespace: #{actionMapping.namespace}"
puts "Method: #{actionMapping.method}"
puts "Result: #{actionMapping.result}"

MythTV allowing schedules but showing none

My MythTV is on the fritz. Maybe because I keep screwing with it... It would let me "schedule" things through mythweb or by Manage Recordings -> Schedule Recording and would save the recording schedule I picked but would not show anything in the upcoming recordings section. This had happened to me once before and luckily I vaguely remembered how to fix it.

If this happens to you, or to me again, check that your recording device is hooked up to your listings. I think it's under Input Devices in setupmythtv, aka the backend setup. In my case I had my card and I had Schedules Direct but my card's television input was not pointing to anything. I pointed it to Schedules Direct and exited out, went back into the frontend and all my upcoming shows were there.

Maemo Mapper paths.db break down

When connected to GPS mapper has a track log in the paths.db file in /home/user/.maemo-mapper/. If you take a look at the file with the sqlite3 client you'll see a table named track_path. This is where the data points are stored but they're not in degrees. A .schema reveals that the columns are called unitx and unity.

A dig through the internet found me source that was using a function called unit2latlon. So I downloaded the source from the garage and grep'd for the definition of this function. It turns out to be a macro in defines.h.

Here's the macro converted to ruby for easy use. It only does long and lat. I still have to find what the altitude is in...

TILE_SIZE_P2 = 8
MAX_ZOOM = 20

WORLD_SIZE_UNITS = 2 << MAX_ZOOM + TILE_SIZE_P2

MERCATOR_SPAN = -6.28318377773622 
MERCATOR_TOP = 3.14159188886811 


def unit_2_lat_lon(unitx, unity)
    lon = ((unitx) * (360.0 / WORLD_SIZE_UNITS)) - 180.0
    lat = (360.0 * (Math.atan(  Math.exp ((unity * (MERCATOR_SPAN / WORLD_SIZE_UNITS)) + MERCATOR_TOP)))) * (1.0 / Math::PI) - 90.0;
    return lat,lon
end

It's line for line the same as the Maemo-Mapper team's code. There was a lot more in the defines.h file, but this is all that's needed to run the function. I ran the last point in the database and checked the output in Google maps (straight copy and paste!) and it is where it stopped logging!

irb(main):001:0> lat,lon = unit_2_lat_lon(157094175,199470581)
=> [41.9127500226541, -74.6601469069719]

Google Maps, Maemo Mapper, GPX, KML all get me to the rally!

I'm a bit of a rally fan. Not the political kind, but the motorsports kind (you may have see it in the x-games...). This weekend is the 2008 Rally New York. If you've ever been to a rally before you know what a pain it is finding and getting to spectator points. If you haven't, I suggest you do, and believe me, it's a pain. There's usually a spectator guide and if you dig around the site some scanned maps that have been marked but these always leave me a little in the dark. So each time I go to one of these I usually take a few hours checking out google maps and figuring out where the spectator points are and how to get there.

It started out with just printing out the pdf's and the maps, but then I got a psp and I started downloading Google maps and drawing on them and loading them as pictures. Then I got the Nokia N800, suddenly there was no need for print outs (yay! I'm going green while driving > 300 miles to a fuel burning frenzy). The N800 can handle PDFs and with the help of Maemo Mapper can handle maps for me. That coupled with a bluetooth GPS receiver gives me a really great tool for navigating to the different points. Now I have an iPhone as well which has maps and PDF abilities (if it only had GPS!)
So what I did this year was plot everything on Google Maps:

View Larger Map

The cameras are spectator points. The orange lines are the closed road courses and the blue lines are the recommended approaches to the spectator areas. While editing a map you can download the kml file to view in Google Earth. This file just contains a link to the actual .kml file that has the lines and points in it. I tried putting this file on a server and hitting it in my iPhone's maps.app. But after getting a polite error about not being able to find anything matching the search I looked on the web and found that the only kml elements Google Maps for Mobile (which maps.app is based on I hear) is the point. So lines were out and my iPhone was out.

Next I looked up what format Maemo Mapper supported, which is GPX. I fired up gpsbabel and converted my real kml file to gpx only to have Maemo Mapper choke on it. It turns out that Mapper wants only one type of dataset at a time. The kml had lines and points which were converted to "trk"'s and waypoints. But I really only needed to store my spectator areas as Points of Interest (POI). Mapper stores it's POI in a sqlite3 database in /home/user/MyDocs/.documents/Maps/poi.db by default. By getting that file I was able to write a ruby script that took out the "Point" elements from the kml file and create sql insert statements. Putting that db back and fixing permissions (I was ssh'd in as root so the file became pwn'd) gave me the spectator points. Since they're the only things in there it makes for very easy viewing!

Then came the "approaches", I have them as line fragments on the google maps but Mapper only supports Routes and tracks. I don't know if tracks would have helped me as I think they're in a db called paths.db but the routes are stored in an xml file in gpx format. Gpsbable dropped the names and descriptions when it converted the lines so I created a new ruby script that took the LineStrings from kml and created trkpt's from them. Reloading the route file in Mapper gave me my route to the place plus the route fragments to the spectator areas. Since they were all in the same route file I could then download tiles around the routes and have everything internet free. I also added a description to some of the points in the line for turn directions/hints. You can count the bends in your google map to know which one to add it to and remember it goes at the beginning of a line not the end. So if you have a right turn between segment 2 and 3 add the turn description to segment 3.

Hopefully that'll be everything I need. I'm bringing my old phone just in case I need to tether (damn you iPhone and your lame bluetooth support) to the N800 for data. Now about that battery life...

Building source on Ubuntu

If you're trying to build anything from source on Ubuntu don't forget to sudo apt-get install build-essential before you start!

I was trying to build the iphone toolchain on Ubuntu 7.10 and was getting this error when trying to ./configure --enable-optimized

checking for C compiler default output file name... configure: error: C compiler cannot create executables
See 'config.log' for more details.

I knew there was some basic install stuff you needed before hand but I had to search for it. Once installed your configure step will work.

[update]
If you have to build ruby gems with native extensions then also add

sudo apt-get install ruby1.8-dev

HTML as a programming language

I read an article a couple days ago about robots that were programmed by genetic algorithms where one of the patterns that evolved was that of a liar. I've been looking for something to try out with genetic algorithms and thought it would be interesting to see if I could create a liar bot as well.
Since I've been reading Programming Collective Intellegence (along with everyone else!) and Dave always telling me this stuff needs to be done as a tree ("LISP, NICK! LISP!") I started thinking down that path.
Then I realized that HTML is a tree, and quickly coded up this useless calculating proof of concept: HTML programming!

<body>
  <div class="add">
      <div class="power">
          <div>x</div>
          <div>2</div>
      </div>
      <div class="multiply">
          <div>5</div>
          <<div>x</div>
      </div>
      <div>4</div>
  </div>
</body>

This is the "Code". It's a polynomial, x^2+5x+4. And then the "interpreter":

function writeExpression(action, actors) {
    //console.debug(action,actors);
    if (action == 'add') action = '+';
    if (action == 'subtract') action = '-';
    if (action == 'multiply') action = '*';
    if (action == 'power') action = 'Math.pow(';

    var openParen = action.indexOf("(");
    if (openParen > -1) {
         action += actors.join(",")+")";
         return action;
    } else {
         return actors.join(action);
    }
}

function depthFirst(el) {
    if (el.childElements().length == 0) return el.innerHTML;
    var children = [];
    for(var i = 0; i < el.childElements().length; i++) {
        children.push(depthFirst(el.childElements()[i]));
    }

    var expression = writeExpression(el.className, children);
    //console.log(expression);
    return expression;
}

function parse() {
    var code = depthFirst($(document.body));
    $(document.body).insert("<br/>"+code);
    x = prompt('X = ?');
    eval("fun = function(x) {return "+code+";}; alert('f(x) = "+code+"\\nf("+x+") = '+fun(x));");
}

It's just a depth first search that then decodes the class name into an operator and returns the equation for that branch.

Reflecting off a cd case 2

Reflecting off a cd case 2
Originally uploaded by joecagit

Progress has been made!
Using GPSD's python wrapper I'm connecting to GPSD running on my desktop machine. I also fixed the spacing of the lines on the speed chart and the number of numbered lines on the compass. I also dropped in the current speed value.

The first test revealed how tiny the font at 25 (pixels? points?) is. So I bumped it up to 50 and doubled the draw size of the two gauges. Either by luck or some math that I some how got right, the fonts fix perfectly.

When trying to take this picture I noticed that the display goes away completely when the any part of the light from the GPS is behind it. That means that the HUD will disappear in other car's lights or street lights, etc. I hope that won't be too much of an issue.

Running on N800!

IMG_2250.JPG, originally uploaded by joecagit.

Thanks to pygame and anyone who worked on porting it and python to Maemo I have the first steps running on the N800!
Fullscreen - check
Flip Horizontal - check

I also found wiimote.py on the internet and will be trying to integrate a wiimote as an "angle of attack" and G-meter.

It's taking a little bit of time since I have to now learn python to get this all to work. But, hey, one more language to add to my collection.

First Prototype HUD GPS Hookup

IMG_2246.JPG, originally uploaded by joecagit.

This is the first test of my prototype for a car HUD. What I'm going for is a fighter jet feel in my car. I'm prototyping using Shoes for now to just work out the details quickly. I'm using gpsd with Net::Telnet in Ruby to access the heading (across the top) and speed (down the side).
The image doesn't really make sense since the heading coming over the wire isn't what's showing on the compass. I think I took the picture as one of the bad sentences came across. (Every once in a while it would change to 358ish degrees when it was supposed to be about 2 degrees)
Also the speed coming across is either in knots or meters/second so that's what's showing on the HUD not the 25 mph on the GPS.
I have to still figure out how to get the waypoint information from the GPS to update the hud. That's what the little circle is. Hardcoded to 285 in the picture.