New Google Calendar Provider for Thunderbird/Lightning

Finally a Google Calendar Provider for Thunderbird/Lightning has been released that's based on the gdata api, and not just a network XML link.
Problem with the later approach was - to me - that you needed to add each of your calendars separately via their URLs. That seemed to have performance impacts as well. I had about seven such calendars, and it significantly (with my other CalDAV calendars) slowed down Thunderbird start-up time.

When the new provider was released a couple of days ago, I first really noticed, because it forced me - with a separate pop-up - to authorize it for my Google account. With seven calendars this step kept popping up - seven times, of course - but still for the old calendar URLs.
Performance was really nasty...  Thunderbird was unusable essentially.

For some weird reason, I tried a "New Calendar" > "On the Network" > "Google Calendar" and found, that it was actually letting my select Google calendars by account, instead of just letting me add the XML link per calendar like the previous version.

So first thing, delete all my old Google calendars in Lightning, and try the "New Calendar" > "On the Network" > "Google Calendar" again, with my Google account, and voila, it presented all the calendars I had on Google. With the color as on the web. I only needed to select the ones I wanted in Lightning, and that was it.

Should have been like this from the start :)

Thunderbird start-up performance is back to a couple of seconds now (from about 1 minute+)

Lightning Drap'n'Drop Problem

In my earlier post (and comment discussion) about Opening ICS Files with Thunderbird/Lightning we mentioned that you can also drag the .ics link directly from the browser to the calendar pane in Thunderbird/Lightning.
I also found, that this is not always working.

So - as promised - I looked into this a bit.
First, narrow down the "not always" to something reproducible:

I took a link that I knew was working with drag&drop (actually from the timetable of the Austrian Federal Railways), and verified this. Yep, working nicely

Then I just downloaded the resulting .ics file to my local hard disk into a folder that was mapped into my webserver (Tomcat 7). I wrote a tiny .html file that mainly included a link to the .ics file.

Then open the html file in Chrome/Firefox and drag the download text/link into (or is it onto?) the Lightning calendar pane in Thunderbird. Ignore the second line for a second (no pun intended).

Nothing.

Well, that was surprising... So I went on to waste a lot of time with checking the mime-type etc etc.

But it was fine by responding with "text/calendar" - see:

So I took some Java drag&drop code I had, and took a closer look at the data included in the DnD object. Also everything fine there (I spare you the details here).

By chance I looked into the access log of the webserver and found the following:

GET /test/download.icsdownload HTTP/1.1" 404 1001

Which essentially says, that Thunderbird requests a broken URL. It should GET /test/download.ics and not /test/download.icsdownload

No wonder it gets a 404 and cannot import anything.

Quick test to include a "?" in the a href (see above, the second line I asked you to ignore earlier).
If you drag this one, you'll get a
GET /test/download.ics?download? HTTP/1.1" 200 949



As we are handling static files here, the "?download?" appended to the filename will gracefully get ignored and the file will get served (as can be seen by the 200 response code)
And with this link, Thunderbird really imports the file and opens the event dialog.

Fine there.

What Lightning does, is - wrongly - append the text of the html anchor ("download" or "download?") to the URL. You can test with naming the anchor asdfgh and will see that /test/download.icsasdfgh will be requested.

Why does it do it ?

It seems to be that Lightning picks up the HTML fragment data type that gets passed by DnD. The browser passes the full anchor, from begin tag to end tag (see above), and Lightning stupidly parses until the first newline (\n), and throws away the HTML tags.
This is wrong.
Should not happen.
I'll file a bug in bugzilla.

Opening ICS Files with Thunderbird/Lightning

Now as the world is slowly progressing towards the 21st century when it comes to calendaring tools, some websites offer to download a calendar file (ical, ics, ...) for events they publish.

The problem in my environment is, that no application on my (Windows) PC is registered to handle this file type.
Thunderbird/Lightning simply does no do this.

The out-of-the-box solution without any hacking is to simply download the .ics file e.g. to the desktop, and then drag&drop it to Thunderbird's today pane. This triggers an import. Quite OK.

Works with TB17 and Lightning 1.9.
(Sometimes it seems I have to hold the Ctrl key to force a copy (instead of a move)... awkward.

Thunderbird left alone

So it appears that the Mozilla Foundation decided to no longer support, i.e. fund, Thunderbird.

There will be only security updates and patches, and the community is of course free to continue developing Thunderbird.

I guess email - and thus Thunderbird - is not innovative enough for Mozilla, and in a way I can see that. but then again, I've been using Thunderbird even as my corporate email client for about 10 years now and never regretted it. Or even envied the Outlook users. Quite on the contrary.

At least since Lightning, the Calendar add-one, got mature. For a global search (indexed, of course) I find Thunderbird even superior. Ctrl-K is (after Ctrl-Shift-K and Ctrl-M) probably the most used short cut for me in Thunderbird.

Well, let's hope the community continues to support it - I'd hate to go back to Outlook.

Thunderbird Clipboard Bug Fixed

Wow, indeed in Thunderbird 10 the clipboard/paste bug from Excel/OpenOffice is indeed fixed.

Weird Thunderbird Bug

Since I upgraded my Thunderbird to version 9 if could no longer paste tables from OpenOffice.org Calc to Thunderbird as a table... they always get inserted as graphics.
Paste-without-format would just copy the plain text, but - of course - no format.
The problem has also been reported with Microsoft Office and LibreOffice.

I did not really notice the first couple of times, until co-workers started to complain that they no longer could copy/paste elements from my "tables" because they were images.

Did some searching on the web and found the bug ... indeed introduced in TB9 supposed to be fixed in FB10 (have to test this on a vbox):

The actual problem is when Thunderbird parses the meta-data of the clipboard, it does a locale-specific parse of a decimal number (the version)... but the version string will always contain a dot (.) and never a comma (,), since it is not really a decimal number. So there is a difference between e.g. German and English system(!) locale, i.e. the problem will appear on German windows, but not on English windows.

Changing the decimal symbol to a dot (.) in the Windows system settings will actually "fix" (circumvent) this problem. I just tested this.
Not sure, which system-wide setting I should keep now...

As I noticed in the bug, the fix is to treat the version as a string, not a (decimal) number.... That's better.

Thunderbird 8 and Lightning 1.0

So, 2 days ago Thunderbird 8 has been released.

The major changes are:

• more restrictive handling of extensions/add-ons, especially those that come from other apps... those will be disabled by default... Thunderbird Installer will guide you through this.

• keyboard short-cuts for the various search options have been changed.

While it has been Ctrl-F for both, the Find within the current message and the QuickFilter for the current folder, it is now

• Ctrl-F for Find (within the current message)
• Ctrl-Shift-K for the QuickFilteron on the folder
• and Ctrl-K is unchanged for the global (indexed) search.

Took me 2 days to get accustomed to the Ctrl-Shift-K.
Well, almost. Not entirely. But I'll manage.

Also I had to "manually" upgrade 3 add-ons that were not declared compatible with TB 8, because no-one cared to update their install.rdf in the XPI file... quite cumbersome... See my rant earlier this year.

Ah, yes, and Lightning finally made it to official release 1.0 ... after years of beta.

Mozilla's new version policy

With Firefox 4 Mozilla introduced their new release policy, which basically means a new release every quarter, and also changing the major version number with every such release. Thus we got Firefox 6 only yesterday.

Since I'm not an enterprise, I do not have a problem with quarterly releases... some IT organizations do.
Also, I personally couldn't care less if they call their releases 4, 5, 6, 7, or Bob, Frank, Josephina, ... or IV/2011, X/2011, Fall 09... whatever.

If it were not for the add-ons... Usually you test an add-on you develop against a certain release, and also declare this in the install.rdf:

<!-- Thunderbird -->
<em:targetApplication>
<Description>
<em:id>{3550f703-e582-4d05-9a08-453d09bdfdc6}</em:id>
<em:minVersion>1.0</em:minVersion>
<em:maxVersion>5+</em:maxVersion>
</Description>
</em:targetApplication>

In the past you knew that the architecture would be stable for a major release, so if you'd tested successfully against a late Thunderbird 3 beta you could increase the maxVersion to "3+". And you'd not have to bother for the next 18 month, until the next major version would go to alpha or beta, you'd have a look at it, test your add-on against it, maybe a tweak here or there, and voila, maxVersion++; and publish it.

Now, you have to do this every 3 months; since everything is now a major version, you either have to declare the add-on as universally compatible (and it might break with version 8) or you have to update the install.rdf every 3 month. Even if there is no major change in Firefox or Thunderbird (as this week with v6).

From a users perspective the same happens, with every update you'll see a couple of add-ons as "incompatible" and disabled. And you either go to your profile and patch the install.rdf (which is what I do) or you have to wait a couple of days (at least) until the developers publish the new "version".

True for both Firefox and Thunderbird.
Somewhat annoying. There's a reason why the separation of major and minor version number has been introduced a couple of decades ago...

Awesome Thunderbird Plugin

So, the other day I sat down to write myself an add-on that was more than overdue for me... or for the way I use mail and Thunderbird.

At that time I called it Domain Specific Move, and it did exactly that. I took the most domain with the most occurrence in an email (scanning to, from, cc) and suggested a folder for this mail; and you were able to train it.

Through Max's comment I learned about the Nostalgy add-on. It actually is about defining keyboard shortcuts and stuff for Thunderbird:

Adds keyboard shortcuts to change folder, move/copy messages, with folder name auto-completion (using only the keyboard). 

The folder name auto-completion is the really awesome stuff. I just have to hit S (for Save == Move) and an entry field pops up at the bottom of the window and I only need to type two or three letters of the folder name and that's it.

When you really use nested folders like I do - with about 5-6 levels deep, this saves a lot of time.

Same for G as GoTo folder. And there's a nice B (whatever that stands for) that Moves the message to the folder and then goes to the folder (sort of S+G).

Great stuff. Thanks Max, for the hint.

Thunderbird hack: Domain Specific Move

One of the most frequent actions in Thunderbird is to move a message that I received from a business partner or customer to a message folder for exactly this partner/customer. Same for messages I sent to them.

However, as you communicate with more customers and partners, the folder hierarchy will become more complex and I already need about 6 clicks to select the specific folder.

Sometimes I'm lucky and it is the most recently used folder, then I can do it with the "move again" function directly in the context menu; sometimes, it is at least in the recent folder menu, still 3 clicks.

WIBNI if TB could just remember that I always move messages from domain A to the folder X, lets say from "ibm.com" to folder "/Vendors/IBM" or something like that, and then present me with a one-click option on the menu.

So I wrote an add-on for this and called it "Domain Specific Move".
It does exactly what I described.

I find the most frequently used domain in the email (counting all from sender, recipient, cc-list).
If I already find a setting for this, I create an additional menu item in the move message menu for a move to this folder.

If not, I present a "learn" menu item, that lets you train the extension on where to put mails for this extension (i.e. register a folder for this domain). You pick the destination folder yourself. No magic included there.

Once I thus learned and stored the folder for this domain, I can - next time this domain appears - present the "Move to " menu item as above.

Configuration is stored in the preferences under "extensions.domainmove.".
Currently I have no options page for this, so if you want to change or remove an entry, you have to go to the prefs.js file or the about: dialog.

Yes, I know, filters can do the same; but when I select to manually run the filter, it will not tell me what exactly it is up to... The beauty of my approach (IMHO) is, that I see it on the menu and can decide otherwise, because not always does the folder registered for this domain really match.

In essence this is only a short cut with an educated suggestion. No behind-the-scenes magic.

Todos:

• more flexibility with domains with more than 2 parts (e.g. at.ibm.com should map to ibm.com if there is no at.ibm.com)
• unlearn domains (without going to the about: dialog)
• option for domains-to-ignore; currently I ignore non-specific domains as gmail.com, a1.net, gmx.at, sun.com, oracle.com [1].
• ignore "my" domain (see comment re oracle.com)

Available for TB3+ only.

I will polish the code and implement some of the above todos, then I will post the first beta.

--
[1] well for me as an former Sun and now Oracle employee, all emails contain either an oracle.com or sun.com address and this domain contains no information on where to archive the email.

Sharing Thunderbird Address Books... some experience

This is a follow up to my post on Sharing Thunderbird Address Books Between Computers with Dropbox.
I've been using this for 3+ weeks now, and here's the experience.

Overall it works quite well, however:

1. I noticed 2 occasions where the filesystem link from the address-book in the Thunderbird profile to the one in the "My Dropbox" folder got lost. The abook.mab in the profile folder then all of a sudden is a real file with not connection whatsoever to the dropbox file.
   
2. Dropbox deteced one replication error and marked it accordingly, by renaming the "older" file as "abook (computername's conflicted copy date).mab"

In both cases replication of the file to and from other computers then fails.
The latter is easy to avoid - at least in my case: Don't have Thunderbird open at the same time on both computers.

Sometimes however, this is tricky, because you need Dropbox to replicate the file before you start Thunderbird on the other computer. So e.g. when you hibernate one PC whilst Thunderbird is still running (keeping the abook.mab file open and locked), the file will not get uploaded to Dropbox. When you then start Thunderbird on the second computer and modify the address-book... voila... replication conflict.

I still don't know what caused the link failure from problem #1. Will continue to monitor this.

Sharing Thunderbird Address Books Between Computers with Dropbox

In the previous post I described how to synchronize your Thunderbird signature (file) across computers using Dropbox.

This was a fairly easy task, because Thunderbird let's you specify the signature file as an absolute path, so you can put your signature anywhere on your computer.
For whatever reasons, the address book has to reside in your profile folder.
So if you want to synchronize Thunderbird address books across computers using Dropbox, this is a bit more tricky.

First, we have to somehow have the address book file in the "My Dropbox" folder, and still in the profile folder.
Luckily, Windows XP (and later) do have links in the NTFS filesystem - contrary to what most people think/know.
Technically they are called NTFS junction points, but usually referred to as links.
You can create them with the FSUTIL utility.

So what we'll do is.

1. Find the "master" address book
2. move it to the Dropbox folder
3. replace the original file in the profile folder with a link to the file in the Dropbox folder
4. repeat step 3 for all computer.

So, assuming you already did register for Dropbox and install it, here are the details:

1. In Thunderbird check the name of he address book you want to synchronize. In the Address Book window, select the address book of your choice and then properties. The only property there is the name.
2. In the Config Editor, locate the file name for your address book. This can easily be done if you filter the view on "ldap_v2.servers.*.descr". This will list all the address book names you have, including the one we are looking for (from step #1).
3. Find the id of your address book.
   It is is right behind "ldap_2.servers.".  Something like "pab" or "default".
4. Once you have that identifier, you can get the file name from "ldap_2.servers.yourid.filename". It is most likely abook.mab. [1]
5. Then locate your Thunderbird profile folder.
   This is (as a default) in the Thunderbird folder of your personal Application Data directory. The fastest way to get there is via Windows Start Menu → Run → "%appdata%" (literally with the percent signs). Then go to Thunderbird, then Profiles and then pick the profile you want to modify (if you have more than one).
6. Now find the file from step #4.
7. Be sure to exit Thunderbird now, so that all files are closed and available to you.
8. Move the file from step #6 to your "My Dropbox" folder (again, this can be easily opened by double clicking on your Dropbox icon in the task bar).
9. Open a command line (Start → Run → "cmd.exe" ) and change directory to your profile folder form step #5.
10. Find the "My Dropbox" folder in your file system. It's quite close to the Application Data folder. Try something like a "dir %homepath%\abook.mab /S /B" to find the exact location of the file.
11. Create the symbolic link with the following command:
    

    fsutil hardlink create abook.mab "..\..\..\..\My Documents\My Dropbox\abook.mab"

    Before you do this, you might want to check with dir, if the abook.mab file can really be found with the above path. Correct your path as appropriate. You can also use the absolute path from step #10.
12. With a "dir abook.mab" verify that the "logical" file is back again with the correct size.
13. Start Thunderbird and verify that your address book is available and correct.
14. Go to computer #2 
15. Repeat steps #1-12 as above, just do not copy the abook.mab (or whatever the name is) to the Dropbox folder. Only delete (or rename) it in the profile folder, before your create the link with fsutil.
16. When you start Thunderbird you should now have all the addresses from computer #1 available.
17. Repeat #15 for all remaining computers

ONE BIG CAVEAT:
I haven't been using this method for long, so I can't really say how stable this works, especially if there are "replication conflicts", i.e. when you update the file on both computers before Dropbox has a chance to synchronize. Should there arise major problems with my setup, I'll post them here.

Comments section is of course open for your findings.

--
[1] in the following paragraphs I will - for brevity - only call it abook.mab; please substitute your file name as appropriate.

Sharing Thunderbird Signature Between Computers with Dropbox

Granted, having the signature file for your e-mail in sync between two or more computers is not amongst the top IT problem of the century.
However, if you change your signature frequently (because you include some simple banner message or seasonal greetings, etc etc) it suddenly becomes important.

In my case this is between my company laptop and my home desktop. Both have the same IMAPS access to our corporate mail server. But as IMAP and Thunderbird[1] go, you are in sync with your mailboxes, but not with all your other data, like address book and signature.

Let's take a look at the signature first, since this is easier (from a Thunderbird perspective).

First we need some file location that can always be accessed from all participating computers. Preferably not only online, but also when offline.

Dropbox is the ideal solution for that: it synchronizes folders of your computer to the (cloud) storage of Dropbox. From and to multiple computers. That's the trick here.

Let's for this use case here ignore all the photo album and public sharing or collaboration features dropbox offers. We just use it for file sync to the cloud (and back again).

So obviously, you have to register for Dropbox and install the software on all participating computers.
[waiting for you to do so ... ]

The default folder is a newly created folder called "My Dropbox" in your home directory. This folder will be in sync across all computers, then.
So let's put our signature file there:

1. Start on the computer, where you have the most appropriate or up-to-date signature file.
2. In Thunderbird go to your account settings (Tools >> Account Settings) and pick your account there.

3. Check where you current signature file resides... 
4. Go there in Windows Explorer and copy (or move) the file to your Dropbox folder.
   You can easily open your Dropbox folder with a double-click on the Dropbox icon in your status bar.
5. Once the file is there, on the Account Settings page change your signature file to the new location. 
6. And then - in Thunderbird - create a new email to check if it has been picked up correctly.
7. Now the other computer. The signature file should already be in your Dropbox folder. If not, trouble-shoot that first.
8. Once you find the file there, only change the location of the signature file in Thunderbird- as above - to the one in the Dropbox folder.
9. Again, create a new email to verify.
10. Repeat for all remaining computers...

Now you can update your signature on any of those computers, and it will be picked up by the others.

Neat, isn't it.
And it didn't cost us a dime.

I started this about 2 months ago and had to change my signature (a least) 3 times or more in the meantime. Works like a charme.

The synchronization of the address book is a bit more complicated, and will be covered in a separate blogpost.

--
[1] and other IMAP clients

How to get Thunderbird attachments as read/write...

This has been bugging me for years now... ( I guess ).
When I double click e.g. an OpenOffice attachment in Thunderbird, the document is then opened as read-only in OpenOffice.

There's 1000+1 reasons on the web why this is a) intended and b) good... but I still did not like it.

Some (short) analysis revealed, that Thunderbird is actually setting the read-only attribute of the attachment, so OpenOffice was not to blame.
Easy to google from that and find a really good tip here:

One has to set the configuration value browser.helperApps.deleteTempFileOnExit to false [1]

E voila... Works fine now.
No unintended side effects so far.
--
[1] you might have to create it...

Thunderbird 3.1 beta 2 out

Finally the beta2 for the new Thunderbird release (no, just an update, actually) is out[2].

It brings 2 important features:

a) the Lightning nightly updates only work with TB3.1 beta for a couple of weeks now... so finally I am up-to-date again with Lightning as well.

b) the fully indexed search introduced in Thunderbird 3 was great, but what annoyed me ever so slightly, was that they used the same search field for filtering and searching and you had to select from the drop-down what you actually wanted to do (filter the current folder view or search all over the mail boxes).

Cumbersome, at least.

Now with TB3.1 they split those into two... a filter bar and a search bar.

Search is hot-keyed [1] with Ctrl-K (kind of a standard anyway) and filter with Ctrl-F (which is mnemonically OK).
Although the filter bar takes away some space from the folder pane, it is worthwhile (to me). You can always hide it again. Not sure if I can make use of the quick filters for unread/starred/contact/... but let's just see.

What still strikes me as odd, that in the ... uhm ... message pane action area (?) ... not sure what it's real name is... you know, that one here, the one that way introduced with TB3...

Ok...that one. So well they are still missing a "File" or "Move to" action there. Do I really have to hack it into TB with an extension ?

Also they are still missing the Favorite sub-menu in the Move to context menu... only recent folders there... what would be so favored with my favorite folders, if not being a frequent move-to-target.

Still, TB3.1b2 seems quite stable... production grade, it'd say...

---
[1] not sure if this is a verb... well now it is.
[2] download here

Regex Pitfall

I guess this was the first time ever that I had to search/match/replace with regular expressions across line boundaries.
Not the usual multi-line ("/m") operation; quite the contrary, I wanted the source string to be treated as one single line regardless of newline-characters in it.

In other words: the dot (".") should also match a newline.

Not that easy, it turns out.
Quoting from the Regex Tutorial:

The dot matches a single character, without caring what that character is. The only exception are newline characters. In all regex flavors discussed in this tutorial, the dot will not match a newline character by default. So by default, the dot is short for the negated character class [^\n] (UNIX regex flavors) or [^\r\n] (Windows regex flavors).

Changing this behavior is actually language dependent.
Since I needed it within JavaScript (XUL/Thunderbird) I had to revert to a [\s\S] instead of the .

JavaScript [does] not have an option to make the dot match line break characters. In
those languages, you can use a character
class such as [\s\S] to match any character.
This character matches a character that is either a whitespace
character (including line break characters), or a character that is not a
whitespace character. Since all characters are either whitespace or
non-whitespace, this character class matches any character.

Weird. Unreadable. Requires a comment.
But it works.

Show sender's picture in Thunderbird 3

With Thunderbird 3 you now can store a photo to each contact.

Nice feature, but totally pointless without an option to actually display this photo next to the emails.

Now for the good part: there is an add-on to do exactly this...

Contact Photos :: Add-ons for Thunderbird

It looks if it can find the sender of the email in any of your address books, and there is a picture stored with this record. If so, it will be displayed in the message header. If not, a dummy picture is displayed.

You can control the behavior a bit...

I know the Mac folks will now brag that their Apple Mail software could do this for years... And you are right... It was overdue.

Now all we need is a way to easily collect those photos (with gravatars not being that popular). Since the address book in Thunderbird still lacks usability, there is of course no way to simple drag and drop images into the address book... Maybe I'll hack this...

Extra Folder Columns :: Add-ons for Thunderbird

For the Thunderbird 3 users out there:

You'll probably be missing the Total/Unread/Size columns in your folder pane...
To get it back, install the Extra Folder Columns :: Add-ons for Thunderbird add-on.
It was specifically created for TB3, but did not support the latest betas and RCs... Now with version 0.4 it supports the final 3.0 release and also works with the smart folders...

Easiest way as usual: open the link in Firefox (or any browser), open the add-on dialog in Thunderbird and then drag the "Download Now" button from the browser to the add-on list in Thunderbird. Voila. No need to download and open the file afterwards (unless you want to archive the .xpi on your local disk)

Now that Thunderbird 3 is out...

... maybe the extension/add-on developers could update their extensions...

The beta-phase was long enough and gave ample time to update... (like I did).

btw: here are the release notes of TB3 in case you haven't installed it yet.

Ctrl-K everywhere

Just noticed the other day, that Ctrl-K (i.e. jump to searchbar) not only works in Firefox and Internet Explorer, but also (tata) in Thunderbird...

Now even more handy with the new indexed full search functionality in Thunderbird 3.0 beta 4....