Terse Systems

This is going to be a long and boring post about managing information. The short version is I’m using Yahoo Mail, courier-imap, Shadow Plan and Ecco Pro and am very happy about it. I previously wrote about this here, and here.

One of my biggest problems since I was about… ten. I’m a total packrat. I want to keep everything I’ve ever done, thought, read about accessible. As a result I’ve had everything from a Word diary to spiral bound notebooks to Palm calendars to Microsoft Outlook to email to well, this blog.

It doesn’t work. I’m always missing some information somewhere, and it bugs the hell out of me. Even worse is the context – the Word diary was great for reading through stuff I did from the time I was 15 to the time I was 21, but after that point I working and I just couldn’t use it to make appointments and todos effectively.

Spiral bound notebooks are great because they’re mobile and they have context. But they don’t scale. I can’t search them, it’s really hard to figure out order sometimes. Pages get ripped out, I have more than one notebook if I’m on different projects sometimes. My typing is faster than my longhand as well. I tried teeline shorthand for a bit to see if that helped, but tht dndnt wrk s wll gvn tht I hd t lve out all th vwls. Oh, and I never really learned how to read teeline. So I switched to e-mail. Search, sort and filtering is built in, and the essay format was perfect – just sit down and type out freeform.

But e-mail had scaling problems. Especially when I had two e-mail accounts and four different computers. I was always missing e-mail somewhere because it had been downloaded onto the wrong machine, and sorting and merging was annoying as hell.

Email got easier when I got tersesystems.com. I set up an IMAP server, and started using Thunderbird as a mail client. Now I have one repository of contact information and history that is pretty comprehensive, and it works great as a fallback. It doesn’t matter if I didn’t make a note or don’t have someone’s phone number – with IMAP’s server side search I can type in the name and filter down to the e-mail with his contact info. Any work information I think I’ll need later (meetings, notes, funny jokes) I’ll just forward home. However, this does have some drawbacks.

I’m not worried about losing any e-mail, as johncompanies.com will backup my maildir nightly and they have good quality hardware. I use SSH port forwarding and SSL, so I’m not too worried about the security. I am, however worried about reliability. When I sent mail directly to will@tersesystems.com, some email would bounces, and sometimes Exim or my collated box would go down.

So I have another level of backup. My home account is will_sargent@yahoo.com, so e-mail gets sent to Yahoo first. I then pull e-mails off Yahoo through POP3 using fetchmail. If my hosted box or IMAP server go down, my e-mails doesn’t bounce and I can still get a minimal level of reliability. I also rely on Yahoo to do my spam filtering. Spamassassin and Vipul’s Razor had to be constantly tweaked to work, and the combination of Yahoo and Thunderbird means that very little spam gets through. For this, I pay them $25 a year, which I think is well worth it.

But it’s difficult to use email for appointments and tasks. It records what you have done, not what you’re about to do. You also need to be in front of a computer that’s both turned on and online. If you’re just trying to keep track of lots of low level stuff, PDAs work far better.

PDAs are great. Take it out, tap out a few notes, put it back. Sync with the desktop later. But it was difficult to write anything of import on the Palm, and the Palm desktop wasn’t exactly wonderful. Notes were pitifully organized, tasks couldn’t be nested, and it didn’t allow linking between tasks. So the desktop UI was out. Outlook had a much better UI, but the same limitations. And Outlook tasks really, really suck. I found a great Task organizer called Shadow Plan. The desktop UI still sucked though. After being used to e-mail and word processors, it was hard to go back to something so primitive.

Which is where Ecco Pro comes in. Ecco basically is a database where all the folders (otherwise known as attributes, columns or properties, depending on what background you’re from) of an item (row, bean, etc) are optional. Attributes can have different data types (boolean, Date, String, etc.), and you can create your own attributes. If you assign an predefined attribute like ‘Appointment’ to an item, then it shows up in the calendar. If you add a predefined attribute like ‘Todo’ to an item, then it shows up in the Task list. In other words, the same item can be both an appointment and a todo, and you can filter by context. Yes, it’s intertwingled. And you can do outlining in it – items can be nested under other items. The desktop UI is perfect.

Edit: Well, it’s sort of perfect. There’s been the criticism that Ecco is an “outliner construction set” that allows you to set up your own system, but is a bit too freeform to do any good in its default mode. I used Dave Gustafson’s GTD template, so I didn’t have that problem.

The trouble is that Ecco is about seven years old and not maintained. However, it does support DDE and Palm Synchronization. The Palm hotsync was a bit dodgy at first (involving the use of regedit), but it just involves making sure that Application0 through 4 have the right DLLs.

But Ecco didn’t know anything about Shadow Plan. Luckily, Shadow uses XML as a file format. I wrote an converter which imports the XML and exports indented text, and vice versa. Then I wrote a Visual Basic macro and called it from a batch file to read the indented items and use DDE to pipe them into Ecco. Then I did that in reverse, and now I have an Inbasket file (imports to Ecco) and an Actions file (exports from Ecco) and all my diary stuff synchronized.

This isn’t a perfect solution. Ecco doesn’t work with Thunderbird or IMAP very well. It links up to URLs okay (it searches for http:// prefixes and lets you launch a browser if it finds one). Can’t google from it, but then again I never could. I found a tool called EccoHelpers that will auto categorize items for me. I thought about using Ecco for time tracking, but using Punch Time Clock from http://www.psync.com on the Palm Pilot works better. And for recurring tasks, I use Karen’s Countdown Timer in conjunction with the command.pl script to drop new tasks into my Inbasket automatically.

Congratulations if you made it this far. I think this stuff is fascinating. Everyone has their own system, but I never hear about it unless I specifically hear about it. But we make our lives and our livelihoods from this stuff.