Terse Systems

I’m on the SHAPE forum for software engineering management, and brought up the question about which methodology is the most appropriate for which situation.

I brought up three situations, and then realized there weren’t any factors I could weight for a heuristic. So I made them up, starting from a list I already had on factors that go into estimating software projects.

In the process, I’ve also made the attempt to list every commonly used software methodology and/or process framework. I’m certain I’ve missed some (such as ICONIX) but at least this is a good start.

Different people made different recommendations. The reasons why they made them was at least as interesting as the decisions they made.

I’ve also found a couple of books, notably Balancing Agility and Discipline which deals explicitly with people who are on the fence when it comes to software methodologies. I’ll come back and update this post as I get a better picture.

I’m commuting down to LA for my current gig. That gives me about two hours free before the plane flight where I have pretty much nothing to do. So I work on my blog and read.

I’ve given up on blocking the comment spam in the current version of the blog. It’s going to make more sense to do it in the next release. So (following the test-first methodology) instead of actually writing code to create and delete comments in Hibernate, I’m researching dbunit and trying to figure out the best ways to strip and reinstate a database in different apps. I can’t decide whether this is wise or not.

I’ve been researching programming methodologies again (here’s last time – even if you don’t read it, read The Inevitable Pain of Software Development, which is by far the best paper I found). The hard bit is getting empirical data which says what methods were successful under which circumstances. However, I did find a review of agile programming methods that gave me some very useful data points. The interesting thing is that it’s all been done before.

I also found the programming podcasts on itconversations. These are mysteriously addictive – I can play them sitting in the airport or waiting for a bus and feel like I’m doing something useful. I have to admit I don’t like the not-so-subliminal ads that the speakers put in for their books, products, and seminars, but you can learn more context in an interview than you can in a book because the interviewer actually understands software development.

Now I want to read Agile Software Development. But how useful is that in practice? How useful is any of this in practice? I don’t know. And getting a good quality filter in place is almost as expensive as ignorance.

I’ve been playing around with the ATG Eclipse plugin. I like the Repository editor, but it can’t cope with more than two layers of XML file combination, so it’s not useful for modifying large scale catalogs. But it is great for proof of concept repositories and prototypes. And I like that I don’t have to run through a build script when using an ATG project in Eclipse.

The side effect of using the plugin is that I’ve been creating a whole bunch of disposable ATG modules. I’m fairly good at cranking these out now, but they’re a hassle in bulk. So I wrote an ant script that creates ATG modules from a template. This will setup the EAR, WAR, and libs for you automatically. Click here for the file.

There’s also some stuff in there about ATG versioned modules. I remember that there was a big annoying problem with CampaignOptimizer (otherwise known as A/B Testing). The details are in build.properties, but I don’t remember precisely.

It’s worth noting that versioned modules could have some advantages over regular modules for large scale enterprise builds. Notably, versioned modules can contain scripts that can automate deployment. Jason Goth goes over this in his presentation at the ATG Open. Relevant bits are as follows:

Versioned Modules
- Each versioned module contains an install script(s).
- The AACom module is a “shell”
– Creates a dependency on the actual version of the site
– The MANIFEST.MF contains just the following:
ATG-Required: AACom-1_36

Versioned modules allow completely automated deployments.
- The deployment script:
– Copies the module files
– Stops a portion of the servers
– Runs the install script(s)
– Starts Dynamo
– Repeat process for remaining servers
- Written in Perl

The deployment script allows us to:
– Upgrade without manual intervention
– Avoid inconsistencies during the restart
– Handle failed file transfers
- Don’t start those servers
– Rollback in case of failure

This is taken from the powerpoint slides. I have no idea exactly what he meant by much of this – since ATG versioned modules are completely undocumented, much of what I already know is trial and error. Still, it’s a good starting point.

Okay, time to go through the toolkit again. It’s been a while since I did this. This time it’s Firefox Extensions.

HTML Validator. If you are an HTML designer, a page developer, or indeed anyone who makes web pages for a living, you need this extension. I can’t tell you the number of headaches it has saved me by pointing out exactly where something doesn’t match up.

Aardvark. This application is superficially similar to the DOM inspector, but it has a far nicer interface. You turn it on, and all the page elements are tagged for you on the page. This sounds a little abstract, so here’s a demonstration:

Firebug. This is a number of different things, and it’s already had very good press. It does a number of things, but the best thing from my perspective is that it validates CSS, debugs Javascript (in a way far superior to Venkman) and can track XMLHTTPRequests. If you’re working on AJAX applications, Firebug pulls many different things together to form a unified whole.

Scrapbook. This extension changes the way I surf the web. You drag and drop a bookmark or a page into it, and it stores the page for you. You can then read the page offline, without being connected to the web. And because it’s local, you never have to worry about losing the URL. I’ll drag pages in and use them for reference at meetings, for long articles, for web page receipts…

That’s it for now – there are other firefox extensions I like, but they’re personal preferences rather than new features.

So it turns out IF you have a Windows machine which uses a wireless connection for the Internet, AND you wire it up to an ethernet network for an internal subnet, AND the ethernet is faster than the wireless connection, THEN Windows thinks “hey, this new network is faster than the old one, let’s send everything there!” And tries to send Internet packets to the ethernet.

So you can’t rely on the automatic metric. More details here.

I had a couple of hours in the airport a couple of weeks ago, and decided to use them to write some Yahoo widgets.

The first one is an expansion on the existing TenPlusTwo widget: I’ve added a whole bunch of options, such as configurable popups, sounds and programs. I tried getting in touch with the original author so I could submit the code back as an enhancement, but he has no email address! So this will have to do.

The user preferences look like this:

The second one is a heavy reworking of the Yahoo Notepad, into a pure GTD inbasket. This thing just takes in text that you type in, then runs a script against it.

It looks like this:

The precise nature of the script is configurable in the preferences, so you can run a Perl, Python or Ruby script against it.
In this case, my script executes the following command

"c:/usr/perl/bin/perl.exe d:\home\wsargent\work\eccoapi\scripts\command.pl 'this is a test'"

In this case, the inbasket dumps into Ecco Pro, the only application that reduces me to zealotry. But you can write your own scripts to write out to a text file, a web application, or wherever.

You can download both widgets here: Inbasket and TenPlusTwo.

Total time spent: two hours, plus the time spent to write this. Widgets rock.

I wrote my blog from the ground up. It seemed like an interesting technical exercise at the time. Now I have actual content and people who comment, I’m in a much better position to appreciate the benefits and drawbacks.

Plus: I know exactly what’s going on. It does exactly what I want.
Minus: If something doesn’t work, I can’t draw on a user community to fix it for me.
Plus: I can use the blog as a proving ground for new technology.
Minus: Spammers also learn new technology.

I thought I was very clever for writing my own blog, because spammers were oriented against WordPress and Blogger and wouldn’t bother with as small a target as one custom written blog. I failed to take into account that spammer technology would advance to the point where they would figure out how to act like users and hit a comment button without regard to the back end technology.

I tried cleaning out the spam manually. This is like trying to stop the tide with a bucket. The problem is to come up with a comment submission scheme that spammers cannot hack, and that I need to think about.

In the meantime, I’ve disabled comments altogether. And in lieu of deleting the comments by hand, I’ve used this problem as an opportunity to think about how to do ad-hoc deletes from the repository.

Hence the BeanshellAdminService. This little beauty runs in the AdminServer, and allows me to run beanshell scripts on the server. It’s so convenient that I can’t believe I didn’t think of it years ago. For development, it lets you do just about anything.

I’ve uploaded it to the usual place. This is useful for your normal services as well. You can override the admin servlet to set your own parameters for the interpreter, so if there’s a function that you want to pull in or some parameters that you want to set, you can set them with an anonymous subclass. Best to do that from another module though, so you don’t inadvertently deploy scriptable code to production…

I’m trying to get a better estimate of how much time I spend on various tasks.

Part of this is project planning, and part of it is genuine curiosity. If I know how much time I’m spending on various tasks, I can provide better feedback to management and better estimates of my milestones as whole.

Accurate time tracking is the core of PSP, and a number of people have independently recommended the practice. Time tracking for programmers is a hard problem in general though, because it must be done on a minute by minute basis. So most programmers don’t do it.

There is software that does time tracking, but if you look more closely at it, you realise that it tracks hours for billing purposes. For that particular problem, my needs are met. I use Punch Time Clock on the Palm, and it does exactly what I want it to.

No, for this particular problem I need software which would automatically look at what I was doing on the computer, and feed it back to me, while I’m focusing on something else. It needs to be able to look over my shoulder when I work, and tell me when I’m writing an email, looking at a database or writing code. Ideally it should be able to tell me what files I’m working on. Hackystat tries to do this, but it really only keeps track of programs which have specific plugins that talk back to the Hackystat server.

Time tracking applications exist in a sea of applications, and there really isn’t one central place where they’re reviewed. (Sadly, this is the best I’ve found.) I reviewed many of the Time Tracking applications (chucking most of them), and have narrowed it down to three candidates:

Allnetic Working Time Tracker: Some limited automatic tracking, and very good manual tracking. It seems to be the leading contender in that it comes up many different times on different sites.

Worktime: This is a good solid automatic tracking utility. It has a nice UI, stays out of my way, and is reasonably powerful.

Smart Worktime Tracker: This is currently the frontrunner. It looks a couple of years old, but has features which seem specifically designed for programmers, such as regex matching on the title so that you can match which files are being worked on inside an application. More importantly, it can do “smart stuff” which apparently means it’ll group applications automatically according to how you use them. More in the FAQ.

We’ll see how useful this is. Now to go block the comment spam from the blog and add tools to remove the existing spam without pain…

The named query feature in ATG 7.0 is very useful. However, it’s actually confusing to use, because it’s not clear how to use it.

You start off by creating the query in the XML:

  <item-descriptor name="person">


 &lt;named-query&gt;
    &lt;rql-query query-name="myQuery"&gt;
       &lt;rql&gt;name = ?0.name AND age = ?1.age&lt;/rql&gt;
    &lt;/rql-query&gt;
 &lt;/named-query&gt;


  </item-descriptor>

And this exposes a NamedQueryView to the repository. However, it doesn’t show how you execute the query with parameters, which is what you used to be able to do with rqlStatement.executeQuery(view, params). This is actually covered in the section before, concerning Parameterized Queries… but you need to put the two chapters together before it makes sense.

RepositoryItem steve, bill; // assume these get initialized correctly
Repository r = getRepository();
RepositoryView view = r.getView("foo");
NamedQueryView namedQueryView = (NamedQueryView) view;
ParameterSupportView parameterSupportView = (ParameterSupportView) view;
Query myQuery = namedQueryView.getNamedQuery("myQuery");
Object[] parameters = { steve, bill };
// Returns an array of person items.
RepositoryItem[] items = parameterSupportView.executeQuery(myQuery, parameters);

And there you go.