Terse Systems

I’m trying to get a better estimate of how much time I spend on various tasks.

Part of this is project planning, and part of it is genuine curiosity. If I know how much time I’m spending on various tasks, I can provide better feedback to management and better estimates of my milestones as whole.

Accurate time tracking is the core of PSP, and a number of people have independently recommended the practice. Time tracking for programmers is a hard problem in general though, because it must be done on a minute by minute basis. So most programmers don’t do it.

There is software that does time tracking, but if you look more closely at it, you realise that it tracks hours for billing purposes. For that particular problem, my needs are met. I use Punch Time Clock on the Palm, and it does exactly what I want it to.

No, for this particular problem I need software which would automatically look at what I was doing on the computer, and feed it back to me, while I’m focusing on something else. It needs to be able to look over my shoulder when I work, and tell me when I’m writing an email, looking at a database or writing code. Ideally it should be able to tell me what files I’m working on. Hackystat tries to do this, but it really only keeps track of programs which have specific plugins that talk back to the Hackystat server.

Time tracking applications exist in a sea of applications, and there really isn’t one central place where they’re reviewed. (Sadly, this is the best I’ve found.) I reviewed many of the Time Tracking applications (chucking most of them), and have narrowed it down to three candidates:

Allnetic Working Time Tracker: Some limited automatic tracking, and very good manual tracking. It seems to be the leading contender in that it comes up many different times on different sites.

Worktime: This is a good solid automatic tracking utility. It has a nice UI, stays out of my way, and is reasonably powerful.

Smart Worktime Tracker: This is currently the frontrunner. It looks a couple of years old, but has features which seem specifically designed for programmers, such as regex matching on the title so that you can match which files are being worked on inside an application. More importantly, it can do “smart stuff” which apparently means it’ll group applications automatically according to how you use them. More in the FAQ.

We’ll see how useful this is. Now to go block the comment spam from the blog and add tools to remove the existing spam without pain…

The named query feature in ATG 7.0 is very useful. However, it’s actually confusing to use, because it’s not clear how to use it.

You start off by creating the query in the XML:

  <item-descriptor name="person">


 &lt;named-query&gt;
    &lt;rql-query query-name="myQuery"&gt;
       &lt;rql&gt;name = ?0.name AND age = ?1.age&lt;/rql&gt;
    &lt;/rql-query&gt;
 &lt;/named-query&gt;


  </item-descriptor>

And this exposes a NamedQueryView to the repository. However, it doesn’t show how you execute the query with parameters, which is what you used to be able to do with rqlStatement.executeQuery(view, params). This is actually covered in the section before, concerning Parameterized Queries… but you need to put the two chapters together before it makes sense.

RepositoryItem steve, bill; // assume these get initialized correctly
Repository r = getRepository();
RepositoryView view = r.getView("foo");
NamedQueryView namedQueryView = (NamedQueryView) view;
ParameterSupportView parameterSupportView = (ParameterSupportView) view;
Query myQuery = namedQueryView.getNamedQuery("myQuery");
Object[] parameters = { steve, bill };
// Returns an array of person items.
RepositoryItem[] items = parameterSupportView.executeQuery(myQuery, parameters);

And there you go.

The ATG security framework has long been a source of fascination to me. Mostly because I couldn’t figure the damn thing out.

The canonical reference is here, but there’s one vital piece of information missing. It doesn’t tell you where you get a secured object from.

I broke down tonight and (using this opportunity to try out Enterprise Architect 6.1) mapped it out.

Security in ATG can be broken down into three areas. There’s user and persona related information. There’s access right information. And finally there’s the “secured object” which says who can do what.

There are the rights:

And finally there are the secured objects. The code sample for determining the rights of a secured object is:

SecurityContext context = securedObject.getSecurityContext()
SecurityPolicy policy = context.getSecurityPolicy();
boolean hasAccess = policy.hasAccess(securedObject, user, StandardAccessRights.EXECUTE);

Most of the security framework assumes secured repository items. I had thought that you could use AccessConfiguration, but those classes are not only undocumented, but they’re not functional. The only way I’ve found to get good secured objects is to use /atg/webservice/security/NucleusSecurityManager, mentioned in the Integration Framework guide. This uses a repository for the secured objects, but since it’s a part of DAS and there’s an Admin interface that allows you to add items yourself, it’s not too much hassle.

So. If you want to use restrict access secured object, create a new user in the ACC. Then add the user to a group. Then bring up the NucleusSecurityManager and add a new functional id to it with the ACL of the group. After that, you just get the SecurityObject from the SecurityManager, and run through it as usual.

ATG has been busy lately. The big news has to be the new ATG Eclipse plugin (remote plugin site is http://www.atg.com/eclipse). It includes a Repository Editor that looks like this:

Right click, expand, and override properties. It works with XML file combination (so it only defines the changes you make), and make viewing and editing repositories a snap.

There also some functionality that I’d somehow missed in the shuffle: repositories can now support many to many tables. It used to be (back in prehistory) that they didn’t support them directly because adding an item to a list automatically added it on the other end as well, and the caches got confused. Now bidirectional links just works, without requiring a link table in the middle to disambiguate them.

I both hate and love little tweaks like this, because they make my life easier when I know about them, and make me feel like an idiot when I miss them. And since ATG doesn’t come out with a “new in this release” feature list, the only recourse I have is to read through the manual and classes in every version. It builds character, but it gets a little old.

Still. Named queries support in ATG repositories. This is not a vastly new feature, but I don’t remember it supporting stored procedures or SQL queries before. I’m in favor of named queries, because it keeps all the query logic together. It’s interesting how many layers queries have gone through. I’ve gone from queries in UI to queries in DAO/manager components, and now the queries are completely outsourced back to the repository. Eventually there will be a dedicated query engine that we’ll communicate with and we’ll have to make queries to get the queries out.

There is a “removeNullValues” attribute for multi-valued collections which will make sure that any null elements will not show up when the collection is called. This is actually important, because there are a number of places in DCS where an order has to be reconstituted and will not value the relationships properly if a pipeline processor fails, leaving it null. This kind of thing causes all kinds of headaches, because no-one ever checks for null elements when iterating through a collection. Ever. So it dies with NPE and doesn’t tell you what didn’t work. So thank goodness for that.

I read about the session backup for repository items feature. I don’t know how useful this really is, but I’m sure someone must want it for something. I also found it’s possible to notify a repository through RMI or SQL JMS that it should invalidate its caches through the GSAInvalidationService. The distributed cache concurrency through SQL JMS is also new.

Added to this, I found some really old functionality in the middle of comparing ATG Nucleus to Spring. It turns out that Nucleus has been able to able to expose JNDI through a component reference (atg.nucleus.JNDIReference), and can even call static methods directly (atg.nucleus.StaticMethodReference). There’s even a system for component aliases built in with atg.nucleus.GenericReference, which explains why I always got confused when looking at price lists and secure price lists (they’re actually the same component).

Other than that… well, I’m still investigating atg.repository.test. I’ll see if I can reuse anything there over the mock repository stuff. It would be nice to be able to define a repository, run some integration tests against it, and then start the server…

Hokay. I have a workable configuration environment now.

There’s one master Spring file, and two subsiduary files that contain environment specific properties.

The main file is below (minus the data source):

<bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
  <property name="exposeTransactionAwareSessionFactory" value="false"/>
  <property name="dataSource" ref="dataSource"/>
  <property name="mappingResources">


&lt;list&gt;
  &lt;value&gt;com/tersesystems/blog/dao/hibernate/PostImpl.hbm.xml&lt;/value&gt;
  &lt;value&gt;com/tersesystems/blog/dao/hibernate/CommentImpl.hbm.xml&lt;/value&gt;
  &lt;value&gt;com/tersesystems/blog/dao/hibernate/CategoryImpl.hbm.xml&lt;/value&gt;
  &lt;value&gt;com/tersesystems/blog/dao/hibernate/MemberImpl.hbm.xml&lt;/value&gt;
  &lt;value&gt;com/tersesystems/blog/dao/hibernate/MediaImpl.hbm.xml&lt;/value&gt;
  &lt;value&gt;org/appfuse/model/Role.hbm.xml&lt;/value&gt;
  &lt;value&gt;org/appfuse/model/User.hbm.xml&lt;/value&gt;
  &lt;value&gt;org/appfuse/model/UserCookie.hbm.xml&lt;/value&gt;
&lt;/list&gt;


  </property>      

  <property name=”hibernateProperties” ref=”hibernatePropertiesBean”/>    

</bean>

Note the references to the hibernatePropertiesBean. This is an instance of PropertiesFactoryBean that will be defined in the environment Spring file.

The JTA Spring file is defined below:

<bean id="hibernatePropertiesBean" class="org.springframework.beans.factory.config.PropertiesFactoryBean">
  <property name="properties">


&lt;props&gt;
  &lt;prop key="hibernate.dialect"&gt;${hibernate.dialect}&lt;/prop&gt;
  &lt;prop key="hibernate.cache.use_query_cache"&gt;${hibernate.cache.use_query_cache}&lt;/prop&gt;         
  &lt;prop key="hibernate.cache.use_second_level_cache"&gt;${hibernate.cache.use_second_level_cache}&lt;/prop&gt;        
  &lt;prop key="hibernate.cache.provider_class"&gt;${hibernate.cache.provider_class}&lt;/prop&gt; 
  &lt;prop key="hibernate.current_session_context_class"&gt;${hibernate.current_session_context_class}&lt;/prop&gt;
  &lt;prop key="hibernate.show_sql"&gt;${hibernate.show_sql}&lt;/prop&gt;        
  &lt;prop key="hibernate.connection.release_mode"&gt;auto&lt;/prop&gt;
  &lt;prop key="hibernate.query.substitutions"&gt;${hibernate.query.substitutions}&lt;/prop&gt;          
  &lt;!-- Must be enabled for JTA transactions... --&gt;    
  &lt;prop key="hibernate.transaction.manager_lookup_class"&gt;${hibernate.transaction.manager.lookup}&lt;/prop&gt;  
&lt;/props&gt;


  </property>  

</bean>

and the JDBC one is defined likewise. Note that we can’t have even an empty property existing for hibernate.transaction.manager.lookup here.

<bean id="hibernatePropertiesBean" class="org.springframework.beans.factory.config.PropertiesFactoryBean">
  <property name="properties">


&lt;props&gt;
  &lt;prop key="hibernate.dialect"&gt;${hibernate.dialect}&lt;/prop&gt;
  &lt;prop key="hibernate.cache.use_query_cache"&gt;${hibernate.cache.use_query_cache}&lt;/prop&gt;         
  &lt;prop key="hibernate.cache.use_second_level_cache"&gt;${hibernate.cache.use_second_level_cache}&lt;/prop&gt;        
  &lt;prop key="hibernate.cache.provider_class"&gt;${hibernate.cache.provider_class}&lt;/prop&gt; 
  &lt;prop key="hibernate.current_session_context_class"&gt;${hibernate.current_session_context_class}&lt;/prop&gt;
  &lt;prop key="hibernate.show_sql"&gt;${hibernate.show_sql}&lt;/prop&gt;
  &lt;prop key="hibernate.connection.release_mode"&gt;auto&lt;/prop&gt;
  &lt;prop key="hibernate.query.substitutions"&gt;${hibernate.query.substitutions}&lt;/prop&gt;   
  &lt;!-- Do not allow transaction.manager.lookup to be defined here. --&gt;       
&lt;/props&gt;


  </property>  

</bean>

This doesn’t include the changes to the transaction manager or the like, but it gets the point across.

Then, since the servlet isn’t around to do it for me, I have to define a hibernate session “in-house” for integration tests:

public void onSetUp() throws Exception
{  



super.onSetUp();

ApplicationContext context = applicationContext;
sessionFactory = (SessionFactory) context.getBean("sessionFactory");
sessionFactory.getCurrentSession().beginTransaction();


}

@Override
protected void onTearDown() throws Exception
{ 



if (transaction != null) 
{
    transaction.commit();
}

super.onTearDown();


}

And now all my integration tests work! Yay!

I used Carlos Sanchez’s post as a guide, and have been looking at Alef’s Configuration Management with Spring for more information.

Still surprised at how much work it is to override properties in Spring. As far as I know, there’s no way you can explicitly say “I’m scribbling over another bean definition” now, apart from where you’ve explicitly set it up that way with PropertyOverrideConfigurer. Among other things, I don’t see any way that you can change a bean’s defined class once it’s set up, which is something you can do in Nucleus right off the bat.

Still, it works, and I should be thankful I can do integration testing at all. Now to go look at Webwork forms.

So. In the previous post, I talked about integration testing. An integration test runs through Webwork, Hibernate and Spring and expects a certain amount of scaffolding to work right.

I’ve been having a very tough time getting that scaffolding to work right. Part of the problem is that there is no mutually agreed pattern for handling Hibernate sessions.

When I am using the Spring design pattern (OpenSessionInView), the Spring integration tests work fine. But that pattern doesn’t allow lazy loading unless you do some fancy acrobatics. And if you write your own code to handle Hibernate sessions… things don’t work so well.

Specifically, I wrote a transaction filter and a session per request filter:

Session session = sessionFactory.getCurrentSession();
try
{


// Joins the existing JTA transaction, or creates one.
session.beginTransaction();
//boolean rollback = true;
try
{
    filterChain.doFilter(request, response);
    // commented out because JTA transaction seems to
    // do this for us...
    //rollback = false;
} finally
{
    //if (rollback)
    //    tx.rollback();
    //else        
    //    tx.commit();
}


} finally
{

session.close();


}

and then just did a very simple DAO pattern that called sessionFactory.getCurrentSession() from inside the DAO. Now, if I were using Hibernate 3.0, then getCurrentSession would require a JTA transaction to be in place, and I wouldn’t be able to run an integration test unless I found a replacement J2EE container. However, I’m using Hibernate 3.1. Hibernate 3.1 does not require an existing JTA transaction to be in place for getCurrentSession(). So I should be fine, right?

Wrong. It turns out that Spring wraps all calls to the sessionFactory object and calls its own utility class instead. Every time you call sessionFactory.getCurrentSession(), Spring calls SessionFactoryUtils.doGetSession() for you. And doGetSession() checks to see if there’s a JTA transaction, and if there isn’t one… it doesn’t let you call getCurrentSession.

In fairness, this has been noted as bug SPR-1354 and fixed in CVS. But that does me no good.

So what I tried to do is setup the integration tests with a different Spring configuration. Instead of setting up the sessionFactory with JTATransactionManager and OrionTransactionLookup, I’d extract all the relevant beans and properties, and have applicationContext-jta.xml and applicationContext-jdbc.xml files only containing the relevant transaction config. The sort of thing you could do in ATG in five minutes using a Nucleus layer.

This… sort of works.

However, I’m having trouble setting up the Hibernate properties. This breaks because when I’m in the app server, I need to set transaction.manager_lookup_class to OrionTransactionLookup. But I need it not to exist for the integration testing. I can define a property with a null value, but Hibernate still sees that the key exists and complains about it.

About the only solution I can think of is to have the props map abstracted out to a properties file and see if I can swap in different properties files. (EDIT: Fixed, see here for details.)

The interception framework should be optional – it should let Hibernate fail on its own. And HibernateTemplate/OpenSessionInView stuff gets around a problem that, as far as I can tell, doesn’t exist.

I don’t get it. I thought this was supposed to be easy.

EDIT: okay, NOW I get it.

EDIT: Also see here for an excellent review of Spring’s “integration” with Hibernate.

So far I’ve been focusing on Webwork. However, I’ve also been trying more automated testing again. My previous effort didn’t work so well, as I spent more time writing the mockobjects library than writing the unit tests, but I was happy to give it another try using TestNG.

Unit tests work fine for me, but I’m still not fascinated by them.

Things got more interesting when I discovered the integration testing classes in Spring. I found that the Spring integration test classes were built on top of JUnit, and didn’t seem to support TestNG. So I switched, and it was so worth it.

Integration testing with Spring is awesome. I can write some code, write an integration test, then run through the test until I’m sure I know what’s going on. This really helps when parsing posts. A raw post goes through several different parsers to do things like colorize Java code, bleep swearwords and escape HTML characters, so there are many opportunities to parse things in the wrong order or not at all.

Here’s a sample:

public class DisplayPostActionTest extends


    AbstractDependencyInjectionSpringContextTests {
@Override
protected String[] getConfigLocations() {
    return fakeDatabasePaths;
}

private static String[] fakeDatabasePaths = {
    "classpath*:/applicationContext-test.xml", "classpath*:/applicationContext.xml", 
};

DisplayPostAction displayPostAction;

/* (non-Javadoc)
 * @see org.springframework.test.AbstractDependencyInjectionSpringContextTests#onSetUp()
 */
@Override
protected void onSetUp() throws Exception {
    ApplicationContext context = applicationContext;        
    displayPostAction = (DisplayPostAction) context.getBean("displayPostAction");
}

public void testExecute() {        
    try {
        displayPostAction.setPostId("43");
        String returnValue = displayPostAction.execute();
        assertNotNull(returnValue);

        assertTrue(Action.SUCCESS.equals(returnValue));

        Post post = displayPostAction.getPost();
        assertNotNull(post);            
    } catch (Exception e) {
        e.printStackTrace();
        fail();
    }
}


}

I have a couple of really big posts going over Spring and Hibernate, but I’m sitting on them until I know what to say. Of course, the longer I think about it, the more there is I could say…

So. Johanna had an interesting post about her office and how she cleaned it up. Here’s a photo of my office.

I’m not telling you if that’s the before or after picture.

However, I will say that I owe my soul to Crate and Barrel. And having an honest to god filing cabinet and labeller makes life so much easier.

Also, I like pizza.

I have a question. Hibernate for the most part does exactly what I expect it to, and all is good. However, the elements() function doesn’t seem to be working as it should. I suspect my mapping may be off.

Is this legal?

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-mapping PUBLIC


    "-//Hibernate/Hibernate Mapping DTD 3.0//EN"
    "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd"&gt;


<hibernate-mapping package=”com.tersesystems.blog.dao.hibernate”>

  <!–   



CREATE TABLE blog_category (
category        varchar2(40)    NOT NULL,
PRIMARY KEY(category)
);    


  –>

  <class name=”CategoryImpl” table=”BLOG_CATEGORY”>  



&lt;id name="name" column="CATEGORY"&gt;      
&lt;/id&gt;


  </class>

</hibernate-mapping>

and

public class CategoryImpl implements Category
{


private String mName;

/**
 * @return Returns the name.
 */
public String getName() {
    return mName;
}

/**
 * @param pName The name to set.
 */
public void setName(String pName) {
    mName = pName;
}

public String toString() {
    return "CategoryImpl: " + mName;
}


}

Because when I do this query:

String queryString = "from PostImpl as post where :category in elements(post.categories) "


         + " and post.state = :state order by post.date desc";


Query query = session.createQuery(queryString);

query = query.setParameter(“category”, pCategory);
query = query.setInteger(“state”, PostState.PUBLISH.asCode());
List posts = (List) query.list();

I get this:

2005-12-31 13:48:46,046 INFO [org.hibernate.event.def.DefaultLoadEventListener] - <Error performing load command>
org.hibernate.ObjectNotFoundException: No row with the given identifier exists: [com.tersesystems.blog.dao.hibernate.CategoryImpl#
Programming, Programming]


    at org.hibernate.ObjectNotFoundException.throwIfNull(ObjectNotFoundException.java:27)
    at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:118)
    at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:75)
    at org.hibernate.impl.SessionImpl.immediateLoad(SessionImpl.java:643)
    at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:59)
    at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:84)
    at org.hibernate.proxy.CGLIBLazyInitializer.intercept(CGLIBLazyInitializer.java:134)
    at com.tersesystems.blog.dao.hibernate.CategoryImpl$$EnhancerByCGLIB$$6d6de9a8.toString(&lt;generated&gt;)
    at java.text.MessageFormat.subformat(MessageFormat.java:1237)
    at java.text.MessageFormat.format(MessageFormat.java:828)
    at java.text.Format.format(Format.java:133)
    at java.text.MessageFormat.format(MessageFormat.java:804)
    at com.tersesystems.blog.dao.hibernate.HibernatePostDAO.findPublishedByCategory(HibernatePostDAO.java:192)
    at com.tersesystems.blog.action.DisplayPostsAction.execute(DisplayPostsAction.java:130)</pre></div>



It’s disconcerting.  I’m not even sure what the error message means.
          
        
      
      
  

• Posted in Blog, Java, Programming
• Meta 2 comments, permalink, rss, atom