2.0EA1 is released!

Finally. It's been quite a long haul to get here, but I've just published the first early access release of OpenQuote 2.0 to source forge as a packaged, downloadable binary.

Download it from here: openquote-community-2.0EA1.zip

Some screenshots... just because I can:

The old adage that in software development the last 20% of progress takes 80% of the effort seems accurate. Since I last blogged, some new features have been added, and some others improved. The intention that 2.0 should effectively deliver the same feature set as 1.4, just on a totally new platform still holds more or less true, but in some cases the new platform made it considerably easier to do things in a very much more technically concise and more user friendly way than was possible in 1.4.

A good example of this is the way the configure system and it's caching mechanism deal with changes at runtime. In the world of 1.4 a product developer would frequently have to manually reset configurations and clear caches to get their latest product change picked up. In 2.0, thanks to a Liferay document library event hook, a bit of JMS magic and some enhancements to the configuration services, it should never be necessary for a user to think about reseting configurations or clearing caches - it will all happen automatically.

A big chunk of the that last 20% was about testing and getting the new Bamboo server working (thanks again to Atlassian for letting the project use their OnDemand service). Historically, OpenQuote's continuous integration build didn't run any tests at all. Obviously, this is not a good thing. In fact, it's rather shameful, and something that 2.0 had to address. Now the unit tests and integration tests are run during each build. The UI tests are still manual.

The move to OnDemand Bamboo was far more involved that expected. Elastic bamboo is a truly fantastic thing though. Bamboo controls the builds, but all of the actual build activity takes place on Amazon EC2 instances which are started/stopped automatically as needed. Getting elastic bamboo up and running wasn't too much of a problem, but some curious build errors that couldn't be reproduced anywhere but on the build server caused a prolonged head scratching episode.

It turned out that the errors were caused by a configuration difference in MySQL. But I only found this out moments before the men in white coats were scheduled to arrive with some new brightly coloured smarties for me to try. On MacOS and Windows, MySQL installs itself by default to use case insensitive queries. On other flavours of Unix (e.g. our elastic bamboo instances) it is case sensitive by default. Simple when you know it, obvious even. But I'd been down any number of dead ends before I discovered this little nugget.

So, 2.0EA1 is finally out of the door. EA2 will not be too far behind it. The priority now is to split PageFlows out and make them independent of the quotation process so that they can be used for other things - like MTAs and Renewals, but also things which aren't related to policies at all. We also need to work on the themes. The new OpenQuote theme (did I mention that?) works well and keeps screen clutter to an absolute minimum, but it doesn't make the quote screens themselves look very pretty and we need new themes for the demo products too.

Much to be done. No time for rest!

2.0EA1 - so close you can smell it

It's quite a while since I last blogged, but we're making swift progress towards the first Early Access (EA) release of OpenQuote 2.0. If the term EA doesn't mean anything to you, there is a page on the wiki that explains all.

So, since the last post, here's what we've been up to:

• The configuration reset mechanism has been enhanced so that it a) happens automatically when the system is first started up, and; b) can be invoked via a web service. OpenQuote has always had some support for web services, but we're building that out to cover more services for 2.0. For the EA release this will include the ability to reset configurations and clear caches.
• The 1.4 integration tests now all pass in 2.0. So we now have both unit tests and integration tests working. This just leaves the product tests, of which the widget showcase test is already passing. Most of the problems found by the integration test were related to class loading. The accessors and the factories do quite a bit of reflection and were, for the most part, picking up the wrong classloaders for use in JBoss 7.
• The Quotation portlet is now working. This was a major step forward. The configuration of the portlet (i.e. the way in knows which product to quote for) has been much simplified. Rather than editing the portlet XML files as the product developers has to do in 1.4, you now simply switch the portlet to edit mode and select the product that you want from a drop down menu. Much simpler.
• The Sandpit portlet is now working. This, again, was a major step forward. Though not as much work as it reuses the Quotation portlet. However, this now means that we're in a position to run all of the product tests.

What's left before we call it an EA? Well, we need the cache manager portlet to be working. Using the web service to do this is okay, but a bit painful for everyday use. The product catalog portlet also needs to be ported across from 1.4. These are both quite simple portlets, but I want to move them on from the current JSF implementations into the brave new world of GWT. There isn't a huge advantage to using GWT in such simple portlets, but we have more extensive places for GWT in future so it makes sense to cut our teeth on something simple first.

Long live OnDemand JIRA

The old OpenQuote JIRA is dead. Long live the OpenQuote JIRA.

I'd been putting off migrating our old JIRA to Atlassian's OneDemand service (which I blogged about a while ago). But now it is done, and it turned out to be a lot simpler than expected.

In principal migrating between instances of JIRA is easy. You simply use the admin functions to export content from one server, and the import it into another. If you have a complex setup there can be more to it, but our setup is simple so that should be all that's necessary. However, our old server was very old. We were running version 3 of JIRA and the OnDemand server is running version 5. You can't import version 3 format exports into version 5.

Thankfully, Atlassian provide very detailed docs on how upgrade between version. Very detailed and very copious docs which rather put me off. But a couple of days ago I found myself with a spare hour or two so I took a punt at it. And in that couple of hours the job was done.

All I needed to do was to create an export from the old server. Install a standalone JIRA 4.0 and import into it. Then immediately export again from that server. This gave me a version 4 export which imported without issues into the OnDemand system.

The URL remains the same: http://openquotecommunity.org/jira, it now redirects to OnDemand. All users have also been migrated. In fact even the site logo and look and feel customisations came across too. Can't be bad.

Protocol handlers and JBoss

The last of the fixes necessary to support the quotation portlet are now committed to the 2.0 branch. It'll be nice to be doing some UI again after such a long spell in the darkness, but more on that later.

The two last updates that were needed were support for the product: protocol, and web service access to the configuration methods - to reset configuration, clear the cache etc. Both had the wrinkles wrt the new 2.0 liferay/jboss stack.

The "product:" protocol handler is pretty well essential for any of OpenQuote's product functionality. The system uses it to access all of the product content held in CMS. The old 1.4 product protocol handler left a little to be desired in terms of configurability, so that has also been addressed for 2.0. But the first problem was how to make a protocol handler visible to JBoss.

This turns out to be simple. Kinda. The first thing you need is the protocol handler class itself. In this case that class is com.ail.core.urlhandler.product.Handler. This has to follow the normal standard for protocol handler's in Java - it has to be in a package named after the handler (product in this case), and it must be called Handler, and it must extend URLStreamHandler.

For JBoss to recognise the hander, you have to do three things.

1. Create a URLFactoryHandler. JBoss will use this class to create instances of your new handler as it needs them. The (very simple) class can be found in com.ail.core.urlhandler.URLFactorHandler, it isn't much more than a switch statement. 
2. Both the URLFactoryHandler and the product Handler itself are packaged up into the com.ail.core JBoss module, but JBoss won't find them itself. It needs to be told where to look. To that end, add the file META-INF/services/java.net.URLStreamHandlerFactory (no file extension), to the same module. The file just contains one line naming the URLFactoryHandler class.
3. Tell JBoss that the module contains URL handlers on the JVM command line. In our case I added the following to <jboss>/bin/standalone.conf (and standalone.conf.bat to accommodate our Windows friends): -Djboss.protocol.handler.modules=com.ail.core

Job done. Product registries can now refer to "product:~/PageFlow/Quotaiton.xml" or whatever, and they will get the content direct from the product tree in CMS.

The other change made to the protocol handler was to paramatise it's configuration. As the location and of the product tree may be moved in CMS (I can't think why anyone would move it, but I don't see why they shouldn't be able to either), and the credentials used to access CMS certainly need to be configurable. These are now configured in the Core's configuration in the ProductURLHandler group.

As before, all of this has been committed into SVN. There is no binary distribution of 2.0 yet, so you will need to build form the source if you want to play.

We'll come back to the addition of web service access to the configuration server next time.

Mostly Earless

OpenQuote 2.0 will be an earless release. As I blogged last time, up until 2.0 OpenQuote was packaged into an EAR, but issues with Liferay's support for portlets within EARs has lead us down a different road now, and it's not one I'm sorry to be travelling down.

So, since the latest commits into the 2.0 tree, the EAR is history. Our various JEE components now all sit in the deployments folder and their dependencies are managed using JBoss modules and are resolved at build time using Ivy.

The move to JBoss modules and Ivy was pretty smooth. Very smooth in fact. The only slight annoyance left is that I can't as yet find a way to have Ant resolve classpaths for javac (etc) with respect to modules. What I'd really like to do is to derive a classpath by referring to a bunch of module names, but right now the build still has to do this using combinations of <fileset> and <dirset>. This leads to some duplication.

Even so, this is a big step forward, so the classpath issue hasn't dampened my sense of satisfaction too much.

Of course, Ivy is only part of the picture. Our old dependency repository, hosted at openquotecommunity.org, has done a fine job, but it would have needed some serious restructuring to make it work well with Ivy. So, enter, Sonatype's Nexus! We now have an instance of the same running on openquotecommunity.org.

I have to say, I'd thoroughly recommend Nexus. It was an absolute breeze to setup and configure. The only wrinkle being that I originally wanted deploy it as a WAR in our existing JBoss development server - which hosts JIRA and Bamboo. The WAR deployed okay, but it exhibited some odd authentication behaviour. Logging into Nexus as the default "admin" user apparently worked, but none of the admin menus appeared. Weird.

I put this down to the fact that we're running a pretty old version of JBoss on that server. As we're going to be moving to Atlassian's on demand service, I didn't want to spend time picking about in that particular can of worms, so I installed Nexus as a standalone (Jetty based) server instead. And that works like a dream.

You gotta love open source!

2.0 platform taking shape

I was expecting that the job of moving OpenQuote from the old JBoss Portal/Alfresco based platform to Liferay would be a pretty tough one, but so far... well... it just hasn't been.

The first set of commits are in the 2.0 SVN branch now and are ready for the intrepid to play with. We're not building a binary bundle yet, so you'll have to build from source. What you'll get as a reward for your efforts is a running instance of Liferay based on JBoss 7.1.1 with the OpenQuote backend components all in place, all the product content in place, and a somewhat familiar home page. The unit tests all pass too. It  really couldn't have gone much more smoothy... so far.

The elements that are left are the portlets, themes, integration and UI tests. Of those, portlets are giving me a headache right now.

It seems that Liferay is really expecting portlets to appear in the domain directory rather than inside an EAR as has been the style of OpenQuote deployments since the project began. There are various workarounds suggested on the web (interesting stuff here and here), but so far these workarounds aren't working in the way I'd like. They're also a bit messy. So messy that they're having the effect of making me rethink the bigger picture: Why are we using an EAR anyway?

Back in the days when OpenQuote was still called MultiQuote (and that is a long time ago), EARs were really the way to go, but JEE has moved on since then and so has JBoss. The primary value that we gained from EARs was all about class loaders. Our JEE components all get so share a common set of dependent jars which sit in the EAR's lib folder. But with JBoss Modules being a fundamental part of JBoss AS now, we have what can be seen as a much more flexible and expressive way of dealing with dependencies.

Ideally, of course, we'd wait for Project Jigsaw to deliver a module system right inside the JVM, but JBoss Modules is here now and could ease our path considerably.

I still have one or two straws to clutch at with regards to getting portlets working from the EAR, but if that fails we may be waving goodbye to our EARs in 2.0.

Change is good!

Thank you Atlassian!

A big thank you goes to Atlassian this week for allowing the OpenQuote project to use their OnDemand service absolutely for free!

Atlassian's JIRA and Bamboo have been a mainstay of OpenQuote project development since we outgrew the support that SourceForge offered many years ago. They are both essential and excellent products.

OnDemand is a cloud based service which will mean that we get all of the benefits of the Atlassian products without the server admin work.

As well as Bamboo and JIRA, we also have the use of Confluence, Crucible and Fisheye.

All I need now is some time to migrate and set them up.