Cocoon

My experiences with Cocoon began in February 2000 during web technology investigation in a project at the Dresdner Bank.

Our task was to build a web based portfolio management system that would replace an existing client-server solution, used to generate pdf and html reports of customer's investment and account data.

The original system, built as a Windows NT client, communicated with an Informix database server. Our job was to develop a web based system that would replace that client, but interface with the existing server technology which at the time retained all of the bank's propriety business logic.

At the time, Dresdner Bank had decided to merge with the Deutsche Bank, and although the merger in the end fell through, it had large implications for us as it forced us to think about the interoperability of our system with those outside of our own organisation, and in particular the portability of our data.

XML was the obvious choice for our data format, and so we started to build an XML based prototype of our system. This was what led us to the Apache XML project, in particular Xerces, the Apache XML parser and Xalan, the Apache XSL processor.

Our initial prototype consisted of a simple servlet which ran under JServ. It performed an SQL query on a database via JDBC, converted the resultant data into XML, and processed it with XSL. The final result was then delivered to the client either as html or pdf depending on a request parameter.

The prototype was quite simple, but effective, and it demonstrated our concept well enough for us to be authorised by management to continue this style of development for our production system.

This is when Cocoon came into the picture. Our prototype was effective at demonstrating our concept, but it was no where near what we required for a production system. This drove us in search of a framework that was built around the same ideas. Cocoon, the very next hyper link in the menu following Xerces and Xalan at xml.apache.org, was exactly what we were looking for.

At the time there were also several other Open Source web development frameworks that we evaluated, namely Struts, and Turbine. While both of these frameworks have matured quite substantially since February 2000, Cocoon, at the time proved to be superior for our needs due to its leverage upon XML (and later Avalon), and it's multi-stage pipeline concept.

And so - Cocoon became our application's development framework.

Initially we started using Cocoon 1.7.4, and then later CVS versions of Cocoon 1.8 to implement the pdf reports and html based data views that our application consisted of.

After about 6 months of development however, we discovered the need to make several enhancements and fixes to fop, the component of Cocoon that generated pdf documents. This presented a problem for us, because at the time the most recent version of fop was incompatible with the 1.x series of Cocoon.

This, and the focus and excitement of the Cocoon developers fuelled our switch from Cocoon 1.x to an early CVS version of the yet to be released Cocoon 2.

During the switch, is when all the fun began! :)

Pre-release Cocoon was much smaller and simpler than what it is today. It consisted of a pipeline and sitemap engine, and not much more :) There was no caching, no support for actions or subsitemaps, certainly no flowscript and there were only a handful of selectors and matchers available.

We also realised that the way you developed an application with Cocoon 2 was different than how you would use Cocoon 1. The concepts were similar, however the implementation details were substantially different.

After becoming familiar with Cocoon 2 though, our application took on a whole new face, and became much easier for us to maintain and extend.

We were able to structure our URI and filesystem space independently, and easily add support for internationalisation and multiple frontends, amongst many other advantages.

Over time our application eventually became a 'pure' Cocoon application, in the sense that we removed all of our 'other' legacy code (jsp's and xalan extensions, for example), in favour of the Cocoon way of doing things.

We developed selectors, matchers and later actions to handle control flow through our sitemaps, and were able to submit a lot of these additions and fixes back to the Cocoon community.

During the development of our application with Cocoon we also discovered Avalon. The Avalon framework is the underlying technology Cocoon (versions 2 onwards) is built upon, and provides a great foundation for IoC based software development.

Avalon is important technology, and I tend to feel that an understanding of Avalon is almost a prerequisite for understanding the power and flexibility of the Cocoon internals.

After learning how Avalon was used to abstract and componentize the internals of Cocoon we also undertook the project of porting our business logic and application code to Avalon components.

Currently the system uses a CVS version of Cocoon 2.1 from February 2003, it serves approximately 3000 users per day, over 4 load balanced application servers.

The main development now focuses on refining the internals to use newer Cocoon features like flowscript, modern form handling, configurable caching, etc.

There's also a continuing effort to re-factor and improve the business logic code which now is mainly implemented as Avalon components.

The adoption of Cocoon as our primary implementation technology has also given us exposure to other useful open source and free software tools such as Avalon mentioned above, Forrest, Axis, Rhino, and others.

More and more of these tools are being used and integrated into our application and Cocoon in general, which is great.

Like all projects based on new technology, we went through many highs and lows while developing and using Cocoon. Interestingly now, after talking with others who were using Cocoon at the same time, I've found many of the experiences to be shared, even though both sides at the time weren't aware of it :)

Often, the issues we experienced were due to the use of unreleased code, and not the officially released versions of Cocoon. When one chooses to be on the cutting edge, they must be prepared to accept the sharpness of the blade.

As Cocoon developers ourselves, quite often we wanted to use and test features we'd developed as soon as possible, so several times, with a degree of risk we choose functionality over stability.

Our largest source of problems was often load related. Often during testing we would observe that our application wouldn't scale, or would exhibit defects with increased amounts of users.

To combat this we tested our application more thoroughly before releasing it into production with tools like LoadRunner and WinRunner. Automated scripts would simulate production load (sometimes 3 to 4 times the normal production load) to anticipate and provoke potential problems.

This up front method of heavy load testing often proved effective and we were able to iron out many issues before our users discovered them.

Documentation was also an issue for the non-developers in our project. In the early days the source code was the documentation :) Today this is no longer a problem though with several excellent books available on the market, and a separate Cocoon documentation effort.

Quite often though, we've found mailing lists and peer group communication to be the best source of documentation and help, as they're the primary medium for discussing current/new features and issues.

Looking back, I personally found the good to outweigh the bad by far.

Dresdner Bank was ecstatic that they they had the source code for everything their system comprised, down to the level of the operating system, they were also amazed that the entire system was available for no charge.

The developers were happy because they were able to work on a project that was technologically extremely interesting and fun, and were also able to work with other like-minded people distributed across the Internet and the world.

Everyone also valued the fact that the knowledge they learnt using and developing Cocoon was completely transferable to other projects.

It was a win-win situation for everyone.

One important and thoroughly enjoyable aspect of Cocoon has been and continues to be the community of people that surround it. I was privileged enough to attend ApacheCON 2000 in London, which is where I met and became good friends with people like Carsten Ziegeler, Torsten Curdt, and many others. Over the past years we've also had several events local and abroad which has brought the community closer together.

The Cocoon Community has also had several real life Cocoon events in the past. Initially as part of larger conferences like ApacheCON, but more recently as separate events like the Cocoon GetTogether and the Stammtisch's held in Frankfurt.

The Cocoon GetTogether in Ghent was quite a remarkable experience. Ghent demonstrated to me the maturity of Cocoon, showcasing the first Cocoon conference with speakers and companies presenting seminars and products based on or built upon Cocoon.

The GetTogether also allowed us to finally meet almost all of the Cocoon committers in person, which was great. It showed me personally how truly global Cocoon had become, and how just a few emails on a mailing list can earn you a little fame :)