Richard Wilkinson’s Blog

More on terracotta and wicket

So recently I have been working on session clustering on Zoomf, which is written in apache wicket.

We decided to go down the terracotta route, because at the time we were running on Jetty and I couldn’t get the WADI clustering to work right, plus terracotta claimed to be easy to get up and running, which in fairness it was. All was going to plan and we deployed the code to our production site. Unfortunately we started running into problems. Basically loads of wicket objects were being created and distributed, because of the way wicket stores pages in its session. With the amount of traffic our site was getting the terracotta distributed garbage collector (dgc) couldn’t keep up, and so the persistent disk store was using up more and more disk space. We’re talking tens of gigabytes here, which is clearly a problem.

After playing around with the different terracotta config options it became apparent that a wicket solution to limit what was distributed was needed.

The simplest solution I could think of was instead of storing the wicket pages as page objects I should serialize them and store them as byte arrays instead. This proved to be as easy as implementing a new wicket IPageMapEntry which does the serialization and deserialization and overriding the getPageMapEntry() method in my base page to make all my pages use the new class.

The new class itself is really simple:

public class NewPageMapEntry extends AbstractPageMapEntry
{
        private transient Page page;
        private byte[] data;
 
        public NewPageMapEntry(final Page page)
        {
                this.page = page;
 
                data = Objects.objectToByteArray(page);
 
                setNumericId(page.getNumericId());
        }
 
        @Override
        public Page getPage()
        {
                if(this.page == null)
                {
                        page = (Page) Objects.byteArrayToObject(data);
                }
                return page;
        }
}

With any luck the new code should be running Zoomf next week and on Tomcat too (tomcats terracotta support is more mature than jettys). If are are interested seeing the progress of this you can read this thread on the terracotta forum, which was extremely useful. A big thanks to all the guys who helped me on there!

EDIT:

I am now working on a better way for to integrate wicket and terracotta, see this thread for more information.

Zoomf does microformats

Last week we released a slightly updated Zoomf.  One of the less visible updates was the support of the hListing microformat.  We also GeoTagged all of the search results and property details pages with both GeoURL and Geo Tag formats.

hListing is a relatively new microformat, but has recently been thrust into the limelight by its adoption on the price comparison site Kelkoo, which is owned by Yahoo.  You can read the original blog post announcing this here.  Fellow property site Nestoria quickly followed suit with hListing support on their results pages.  I should add that I didn’t copy Nestoria in adding hListing support, its just that we were working on other important updates to Zoomf and didn’t have time to do an update of the main website straight away :).

GeoTagging is also quite new and basically consists of embedding geographic location meta data into your web page so that the content of a page can be linked to a geographic location.  For Zoomf this means that on any results page (eg for clowne) the page is geotagged to the search location, and for any property detail page the page is geotagged to the location of that particular property.

You may ask whats the point of all this meta data and embedded microformats, especially when afaik Google doesn’t take account of this stuff when they index web pages, and it is invisible to a user using their browser.  The answer is that someday the big search engines may take notice of this content and use it to enhance their search results, Yahoo recently announced that their search results will start to extract more semantic meaning from pages using microformats, so the so called semantic web, of which microformats are a key part may be closer than you would think.

Until then smaller more specialised applications will be able to use this extra meta data to do cool things with the content on your pages, one example is the Optimus microformat translator which can transform pages with microformat data to JSON, XML or RSS formats, try it on a Zoomf results page or a linkedin public profile to see what I mean.

Updates to zoomf

You may have noticed a few changes at zoomf lately. Over the last month or so the entire site has been given a face lift, that much is obvious. What is not so obvious are some of the less visible new features, such as GeoRss, Kml, better tag filtering, better sorting, better content and Opensearch plugins.

Since the geeky stuff never gets talked about on the main zoomf blog, i feel it is my duty to highlight it here.

• GeoRss - GeoRss is best described on the wikipedia page here. Basically we now put location data into our rss feeds, which can be used by all sorts of different applications, the most useful of which is google maps. If you paste any GeoRss enabled feed into the google maps search box it will instantly display the contents of that feed in the map. Check out my home town here. For anyone thats interested we also tag our RSS feed with photos using the Media RSS extensions.
• KML - Again its probably best to look on wikipedia. KML is the markup used by google earth, and many other applications such as google maps. You can either download the kml and open it with google earth, or paste the kml url into the google maps search bar (as with the rss url) to view the results either google earth or google maps respectively.
• Better tag filtering - The tag filtering (eg garden, victorian) now works as an OR query rather than an AND query as before. We felt this gave a better user experience as people can now search for properties on either the 1st or 2nd floor, which was previously impossible.
• Better sorting - You can now sort by price, number of bedrooms and freshness, both ascending and descending.
• Better content - Using the Geonames.org wikipedia web service we display a random wikipedia article based within 10km of your current search area on every zoomf results page. Check out my hometown results page and you will see wikipedia entries for such interesting places as Creswell Crags and Bolsover Castle. The webservice itself is straightforwards to implement and our thanks go to the people at Geonames.org for providing it free of charge.
• Opensearch plugins - Opensearch is yet again best described by wikipedia, however all you really need to know is that it lets users of firefox and internet explorer 7 (and do a zoomf search from their browser. For firefox users, all you do is go to any zoomf page and click on the search engine selector icon in the top right corner (it probably has a big google ‘G’ in it), then click on ‘add Zoomf.com sales’ or ‘add Zoomf.com’ lets in the drop down bar. We even support search suggestions like google does. More details of how to do this will be posted to the main zoomf site as soon as we get chance.

Thats it for now, expect another geeky zoomf tech update when we bring back improved visual search in the next few weeks.

Richard