Reply to comment

It's been nearly one month since I started full speed at Goojet, and I already learned a lot on the mobile web and J2ME.

One of the key issues in the mobile web is latency: connections are long to establish, and data transfer rates are low. Not everybody has a 3G phone nor an unlimited data plan.

Connection establishment latency can be mitigated in two ways:

• Use persistent HTTP/1.1 connections. By reusing the same TCP socket for several HTTP requests, you save the non-negligible connection setup overhead. To achieve this, every request and response must carry a Content-Length header.
  
• Open several connections in parallel when you have several items to load (e.g. images in a page). This might seem contradictory to the above, but a single connection doesn't use all the available bandwidth. There are lots of idle periods during connection establishement and data transfer, waiting for handshake packets on the slow link. So by opening several connections and handling them in parallel, more of the available bandwidth is actually used leading to shorter overall load time.

Having responsive connections is good to speed-up transfer, but even better is not to have to transfer data at all, both for response time and user phone bill. So HTTP cache headers must be handled with great care, which also means that the J2ME application has to implement an HTTP cache.

Cache headers for static resources like images or CSS files are most often handled by the web server itself, by means of Last-Modified and/or ETag headers. You have to be careful with ETags though in server farms, since some web servers include the file inode in it, which essentially makes its value different on every server for the same replicated resource, thus breaking the intented purpose.

Static resources can also benefit from using the Expires header, or better the HTTP/1.1 Cache-Control header that avoids the date parsing and clock synchronization issues associated to Expires. The use of versioned URLs even allows to set an infinite expiration date.

But cache headers for applications are way more tricky. There is often no concept of "last modified", and computing an etag that reflects the state of all the elements that contribute to the response is not always possible or would lead this concern to creep into all application layers.

So I've taken a different approach in the Goojet backend: when receiving the first byte of the response body, a servlet filter checks if the application has set cache headers. If not, the response is buffered so that we can compute a hash code once the application has finished producing it. We could have used MD5 for the hash, but it's a bit costly to compute and we don't need something cryptographically secure. So we use a 64 bit FNV-1 hash that is very fast to compute and has a low collision rate, even for small changes in the data.

The result is that even for highly dynamic responses, we are able to provide cache headers that allow the mobile application to issue conditional requests and download data only when actually needed.

All these techniques combined really make a difference to have a more responsive application and a lower phone bill!