We use map-reduce at
Joost to process usage data (roughly equivalent to the log files on a web server) and extract lots of useful information about the usage of the platform. This is built on
Apache Hadoop, an open source implementation of map-reduce.
Considering the still limited numbers of users we have, map-reduce isn't absolutely necessary and a SQL database could have done the trick, but with the expected huge user base when Joost becomes generally available, having a solution that is able to scale mostly by throwing in more machines is a must have.