SZE, Tsz Wo (Nicholas)

I am currently working on the Apache Hadoop, an open source distributed computing software system,
at Yahoo!.

I used Hadoop to compute π, more specifically, the n^th bit of π. A mapreduce program, called distbbp, was developed for such purpose. It was checked in to Hadoop 0.21 as an example program. The results are shown in the table on the right.

• Except for the first and the last rows, bits starting at positions n–4 and n+4 are also computed for the sake of correctness.
• Row 0 to Row 7
  • They were computed by a single machine.
  • A single run of Row 7 took several seconds.
• Row 8 to Row 14
  • They were computed by a 7600-cpu-core cluster.
  • A single run of Row 14 took 27 hours.
• The first part of Row 15 (6216B06)
  • The result was posted in this YDN blog.
  • The first 30% of the computation was done in idle cycles of some clusters spread over 20 days.
  • The remaining 70% was finished over a weekend on Hammer, a 30,000-cpu-core cluster.
  • The log files are available here.
• The second part of Row 15 (D3611)
  • The computations were completed on June 30, 2009.
  • The starting position is 1,000,000,000,000,053, totally 20 bits.
  • The last bit, the 1,000,000,000,000,072^nd bit, probably is the highest bits computed ever in the history.
  • Two computations, at positions n and n+4, were performed.
  • A single computation was divided into 14,000 jobs and 7,000,000 tasks. It took 208 years of cpu-time and 12 days of cluster time.
  • The log files are available here.