Nice article about multicore processing in the  July 2010 of IEEE Spectrum by David Patterson . Interesting analogy about parallel computing and news reporting.

But why is parallel processing so challenging? An analogy helps here. Programming is in many ways like writing a news story. Potentially, 10 reporters could complete a story 10 times as fast as a single reporter could ever manage it. But they’d need to divide a task into 10 equally sized pieces; otherwise they couldn’t achieve a full tenfold speedup.

Complications would arise, however, if one part of the story couldn’t be written until the rest of it was done. The 10 reporters would also need to ensure that each bit of text was consistent with what came before and that the next section flowed logically from it, without repeating any material. Also, they would have to time their efforts so that they finished simultaneously. After all, you can’t publish a story while you’re still waiting for a piece in the middle to be completed. These same issues—load balancing, sequential dependencies, and synchronization—challenge parallel programmers.

Further along:

All in all, things look pretty bleak.

But he continues:

Nevertheless, there has been progress in some communities. In general, parallelism can work when you can afford to assemble a crack team of Ph.D.-level programmers to tackle a problem with many different tasks that depend very little on one another. One example is the database systems that banks use for managing ATM transactions and airlines use for tracking reservations. Another example is Internet searching. It’s much easier to parallelize programs that deal with lots of users doing pretty much the same thing rather than a single user doing something very complicated. That’s because you can readily take advantage of the inherent task-level parallelism of the problem at hand.

Other success stories referred to are Computer Graphics, Weather Prediction (one of the workhorse applications for supercomputing – one of PARAM’s main use has been in the National Centre for Medium Range Weather Forecasting (NCMRWF)) and simulations.

The La-Z-Boy era of program performance is now officially over, so programmers who care about performance must get up off their recliners and start making their programs parallel.

Despite these reasons for hope, the odds are still against the microprocessor industry squarely completing its risky Hail Mary pass and finding some all-encompassing way to convert every piece of software to run on many parallel processors. I and other ­researchers at the main centers of parallel-­computing research—including Georgia Tech, the University of Illinois, Rice University, Stanford, and the University of California, Berkeley—certainly don’t expect that to happen. So rather than working on general programming languages or computer designs, we are instead trying to create a few important applications that can take advantage of many-core microprocessors. Although none of these groups is likely to develop the ultimate killer app, that’s not the intention. Rather, we hope that the hardware and software we invent will contain some of the key innovations needed to make parallel programming straightforward. If we’re successful, this work should help to usher in whatever application ultimately wins the “killer” distinction.

He finishes off with:

No matter how the ball bounces, it’s going to be fun to watch, at least for the fans. The next decade is going to be interesting.

via IEEE Spectrum: The Trouble With Multicore ^[1].