High Performance Computing - Return of Mainframes?

On this Monday, I attended a conference on High Performance Computing sponsored by Microsoft. The keynote was delivered by Microsoft HPC Division General Manager. It was good to see that HPC is slowing transitioning from Research use to commercial use. Of late, there has been lot of interest in Manufacturing companies to explore HPC capabilities for Engineering and Simulation. I see lot of possibilities in Global Datawarehousing and Fraud Detection in Financial services too.

But, we have miles to go before we those apps in mainstream. As we have witenessed in the industry for quite sometime, the progress in Hardware space always outperforms the progress in software product industry. Today, we are capable of talking about HPC, Grid and Large scale computing. However, we dont have intelligent software products or applications that can leverage the full power of those powerful systems. There are couple of reasons - 1. The fundamental way of programming software applications has so far been single threaded. We dont have an out-of-the-box programming model or a product infrastructure that would seamlessly enable today's developers to write applications that can execute in parallel within themselves. Of course, Intel and Microsoft are working hard in bringing advancements in those areas. 2. Only when the foundational infrastructure software becomes available, the toppings or so-called end-user applications will start to emerge (for example, engineering applications or animation applications) including Developer tools.

On the same conference, I asked the Microsoft GM about the possibilities of running MS SQL Server on Windows 2008 HPC server and the answer was 'Not Yet'. Of course, the SQL Server product has to be re-architected to run on the HPC platform. The same applies to rest of the commercial products like analytics services or integration services.

It sounded to me that we are returning to Mainframes era with Open standards! :-). Here too, Windows HPC has the concept of Job Schedulers which the old mainframes used to have. But this time, You can execute WCF compliant services too as part of Job. So, the HPC platform is 'Service-Oriented'.