Supercharge your cluster with Tesla S1070 systems from NVIDIA or Tesla M1060 processors integrated into servers from leading OEMs. Experience the performance of a large cluster with just a small cluster of Tesla solutions. Tesla-based clusters deliver up to 30 times the performance of CPU-only clusters with lower power and less space. Featuring the revolutionary NVIDIA CUDA™ parallel computing architecture and powered by 240 parallel processing cores in each Tesla processor, the Tesla pre-configured solutions shatter your performance per watt expectations to help you solve the toughest computing problems faster.
You can build your own GPU cluster with Nvidia S1070 1Us connected to rack mounted servers. You will still need some way to distribute your workload like Condor or a 3rd party grid software provider. Or you can buy preconfigured GPU clusters from companies like Cray (yes... the inventors of the super computer).
For more information http://www.nvidia.com/object/preconfigured_clusters.html