Storage Bits
Robin HarrisA $525, 2048-core cluster in 45 minutes
By Robin Harris | December 16, 2010, 8:32am PST
Summary
10 years ago that would have been a top 10 supercomputer. Now all you need is a credit card and the Internet. Still think cloud computing is a fad?
Topics
Blogger Info
Robin Harris
Biography
Robin Harris
Harris has been messing with computers for over 30 years and selling and marketing data storage for over 20 in companies large and small. He introduced a couple of multi-billion dollar storage products (DLT, the first Fibre Channel array) to market, as well as a many smaller ones. Earlier he spent 10 years marketing servers and networks. After leaving corporate life he founded TechnoQWAN, a consulting and analyst firm. He also developed StorageMojo into one of the top storage industry blogs.
Robin writes, consults, coaches and lives among the mountains of northern Arizona.
10 years ago that would have been a top 5 supercomputer. Now all you need is a credit card and the Internet. Still think cloud computing is a fad?
A large biotech research group needed a lot of CPU cycles to run some simulations. Their own 2000-core cluster was fully booked. Call of Duty: Black Ops was just out!
They turned to CycleComputing’s CycleCloud automated compute cluster creation system. You can build a cluster without them, but CycleCloud also handles details like:
- Scheduling
- Maintaining OS images
- Encryption and associated key management
- Deploying file systems
- Load-based scaling
Amazon Web Services (AWS) provided the horsepower.
Benchmarking AWS
With AWS rolling out new services monthly it seems, figuring out what to use is a problem. CycleComputing ran some benchmarks to understand the cost-efficiency of AWS cluster options.This job was compute, not I/O, intensive. AWS offers new Cluster Compute instances configured with storage and I/O to handle either 8 or 16 concurrent jobs, as well as the older 8-way High-CPU instances.
They found that while the 8 & 16 CC instances were faster than the 8 H-CPU instance, the latter was more cost-effective because it cost so much less. The 8 H-CPU got 25% more work done per dollar.
I know, it seems almost un-American for the latest and greatest to not deliver the best bang for the buck. Know your application!
Set up, take down
They started setting up the cluster at 1030am and by a 1115 they were maxing out over 2000 cores. The job was done by 2pm and it took another 10 minutes to bring the cluster down. Total AWS price: $525.The Storage Bits take
With 1.7 TB of RAM and 2048 cores, this rent-a-cluster’s computing power was mind-boggling a decade ago. There is a new job here: cloud computing architect.But there’s a larger lesson for the industry here: this was all done with commodity hardware. All the fancy branded storage, servers and networking that the BigCo’s are peddling just isn’t part of this equation.
Which is why you can do this for $525.
Cloud computing and storage are driving a new round of competition and consolidation in the IT industry. It will be rough on the companies, but good for us users.
Comments welcome, of course. CycleComputing wrote this up in their blog.
Kick off your day with ZDNet's daily e-mail newsletter. It's the freshest tech news and opinion, served hot. Get it.
Robin Harris has been messing with computers for over 30 years and selling and marketing data storage for over 20 in companies large and small.
Disclosure
Robin Harris
Robin Harris is a president of TechnoQWAN, a consulting and analyst firm in northern Arizona. He also writes StorageMojo.com, a blog which accepts advertising from companies in the storage industry, and has a 25 year history with IT vendors. He has many industry contacts, many of whom are friends and all of whom he has opinions about. Robin has relationships with many companies in the technology industry. Every company he writes about may have sought to influence his opinion through carefully-crafted marketing messages and self-serving white papers, gifts ranging from desk calendars, t-shirts, lunches and trips as well as analyst or consulting assignments. He also invests in some technology companies. He may accept payment for services in stock as well. Robin discloses financial investments in or client relationships with companies named in Storage Bits. To help readers sort out the gold from the dross in his writings, Robin tries to communicate his reasons as clearly as he can. If you agree, you are intelligent and discerning. If you disagree, well, you disagree. In all cases, Robin encourages readers to subject everything they read, see or hear on the internet or from politicians to some simple questions: * What assumptions are implicit in the world view and judgments of the author? * What, if any, is the factual basis for the opinions the author expresses? * Is it reasonable, logical and clear? Your critical faculties: use ‘em or lose ‘em!
Biography
Robin Harris
Harris has been messing with computers for over 30 years and selling and marketing data storage for over 20 in companies large and small. He introduced a couple of multi-billion dollar storage products (DLT, the first Fibre Channel array) to market, as well as a many smaller ones. Earlier he spent 10 years marketing servers and networks. After leaving corporate life he founded TechnoQWAN, a consulting and analyst firm. He also developed StorageMojo into one of the top storage industry blogs.
Robin writes, consults, coaches and lives among the mountains of northern Arizona.
More from “Storage Bits”
Talkback Most Recent of 14 Talkback(s)
- Follow via:
- RSS
- Email Alert
A market emerges...
I'm picturing a future where excess computing power is sold like excess electricity. Today many power companies choose, or are lawfully bound, to buy excess power generated by privately owned sources. This electricity may be coming from solar panels, wind turbines, small hydroelectric plants...you name it. The electricity can be routed back to the power company and resold. Meters monitor and keep track of these transactions. We're not too far away from being able to do the same thing with excess CPU cycles. Individuals aren't likely to ever get rich by selling off their excess computing power, but they may be able to subsidize their own internet usage. There are already frameworks being used for this kind of computational sharing (SETI@Home, Einstein@Home, Distributed.net) but I believe all of these are volunteer based. I've not yet heard of any that offer up compensation for distributed computing projects. I can't imagine they won't come around at some point though.
RE: A $525, 2048-core cluster in 45 minutes
@jasonp@... Err, you think? It seems the price of computing cycles will continue to fall, the laws of supply and demand make it really hard to sell something that is so abundant.RE: A $525, 2048-core cluster in 45 minutes
This is pretty awesome. I used to intern at a video production house where we could pull the extra CPU cycles from all the networked computers around the office to help do video rendering for 3DSMAX, LightWave and Maya projects. There were times though when there simply wasn't enough CPU "in the building" for everything going on and timelines could get really tight. This would have been a very handy service to leverage for just those occasions.
This isn't a cloud
It's grid computing. Intel invested heavily in this the last time this fad came around, and they lost big time.
thank you
for pointing that out because some of these people are so overhyped on the whole "cloud computing" that they want you to forget things like grid computing so that when they roll it out of the closet again they can say "see, even more cloud computing things more proof where all going to the cloud!!".next he'll point to some company putting out a new email client telling everyone "look mail though the clouds - how much more proof do you need that were all moving to the cloud!?!?!"
ZDNet Blogger
RE: A $525, 2048-core cluster in 45 minutes
@happyharry_z Sorry, grid is dead. And has been for several years. See my post at http://storagemojo.com/2007/11/26/grid-is-dead/Robin
Grid is dead...
long live grid.jasonp@...12/16/2010 01:20 PMeither it is time to retire
after this article or too much partiyng. there is nothing new here. It looks like an advertisement to me.
I thought the "Grid" had an 8088 ....
and a plasma display? Titanium case though!
RE: A $525, 2048-core cluster in 45 minutes
I have just finished a 3 day course on #Azure and a recurring them in my head the whole time was that of the Cloud Computing Architect,The possibilities at a ridiculously low cost are mind boggling but you do need to understand the options available and look at how best to solve the problem.
This, at least for some time, is ripe for the new role of Cloud Computing Architect for which you would expect to get paid rather handsomely. The cost savings to organizations could be enormous.
BondiGeek
Http://www.bondigeek.com
If you have a right algorithm/problem...
If you have a problem that can be distributed to 2048 CPUs nicely then you can do it on office PCs. No need to do it in the cloud.
RE: A $525, 2048-core cluster in 45 minutes
@man_28 But why do it yourself at the massive expense of all that infrastructure (which no IT department would probably get the go ahead for from the CFO anyway) when the infrastructure is already provided for you and you can just rent it for half a day as needs be.You company accountant will love you trust me
RE: A $525, 2048-core cluster in 45 minutes
I also believe that this idea of everything in the cloud is passing .... I know many people talk about it when I see the "wonder", people still do not trust these systems, at least that's what several people feel they have fear and safety etc. ....
acompanhantes
ZDNet Blogger
RE: A $525, 2048-core cluster in 45 minutes
@fernandoacflog Private clouds will be popular as well among the world's largest organizations and enterprises. As noted in Moore's Wall, our ability to scale up is limited, so scale out is the only viable option.Talkback - Tell Us What You Think
Friday, 17 December 2010
A $525, 2048-core cluster in 45 minutes | ZDNet
via zdnet.com
