High Performance Community

Last week I was on the road and out of my home office. Even with my little Eee netbook and plenty of spotty wireless access I still managed to get behind in just about everything. My trip was non-technical in nature so a few days away from the high-tech hub-bub was useful. I was not on vacation either. My trip was to attend meetings as part of my work with a local non-profit organization. It turns out that having geek mojo helps outside of HPC as well. I managed to set up a Wiki and an on-line survey that has been received quite well. (As a side note, I used LimeSurvey. It made the job so easy and made me look like and an uber geek. Nice work Lime team.)

Getting back to HPC. I had some time to think about the annual HPC mega-show and conference; SC10 in New Orleans. I will be attending again this year and be more stationary than usual because I will be in the SICORP Booth (#1731) promoting a new community initiative. Also, the Beowulf Bash will return again this year. More information on both activities will be forthcoming very soon. If you keep and eye on my twitter posts and Cluster Monkey you can get the latest on my SC10 activities.

As we have done in previous years, the Bash will be Monday night (November 15th, 2010) starting at 9PM (after the Opening Gayla). We are planning for more space this year as this event seems to keep growing. For those that may not know, the first Beowulf Bash was an attempt by Don Becker and myself to invite the small fledgling cluster community at SC to gather in one room. We basically drank beer and talked of world domination. Seems to have worked now that I think about it.

As weird as it sounds, I do consider the Beowulf Bash and other socializing events to be an integral part of cluster HPC. Community is a particularly important part of the open source movement and is defined by common goals and common ownership of ideas. Open source is an embodiment of many communities each sharing some aspect of the whole. Conversations are the life blood of any community and the Internet has made global HPC communities a reality. In as much as I like email and the whole web thing, I find that there is no substitute for real conversation. Sharing face time with real people is just as important as with your keyboard and monitor. To put it in more geeky terms, real live conversations have a staggering amount of information that is lost in an email, blog, or other electronic medium. Some may argue all the extra information is not useful and I would agree that sometimes it is. In other cases, it is just that information that conveys passion, disdain and everything in between that I find useful. Then there is the “scuttlebutt” factor — the stuff you only here outside the meeting rooms.

I also want to go on the record and state that I believe conversation about commercial codes (often closed source software) and hardware is as important as open source. It is just that in the open source world there is so much more to talk about!

Hopefully, I have convinced you to come out and play if you are attending SC10. For those that cannot make it, don’t ignore your local resources like user groups and the occasional cup of coffee. A few years back, I was running a NYC/New Jersey HPC User Group (HUG). We would meet over dinner and discussed HPC issues and problems. I met quite a few people with which I still keep in contact and I learned a few technical things that I never would have gleaned from an email conversation. I’ll wager that if you live and work in an area where there are other HPC users like you it would not take much to get together and complain about a few things. It helps, trust me.

Speaking of community, I just read this story on over at HPCWire. It seems a community of companies Cray, Data Direct Networks, Lawrence Livermore National Laboratory (LLNL) and Oak Ridge National Laboratory (ORNL) have come together to create Open Scalable File Systems, Inc. (OpenSFS). The organization will focus will on Lustre and other open source file system technologies for HPC. They expect other companies and organizations will be joining the effort.

One might assume that this in response the the recent acquisition of Lustre by Oracle (via the Sun purchase), but OpenSFS CEO Norman Morse states “We absolutely refuse to fork the system. We intend for Oracle to be the canonical definition of Lustre.” Well maybe not a fork, but let’s just call it what it it is; insurance.

Here is the interesting thing about this type of arrangement. If Cray went to Data Direct Networks, LLNL, and ORNL and said “Let’s do a parallel file system we all can use.” The response would be, “Sure, once we get legal, R&D, and marketing involved to figure out how to share the Intellectual Property (IP) we bring to the deal and make sure we get a return on our investment.” Such an arrangement may take 1-2 years to get off the ground.

Now consider the Open Source mantra “give a little, get a lot.” All the organizations benefit from the combined efforts and the cost is cheaper than if they were to go at it alone or try to create complex IP agreements among the interested parties. A fair sharing model based on copyright, like the GNU license, short circuits many of the traditional impediments to cooperation. In addition, you get this thing called “a community” around your project. Within this community are your beta-testers, developers, reviewers, first customers, and most importantly conversations about your project/product.