Sam Aparicio

Today, in a discussion with some engineers,  we wondered if it is possible to replicate some of the attributes that make some open source projects so incredibly powerful in a more commercial setting. We’re not talking about making money off of open source; instead, we’re interested in cataloguing what can be learnt from the open source engineering approach and seeing how it could be leveraged for businesses that have a commercial R&D function.

Successful open source projects:

• Produce very high quality software
• Have generally very motivated developers
• Get some of the best talent in the world
• Are highly distributed, enabling individuals who participate the ultimate in lifestyle flexibility. You live where you want, you work however much you want, whenever you want.

Some of their observations were:

• All successful open source projects have a benevolent dictator in charge. Companies are afraid to confer so much power to one person and instead tend to decide by committe
• The lack of commercial pressure allows engineers to progress towards goals at their own pace, many open source projects produce “progress” very slowly.
• The natural selection among open source projects is brutal. For every successful project with the right idea, the right people, the right market, there are hundreds which fail, either because of the wrong dictator, the wrong idea or the wrong timing

Some of my counters:

• Companies, especially startups, should excel at empowering one individual to steer a project. A company can eliminate all barriers for one individual to lead without having to worry about perception, mundane tasks, or financial worries.
• Product companies, especially those that are targetting thousands of users, have to progress very slowly anyways, giving a lot of consideration to quality, operations, code health, etc.
• The natural slection of commercial endeavours is just as brutal. But if you just could get top notch software out of an engineering team in a reliably manner, then you’ve eliminated one more source of risk.

Wondering what other people’s experiences are in building highly distributed software engineering teams made of highly talented, highly motivated engineers. My gut feeling is that this kind of approach hinges on the quality of the process. The distributed approach forces a degree of formality that correlates with good engineering practices. This raises two interesting questions: a) is it really the process that makes the success, or is there something inherently different when you take money out of the equation that makes it possible for these open source projects to succeed (to me, open source looks like an aberration, it shouldn’t work yet it works beautifully sometimes) b) if it is the process, what comes first, the distribution forcing the practices or developers who wouldn’t work any other way self-organizing around those practices?

In any case, I don’t think that the central motivation of pursuing open source engineering approaches in commercial settings should be the lowering of R&D costs. It makes more sense if the goal is the raising of engineering standards and what that can enable for a certain kind of business.

Zed Shaw has a nice essay on the state of programming, drawing analogies from the attainment of mastery from martial arts. Here’s an excerpt:

Programming is a very new discipline, so there’s not too many master programmers out there. What’s worse is that the few people I would consider masters aren’t very exemplary of the software profession and art. They are typically professors who never write anything under a deadline and are given complete artistic freedom to develop whatever they want. Take Donald Knuth, who was able to take three years off from teaching in order to complete TeX. There’s no way I could get away with telling my employer that it’ll take me three years to finish their product. Knuth is basically a “master amateur”. A guy who worked in a complete utopia and was able to hone his skills without interference. I would compare him with a man who became a master by studying at a monastery for for his entire life.

[From The Master, The Expert, The Programmer]

Very nice thoughts overall. I think, though, that the analogy is flawed. At least as it pertains to commercial development. Comparing programming to an art form, and the process of maturing as a developer to the process of becoming a master, comes from the mindset of somebody who is trying to create something intrinsically beautiful.

And while beauty is a powerful concept, most software projects are trying to make something functional. Just like a building needs an inner structure, a piece of software needs code. What one demands of a building structure, above all, is that it be sound. Not beautiful. Structural engineers are not pursuing an art. Nor a science. They’re pursuing to apply a body of systematically gathered engineering knowledge to a problem, and maybe, in the process, advance the state of engineering.

A lot of programmers, especially in the US, are oblivious to the fact that their profession is 10 years away from disappearing as we know it today. What is causing development to move to lower cost of employment locations is precisely the underlying fact that programming is an engineering discipline, not an art form.

There are some businesses with obscenely rewarding markets which can afford the artists, but there are fewer of those every day. For the rest of programmers, the choices are: to milk the good pay while it lasts, to become an artist and find a patron, to retrain in another profession, or to disintermediate, become entrepreneurs and hope that they come across one of those obscenely rewarding markets.

I met Yakov Fain a couple of years ago during a blustery NJ weekend at a Flex bootcamp course he was giving. He is damn smart, and is publishing his second Flex book on OReilly soon. He’s also funny, in a non-self-aware way. Hope you meet him at some conference soon.

One of his russian-english signature sentences: “The mankind went nuts”. Which he used when speaking about the web development community when AJAX was published.

When talking about Flex vs AJAX, he made a really good analysis, and somehow, the bad parts of AJAX stuck with me:

• Code is in the clear
• A refresh forces a reload
• Need to write code to check that all the Javascript code has arrived before app can correctly execute
• AJAX is a different kind of lock-in. (Flex is big-guy lockin and Ajax small guy lockin.)
• Cannot jump out of the browser
• Need to check cross-browser compatibility
• No data bindings and no closures.

So there you go.