A lot of times at the Snow Sprint I was struck by a feeling that I was in the middle of a group of people who were really doing open source right. So on the plane home I started thinking about what that meant and why I haven’t felt that way at TOPP.
Well, a lot of us have said at various times that our attempts at doing open source development have often been frustrating and rarely seem to be worth the effort. Coming out of the sprint, I think that’s because we are missing a very big piece of the “True Spirit of Open Source”. Doing open source properly is not just a matter of putting the GPL on our code—it is not about making a legal contract but about making a social one. It is about committing to reuse and collaboration both internally and externally and it is well worth the effort.
Writing, finding, and using software all take time and care and with someone paying for our time and care we ought to think of these acts as investments. We should make the most of those investments by seeking code reuse and encouraging collaboration. The more we encourage heavy use of software and heavy reuse of code, the better our ability to improve and solidify it. Whether or not we’ve read Dreaming in Code (though everyone here should—as soon as possible—seriously) we all know that software is hard to do well. It’s full of unexpected behavior, prone to breaking when you touch it, and rarely optimized. We can improve it only by testing and exercising it: give it to people to use, poke it in new ways, put it under stress, see what breaks, fix it and don’t let it break in the same way again.
Laziness is a virtue
Because software is expensive to write, it is most responsible of us to write as little as we can. And because software is hard to write, it’s rarely done once it’s written; most software has tremendous room, and often necessity, for improvement. So our goal should be to maximize the impact of every improvement we make. This primarily means seeking to increase the breadth of our code’s effect: convincing other people to use our software, sharing code across multiple products, and reusing products in as many ways as we can across everything we develop.
To give a specific, if vacuous, example, we have five or more different pieces of code deployed on OpenPlans that generate and send email to users: Listen sends mail, Twirlip sends mail, WordPress sends mail, and Opencore sends mail in at least two different ways. Suppose, instead, that we had a single shared way to send mail to users. Aside from the immediate benefit of less overall knowledge to store, the different mail-sending needs of all these applications would force us to develop a stable and reliable method for sending mail, and when heavy usage of one of those dependent applications inevitably reveals an inefficiency or bug in the mail-sending code, multiple otherwise unconnected parts of our site would instantly benefit from our resulting improvements. Put that same mail-sending code on two or more websites and encourage other developers to deploy it in their applications, and we can tremendously increase both the number and variety of stress points and the magnitude of impact of each strengthening, both the opportunity and the payoff for improvements.
So maximizing adoption of every piece of software by both users and developers is a forward-thinking and cost-effective stance. It gives programmers the most opportunities to strengthen their (buggy, brittle) code and ensures that every (hard, pricey) act of improvement has a large and wide-reaching effect.
We’re not alone
It is not only creating software that needs be treated as an investment. In a true open source environment using others’ software, too, is an investment that we should make the most of. When we choose to depend upon existing free technology—in any way: WordPress, Xinha, Zope, Pylons, Trac—we will benefit ourselves and others by doing all that we can to improve it, participate in its communities of users and developers, and encourage the growth of those communities. The core reasons are the same as with our own code: the more eyes and hands that are on a piece of software, the better it can become.
There is, of course, an additional benefit to encouraging outside developers, namely lessening our internal development and maintenance burden by gaining volunteer contributors. When we create our own software we thus have the additional responsibility of building these communities from nothing. But when we are dealing with existing open source software this goal is all the easier to achieve. A community of developers already exists; our only responsibility is to gain admittance. We should treat our lack of commit rights not as an obstacle, but as a challenge.
Alongside the fuzzy feelings that come with earning the respect of an established group, there is, in keeping with the best of the open source mentality, great potential for everybody to win when we are welcomed into an existing community. We stand to reap the benefits of the efforts of the intelligent, experienced and caring people who develop the software that we have decided is worthy of our use, and we gain the potential to influence the path and priorities of their development with our input and interest. They, meanwhile, gain the feedback and work of a new set of smart (and funded) contributors with a real interest in the success of their project.
Finding those unicorns
I’ve framed this primarily as an economic argument to be concrete, but of course there’s more to it than that. The same stance is also about committing to respectful and positive relationships with outside agents; putting our faith in open and collaborative development environments and in the potential for groups of smart, committed people to produce great things even with differing agendas; taking the time and care to make our work as good as it can be and letting the rest of the world be the judge of it. It’s about being responsible consumers who contribute back to the products we consume, and being responsible producers who proudly stand by what we produce. And then there’s our aspirations at TOPP to do good and produce real positive change: all the time and money we can save by reusing and building on existing solutions can be diverted toward new projects to do even more good in the world, with cascading effects.
So let’s build and get involved with communities, submit and fix bugs in others’ code, give thanks and contribute ideas to the people whose software we use.
Let’s release Flunc, Externalator and Deliverance to the world for real, with welcoming websites and helpful documentation, and encourage people to use them and talk to us about them on mailing lists and IRC channels. Along with everything else it’s a good way to put our software to the test: to put a different spin on the dogfood concept, if no one else wants to use our software, why should we? Let’s keep up with Xinha development and submit behavioral patches to its authors instead of keeping our work isolated on our “vendor branch.” Let’s tell people about the recommendations of our smart interaction designers and submit our implementations of that advice; how many open source software communities have an excess of input from web software designers? Let’s release WordPress and Trac plugins to their wider communities and let’s solicit contributions and feedback to Listen and Opencore.
In short, let’s try to practice better open source. After all, we can only bring the open source ethos and philosophy to the broader world after we get it right in our software.
A big +1 from me, though I admit that some of this actually surprises me - we have a vendor branch of Xinha? We’re not submitting patches to the libraries we’re using? We haven’t released wordpress and trac plugins? I believe the choice to build on so many open source projects was to join their communities and help them get better - if we’re not doing that then there’s little benefit of open source. With GeoServer the choice to build upon GeoTools was made before I even got there, and at first it felt like this huge overhead - most every change you wanted to do you’d have to discuss with a bunch of people, we wrote code that we wouldn’t even directly use, it felt like we were wasting our time with all this collaboration. Within a year or two the scales completely tipped, and we started getting more and more contributions to GeoTools. More people joined because they felt it was a strong community, and a lot of that was due to our contributions. When we started there were 6 developers on GeoTools (and two of them were us), now 33 developers have contributed in the past year, 67 all time.
At the same time there was another project that was very similar to GeoServer - open source written in java and implementing all the same standards, indeed they were even a bit ahead. They made a choice to build their own library, and to make collaboration an after thought. We definitely moved slower than them, pausing to collaborate and get our core in to GeoTools. But four years later we’re not even in the same league - GeoServer has pretty clearly surpassed in just about every dimension, and it’s directly because we committed to reuse and collaboration very early on.
It is really, really hard to build a successful open source community, but the best way I’ve found is to build upon an existing community that you help grow, becoming incredibly active members there. That imbues your project with a strong open source DNA of sorts, that eventually attracts people to what you’re working on. But one key is definitely to split things up in to discrete parts that others can use on their own. We’re already building on lots of other tools for openplans, indeed I have a suspicion if we jumped in to it we could be the difference in making Xinha the best open source text editor. Which in time will inevitably mean the best text editor, period. Nail linking and some general UI improvements, and implement auto-save that makes an easily implementable rest call that other languages can support (with reference implementations in python and php for wordpress), and it will be better than just about anything.
Also one thing I think you skirt around but don’t quite come out and say - there’s a joy to contributing to open source, a glorious feedback loop of positive esteem, that may be missing from openplans development. Though I started doing open source for the ‘moral’ reasons - it lines up with my philosophies - the reason I stick with it is extremely personal and almost selfish. If I were to work for a proprietary company I wouldn’t get people emailing out of the blue thanking me for my work, telling us how much they love what we’ve built. I’m a junkie for that positive feedback, and though the end product of openplans.org will get there, I am confident it can be found in the short term by active participation and contribution to the communities we build upon.
Comment by cholmes on January 29, 2008 at 11:34 am
Not much to add here except all the + I can muster.
Comment by sbenthall on January 31, 2008 at 11:06 am
+1, of course. I think it’d be useful to see a breakdown of what open source products we use, which of our developers work on them, and what (if anything) we already have to contribute. Basically, I think this is something worth committing to and I want to see some concrete steps towards being involved in other communities.
Comment by rpenate on January 31, 2008 at 2:12 pm