Please ping Mel (mchua) with feedback and questions, or leave notes below.

comments

• leave your thoughts here!
   

overview of opdev (openplans dev center)

Opdev is a developers' center for standalone components of the openplans stack (in particular: deliverance, flunc, and listen, to be integrated in roughly that order as a starting point). It is primarily intended as a communications and documentation hub for developers, both TOPP and non-TOPP coders, working on the component projects themselves. It also incidentally serves as a test/demo ground for some of these components, but only when it makes sense to do so for a development environment (i.e. we dogfood, but only in situations where we would use the component even if we hadn't made it ourselves). 

[while I generally agree with this sentiment, I feel it should be acknowledged that we've gained a lot by using our tools -- we aren't the target users, but we are users, and we aren't trans-human or otherwise all that odd.  My real concern has been more that we don't hurt our productivity by forcing ourselves to use inappropriate tools.  So I would really like to keep using our tools when it at all makes sense (though also with different tolerance depending on how important the tool is -- we need a good bug tracker and good versioning, but what else do we need? -- Ian]

[Bug tracker, versioning, and a stable place for docs that's as easy for us to use as possible - interfaces with your favorite text editor, say, and nicely formats code snippets placed inline. Plus some sort of real-time chat. That's my personal minimal-features list, and we have most of it already, in varying forms of usability... the usability barrier is pretty crucial esp. for docs, though, because folks won't do it if it isn't easy. I agree that we're not trans-human and do want to dogfood as much as possible, but what's "easiest" for developers isn't always what's easy for non-devs... we've climbed the learning curve on tools like trac, so why not use them if we think they're "better" for our (techie-person) purposes? I think I've just repeated Ian here, though - my impression is that we agree on this count, but that everyone's threshold on when openplans is appropriate for dev use is probably different. My primary choice of when to switch platforms isn't so much driven by my personal feelings about ease-of-use of a tool - I have a high tolerance for gritting teeth and hacking around things, if needed - but having something the rest of the development community will use. I tend to follow the crowd I hang with, since persistent documentation is a really big need for me to develop.  --Mel]

[as a side note, I can't wait until Jeff releases his KSS thing for commenting more easily on pages like this. --Mel]

Opdev will hopefully save time for non-TOPP coders by giving them easily accessible new tools to use, and save time for TOPP coders by giving them a development community that contributes back to code without the current overhead of new developers spending several weeks asking questions incessantly in the NYC office. It seeks to automate (or come as close as possible to automating) repetitive tasks and commonly-asked questions through the usage of things like docstrings, searchable communications archives, and a push-out publishing platform for making announcements and starting discussions. (Note that I am deliberately avoiding usage of the terms "blog," "forum," "wiki," and "mailing list" here.)

It will hopefully become a home for "living" documentation that is not only up to date but tied in and integrated to actual code, running demos, and repositories as well as the development community at large. Opdev will also hopefully eventually create a "better" (whatever that means) channel for user/dev feedback on the software we're writing so that questions are available for the TOPP developers to work on, but also for the external community to see so that everyone who has a chance of solving the problem can know that there's a problem and how they might be able to fix it.

Some of these problems are technical; some are social/cultural. Opdev, as a developers' center, must out of necessity be a combination of both.

opdev is NOT

1. a developers' center for openplans-the-stack (this is still on openplans.org)
   [I think by moving to a more explicit outward-facing developer center, we would have something that would at least be useful for openplans-the-stack, which I see as just one particular combination of features that could be a project just like any other -- Ian] [That's my eventual hopeful master plan, but openplans is Really Big and changing rapidly and it makes sense to do the small-scale components as a kind of 'pilot' first. --Mel]
   
2. a developers' center for openplans.org and nycstreets.org the site deployments (this is still on openplans.org)
3. a separate software project/stack of its own [Please shoot me if this ever happens. --Mel]
   
4. a place for external code-hosting in general (all use-cases discussed are currently TOPP projects, although we could discuss this later if opdev is stable and working with several projects and the issue comes up again)

It is possible that opdev may expand to cover other projects in the future ("toppdev"), including things like geoserver and its related subcomponents, if the people on those projects so desire. It is also vaguely possible that it could someday be used as a developers' center for opencores and the openplans stack. However, all these things are blue-sky ideas firmly fixed in the "we'll keep it in mind for later, but definitely not for now" stage as of Feb 2008; only after the initial deliverance/flunc/listen implementation is underway will further additions be considered.
 

motivation: why opdev?

TOPP coders are currently keeping their code in SVN, their tickets in Trac, and everything else on openplans.org. While the first two are beautiful tools designed specifically for the needs of coding teams, the third is a beautiful tool designed specifically for the needs of non-coding teams. Dogfooding is awesome, but only when the reason you're eating the food is that... well, you're a dog.

Separately, there are lots of awesome software tools that were produced in conjunction or as part of the openplans development process that would be super for other coders to be able to use on their non-openplans.org/nycstreeets.org projects. In particular, many Plone bloggers have already been calling for Deliverance, which would also be useful for non-Plone sites. But it's hard for most people to discover Deliverance exists, to figure out how to install it in use cases different from our specific deployments of openplans.org and nycstreets.org, and consequently difficult for them to contribute back to.

Another side result of this is our code is open-as-in-license but not fully open-as-in-spirit; since barriers to adoption and hackery by outsiders are pretty high, this keeps our code from joining the big happy ecosystem of OSS as fully, and discourages external devs from joining the team (which means we have to do all the work). An active volunteer community giving positive feedback, publicity, and bugfixes is a good, good thing. One crucial barrier to getting external developers to contribute is the lack of documentation and support on how they can do so.

features

vital

if possible

1. "forums" or some other way of accessing/contributing-to mailing lists other than an email interface (integrated with lists) [Really?  I never use anything but email to discuss projects; this seems like the norm for open source projects -- Ian] [I think it was Ethan who requested this? I never use forums myself - which is why I'd like them to be synced to a mailing list (apparently only phpBB currently does this, so we'd have to write a plugin for... vanilla, or whatever ends up being used instead). I think it could serve two usually-unfilled-in-OSS-projects roles, though. First is to make the archives more usable to encourage folks to mine information from them - although this theoretically could be done with CSS and non-transient information on these lists really ought to get pushed to wiki anyway. Second - which I think is more important - is to provide a community feedback bridge where nontechnical users can talk with developers. My impression is that less-techie people are sometimes less comfortable subscribing to (sometimes high-traffic) dev lists with lots of conversations they don't understand, and they're more likely to chime in with a "use perspective" at one point on a convo and then pop out... that's something I'd like to cultivate as it seems like our feedback loops from openplans (and subcomponents) users are not terrifically amplified right now. --Mel]  [No, not me; I don't remember who it was, though.  Personally I don't care about forums and would register a weak -1 on forums as a separate feature -- the only reason I care about forums is that I feel very strongly that we should be using Listen for our mailing lists (good dogfooding) and Listen's mailing lists are also forums. -egj]
   
2. integrated search across all individual projects [dogfoodable! -- Ian] [Oh yes. --Mel]
   
3. contributing guidelines/procedures (It's unclear what these are or how strict we should make them; they'll probably emerge organically, at which point we can notice and write down our observations, and decide if we like what we're doing. The consensus seems to be that we shouldn't force any protocols on anyone, but instead make certain practices available, and perhaps "the usual thing to do.") [some guidelines can be helpful; we can mostly copy these from several existing documents -- I have a couple, PEP 8, Zope has a bunch of documents, and maybe Django people? -- Ian] [+1 to "stealing" stuff - at least as a strawman. It would be interesting to see what eventually evolves from the (presumably non-technical) community - folks who have a tons of experience with grassroots but not OSS-culture - might (organically) develop and replace it with, given their massive amount of experience in handling large numbers of volunteers. --Mel]
   

design thoughts

open questions

1. One trac instance or many? [Maybe many, though cohosting many distinct Trac instances in one install is easy enough -- Ian] [I'm not clear yet on how much cohosted Trac instances can be integrated with each other, or how distinct we want to make the projects (I'm thinking "very") but this is something I definitely have to look into. Thanks for the prod. --Mel]
   
2. One plone instance or many?
3. One deliverance theme or many? (The trend seems to be towards "one.") [I think this fits with branding -- Ian]
   

big questions - beyond the scope of this project, but still affecting it

1. Is our product software, or is our product a (hosted) service? (Big Question beyond the scope of this particular project.) [If we make this center, then we're talking about software, right?  This isn't an exclusive choice; just because we produce software, we can also produce a hosted service -- Ian] [End goal is different.
   
2. Who should use openplans? Openplans.org? The components of openplans?
3. What kind of external development should we support for openplans?

alternatives/tools considered

1. launchpad (why: awesome. why not: not open-source, we can't hack it.) [There are some things that seem interesting, but there's a lot of features that feel funny to me on Launchpad, and it doesn't feel as brandable as other options.  But I'm interested in giving this a go with a project like virtualenv -- Ian] [I'd love to check this out when Launchpad "someday" becomes open-source. It'll be TOPP-hostable/brandable/whee then, I presume. But for now I'd rather steer clear of it, because... I feel like it won't give us enough control, as vague as that sounds. --Mel]
   
2. google code hosting (why: awesome. why not: not open-source, we can't hack it.) [I find it a little too primitive -- it's fine for small projects, which is a lot of projects, but we have quite a few projects. -- Ian] [Ian+1 --Mel]
   
3. trac (why: we have it already, good proven tool, most of us like it. why not: doesn't have the complete feature set we want on its own.) [It's also very common and familiar to potential contributors; nearly every project I work with uses Trac -- Ian]
   
4. openplans.org (why: dogfood, our current solution. why not; really not made for code projects.)
   
5. openplans stack separate from openplans.org (why: dogfood with more flexibility to customize than openplans.org. why not: still not made for code projects, changing too rapidly to be a good stable communications base for developers.)
   
6. sourceforge (why: it's there. why not: we don't like the toolsets offered.) [SF is super lame -- Ian] [Yeah, but it exists, and something something writing everything down from brainstorms something something. ;-) But it's my lowest-ranked choice of the options on this list. --Mel]
   
7. plone (why: the openplans stack is based on plone, partial-dogfooding. why not: we already have Too Many Plone Installs to maintain, doesn't have any compelling factors over other options in terms of feature set.)
   
8. pypi (why: free hosting, python rocks. why not: we code in non-python languages as well.) [very limited featureset; only works for tiny projects with no community -- Ian]
   
9. confluence (why: the geoserver team is using it. why not: it's commercial and there's no particularly strong reason to keep it.) [Pylons is also using it.  But eh.  -- Ian]
   

 

questions raised

 

Dogfooding is good. Why are we stopping?

We're not.

Openplans stack development (as a whole, integrated piece of software) will continue on openplans.org. Openplans.org and nycstreets.org development work (these specific instances of the openplans site) will continue on openplans.org. TOPP office stuff will still continue on openplans.org, and some TOPP people will still have non-TOPP projects they're involved in on openplans.org.

Opdev is for individual components that are also used in openplans - the "upstream" packages, if you will (although they're being packed up later than the "downstream" openplans stack). This is an attempt to partition out their development and their interfaces so that they become usable independent of the openplans software stack. It's almost like a retroactive branch-back of the openplans component, except the code itself probably won't be changing much, just the way it's presented, installed, and used.

Besides, dogfooding only works if you use a product for its intended end-use. If your company makes wheelchairs, you don't have to use them for furniture around your conference room table; it's more useful to have normal chairs at your meetingtable and take your wheelchairs on test drives around the office hallways.

Will this interfere with woonerf deployment (the next big openplans launch)?

I hope not. All the existing docs/code/stuff will stay untouched so the woonerf crunch won't be affected; I'll just be copying/porting/updating everything to the new site, which will at first live only on the tower I'm working on (and then will be put live online, but with a big PRE-ALPHA!!! banner to warn potential testers). In other words, you don't need to pay attention to anything I'm doing, if you're so inclined.

implementation

1. Trac - for docs, task/bug tracking, and a blog if possible. [I'm not too psyched about a blog; WordPress works well, though I'd want to add an authoring option that is more appropriate for code content -- Ian] [It's not so much "a blog!" but "a way to get a feed of community updates that can also include explanatory prose in between the "BUILD #NNNN HAS COMPLETED" messages," methinks. For wordpress, how does this plugin look, or something like it? Also, Trac apparently has a blog plugin (I'd prefer to use as few components as possible if functionality/usability aren't sacrificed, to avoid cobbling together The Openplans Stack Version 2.0). Must play with trac-blog-plugin at some point. --Mel]
2. Deliverance to skin/theme Trac. [Repoze is doing this -- Ian] [Who did the repoze site? --Mel][the agendaless guys: http://agendaless.com/ http://repoze.org/contact.html  -- PW]

3. Plone for mailing lists, skinned with Deliverance so it looks like the Deliverance-skinned opdev Trac.
4. (if Trac has no blog, we'll use) Wordpress for blogging, also skinned by Deliverance.