OpenCore Software

This showed up on my radar today. It’s a cute little hack, if a bit rough. (I’ve already posted a couple patches).

It could easily be generalized, there’s nothing really zope-specific about it, all it really cares about is a path to a common-apache-format access log and the PID of a process to watch memory. The result is a nice little Flot graph where you can select a range and see what URLs were hit during that time.

Here’s a screenshot of an example from a flunc run against opencore (the “basic” suite):

So what does this tell us really? I’m not sure. Multiple runs show that the graph generally starts around 400 MB and ends around 500 MB, but what happens between does not seem consistent. So I’m not sure if this little app is really that useful for opencore.

I spent the last two days at the Cisco facility in San Jose, attending the third NetSquared conference. NetSquared is a conference that focuses on the intersection of activism and technology. It brings together people from NGOs, foundations, and technology providers to foster conversation, collaboration, fund-raising, and just to generally get folks connected and excited about the interesting ways that the net is being used to enable social change.

There were a number of talks, panels, and conversations, but the central focus of the event was a contest between 21 featured projects. Early on the first day, each of these projects was given 2 minutes to describe themselves; this was immediately followed by an hour-long session where attendees could walk around and chat w/ project members to learn more. Near the end of the last day, they were all given another 2 minutes to make their closing pitch, after which attendees were able to vote for which projects they felt were most deserving. Each person was given 3 votes, which could be distributed as desired (i.e. all for one project or spread among 2 or 3). the top three vote winners received $25000, $15000, and $10000, respectively; the rest split equally the remaining available grant money, which rumor had it came to about $2000 per.

It was a lot fun and inspiring to see what good work people are doing in the world. The coolest thing I saw was by Ushahidi, the winner of the $25000 prize. They’ve been mapping outbreaks of violence and peace as a reporting and research tool during Kenya’s post-election tumult. They’ve got a great timeline map, with which you can get a great visual representation of the way events developed through time. The timeline component is something out of MIT’s SIMILE project, and thus is free for us to use. This could be very useful for TOPP; it’d be a great way to present the summer’s worth of block parties, for instance. And it’d be fun to add geocoded event support to OpenPlans; users who are producing events (sprints, protests, fundraisers, whatever) would be able to enter the time and location, and their event would show up on a regional timeline map. Other folks in that region would get a great visual overview of what was going on around them.

Beyond just generally learning what was out there, the time spent talking to people and listening to talks left me with a few specific takeaways:

1. Mapping is the new black
   
   Nearly everybody is doing mapping related work. Maybe this was a bit slanted, because “Mash-ups” was one of the themes of the conference and, frankly, I can’t think of a compelling mash-up I’ve seen that wasn’t mapping-related. But I think it’s more than that; mapping is very useful, and people are putting mapping tools to good use in all sorts of ways.
2. TOPP’s Geo group is in a great position
   
   While there are a lot of people doing mapping work out there, TOPP has been doing geo stuff for longer than just about anybody I’ve seen in the open source-y non-profit world. And the depth of the work TOPP has done is impressive; we haven’t just used existing tools in clever ways, we’ve been one of the driving forces behind the very existence of a high-powered open source end-to-end GIS system. I spoke to a couple of folks who had been in consideration for larger geo-related contracts w/ gov’t and NGO organizations where the had really wanted to use the open source offering, but had balked in the end from fear of becoming dependent on a one- or two-person consultant team. With TOPP’s depth of experience and resources (human and otherwise), we’ll be able to make such organizations feel much more comfortable. This has been the plan all along, but it’s really exciting to talk to people who are out there in the middle of it confirming that the opportunities really are there, and that we’re in a great place to be able to take advantage of them.
3. TOPP itself is in a great position
   
   Most of the people who are pouring their lives into using technology for good causes are constantly hustling for funding. Either they’re freelancing and are trying to land their next contract (even as they’re working on their current one), or they’re a non-profit that’s constantly pounding the pavement for grants and other funding. TOPP is very unusual, and very lucky, in that we’re given the opportunity to develop our offerings without a lot of pressure to monetize them; only after we’ve established a reputation and developed demonstrable strengths do we start to focus those strengths towards generating a revenue stream. This seems to be working fabulously for the Geo group (see above); I see no reason why it won’t work similarly for OpenPlans, and other projects that we develop, as long as we continue to produce working, usable software.

I managed to talk to a lot of folks throughout the event, folks were generally very interested in what we’re doing. I had a number of occasions where someone would approach me and introduce themselves, saying “Hi! So-and-so said I should come over and talk to you, that I’d be very interested in what you’re doing. What are you doing?” While it was hard to keep describing our organization and our efforts over and over again without sounding lame and canned, that’s a pretty good problem to have. A couple of the connections I made may end up bearing some fruit down the road. In particular, I’d guess that our Geo folks would be interested in talking to the folks who find out about but do not end up getting those bigger contracts I was mentioning above, so we can decide whether we might be interested in trying to land those jobs. Also, there were a couple of nice mapping front end tools that would probably add value to our greater GIS offerings, if we made sure those tools work with our stack.

All in all I think it was valuable for me to be there. And I think that we should have a slightly bigger presence next year, including giving a presentation of our own. I’m sure they’d be happy to have us.

I’ve stated before one thing I like about openplans is the “simple” permission simple.  I use the quotes because there are several caveats, one of which I’ll address here.  For a particular project, the permission system is essentially a 3×4 table:

  This is pretty simple and intuitive I think.  We also have a status of being listed on a project.  That is, can people see you’re part of the team.  If you’re a whistleblower at exxon, you might not want your company to see that you’re part of the ExxonIsSatan project, for instance.  For a particular project, other members can see information about your affiliation according to the following rubrik:

 I’m being intentionally over-verbose here for illustration.  If user A wants to see information concerning member B about his affiliation with project C, how does this work?

- can user A view project C?

- can user A view member B’s affiliation with project C?

So its a cascaded permission system, if you will (or something like that, anyway).  My main reason for posting this is to put this in programmatic terms and to see what others thought about this. We’re not really taking advantage of as much as we can here, and I’m tempted to say that intelligent use of roles could make this all happen behind the curtain, which would be my preference.  Also, why not have a reference on line?

 [Disclaimer:  no exxon executives, stakeholders, or stock prices were harmed either financially or materially in the writing of this post]

I would like us to figure out our licensing policy.  So far the policy has been:

• If we’re building on some other code base, use the existing license.
• If we’re building our own code, make something up.

So, sometimes people will ask about some piece of code and its licensing, and I’m forced to kind of pull an answer out of thin air, obviously without a great deal of thought.  It’s nice that we make everything open source — and honestly that’s pretty much all I care about — but we need to be a little more clear about what that means for other people.

So: what licensing should we use?  I wrote a post about my own thoughts on licensing.  In summary, I think the most practical choice is a permissive license on libraries (MIT, BSD, etc), and perhaps the GPL (v3?) on applications.  Though if applications have functionality extracted into libraries then that involves license switching — fine to do internally, but if there’s external contributions it can become complicated.

Ideally, I would like as an end product a document that makes it pretty clear to someone when they start a project (of any size) what how that should be licensed.  I’d rather it not document licensing in terms of options, as I honestly don’t think it’s worth the time to put a lot of thought into licensing on a per-project basis.

Looking at our code (that’s how it always starts) I was again bitten by that disgust for DRY code. Some of this is inevitable in a web framework, I think. Web frameworks are by their nature complex programs. Should a web framework handle authentication and permissions? Almost definitely. Should a web framework handle unicode and i18n and localization issues? One would hope so.

Python has a bunch of these frameworks and I think this is a good thing. What I do question is how much functionality lives in the framework that could be abstracted outside of the framework.  What is a framework but a tool kit that you want to apply to HTTP requests and responses?  Thinking that way, issues like handling unicode, HTML escaping, authentication, etc are really library type functions.  If Bob doesn’t like pylons but likes how they do auth, that part should be pretty libraried-out (well, there’s authkit, which is actually a good example of what I consider *bad* library code).

This isn’t a magic bullet, not do I encourage programmers to prematurely make their code library-like.  I’ve been bitten by that too.  But once one figures out process and what one *really* wants to do, its easy to figure out the pattern, figure out which parts are actually *part* of the web framework and which parts are better *consumed* by the web framework.