Last problem … the build needs to download two large tarballs (Zope, and opencore’s Plone Product bundle) from dist.socialplanning.org, which is a Github Pages site.  It seems that Github has decided to throttle downloads aggressively from the server I’m building on: both downloads get 20-80% complete and then hang indefinitely, whether I’m using the fassembler build code or wget.  So I had to download the tarballs to my local computer, scp them to the server, put them in the root of the build directory, and then convince fassembler not to re-download them.  I haven’t yet cleaned up & committed the code for this one — I just edited it locally to comment out the download logic — so I’ll circle back to it before my next rebuild.

Next rebuild in progress but I haven’t circled back to it yet.  :-(  I need to fix this. 

Filed January 26th, 2014 under openfsm, opencore

Actionkit donations have a few important properties that can vary independently:

  • Whether the donation was processed “directly” from an Actionkit form, or via an API call or bulk import
  • Whether it’s a one-time donation or a recurring donation
  • What payment processor was used (PayPal, BrainTree, or Auth.net)
  • Whether the payment succeeded or not

The actual amount of money transferred from a user to the organization is found in a different place in the database (and interpreted differently) depending on these properties.  So answering the question “How much money has User X given us?” is nontrivial.

(Note: even answering that question does not provide the full picture, because additional information is captured for product orders and candidate contributions. So “How much money has the organization raised from User X?” is an even harder question to answer.) 

Here’s some important details:

  1. If the order was processed directly from actionkit, there will be a core_transaction record with an order_id foreign key.  The order will have import_id = NULL.  The transaction will have a status, which must be “completed” for the payment to count.  The order will also have a status.  An order with status=”completed” might still refer to failed or cancelled transactions.  And a transaction with status=”completed” might be attached to a non-completed order!  So both order and transaction must have status=”completed” for the record to truly represent money successfully transferred.  The transaction’s amount field will be the money given.
    • If it’s a one-time donation, the core_order and core_transaction will be one-to-one.
    • If it’s a recurring donation, the core_order will have many core_transactions attached, one per payment period.  
      • There will be additional core_transaction records that represent non-payment events, including the setup of the recurring donation.
      • If the user set up a recurring donation and made the initial payment at the same time, then there will initially be at least two core_transactions for that core_order.
      • These additional non-payment events in core_transaction can be safely included in your tally, because they will have `amount=0`.
    • So, as long as core_transactions with transaction.status <> “completed” or order.status <> “completed” are excluded, it’s safe to just tally the `core_transaction`.`amount` field for all core_orders attached to the user in question.
  2. If the order was processed through an import action or API call, there will not be a core_transaction record.  The order will have a non-NULL import_id.  For these orders (if you wish to include them in the tally) core_order.total should be tallied after exclusion of rows with core_order.status <> “completed”. 

I’m pretty sure that every core_order with an import_id will have no core_transactions attached; and every core_order with core_transactions attached will have no import_id.

So, to collect a user’s total donations, you can make two queries and sum the result:

  • One to `select amount from core_transaction join core_order where t.status = “completed” and o.status=”completed”`
  • One to `select total from core_order where import_id is not null and status = “completed”`   

This should include the total dollar value of all successfully completed transactions, including individual transactions within a recurring donation.  It will not include pledges of future recurring donations.  It will include past transactions that are not donations to your organization, like product orders and candidate contributions.

Filed January 11th, 2014 under actionkit

The wordpress database is supposed to maintain its own copy of all users known to the opencore system.  When a new user account is created, a Zope event subscriber in the oc-wp package makes a signed post subrequest to the Wordpress system with the new user’s information, which then gets stored in the Wordpress database.  Sometimes this fails for some reason, like wordpress being down when the event gets fired, or the request timing out.  This means the wordpress database will be missing a user that it’s expected to know about.

Unfortunately this can cause all kinds of problems subsequently.  For example, when a user joins a project, another Zope event subscriber tries to notify Wordpress about that, to grant the user additional privileges on that project’s blog.  If that subrequest fails, the entire request will raise an exception and the transaction will be aborted.  This means the user will be unable to join the project, and an error will show up in the Zope event log looking like this:

2014-01-03T16:21:51 ERROR Zope.SiteErrorLog [...] http://site.com/people/USERNAME/account

Traceback (innermost last):
  Module ZPublisher.Publish, line 57, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 42, in call_object
  Module opencore.browser.formhandler, line 199, in __call__
  Module opencore.browser.formhandler, line 268, in __delegate
  Module opencore.browser.formhandler, line 96, in __call__
  Module opencore.member.browser.account, line 322, in accept_handler
  Module zope.event, line 23, in notify
  Module zope.component.event, line 26, in dispatch
  Module zope.component._api, line 130, in subscribers
  Module zope.component.registry, line 290, in subscribers
  Module zope.interface.adapter, line 535, in subscribers
  Module zope.component.event, line 33, in objectEventNotify
  Module zope.component._api, line 130, in subscribers
  Module zope.component.registry, line 290, in subscribers
  Module zope.interface.adapter, line 535, in subscribers
  Module <string>, line 1, in <lambda>
  Module opencore.wordpress.subscribers, line 37, in project_contains_blog
  Module opencore.wordpress.subscribers, line 114, in notify_wordpress_user_joined_project
  Module opencore.wordpress.subscribers, line 73, in send_to_wordpress
AssertionError: Failed to update wordpress: 400 User with name USERNAME does not exist! :

There will be plenty of other places where functionality will be broken for this user: leaving a project, updating your profile or email address, being promoted or demoted in a project.

When this happens, we need to manually notify wordpress about the user’s existence, to fix up the account.

You can do this in a zopectl debug prompt by pasting the following code:

from opencore.utils import setup
app = setup(app)
mem = app.openplans.portal_memberdata['USERNAME']

from opencore.wordpress.subscribers import notify_wordpress_user_created
notify_wordpress_user_created(mem, None)

After you’ve done that, the user’s functionality should be restored.

Filed January 3rd, 2014 under opencore