Jeffrey Hearn and Joss Ingram have published examples of using Wagtail’s hooks system to register custom hallo.js plugins for additional WYSIWYG features.  Using their examples I was able to quickly wire up some plugins that I needed for a Wagtail project, including:

  • Indent and outdent buttons, for nested lists and indented text blocks
  • Superscript and subscript buttons
  • Underline and strikethrough buttons
  • “Paste as plain text” and “edit raw html” buttons (but more on these in another post) 

There was one missing piece that was bugging me though — how to make the buttons stateful, so that (e.g.) when the cursor is within a block of text that’s already superscript, the “superscript” button would show up as already on, and clicking it would disable superscript.

Digging into Wagtail and hallo.js source I found an answer — the `queryState` option to the `hallobutton` constructor.  This option should be sent in as a callback function which can tell the hallo button whether it should be drawn as enabled or disabled: 

[…]

populateToolbar: function(toolbar) {
 var button, widget;

  var getEnclosing = function(tag) {
                    var node = widget.options.editable.getSelection().commonAncestorContainer;
                    return $(node).parents(tag).get(0);
  };



  button = $(’<span></span>‘);
  button.hallobutton({
                    uuid: this.options.uuid,
                    editable: this.options.editable,
                    label: ‘Strikethrough’,
                    icon: ‘fa fa-strikethrough’,
                    command: null,
                    queryState: function(event) {
                        return button.hallobutton(’checked’, !!getEnclosing(”strike”));
                    }
  }); 

  toolbar.append(button);

  button.on(’click’, function(event) {
                    return widget.options.editable.execute(’strikeThrough’);
  });

}

[…] 

This handles the UI aspects of statefulness.  

In some cases — like strikethrough and underline — nothing else is needed: the browser’s `execCommand(”strikeThrough”)` will already check whether the selection is already enclosed in a strike tag, and disable strikethrough if so.

In other cases — like superscript and subscript — the browser’s execCommand function doesn’t seem to implement the toggle behavior; in those cases, the plugin button’s onclick function will need to check for an enclosing sup/sub tag and do the work of removing the formatting instead:

  button.on('click', function(event) {
      return widget.options.editable.execute(
        getEnclosing("sup") ? 'removeFormat' : 'superscript');
  }); 
Filed August 9th, 2014 under wagtail

Wagtail has a built in “Redirects” system that lets you set up redirects for particular URLs.  These are managed independently of your site’s Page tree, which has a few disadvantages:

  • The editor’s user interface just provides a raw URL field for the source of the redirect, instead of letting you add redirects from within the Explorer and see them in their urlspace context.
  • If you set up a redirect at /my-page/redirect-url/ then the “my-page” parent won’t have any knowledge of the redirect that’s underneath it.  This means you’d need to do extra work and database queries any time you want to do something with those redirects in the page hierarchy, like displaying them in their parent page’s navigation menus.
  • You lose all the other built in features of pages, like search integration, workflow, versioning, etc.

So, for typical usage (e.g. avoiding link breakage when pages move around) the built in system is appropriate, but sometimes you may need to treat redirects a bit more like fully managed content.

Luckily it’s easy to set up a custom Page type which acts as a redirect.  You just need to override the “serve” method from the core Page class, for example:

class PageAlias(Page):

    alias_for_page = models.ForeignKey(’wagtailcore.Page’, related_name=’aliases’)

    def serve(self, request):
        return redirect(self.alias_for_page.url, permanent=False)

Now you have a new Page type which editors can add anywhere in their site tree, using the standard user experience including the page chooser interface, which will redirect to your chosen page instead of rendering a template.

Filed May 10th, 2014 under wagtail

Each wagtail Page class has a “template” attribute which tells the system what template (as a string identifier) should be loaded when rendering instances of that Page.  As Serafeim Papastefanos writes in his Wagtail tutorial:

A normal Django template can be used to display each page type. Wagtail either generates automatically (by seperating underscors with capital letters in camelcase for instance BlogIndexPage -> blog_index_page) or you can use the template class attribute. Let’s add a templates foler within the tutorial foler and add another folder named tutorial inside templates and ten add a file named blog_index_page.html to templates with the following content (you must have the following hierarchy wagtailtutorial/tutorial/templates/tutorial/blog_index_page.html):

This lets you specify a custom template per page type.  But what if you want to allow administrators to choose from a set of templates for each page?

The Page.template attribute is only accessed when rendering a page instance, so it’s possible to set up a dynamic template per page instance using a property, for example:

class CustomPage(Page):

    template_string = models.CharField(max_length=255, choices=(
                                         (”myapp/default.html”, “Default Template”), 
                                         (”myapp/three_column.html”, “Three Column Template”,
                                         (”myapp/minimal.html”, “Minimal Template”)))
    @property
    def template(self):
        return self.template_string 

Now your editors can dynamically set the chosen template for each page instance.

Of course in this example, you could also just define a “template” CharField directly in the model, instead of using a property.  But using a property allows you to do even more dynamic things like referencing a foreign key to a database-backed template class that admins can edit through the web, or inspecting the page object’s attributes and returning a different template based on which ones are filled in, etc.

Interactions with Abstract Models 

One caveat, though.   Wagtail uses some metaclass magic to add the default “{{ app-name }}/{{ page-type }}.html” template attribute to each class on initialization, by inspecting the existing class attributes and only adding the default value if the class itself doesn’t provide a “template” attribute:

class PageBase(models.base.ModelBase):
    “”"Metaclass for Page”"”
    def __init__(cls, name, bases, dct):
        super(PageBase, cls).__init__(name, bases, dct)
        […]
        if ‘template’ not in dct:
            # Define a default template path derived from the app name and model name
            cls.template = “%s/%s.html” % (cls._meta.app_label, camelcase_to_underscore(name))

(That’s in wagtail/wagtailcore/models.py)

If you’re using a class inheritance hierarchy for multiple page types which all share a common abstract base — a pattern that’s encouraged in the wagtaildemo source — then you’ll need to make sure you define your custom “template” property on each concrete subclass, rather than on the abstract base class.  It seems that when you define it on the base class, the “template” doesn’t appear in the dct received by that PageBase.__init__ metaclass method, so wagtail will end up overriding your customization.

In case that’s not clear — don’t do this, it won’t work:

class BasePage(Page):
    template_string = models.CharField(max_length=255)

    class Meta:
      abstract = True

    @property
    def template(self):
        return self.template_string

class BlogPage(BasePage):

    author = models.CharField(max_length=255)



class EventPage(BasePage):
    date = models.DateField()
    location = models.CharField(max_length=255)

Instead you’ll need to do something like this, violating DRY:

class BasePage(Page):
    template_string = models.CharField(max_length=255)

    class Meta:
      abstract = True

class BlogPage(BasePage):

    author = models.CharField(max_length=255)

    @property
    def template(self):
        return self.template_string

class EventPage(BasePage):
    date = models.DateField()
    location = models.CharField(max_length=255)

    @property
    def template(self):
        return self.template_string  

There’s probably a way around this with even more metaclass magic, but this is good enough for me.

Filed May 10th, 2014 under wagtail

By default wagtail displays its add/edit view for all pages with two tabs — “Content” and “Promote” — whose contents you define for each page type by setting the .content_panels and .promote_panels like so:

 HomePage.content_panels = [
    FieldPanel('title', classname="full title"),
    FieldPanel('body', classname="full"),
    InlinePanel(HomePage, 'carousel_items', label="Carousel items"),
    InlinePanel(HomePage, 'related_links', label="Related links"),
]

HomePage.promote_panels = [
    MultiFieldPanel(Page.promote_panels, "Common page configuration"),
]

You can also customize the set of panels themselves.  For example, a particular page type might not need to have a “Promote” tab, or you might want a separate “Sidebar” tab to better organize the editor experience.

Here’s how to do that; suppose you have a custom page class called CustomPage:

from wagtail.wagtailadmin.views.pages import (PAGE_EDIT_HANDLERS,
                                              TabbedInterface,
                                              ObjectList)

PAGE_EDIT_HANDLERS[CustomPage] = TabbedInterface([
        ObjectList(CustomPage.general_panels, heading="General"),
        ObjectList(CustomPage.content_panels, heading="Content"),
	ObjectList(CustomPage.sidebar_panels, heading="Sidebar"),
	])

Note that if you do this, your CustomPage class should define the above attributes (in this example, general_panels, content_panels and sidebar_panels) instead of the typical content_panels and promote_panels. 

I’ve filed a pull request with a cleaner API for the sake of argument, though I guess I’m just curious whether this feature can be relied on moving forward.

Filed May 10th, 2014 under wagtail
  • You can limit a PageChooserPanel to a particular type of page:
       PageChooserPanel(‘project’, page_type=‘myapp.SpecialPurposePage’)

 …ok well just one note for now.

Filed May 9th, 2014 under wagtail

Remind me that there’s a post I need to write about converting a brownie recipe from unsweetened chocolate to semisweet chocolate plus cocoa powder.

Filed February 8th, 2014 under Uncategorized

Last problem … the build needs to download two large tarballs (Zope, and opencore’s Plone Product bundle) from dist.socialplanning.org, which is a Github Pages site.  It seems that Github has decided to throttle downloads aggressively from the server I’m building on: both downloads get 20-80% complete and then hang indefinitely, whether I’m using the fassembler build code or wget.  So I had to download the tarballs to my local computer, scp them to the server, put them in the root of the build directory, and then convince fassembler not to re-download them.  I haven’t yet cleaned up & committed the code for this one — I just edited it locally to comment out the download logic — so I’ll circle back to it before my next rebuild.

Next rebuild in progress but I haven’t circled back to it yet.  :-(  I need to fix this. 

Filed January 26th, 2014 under openfsm, opencore

Actionkit donations have a few important properties that can vary independently:

  • Whether the donation was processed “directly” from an Actionkit form, or via an API call or bulk import
  • Whether it’s a one-time donation or a recurring donation
  • What payment processor was used (PayPal, BrainTree, or Auth.net)
  • Whether the payment succeeded or not

The actual amount of money transferred from a user to the organization is found in a different place in the database (and interpreted differently) depending on these properties.  So answering the question “How much money has User X given us?” is nontrivial.

(Note: even answering that question does not provide the full picture, because additional information is captured for product orders and candidate contributions. So “How much money has the organization raised from User X?” is an even harder question to answer.) 

Here’s some important details:

  1. If the order was processed directly from actionkit, there will be a core_transaction record with an order_id foreign key.  The order will have import_id = NULL.  The transaction will have a status, which must be “completed” for the payment to count.  The order will also have a status.  An order with status=”completed” might still refer to failed or cancelled transactions.  And a transaction with status=”completed” might be attached to a non-completed order!  So both order and transaction must have status=”completed” for the record to truly represent money successfully transferred.  The transaction’s amount field will be the money given.
    • If it’s a one-time donation, the core_order and core_transaction will be one-to-one.
    • If it’s a recurring donation, the core_order will have many core_transactions attached, one per payment period.  
      • There will be additional core_transaction records that represent non-payment events, including the setup of the recurring donation.
      • If the user set up a recurring donation and made the initial payment at the same time, then there will initially be at least two core_transactions for that core_order.
      • These additional non-payment events in core_transaction can be safely included in your tally, because they will have `amount=0`.
    • So, as long as core_transactions with transaction.status <> “completed” or order.status <> “completed” are excluded, it’s safe to just tally the `core_transaction`.`amount` field for all core_orders attached to the user in question.
  2. If the order was processed through an import action or API call, there will not be a core_transaction record.  The order will have a non-NULL import_id.  For these orders (if you wish to include them in the tally) core_order.total should be tallied after exclusion of rows with core_order.status <> “completed”. 

I’m pretty sure that every core_order with an import_id will have no core_transactions attached; and every core_order with core_transactions attached will have no import_id.

So, to collect a user’s total donations, you can make two queries and sum the result:

  • One to `select amount from core_transaction join core_order where t.status = “completed” and o.status=”completed”`
  • One to `select total from core_order where import_id is not null and status = “completed”`   

This should include the total dollar value of all successfully completed transactions, including individual transactions within a recurring donation.  It will not include pledges of future recurring donations.  It will include past transactions that are not donations to your organization, like product orders and candidate contributions.

Filed January 11th, 2014 under actionkit

The wordpress database is supposed to maintain its own copy of all users known to the opencore system.  When a new user account is created, a Zope event subscriber in the oc-wp package makes a signed post subrequest to the Wordpress system with the new user’s information, which then gets stored in the Wordpress database.  Sometimes this fails for some reason, like wordpress being down when the event gets fired, or the request timing out.  This means the wordpress database will be missing a user that it’s expected to know about.

Unfortunately this can cause all kinds of problems subsequently.  For example, when a user joins a project, another Zope event subscriber tries to notify Wordpress about that, to grant the user additional privileges on that project’s blog.  If that subrequest fails, the entire request will raise an exception and the transaction will be aborted.  This means the user will be unable to join the project, and an error will show up in the Zope event log looking like this:

2014-01-03T16:21:51 ERROR Zope.SiteErrorLog [...] http://site.com/people/USERNAME/account

Traceback (innermost last):
  Module ZPublisher.Publish, line 57, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 42, in call_object
  Module opencore.browser.formhandler, line 199, in __call__
  Module opencore.browser.formhandler, line 268, in __delegate
  Module opencore.browser.formhandler, line 96, in __call__
  Module opencore.member.browser.account, line 322, in accept_handler
  Module zope.event, line 23, in notify
  Module zope.component.event, line 26, in dispatch
  Module zope.component._api, line 130, in subscribers
  Module zope.component.registry, line 290, in subscribers
  Module zope.interface.adapter, line 535, in subscribers
  Module zope.component.event, line 33, in objectEventNotify
  Module zope.component._api, line 130, in subscribers
  Module zope.component.registry, line 290, in subscribers
  Module zope.interface.adapter, line 535, in subscribers
  Module <string>, line 1, in <lambda>
  Module opencore.wordpress.subscribers, line 37, in project_contains_blog
  Module opencore.wordpress.subscribers, line 114, in notify_wordpress_user_joined_project
  Module opencore.wordpress.subscribers, line 73, in send_to_wordpress
AssertionError: Failed to update wordpress: 400 User with name USERNAME does not exist! :

There will be plenty of other places where functionality will be broken for this user: leaving a project, updating your profile or email address, being promoted or demoted in a project.

When this happens, we need to manually notify wordpress about the user’s existence, to fix up the account.

You can do this in a zopectl debug prompt by pasting the following code:

from opencore.utils import setup
app = setup(app)
mem = app.openplans.portal_memberdata['USERNAME']

from opencore.wordpress.subscribers import notify_wordpress_user_created
notify_wordpress_user_created(mem, None)

After you’ve done that, the user’s functionality should be restored.

Filed January 3rd, 2014 under opencore

Bitballoon’s (POST) form handler normally responds with a generic Bitballoon-branded “thanks” page.  If you have a reseller platform, you can customize this across your customer sites to brand it on your own. 

But individual sites, and in fact individual forms, can customize their own “thanks” page.  To set up a custom “thanks” page:

  • Build a page in your site tree called “thanks.html” (or anything else — but it must have the .html suffix)
  • In your <form> HTML, set the form’s @action to point to the thanks page, with the .html suffix.  

So, for example: <form action=”/thanks.html”>

Bitballoon will strip out the .html suffix on the page itself and in the form action attribute, so the deployed site will have <form action=”/thanks”> instead.  The form submission will be captured in the Bitballoon database and the user will see your “thanks” page as the response (including seeing the URL /thanks in the address bar).

Using this technique you can point all of the forms on a Bitballoon site to the same thanks page, or to a distinct page per form.  In fact (though I haven’t tested this) you could even point a form to submit back to the same page that the form is on.

When Bitballoon responds to a form POST, it will also inject a special Javascript snippet into the page head.  This javascript will set a global variable called “FormSubmission”, which will be an object containing all the submitted form data (in FormSubmission.data) and some metadata like the submission’s unique non-sequential id, the timestamp of the submission, and the name of the form that was submitted.

You can use this to place conditional content on your page: with Javascript, check for the existence of the FormSubmission variable.  If it’s set, then you know that this is the response to a POST request that submitted a particular form, and you can display a “thanks” message, or fill in the data on the form if you’re submitting back to the same page, etc.

Filed December 21st, 2013 under bitballoon
Next Page »