• Flunc Project Home

last modified September 30, 2012 by egj

QuickStart


If you just want to use flunc, the quickest way to install it is using easy-install:

easy_install flunc 
If you don't want to install it globally on your system, you can also use virtualenv:

virtualenv myflunc
cd myflunc
source bin/activate
easy_install flunc

If you want to develop flunc itself, you can install from source:


virtualenv myflunc
source myflunc/bin/activate
mkdir myflunc/src
cd myflunc/src
svn co https://svn.socialplanning.org/svn/flunc/trunk flunc
cd flunc
python ./setup.py develop


Example Flunc tests

Examples of tests and suites can be found at https://svn.socialplanning.org/svn/opencore/trunk/ftests

 

Tests, Suites and Configurations

Flunc uses browser simulation to do rudimentary functional testing over the web using Twill scripts chained together by a simple harness. The testing harness understands 3 things: tests, suites and configurations.

Tests

Tests are just simple twill scripts. The tester recognizes files with the extension '.twill' as tests

go ${base_url}/login_form
fv login_form __ac_name $user
fv login_form __ac_password $password
submit submit
find 'You are now logged in'

Suites

Suites may simply be newline separated lists of test names (and possibly suite names). When run, the tests listed are run in the order specified. The tester recognizes files with the extension '.tsuite' as suites.

create_user
login
create_project
edit_project
logout
destroy_project
destroy_user

Configurations

Configurations are also twill scripts, but by convention they contain only statements which set the value of variables used in tests. The tester recognizes files with the extension '.conf' as configurations.

setglobal user      'tester'
setglobal password  'tester1'

Stringing Things Together

The test runner will run suites and tests in the order specified on the command line. When a suite is encountered, if a configuration with a corresponding name is found, the configuration is loaded, eg all.tsuite and all.conf.

In addition, a configuration file may be specified on the command line using the -c option. This configuration will partially override any suite configurations that are loaded, eg this configuration might only override $user.

Flunc only looks for configuration files for suites, not for individual twill files.

Example Usage

$ flunc --help
show available command line options
$ flunc -l
list test suites
$ flunc all
run the suite 'all.tsuite' against the default host using the default configuration
$ flunc -c basic create_user login create_project destroy_project destroy_user
run the specified tests specified in order using 'basic.conf' for configuration
$ flunc -t http://localhost:8080/some_portal all
run 'all.tsuite' against the portal specified (set base_url to the url given)

Creating New Tests

Since simple tests are just twill scripts, all you need to do to create a new test is create a file that ends in .twill and put in a few twill commands.

By default flunc will look for tests in the ftests folder within the flunc egg. You can run tests in the current
directory with the -p. command line flag.

The following asserts that we can load http://www.google.com.

$ echo "go http://www.google.com" > go_google.twill
$ echo "url http://www.google.com" >> go_google.twill
$ flunc -p. go_google

[*] scanning for tests in '.'
[*] Running against http://localhost:8080/openplans, host: localhost:8080 path=/openplans
[*] running test: go_google
==> at http://www.google.com
[*] All Tests Passed!

Often when creating tests, it is helpful to abstract out specific test data into twill variables. This includes data like:
username, password, base url. The goal is to make it easy to run the tests in an slightly different context.

Builtin Variables

Flunc always defines a handfull of twill variables which are often useful in writing retargetable tests.

variable name description
base_url this is the full url provided on the flunc command line representing the site that should be tested
base_host this is just the hostname contained in the base url
base_path
this is just the path portion of the base url
test_path
this is the root of the flunc search path on the local filesystem
The base url defaults to http://locahost/, but can be set to anything using the -t command line flag.

The following asserts that we can go to the base url.
$ echo "go \${base_url}" > go_somewhere.twill
$ echo "url \${base_url}" >> go_somewhere.twill
$ flunc -p. go_somewhere -t http://www.yahoo.com

[*] scanning for tests in '.'
[*] Running against http://www.yahoo.com, host: www.yahoo.com path=
[*] running test: go_somewhere
==> at http://www.yahoo.com
[*] All Tests Passed!


Debugging Tests

There are some useful command line options that aid while debugging tests

-i, --interactive-debug
Fall into twill shell on error
-d DUMP_FILE, --dump-html=DUMP_FILE
dump current HTML to file specified on error. specify - for stdout.
-w, --show-error-in-browser
show dumped HTML in a web browser on error, forces interactive mode
-v, --verbose
Display stack traces from twill
-F, --ignore-failures
continue running tests after failures
-C, --cleanup-only
run only the suite cleanup and not the tests
-X, --no-cleanup
do not run cleanup handlers for suites

Various combinations of these options can allow of interesting possibilities. Here are some useful examples:

$ flunc -i all
Drop into an interactive shell on the first failure. The twill shell provides excellent commands to help debug the problem. Some useful ones to start off with are: showforms, showlinks, show, save_html, and find.
$ flunc -w all
Drop into an interactive shell on the first failure and display the current page in a web browser
$ flunc all -wF
This command will launch a web browser with the page it failed with on error, as well as drop you into an interactive twill shell. You can fix the issue in the twill script, while verifying visually what went wrong through your browser. After making the correction, the runner will continue execution of the remaining suites starting with the next test.
$ flunc -C basic
Runs the cleanup for the basic suite without executing any of the tests

Advanced Usage

Test Arguments

It is sometimes useful to provide arguments to a test inside a suite to avoid repeating a series of similar steps. To support this, suites support a simple argument passing syntax to override variables in a test. The assignments apply only during the execution of the particular test. Values may be strings or refer to global twill variables which have been assigned in configurations. n.b this syntax currently has no meaning when applied to suites.

login(user='joe')
do_something # as_joe
logout

login(user=${admin_user})
do_something_fancy(blah=${somearg}, foo='joop') # as admin
logout

Cleanup Handlers

flunc will recognize suites which end in '_cleanup' as cleanup handlers, eg 'policy_cleanup.tsuite' as the cleanup handler for 'policy.tsuite'. Cleanup handlers will always run after a suite regardless of failure. In interactive mode, cleanup handlers are not run until the user has exited the twill shell.


Host Redirection

Flunc can be told to send requests to different hosts. All this affects is where the request is sent, the old host header is still set normally.
This can be accomplished in one of 2 ways:

host_alias extension
- this is an extension to the twill syntax. You need to let twill know about it by running:
extend_with flunc.redirect_host.
host_alias new_hostname actual_url

host mapping file
- flunc can be given the -m option. Pass in the full path of the host mapping file to use.
Each line of the file contains one mapping:
new_hostname actual_url

If the port is 80, then the port does not have to be specified on the host. However, if the port should be changed, then you specify it like: host:port

extend_with flunc.redirect_host
host_alias foo.openplans.org www.openplans.org
go http://foo.openplans.org
url http://foo.openplans.org
host_alias bar.openplans.org:8080 www.openplans.org
go http://bar.openplans.org:8080
url http://bar.openplans.org:8080

In addition to just simple host redirection, you can you can also redirect to a new host with a selector (portion of url) prepended to it, almost like a url mapping. This is useful for testing different virtual host mappings.
extend_with flunc.redirect_host host_alias quux.openplans.org http://www.openplans.org/VirtualHostBase/http/quux.openplans.org/openplans/projects/someproject/VirtualHostRoot 

Now anything appended to quux.openplans.org will be tacked on to the host that it was aliased to. For example:
http://quux.openplans.org/baz.html -> http://www.openplans.org/VirtualHostBase/http/quux.openplans.org/openplans/projects/someproject/VirtualHostRoot/baz.html