• flunc_gen

last modified December 26, 2010 by ejucovy

FLUNC_GEN


flunc_gen is a helper application which can be used to automatically generate and maintain flunc suites for situations where a set of tests must be run in combinations of different states with varying expected results.

Its implementation (which may not match this document) currently lives at https://svn.socialplanning.org/svn/opencore/trunk/opencore/ftests/gen/generate_ftests.py

A CSV file is used to describe the states of the system, the actions to perform and the expected result of the
action in a given state.

Example 1: Testing a simple wiki
Example 2: Expanding the wiki
Generation Extras
Generating Test Stubs

Example 1

Testing a simple wiki with different user types.

Imagine you are testing a wiki with three types of users: an administrator, a member and an anonymous user. Wiki pages may be viewed, edited, and deleted.  A member can view and edit.  An anonymous user may only view. Administrators may do anything.



Anonymous
Member
Administrator
View Page
Y Y Y
Edit Page
N
Y
Y
Delete Page
N N Y

(see how this is represented)

Barring setup, there are 9 tests to perform, but using flunc_gen you can write only 6: one for each of the user types, and one for
each of the actions.

neccessary tests:

test
description
setstate_<state id>
a test or suite that put the system in the state specified
<action>_yes a test or suite which asserts that the action can be performed

Output for Example 1

flunc_gen generates the following suite in this situation.  (Note that X: indicates a test is required to fail)
## Enter state [anonymous]
setstate_anonymous
view_page_yes
X:edit_page_yes
X:delete_page_yes
exitstate_anonymous
## Exited state [anonymous]
## Enter state [member]
setstate_member
view_page_yes
edit_page_yes
X:delete_page_yes
exitstate_member
## Exited state [member]
## Enter state [administrator]
setstate_administrator
view_page_yes
edit_page_yes
delete_page_yes
exitstate_administrator

Input for Example 1

flunc uses a CSV file of a certain form to describe the expecations in different states. You can export one from your favorite
spreadsheet application or write it by hand:

example
anonymous, member, administrator

view page, y, y, y
edit page, n, y, y
delete page, n, n, y
line
description
example the first line is the name of the test suite
anonymous, member, administrator
each subsequent line indicates a set of states to cycle through. one state per column
  blank line indicates end of states
view page, y, y , y
an action name followed by a column with a y or n for each state indicating the expected success or failure of the action
edit page, n, y, y
...

Example 2

Expanding the wiki.

Imagine we add the ability for the creator of a page to delete a page. We also add the ability to
lock a page so that only the owner and the administrator can edit it.



Page Unlocked



Page Locked




Anonymous
Member
Owner
Administrator
Anonymous
Member
Owner
Administrator
View Page
Y Y Y Y
Y
Y
Y
Y
Edit Page
N
Y
Y
Y
N
N
Y
Y
Delete Page
N N Y Y
N
N
Y
Y
Unlock Page
N
N
N
N
N
N
Y
Y
Lock Page
N
N
Y
Y
N
N
N
N

(see how this is represented)

Now there are 40 tests to perform, but using flunc_gen you can write only 11.


Output of Example 2

flunc_gen generates the following suite in this situation.  (Note that X: indicates a test is required to fail)
## Enter state[locked,anonymous]
setstate_locked
setstate_anonymous
view_page_yes
X:edit_page_yes
X:delete_page_yes
X:unlock_page_yes
X:lock_page_yes
exitstate_locked
exitstate_anonymous
## Exited state [locked,anonymous]
## Enter state[locked,member]
setstate_locked
setstate_member
view_page_yes
edit_page_yes
X:delete_page_yes
X:unlock_page_yes
X:lock_page_yes
exitstate_locked
exitstate_member
## Exited state [locked,member]
## Enter state[locked,administrator]
setstate_locked
setstate_administrator
view_page_yes
edit_page_yes
delete_page_yes
X:unlock_page_yes
lock_page_yes
exitstate_locked
exitstate_administrator
## Exited state [locked,administrator]
## Enter state[unlocked,anonymous]
setstate_unlocked
setstate_anonymous
view_page_yes
edit_page_yes
delete_page_yes
X:unlock_page_yes
lock_page_yes
exitstate_unlocked
exitstate_anonymous
## Exited state [unlocked,anonymous]
## Enter state[unlocked,member]
setstate_unlocked
setstate_member
view_page_yes
X:edit_page_yes
X:delete_page_yes
X:unlock_page_yes
X:lock_page_yes
exitstate_unlocked
exitstate_member
## Exited state [unlocked,member]
## Enter state[unlocked,administrator]
setstate_unlocked
setstate_administrator
view_page_yes
X:edit_page_yes
X:delete_page_yes
X:unlock_page_yes
X:lock_page_yes
exitstate_unlocked
exitstate_administrator
## Exited state [unlocked,administrator]


Input for Example 2


example2
locked, unlocked
anonymous, member, administrator

view page,   y, y, y, y, y, y, y, y
edit page,   n, y, y, y, n, n, y, y
delete page, n, n, y, y, n, n, y, y
unlock page, n, n, n, n, n, n, y, y
lock page,   n, n, y, y, n, n, n, n
line
description
example2 the first line is the name of the test suite
locked, unlocked
a set of states to run through
anonymous, member, administrator
all combinations of a single element from each row are run through
  blank line indicates end of states
view page, y, y , y, y, y, y, y, y
an action name followed by a column with a y or n for each combination of states indicating the expected success or failure of the action.
edit page, n, y, y, y, n, n, y, y
...

Generation Options

There are a number of optional tests which will be included in the autogenerated suite if they are present in the
search path at generation time.


File
Description
setup_gen_<suite>
Global setup run before anything
setup_<action>
Action specific setup run between gloabl setup_gen and after_setup
after_setup_<suite>
Run after all action specific setups
prelude_<action>
Optional action specific stuff run before entering a new state
<action>_no If there is an explicit "no" test for an action it will be used in lieu of expecting failure in the _yes test
postlude_<action>
Optional action specific stuff run after exiting a state


Generating Test Stubs

flunc_gen can be used to generate a skeleton directory of test stubs for the states and tests found in a given
CSV file.

In write-skel mode flunc_gen creates:
File
Description
predicates/<action>_yes.twill  a test for the success of the action named
states/setstate_<state>.tsuite  a suite to set the state to the state named
states/exitstate_<state>.tsuite
a suite to exit the state named
setup/setup_gen_<suite>.tsuite
a global setup script
setup/after_setup_<suite>.tsuite
a script run after per test setups
Example:
$ flunc_gen -p. ../example2.csv --write-skel
[*] created directory ./predicates
[*] created directory ./setup
[*] created directory ./states
[*] created ./setup/setup_gen_example2.tsuite
[*] created ./setup/after_setup_example2.tsuite
[*] created ./states/exitstate_locked.tsuite
[*] created ./states/exitstate_anonymous.tsuite
[*] created ./states/setstate_administrator.tsuite
[*] created ./states/exitstate_unlocked.tsuite
[*] created ./states/setstate_locked.tsuite
[*] created ./states/setstate_unlocked.tsuite
[*] created ./states/setstate_anonymous.tsuite
[*] created ./states/exitstate_administrator.tsuite
[*] created ./states/setstate_member.tsuite
[*] created ./states/exitstate_member.tsuite
[*] created ./predicates/lock_page_yes.twill
[*] created ./predicates/unlock_page_yes.twill
[*] created ./predicates/view_page_yes.twill
[*] created ./predicates/edit_page_yes.twill
[*] created ./predicates/delete_page_yes.twill