Tollium headless testing

Hints about using tollium's testframework

Basic test setup:

LOADLIB "mod::system/lib/testframework.whlib";
LOADLIB "mod::tollium/lib/testframework.whlib";

ASYNC MACRO TestApp()
{
  // Launch app
  AWAIT ExpectScreenChange(+1, PTR TTLaunchApp("system:webservers"));

  // Actions..
  TT("webserverlist")->selection := TT("webserverlist")->rows[0];
  AWAIT ExpectScreenChange(+1, PTR TTClick("open"));
  AWAIT ExpectScreenChange(-1, PTR TTEscape);
  TestEQ(TRUE, TTIsActionEnabled(":Delete"));

  // Filling fields
  // Set the value of the select with title 'Selection' to the option with title 'Second option'
  TTFillByTitle(":Selection", "Second option");
  // Set the selection of the first of two lists (with any title) to the row with "First text column value" in the first text column.
  TTFillByTitle("{list}:*[1/2]", "First text column value");

  // Close app
  AWAIT ExpectScreenChange(-1, PTR TTEscape);
}

RunTestFramework([ PTR TestApp
                 ], [ testusers := [[ login := "sysop", grantrights := ["system:sysop"] ]
                                   ]
                    ]);

General testing tips

Lists

Common actions:

When dealing with editable lists (listedit, arrayedit, lists with custom add/edit buttons..) test the Edit action after testing an Add action (even if just to read and verify a single field in the Edit screen) and dismiss the screen with TTClick(":Ok"). You shouldn't have to update focus or list selection between dismissal of the Add screen and opening the Edit screen. Eg:

AWAIT ExpectScreenChange(+1, PTR TTClick(":Add"));
TTFillByTitle(":Name", "my filename");
AWAIT ExpectScreenChange(-1, PTR TTClick(":OK"));

AWAIT ExpectScreenChange(+1, PTR TTClick(":Edit"));
TestEq("my filename", TT(":Name")->value);
AWAIT ExpectScreenChange(-1, PTR TTClick(":OK"));

TT pattern matching

The TT function can find elements by their name or by their title. If an element has no title, TT will look at its errorlabel and then its label (for radiobuttons/checkboxes) or hint (for actions/images)

// Find an element named 'tabs' in the current toplevel screen
TT("tabs");

// Find an element named 'questions' inside an element named 'doceditor'
TT("doceditor->questions");

// Find an element by title (prefix with colon)
TT(":Title");

// Find a element of elementyype 'arrayedit' with errorlabel 'Answers'
TT("{arrayedit}:answers");

/* Select the first of the the 3 elements with title 'Add'.
   This only works for title matches.
*/
TT(":Add[1/3]");

// Select the 2nd of 4 arrayedits, ignore the title ('*' can be used to match all titles)
TT("{arrayedit}:*[2/4]");

Common actions

Handling message boxes

Message boxes (as generated by TolliumScreenBase::RunSimpleScreen) can be handled like any other Tollium screen but are more efficiently handled by using ExpectAndAnswerMessagebox

// Click a delete action and confirm:
AWAIT ExpectAndAnswerMessageBox("Yes", PTR TTClick(":Delete"));

// Verify a specific text appears in the message box text
AWAIT ExpectAndAnswerMessageBox("ok", PTR TTClick("import"), [ messagemask := "*5 changes*" ]);

Handling return values

To get the return value of an opened dialog (eg when testing common dialogs) you can use expectcallreturn. Example:

// Create a 'normal folder' (type 0) in the repository (folder 1)
RECORD screenchange := AWAIT ExpectScreenChange(+1, PTR RunNewFSObjectDialog(GetTestController(), 1, [ type := 0, isfolder := TRUE ]));
// Set a name
TT(":Name")->value := "mysubfolder";
// Dismiss the dialog
AWAIT ExpectScreenChange(-1, PTR TTClick(":Ok"));
// Get the ID of the new folder
INTEGER folderid := AWAIT result.expectcallreturn();
TestEq(TRUE, folderid > 0);

(expectcallreturn is a function call so the testframework can detect that you're now waiting for the dialog call to return)

Opening subapplications

If an action triggers a new application, you can open this new app using TTLaunchStartedApp. Example:

// Open publisher
AWAIT ExpectScreenChange(+1, PTR TTLaunchApp("publisher:app", [ params := [ "/webhare-tests/webhare_testsuite.testfolder/webhare_testsuite.site/" ] ]));

// Select and 'double click' file
TT("filelist")->selection := TT("filelist")->rows[0];
TT("filelist")->openaction->TolliumClick();

// Open rich document editor triggered by the above click
rec := AWAIT ExpectScreenChange(+1, PTR TTLaunchStartedApp);

Uploads

To simulate an upload, run ExecuteUpload on an upload action component, eg:

TT("upload")->ExecuteUpload([[ filename := "filemanager_root_uploadtest2.dat", data := DEFAULT BLOB]]);

If you need to upload a XLSX file for your test, considering generating it using GenerateXLSXFile.

Downloads

Getting a file from a <downloadaction>:

// Returns a wrapped blob
RECORD download := (AWAIT ExpectSentWebFile(PTR TTClick("reporturlhistory"))).file;

If the tested function returns a XLSX file, you should be able to parse it with GetOOXMLSpreadsheetRows.

A more complex example testing a RunColumnFileExportDialog:

  AWAIT ExpectScreenChange(+1, PTR TTClick("export"));
  RECORD file := (AWAIT ExpectSentWebFile(PTR TTClick(":Download"))).file;
  RECORD ARRAY testrows := SELECT * FROM GetOOXMLSpreadsheetRows(file.data) WHERE title LIKE "TEST: *" ORDER BY id;
  TestEqMembers([ maincategory := "TEST: another of my categories", sub1 := "TEST: A Subcategory" ], testrows[2],"*");
  AWAIT ExpectScreenChange(-1, DEFAULT MACRO PTR); //download self-closes

Drag and drop

To execute a drag and drop action, use ExecuteDragDrop.

/* Drags an overlay from table 'mytable' to the row in list 'mylist' with
   rowkey 'rowkeyvalue', with a copying drag action
*/
ExecuteDragDrop(
    TT("mytable"),
    selected_overlay,
    TT("mylist"),
    [ dragmode := "copy"
    , target := "rowkeyvalue"
    ]);

Selecting list/select rows

To select a list row based on a text column use TTFillByTitle:

// Selects an option in a select by title
TTFillByTitle(":Duur", "Een kwartier");

// Selects an list row base on the value of the first text column
TTFillByTitle("mylist", "Reparatie 1");

// Selects an list row base on the value of another cell
TTFillByTitle("mylist", "Een kwartier", [ cellname := "duration" ]);