Testframework and Continuous Integration

• Validation
• Testframework
• Testfw
• JS Tests
• Tollium headless testing

Testframework API

• OBJECTTYPE AsyncEventWaiter
  

  This objecttype provides facilities for waiting on specific events. It also allows events expanding into new waits

• MACRO DumpStackTrace(RECORD ARRAY trace)
  

  Prints a stacktrace

• OBJECT FUNCTION EnsureTestSite(STRING sitename, RECORD options)
  

  Setup a test site. It will be created as /webhare-tests/ and use WebHare backend:/testoutput/ as output

• OBJECT FUNCTION EnsureWHFSTestRoot()
• RECORD FUNCTION ExtractWebdesignRenderingSummary(OBJECT htmldoc)
  

  Extracts the webdesign rendering summary from an HTML document

• STRING FUNCTION GetRandomTestModuleName()
• STRING FUNCTION GetTestSecret(STRING varname)
  

  Get a test secret, either from the environment or set using wh debug setsecret

• BOOLEAN FUNCTION IsBeingPublished(INTEGER objid)
  

  Returns true when a file is currently being published

• BOOLEAN FUNCTION IsBeingPublishedSince(INTEGER objid, DATETIME when)
  

  Returns true when a file is currently being published

• BOOLEAN FUNCTION IsTestModuleName(STRING module)
• MACRO MarkAsPublished(INTEGER objid, BOOLEAN needswebdesign, DATETIME setlastpublishdate)
  

  Marks a file as published

• OBJECT FUNCTION OpenTestsuiteWRDSchema()
  

  Opens temporary the WRD schema created by the testsuite

• VARIANT FUNCTION RecurseDeleteSettingIds(VARIANT data, STRING ARRAY extra_deletes)
  

  Deletes all setting ids (wrd_settingid, fs_settingid, __blource, etc) from records, to be able to compare the content

• RECORD FUNCTION RunFilePublish(INTEGER fileid, BOOLEAN profile)
  

  Runs an inline file publication

• OBJECT FUNCTION RunTestframework(FUNCTION PTR ARRAY tests, RECORD options)
  

  Run a number of tests within the framework

• MACRO SetupSiteTestOutput(OBJECT site, STRING subfolder)
  

  Setup test output for a site

• MACRO SnoozeRateLimits()
  

  Request an extension on all rate limits. Must currently be renewed every 10 minutes, invoke testframework with keepratelimits := TRUE if you don't want this bump (but previous tests can currently still have a lingering enabling)

• MACRO TestAssert(BOOLEAN to_assert, STRING annotation)
  

  Test if an expression is TRUE

• MACRO TestEq(VARIANT expected, VARIANT actual, STRING annotation)
  

  Compare two values by value

• MACRO TestEqCSSColor(STRING expected, STRING actual, INTEGER max_deviation, STRING annotation)
  

  Tests if a CSS color matches the expected value

• MACRO TestEqFloat(FLOAT expected, FLOAT actual, FLOAT max_deviation, STRING annotation)
  

  Test if a float value matches the expected value, with a specific maximum deviation

• MACRO TestEqLike(STRING mask, STRING actual, STRING annotation)
  

  Test if a value matches a LIKE mask

• MACRO TestEqMembers(VARIANT expected, VARIANT actual, STRING memberstext, STRING annotation)
  

  Compare specific cells of two values (recursive)

• MACRO TestEqStructure(VARIANT expect, VARIANT input, STRING annotation)
  

  Tests if a value follows a structure

• OBJECT testfw
  

  Test framework

• CONSTANT STRING testfw_testmodule
  

  Name of the testmodule

• RECORD ARRAY FUNCTION TestInvoke_ExtractAllMailFor(STRING emailaddress, RECORD options)
• OBJECT FUNCTION TestRejectedLike(STRING mask, OBJECT promise, STRING annotation)
  

  Waits until a promise fulfulls, and tests if it is rejected

• OBJECT FUNCTION TestThrowsLike(STRING mask, FUNCTION PTR testfunc, STRING annotation)
  

  Tests if a function throws an exception that matches a LIKE mask

• MACRO TestUnreachable(STRING annotation)
  

  Reports an error when this statement is reached

Publisher test APIs

• OBJECTTYPE FormsapiFormTester
  

  Publisher forms (CI) tester returned by OpenFormsapiFormTester

• OBJECT FUNCTION OpenFormsapiFormTester(OBJECT browser, RECORD options)
• OBJECT FUNCTION SetupTestWebsite(STRING webdesigntemplate, RECORD options)
  

  Setup a website for continuous integration tests

Tollium test APIs

• ASYNC FUNCTION AnswerMessageBox(STRING answer, RECORD options)
  

  Answers the currently open messagebox

• MACRO DumpRunningExpectCalls()
• BOOLEAN FUNCTION ExecuteDragDrop(OBJECT sourcecomp, VARIANT selection, OBJECT targetcomp, RECORD options)
  

  Do a drag-drop onto another component

• BOOLEAN FUNCTION ExecuteListDragDrop(OBJECT sourcecomp, VARIANT rowkeys, OBJECT targetcomp, RECORD options)
  

  Do a drag-drop from list to list

• RECORD FUNCTION ExecuteWindowOpenAction(OBJECT actioncomponent, RECORD options)
  

  Execute an open-new-window action

• ASYNC FUNCTION ExpectAndAnswerMessageBox(STRING answer, MACRO PTR call, RECORD options)
  

  Expect a message-box to open after a trigger-call, and answer it

• ASYNC FUNCTION ExpectNoScreenChange(MACRO PTR call)
  

  Expect the screen stack to not change after a call

• ASYNC FUNCTION ExpectOptionalProgressWindow(MACRO PTR call, RECORD options)
  

  Expect that an async action (triggered by call, if specified) may result in a progress window, process that progress window

• ASYNC FUNCTION ExpectScreenChange(INTEGER diff, MACRO PTR call, RECORD options)
  

  Expect the screen stack to change

• ASYNC FUNCTION ExpectSentWebFile(MACRO PTR call, RECORD options)
  

  Expect a file to be sent to the user

• OBJECT FUNCTION ExpectWorkErrors(MACRO PTR call, RECORD options)
  

  Expect work errors, auto 'ok' them

• OBJECT FUNCTION GetTestController()
• OBJECT FUNCTION GetTestsuiteWRDauthPlugin()
• STRING FUNCTION InlineScreens(STRING content, STRING hash)
• RECORD FUNCTION TestInvoke_GetFileTransferData(STRING url)
• OBJECT topscreen
• OBJECT FUNCTION TT(STRING compname, RECORD options)
  

  Look for a component in the current toplevel screen

• BOOLEAN FUNCTION TTClick(STRING compname, RECORD options)
• MACRO TTEscape()
• MACRO TTFillByTitle(STRING compname, VARIANT val, RECORD options)
  

  Sets a component value

• BOOLEAN FUNCTION TTIsActionEnabled(STRING compname, RECORD options)
  

  Test is an action is enabled wrt current selection and flags

• BOOLEAN FUNCTION TTIsVisible(STRING compname, RECORD options)
  

  Check whether the requested component is visible */

• ASYNC MACRO TTLaunchApp(STRING appname, RECORD options)
  

  Launch an application

• ASYNC MACRO TTLaunchStartedApp()
  

  Launch the most recently requested application (eg after a click)

WRD test APIs

• MACRO CompleteWRDTestpayment(STRING payurl, STRING approvetype)
• RECORD FUNCTION CompleteWRDTestpayment2(STRING payurl, STRING approvetype, RECORD options)
  

  Accept or reject testprovider payment

• MACRO DumpWRDEntityHistory(OBJECT entity, RECORD options)
  

  Dumps the history changes for a specific entity

• MACRO PushWRDTestPaymentButton(STRING approvetype)
  

  Click testprovider payment approval button

Syntax/file validation

• STRING FUNCTION FormatValidationError(RECORD msg)
  

  Returns the string for a validation error from an XML (without column number)

• STRING FUNCTION FormatValidationWarning(RECORD msg)
  

  Returns the string for a validation warning from an XML (without column number)

• RECORD FUNCTION GetModuleValidationConfig(STRING modname)
  

  Returns the validation configuration for a module

• STRING ARRAY FUNCTION GetValidatableFiles(RECORD validateconfig, STRING modulename, RECORD options)
  

  Gathers the list of files that must be validated

• MACRO PrintValidationMessages(RECORD file)
  

  Prints (colored) validation errors and warnings to the console

• RECORD FUNCTION ScanModuleTids(RECORD validationconfig, STRING module)
  

  Returns the list of tids in a module

• RECORD FUNCTION ValidateSingleFile(STRING resourcename, RECORD options)
  

  Validates a single file

• RECORD FUNCTION ValidateSingleFileAdhoc(STRING resname, RECORD options)
  

  Validate a single file using the module configuration for validation options