CodeAssurance`
CodeAssurance`

TestFiles

TestFiles[symbol]

gives test files associated with symbol.

TestFiles["symbol"]

gives test files associated with symbol named symbol.

TestFiles["symbol/name"]

gives test files associated with symbol and categorized according to name.

TestFiles[PacletObject[]]

gives test files associated with a paclet.

TestFiles[spec,lev]

gives test files in the directories specified by spec and subdirectories down to level lev.

Details and Options

  • TestFiles returns {File[], }, a list of test files.
  • TestFiles["symbol"] resolves to a list of files of the form {File["root/symbol/ *.wlt"],} or {File["root/symbol/ *.nb"],} .
  • TestFiles["symbol/name"] resolves to a list of files of the form {File["root/symbol/name/ *.wlt"], } or{File["root/symbol/name/ *.nb"], } .
  • TestFiles[{PacletObject[]}] resolves to a list of files of the form {File["root/ / *.wlt"], } or {File["root/ / *.nb"], } .
  • When TestFiles["spec"] returns a list of files of the form {File["root/ "], }, the root directory resolves to the "TestFiles" directory in the paclet extracted from spec.
  • The "TestFiles" root of a paclet is defined in its PacletInfo.wl file with the Extensions key, {"TestFiles","Root""TestFiles"}.
  • Whether test files with extensions .wlt or .nb or both are returned depends on the "TestFileExtension" option.
  • TestFiles[PacletObject[]] is equivalent to TestFiles[PacletObject[], Infinity] and lists all test files in all sub-directories of the paclet's "TestFiles" directory.
  • TestSummary and TestReport can be applied to the list of test files returned by TestFiles.
  • TestSummary[TestFiles["spec"]] can be abbreviated to TestSummary["spec"] when spec resolves to a test file directory.
  • The level specification lev takes the same form as that used in FileSystemMap, FileNames etc.
  • In TestFiles["symbol/dir1/dir2/"] more finally granulated categorizations of test files are also supported.
  • TestFiles["symbol/dir1/dir2/"] is equivalent to TestFiles[{"symbol","dir1","dir2",}] .
  • TestFiles[{paclet, "dir1/dir2/"}] is equivalent to TestFiles[{paclet, "dir1","dir2", }] .
  • The list of all implicit TestFiles directories defined by all paclets is returned by PacletManager`PacletResources["TestFiles"].
  • The following options can be given:
  • "TestFileExtension"".wlt"the test file extension to search for
    "ExcludedTestFiles"$ExcludedTestFilestest files excluded during search
  • Possible settings for "TestFileExtension" include:
  • ".wlt"search only for test files with extension .wlt
    ".nb"search only for test files with extension .nb
    Allsearch for test files with extension .wlt or .nb
    Automaticequivalent to the setting ".wlt"
  • Possible settings for "ExcludedTestFiles" include:
  • pattexcluded test files match the string pattern patt
    RegularExpression["regex"]excluded test files match the regular expression
    {patt1,patt2,}excluded test files match any of the patti
    Noneno excluded test files
  • $ExcludedTestFiles is set by default to "*FileSystemModify.*".

Examples

Basic Examples  (1)

Show all test files associated with TestFiles.

Show a test summary of all the tests associated with TestFiles.

Show a test summary restricted to TestFiles's core and documentation tests.

Show a test summary restricted to TestFiles's documentation and exception-handling tests.

A convenient syntax when applying TestSummary to the output of TestFiles can be obtained by feeding its file-specification directly to TestSummary.

By default, test files do not contain files that match the specification in $ExcludedTestFiles which by default specifies excluding files with names matching "*FileSystemModify*".

CreateDeveloperPaclet contains a test file with tests that create and delete paclets that modify the underlying filesystem and hence are all stored in a "FileSystemModify" test file. Identify this test file.

Tech Notes
  • Testing that Scales
  • Paclet Workflow
  • Using the Testing Framework