ePubs
The open archive for STFC research publications
Home
About ePubs
Content Policies
News
Help
Privacy/Cookies
Suggest an Enhancement
Contact ePubs
Full Record Details
Persistent URL
http://purl.org/net/epubs/work/40887
Record Status
Checked
Record Id
40887
Title
TolDiff: A regression testing tool for validating results from scientific codes
Contributors
HJJ van Dam (Science and Technology Facilities Council)
Abstract
The current practices in testing scientific and legacy codes are analysed. In particular problems with validating results in regression testing are highlighted. Commonly ad hoc solutions are implemented to deal with these problems. These solutions are typically code specific, require a non-trivial initial development effort and at least some ongoing maintenance, and even then some code errors that express themselves in the output may go undetected. To counter this unsatisfactory state of affairs an alternative approach is proposed that is based on the well known Diff program. The approach is derived from the realisation that a traditional Diff program would offer a good solution if previously identified insignificant differences could be suppressed and only potentially important differences reported. Such a tolerant diff approach requires a reference file that defines the ?gold standard? and a set of tolerances. It should support processes to diff a test result, to update the tolerances with newly discovered unimportant differences, and to transfer existing tolerances to new reference files. It is shown that such an approach can be implemented in such a way as to fully automate the diff, update, and transfer operations. It is shown that about 13 updates are needed to suppress all insignificant differences. The current implementation performs any of its operations in at worst 0.39 seconds on files of 3000 lines using a 2.8 GHz Pentium 4 machine, thus showing that practical application of this approach is feasible.
Organisation
CCLRC
,
CSE
,
CSE-CCG
Keywords
file differencing
,
regression testing
,
scientific computing
Funding Information
Related Research Object(s):
Licence Information:
Language
English (EN)
Type
Details
URI(s)
Local file(s)
Year
Journal Article
software-practice & experience (2007).
2007
Showing record 1 of 1
Recent Additions
Browse Organisations
Browse Journals/Series
Login to add & manage publications and access information for OA publishing
Username:
Password:
Useful Links
Chadwick & RAL Libraries
SHERPA FACT
SHERPA RoMEO
SHERPA JULIET
Journal Checker Tool
Google Scholar