34 lines
1.4 KiB
Plaintext
34 lines
1.4 KiB
Plaintext
Adding a regression test
|
|
========================
|
|
|
|
A regression test runs an entire system from start to finish (training and test),
|
|
generates a score, and compares against a hard-coded baseline. This is to ensure
|
|
that code changes don't adversely affect final system performance.
|
|
|
|
Each file in this directory (besides this one) that starts with the prefix
|
|
"test-" will be treated as a {shell script / perl script / something else
|
|
executable} regression test. To add a regression test, simply add such a file
|
|
to this directory.
|
|
|
|
The script:
|
|
- SHOULD run an entire training and test cycle on some fixed data, generate a
|
|
score, and compare that score against a fixed baseline (probably hard-coded
|
|
in the script itself.)
|
|
- MAY assume a correctly set up JavaNLP environment, including access to files
|
|
on NFS.
|
|
- SHOULD NOT take more than half an hour or so to run on a reasonable machine.
|
|
(So downsample your data until it fits in that limit!)
|
|
|
|
The final line of standard output of the script should either:
|
|
|
|
- start with "PASS", in which case the test is assumed to pass; OR
|
|
- start with "FAIL" followed by a one-line description of the error,
|
|
(e.g. "baseline F-measure 0.82 but only acheived 0.75")
|
|
|
|
Any other output will be considered an error. Successes, failures and errors
|
|
will be reported to java-nlp-list at some regular interval.
|
|
|
|
Don't forget to check in all your files!
|
|
|
|
-- wtm 10/17/2006
|