Details
-
Type:
Bug
-
Status: Open
-
Priority:
Critical
-
Resolution: Unresolved
-
Affects Version/s: None
-
Fix Version/s: None
-
Component/s: Command Line Interface, Test suite
-
Labels:None
-
Epic Link:
Description
Our CLI test suite is not reproducible because arbitrary timeouts are used to decide if a particular CLI execution has succeeded or failed.
A CLI test should:
- launch Jalview
- monitor Jalview's progress processing the arguments
- verify any output as being correct
- fail if Jalview does not appear to progress within given performance expectations
TImeouts are necessary, but we have frequently had to increase these timeouts to account for people working on different machines. This impacts of the effectiveness of the test suite because it takes longer for the suite to run.
Timeouts should only be used when waiting for fine-grained events - not as an arbitrary sleep period that blocks the suite's progress.
A CLI test should:
- launch Jalview
- monitor Jalview's progress processing the arguments
- verify any output as being correct
- fail if Jalview does not appear to progress within given performance expectations
TImeouts are necessary, but we have frequently had to increase these timeouts to account for people working on different machines. This impacts of the effectiveness of the test suite because it takes longer for the suite to run.
Timeouts should only be used when waiting for fine-grained events - not as an arbitrary sleep period that blocks the suite's progress.