Academic Journal
Empirically revisiting the test independence assumption
| العنوان: | Empirically revisiting the test independence assumption |
|---|---|
| المؤلفون: | Sai Zhang, Darioush Jalali, Jochen Wuttke, Wing Lam, Michael D. Ernst, David Notkin |
| المساهمون: | The Pennsylvania State University CiteSeerX Archives |
| المصدر: | http://homes.cs.washington.edu/~mernst/pubs/test-independence-issta2014.pdf. |
| سنة النشر: | 2014 |
| المجموعة: | CiteSeerX |
| مصطلحات موضوعية: | Categories and Subject Descriptors, D.2.5 [Software Engineering, Testing and Debugging. General Terms, Reliability, Experimentation. Keywords, Test dependence, detection algorithms, empir- ical |
| الوصف: | In a test suite, all the test cases should be independent: no test should affect any other test’s result, and running the tests in any order should produce the same test results. Techniques such as test prioritization generally assume that the tests in a suite are independent. Test dependence is a little-studied phenomenon. This paper presents five results related to test dependence. First, we characterize the test dependence that arises in practice. We studied 96 real-world dependent tests from 5 issue tracking systems. Our study shows that test dependence can be hard for programmers to identify. It also shows that test dependence can cause non-trivial consequences, such as masking program faults and leading to spurious bug reports. Second, we formally define test dependence in terms of test suites as ordered sequences of tests along with explicit environments in which these tests are executed. We formulate the problem of detecting dependent tests and prove that a useful special case is NP-complete. Third, guided by the study of real-world dependent tests, we propose and compare four algorithms to detect dependent tests in a test suite. Fourth, we applied our dependent test detection algorithms to 4 real-world programs and found dependent tests in each human-written and automatically-generated test suite. Fifth, we empirically assessed the impact of dependent tests on five test prioritization techniques. Dependent tests affect the output of all five techniques; that is, the reordered suite fails even though the original suite did not. |
| نوع الوثيقة: | text |
| وصف الملف: | application/pdf |
| اللغة: | English |
| Relation: | http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.646.2532; http://homes.cs.washington.edu/~mernst/pubs/test-independence-issta2014.pdf |
| الاتاحة: | http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.646.2532 http://homes.cs.washington.edu/~mernst/pubs/test-independence-issta2014.pdf |
| Rights: | Metadata may be used without restrictions as long as the oai identifier remains attached to it. |
| رقم الانضمام: | edsbas.22DC1EE0 |
| قاعدة البيانات: | BASE |
| الوصف غير متاح. |