Automatically classifying test results by semi-supervised learning

Almaghairbe, Rafig and Roper, Marc; (2016) Automatically classifying test results by semi-supervised learning. In: 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE). IEEE, CAN, pp. 116-126. ISBN 978-1-4673-9003-3 (https://doi.org/10.1109/ISSRE.2016.22)

[thumbnail of Almaghairbe-Roper-ISSRE2016-Automatically-classifying-test-results-by-semi-supervised-learning]
Preview
 Text. Filename: Almaghairbe_Roper_ISSRE2016_Automatically_classifying_test_results_by_semi_supervised_learning.pdf
Accepted Author Manuscript
Download (189kB)| Preview

Abstract

A key component of software testing is deciding whether a test case has passed or failed: an expensive and error-prone manual activity. We present an approach to automatically classify passing and failing executions using semi-supervised learning on dynamic execution data (test inputs/outputs and execution traces). A small proportion of the test data is labelled as passing or failing and used in conjunction with the unlabelled data to build a classifier which labels the remaining outputs (classify them as passing or failing tests). A range of learning algorithms are investigated using several faulty versions of three systems along with varying types of data (inputs/outputs alone, or in combination with execution traces) and different labelling strategies (both failing and passing tests, and passing tests alone). The results show that in many cases labelling just a small proportion of the test cases – as low as 10% – is sufficient to build a classifier that is able to correctly categorise the large majority of the remaining test cases. This has important practical potential: when checking the test results from a system a developer need only examine a small proportion of these and use this information to train a learning algorithm to automatically classify the remainder.

ORCID iDs

Almaghairbe, Rafig ORCID logoORCID: https://orcid.org/0000-0002-8250-3909 and Roper, Marc ORCID logoORCID: https://orcid.org/0000-0001-6794-4637;
  • Item type:Book Section
    ID code:58652
    Dates:
    Date
    Event
    8 December 2016
    Published
    16 July 2016
    Accepted
    Notes:© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
    Subjects:Technology > Electrical engineering. Electronics Nuclear engineering
    Department:Faculty of Science > Computer and Information Sciences
    Depositing user: Pure Administrator
    Date deposited:15 Nov 2016 11:44
    Last modified:11 Nov 2024 15:08
    Related URLs:
    URI:https://strathprints.strath.ac.uk/id/eprint/58652
CORE (COnnecting REpositories)