Publication
A. Ramírez, J.R. Romero and C.L. Simons. “A Systematic Review of Interaction in Search-Based Software Engineering”. IEEE Transactions on Software Engineering, vol. 45(8), pp. 760-781. 2019.
Abstract
Search-Based Software Engineering (SBSE) has been successfully applied to automate a wide range of software development activities. Nevertheless, in those software engineering problems where human evaluation and preference are crucial, such insights have proved difficult to characterize in search, and solutions might not look natural when that is the expectation. In an attempt to address this, an increasing number of researchers have reported the incorporation of the ‘human-in-the-loop’ during search and interactive SBSE has attracted significant attention recently. However, reported results are fragmented over different development phases, and a great variety of novel interactive approaches and algorithmic techniques have emerged. To better integrate these results, we have performed a systematic literature review of interactive SBSE. From a total of 669 papers, 26 primary studies were identified. To enable their analysis, we formulated a classification scheme focused on four crucial aspects of interactive search, i.e. the problem formulation, search technique, interactive approach, and the empirical framework. Our intention is that the classification scheme affords a methodological approach for interactive SBSE. Lastly, as well as providing a detailed cross analysis, we identify and discuss some open issues and potential future trends for the research community.
Additional material
The following document describes in detail the ”systematic review protocol” defined for this review paper. A relevant part of this process refers to the adaptation of query strings to each specific search engine.
Review protocol (including search queries and results)
As a result of the queries defined by the protocol, their outcomes are listed in the following document in terms of a ”list of papers per database”.
List of returned papers per database
From the list of papers returned by the search engines, a first filtering process is perform to get the list of ”candidate papers”. Inclusion and exclusion criteria, as defined by the review protocol, are applied to this list to obtain the list of primary studies.
List of candidate papers, excluded and included papers
The following document describes the categories, and their respective subcategories, adopted for the classification of primary studies in the field of interactive SBSE. This guide is intended to be a separate contribution of the review, and could serve as a guideline in future research.
Guide for the classification of interactive search-based studies
The following document defines the ”data extraction form” used for the analysis of primary studies, and contains the information extracted after a peer-review process.