Empirical studies on exploratory software testing

 |  Login

Show simple item record

dc.contributor Aalto-yliopisto fi
dc.contributor Aalto University en
dc.contributor.author Itkonen, Juha
dc.date.accessioned 2012-08-31T11:09:25Z
dc.date.available 2012-08-31T11:09:25Z
dc.date.issued 2011
dc.identifier.isbn 978-952-60-4339-5 (PDF)
dc.identifier.isbn 978-952-60-4338-8 (printed)
dc.identifier.issn 1799-4942
dc.identifier.uri https://aaltodoc.aalto.fi/handle/123456789/5054
dc.description.abstract Exploratory software testing (ET) is a practically relevant approach to software testing that lacks scientific knowledge. In ET, the tester's work is not based on predesigned and documented test cases. Instead, testing is guided by a higher-level plan or mission, and the testing work involves parallel test design, test execution, and learning. One of the distinct characteristics of ET is that the tester designs the tests during ET and uses information gained to design new and better tests continuously. The ET approach relies on testers' skills and experience. The main claimed benefits of ET are the tester's ability to apply personal knowledge and creativity during testing as well as effectiveness, efficiency, and agility in terms of adapting to changes and working with imperfect documentation. In this thesis, the ET approach has been studied using empirical research methods. Two case studies, one controlled experiment, and two field studies were performed to address three research goals: defining ET and understanding its applicability based on the literature; empirically investigating the benefits and shortcomings of ET; and providing empirically based results on how the ET approach is applied in practice. This research identifies different approaches to ET in industry and describes concrete testing practices. The role of the tester's personal knowledge is identified in the literature, and this research provides a detailed analysis of the application of personal knowledge in failure detection using ET. The main conclusions of this work are that ET can be as effective as test case-based approaches and even more efficient in certain contexts. The testers are capable of utilizing their personal knowledge in failure detection, and the role of personal knowledge is important in the ET approach. In addition, software testing in product organizations seems to involve multiple diverse organizational groups, and ET was found to be an applicable approach to engage domain experts in testing. The main implications of this thesis are introducing the exploratory testing approach to the research community and motivating its relevance by providing empirical studies in industry. In addition, the results of the effectiveness and efficiency of ET as well as the qualitative data on exploratory testing practices and the detailed analysis of knowledge in exploratory testing work are valuable for the research community. The main practical implications include presenting the benefits and applicability of the ET approach along with the potential shortcomings and providing empirical evidence regarding the benefits of ET. en
dc.format.extent Verkkokirja (2236 KB, 74 s.)
dc.format.mimetype application/pdf
dc.language.iso en en
dc.publisher Aalto University en
dc.relation.ispartofseries Aalto University publication series DOCTORAL DISSERTATIONS , 107/2011 en
dc.relation.haspart [Publication 1]: Juha Itkonen, Kristian Rautiainen, and Casper Lassenius. 2005. Toward an understanding of quality assurance in agile software development. International Journal of Agile Manufacturing, volume 8, number 2, pages 39-49. © 2005 International Society of Agile Manufacturing (ISAM). By permission. en
dc.relation.haspart [Publication 2]: Juha Itkonen and Kristian Rautiainen. 2005. Exploratory testing: A multiple case study. In: Proceedings of the 4th International Symposium on Empirical Software Engineering (ISESE 2005). Noosa Heads, Queensland, Australia. 17-18 November 2005. IEEE. Pages 84-93. ISBN 0-7803-9507-7. © 2005 Institute of Electrical and Electronics Engineers (IEEE). By permission. en
dc.relation.haspart [Publication 3]: Juha Itkonen, Mika V. Mäntylä, and Casper Lassenius. 2007. Defect detection efficiency: Test case based vs. exploratory testing. In: Proceedings of the First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007). Madrid, Spain. 20-21 September 2007. Los Alamitos, California, USA. IEEE Computer Society. Pages 61-70. ISBN 978-0-7695-2886-1. © 2007 Institute of Electrical and Electronics Engineers (IEEE). By permission. en
dc.relation.haspart [Publication 4]: Juha Itkonen, Mika V. Mäntylä, and Casper Lassenius. 2009. How do testers do it? An exploratory study on manual testing practices. In: Proceedings of the Third International Symposium on Empirical Software Engineering and Measurement (ESEM 2009). Lake Buena Vista, Florida, USA. 15-16 October 2009. IEEE. Pages 494-497. ISBN 978-1-4244-4842-5. © 2009 Institute of Electrical and Electronics Engineers (IEEE). By permission. en
dc.relation.haspart [Publication 5]: Juha Itkonen, Mika V. Mäntylä, and Casper Lassenius. The role of knowledge in failure detection during exploratory software testing. IEEE Transactions on Software Engineering, May 2011, 17 pages, submitted for publication. © 2011 by authors and © 2011 Institute of Electrical and Electronics Engineers (IEEE). By permission. en
dc.relation.haspart [Publication 6]: Mika V. Mäntylä, Juha Itkonen, and Joonas Iivonen. 2012. Who tested my software? Testing as an organizationally cross-cutting activity. Software Quality Journal, volume 20, number 1, pages 145-172. © 2011 by authors. en
dc.subject.other Computer science
dc.title Empirical studies on exploratory software testing en
dc.type G5 Artikkeliväitöskirja fi
dc.contributor.school Perustieteiden korkeakoulu fi
dc.contributor.department Tietotekniikan laitos fi
dc.contributor.department Department of Computer Science and Engineering en
dc.subject.keyword software testing en
dc.subject.keyword exploratory testing en
dc.subject.keyword defect detection en
dc.subject.keyword effectiveness en
dc.subject.keyword experience en
dc.subject.keyword domain knowledge en
dc.subject.keyword case study en
dc.subject.keyword controlled experiment en
dc.subject.keyword field observation en
dc.identifier.urn URN:ISBN:978-952-60-4339-5
dc.type.dcmitype text en
dc.type.ontasot Väitöskirja (artikkeli) fi
dc.type.ontasot Doctoral dissertation (article-based) en
dc.contributor.supervisor Lassenius, Casper, Prof.


Files in this item

This item appears in the following Collection(s)

Show simple item record

Search archive


Advanced Search

article-iconSubmit a publication

Browse

My Account