Michael's PhD Blog

After 3 years of hard work, writing, publishing and several discussions my dissertation “project” is finally ready! In April 2011 I’ve submitted my dissertation and in June 2011 I’ve successfully defended it In the next weeks a long blogpost about PhD Best Practices and How to use Social Networks & Cloud Computing for dissertation writing will follow. Stay tuned!

I have placed a list of the talks I have given in the last years under My Talks. Slides for each talk will come in short!

As you may know, my research work is focusing on the holistic view which is missing in nowadays (model-based) software testing. Complete abstract of my phd thesis can be found in About. But, let’s start with the simple question: What does holistic mean?

The word holistic comes from the Greek word holos, which means all, whole, entire and total. The general principle of so called holism was already summarized by Aristotle as “The whole is different from the sum of its parts”. There exist several different interpretations of holism in different domains. The one interesting for me is the holism in science, which was summarized in an Wikipedia article. The main idea is to analyze complex systems by decomposing it into several parts (which can also be systems) and establishing the relations between them. Only this kind of view on the whole landscape supports the identification of complex coherences.

What does this definition has to do with software engineering and especially software testing?

In software engineering we build very complex systems. We create specifications and designs of such systems. This is the constructive parts of software engineering. The analytical one is software testing, where those complex systems have to be validated (or even verified; for discussion about V&V look here). Testers have to obtain the high-level view on the whole system and have to analyze the relations between all parts of the complex system. They can gain this knowledge by analyzing specifications or by exploratively executing the system under test. Let’s take a look at the first case…

In my research work I concentrate on the field of model-driven development. Here models are used as specifications of the system. There are several advantages of using visual models rather than tons of natural language or even formal specifications. For detailed discussion look at the German article from Informatikspektrum. However, models depict the system which has to be build (and tested). In the world of the UML three so called viewpoints can be identified: structure, behaviour and interaction. Thinking back about model-based testing (as one possible MDD activity), we use the mentioned UML models for test generation. I call it reuse, because developers (i.e. business analysts) create those models and we as testers reuse them for testing purposes. There is a certain problem about the reuse of developer models which was discussed by Pretschner and Philips. But back to the main problem: While reusing developer models the usage of the mentioned viewpoints with all relations between them has to be guaranteed! This kind of reuse is called holistic!

Who uses holistic (model-based) testing?? I’ve found that each test designer doing his job has to think in the holistic kind of way. I’ve done several empirical observations at all kind of projects (small, middle and large-scale) at Capgemini CSD, which is a software engineering and IT consulting company. Further I’ve analyzed about 160 papers and books of the current literature. The term holistic (model-based) testing has been used by only two researchers worldwide. The first mention of holistic testing was done by Prof. Dr.-Ing. habil. Fevzi Belli in his paper called “A holistic view for finite-state modeling and testing of user interactions”. Belli defines holistic as testing not only the positive (and specified / modeled) behaviour of the system, but also the negative one. The second researcher is my minority. Different as Belli I define holistic model-based testing in relation to the conceptual viewpoints on (meta-)models which are used for test generation. This is way the Bellis holistic testing approach can be integrated in the more “high-level” holistic testing which I promote.

In my phd thesis I describe the need for holistic model-based testing and its influence on test case quality in more detail. I have performed some experiments and case studies which provide first evidence. Besides my work, further research on this topic is still needed. Especially more empirical observations in large-scale projects for longer time periods at different companies would be helpful. As I plan to finalize my dissertation in March, this could be a possible future research opportunity. If someones interested in it, I can provide a lot of details

Right before christmas I received an very interesting e-mail from a well-known MBT researcher, Prof. Atif Memon. He invited me to write a book chapter for one of the widely known Elsevier books, namely “Advances in Computers”. This was like the best christmas gift for a researcher Of course I accepted the invitation and will start writing on this chapter shortly. It is also a perfect continuation of my research / phd work. I plan to finalize my dissertation in March 2011. The book will appear early in 2012. Seems like my affinity to model-based testing will continue in the next years

First we couldn’t easily read our analysis model. We managed to create a simple meta-model for the analysis model by hand. There were plenty of errors while trying to read our model even with the created meta-model. Then we tried to use some automation tools to create such a meta-model (part of oaw). Still several errors prevented the framework to execute some model transformation rules. After several days we figured out that the Enterprise Architect cannot export correct xmi files. While there is no ECore model for the information which EA is exporting, we tried to create such one. This was a tremendous task and after several days of try&error we decided not to use the professional frameworks due to the export/import problem. Of course the same problem arises while trying to read or write the test model described with the UML Testing Profile. Afterwards we find out, that an ECore meta-model for UTP exists, we couldn’t manage to solve the missing ECore meta-model for EA.

That is why we finally implemented the model transformation frameworks with Java and Groovy. It isn’t a nice technical solution, but it works. I have to admit that I’m really disappointed with this xmi / ECore conformance problem. While writing my own master thesis I faced the same problem, and this was more than 2 years ago! Hope that somebody in the Eclipse community or at Sparx Systems solves it one day…

This is a short intermediate post about my dissertation status. Somewhere at the end of 2009 / beginning of 2010 I decided to start writing. Until now almost 4/7 chapters are written down and some of them finally reviewed. The “big” ones are the introduction, related work and solution chapter. Some parts of the foundations are also written down. It may sound like a good status, but it isn’t. There are still several questions which I have to think of, find related work and write it down. My deadline is March 2011 and therefore only 5 months have left. I hope to better manage my project work at Capgemini with the dissertation writing. Besides this, two papers are in work and I hope to get them accepted soon. Happy dissertating for all the phd students out there!

Recently my collegue Baris Güldali and I were invited to give a talk at the 1st international workshop on evolution in model-driven testing (emdt2010). Since in the last months we were strongly discussing the topic of agile software development together with model-based testing we decided to put our thoughts into a paper and present it there. The main idea was to relate the 12 principle of the agile manifesto together with some characteristics of model-based testing. The later one we based on current literature (e.g. The mbt taxonomy from Utting et al.) and our experiences with mbt. We show that several principles of the agile manifesto are fulfilled by mbt! Of course those are abstract investigations and for practical usage the instances, for example scrum and a lightweight mbt approach using UML should be considered. We show such a combination in a recent publication from our research group.
Our paper was published in the proceedings of the IWK conference which hosted the emdt2010 workshop. Here is the abstract:

Agile manifesto defines principles for a light-weight software development process aiming at an improved customer satisfaction. Automated testing plays an important role in fulfilling these principles, because it enables efficient execution of test scripts for checking the quality of delivered software. However, the implementation and the maintenance of the test scripts can be very tedious and error-prone. In order to deal with that, model-based testing extends the automated test execution by test design and test implementation. Thus, model-based testing can speed up the test automation and improve the maintenance of test scripts. Nevertheless, introducing model-based testing requires some initial and some continual efforts, like creating test models, buying or developing tools, etc. In this talk, we will discuss how model-based testing can support agile development without conflicting with the principles of agile manifesto.

Some time ago I was invited by Prof. Dr. Andreas Spillner (FH Bremen) to give a talk about “Testing in large-scale projects” (German: Testen in Großprojekten) at the annual meeting of the SIG Software testing belonging to ASQF. As my research project at Capgemini TS CSD (aka Capgemini sd&m) deals with the development of a corporate test methodology (especially for large-scale projects), it was a perfect occasion to present some of our results. I’ve managed to convince a “real-life” test manager from Capgemini to hold the talk with me. This way we could show the typical problems in such a context, the methodological solutions and their applicability in real-life projects. The slides from our talk (in German) will be published at the homepage of ASQF soon.

While preparing for the talk in Bremen I received an invitation to give the same talk at the ASQF Testing Day in Ludwigsburg (near Stuttgart). The event manager mentioned that the audience should be larger than in Bremen. I’m looking forward to present and discuss the typical testing problems of large-scale projects there!

Together with Melanie Wohnert we have written a paper about the main problems of agile testing in large-scale projects for the ICST2010 conference. Our paper was declined, but we had some very constructive comments. After rethinking the paper we corrected the paper and submitted it for the 30. GI TAV workshop. Our paper was accepted and because Melanie is working for Capgemini sd&m (co-organisation of the workshop), the PC decided to present is as a keynote.

The paper deals with the typical problems of agile testing which we have collected during some interviews with project managers and architects. Based on the identified problems we propose to use the most known agile test method: Test-Driven Development (TDD). Besides TDD it is very important to have a good understanding of what has to be tested in lower test levels like unit testing. Therefore we combine TDD with a solution called Test Aspect Design which solves several problems identified during the interviews. We will hold the talk in Munich at 17. June.

Recently we extended our paper about the tool evaluation for TDD within acceptance testing together with Marian Jureczko. We were invited to publish it in the Electrical Review, one of the oldest journals from the so called philadephia list. If accepted, then it would be my first international journal article. Here the abstract of our paper:

In the last years the software engineering community pays a strong interest in agile development methods. Those methods place software testing for example the Test-Driven Development method as an important task of the development process. Agile projects rely on good test automation tools. In this paper we evaluate five test automation tools for their usage in acceptance testing for web applications using Test-Driven Development.