This was the first ever global software testing competition and the European qualification round was held online on June 13, 2014.
The Software Testing World Cup (STWC) is an event for testing practitioners to show off their skills and compete with other testing professionals. It brings the testing craft into the spotlight and gives the profession a competitive event on a global scale.
Last year a proof of concept of STWC was run with 16 teams. This year is the first full scale competition running on all six continents with a maximum of 250 teams per continent. In the European qualification 250 teams participated. From Ericsson Finland Kjell Lauren and Tomi Juslin participated together with Ismo Aro and Toni Kortepohja from Omenia Oy as Team Bug Circus.
We joined the competition in order to challenge ourselves by doing something completely different than what we are doing on daily basis, to find out how good testers we are, curiosity, fun, and of course fame & glory.
SOFTWARE UNDER TEST
A few minutes before the actual competition started the participants got the link to the software to be tested. It was a product that creates demo accounts and reports for small businesses. Our mission was to investigate and report on the current stability of the system. The latest addition to the product was a function that emails out invites to the demo account and to a customer center to view the demo accounts. The customers’ primary concern beyond the email function was to have the system tested on as many devices and screen sizes as possible, but having the Apple devices as primary focus. Additionally, a login account should not be able to see locations created by other login accounts. Load and performance testing were out of scope during the session. Also security testing had a low priority although this was one evaluation criteria in previous qualifications.
The competition time was in total three hours which did not leave too much room outside the test execution, bug reporting and writing the test report. It started with the judges and customer representative presenting the System Under Test (SUT) on the live YouTube channel. After the SUT presentation the judges were available for answering questions of the teams and keeping the teams informed with important information.
In the beginning we struggled to understand what the SUT was supposed to be used for and how it really should work. It took us quite some time to figure it out. The SUT was quite unusable and a specialized application, in our opinion not very adequate for a three hour testing competition. We were struggling with the basic features quite a long time and not focusing on the main features we were supposed to test. Even creating and managing accounts to test the mailing functionality was quite a challenge for some of us. Not to mention the user settings functionality which turned out to be quite buggy too.
During the competition the found bugs had to be filed in HP Agile Manager, in which some of the teams found a bug making it possible to copy reported bugs from other teams. Each team had to prepare a test report, including the status and how that was achieved as well as how the testing was performed. In total, our team filed 28 bug reports. During the competition a total of 3169 bugs from 250 teams was filed.
We also had an active supporting group of colleagues cheering for us throughout the whole competition; mostly from the sauna and palju (a hot tub). The results are not yet available; they will be announced by end of July 2014.
LESSONS LEARNED AKA AFTERMATH
Preparing for a short three hour competition is not that easy when you have no clue about the SUT or what is required to be tested. From the previous qualifications we noticed the SUT can be a web application or a downloadable application doing just about anything. Also thinking about load, performance and security testing it requires special tools to be able to execute those tests. One thing is for sure, the SUT used in this kind of competition should be at least on some stable level so that the teams could concentrate on testing the features customer requires, not struggling with the basic functionalities.
We had agreed beforehand that one of us will follow the live YouTube channel and one is responsible for the test report leaving two guys to concentrate fully on testing for the whole time. This split of work worked quite well. In the future it would be good to add a couple of short debriefing slots to be able to better focus and change strategy during the competition.
For testing technique we chose exploratory testing to cover the different areas of the application. It was the right choice as there was no time to create any automated tests. We used also a variety of platforms and browsers to gain coverage of the application and see how it worked on different devices with different screen resolutions.
It turned out that writing the test report took more than the 30 minutes we had reserved for it. One hour would have been needed to write a good report. One major input for the test report was the bug reports, which we categorized when writing them. This made writing the test report easier as the info was already available. At the end we didn’t have the time to review the written reports, what would have been needed to make them even better. So at least this is something we need to improve for the next time.
Based on the feedback and comments from the previous qualifications we decided to be co-located in the same room. This turned out to be a perfect choice and helped a lot as we were able to discuss and show each other’s screens all the time.
The overall feeling from the event was really positive. Three hours of full concentration is very exhausting but at the same time the excitement keeps you trying even harder. And YES, we will participate next time, too. In the end, it was about challenging us, sharing knowledge, learning new things, working and having fun with great people.
THE ROAD TO VICTORY
The Software Testing World Cup is split into two parts: the online continental preliminary/qualifying rounds and the world cup finals during Europe’s greatest Agile event of the year, the Agile Testing Days 2014. The winning team from each continent will be invited to the finals. Up to 8 teams, watched by an international crowd of testers, will fight for the global crown of software testing.
The continental rounds are done locally and online. The final round is done on the spot and will be supported with online and offline media presence (live streaming, audience, stage, etc.). It will be the social highlight at the Agile Testing Days on November 10, 2014, followed by an award ceremony during the popular MIATPP Award Night.
Further Information on Software Testing World Cup (STWC)