. : News | Screen shots | Bugs | TODO

Supported cards


UMTS info



Extra docs

mailing list

How to enable code coverage on the umtsmon module tests


The umtsmon project contains several module tests in the umtsmon/test directory. Of course, it would be interesting to see how much of the code actually gets covered by these module tests. This page explains how to use gcov (see its man page) to do code coverage analysis.


In order to enable code coverage logging, you need to compile all code again with specific extra compiler flags. These are already mentioned in test/GenericTestPro.inc but are outcommented in CVS:

### uncomment these two lines for compiling with code coverage instrumentation
#QMAKE_CXXFLAGS_DEBUG += -fprofile-arcs -ftest-coverage
#QMAKE_LFLAGS_DEBUG += -fprofile-arcs -ftest-coverage
Secondly, you need to modify the location of the object files to the current directory. This is needed because otherwise, gcov won't be able to figure out which graph files belong to which source files. In the same file, you have to change OBJECT_DIR = .obj to read . instead of .obj.


Now we need to run QT3's qmake and make to actually recompile all source code. Note that all required source files will be recompiled: the location changed, remember?
Anyway, we now have the final executable again, but instrumented. And the whole directory is filled with object files...


This is the easy part: just run the application as usual. It will create *.gcda and *.gcno files for each source file or include file. Of course, the execution speed drops significantly due to all the logging it needs to do.
Then we run gcov on the main source file to generate the reports. The following are some examples for files in different source directories:

   gcov NetworkChangerTest_main.cpp
   gcov ../../src/controller/NetworkChanger.cpp
   gcov ../../src/model/Query.cpp
   gcov ../../.ui/networkoperatorwizard.cpp
will create reports (the *.gcov files) for the relevant headers and source files which have a graph file in the current directory. These files contain ASCII text with the results


Let's look at a fragment of such file:

        8:  108:Query::ReturnCode Query::run()
        -:  109:{
        -:  110:        // safety net
        8:  111:        assert(theSerialPortPtr != NULL);
        8:  112:        if (theSerialPortPtr == NULL)
        -:  113:        {
    #####:  114:                return NO_DEVICE;
        -:  115:        }
        -:  116:
        -:  117:        // more safety net: lock the mutex
        8:  118:        assert(hasLock == false);
        8:  119:        if (theQueryMutex.tryLock()==FALSE)
        -:  120:        {
    #####:  121:                hasLock = false;
    #####:  122:                DEBUG5("Query instance %p trylock FAILED\n", this);
    #####:  123:                return NO_LOCK;
        -:  124:        }
As you can see, the output contains all of the source code, including line numbers. Any line that doesn't contain executable code starts with a '-', otherwise it will either contain a number (the line has been executed X times) or '####', indicating that the line never has been executed.

Viewing the above output, it shows that all special situations, like error conditions, have not been executed. That is usually unwanted behaviour - a good test case excerts both good and bad behavior. Browsing through results, you'll notice that not all lines have been executed. gcov actually tells how much lines were touched during the run:

File '../../src/base/SerialPort.cpp'
Lines executed:25.29% of 174        
../../src/base/SerialPort.cpp:creating 'SerialPort.cpp.gcov'
Note that a percentage below 80% is generally considered to be too low. Achieving 100% is rather difficult - and many obscure error situations may not warrant spending time to write test code for.

Advanced Topics

Of course, some files are exerted by several tests - a good example is the Query class, which is used by the majority of the test cases, and each test case touches diffent parts of the code. gcov by itself is capable to add data of multiple runs into the same graph files - allowing to run all tests and only then generate reports.
The easiest way to do this, is by copying all of the sources into a single new directory in the test directory and then qmake,make,run the first test, then qmake,make,run the second test, qmake,make,run the third test, etc. Once all runs are finished, you can then run gcov on the source file.

This way, running both NetworkOperatorWizardTester and NetworkChangerTester, you get a code coverage of 81.7% on NetworkChanger.cpp, 70.8% on networkoperatorwizard.ui.h and 97.3% of OperatorList.cpp. That's good.
Another interesting example is Query.cpp. After running the above two tests, 77.4% of the code of Query is touched. Adding the QueryTester then improves the coverage of Query to 78.3%.

UMTSmon (C) 2006,2007,2008 Klaas van Gend and others, licensed under the GPL v2 license hosted by SourceForge.net: Sponsored by NLNet Foundation:
  Get control/monitor your UMTS card at SourceForge.net. Fast, secure and Free Open Source software downloads NLNet Logo