When I started Unit Testing, I was like most people at the beginning – I wanted to have full code coverage and test every line that I wrote in production code. But is this desirable, let alone achievable?
As a general rule you should aim for 80% Unit Test Code Coverage. Anything less would leave too much code untested. While you can have a higher coverage than 80%, a full Coverage of 100% is almost never possible.
But if we don’t test every line, how can we be sure that we don’t break anything through refactoring or adding new features? We’ll cover this in the following sections.
What Unit Test Coverage Do I Need in C?
Let’s start with defining what Unit Test Coverage (or Code Coverage) means. It is the percentage of lines in the source code that is called from at least one Unit Test. If a Unit Test executes functions from the production code, every executed line is counted as “covered”. The calculation therefore is rather simple:
(Lines of Code under Coverage / Lines Of Production Code) * 100 = Code Coverage.
When you begin a new project, and especially when you are doing Test Driven Development, your Unit Test Coverage will be very high, 95% and above. Unit Tests mostly cover the (business) logic of the production code and if you focus on this at the beginning, your Coverage is high.
UI and Database operations on the other hand are something that should not be tested with Unit Tests. They make the tests slow and sometimes need costly initialization. Also there are other types of tests that suit these parts of the software better (Integration Tests, Acceptance Tests, UI Tests, etc.).
So the more of this becomes part of your program, the less code coverage you will have. You could try to go higher than the average of 80%, but the added benefit would not justify the effort.
This is true not only for C but for all Programming Languages (at least the ones I know of). C is not different to, say, Python, when it comes to Unit Testing Code Coverage. Even if Python is totally different in approach and syntax you will strive for 80% there too.
How Do You Determine That The Test Coverage Is Enough?
One of the “Fathers” of Test Driven Development, Kent Beck, wrote in his Book “Test Driven Development: By Example” about how much testing one should do. There is a now famous quote that says:
Write tests until fear is transformed into boredom
Kent Beck, “Test Driven Development: By Example”
I pretty like this quote because it essentially states the point. There is no definite number of tests or test coverage that your program should have. If you, the programmer, are afraid to change a line of code because you are afraid to break something, then you clearly don’t have enough tests.
But if you, the programmer, just copy and paste the last test and just change tiny details in order to cover the very last border case (although you know it will work), then this could be described as boredom. You are not afraid anymore and therefore the tests are enough.
I know that this may a sound a little bit esoteric to you, but in the end Unit Testing is a lot about practice and experience. I remember testing way too much in the beginning, testing the wrong things (which essentially proved nothing in my code), then testing too little and being caught by a bug in production. I have been there, and still sometimes are.
Don’t worry about getting it right from the start. Just go and keep walking, you will soon develop a gut feeling about how much testing is enough.
How To Plan Unit Test Coverage?
It is important to write Unit Tests for the Business Logic and the Interfaces to other Parts of the Software like UI, Database, Input, etc. Therefore you shouldn’t aim for a specific Coverage, but instead focus on writing the tests.
You cannot really plan for Code Coverage because you don’t know up front how much of your code is testable in a proper manner. Unit Test Coverage is the result of careful and mindful testing. Programming methods like Test Driven Development (TDD) can help building the necessary discipline.
Tools for Measuring Unit Test Coverage in C
How do you know how much of your Code is under Coverage? There are tools that will find untested sections in your Code and calculate the overall Coverage for you. In this section you will find three of this tools, but these are not the only ones.
Please note that I have included Links to External Sites. Although I checked them conscientiously, I am not responsible for the content on these sites.
Bullseye
https://www.bullseye.com/product.html
This Code Coverage Tool has a Coverage Browser. It differentiates between function coverage and condition/decision coverage which is an interesting approach. You can request a free trial, after that you have to buy a license.
Squish Coco
https://doc.froglogic.com/squish-coco/latest/
Squish Coco also has a Coverage Browser. The main features are finding untested sections, finding redundant test (which can be eliminated), finding dead code (which is never executed), measuring execution time of tests and programs and calculation the optimal test execution for maximum test coverage.
You can also get a free trial here if you contact the company. They offer qualification-kits for several standards like DO-178C, ISO 26262 and more.
Testwell
http://www.testwell.fi/ctcdesc.html
The third tool presented here is Testwell. In contrast to the first two tools, this one is console only. You can determine your Code Coverage by typing specific commands and options. Do not underestimate its features because of the missing UI though.
As with the other ones you can request a free trial version on their site. They have several qualification kits as well (like the ones squish coco has).
To recap, achieving 100% Unit Test Code Coverage is not necessary or desirable. Aim for 80% and trust your gut feeling about ho much Unit Test Coverage is enough.
