antek's tech blog - testingZola2015-01-12T00:00:00+00:00https://anadoxin.org/blog/tags/testing/atom.xmlType-parametrized unit testing in GTest2015-01-12T00:00:00+00:002015-01-12T00:00:00+00:00Unknownhttps://anadoxin.org/blog/typeparametrized-unit-testing-in-gtest.html/<p>Google's testing framework, GTest, goes beyond writing basic unit tests. It has
some more advanced functionality that allows the user to cover more code with
less tests, which -- considering the fact that unit tests always make a
significant share of the source code ratio in a project -- is always a good
thing.</p>
<p>Test parametrization reduces the need to duplicate code inside tests. It's
really best suited for testing interfaces and underlying implementations of
these interfaces.</p>
<p>Consider a following example. We have a few container classes inside our
application. To keep things simple let's use <code>std</code> containers to implement our
own. So, we will build three containers:</p>
<pre data-lang="text" class="language-text "><code class="language-text" data-lang="text">VectorContainer, based on `std::vector`,
ListContainer, based on `std::list`,
QueueContainer, based on `std::deque`.
</code></pre>
<p>As for generalization, let's create a simple interface <code>IContainer</code> which will
define all the functionality needed to form a simple container for our needs.
We'll use only a few methods:</p>
<ol>
<li>Adding to the container by the <code>add()</code> method,</li>
<li>Iterating through the container by the <code>forEach()</code> method,</li>
<li>Reporting the size of the container by the <code>count()</code> method.</li>
</ol>
<p>Here's our interface:</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">#pragma once
#include <functional>
typedef std::function<void (const size_t num)> IteratorFuncT;
class IContainer {
public:
virtual void add(size_t num) = 0;
virtual size_t count() const = 0;
virtual void forEach(IteratorFuncT closure) const = 0;
};
</code></pre>
<p>The interface will allow the user of our container to add some files into it,
ask about the number of elements currently stored inside, and iterate through
all added elements. Pretty basic stuff.</p>
<p>Let's create our first implementation, based on <code>std::vector</code>. Please note that
"pure" TDD philosophy advocates writing tests <em>before</em> the implementation, but
since we're not producing production code here, we'll try to get away with Test
Aware Development, instead of Test Driven Development ;).</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">// VectorContainer.h
#pragma once
#include "IContainer.h"
#include <vector>
class VectorContainer: public IContainer {
private:
std::vector<size_t> items;
public:
void add(size_t num) override {
items.push_back(num);
}
size_t count() const override {
return items.size();
}
void forEach(IteratorFuncT closure) const override {
for(const size_t item: items) {
closure(item);
}
}
};
</code></pre>
<p>OK, since we have our first implementation, it's possible to write some tests.</p>
<p>One way would be to create multiple tests tailored for <code>VectorContainer</code>, and
later copy and paste them to another file, while performing refactoring of the
<code>VectorContainer</code> type to <code>ListContainer</code>, like this:</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">// Test.cpp
#include <gtest/gtest.h>
#include "VectorContainer.h"
TEST(ContainerTest, ShouldBeEmptyOnStartup) {
VectorContainer container;
ASSERT_EQ(container.count(), 0);
}
TEST(ContainerTest, ShouldAdd1) {
VectorContainer container;
container.add(1);
ASSERT_EQ(container.count(), 1);
}
</code></pre>
<p>But, of course, we can do better.</p>
<p>We can define a generic test fixture first. Then, we can bind multiple types to
that fixture, so that when it will finally be instantiated by the test runner,
the runner will create multiple instances, each instance for each declared type
of the test fixture. Then, each test for each fixture will be invoked and
executed, testing all of the types that we declared.</p>
<p>In other words, we can bind multiple types to a test class. Later, we can bind
multiple tests to the same class. When we do both things, each test will be
executed multiple times, depending on how many types we declared.</p>
<p>So, scratch your current <code>Test.cpp</code>, and let's start again. First, usual set of
preprocessor includes:</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">#include <gtest/gtest.h>
#include "VectorContainer.h"
</code></pre>
<p>Then, we need to create our fixture class. This class will be the central point
of our test collection.</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">template <typename T>
class ContainerTest: public ::testing::Test { };
TYPED_TEST_CASE_P(ContainerTest);
</code></pre>
<p>Our fixture class should be a subclass of <code>::testing::Test</code> class, which is a
part of GTest framework. For our current purposes it can be empty, that's why it
has no body, not even a constructor nor destructor.</p>
<p>Then, the <code>TYPED_TEST_CASE_P</code> macro initializes the code needed to turn this
fixture class into a parametrized one.</p>
<p>Now we can write our tests. But let's skip that for a minute and do a
different thing: let's specify what types the framework should instantiate in
order to test the interface. This is done by the following code:</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">// placeholder for test bodies
typedef ::testing::Types<VectorContainer> ContainerTypes;
INSTANTIATE_TYPED_TEST_CASE_P(ContainerTypesInstantiation, ContainerTest, ContainerTypes);
</code></pre>
<p>First we need to create a typedef of the <code>::testing::Types</code> class. I will name
this new type <code>ContainerTypes</code>, and the type of it will be specified as:</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp"> ::testing::Types<VectorContainer>
</code></pre>
<p>So, we will specify here that we want to instantiate the <code>VectorContainer</code> class
inside our parametrized tests, that we'll write in a minute or two.</p>
<p>After the typedef has been created, it needs to be bound to our fixture class by
using the <code>INSTANTIATE_TYPED_TEST_CASE_P</code> macro:</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp"> INSTANTIATE_TYPED_TEST_CASE_P(PlaceholderName, OurFixtureClass, OurTypedef);
</code></pre>
<p><code>PlaceholderName</code> is a name of some kind. To be completely honest, I have no
idea what is its purpose, it seems that it can be anything.</p>
<p><code>OurFixtureClass</code> is a name of our fixture class: <code>ContainerTest</code>.</p>
<p><code>OurTypedef</code> is a typedef we have just created, that points to the types we
would like to be instantiated inside test cases.</p>
<p>Now, we can write some unit tests, but instead of specifying <code>TEST</code> macro, we
need to use <code>TYPED_TEST_P</code>. <strong>Note:</strong> Remember to put those tests <em>before</em> the
typedef we just declared, not <em>after</em>. See the "placeholder for test bodies"
comment above -- that's the spot you need to put your tests into.</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">TYPED_TEST_P(ContainerTest, ShouldBeEmptyOnStartup) {
TypeParam container;
ASSERT_EQ(container.count(), 0);
}
TYPED_TEST_P(ContainerTest, ShouldAdd1) {
TypeParam container;
container.add(1);
ASSERT_EQ(container.count(), 1);
}
REGISTER_TYPED_TEST_CASE_P(ContainerTest,
ShouldBeEmptyOnStartup,
ShouldAdd1
);
</code></pre>
<p>Inside each test I use <code>TypeParam</code> to specify the type of the container to be
tested. In runtime, it will point to a concrete type, specified in the
<code>::testing::Types<></code> typedef that was created earlier.</p>
<p>Also, have you seen the last 4 lines? Each test should be included inside the
<code>REGISTER_TYPED_TEST_CASE_P</code> macro. Without it, the test runner will abort in
runtime reminding you about any test cases that aren't specified here.</p>
<p>Let's compile whole project and see if it works:</p>
<pre data-lang="text" class="language-text "><code class="language-text" data-lang="text">g++ Test.cpp -o Test -lgtest -lgtest_main -std=c++11
</code></pre>
<p>Linking <code>gtest</code> along with <code>gtest_main</code> will allow you to skip defining the <code>main()</code>
function. <code>gtest_main</code> has its own <code>main()</code>.</p>
<pre data-lang="text" class="language-text "><code class="language-text" data-lang="text">$ ./Test
Running main() from gtest_main.cc
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from ContainerTypesInstantiation/ContainerTest/0, where TypeParam = VectorContainer
[ RUN ] ContainerTypesInstantiation/ContainerTest/0.ShouldBeEmptyOnStartup
[ OK ] ContainerTypesInstantiation/ContainerTest/0.ShouldBeEmptyOnStartup (0 ms)
[ RUN ] ContainerTypesInstantiation/ContainerTest/0.ShouldAdd1
[ OK ] ContainerTypesInstantiation/ContainerTest/0.ShouldAdd1 (0 ms)
[----------] 2 tests from ContainerTypesInstantiation/ContainerTest/0 (0 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (0 ms total)
[ PASSED ] 2 tests.
</code></pre>
<p>Well, nothing extraordinary here, the test runner has executed two tests that we
have defined. So where is the justification for all those macros that we had to
use?</p>
<p>It will be more clear when we'll create another class implementing the
<code>IContainer</code> interface, <code>ListContainer</code>. Here's the code:</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">// ListContainer.h
#pragma once
#include "IContainer.h"
#include <list>
class ListContainer: public IContainer {
private:
std::list<size_t> items;
public:
void add(size_t num) override {
items.push_back(num);
}
size_t count() const override {
return items.size();
}
void forEach(IteratorFuncT closure) const override {
for(const size_t item: items) {
closure(item);
}
}
};
</code></pre>
<p>It looks like the previous class, but this one uses <code>std::list</code>, so it's
different.</p>
<p>Now what do we do to test this class? We modify our <code>ContainerTypes</code> typedef by
adding a new type:</p>
<pre data-lang="c++" class="language-c++ "><code class="language-c++" data-lang="c++">typedef ::testing::Types<VectorContainer, ListContainer> ContainerTypes;
</code></pre>
<p>And we recompile the project. After we run the test runner, it will
automatically call the test cases for our new interface:</p>
<pre data-lang="text" class="language-text "><code class="language-text" data-lang="text">$ ./Test
Running main() from gtest_main.cc
[==========] Running 4 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 2 tests from ContainerTypesInstantiation/ContainerTest/0, where TypeParam = VectorContainer
[ RUN ] ContainerTypesInstantiation/ContainerTest/0.ShouldBeEmptyOnStartup
[ OK ] ContainerTypesInstantiation/ContainerTest/0.ShouldBeEmptyOnStartup (0 ms)
[ RUN ] ContainerTypesInstantiation/ContainerTest/0.ShouldAdd1
[ OK ] ContainerTypesInstantiation/ContainerTest/0.ShouldAdd1 (0 ms)
[----------] 2 tests from ContainerTypesInstantiation/ContainerTest/0 (0 ms total)
[----------] 2 tests from ContainerTypesInstantiation/ContainerTest/1, where TypeParam = ListContainer
[ RUN ] ContainerTypesInstantiation/ContainerTest/1.ShouldBeEmptyOnStartup
[ OK ] ContainerTypesInstantiation/ContainerTest/1.ShouldBeEmptyOnStartup (0 ms)
[ RUN ] ContainerTypesInstantiation/ContainerTest/1.ShouldAdd1
[ OK ] ContainerTypesInstantiation/ContainerTest/1.ShouldAdd1 (0 ms)
[----------] 2 tests from ContainerTypesInstantiation/ContainerTest/1 (0 ms total)
[----------] Global test environment tear-down
[==========] 4 tests from 2 test cases ran. (0 ms total)
[ PASSED ] 4 tests.
</code></pre>
<p>Much better than copy, paste and refactor!</p>
<p>We can create another test just to see that it's really working:</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">TYPED_TEST_P(ContainerTest, ShouldIterate) {
TypeParam container;
container.add(1);
container.add(2);
container.add(3);
std::vector<size_t> verification;
auto proc = [& verification, & container] (const size_t item) -> void {
verification.push_back(item);
};
container.forEach(proc);
ASSERT_EQ(verification.size(), 3);
ASSERT_EQ(verification[0], 1);
ASSERT_EQ(verification[1], 2);
ASSERT_EQ(verification[2], 3);
}
</code></pre>
<p>We compile the project and run it:</p>
<pre><code>$ g++ Test.cpp -o Test -lgtest_main -lgtest -std=c++11 && ./Test
Test.cpp:41: You forgot to list test ShouldIterate.
[1] 12001 abort (core dumped) ./Test
</code></pre>
<p>Yes, I forgot to include <code>ShouldIterate</code> in <code>REGISTER_TYPED_TEST_CASE_P</code>...</p>
<pre data-lang="cpp" class="language-cpp "><code class="language-cpp" data-lang="cpp">REGISTER_TYPED_TEST_CASE_P(ContainerTest,
ShouldBeEmptyOnStartup,
ShouldAdd1,
ShouldIterate
);
</code></pre>
<p>And the test has been automatically bound to both types.</p>
<pre data-lang="text" class="language-text "><code class="language-text" data-lang="text">$ g++ Test.cpp -o Test -lgtest_main -lgtest -std=c++11 && ./Test
Running main() from gtest_main.cc
[==========] Running 6 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from ContainerTypesInstantiation/ContainerTest/0, where TypeParam = VectorContainer
[ RUN ] ContainerTypesInstantiation/ContainerTest/0.ShouldBeEmptyOnStartup
[ OK ] ContainerTypesInstantiation/ContainerTest/0.ShouldBeEmptyOnStartup (0 ms)
[ RUN ] ContainerTypesInstantiation/ContainerTest/0.ShouldAdd1
[ OK ] ContainerTypesInstantiation/ContainerTest/0.ShouldAdd1 (0 ms)
[ RUN ] ContainerTypesInstantiation/ContainerTest/0.ShouldIterate
[ OK ] ContainerTypesInstantiation/ContainerTest/0.ShouldIterate (0 ms)
[----------] 3 tests from ContainerTypesInstantiation/ContainerTest/0 (0 ms total)
[----------] 3 tests from ContainerTypesInstantiation/ContainerTest/1, where TypeParam = ListContainer
[ RUN ] ContainerTypesInstantiation/ContainerTest/1.ShouldBeEmptyOnStartup
[ OK ] ContainerTypesInstantiation/ContainerTest/1.ShouldBeEmptyOnStartup (0 ms)
[ RUN ] ContainerTypesInstantiation/ContainerTest/1.ShouldAdd1
[ OK ] ContainerTypesInstantiation/ContainerTest/1.ShouldAdd1 (0 ms)
[ RUN ] ContainerTypesInstantiation/ContainerTest/1.ShouldIterate
[ OK ] ContainerTypesInstantiation/ContainerTest/1.ShouldIterate (0 ms)
[----------] 3 tests from ContainerTypesInstantiation/ContainerTest/1 (0 ms total)
[----------] Global test environment tear-down
[==========] 6 tests from 2 test cases ran. (0 ms total)
[ PASSED ] 6 tests.
</code></pre>
<p>Now you can create the third type, <code>QueueContainer</code>, based on <code>std::deque</code>, and
see how easy it is to create new parametrized tests!</p>