Path: ...!3.eu.feeder.erje.net!feeder.erje.net!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: Stefan Reuther <stefan.news@arcor.de>
Newsgroups: comp.arch.embedded
Subject: Re: Unit Testing: from theory to real case
Date: Fri, 30 Aug 2024 18:23:21 +0200
Lines: 91
Message-ID: <vat2mq.220.1@stefan.msgid.phost.de>
References: <vakb55$2sp38$2@dont-email.me>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Trace: individual.net ZZaSAMrBhGxJwhqaXg5z8wA3os0eJaWoTfA7VEy/AnqCVkRwis
Cancel-Lock: sha1:2UKDZ9t97ctfKgmeoQ6ZUnfg9Bs= sha256:6FYYgJjkHE9tdVhNSZMIwgRx1Se3WFFCuIQqmUxTyOs=
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.1 Hamster/2.1.0.1538
In-Reply-To: <vakb55$2sp38$2@dont-email.me>
Bytes: 4458

Am 27.08.2024 um 12:52 schrieb pozz:
> I know the importance of testing, but we have to admit that it increases
> the cost of software development a lot, at least at the beginning. Not
> always we have the possibility to invest this price.

Not investing at the start means: pay it later. In longer debugging
sessions, or more bugs. Or more effort when you start testing.

That's part of the reason why testing is perceived as a burden, not as a
help: you have this big pile of code and no idea where to start. It has
not been designed for testing, so you sit there: how the fuck am I going
to test this beast, simulate time, simulate file access, etc.?

> First of all, I have a great confusion in my mind about the subtle
> differences about mocks, stubs, fakes, dummies and so on. Anyway I think
> these names are not so important, so go on.
[...]
> The interface is very simple. I have some functions to initialize the
> configuration of an event (a simple C struct):
[...]
> I have a function that is called every second that triggers actions on
> occurrences:
> 
>   void calendar_task(void);

If I were to design this interface from scratch, with testing in mind,
it would look like:

    void calendar_task(struct Calender* p, time_t t);

One: the calendar should be an object, not global state. This means I
can create as many calendars as I want for testing. Making this possible
usually immediately improves the flexibility of your code, because you
can of course also create multiple calendars for production. New
feature: product now can support multiple users' calendars!

Two: do not have the function ask the system for the time. Instead, pass
in the time. So, no more need to fake the time as global state.

At least for me, it makes testing feel quite natural: each of the tests
I build is a tiny system setup. Not some Frankenstein monster with
replaced system calls and linker tricks.

(One more change I would most likely make to the interface: have it tell
me the time to next event, so I do not have to wake up every second;
only, when an event is due.)

> Now the problem is... which tests to write?

Depends on your requirements (black-box), and your implementation
(white-box).

A calendar with zero events.

A calendar with one event, per type.

A calendar with multiple events of same or different type.

Some border cases that you happen to find in your implementation (e.g. a
one-shot event that has long passed, or a repeated event with interval
"every 0 seconds").

> The combinations and possibilities are very high. calendar_init() can be
> called with only 1 event, with 2 events and so on. And the behaviour for
> these cases must be tested, because it should behaves well with 1 event,
> but not with 4 events.

Does this really make a difference with your implementation? Isn't it
just a for loop, with the cases "zero" and "one or more"? The insertion
logic probably distinguishes between "one" and "two or more".

> I'm confused. How to scientifically approach this testing problem? How
> to avoid the proliferation of tests?

Are you measuring coverage (e.g. lcov)? That should tell you your blind
spots.

> Which tests are really important and how to write them?

The simple every-day tests make sure you do not accidentally break
something simple.

The boundary case tests make sure that bringing your software to its
boundaries does not break it (e.g. "nobody can hack it").

I wouldn't say one is more important than the other. But both help me
sleep better.



  Stefan