Path: ...!news.mixmin.net!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: pozz <pozzugno@gmail.com>
Newsgroups: comp.arch.embedded
Subject: Re: Unit Testing: from theory to real case
Date: Fri, 30 Aug 2024 18:00:39 +0200
Organization: A noiseless patient Spider
Lines: 269
Message-ID: <vasqb6$ct06$3@dont-email.me>
References: <vakb55$2sp38$2@dont-email.me> <vaqnag$45ii$1@dont-email.me>
 <varv8e$ct05$1@dont-email.me> <vasdg1$fl41$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 30 Aug 2024 18:00:39 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="0525a567e37dc322fa080b3fd6cf6613";
	logging-data="422918"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX199ksQwwIbe7czHKz1xCfBbthIm0V/0Td0="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:Yc37pteQWCooUuDDpHamn1VY6Cs=
In-Reply-To: <vasdg1$fl41$1@dont-email.me>
Content-Language: it
Bytes: 11244

Il 30/08/2024 14:21, Don Y ha scritto:
> On 8/30/2024 1:18 AM, pozz wrote:
>> When you write, test for this, test for that, what happens if the 
>> client uses the module in a wrong way, what happens when the system 
>> clock changes a little or a big, and when the task missed the exact 
>> timestamp of an event?
>>
>> I was trying to write tests for *all* of those situations, but it 
>> seemed to me a very, VERY, *VERY* big job. The implementation of the 
>> calendar module took me a couple of days, tests seem an infinite job.
> 
> Because there are lots of ways your code can fail.  You have to prove
> that it doesn't fail in ANY of those ways.

So you're confirming it's a very tedious and long job.


> uint multiply(multiplicand, multiplier) {
>      return(6)
> }
> 
> works well for the test cases:
>    2,3
>    3,2
>    6,1
>    1,6
> but not so well for:
>    8,5
>    17,902
>    1,1
> etc.

This is a simpler case. Just test for a couple of different result and 
you're almost sure it works for normal case (positive integers).


>> I have four types of events, for each test I should check the correct 
>> behaviour for each type.
>>
>> What happen if the timestamp of an event was already expired when it 
>> is added to the system? I should write 4 tests, one for each type.
>>
>> AddOneShotEventWithExpiredTimestamp_NoActions
>> AddWeeklyEventWithExpiredTimestamp_NoActions
>> AddMonthlyEventWithExpiredTimestamp_NoActions
>> AddYearlyEventWithExpiredTimestamp_NoActions
> 
> Chances are, there is one place in your code that is aware of the fact that
> the event is scheduled for a PAST time.  So, you only need to create one 
> test.
> (actually, two -- one that proves one behavior for time *almost* NOT 
> past and
> another for time JUST past)

I read that tests shouldn't be written for the specific implementation, 
but should be generic enough to work well even if the implementation 
changes.


> Your goal (having already implemented the modules) is to exercise each
> path through the code.
> 
> whatever() {
>     ...
>     if (x > y) {
>        // do something
>     } else {
>        // do something else
>     }
>     ...
> }
> 
> Here, there are only two different paths through the code:
> - one for x > y
> - one for !(x > y)
> So, you need to create test cases that will exercise each path.

Now I really know there are only two paths in the current 
implementation, but I'm not sure this will stay the same in the future. 
What happens if a developer changes the code in:

   if (event_type == SINGLE) {
     if (event_timestamp < current_time) {
       // Event expired, ignore it
     } else {
       add_event_to_active_events();
     }
   } else if (event_type == MONTHLY) {
     if (event_timestamp <= current_time) {
       ...
     } else {
       ...
     }
   } else ...

The paths could multiply.


> To verify your "x > y" test, you would want to pick an x that is
> just detectably larger than y.  And, another case where x is as
> large as possible WITHOUT exceeding y.  You can view this as defining
> the "edge" between the two routes.

Yes, and the test cases proliferate like ants... sigh :-(


> If, for example, you picked x = 5 and x = 3 as your test cases
> (where y = 4), then you WOULD exercise both paths.  But, if you had
> mistakenly coded this as
>     if (x >= y) {
>        // do something
>     } else {
>        // do something else
>     }
> you wouldn't be able to detect that fault, whereas using x = 5 and x = 4
> would cause you to wonder why "do something else" never got executed!
> 
>> What does it mean "expired timestamp"? Suppose the event timestamp is 
> 
> A time that is "in the past".  If it is time 't', now, what happens if the
> client specifies an event to happen at time t-1?  Should you *immediately*
> activate the event (because NOW it is t > t-1?)  Or, should you discard it
> because it was SUPPOSED to happen 1 second ago?
> 
> What if t-495678?  Is there a different type of action you expect if the
> time is "a long time ago" vs. "just recently"?
> 
> Do events happen at *instants* in time?  Or, in CONDITIONS of time?
> 
> If they happen at instants, then you have to ensure you can discern one
> instant from another.

Yes, yes, yes, please add tests, tests, tests.

My trouble isn't inventing new tests for normal and edge and corner 
cases, but limiting them to the real necessity, otherwise the software 
development stucks on tests.


>> "01/01/2024 10:00:00". This timestamp could be expired for a few 
>> seconds, a few minutes or one day or months or years. Maybe the module 
>> performs well when the system time has a different date, but bad if 
>> the timestamp expired in the same day, for example "01/01/2024 
>> 11:00:00" or "01/01/2024 10:00:01".
>> Should I add:
>>
>> AddOneShotEventWithExpiredTimestamp1s_NoActions
>> AddOneShotEventWithExpiredTimestamp1m_NoActions
>> AddOneShotEventWithExpiredTimestamp1h_NoActions
>> AddOneShotEventWithExpiredTimestamp1d_NoActions
>> AddWeeklyEventWithExpiredTimestamp1s_NoActions
>> AddWeeklyEventWithExpiredTimestamp1m_NoActions
>> AddWeeklyEventWithExpiredTimestamp1h_NoActions
>> AddWeeklyEventWithExpiredTimestamp1d_NoActions
>> AddMonthlyEventWithExpiredTimestamp1s_NoActions
>> AddMonthlyEventWithExpiredTimestamp1m_NoActions
>> AddMonthlyEventWithExpiredTimestamp1h_NoActions
>> AddMonthlyEventWithExpiredTimestamp1d_NoActions
>> AddYearlyEventWithExpiredTimestamp1s_NoActions
>> AddYearlyEventWithExpiredTimestamp1m_NoActions
>> AddYearlyEventWithExpiredTimestamp1h_NoActions
>> AddYearlyEventWithExpiredTimestamp1d_NoActions
>>
>> They are 16 tests for just a single stupid scenario. If I continue 
>> this way, I will thousands of tests. I don't think this is the way to 
>> make testing, do I?
> 
> You declare what scenario you are testing for as a (commentary) preface
> to the test stanza.
> 
> If you are testing to ensure "NoActions" is handled correctly, then
> you look to see how many ways the "NoActions" criteria can tickle
> the code.
> 
> If there is only ONE place where "NoActions" alters the flow through
> the code, then you only need one test (actually, two as you need
> to cover "SomeAction" to show that "NoAction" is different).
> 
========== REMAINDER OF ARTICLE TRUNCATED ==========