Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: pozz <pozzugno@gmail.com>
Newsgroups: comp.arch.embedded
Subject: Re: Unit Testing: from theory to real case
Date: Fri, 30 Aug 2024 10:37:31 +0200
Organization: A noiseless patient Spider
Lines: 168
Message-ID: <vas0cb$ct06$2@dont-email.me>
References: <vakb55$2sp38$2@dont-email.me> <vaqnag$45ii$1@dont-email.me>
 <varv8e$ct05$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 30 Aug 2024 10:37:31 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="0525a567e37dc322fa080b3fd6cf6613";
	logging-data="422918"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/uqc+LQgcgAJVN/KUUAJazCM1cd7A0S1o="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:3j3C2QL+eazn9rfp7Y2+usdAk5E=
Content-Language: it
In-Reply-To: <varv8e$ct05$1@dont-email.me>
Bytes: 8972

Il 30/08/2024 10:18, pozz ha scritto:
> Il 29/08/2024 22:56, Don Y ha scritto:
>> On 8/27/2024 3:52 AM, pozz wrote:
>>> I read a lot about unit testing, but unfortunately I usually work on 
>>> single-developer projects with stressing time constraints, so I never 
>>> created full tests for an entire project in the past. This means I'm 
>>> a newbie in this aspect of software development.
>>
>> Fix it now or fix it later -- when you have even LESS time (because 
>> customers
>> are using your defective product)
>>
>> Testing should start when you define the module, continue while you are
>> implementing it (you will likely notice "conditions" that could lead to
>> bogus behavior as you are writing them!), and when you consider it 
>> "done".
>>
>> Thinking about testing when you draft the specification helps you
>> challenge your notion of the suitability of such a module for the task(s)
>> at hand as you imagine use cases (and MISuse cases).
>>
>>> I know the importance of testing, but we have to admit that it 
>>> increases the cost of software development a lot, at least at the 
>>> beginning. Not always we have the possibility to invest this price.
>>
>> If you assume there are two types of "software" -- stuff that TRIES to 
>> work
>> and stuff that HOPES to work, then the cost of the latter can be a lot 
>> less...
>> because you really don't care *if* it works!  Apples; Oranges.
>>
>>> These days I'm working on a calendar scheduler module. The client of 
>>> this module can configure up to N events that could be:
>>> - single (one shot)
>>> - weekly (for example, on Monday and Saturday of every weeks)
>>> - monthly (for example, the days 3-5-15 of every months)
>>> - yearly (for example, the day 7 of months Jan, Feb and Mar)
>>> Weekly, monthly and yearly events have a starting time and *could* 
>>> have a maximum number of repetitions (or they could be forever).
>>
>> Testing aims to prove that:
>> - your specification for the module accurately reflects its need 
>> (suitability)
>> - the module actually implements the specification (compliance)
>> - the module is well-behaved in "all" possible scenarios, even when 
>> misused
>> - changes to the module haven't compromised past performance (regression)
>>
>> It also gives you an idea of how your "process" is working; if you are
>> finding *lots* of bugs, perhaps you should be testing more aggressively
>> earlier in the process (there is a tendency to NOT want to make lots of
>> changes/fixes to code that you've convinced yourself is "almost done")
>>
>> And, it provides exemplars that you can use to evaluate performance.
>>
>>> The calendar module depends on some other modules. First of all, it 
>>> asks for the current time as time_t. It calls make_actions() 
>>> function, with certain parameters, when an event occurrence expired.
>>
>> Treat each as an independent, testable entity.  This makes it easier to
>> design test cases and easier to isolate anomalous behavior(s).
>>
>>> I'm confused. How to scientifically approach this testing problem? 
>>> How to avoid the proliferation of tests? Which tests are really 
>>> important and how to write them?
>>
>> Make a concerted effort thinking of how to *break* it.  E.g., If you 
>> try to
>> schedule an event for some time in the past, how should it react? 
>> Should it
>> immediately "trigger" the event?  Should it silently dismiss the event?
>> Should it throw an error?
>>
>> What if "the past" was just half a second ago and you've been unlucky
>> enough that your task was delayed a bit so that the clock ticked off
>> another second before you got a chance to schedule your event AHEAD of
>> time?
>>
>> If there are multiple steps to scheduling an event (e.g., creating a 
>> structure
>> and then passing it on to a scheduler), consider if one of the steps 
>> might
>> (intentionally!) be bypassed and how that might inject faulty behavior 
>> into
>> your design.  E.g., if you do all of your sanity checks in the "create
>> structure" step, BYPASSING that step and passing a structure created
>> by some other means (e.g., const data) avoids that sanity checking; will
>> the scheduler gag on possibly "insane" data introduced in such a manner?
>>
>> Can a client become confused as to which structures are "still active"
>> vs. "already consumed"?  If an active structure is altered, can that
>> lead to an inconsistent state (e.g., if the scheduler has acted on *part*
>> of the information but still relying on the balance to complete the 
>> action)?
>>
>> Can a client safely repurpose an event specification?  Or, once created,
>> does the scheduler "own" it?  Is there some safety window in which such
>> alterations won't "confuse" the scheduler, outside of which the scheduler
>> may have already taken some actions on the assumption that the event IS
>> still scheduled?
>>
>> What happens if someone changes the current *time*?  Do all events 
>> that are
>> now "past" instantly trigger?  Are they dismissed?  Do they move 
>> forward or
>> backwards in time based on the delta introduced to the current time?
>>
>> [This is a common flaw in folks trying to optimize such subsystems. 
>> There is
>> usually a need for relative events AND absolute events as an 
>> acknowledgement
>> that "time" changes]
>>
>> These interactions with the rest of the system (clients) can help you
>> think about the DESIRED functionality and the actual use patterns.  You
>> may discover your implementation strategy is inherently faulty 
>> rendering the
>> *specification* defective.
>>
> 
> Thank you for your reply, Don. They are valuable words that I read and 
> hear many times. However I'm in trouble to translate them into real 
> testing.
> 
> When you write, test for this, test for that, what happens if the client 
> uses the module in a wrong way, what happens when the system clock 
> changes a little or a big, and when the task missed the exact timestamp 
> of an event?
> 
> I was trying to write tests for *all* of those situations, but it seemed 
> to me a very, VERY, *VERY* big job. The implementation of the calendar 
> module took me a couple of days, tests seem an infinite job.
> 
> I have four types of events, for each test I should check the correct 
> behaviour for each type.
> 
> What happen if the timestamp of an event was already expired when it is 
> added to the system? I should write 4 tests, one for each type.
> 
> AddOneShotEventWithExpiredTimestamp_NoActions
> AddWeeklyEventWithExpiredTimestamp_NoActions
> AddMonthlyEventWithExpiredTimestamp_NoActions
> AddYearlyEventWithExpiredTimestamp_NoActions

That is true if the repeated events (weekly, monthly, yearly) have a 
limited number of repetitions *and* the system time is over the last 
repetition when they are added to the system.
Otherwise, even if the system time is over the event timestamp (first 
occurrence), other repetitions could happen in the future, so they 
should be added to the list of active events.

So there isn't just one test for the WeeklyEvent, but there are more:

WeeklyEventWith10Repetions_SystemTimeIsOverAllOfThem_IgnoreEvent
WeeklyEventWith10Repetions_SystemTimeIsOverOnlyTwoOfThem_AddEvent
WeeklyEventWithInfiniteRepetions_SystemTimeIsOverSomeOfThem_AddEvent

This for weekly, but also for monthly and yearly.

And what if the addition of events (the call to calendar_init) is made 
with multiple events and not only one? Maybe the module behaves 
correctly when just one event is added (expired or not), but behaves bad 
when two or more events are added.

Maybe it behaves well when all the events are completely expired, but 
bad when one is partially expired and on the totally expired.

The combinations that we could imagine are infinite.