Path: ...!3.eu.feeder.erje.net!feeder.erje.net!news.mb-net.net!open-news-network.org!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: pozz <pozzugno@gmail.com>
Newsgroups: comp.arch.embedded
Subject: Re: Unit Testing: from theory to real case
Date: Fri, 30 Aug 2024 10:18:22 +0200
Organization: A noiseless patient Spider
Lines: 162
Message-ID: <varv8e$ct05$1@dont-email.me>
References: <vakb55$2sp38$2@dont-email.me> <vaqnag$45ii$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 30 Aug 2024 10:18:22 +0200 (CEST)
Injection-Info: dont-email.me; posting-host="0525a567e37dc322fa080b3fd6cf6613";
	logging-data="422917"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1/38OWhVUwdop3qfUP6yVtdZ2ke0xfTM1s="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:HHjXzzgAV5f1cvRm7aMa1MIS3wU=
Content-Language: it
In-Reply-To: <vaqnag$45ii$1@dont-email.me>
Bytes: 8986

Il 29/08/2024 22:56, Don Y ha scritto:
> On 8/27/2024 3:52 AM, pozz wrote:
>> I read a lot about unit testing, but unfortunately I usually work on 
>> single-developer projects with stressing time constraints, so I never 
>> created full tests for an entire project in the past. This means I'm a 
>> newbie in this aspect of software development.
> 
> Fix it now or fix it later -- when you have even LESS time (because 
> customers
> are using your defective product)
> 
> Testing should start when you define the module, continue while you are
> implementing it (you will likely notice "conditions" that could lead to
> bogus behavior as you are writing them!), and when you consider it "done".
> 
> Thinking about testing when you draft the specification helps you
> challenge your notion of the suitability of such a module for the task(s)
> at hand as you imagine use cases (and MISuse cases).
> 
>> I know the importance of testing, but we have to admit that it 
>> increases the cost of software development a lot, at least at the 
>> beginning. Not always we have the possibility to invest this price.
> 
> If you assume there are two types of "software" -- stuff that TRIES to work
> and stuff that HOPES to work, then the cost of the latter can be a lot 
> less...
> because you really don't care *if* it works!  Apples; Oranges.
> 
>> These days I'm working on a calendar scheduler module. The client of 
>> this module can configure up to N events that could be:
>> - single (one shot)
>> - weekly (for example, on Monday and Saturday of every weeks)
>> - monthly (for example, the days 3-5-15 of every months)
>> - yearly (for example, the day 7 of months Jan, Feb and Mar)
>> Weekly, monthly and yearly events have a starting time and *could* 
>> have a maximum number of repetitions (or they could be forever).
> 
> Testing aims to prove that:
> - your specification for the module accurately reflects its need 
> (suitability)
> - the module actually implements the specification (compliance)
> - the module is well-behaved in "all" possible scenarios, even when misused
> - changes to the module haven't compromised past performance (regression)
> 
> It also gives you an idea of how your "process" is working; if you are
> finding *lots* of bugs, perhaps you should be testing more aggressively
> earlier in the process (there is a tendency to NOT want to make lots of
> changes/fixes to code that you've convinced yourself is "almost done")
> 
> And, it provides exemplars that you can use to evaluate performance.
> 
>> The calendar module depends on some other modules. First of all, it 
>> asks for the current time as time_t. It calls make_actions() function, 
>> with certain parameters, when an event occurrence expired.
> 
> Treat each as an independent, testable entity.  This makes it easier to
> design test cases and easier to isolate anomalous behavior(s).
> 
>> I'm confused. How to scientifically approach this testing problem? How 
>> to avoid the proliferation of tests? Which tests are really important 
>> and how to write them?
> 
> Make a concerted effort thinking of how to *break* it.  E.g., If you try to
> schedule an event for some time in the past, how should it react?  
> Should it
> immediately "trigger" the event?  Should it silently dismiss the event?
> Should it throw an error?
> 
> What if "the past" was just half a second ago and you've been unlucky
> enough that your task was delayed a bit so that the clock ticked off
> another second before you got a chance to schedule your event AHEAD of
> time?
> 
> If there are multiple steps to scheduling an event (e.g., creating a 
> structure
> and then passing it on to a scheduler), consider if one of the steps might
> (intentionally!) be bypassed and how that might inject faulty behavior into
> your design.  E.g., if you do all of your sanity checks in the "create
> structure" step, BYPASSING that step and passing a structure created
> by some other means (e.g., const data) avoids that sanity checking; will
> the scheduler gag on possibly "insane" data introduced in such a manner?
> 
> Can a client become confused as to which structures are "still active"
> vs. "already consumed"?  If an active structure is altered, can that
> lead to an inconsistent state (e.g., if the scheduler has acted on *part*
> of the information but still relying on the balance to complete the 
> action)?
> 
> Can a client safely repurpose an event specification?  Or, once created,
> does the scheduler "own" it?  Is there some safety window in which such
> alterations won't "confuse" the scheduler, outside of which the scheduler
> may have already taken some actions on the assumption that the event IS
> still scheduled?
> 
> What happens if someone changes the current *time*?  Do all events that are
> now "past" instantly trigger?  Are they dismissed?  Do they move forward or
> backwards in time based on the delta introduced to the current time?
> 
> [This is a common flaw in folks trying to optimize such subsystems.  
> There is
> usually a need for relative events AND absolute events as an 
> acknowledgement
> that "time" changes]
> 
> These interactions with the rest of the system (clients) can help you
> think about the DESIRED functionality and the actual use patterns.  You
> may discover your implementation strategy is inherently faulty rendering 
> the
> *specification* defective.
> 

Thank you for your reply, Don. They are valuable words that I read and 
hear many times. However I'm in trouble to translate them into real testing.

When you write, test for this, test for that, what happens if the client 
uses the module in a wrong way, what happens when the system clock 
changes a little or a big, and when the task missed the exact timestamp 
of an event?

I was trying to write tests for *all* of those situations, but it seemed 
to me a very, VERY, *VERY* big job. The implementation of the calendar 
module took me a couple of days, tests seem an infinite job.

I have four types of events, for each test I should check the correct 
behaviour for each type.

What happen if the timestamp of an event was already expired when it is 
added to the system? I should write 4 tests, one for each type.

AddOneShotEventWithExpiredTimestamp_NoActions
AddWeeklyEventWithExpiredTimestamp_NoActions
AddMonthlyEventWithExpiredTimestamp_NoActions
AddYearlyEventWithExpiredTimestamp_NoActions

What does it mean "expired timestamp"? Suppose the event timestamp is 
"01/01/2024 10:00:00". This timestamp could be expired for a few 
seconds, a few minutes or one day or months or years. Maybe the module 
performs well when the system time has a different date, but bad if the 
timestamp expired in the same day, for example "01/01/2024 11:00:00" or 
"01/01/2024 10:00:01".
Should I add:

AddOneShotEventWithExpiredTimestamp1s_NoActions
AddOneShotEventWithExpiredTimestamp1m_NoActions
AddOneShotEventWithExpiredTimestamp1h_NoActions
AddOneShotEventWithExpiredTimestamp1d_NoActions
AddWeeklyEventWithExpiredTimestamp1s_NoActions
AddWeeklyEventWithExpiredTimestamp1m_NoActions
AddWeeklyEventWithExpiredTimestamp1h_NoActions
AddWeeklyEventWithExpiredTimestamp1d_NoActions
AddMonthlyEventWithExpiredTimestamp1s_NoActions
AddMonthlyEventWithExpiredTimestamp1m_NoActions
AddMonthlyEventWithExpiredTimestamp1h_NoActions
AddMonthlyEventWithExpiredTimestamp1d_NoActions
AddYearlyEventWithExpiredTimestamp1s_NoActions
AddYearlyEventWithExpiredTimestamp1m_NoActions
AddYearlyEventWithExpiredTimestamp1h_NoActions
AddYearlyEventWithExpiredTimestamp1d_NoActions

They are 16 tests for just a single stupid scenario. If I continue this 
way, I will thousands of tests. I don't think this is the way to make 
testing, do I?