Path: ...!3.eu.feeder.erje.net!feeder.erje.net!news.mb-net.net!open-news-network.org!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail From: pozz Newsgroups: comp.arch.embedded Subject: Re: Unit Testing: from theory to real case Date: Fri, 30 Aug 2024 10:18:22 +0200 Organization: A noiseless patient Spider Lines: 162 Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Date: Fri, 30 Aug 2024 10:18:22 +0200 (CEST) Injection-Info: dont-email.me; posting-host="0525a567e37dc322fa080b3fd6cf6613"; logging-data="422917"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/38OWhVUwdop3qfUP6yVtdZ2ke0xfTM1s=" User-Agent: Mozilla Thunderbird Cancel-Lock: sha1:HHjXzzgAV5f1cvRm7aMa1MIS3wU= Content-Language: it In-Reply-To: Bytes: 8986 Il 29/08/2024 22:56, Don Y ha scritto: > On 8/27/2024 3:52 AM, pozz wrote: >> I read a lot about unit testing, but unfortunately I usually work on >> single-developer projects with stressing time constraints, so I never >> created full tests for an entire project in the past. This means I'm a >> newbie in this aspect of software development. > > Fix it now or fix it later -- when you have even LESS time (because > customers > are using your defective product) > > Testing should start when you define the module, continue while you are > implementing it (you will likely notice "conditions" that could lead to > bogus behavior as you are writing them!), and when you consider it "done". > > Thinking about testing when you draft the specification helps you > challenge your notion of the suitability of such a module for the task(s) > at hand as you imagine use cases (and MISuse cases). > >> I know the importance of testing, but we have to admit that it >> increases the cost of software development a lot, at least at the >> beginning. Not always we have the possibility to invest this price. > > If you assume there are two types of "software" -- stuff that TRIES to work > and stuff that HOPES to work, then the cost of the latter can be a lot > less... > because you really don't care *if* it works!  Apples; Oranges. > >> These days I'm working on a calendar scheduler module. The client of >> this module can configure up to N events that could be: >> - single (one shot) >> - weekly (for example, on Monday and Saturday of every weeks) >> - monthly (for example, the days 3-5-15 of every months) >> - yearly (for example, the day 7 of months Jan, Feb and Mar) >> Weekly, monthly and yearly events have a starting time and *could* >> have a maximum number of repetitions (or they could be forever). > > Testing aims to prove that: > - your specification for the module accurately reflects its need > (suitability) > - the module actually implements the specification (compliance) > - the module is well-behaved in "all" possible scenarios, even when misused > - changes to the module haven't compromised past performance (regression) > > It also gives you an idea of how your "process" is working; if you are > finding *lots* of bugs, perhaps you should be testing more aggressively > earlier in the process (there is a tendency to NOT want to make lots of > changes/fixes to code that you've convinced yourself is "almost done") > > And, it provides exemplars that you can use to evaluate performance. > >> The calendar module depends on some other modules. First of all, it >> asks for the current time as time_t. It calls make_actions() function, >> with certain parameters, when an event occurrence expired. > > Treat each as an independent, testable entity.  This makes it easier to > design test cases and easier to isolate anomalous behavior(s). > >> I'm confused. How to scientifically approach this testing problem? How >> to avoid the proliferation of tests? Which tests are really important >> and how to write them? > > Make a concerted effort thinking of how to *break* it.  E.g., If you try to > schedule an event for some time in the past, how should it react? > Should it > immediately "trigger" the event?  Should it silently dismiss the event? > Should it throw an error? > > What if "the past" was just half a second ago and you've been unlucky > enough that your task was delayed a bit so that the clock ticked off > another second before you got a chance to schedule your event AHEAD of > time? > > If there are multiple steps to scheduling an event (e.g., creating a > structure > and then passing it on to a scheduler), consider if one of the steps might > (intentionally!) be bypassed and how that might inject faulty behavior into > your design.  E.g., if you do all of your sanity checks in the "create > structure" step, BYPASSING that step and passing a structure created > by some other means (e.g., const data) avoids that sanity checking; will > the scheduler gag on possibly "insane" data introduced in such a manner? > > Can a client become confused as to which structures are "still active" > vs. "already consumed"?  If an active structure is altered, can that > lead to an inconsistent state (e.g., if the scheduler has acted on *part* > of the information but still relying on the balance to complete the > action)? > > Can a client safely repurpose an event specification?  Or, once created, > does the scheduler "own" it?  Is there some safety window in which such > alterations won't "confuse" the scheduler, outside of which the scheduler > may have already taken some actions on the assumption that the event IS > still scheduled? > > What happens if someone changes the current *time*?  Do all events that are > now "past" instantly trigger?  Are they dismissed?  Do they move forward or > backwards in time based on the delta introduced to the current time? > > [This is a common flaw in folks trying to optimize such subsystems. > There is > usually a need for relative events AND absolute events as an > acknowledgement > that "time" changes] > > These interactions with the rest of the system (clients) can help you > think about the DESIRED functionality and the actual use patterns.  You > may discover your implementation strategy is inherently faulty rendering > the > *specification* defective. > Thank you for your reply, Don. They are valuable words that I read and hear many times. However I'm in trouble to translate them into real testing. When you write, test for this, test for that, what happens if the client uses the module in a wrong way, what happens when the system clock changes a little or a big, and when the task missed the exact timestamp of an event? I was trying to write tests for *all* of those situations, but it seemed to me a very, VERY, *VERY* big job. The implementation of the calendar module took me a couple of days, tests seem an infinite job. I have four types of events, for each test I should check the correct behaviour for each type. What happen if the timestamp of an event was already expired when it is added to the system? I should write 4 tests, one for each type. AddOneShotEventWithExpiredTimestamp_NoActions AddWeeklyEventWithExpiredTimestamp_NoActions AddMonthlyEventWithExpiredTimestamp_NoActions AddYearlyEventWithExpiredTimestamp_NoActions What does it mean "expired timestamp"? Suppose the event timestamp is "01/01/2024 10:00:00". This timestamp could be expired for a few seconds, a few minutes or one day or months or years. Maybe the module performs well when the system time has a different date, but bad if the timestamp expired in the same day, for example "01/01/2024 11:00:00" or "01/01/2024 10:00:01". Should I add: AddOneShotEventWithExpiredTimestamp1s_NoActions AddOneShotEventWithExpiredTimestamp1m_NoActions AddOneShotEventWithExpiredTimestamp1h_NoActions AddOneShotEventWithExpiredTimestamp1d_NoActions AddWeeklyEventWithExpiredTimestamp1s_NoActions AddWeeklyEventWithExpiredTimestamp1m_NoActions AddWeeklyEventWithExpiredTimestamp1h_NoActions AddWeeklyEventWithExpiredTimestamp1d_NoActions AddMonthlyEventWithExpiredTimestamp1s_NoActions AddMonthlyEventWithExpiredTimestamp1m_NoActions AddMonthlyEventWithExpiredTimestamp1h_NoActions AddMonthlyEventWithExpiredTimestamp1d_NoActions AddYearlyEventWithExpiredTimestamp1s_NoActions AddYearlyEventWithExpiredTimestamp1m_NoActions AddYearlyEventWithExpiredTimestamp1h_NoActions AddYearlyEventWithExpiredTimestamp1d_NoActions They are 16 tests for just a single stupid scenario. If I continue this way, I will thousands of tests. I don't think this is the way to make testing, do I?