Path: ...!weretis.net!feeder6.news.weretis.net!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!feeder.usenetexpress.com!tr1.iad1.usenetexpress.com!69.80.99.26.MISMATCH!local-2.nntp.ord.giganews.com!nntp.supernews.com!news.supernews.com.POSTED!not-for-mail NNTP-Posting-Date: Tue, 16 Apr 2024 15:17:45 +0000 From: John Larkin Newsgroups: sci.electronics.design Subject: Re: Predictive failures Date: Tue, 16 Apr 2024 08:16:04 -0700 Organization: Highland Tech Reply-To: xx@yy.com Message-ID: <5c4t1jt3er7a51macq5mnl8gfsuaipuv2b@4ax.com> References: <0s1r1jhb5vfe7lvopuvfk4ndkbt54ud3d9@4ax.com> X-Newsreader: Forte Agent 3.1/32.783 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Lines: 146 X-Trace: sv3-7Pe7FXM0LIjZo8Id0Pb6jVikoTV67wIpak6ctex3rWvC1CzE2lgnCADu0YJxn8ortez2p2LaEDXpETW!WXASGuLv9sINzCmBSvEpfoMeXD55S9cePA2vSY5U13cWQo4ALNaLhywE6+Vqfr5/Gca9jSLhUyux!aD7oTQ== X-Complaints-To: www.supernews.com/docs/abuse.html X-DMCA-Complaints-To: www.supernews.com/docs/dmca.html X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly X-Postfilter: 1.3.40 Bytes: 7104 On Tue, 16 Apr 2024 10:19:00 -0400, Joe Gwinn wrote: >On Mon, 15 Apr 2024 16:26:35 -0700, john larkin wrote: > >>On Mon, 15 Apr 2024 18:03:23 -0400, Joe Gwinn >>wrote: >> >>>On Mon, 15 Apr 2024 13:05:40 -0700, john larkin wrote: >>> >>>>On Mon, 15 Apr 2024 15:41:57 -0400, Joe Gwinn >>>>wrote: >>>> >>>>>On Mon, 15 Apr 2024 10:13:02 -0700, Don Y >>>>> wrote: >>>>> >>>>>>Is there a general rule of thumb for signalling the likelihood of >>>>>>an "imminent" (for some value of "imminent") hardware failure? >>>>>> >>>>>>I suspect most would involve *relative* changes that would be >>>>>>suggestive of changing conditions in the components (and not >>>>>>directly related to environmental influences). >>>>>> >>>>>>So, perhaps, a good strategy is to just "watch" everything and >>>>>>notice the sorts of changes you "typically" encounter in the hope >>>>>>that something of greater magnitude would be a harbinger... >>>>> >>>>>There is a standard approach that may work: Measure the level and >>>>>trend of very low frequency (around a tenth of a Hertz) flicker noise. >>>>>When connections (perhaps within a package) start to fail, the flicker >>>>>level rises. The actual frequency monitored isn't all that critical. >>>>> >>>>>Joe Gwinn >>>> >>>>Do connections "start to fail" ? >>> >>>Yes, they do, in things like vias. I went through a big drama where a >>>critical bit of radar logic circuitry would slowly go nuts. >>> >>>It turned out that the copper plating on the walls of the vias was >>>suffering from low-cycle fatigue during temperature cycling and slowly >>>breaking, one little crack at a time, until it went open. If you >>>measured the resistance to parts per million (6.5 digit DMM), sampling >>>at 1 Hz, you could see the 1/f noise at 0.1 Hz rising. It's useful to >>>also measure a copper line, and divide the via-chain resistance by the >>>no-via resistance, to correct for temperature changes. >> >>But nobody is going to monitor every via on a PCB, even if it were >>possible. > >It was not possible to test the vias on the failing logic board, but >we knew from metallurgical cut, polish, and inspect studies of failed >boards that it was the vias that were failing. > > >>One could instrument a PCB fab test board, I guess. But DC tests would >>be fine. > >What was being tested was a fab test board that had both the series >via chain path and the no-via path of roughly the same DC resistance, >set up so we could do 4-wire Kelvin resistance measurements of each >path independent of the other path. Yes, but the question was whether one could predict the failure of an operating electronic gadget. The answer is mostly NO. We had a visit from the quality team from a giant company that you have heard of. They wanted us to trend analyze all the power supplies on our boards and apply a complex algotithm to predict failures. It was total nonsense, basically predicting the future by zooming in on random noise with a big 1/f component, just like climate prediction. > > >>We have one board with over 4000 vias, but they are mostly in >>parallel. > >This can also be tested , but using a 6.5-digit DMM intended for >measuring very low resistance values. A change of one part in 4,000 >is huge to a 6.5-digit instrument. The conductivity will decline >linearly as vias fail one by one. > > Millikelvin temperature changes would make more signal than a failing via. >>>The solution was to redesign the vias, mainly to increase the critical >>>volume of copper. And modern SMD designs have less and less copper >>>volume. >>> >>>I bet precision resistors can also be measured this way. >>> >>> >>>>I don't think I've ever owned a piece of electronic equipment that >>>>warned me of an impending failure. >>> >>>Onset of smoke emission is a common sign. >>> >>> >>>>Cars do, for some failure modes, like low oil level. >>> >>>The industrial method for big stuff is accelerometers attached near >>>the bearings, and listen for excessive rotation-correlated (not >>>necessarily harmonic) noise. >> >>Big ships that I've worked on have a long propeller shaft in the shaft >>alley, a long tunnel where nobody often goes. They have magnetic shaft >>runout sensors and shaft bearing temperature monitors. >> >>They measure shaft torque and SHP too, from the shaft twist. > >Yep. And these kinds of things fail slowly. At first. They could repair a bearing at sea, given a heads-up about violent failure. A serious bearing failure on a single-screw machine means getting a seagoing tug. The main engine gearbox had padlocks on the covers. There was also a chem lab to analyze oil and water and such, looking for contaminamts that might suggest something going on. > > >>I liked hiding out in the shaft alley. It was private and cool, that >>giant shaft slowly rotating. > >Probably had a calming flowing water sound as well. Yes, cool and beautiful and serene after the heat and noise and vibration of the engine room. A quiet 32,000 horsepower. It was fun being an electronic guru on sea trials of a ship full of big hairy Popeye types. I, skinny gawky kid, got my own stateroom when other tech reps slept in cots in the hold. Have you noticed how many lumberjack types are afraid of electricity? That can be funny. > >Joe Gwinn