Path: news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: Chris <ithinkiam@gmail.com>
Newsgroups: alt.comp.os.windows-11,comp.os.linux.advocacy
Subject: =?UTF-8?Q?Re:=20Windows-on-ARM=20Laptop=20Is=20A=20=E2=80=9CFrequently-Returned?=
 =?UTF-8?Q?=20Item=E2=80=9D=20On=20Amazon?=
Date: Tue, 25 Mar 2025 08:26:15 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 53
Message-ID: <vrtpb7$2u0fo$1@dont-email.me>
References: <vrnbks$rkfk$1@dont-email.me>
 <vrnqf9$18esr$1@dont-email.me>
 <WYRDP.1210933$_N6e.547203@fx17.iad>
 <a9rvtjtpp62h8hihlc3b9mmlbbf03nm885@4ax.com>
 <vrpmac$315f4$1@dont-email.me>
 <vrq9e1$3klvh$1@dont-email.me>
 <m4cc06FbfhrU5@mid.individual.net>
 <vrrsuk$15shc$1@dont-email.me>
 <vrs5t9$1dqfo$1@dont-email.me>
 <m4eeltFktmlU8@mid.individual.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 25 Mar 2025 09:26:16 +0100 (CET)
Injection-Info: dont-email.me; posting-host="c04ab532fdffa39f615f0ee295cf3770";
	logging-data="3080696"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX18uNvgMC+xXJDHaXGcW100K4g4T47ti9hc="
User-Agent: NewsTap/5.5 (iPhone/iPod Touch)
Cancel-Lock: sha1:AiASSMDLnqYReqs5U4TpaJ77BXM=
	sha1:RmnR+smFInYJgA+Aa6BRl6tWFSM=

rbowman <bowman@montana.com> wrote:
> On Mon, 24 Mar 2025 17:48:25 -0000 (UTC), Chris wrote:
> 
>> Not quite sure what exactly you mean by "inference", the latest Mac
>> Studio that can go up to 512GB RAM is certainly heading in the right
>> direction for local LLM training.
> 
> Inference is when new data is fed into an existing model to reach 
> conclusions. 

Yes, I know. Paul seems to have a different definition, however, as
inference does not need a 4MW supercomputer as he seems to be implying. I
have Ollama running completely locally on mac laptop with 32GB RAM. 

> This is a decent overview of creating and training a model.
> 
> https://www.nitorinfotech.com/blog/training-large-language-models-llms-
> techniques-and-best-practices/
> 
> I haven't looked at this series but there a many PyTorch or TensorFlow 
> tutorials online.
> 
> https://pytorch.org/tutorials/beginner/basics/intro.html
> 
> The original MNIST database was 60,000 pre-scaled images of hand written 
> digits. Fashion MNIST has 60,000 grayscale  clothing images and 10,000 
> test images. This isn't NLP and the datasets are laughingly small. If 
> you're lucky you have a Nvidia GPU. So far most packages use CUDA which is 
> a Nvidia thing. Some can be set to use the CPU. In either case you might 
> want to go for a cup of coffee while the epochs spool by minimizing the 
> loss function.

Yes, CUDA is the dominant interface, but not the only game in town. There
are other NPUs that can give nVidia a run for its money. 

> You do realize the players in the field are using banks of $40,000 Nvidia 
> GPUs and are trying to buy nuclear plants to power them? If you want to 
> talk about ChatGPT here are some numbers.

Sure, but there's a whole spectrum of needs for deep learning methods that
are far more modest and still very useful. 

Machine learning has been around since the 1960s and has had real world
uses for a lot of that time. 

> https://www.moomoo.com/community/feed/109834449715205
> https://www.forbes.com/sites/katharinabuchholz/2024/08/23/the-extreme-
> cost-of-training-ai-models/
> 
> Not coming to a desktop near you any time soon.

I created my first model in 2006/7 with no need for a GPU.