Path: eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: Paul <nospam@needed.invalid>
Newsgroups: alt.comp.os.windows-10
Subject: Re: Saving searches in File Explorer.
Date: Fri, 06 Apr 2018 08:20:28 -0400
Organization: A noiseless patient Spider
Lines: 133
Message-ID: <pa7oma$91q$1@dont-email.me>
References: <3m70cd988oupe2trnbd00d5a8j0oegfb1k@4ax.com> <p9p973$flh$1@dont-email.me> <dok0cdpf62cnu42i4knog081tam7sp6a8s@4ax.com> <p9po0k$gtq$1@dont-email.me> <ek33cd5np4s68sop3pok9n9igkna85ou09@4ax.com> <p9sbj2$76k$1@dont-email.me> <5af5cd9cvbqfr1075vgts6sfqa0djisrlp@4ax.com> <p9uu2v$n1f$1@dont-email.me> <8356cdp9dgf4qu4lf0jqdfi2j3ka0ef18s@4ax.com> <pa4p74$3on$1@dont-email.me> <eendcd9t12q5chbcvfk92q74gk2kscsrf9@4ax.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Fri, 6 Apr 2018 12:20:26 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="8cf2fc9f7f695dc42894f3003c0f1433";
	logging-data="9274"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX18S/Y6czCtPvTeSnh4ANrUquM72nPKyv/M="
User-Agent: Ratcatcher/2.0.0.25 (Windows/20130802)
In-Reply-To: <eendcd9t12q5chbcvfk92q74gk2kscsrf9@4ax.com>
Cancel-Lock: sha1:9Csj6n5jM1OkPA/1oi8Md8XuCnE=

Peter Jason wrote:

> 
> Thanks Paul, I have stopped the shutdowns (tentatitavly) by turning
> off the CCTV cable to the main computer Video card.  I hav'nt the time
> just yet to play with ther prcmon (if the Promon screen shows several
> "Name not Found" in the Result column; may I delete/exclude the
> entries?

I got another method kinda working.

First, you need a copy of your Windows.edb, captured when
Windows 10 is shut down.

C:\ProgramData\Microsoft\Search\Data\Applications\Windows\Windows.edb

I booted up a Ubuntu 17.10 DVD, and installed esedb-utils.
(You could have done this from the Windows 10 Bash prompt,
except the package in there is two years old.)

https://s18.postimg.org/y6f6x022h/esedbexport_of_esedb-utils.gif

    esedbexport -m all Windows.edb

That creates close to 2GB of text files, for a 2GB Windows.edb.
It creates a folder tree, right next to the location of Windows.edb
you selected. So make sure you put a copy of Windows.edb
in a "scratch" location big enough so the folder with the
table dumps has enough room right next to the file.

Now that I know which files might be useful, a more efficient
version of the command might be crafted. But the first run,
that's a good way to get to see all the garbage in there.

*******

Back in Windows 10, we can work on our captured output folder.

Take, for example, the output file

    SystemIndex_Gthr.6

You take that file, and modify the extension to .txt

    SystemIndex_Gthr.6.txt

Then, open it in WordPad and "save" to put Windows line
endings on the file. (The Bash shell "unix2dos" can do
this too, but I like to look at the file first, before
conversion, to be safe.)

Now, you can launch the .txt file in Notepad any time you want.

This is a selected excerpt. Notice that it doesn't give
the whole path. Because it's a database, the person doing the
dump in this case, would have to do extra work to build a path
using db queries.

    SystemIndex_Gthr.6

    ScopeID  DocumentID  FileName
    47748    311561      MpKsl40ff2a5d.sys

And what "scope" means, is where the top of the file tree
was at the time. That means it was likely in "the 47748th folder"
doing that file. Looking up the ScopeID in one of the other
database tables, might give the path.

And the following file, has the folder tree. In the example here,
I've extracted scope and parent, so I can "walk up" the tree
and build a path. The 47748 folder has a parent of 19443.
The 19443 folder has a parent of 19313. And so on.

    SystemIndex_GthrPth.7

    Scope    Parent       Name
    47748    19443        {03250660-F6ED-45CE-A9DE-D9D1053A0922}/
    19443    19313        Definition Updates/
    19313    10           Windows Defender/
    10       9            Microsoft/
    9        5            ProgramData/
    5        4            [4de3b624-438b-455e-8975-aa06de823f68]/

Now I know item 311561 is located at:

ProgramData\Microsoft\Windows Defender\Definition Updates\{03250660-F6ED-45CE-A9DE-D9D1053A0922}\MpKsl40ff2a5d.sys

Where did the Gatherer go ?

File 6 and File 7 tell me. And the DocumentID
tells me what the "counter" on the SearchIndexer
control panel happened to be at the time.

That should give you some idea how to figure out
why your SearchIndexer gatherer count is so high.

Now, obviously, when you delete a hundred thousand files,
those "gatherer" references are no longer going to be valid.
So typically you would do this analysis right after the
SearchIndexer has built from scratch and it's stopped adding
things. Then you'd snapshot the Windows.edb (like I did for
this exercise) and work out the file list.

*******

The code that does that basic dumping, is located here.

https://github.com/libyal/libesedb/

The utility that does the exporting, is here.

https://github.com/libyal/libesedb/blob/master/esedbtools/esedbexport.c

Perhaps you could build a copy of that thing in MinGW
or Cygwin and have the solution live entirely in
Windows, but I don't need to try this for this quick
demo.

*******

I made a simple gawk script to convert the two text files
from the database dump, into regular file names.

https://pastebin.com/Fzzjv0np

The output of the gawk script looks like this.

0000311583	file:C:/[4de3b624-438b-455e-8975-aa06de823f68]/ProgramData/Microsoft/Windows Defender/Definition Updates/{03250660-F6ED-45CE-A9DE-D9D1053A0922}/mpavdlta.vdm
0000311584	file:C:/[4de3b624-438b-455e-8975-aa06de823f68]/ProgramData/Microsoft/Windows Defender/Definition Updates/{03250660-F6ED-45CE-A9DE-D9D1053A0922}/mpengine.dll
0000311561	file:C:/[4de3b624-438b-455e-8975-aa06de823f68]/ProgramData/Microsoft/Windows Defender/Definition Updates/{03250660-F6ED-45CE-A9DE-D9D1053A0922}/MpKsl40ff2a5d.sys

HTH,
    Paul