CHKDSK, ObjectID, etc. - the real cause of your problems Options

[Autumn Breeze]

I’ll bet that none of the people that are having CHKDSK, ObjectID and other problems are using Diskeeper Pro or even better, Diskeeper Pro Premier or a similiar product. Or if they are, they haven't used it properly.

My laptop has 3 partitions on its internal HDD and I have 2 external drives – 350GB and 500GB with a huge number of files and I’ve only seen a slight increase in CHKDSK Step 2 time, no problems with any software including True Image and Retrospect Pro. I’m a developer and have a bunch of apps.

The defrag tool that comes with Windows is a lite version of Diskeeper. Unlike the Pro versions, it can’t resize the NTFS MFT which Windows creates much too small. The first pgm I install after a new Win install is Diskeeper Pro Premier and the first thing I do is increase the size of the MFT (Conifg Properties>Frag Shield) then do a defrag. I try to never let my MFT get over 50% used, and usually much less.

For those that don’t know, from Microsoft “The NTFS file system contains at its core, a file called the master file table (MFT). There is at least one entry in the MFT for every file on an NTFS volume, including the MFT itself. Because utilities that defragment NTFS volumes cannot move MFT entries, and because excessive fragmentation of the MFT can impact performance, NTFS reserves space for the MFT in an effort to keep the MFT as contiguous as possible as it grows.”

“Because of the importance of the MFT to NTFS and the possible impact on performance if this file becomes highly fragmented, NTFS makes a special effort to keep this file contiguous.”

By allocating a large MFT long before it gets fragmented, you will never have a problem with it. btw, don’t be afraid of making the MFT too large. Windows will use all available free space before touching the MFT. If it has to use reserved MFT space for regular files, it will start at the end of it (as opposed to the beginning where it adds file index entries).

If you want to see how much your MFT is fragmented, try out the free trial of Diskeeper Pro and look at the reserved system space/MFT (white with green diagonal lines). Defrag and enlarge your MFT and see if things change. Do a boot time defrag, something the lite version can't do, where it can move system items. Here’s the link to the free trials http://www.diskeeper.com/downloads/downloa...id=0&Apid=0

The Pro version has made my clients think I'm terrific. I cleanup, defrag, etc. their systems and set it to run automatically. (I also install my own custom pgm that deletes all the temp files that get created, and not deleted, during a Windows’ session.) Their machines don't gradually get slower and slower as most users are accustomed to.

Some Microsoft Knowledgebase articles about all this:

http://support.microsoft.com/kb/130539/en-us

http://support.microsoft.com/kb/174619/en-us

http://support.microsoft.com/kb/227350/en-us

This post has been edited by Autumn Breeze: 9.12.2007 17:39

[markf]

Thanks for pointing out the importance of defrag.

After using DK for years I switched to PerfectDisk about 10 months ago. At that time DK was creating problems during boot time defrags on two of my machines, I did not like its real time scheduling, and its extensions of my MFT were wildly too large. So far, PD has been great for me.

I am no expert on chkdsk, but I recall that many of the people who posted noted that defrag did not have a significant effect on their issue. I suspect that at least some of them included boot time defrag on $Extend\$ObjID - I know that PD can defrag this file during a boot time run.

This post has been edited by markf: 9.12.2007 20:14

[Autumn Breeze]

Thanks for pointing out the importance of defrag.

After using DK for years I switched to PerfectDisk about 10 months ago. At that time DK was creating problems during boot time defrags on two of my machines, I did not like its real time scheduling, and its extensions of my MFT were wildly too large. ...

it really doesn't make any difference how large the MFT is (within reason) because, as i pointed out, it is not a totally untouchable area. when/if more disk space is needed and all that's available is MFT reserved space, then it'll start using as much of it as it needs just as if it was regular disk space. it just starts at the back to leave as much contiguous space for the MFT/indexes as possible.

that's why i make my MFT's large from the getgo... if the file system needs it for indexes, GREAT! it's contiguous, no fragmenting... if it's needed for files, that's great too, it'll use it for that, no biggie...

but the real point i wanted to make to those having probs is that a lot of fragmentation, especially MFTs, will slow down your system, CHKDSK, etc. like nothing else.

This post has been edited by Autumn Breeze: 9.12.2007 21:16

[BlueZannetti]

but the real point i wanted to make to those having probs is that a lot of fragmentation, especially MFTs, will slow down your system, CHKDSK, etc. like nothing else.

But for the folks who reportedly have real problems, that's not the issue. Fragmentation will not prevent chkdsk from completing, which is the real problem, and which may set up a cascade of events that leads to genuine and irreversible file system corruption.

Blue

[Autumn Breeze]

But for the folks who reportedly have real problems, that's not the issue. Fragmentation will not prevent chkdsk from completing, which is the real problem, and which may set up a cascade of events that leads to genuine and irreversible file system corruption.

Blue

wasn't aware that people had problems with chkdsk not completing... everything i've seen was about ppl complain' that step 2 was really slow... don't have a clue why chkdsk wouldn't complete... i'd be doin' some serious research and dianostics if that happened...

[BlueZannetti]

wasn't aware that people had problems with chkdsk not completing... everything i've seen was about ppl complain' that step 2 was really slow... don't have a clue why chkdsk wouldn't complete... i'd be doin' some serious research and dianostics if that happened...

You are correct in your impression that many people have complained about the time that stage 2 of chkdsk takes to complete. They really shouldn't since it's a complete non-issue (unless one believes that the laws of physics are only suggestions....).

The problem with the whole situation is the convolution and merging of the stage 2 time increase (not a problem) with cases in which there is an inability of chkdsk to successfully complete. The latter is a clear problem, although the direct relation to KAV/KIS generated file object ID's remains obscure at best, at least in my opinion, although there are certainly a number of people who differ on that point.

Blue

[gazer]

For most people the chkdsk stage 2 "problem" came up all of the sudden. It didn't slowly deteriorate over time. Wouldn't this seem to mean that it is not related to an MFT fragmentation.

Also, assuming your MFT is completely fragmented, will Diskkeeper Pro actually defrag the MFT itself? In one of those Microsoft articles you linked to, it said the MFT can not be defragmented.

[Autumn Breeze]

You are correct in your impression that many people have complained about the time that stage 2 of chkdsk takes to complete. They really shouldn't since it's a complete non-issue (unless one believes that the laws of physics are only suggestions.... ). ...

OMG TOO FUNNY! thank you for a good laff on a Monday mornin'...

[Autumn Breeze]

For most people the chkdsk stage 2 "problem" came up all of the sudden. It didn't slowly deteriorate over time. Wouldn't this seem to mean that it is not related to an MFT fragmentation.

Also, assuming your MFT is completely fragmented, will Diskkeeper Pro actually defrag the MFT itself? In one of those Microsoft articles you linked to, it said the MFT can not be defragmented.

well first off, that's the reason i suggested makin' your MFT's as large as you think you'll ever need in the first place....

second, the above is totally incorrect... i assume the article was out of date but don't really know... here's the truth:

in NTFS 3.0 (W2k) you can do a boot time defrag of the MFT's with Diskeeper (Pro vers)... and in NTFS 3.1 (XP & 2003) you can do an online defrag of MFT's however the boot time is still better since nothing is running then and it can do a better job...

here's a link to get NTFSInfo which might be of interest but far more info will be gleaned from the discussion i mention below http://www.microsoft.com/technet/sysintern...k/NtfsInfo.mspx

btw, NTFSInfo is from MS's Sysinternals, which is a collection of utilities that most ppl aren't aware of... if you're interested here's the link http://www.microsoft.com/technet/sysinternals/default.mspx ... you can also Google it to find out other interestin' articles...

you might ask why i keep talking about Diskeeper... well they have more than a 90% market share and their lite ver, as i mentioned above, is what's included in Windows... their products were good enough to run Norton/Symantec out of the defrag market - no smal feat... their income from individual/home users is very small, the big market is corp customers - IT pros who know how to benchmark defragin' servers/apps/etc and what they want to rest their career on... here's a link to Diskeeper's blog http://www.diskeeperblog.com/

if you really wanna know about defragin', here's an excellent dicussion between reps from Diskeeper and PerfectDisk (it's long but you will learn a great deal about defragin') In the thread, Mmaterie is Product Manager for Diskeeper – at least read his posts and you’ll lean a lot. For instance, “As for Windows XP's boot optimization, that was a technology co-developed by Diskeeper and Microsoft. MS built the prefetch system and the filter that determines the files and their order and Diskeeper wrote the code to sequence the files on the disk (for the built in defragmenter).” Unfortunately there is one guy who knows nothin’ about correct benchmarkin’ who wastes a lot of time talkin’ about “his benchmarkin’ tests”…

and another quote from it (btw, FragShield is Diskeeper's tool/option for increasing the size of MFT's) "...FragShield's purpose mirrors what many database administrators (DBAs) do as a best practice. A DBA typically creates a large database in advance, even though the total number of records only consume a small percentage of the size of that file on disk.

Pre-allocating the MFT with FragShield offers similar capabilities of what a DBA would do for their database. If you have a Win2k system and know that you will need more than 12.5% of the volume, FragShield can pre-expand the MFT and prevent it from fragmenting - mitigating the need for a bootime defrag. Diskeeper will advise you on the need.

On an XP/2003/Vista system FragShield offers the same functionality. As the zone is more dynamic, fragmentation of the MFT often occurs more rapidly than in Win2k. If you install a large app (like a software suite or a game with many files), run a defrag analysis - you'll often see MFT fragmentation. Fortunately an online defrag can address this fragmentation, but using FragShield in advance will prevent the fragmentation from occurring in the first place. ..."

here's the link: http://www.driverheaven.net/applications-s...t-disk-8-a.html

This post has been edited by Autumn Breeze: 10.12.2007 19:36