Gripes about Diskeeper 5

Diskeeper is a file defragmenter. These comments are about the 30 day, full featured, free trial version of Diskeeper 5 for Windows NT4 workstation. NT 4 does not include a defrag program. Windows 2000 includes a stripped down version of Diskeeper.

I installed and ran Diskeeper 5.01.330t under Windows NT4 workstation at Service Pack 3. The machine has two partitions. The C disk is 1.2 gig and formatted as NTFS. It is very fragmented and has about 20% free space. The D disk is about 340 meg and also has about 20% free space.

The install instructions were confusing at one point. It said that it required the Microsoft Management Console v1.1. Then it said do you want to install this software and it wasn't clear to me whether this meant MMC or Diskeeper.

It never cleaned up after itself. What seems like all the installation files were left behind in the C:\TEMP directory.

A defrag of the NTFS partition (C disk) made things worse. There were more file fragments afterwards than before. Diskeeper said this was because there was not enough free space. I made a bit more free space but another defrag still did not improve things much. Then I figured it was time for a boot time defrag to clean up the directories and the Master File Table (MFT).

The first time I went to set up boot time defrag options, nothing ran at boot time. The window where you set these options is confusing. I turned on all the check boxes that I wanted and clicked on a CLOSE button. No good. It turns out you have to click on a SET button, which I failed to see in the row of buttons. Nonetheless, CLOSE usually means to accept user input and close the window. In this case, it acted like a CANCEL button.

When I got this right, the boot time defrag of a 1.2 gig NTFS partition ran for about 2 hours. Partly this was because it did a full defrag of the data in the partition after defragging the directories and MFT (which I had asked for). This was a total surprise, as I had not requested it in the boot time options window. It did a much better defrag than earlier online ones, but there was no warning that this was coming. This took about an hour and deprived me of the use of the machine (its not the sort of thing you interrupt). A boot time defrag of a FAT partition (at a different boot), did not do a full defrag of the data. The machine was a Pentium 200 with 96 meg of RAM.

I ran an analysis of a FAT partition, then went to set up boot time defrag options for that partition. It said
"A more recent analysis of this volume is needed".
I had done the analysis of this partition/drive about 90 seconds prior to this.

Frag Guard seemed very active, even when the system wasn't doing anything. I often seemed to wake up and cause disk I/O, although the cpu usage shown for it in Task Manager was small. I turned it off.

There is no File option on the menu bar; also no Help option on the menu bar. There is no Exit option at all off anything on the menu bar. In fairness, this is probably due to the Microsoft Management Console that Diskeeper seems to run under. Still, it was a surprise to me as an NT4 user.

Diskeeper is not consistent with how it reports Free Space. Sometimes it reports it in megabytes (MB) and sometimes in kilobytes (KB).

I installed and ran the 30 day trial of Diskeeper 5.01.330t on another Windows NT4 computer, this one at Service Pack 6a.

Despite the latest NT service pack, it still needed to install the MMC. I ran a normal defrag on the NTFS C disk and then ran the full range of boot time defrags. The partition is about 1.3 gig and had about 400 meg of free space. The cluster size is 512 bytes. Excluding the time for an NT CHKDSK, all the boot time defragging took 27 minutes. Then I analyzed the partition again Diskeeper reported 26% total fragmentation, 0% file fragmentation and 90% free space fragmentation. It recommended yet another defrag!

I took its advice and ran a third defrag. The number of excess file fragments went from 27 to 15. Total fragmentation went from 26% to 15%. Free space fragmentation went from 90% to 54%. Perhaps the boot time defrag created the need for another regular defrag run? Below is the result of an "Analyze" after this third defrag.

There is a very large part of this partition devoted to "Reserved system space" (green stripes). The Diskeeper help said this was space reserved by NT4 for expansion of the MFT. It explains that NT prohibits anyone and anything from writing files into this space, but does allow applications such as Diskeeper to move files out of it. As a result, you may end up with less free space on an NTFS partition after defragging it should Diskeeper chose to move files out. I did not check free space on this partition before and after defrags.

The paging file, in yellow above, is 40 megabytes, so an eyeball based guess puts the "Reserved system space" at almost four times that or about 150 megabytes. Quite a lot of dead wood in a partition that is using 922 megabytes in total. The Diskeeper help says nothing about decreasing this space, so I suppose gripes about it are directed at Microsoft and the NTFS file system.

Another conclusion from the above report is that Diskeeper did not do a good job of consolidating the green System files. The Diskeeper help says this the MFT and several other file system files.

The installation of Diskeeper did not clean up after itself. In C:\TEMP it left a directory called "Diskeeper Trialware for Windows NT" with 5.4 megabytes of files in it.

The installation procedure does not show you the fairly extensive instructions on how to install the product that are in the the readme file. You don't see the readme file (mine was dated September 3, 1999) until after the product is installed. In fact, unless you went poking around in the directory where Diskeeper was installed, you would not even know there was a readme file. Likewise there is help about installing the product in the Help files for the product which you can't read until after it has been installed.

At boot time, Diskeeper warns you when there are 5 days left in the trial. It has insinuated itself into the boot time start-up processing without informing me. I ran Diskeeper and looked over every option for some indication that it was running at boot time or a way to turn it off. There isn't any. I am not running any feature of Diskeeper that requires it to be constantly active. It's a stealth program, not very polite. The NT task manager shows that there is a process running called DkService.exe. Sure enough, the Control Panel Services applet shows a Diskeeper service that is started automatically.

Every time the machine boots during the last 5 days of the trial, Diskeeper puts up a window reminding you of the number of days left in the trial. After the trial period is over, it continues to nag at boot time.

Diskeeper nags and nags and nags ...

And nag, and nag, and nag ... I had to go to the Services applet and change the Startup option to disabled. This should not be necessary, it's one thing to remind the user when an expired trial version is invoked, but another to constantly nag at boot time.

I ran a defrag today on the same computer as above and the "Reserved system space" (green stripes in example above from June 3,2000) on my NTFS partition got larger! I watched Diskeeper move contiguous (blue) files from just to the right of the "Reserved system space" to just after the last file in the partition (the very end of the partition was freespace). As it moved files, the space they used to occupy did not become free, instead it became part of the "Reserved system space".

The help for Diskeeper has no entry for "reserved" or "system" or "space". I did a search in the Diskeeper help for "reserved system" and found a paragraph explaining what this area of an NTFS partition is. It said "Reserved system space" is reserved for expansion of the Master File Table (MFT) and can not be used by applications. NT will write files into this area when the partition is extremely full and there is no other freespace available. Diskeeper can move files out of this reserved area to ensure that the MFT can expand contiguously, but it can not move files into it.

If this area is only for MFT expansion (as the Help says), why is it labeled "reserved system space" as opposed to "reserved MFT space" or "reserved MFT expansion"?

In my case, there was no reason for NT to have moved files into this area as the partition has consistently had about 40% free space. In fact, the last defrag had been only a week or two earlier and I'm fairly sure that freespace did not fall much below 40% between then and now.

The text based report produced by an analysis of the partition mentions nothing about the "Reserved system space". This is an annoying omission as an eyeball look at the graphical analysis shows that the "Reserved system space" seems to be about 200 meg (about 12% of the the1.7 gig partition). All the text report says is that the MFT itself is 25 meg and that it is 96% used.

I went to the web site for Executive Software at http://www.execsoft.com and searched their Knowledge Base for "RESERVED SYSTEM SPACE". Nothing. Then I searched for "NTFS RESERVED". Nothing. Then I searched just for "NTFS" and got 31 hits. Looked at them all. Nothing relevant.

As I am running a free trial, there is no tech support. Nonetheless, I emailed them at tech_support@executive.com just to see what they would say.

Besides email, you can also submit bug reports on their web site. Quoting from the web site:
If you have encountered a particular bug with our software, please fill out a Tech Support
Request Form, and someone will get back to you right away.
Doing so requires registering with Executive Software and answering a bunch of questions, which I did and was assigned a ticket number of 20006275348.
FYI: The web site does not validate the product number. Since I am running the trial edition, I have no product number and I entered 123456 which it accepted.

The first response from my email message to tech support was a knee-jerk. The product is working as designed, it just moved files out of the "Reserved system space". The second response was:

What you have here is a partition that was originally formatted as FAT and was converted to NTFS. This causes the MFT to be written in what ever free space is available at the time of the convert. Your reserved MFT space sets itself up in a similar fashion. DISKEEPER will them move files out of what ever area the reserved MFT space, (12.5% of the total partition), has "staked out". The reserved MFT space itself never expanded, it was specified under files that were already there prior to the convert. The best way to handle this is not to convert FAT partitions to NTFS.

If you want to have an NTFS partition, format the partition NTFS from the beginning. Since this is the C: partition which is most likely the partition that Windows NT was installed to, you would need to format the partition NTFS by placing the physical disk in another system that is already running Windows NT and performing the NTFS format from Disk Administrator. If you do not have another Windows NT system on hand to do this, simply leave your C: partition FAT.
If you have any questions, you can call me at (800) . . .

Pat

The last point is to leave my C partition FAT. It is not FAT, it's NTFS. It says that the reserved MFT space never expanded. Perhaps not, but the green striped area did. It is focused on the fact that the partition was converted from FAT to NTFS. I wrote about a problem that I noticed between two defrags. The partition in question was NTFS during both defrags.

This response did make me think about the issues in more detail and it's not clear how the Diskeeper GUI reports on files that are in the "reserved system space". On the one hand they are files, so they might show up as red/blue depending on whether they are fragmented or not. On the other hand, they are occupying space that NT wants for itself, the "reserved system space." Based on what tech support later said, it appears that files in this condition are shown as red/blue.

Diskeeper reports on free space in a partition. It's not clear whether this includes the "reserved system space" or not. On the one hand it is available for NT to write files to. On the other hand, it is "reserved". Based on what tech support later said (see below) it does include the "reserved system space". However, Diskeeper can not use the freespace in the "reserved system space" while performing a defrag. They really should break down the freespace they report into regular freespace and freespace within the "reserved system area."

Below is the response from filling out forms on the Executive Software web site:

Oh. I wondered how Pat figured out that the "reserved system space" was 12.5%. Interestingly, no mention is made of converting a FAT partition to NTFS and its effect, which Pat thought was critical. This may make sense to Lance but not to me. Is the MFT reserved space always 12.5%? Also, without knowing the answers to the above questions (such as how the GUI reports on files in the reserved space), it's impossible to get a handle on just what is going on.

At this point I'm afraid to defrag for fear that the "reserved system space" will continue to grow, leaving me with a large wasted area of disk space. If this area is reserved by Windows NT why is Diskeeper enlarging it?

...Let me begin by giving you the full history on the MFT Reserve Space along with steps that you may use to expand it.

In NTFS on Windows NT 4.0, the MFT Reserve Space is a static area created at the boot of the system. The default size is 12.5% of the full size of the partition. In compliance with Microsoft's wishes, DISKEEPER will not move files into the MFT Reserve Space; I do not believe that all of our competitors have followed this Microsoft recommendation. In our first version of DISKEEPER for Windows NT 4.0, (2.0), our developers did not include the green candy stripe area we are discussing, so the task fell upon tech support to explain what was occurring on all NTFS partitions. Upon the release of a later build of DISKEEPER, the green candy stripe area was included, providing the data that DISKEEPER uses for all to see.

You can increase the MFT Reserve Space in 12.5% increments up to 50% of all local NTFS partitions, by adding the following registry key; I'm afraid there is no way to reduce it below 12.5%:
<--- instructions deleted ---->

Having supplied the above data on the MFT Reserved Space, I would like to respond to your specific questions below:

1. You are correct that the text based analysis reports has no data on the MFT Reserved Space. As I mentioned above, the MFT Reserved Space is static so our developers decided that data in the text base report was not needed. I agree that 12.5% is a sizable area, but it once again it is static.

2. I agree that there is no evidence that Windows NT moved files into the MFT Reserved Space. As a general rule when you have a partition that is NTFS format, Windows NT will not move files into the MFT Reserved Space until there is no other free space available. That would mean that you at one point had less than 12.5% free space; not 40% free space as you mention. This does not appear to be the case. I agree that there is no way to be certain on the amount of free space during a previous time, so I am willing to make this assumption along with you. I believe that the files were written into this area when the partition was FAT16, not after the conversion to NTFS. FAT has no restriction on where files are written, making this assumption more likely.

3. You mention that this was your second defrag, not the first after you converted the partition to NTFS. While my data on DISKEEPER operation would indicate that the movement of these files should occur the first time a defragmentation occurs, it is not out of the realm of possibility that other conditions at that time may have inhibited this action. You experienced the movement of files from the MFT Reserved Space on the second run of DISKEEPER; while less likely, that is possible.

4. In your question about "reserved system space" and "normal files" it appears that you may have some confusion on the difference between free space and allocated space. Free space, in this case "reserved system space", can be occupied by allocated space, in this case red/blue (normal files). DISKEEPER simply moved the normal files out the reserved system space. The normal files were in the reserved system space, so moving them exposes that reserved system space that they previously occupied.

5. The percentage shown in the far right hand column of the DISKEEPER GUI includes the MFT Reserved Space. A decision was made to report free space in this fashion do to all other reporting mechanisms in Windows NT report free space in this fashion and thus likely reducing tech queries on the difference that would be reported by not included the MFT Reserved Space. ...

Pat

A very thorough response! I deleted the instructions Pat included for zapping the registry to enlarge the size of the "reserved MFT space" as I wrote to Executive Software because I thought the size was already too big. This reply ended the issue as far as I was concerned. My only gripe about the above is point 4 where Pat says "Free space...can be occupied by allocated space...", not the best choice of terminology. The important point here though is there is no way to know when this happens.

Later in the day, Pat follow-up on my point that it seemed like running a defrag resulted in less free space.

No doubt, Pat was referring to the fact that Diskeeper can not write files into freespace that is part of the "reserved system space."

The actions I saw were most likely not a bug in Diskeeper at all. However, the product does not provide enough information to be sure. If the provided information were more complete, I never would have assumed there was a bug in the first place. Even if the green candy striped section ("reserved system space") of an NTFS partition did occupy more than 12.5%, it would not necessarily indicate an error in Diskeeper, but rather could result from valid registry settings. There is just now way to know for sure. Again, for a version 5 product, guessing and assuming should not be necessary.

Earlier I suggested that "reserved system space" be called "reserved MFT space" or "reserved MFT expansion". It turns out the word "reserved" is also a misnomer. It's not really reserved, it can be used by any file when NT judges there is not enough normal freespace in the partition. It's also not reserved for use by the operating system as Diskeeper will move files out of it.

It occurred to me that Diskeeper might be indirectly telling me the size of the "reserved system space".

I ran an analysis of an NTFS volume that had 19% freespace. The pop-up window after the analysis warned that only 8% of the total volume was available for defragmentation purposes. Perhaps the 11% difference between the 19% freespace and the
8% for defragging is accounted for by the reserved MFT space? I asked this question of Executive Software via email and Pat responded:

In my, now expired, copy of Diskeeper, the View option on the menu bar has no text option (see below).

Also, Pat assumes the "reserved system space" is always 12.5% which is not true. In fact, the example I cited in my email was of a case where it was 11%.

Nonetheless, you can calculate the size of the "reserved system space" if Diskeeper tells you the percent of freespace available for defragmentation purposes. Still another problem however, is that it does not always provide this information when analyzing an NTFS partition. Why? Beats me.

My trial edition of Diskeeper now expired, I am back to Diskeeper Lite, a free version of Diskeeper that pre-dates version 5. Below are condensed images of before (left) and after (right) the de-fragmentation of an NTFS partition. The partition was 2.11 gig, of which 1.3 gig was used and 827 meg (39%) was free. Free space is shown in solid light gray. For a full size copy of each image, click on the image.

The first gripe with Diskeeper Lite is the amount of freespace before and after. As shown in the screen shots below, there is a lot more light gray after the defrag than there was before (click on each image for a full size version that will open in a new browser window). How can this be? Diskeeper should be combining small chunks of freespace into larger chunks, but there is no way it can make freespace where it did not exist before. I suspect this is a bug with Diskeeper Lite.

The second point here is not necessarily a gripe. The Reserved Free space, of which much was made in my prior gripes, is pretty much the same after de-fragmentation as it was before. In these pictures it is the entire second row and parts of the first and third row. Very little was removed from within the Reserved Free space, despite the vendor saying above that Diskeeper will move files out of it. It could be that this functionality was added to later versions of Diskeeper and is not in the Lite version. It's actually fine by me that it works this way.

Before	After