RMdir /r followed by File /r generating random i/o errors

I have found that the following code:

RMDir /r "$INSTDIR\Documents"
SetOverwrite ON
File /r "..\BPQ32\Files\Documents" 

will generate random i/o failures thus:

Error opening file for writing
C:\Program Files (x86)\BPQ32\Documents\Thumbs.db

Installer stops at
Output folder: c:\Program Files (x86)\BPQ32\Documents.

This i/o error (although it isn't always the same file name) has occurred on several Windows 7 machines and one XP machine. The occurance rate is low, perhaps one in 10 runs of the installer.

To try to isolate this I placed a MessageBox immediately following the RMDir /r and with that I could never reproduce it. I then replaced the MessageBox with a Sleep 1000 and could never reproduce it. Removing the Sleep 1000 results in the error returning.

I suspect that the RMDir /r operation is using proceed i/o (overlapped, Asynchronous) when it should be using wait i/o (Synchronous).

The RMDir /r operation is apparently not yet completed when the write from the File operation is attempted.

How about adding /x Thumbs.db to your File instruction.

Stu

Since it is occurring at multiple places in my installer and for multiple files it would do not good to pick just one file. The filename can be any one of the potentially hundreds in the folders I am replacing. The documents folder contains 152 files and 6 folders, several deep. I can see that it is a bit time consuming for the system to completely delete it.

When I insert a 1 second delay following the RMDir the error never occurs. I would guess 1 second is entirely too long, but it works so I am using it.

So it's not just Thumbs.db? I can see that file in particular causing problems as it's the thumbnail cache and it can still be in use by Windows. It also has hidden+system attributes which can prevent write access. Either way, you should exclude it from your installer as it will be adding extra size unnecessarily.

Stu

No it is not for any particular file, rather it can be apparently any file that at that point is being allocated or written. Some that it has failed on are needed exe's for example. With 'sleep 1000' inserted after the rmdir /r the problem is gone. I have had to do this at several places in my code. Not an attractive work around, but it gets me by.

My conclusion is that the rmdir should be using wait i/o.

How can I get this issue to the attention of one of the NSIS developers?

Originally Posted by Ron.Stordahl View Post

How can I get this issue to the attention of one of the NSIS developers?

You should file a bug report in the bug tracker, or submit a code page if you know the solution. You could also drop by in the IRC channel if you want to discuss it realtime.

NSIS does not do any async I/O IIRC, maybe antivirus or something like that is getting in the way?

Ill file a report in the bug tracker, once I find where that is..perhaps you have a link?

No it's not a virus issue, I just reproduced it on a new W7 laptop fresh out of the box. The occurrence rate is low, maybe 1 out of 20 runs, but unacceptable of course. And yes I know a workaround, put a 'sleep 1000' after every RMDir /r.

Now to try to find the bug tracker page.....

He didn't say virus issue, he said antivirus. Antivirus software locks all accessed files for a short while while it scans them, before any operation can be performed on them. You can test this by disabling your virusscanner, and checking whether that solves the problem.

Sorry...I misunderstood. Actually the Windows 7 Laptop I just tested this on, obtaining the failure after about 15 consecutive runs of the installer, was right out of the box and had no antivirus scanner on it...it keeps warning me that I need to get one! Perhaps Ill download MSE at some point, after I first do about a years worth of security updates.

I still suspect the RMDir is not using wait i/o. I never see the issue if I add a 'sleep 1000' after each RMDir. Makes the installer kind of jerky, but gets me by for now.

If you don't pass /REBOOTOK we just use the plain Find*, RemoveDirectory and DeleteFile, none of them are async. (See util.c in exehead source)

MSDN says "The DeleteFile function marks a file for deletion on close. Therefore, the file deletion does not occur until the last handle to the file is closed." This smells like a antivirus and/or indexer issue to me...

Since it occurs on machines with and w/o antivirus software I am dismissing that possibility. As to indexing...perhaps there is a command that can be invoked from within NSIS to turn off indexing, where and if it was on, then turn it on again when the installer finishes? If so I can try that.

As I said I have 'resolved' the problem by inserting 'sleep 1000' after each RMdir, but that does not get to the root of the problem..it only masks it.

When time allows I will post a bug report, when I find where that is done.

It would really help if you could come up with a minimal sample script (and any required files) so I could try to reproduce this (Please come up with a sample that does not include Thumbs.db or other internal windows files)

You report bugs on the NSIS sourceforge tracker (project page on SF)

One way to reproduce this issue is to do a Rmdir /r of a JVM and then follow that with an unzip to the same location. It will fail on random files during the unzip. We are experiencing this issue at present in an older NSIS 2.50 Linux special build.

Are the files random or are they files that are in use? Add a messagebox after rmdir /r and verify that the directory is empty. 3rd-party anti-virus installed?