Ron.Stordahl
24th October 2011 22:25 UTC

RMdir /r followed by File /r generating random i/o errors
  I have found that the following code:

RMDir/r "$INSTDIR\Documents"

>SetOverwrite ON

File/r "..\BPQ32\Files\Documents" 

will generate random i/o failures thus:

Error opening file for writing
C:\Program Files (x86)\BPQ32\Documents\Thumbs.db

Installer stops at
Output folder: c:\Program Files (x86)\BPQ32\Documents.

This i/o error (although it isn't always the same file name) has occurred on several Windows 7 machines and one XP machine. The occurance rate is low, perhaps one in 10 runs of the installer.

To try to isolate this I placed a MessageBox immediately following the RMDir /r and with that I could never reproduce it. I then replaced the MessageBox with a Sleep 1000 and could never reproduce it. Removing the Sleep 1000 results in the error returning.

I suspect that the RMDir /r operation is using proceed i/o (overlapped, Asynchronous) when it should be using wait i/o (Synchronous).

The RMDir /r operation is apparently not yet completed when the write from the File operation is attempted.

Afrow UK
24th October 2011 22:41 UTC

How about adding /x Thumbs.db to your File instruction.

Stu

Ron.Stordahl
24th October 2011 23:07 UTC

Since it is occurring at multiple places in my installer and for multiple files it would do not good to pick just one file. The filename can be any one of the potentially hundreds in the folders I am replacing. The documents folder contains 152 files and 6 folders, several deep. I can see that it is a bit time consuming for the system to completely delete it.

When I insert a 1 second delay following the RMDir the error never occurs. I would guess 1 second is entirely too long, but it works so I am using it.

Afrow UK
24th October 2011 23:14 UTC

So it's not just Thumbs.db? I can see that file in particular causing problems as it's the thumbnail cache and it can still be in use by Windows. It also has hidden+system attributes which can prevent write access. Either way, you should exclude it from your installer as it will be adding extra size unnecessarily.

Stu

Ron.Stordahl
25th October 2011 02:47 UTC

No it is not for any particular file, rather it can be apparently any file that at that point is being allocated or written. Some that it has failed on are needed exe's for example. With 'sleep 1000' inserted after the rmdir /r the problem is gone. I have had to do this at several places in my code. Not an attractive work around, but it gets me by.

My conclusion is that the rmdir should be using wait i/o.

How can I get this issue to the attention of one of the NSIS developers?

MSG
25th October 2011 07:40 UTC

Originally posted by Ron.Stordahl
How can I get this issue to the attention of one of the NSIS developers?

You should file a bug report in the bug tracker, or submit a code page if you know the solution. You could also drop by in the IRC channel if you want to discuss it realtime.

Anders
25th October 2011 14:29 UTC

NSIS does not do any async I/O IIRC, maybe antivirus or something like that is getting in the way?

Ron.Stordahl
25th October 2011 15:27 UTC

Ill file a report in the bug tracker, once I find where that is..perhaps you have a link?

No it's not a virus issue, I just reproduced it on a new W7 laptop fresh out of the box. The occurrence rate is low, maybe 1 out of 20 runs, but unacceptable of course. And yes I know a workaround, put a 'sleep 1000' after every RMDir /r.

Now to try to find the bug tracker page.....

MSG
25th October 2011 16:10 UTC

He didn't say virus issue, he said antivirus. Antivirus software locks all accessed files for a short while while it scans them, before any operation can be performed on them. You can test this by disabling your virusscanner, and checking whether that solves the problem.

Ron.Stordahl
25th October 2011 17:18 UTC

Sorry...I misunderstood. Actually the Windows 7 Laptop I just tested this on, obtaining the failure after about 15 consecutive runs of the installer, was right out of the box and had no antivirus scanner on it...it keeps warning me that I need to get one! Perhaps Ill download MSE at some point, after I first do about a years worth of security updates.

I still suspect the RMDir is not using wait i/o. I never see the issue if I add a 'sleep 1000' after each RMDir. Makes the installer kind of jerky, but gets me by for now.

Anders
25th October 2011 18:03 UTC

If you don't pass /REBOOTOK we just use the plain Find*, RemoveDirectory and DeleteFile, none of them are async. (See util.c in exehead source)

MSDN says "The DeleteFile function marks a file for deletion on close. Therefore, the file deletion does not occur until the last handle to the file is closed." This smells like a antivirus and/or indexer issue to me...

Ron.Stordahl
25th October 2011 19:43 UTC

Since it occurs on machines with and w/o antivirus software I am dismissing that possibility. As to indexing...perhaps there is a command that can be invoked from within NSIS to turn off indexing, where and if it was on, then turn it on again when the installer finishes? If so I can try that.

As I said I have 'resolved' the problem by inserting 'sleep 1000' after each RMdir, but that does not get to the root of the problem..it only masks it.

When time allows I will post a bug report, when I find where that is done.

Anders
25th October 2011 20:51 UTC

It would really help if you could come up with a minimal sample script (and any required files) so I could try to reproduce this (Please come up with a sample that does not include Thumbs.db or other internal windows files)

You report bugs on the NSIS sourceforge tracker (project page on SF)