SunToo
21st May 2007 07:55 UTC

cannot disable file system redirection on 64-bit Windows
Hi,
I wrote an installer that detects during initialization whether it is running on 32-bit or 64-bit Windows. If the latter is true it disables file system redirection for the following reasons:

1) I want to write to the system32 directory.
2) I want to use the same script for both 32-bit or 64-bit Windows.

But it doesn't work. Please try running the following example installer (compiled with NSIS v2.27) on 64-bit Windows.

OutFile "test64.exe"
Name "test64"
AllowSkipFiles off
SetOverwrite try


Function .onInit
push $0
push $1
push $2

;disable the file system redirector for the current thread
System::Call 'kernel32::Wow64DisableWow64FsRedirection(*i .r1) i .r2 ?e'
Pop $0
IntCmp $2 0 0 label_OK label_OK
;The call failed.
MessageBox MB_OK "Wow64DisableWow64FsRedirection failed with error $0"
Abort
label_OK:
;Installation of a dummy file: This installation works.
File "/oname=$SYSDIR\test1.txt" "${NSISDIR}\Docs\Splash\splash.txt"

pop $2
pop $1
Pop $0
FunctionEnd

Section "Install"

;Installation of another dummy file: This file is installed in the wrong directory!!!
File "/oname=$SYSDIR\test2.txt" "${NSISDIR}\Docs\Splash\splash.txt"

SectionEnd

While test1.txt is installed correctly in the system32 directory test2.txt is installed in the wrong directory SysWOW64. Why?

Afrow UK
21st May 2007 09:33 UTC

Why not put the code in a dummy section before all other sections instead.

Section -DisableWow64FsRedirection
...
SectionEnd

Maybe it's because the UI is not initialised until after .onInit perhaps :S

Stu

SunToo
21st May 2007 09:54 UTC

Thanks for you reply, Afrow UK.

The example only demonstrates the problem I detected. Actually my installer does something more. Since NSIS doesn't allow to compile real 64-bit installer, I try to simulate it as best I can. E.g., I disable file system and registry redirection during initialization.

The workaround you proposed may work for some specific installations (I didn't tested it). But installation of files doesn't only occur during installation part. And I don't want to rewrite the installer completely new for every installation task. I want to create a framework which is suitable for most installation tasks.

So I think the better solution would be to identify and solve the problem.

Afrow UK
21st May 2007 10:11 UTC

Try moving it to .onGUIInit then.
If using Modern UI you will have to declare your own using !define MUI_CUSTOMFUNCTION_GUIINIT.

On a side note, you shouldn't really modify the user's system before the install files page in case the user chooses to cancel.

Stu

SunToo
21st May 2007 10:23 UTC

Regarding your side note: File system redirection is only disabled for the thread that calls Wow64DisableWow64FsRedirection. So it only affects the installer. SetRegView also only affects the installer.

Regarding the failed redirection: By what is it caused?

kichik
21st May 2007 20:05 UTC

As you said, Wow64DisableWow64FsRedirection only affects the current thread. The thread that runs .onInit is not the same thread that runs the sections, unless you're running the installer silently.

You should not disable redirection for the entire installer but only for those specific files that you do wish to put in system32. That could get Windows to load the wrong DLL files in some cases.

You should also use the macros from x64.nsh which make it a bit easier.

SunToo
22nd May 2007 07:58 UTC

Originally posted by kichik
As you said, Wow64DisableWow64FsRedirection only affects the current thread. The thread that runs .onInit is not the same thread that runs the sections, unless you're running the installer silently.

You should not disable redirection for the entire installer but only for those specific files that you do wish to put in system32. That could get Windows to load the wrong DLL files in some cases.

kichik
22nd May 2007 12:03 UTC

Not too much to tell. One thread runs the GUI, the other executes the sections.

As for the DLL files, you should disable redirection right before you drop a file into system32 and enable it back again right after. If you don't, the wrong DLL files may be loaded as shown in the link. It could be just a bug in RichEdit, but I'd restrict the disabling of redirection to the bare minimum anyway.

Comperio
22nd May 2007 22:32 UTC

Here's another way to look at it:

If you are running a 32-bit app under 64-bit, Windows redirection will take over and redirect that exe back to the 32-bit sections of the file system and registry.

If you force the install to install that EXE's components under the 64-bit sections, then it will likely fail when you run it because it won't be able to find its "parts" in the proper places.

SunToo
23rd May 2007 14:33 UTC

Now the penny has dropped. In another thread I mentioned myself the problems of loading 64-bit DLLs by a 32-bit application. But I didn't realized the problem of finding the right ones. Sorry and thanks for your patience.