xsinx
19th November 2009 20:28 UTC
32bit Application on 64 bit Windows Vista Home issue
Hello everyone,
I'm not sure this is even NSIS related but I figured I might be able to find an answer here. We have this 32bit application that we install using NSIS. This is a mainstream application and have probably around 100K clients using this application right now.
The application EXE is VB.NET assembly forced to x86 mode. It uses about 75 external DLL's, a third of which are COM ATL 32bit DLL's, an other third are plain 32bit DLL's and the last third are .NET DLL assemblies forced to x86. It has been tested successfully on pretty much all Windows versions, including XP 32/64, Vista 32/64, Windows 7 32/64, be it home, developer, Ultimate, you name it we tested it.
We have 2 clients for whom the COM ATL DLL's won't load (CLSIDFromProgID fails with 800401F3 "Invalid Class String"). First reaction was to check the registry and indeed, something weird is going on for these clients, the class in question is registered in CLASSES_ROOT instead of Wow6432Node as it should on a 64 bit system. I managed to get around this by forcing NSIS to use the REGSVR32 in the SysWow64 folder but the error remains exactly the same.
After a lot of messing around, we found out that running the application as an admin for these clients fixes the issue.
Now... Could someone explain to me:
1. How did stuff ever get outside of the Wow6432Node in the registry? I was under the assumption that 32 bit code was restricted to that node by the virtualization system. The erroneous entries were apparently created by NSIS, we using the internal RegDLL function to register our DLL's.
2. Why does running the app as an admin have any effect on this problem? If this is a registry issue, shouldn't an elevated account have the same problem?
Any other info to clarify this issue would be appreciated. For now, the users are forcing their app to run as an admin but it won't be too long before another comes up which can't get an admin account (using the software in a secured workplace for instance).
I've worked like 15 hours straight on this and I'm about to go crazy! Thanks in advance for any help you can provide.
redxii
19th November 2009 23:46 UTC
Try Library.nsh: http://nsis.sourceforge.net/Docs/AppendixB.html
You're thinking of redirection, and I don't believe it restricts anything from anything.
xsinx
20th November 2009 03:14 UTC
How is it different from using RegDLL? The DLL is installed in the proper folder using a "File" command in NSIS and I'm using RegDLL to register it. This is working for 99.99% of our users, why would those two have a different outcome?
Before I even fix this, I'd like to understand what's going on... How could using an elevated account vs a non-elevated account (same account, which has admin rights) change the behaviour of CLSIDFromProgID? I want to get to the deep of this and really understand what we are doing wrong. Is this a .NET bug? a Windows bug? Some security loophole? A virus or other similar interference? A recent Windows Update that could have trigged this?
One detail I forgot to mention is that one of these users, the one I spent the most time debugging, had this software installed for months without any issue. One morning it just didn't work anymore without any software or hardware change, it just stopped being able to load the COM object. Using elevation, the software runs as expected with no issue. How is this possible? We cleaned all reference to our software in the registry, uninstalled the software and reinstalled from scratch with the same installer that works for all our other users and it's still not working, so it's not a case of corruption of our software in particular.
It's as if the 32 bit redirection is partially failing only in non-elevated mode, I'm totally confused by this.
redxii
20th November 2009 06:48 UTC
Are any of the other 99.99% of users running a 64-bit OS? If it happened months after being installed then it isn't the installer.
There is a program called Process Monitor by Microsoft (SysInternals), it can show you everything the program is accessing, whether or not access was denied to a particular file or registry key, and compare it to what happened ran as admin vs non-admin. I used it a lot to find out why programs don't run as administrator (usually trivial reasons like write access to a folderor registry key).