theTallGuy
30th November 2006 00:17 UTC

Vista: Making installed app run in XP compatibility mode
I've been testing my installer under Vista lately, and discovered that the installed app doesn't play well with Vista's Aero (glass) UI. (The installer, amazingly, works great without modification. Great work, everyone!)

I would like to somehow mark the app to run in XP compatibility mode, which will force the app into Vista Basic UI (non-glass) mode. In other words, I would like to do the equivalent of right-clicking on the app's .exe file, choosing Properties then Compatibility, and selecting XP compatibility.

Does anyone know how to do this? Or, better yet, avoid this hassle completely? (I was hoping there would be some setting in the Vista manifest to do this, but I can't find it if there is.) Changing the software to be glass-friendly is, unfortunately, not an option giving our release schedule.

Thanks in advance for any help you can offer!
tTG

onad
30th November 2006 10:38 UTC

Would you be so kind as to post a screenshot of what you think is not working with AeroGlass and NSIS in Vista?

In the mean time, you could change the message send to a dialog window in .onInit see :
http://www.installationexcellence.co...phi/Index.html

for some hints which API calls.

theTallGuy
30th November 2006 16:54 UTC

Thanks for the link, but the app problem is trickier than that, as certain features need to tie in to the underlying graphics system, and for some reason, they don't work quite correctly with Glass.

I'm sure the problem can be solved, but as we're trying to finish our release candidate by tomorrow, there's no time to break the underlying system. For the moment, the workaround will have to do. We'll solve the Vista issues in the next release when we have more time.

Just to be clear, I'm looking for a way to modify the app's compatibility from the installer. (The problem is *NOT* that the installer has issues with Glass.) I've found one way, which is really, er, a kludge; it involves setting an special environment variable that causes any apps run to work in XP compatibility mode. (I didn't even know that Windows cared about what's in environment variables!) For the curious, you can read about it here:

http://support.microsoft.com/default.aspx/kb/286705

But I would prefer a way that only affects my own app. I'm hoping that someone in this forum knows a way to modify an app's compatibility properties, either via plugin or by executing a command-line utility.

Thanks!
tTG

kichik
30th November 2006 18:29 UTC

A quick Google search for "disable dwm" came up with a related MSDN article. It doesn't say that'd it cancel everything, but you can control at least some of it.

theTallGuy
1st December 2006 20:03 UTC

Thanks, Kichik! I'll pass this on to the developers for when we change the code to truly support Vista.

In the meantime, I'm guessing that there isn't any way to set the OS compatibility properties for an app from NSIS other than by the environment string approach. Sigh. Sometimes these new Windows version are really hard on what remains of my hair... :)

Thanks to everyone who responded and tried to help!

tTG

kichik
2nd December 2006 09:28 UTC

As far as I remember, when you select compatibility mode for a program, it writes something about it in the registry. You could make that change while running regmon in the background to see what you need to write in the registry to set it.

theTallGuy
5th December 2006 05:46 UTC

RegMon did the trick! For those of you who want to force an app to run in XP compatibility mode, just set the following registry key:

HKCU "Software\Microsoft\Windows
NT\CurrentVersion\AppCompatFlags\Layers" "[full application path goes here]" "WINXPSP2"

This will hold us until we can do a real fix.

Thanks, Kichik!

tTG

dbach
12th December 2007 10:03 UTC

Thanks theTallGuy/Kichik. I had the same problem. Now, using the key provided by theTallGuy, I am able to solve this problem. Thanks a lot!

cchian
12th December 2007 16:47 UTC

In some cases you may want to make that change in HKLM if you want it applied to All Users, I have only used it in XP but should work the same way under Vista.