RDaneel
24th February 2003 05:00 UTC

"Official" application data folder support?
Hi all! I am on the verge of completing the transition from my own home-grown install system to NSIS 2.x (using MUI).

As you can guess, I have a question (as well as an observation or two). My question has to do with support in NSIS for detecting (and creating?) the "official" folder for application data... please correct me if I am wrong, but there doesn't seem to be any. :(

My own (in my app) logic uses the SHGetSpecialFolderLocation (with CSIDL_APPDATA)... if that fails (like on an older Windows), then <windows dir>\Application Data\ is used as the folder. But between handling errors and the ITEMIDLIST return from this call in NSIS-scripting land, it seems that the NSIS community *could* benefit from having builtin support for an "$APPDATA" variable during [un]installation.

I have (so far) resisted the temptation to stash a pointer to this folder in the app's reg entries, as I don't like creating gratuitous reg data... as an example, I save the app's menu folder location as command-line arg to the uninstaller.

In any case, I am certainly interested in any comments/solutions from those more experienced with NSIS than I! :)

I also have a comment on the RegDLL and UnRegDLL primitives in NSIS: if the DLL that one is dealing with is anything like a Shell extension, then an SHChangeNotify call will most likely be right after the RegDLL, and the UnRegDLL will very likely need to be followed by a call to CoFreeUnusedLibraries... perhaps each of these operations could be optionally performed in NSIS code rather than using the System plug-in?

Yeah, I know, that way lies bloat, but really, each one of these is a single Win32 call that doesn't require anything ugly like storage management or strings... and for this (which looks fairly simple) or possibly even for the $APPDATA thing, I would be willing to do some coding (but note that I am a Perforce rather than a CVS kind of guy).

Thanks for reading this far if you did!

Sunjammer
24th February 2003 09:49 UTC

but note that I am a Perforce rather than a CVS kind of guy

RDaneel
25th February 2003 22:36 UTC

Thanks, Sunjammer, for the TortoiseCVS tip... interestingly, I installed it and then uninstalled it when I noticed that their Shell extension was giving me serious grief - it caused the time to get a file's properties (from, say, Windows Explorer) to go WAY up. This might not be noticeable for a single file, but try multi-selecting a thousand or more files and things get real ugly. :(

After seeing the hordes of people jumping in to this discussion with comments and advice on the "$APPDATA" thing, I think I will go ahead and write my own support for this function. Sigh. :)

Sunjammer
25th February 2003 23:31 UTC

Well I was tempted to comment along the lines of "why would you want this appdata thing?" but figured that wasn't very constructive :)

RDaneel
26th February 2003 00:44 UTC

OK... perhaps I am missing something here, but this "Application Data" folder is *the* place for apps to stash any per-user data that the app needs to save (and isn't in the registry). As such, I had assumed it was of interest to almost anyone dealing with install/uninstall issues? Well, at least for apps that *have* this sort of storage requirements... :)

Anyway, a search on MSDN (CD or site) for "CSIDL_APPDATA" provides some background on this for anyone who is interested. But basically, it provides a place to store PER-USER data for apps, and is particularly important when dealing with roaming profiles or multi-user/network installations in general where the app's install directory may not be writable or is otherwise the "wrong place" to store per-user data.

In any case, take a look at the user profile folders (under "Documents and Settings" on 2K/XP) for examples of this in use... Microsoft isn't the only SW outfit using this - heck, even Sonique2 uses it! :)

RDaneel
26th February 2003 00:51 UTC

Hmmm... another quick comment in the unlikely event that anyone jumps right into this without checking out the docs first: the expected usage of the "Application Data" folder is like, say, HKCU\Software - a company/org name should be the next level, followed by a specific app name, e.g. "<Application Data>\Microsoft\Media Player"... OTOH, Mozilla just uses "Mozilla".

Sunjammer
26th February 2003 01:40 UTC

Yeah I kinda new all that, but you won't catch me storing anything in that folder. I hate programs that use it (just a foible of mine I guess). My preference is that a program store it's data in it's own installation directory so that it is all nicely self contained.

RDaneel
26th February 2003 02:39 UTC

I guess that's cool and all that (and it's DEFINITELY the way I feel about, say, DLLs that the app may need), but that model just doesn't support multiple users/profiles or separation of code and data (or R/O code trees) as easily.

Put another way, do you make use of the registry, or just use INI files in the install tree for apps you build? In the end, the Application Data tree could be seen as an extension of / replacement for the HKCU hive of the registry, for when you don't think the registry is appropriate for your purposes... hey, Mozilla uses it, and most of us probably wouldn't accuse them of trying to suck up to Microsoft! :)

kichik
26th February 2003 10:37 UTC

This archive page explains how to get shell folders that aren't avaiable using $* NSIS variables:
http://nsis.sourceforge.net/archive/...e.php?pageid=6

To get the application data folder read AppData.

I don't think calling SHChangeNotify after every RegDLL call is a good idea. Not all DLLs are shell extensions and calling this function will just slow things down when getting the shell to reload data.

RDaneel
28th February 2003 23:03 UTC

No, I am really still alive... :)

Thanks for taking the time to point me to the archived suggestion(s) kichik, but it's not clear that that approach handles all the cases I am concerned with (quite).

All of this started to really bug me, especially since Microsoft tried to introduce a DLL (SHFolder.dll) provide a uniform interface to Shell folder info, and then THAT broke because of some of their "security" fixes... so I have put together fresh installs of Win95, Win95 OSR2, Win98, WinNT4, as well as my day-to-day Win2K and notebook WinXP, to see what the real deal is with both the "SH" APIs and the "Explorer\Shell Folders" data in the registry across all of these (except Me, which I never could bring myself to get).

I'll be back (after running my tests with Beta 2)! :)

And regarding the SHChangeNotify and the CoFreeUnusedLibraries after [Un]RegDLL calls, I did mention "optional"... and when you need them, you do need them, so if the System plugin is still needed to get things done, then that is the way it goes. :(

RDaneel
6th March 2003 00:05 UTC

After examining clean installs of 95, 95 OSR2, 98, NT4, as well as "running" versions of 2000 and XP with respect to the following questions, here is what we get (regarding the "Application Data" issue):

Q: Is "\Windows\Application Data" present?

A: In 95, you would get to create it, in 98 it is already there, and for NT4 and later, you actually have a "profile" folder in which "Application Data" resides.

Q: Does SHGetPathFromIDList() give useful results?

A: This API is functional in ALL releases of Windows starting with 95, but only supplies useful info for "Application Data" starting with 98 (where it returns "\Windows\Application Data").

Q: Does SHGetFolderPath() give useful results?

A: This API is only functional in releases of Windows starting with 2000. It MAY be supported if the SHFolder.DLL redistributable has been installed, BUT see notes below.

Q: What about the "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders" key?

A: See comments below, but as of 98, this key is populated with "App Data" info, but requires expansion of the environment var "%USERPROFILE%" for NT4 and later.

Q: What about the "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders" key?

A: See comments below, but as of NT4, this key is populated with "App Data" info (and always requires expansion of "%USERPROFILE%").

Q: What does Microsoft say about locating Shell folders?

A: Previously, they said to use SHGetPathFromIDList()... which is something of a hassle because of having to deal with the ITEMIDLIST return. Later, they starting pushing SHGetFolderPath(), which is built into 2000 and later (i.e., XP), and even suggested that developers start including a redistributable containing SHFolder.DLL to use this latest "uniform" API for accessing Shell folder info. Unfortunately, code in this DLL asked for R/W access to the registry, and ran afoul of some of the Microsoft security patches. Sigh. So this "solution" became a "problem".

Comment1: So, API-wise, SHGetPathFromIDList() has the most uniform support, only requiring 98 or later. And the API-level way of getting Shell folder info is the ONLY one that has and will continue to have long-term support from Microsoft.

Comment2: The Explorer "Shell Folders" registry key is somewhat mixed in its usefulness... while the values SEEM to look good starting with 98, there are issues (besides using reg keys rather than APIs) - officially, the "User Shell Folders" Explorer key takes precedence over "Shell Folders", and it doesn't show up until NT4. As a practical note, don't forget to expand the environment strings which can be embedded in the values from these keys!

Comment3: The whole topic of per-user "Application Data" storage has gotten uglier, now that Microsoft has "refined" the model to deal with "roaming" profiles... the Registry is not cool any longer, you are supposed to split up user data into "roaming" and "local" chunks, with the original "Application Data" becoming "roaming" and a new "Application Data" under "Local Settings" becoming local or non-roaming. Sigh.

Summary: I will continue to use the following logic: use SHGetPathFromIDList() to get an ITEMIDLIST and SHGetPathFromIDList() to transform that into a string. If we can't get the ITEMIDLIST, we are on 95, so use "$WINDIR\Application Data" - creating it if we have to.

I believe that it will be increasingly useful for NSIS to support "$APPDATA" and "$LOCALAPPDATA" variables, as it can be done with a minimum of code and saves the script-writer from having to use the System plugin - as cool as it is! :)

But then, I also think that the [Un]RegDLL primitives should OPTIONALLY invoke SHChangeNotify (to update the Shell's view of "associations" and filetypes) and CoFreeUnusedLibraries (at times important if you need to do something like, say, deleting your COM DLL during an uninstall), as these are essential for full support of [un]installation of ActiveX/Shell extensions/COM objects etc., and not everyone sees it that way. :)

If anyone is interested, I will publish my function for obtaining "$APPDATA", but remember that its use will also bring in the System plugin - for now.

virtlink
6th March 2003 10:59 UTC

NSIS wants to be the InstallShield concurrent, or at least a good, free installer, capable of virtually anything, right?
I like the $APPDATA suggestion and it is one step further to InstallShield.

Sunjammer
6th March 2003 12:06 UTC

I've updated a feature request at the nsis project page. That feature request related to this kind of subject matter so I figured I'd link to this thread in it. That way when whoever gets around to looking at that feature request this information won't be forgotten.

Mad Doggie
12th October 2003 05:45 UTC

What ever happened with this one? I am urgently needing to update an INI file located under the current user's Application Data folder.

Cheers!

Luke

Mad Doggie
12th October 2003 05:50 UTC

OK, realized that I was looking at the wrong hive in RegEdt32.... I see that the current folder is stored in HKCU. We'll keep our fingers crossed.

Cheers!

Luke

Joost Verburg
27th December 2003 23:18 UTC

NSIS 2 RC1 has a constant for Application Data: $APPDATA.

Mad Doggie
28th December 2003 03:46 UTC

Originally posted by Joost Verburg
NSIS 2 RC1 has a constant for Application Data: $APPDATA.

Anders
29th December 2003 20:13 UTC

fun fact: you are not supposed to use the shell folders key




The long and sad story of the Shell Folders key

Once upon a time, in what seems like a galaxy far far away (a Windows 95 beta release known as "M3"), we documented a registry key called "Shell Folders" that programs could read to obtain the locations of various special folders like the Fonts folder or the My Documents folder.

The developers who received Windows 95 M3 Beta followed the documentation and used that key.

In the meantime, Windows 95 work continued, and we realized that a registry key was the wrong place to store this information. In part, because a lot of things (like the Control Panel) aren't disk directories so they wouldn't be expressible there. And in another part, because we had forgotten to take into account a feature of Windows NT called roaming user profiles, where your user profile can move around from place to place, so a hard-coded path in the registry is no good.

RDaneel
29th February 2004 21:07 UTC

"fun fact: you are not supposed to use the shell folders key"

Indeed. And after I had gone to some lengths in this thread to explain why this was a bad idea and what to do instead.

Sigh.

I have been away from NSIS for a while, and can still hope that the APPDATA implementation doesn't match the documentation that is currently published. I will go look...

<a short time passes>

Cool. The docs are incorrect (or at least I find them misleading), and the current code in an NSIS-generated installer does not (in general) rely on the registry for most of the "interesting" new variables - like APPDATA. :)

Or at least this would seem to be the case from a quick perusal of "GetNSISString" in "exehead\util.c".

Thanks, Kichik and Joost and whoever else helped on this one!

Now I am off to see why the latest iteration of NSIS (as with most of its "interesting" releases) no longer likes my installer script. :)