Unicode

I don't think there is any possibility to convert existing scripts automatically.

There is no way identify the codepage of a script file, the bytes just have a different character representation depending on the users codepage. Script files can even contain multiple codepages (for example, if language strings are combined in a single file).

On run-time, texts that are sent to the dialogs can come from any source on the system, complete outside the installer itself. Again, no automatic conversion is possible. Things become even more complicated when external DLLs (including NSIS plug-ins) are called that return ANSI strings.

Taking this all into account, I think the only realistic thing is to release a new major version that will be Unicode-only.

Of course not, and you're right - those scripts would need to be converted. But we're not going to get around such an issue no matter what we do; and I don't see it as a good idea to cause extra hassle for everyone writing non-English single-language installers, compared to causing a little more work for those that would actually benefit greatly from converting it to Unicode - at least not for the first official Unicode release.

Also, are the amount of "non-native-codepage"-dependent scripts really that big? I'm under the impression that many people stick to the bundled stuff, which would already be Unicode, or have already switched to the Unicode build.

EDIT: Good point, Joost - I hadn't considered external calls. Alright, never mind the ANSI then.

IMHO two article developers or NSIS script creators should read (I mean really READ...)

http://www.microsoft.com/globaldev/h..._announce.mspx



Note that this message is NOT to insult, it's purpose is to help getting Unicode support in a decent smooth upgrade path to NSIS. Lots of people have lots of good ideas but somehow we have to create it...that will mean some fierce pro/cons discussions. But take for example Jim Park's initiative, it is *great*, he got things rolling for Unicode for him because he has a need and just created his own build!

It is also not that I personally still would support for Win9x/ME, but a lot of third pary software uses NSIS e.g. the Firefox installer on Win32, they are affected if there is an "Only" unicode build" with no backwards compatibility layer.

For those who do not know: "kichik" knows possibly the best what path to choose, and if he doesn't he still does 😉 after all he does a major amount of the NSIS development.

But I also see the points by Joost, Unicode only major release to avoid confusing people which NSIS to use, and after all still 2.x can be used for non Unicode releases.

And for the plugins, most of them come with source, should we already ask the plugin creaters nicely to make them Unicode compatible if needed and not done so already?

...so now MY first step will be, make the plugins I've created unicode compatible, even if not used yet...

Windows 95/98/Me support would require NSIS itself and all plug-ins to provide both an ANSI and a Unicode user interface. If external APIs are used to modify the interface, scripts would also have to check whether ANSI or Unicode is used and do the required conversions.

All scripts need to updated anyway if NSIS uses Unicode internally, even if the user interface is still ANSI. The fact that an ANSI interface is still available would only improve backwards compatibility a little.

Because this is all very complicated, I'm afraid we don't have the manpower to implement it within a reasonable time.

Almost all new applications that are developed (e.g. Firefox 3) don't support 95/98/Me anymore. If legacy Windows versions still need to be supported for an installer, NSIS 2 can be used.

As an interested outsider here, I can say I would largely prefer having an officially maintained Unicode NSIS (NSIS3), with an unofficial ANSI legacy version (for example if new features would be backported to NSIS2). That is to say, of course, if a choice has to be made, since limited resources always push to make choices.

I think both NSIS 2 (ANSI) and NSIS 3 (Unicode) can remain official versions. However, we won't really have enough resources to backport features.

If we use TCHAR style macros in the C/C++ source, we can use one set of source files to generate both the ANSI and Unicode binaries. I know some people don't like the use of TCHARs but that's the choice I made because of the convenience of having one common set of source files for both binaries. So I know I can make changes to the source that will be reflected in both the ANSI and the Unicode versions. This should help in keeping NSIS ANSI and NSIS Unicode features in sync.

The other problem is the script files -- it's painful keeping two sets of script files as kichik pointed out. But I think most of the behavior enhancing script files are plain ASCII files which can easily be converted to Unicode and back to ANSI.

The language files, however, I think we need to keep separate because the two sets will have to be different. For one, the number of Unicode language files will be bigger than the ANSI. In fact, it already is in my build. And two, the Unicode script files can have the language name itself in its vernacular because the Unicode binaries can show them all at the same time.

Consider the multi-language installer demo picture on my website. It's a very powerful argument for being able to generate Unicode installers. I know of one case where a software targeted to the Chinese audience had two installers -- one in Traditional Chinese and the other Simplified Chinese. And they were able to use the NSIS Unicode multi-language installer to show the instructions / license agreement in both Traditional and Simplified at the same time. That's over a billion people that can benefit from that sort of flexibility. India also has a huge set of scripts for its languages. With China and India together, you've got almost half of the world's population! 🙂

A mixed ANSI/Unicode (ANSI on 95/98/Me and Unicode on NT/2000/XP/2003/Vista) stub is almost impossible as I pointed out, so building ANSI-only and Unicode-only stubs from the same source may indeed be a good option!

If the compiler would automatically convert Unicode scripts to ANSI (using multiple codepages in a script if necessary, just like the situation is right now), language files and example scripts not using special features would be compatible with both versions (solving the issue kichik pointed out). A simple define (!ifdef UNICODE ...) can allow more advanced scripts (such as the Modern UI) to support both stubs. A script command (independent of the script encoding) can then be used to select what stubs should be used.

We would need two folders for the plug-ins, because the ANSI stubs can only use ANSI plug-ins and the Unicode stubs can only use Unicode plug-ins. All standard NSIS plug-ins can of course support both (using the TCHAR thing).

You'll have to ask kichik about his opinion. Maybe you can work on implementing Unicode in a different branch of the official NSIS SVN repository.

What about FileOpen function in Unicode version? If a file does not exist, does it create an ANSI or Unicode file? There should be a way of creating an Unicode file, and maybe there is, but I didn't find it. Sorry if my question is out of place here, but I did try general forum and got no replies.

Use FileOpen and use FileWriteUTF16LE. If you want to write a BOM at the beginning of the file, you should be able to using FileWriteWord file_handle "65279" which is U+FEFF

Did that and it works perfect, thanks. However, if I try same approach with header I'm tinkering with to make it work with Unicode version of NSIS, I get garbage in file. I'm no expert in NSIS, but I have tried to find the cause of it, and I think that usage of FileJoin from TextFunc.nsh is causing the garbage in final file, since header uses that function to join two files. Files are utf16le, and here is a part of the function :


define ${_TEXTFUNC_UN}FileJoin `!insertmacro ${_TEXTFUNC_UN}FileJoinCall`
...
FileOpen $3 $4 a
IfErrors error
FileSeek $3 -1 END
FileRead $3 $5
StrCmp $5 '$\r' +3
StrCmp $5 '$\n' +2
FileWrite $3 '$\r$\n'
FileOpen $0 $1 r
IfErrors error
FileRead $0 $5
IfErrors +3
FileWrite $3 $5
goto -3
FileClose $0
FileClose $3
...

Maybe the bold part is causing problems? Should there be FileWriteUTF16LE and FileReadUTF16LE calls? Or maybe not, beacuse of the define? I tried to place UTF functions, but this function produces no file then. Any thoughts, Jim?

If you want this function to work for Unicode files, I think you'll have to create a Unicode version. Copy the macro and as you suspected, you need to convert the FileRead and FileWrite calls to the UTF16LE versions. You might have to open the files as binary as well.

Got it working. Thanks for all the help and sorry for bugging you. The function does not work with Unicode files - had to change calls, that was obviuos. But this script with UTF produces errors, and I am not sure why - so I had to do this:
(changes are bolded).

FileOpen $3 $4 a
;IfErrors error
FileSeek $3 -1 END
FileRead $3 $5
StrCmp $5 '$\r' +3
StrCmp $5 '$\n' +2
FileWrite $3 '$\r$\n'
FileOpen $0 $1 r
;IfErrors error
ClearErrors
FileReadUTF16LE $0 $5
IfErrors +3
FileWriteUTF16LE $3 $5
goto -3
FileClose $0
FileClose $3

The header in question is Advanced Uninstall Log, and I'm going to send changes to RedWine. I also had to change Serbian.nlf as UTF16LE is a pile of rubbish.
This
Ďđčőâŕňŕě óńëîâĺ äîăîâîđŕ î ďđŕâó ęîđčřžĺśŕ
should be, in fact, this
Прихватам услове договора о праву коришћења
but I guess it's all Greek to you. 😁
Anyway, thanks, Jim - it sure is great to have a Unicode installer which produces Unicode log, and I can install in Unicode named folder, and uninstall files automatically.
Going to send you correct Serbian.nlf.

I appreciate that. Looking forward to getting those files. It's good to know that the Unicode version of NSIS is actively being used.

NSIS Unicode has been updated to 2.35. Please go to the usual spot to download the binaries and the modified source: http://www.scratchpaper.com.

Is there any particular reason why there are no registry entries at HKLM -> Software -> NSIS for VersionMajor and VersionMinor? ANSI release writes current release values there. Is there another way to get version of NSIS with which the script was compiled?
Regards.

It's because I've been building without specifying revision number and build number. I'm surprised you're the only one who brought this up. Thank you. The files have been updated on my site. Now NSIS Unicode will put the version info under Software/NSIS/Unicode.

Thanks. I was using this to read the version of NSIS I've used for buliding the installer so noticed it was missing...

Is there a way to detect if the compiler is unicode, meaning, is there a defined symbol like __NSIS_UNICODE__ in the unicode build? If not, you should add it so that 3rd party header files for stuff can support both ansi and unicode in one file

That's not a bad idea. I will add it to the next version.

I just made the tweak and modified the binary on my website to include the NSIS_UNICODE predefine for the Unicode build. Enjoy.

7zip will not open the installer archive properly. I get rubbish filenames. Not sure if it's something that 7zip would need to fix, though.

7-zip expects a very specific format from the installer. One of its assumptions is that the string table is saved as ANSI. It's not such a faulty assumption when you add in the no-real-nsis-binary-specification factor.

So does the installer still work? Is it just that a 3rd party app can't open the Unicode-NSIS generated file? If so, then I don't think we can do much about that. If the installer doesn't work with 7zip compression algorithm, that's something I can look into. But the notion I had was that NSIS used those compression algorithms for its own nefarious purposes, not that it had to adhere to some binary format.

The installer works great. It's just that I often use 7zip's ability to open the installer in order to extract the binaries for crash dump debugging. Not a big deal, but I miss the feature 🙂 Maybe there's a way for the 7zip author to add special handling for Unicode NSIS.

A helpful hint to anyone converting their NSIS scripts to Unicode. Be sure to change your System::Call stuff to use w. instead of t. for string types!

but is that the way it should work? The system plugin should probably be changed so that "t"="w" in the unicode build and you can use "a" for forcing ansi strings. Should help keep things portable, but some stuff might break

Yeah, I agree with that idea. Although 'a' is already taken to be a shortcut for the chosen language

"a" is not taken as a param type, only source/destination, but I'm not sure if the parser could handle using "a" for two different things. using @ might be a little weird, maybe "z" for zero terminated or whatever unless someone has a better idea

OT: also, adding support for "p" for pointers might ease a future move to 64 bit

Hello jimpark,

Your unicode nsis has been of immense help to me. There seems to be a slight problem with the language selection dialog - the languages that aren't supported by the system also show up and I'm NOT using MUI_LANGDLL_ALLLANGUAGES .

For example , I'm running the installer on Windows XP with English as the OS language (and I haven't installed the files required for displaying CJK languages). On running the installer , Japanese (actually garbled boxes) also appears in the drop down menu .

It works as expected on my system where i have installed the files for Japanese i.e., both English and Japanese are displayed.

It will be great if you can help.