Unicode

2.45 is out!

I've just released the Unicode version of NSIS built around NSIS 2.45 along with the source code and an ANSI version built from the same source. You can find it at http://www.scratchpaper.com as usual.

And I recently created a plugin that can call a ansi plugin in a unicode installer. It is very much in the testing stage, so get it @ http://nsis.sourceforge.net/CallAnsiPlugin_plug-in and let me know of any problems

Re: 2.45 is out!

Originally posted by jimpark
I've just released the Unicode version of NSIS built around NSIS 2.45 along with the source code and an ANSI version built from the same source. You can find it at http://www.scratchpaper.com as usual.

Congratulations for the new release. I don't understand why this is not merged into mainline.

The code is using the generic-text mappings found in tchar.h (http://msdn.microsoft.com/en-us/libr...8VS.71%29.aspx). This is the documented way of having the same code compiled for ansi and unicode on Windows platforms.

Cheers,
Cristian.

I have a small problem with NSISu: it does always print messages about including files.

The makensis log is full on inclusions like the one below and I discovered that using /V does not help. Is there any way to hide this and still be able to log any warnings or errors?

Opening 'C:\dev\pj\src\contrib\nsis\Contrib\Language files\English.nsh' as UTF-16LE.
File 'C:\dev\pj\src\contrib\nsis\Contrib\Language files\English.nsh' has a BOM marked as UTF-16LE.

Fonts

I'm writing a multilingual install script that needs to support English, Chinese, Japanese, and Turkish among others. I've been able to convert my nsi files to utf-16.

However it seems that the font that the installer is displayed in doesn't have the Japanese characters that I need, instead I just see boxes where the characters should be (the Turkish seems to render fine). I know the computer must have some font that displays these Japanese characters because when I open the .nsi file in wordpad.exe I can view the characters.

Any suggestions? Is it possible to make the installer force display in a certain font? Thanks.

Either create a unicode installer, or set your windows's default codepage for non-unicode applications to Japanese. (Or run the installer through applocale, which is equivalent.)

ngroman, are you posting your problem here because this is a problem you are seeing in the Unicode NSIS? Or are you trying to use the standard NSIS to achieve this? With the standard NSIS, you aren't going to be able to create one installer that can display all those languages together. You'll need Unicode NSIS for that, as MSG stated.

jimpark:
I'm using nsis-2.45-Unicode to compile my .nsi file. My nsi script file is encoded in utf-16LE (I saved it as unicode in wordpad). I verified with chardet in python that it is in fact saved as utf-16LE. Also the output from makensis showed that my file was being opened in utf-16 mode.

The computer I'm running this has XP with the MUI packs installed for French, German, and Spanish (I do not have the MUI for any Asian languages). However the fact that I can see the Japanese characters in wordpad means there is something on my computer that can display these characters.

Have you tried installing the Asian language support? If not, why not?

I realize that I could do that, and it would probably solve my issue...for me; however, most of my users will have standard XP installs that don't have Asian language support either. If I'm going to show a drop-down menu of languages it wouldn't look very good to have some of the languages show up as a bunch of blank boxes to some users.

The NLF files in "Contrib\Language files" specifies which font to use. These have been chosen by the translators so that they look right. But if you want to change the font for a language, you do have the option of using SetFont /LANG=lang_id.

Originally posted by ngroman
I realize that I could do that, and it would probably solve my issue...for me; however, most of my users will have standard XP installs that don't have Asian language support either. If I'm going to show a drop-down menu of languages it wouldn't look very good to have some of the languages show up as a bunch of blank boxes to some users.

Don't show the drop down list. Just detect locale or Windows UI language and auto set installer language to be consistent with locale/UI language.

Originally posted by grzech
Don't show the drop down list. Just detect locale or Windows UI language and auto set installer language to be consistent with locale/UI language.

Autosetting the language to the Windows locale is very bad practice. I for one am running in Japanese locale, while I can't read Japanese without a dictionary (and copy-paste, at that). I keep Windows in Japanese locale because of our translation projects, but it is also very common practice in general in communities of Asian pop-culture fans.

Installers such as nVidia's drivers are extremely annoying in doing everything in Japanese automatically, so the user ends up having to guess which buttons to push.

Yeah, autosetting to Windows UI language is safer. Guys who can't read Japanese rather won't be running Japanese Windows. 😁

I agree with MSG on that one. I hate it when the installer thinks it knows in which language I'd prefer to read my installation instructions. I may have the locale set to some other language that I can read and write but not comfortable enough to speak "geek" in. I'd much rather be given a choice: at the very least, the locale or English? (Okay, maybe I'm being English-centric here.) Or maybe at the download site, I should be given a choice as to which language installer I want.

Originally posted by sealite
I have a small problem with NSISu: it does always print messages about including files.

The makensis log is full on inclusions like the one below and I discovered that using /V does not help. Is there any way to hide this and still be able to log any warnings or errors?

Opening 'C:\dev\pj\src\contrib\nsis\Contrib\Language files\English.nsh' as UTF-16LE.
File 'C:\dev\pj\src\contrib\nsis\Contrib\Language files\English.nsh' has a BOM marked as UTF-16LE.

I have this problem too... It's very annoying... I want to turn off this messages, because they spamming my command window. How can I do it?

jimpark:

All of your downloads have a wrong file extension. .exec instead of .exe.

You did a great job, except for some old plugins, like delay.dll, which won't work, it's just great. I've converted all my scripts to Unicode, including the Essentials Pack.

The abilities with the Unicode version are just awesome.

Keep on your good work, I hope you'll provide updates in future too.

-Chris

In the next release I will try to address both of the issues you've listed above. The "exec" extension is just because of google sites which keeps me from uploading anything with the .exe extension as I've explained on the site. I may have to find another server, though, because it is highly annoying.

Originally posted by scully13
It appears that somebody did add Unicode support to the AccessControl plug-in but the download is missing from the plug-in page. http://nsis.sourceforge.net/AccessControl_plug-in
Does anybody have a copy of this somewhere or know of something else I can use from the Unicode install to change file user permissions?

I just a made a NSIS-Unicode port of it, and uploaded it on

Originally posted by Anders
When hwndNSIS is non NULL you can check that, but if you also support .onInit or silent installers, the only thing I could come up with was checking $pluginsdir:
if the 2nd BYTE is 0, you can be (pretty) sure its UNICODE

Hi Anders, I don't see how this could work :
To reach variable INST_LANG+2, you need to compute g_variables[(INST_LANG+2)*g_stringsize]
but you don't know if you're dealing with CHAR or TCHAR g_variables (which would need to multiply the index by 2)

@Wizou: If you check my BgWorker plugin, you can see that I'm checking for kernel32::lstrcatW in the import table.

I also have some sample code in there that checks $pluginsdir, but like you said, you don't know the offset, so it has to use IsBadReadPtr() and there could be false positives.

Unfortunately, it turns out that writing hybrid plugins is a huge pain in the ass, and I can't recommend doing it on anything except very simple plugins

For the next release, I plan on adding to pluginapi.h:

// True if NSIS is built with Unicode support.
bool NSISCALL IsUnicodeNSIS();

Also, Olivier noted that you could use IsWindowUnicode(hwnd). But this may not work on silent installers.

However, I would still encourage that the plugin writers use TCHARs or a strategy similar to it and compile and link with the right Unicode declarations and definitions. I know there is a temptation to try to generate a single binary that could work on both the ANSI and the Unicode NSIS variants but that seems a bit dangerous to me.

@Anders: IsBadReadPtr won't help you here as it will be valid memory range (.bss variables area), just you're not checking a valid Unicode 2nd byte.
Checking 'lstrcatW' import from main module is however a good idea 🙂

@jim park: I still wonder how you would create such a IsUnicodeNSIS, unless you pass special exec_flags_t value from NSIS to the DLL


If we have such a way for the DLL to detect the version of NSIS, I'm thinking about creating a new pluginapi.h/pluginapi.lib that could transparently convert variables and stack from ANSI or Unicode to the plugins choice depending on whether it is called from Unicode-NSIS or not
This would make plugins creator's life easier.

@jimpark: What about turning on/off "messages about including files" ? NSIS Unicode is great tool and I ask You to add this option, please.

Noooooooooo, we don't need a IsUnicodeNSIS() function or any kind of pluginapi helpers to do both.

It has already been done with the unicode layer for win9x, just use that.

I'm with Jim on this, plugin devs should use TCHAR and compile two versions. There is also my CallAnsiPlugin plugin for older stuff with no source.

@Wizou: well, I'm sure it can be done without checking the import table. The IsBadReadPtr usage was just a big hack and I never finished that type of detection.

@Wizou: That's a good point. It's very difficult to write an IsUnicodeNSIS(). The only thing I could do is try validating the string (if anything) to see if it's good UTF16LE. Look in validateunicode.cpp which is in Source. You'll see a class there that can validate UTF16LE/BE as well as UTF8.

@Anders: I agree with you in that this API, even if accurate, is a bad idea. I'm pulling IsUnicodeNSIS() from pluginapi.h. It will NOT be in the next release.

Does someone know why I'm always getting empty unicode files when I'm using:

unicode::FileUnicode2Ansi "${_HDDS_countFile}" "${_HDDS_countFile}_tmp" AUTO

Pop $R0

${If} "$R0" != 0

  # error handling

${EndIf}

The source files type is "UTF-16LE|UCS-2LE".

-- Problem solved ---

I was using Unicode plugin v1.1.
I've tried version 1.0 and everything is fine now.

If we are going to have any chance of a merge, the examples and header files need to be 100% the same, meaning that both the current A and U builds need some changes:

*The ansi version needs to change its handling of &wXX so that XX is a count of characters and not bytes (and update the docs) (The unicode version already has this fixed)

*The unicode system plugin needs to default to the W version of functions when autodetecting (foo >> fooW and not fooA)


To reduce the NSIS_MAX_STRLEN confusion, we should probably make it really clear in the docs that NSIS_MAX_STRLEN is a count of TCHAR's (Or maybe even rename it). We should also add a predefine for the count of bytes, I suggest we name it NSIS_MAX_STRCB

Both versions currently fail this little test script

Originally posted by Anders
*The unicode system plugin needs to default to the W version of functions when autodetecting (foo >> fooW and not fooA)

We actually do default to the "W" version. Let me explain a little with how the "T" versions of the functions work for those who don't know.

We write in our code:

::CreateFile(...);

When you look at the definition of CreateFile, you see that there really is no function called "CreateFile" but two functions called "CreateFileW" and "CreateFileA." This is true for MOST of the functions that have both the A and the W versions.

#ifdef UNICODE

#define CreateFile  CreateFileW

#else

#define CreateFile  CreateFileA

#endif // !UNICODE

There are functions which only have one name, with no W or A prefixes.

So the logic that is in the system plugin currently is to try to load the function with the name as is, then failing that, try to add "W" for the Unicode build (or "A" for the ANSI build) and try again.

Unfortunately, for older string functions like lstrlen, the lib you are trying to load DOES have a lstrlen which is ANSI only. It's probably there for backward compatibility since it mimics standard library string functions. So for lstrlen, you need to specify kernel::lstrlenW. But as far as I've seen, it's only these string functions that look and behave like the standard library ones that have this problem.

A possible change could be that we always suffix with a "W" or "A" and see if that loads something and try that function first before we try the name of the function unmodified, but I'm uncomfortable with that since I'd rather give the programmer the benefit of the doubt and let him/her decide what s/he really wants to load.

I don't mind the plugin second guessing after failures but I don't want it to think it's smarter than the programmer.

To reduce the NSIS_MAX_STRLEN confusion, we should probably make it really clear in the docs that NSIS_MAX_STRLEN is a count of TCHAR's (Or maybe even rename it). We should also add a predefine for the count of bytes, I suggest we name it NSIS_MAX_STRCB

Both versions currently fail this little test script [/B]

I agree that we should note that NSIS_MAX_STRLEN is in TCHARS. But going through the documentation, they all state that NSIS_MAX_STRLEN means a number of characters, not bytes. So I'm actually okay with the documentation as-is.

As for adding NSIS_MAX_STRCB, I think that's a great idea, although, it can easily be calculated by NSIS_MAX_STRLEN * NSIS_CHAR_SIZE. I like NSIS_MAX_STR_BYTES better as a name, though.

Unless the NSIS trunk has NSIS_MAX_STR_BYTES, it would be useless for me to add it just to the Unicode version as it's meant for portability across the two versions.

So perhaps we can request NSIS_MAX_STR_BYTES to be added to the NSIS trunk?

For your information, I'm starting to merge Unicode port of NSIS version into NSIS repository.

I will try to take the most of what has been discussed here into account.
However my plan is first to make it the most transparent possible for the user and the plug-ins developer.