Anouther Newbish System Plugin Question

I have a DLL called SbAdmDll.dll.

The documentation has a function defined as follows...
Here is what I interpreted is to be using NSIS....

InitPluginsDir
SetOutPath $PLUGINSDIR
File SbAdmDll.dll
StrCpy $1 `${XMLStuff}`
ClearErrors
System::Call `SbAdmDll::SBADM_execute(w r1,i 0)w .r0`
DetailPrint $0 

The returned value is "error" which tells me that system::call is choking on something... Any ideas what I may be doing wrong here?

Check with Depends that your entypoint is propperly exported.

I think I found the problem...

I'm recreating the NSIS script using AutoIt's DLL calls. AutoIt is telling me that there are missing DLLs. Good to know...

I guess I'll be prototyping all my NSIS scripts with AutoIt from now on.

I checked with Dependency Walker and everything is in place now but now I'm getting a "function not found in the DLL file" error although I'm using the named as supplied by Dependency Walker:

unsigned short * SBADM_execute(unsigned short const *,int (*)(unsigned long,void *))

I have the function entry point address.. Can I use that in place of the name? At least for this instance / version of the dll?
Dose the system plugin allow for this? If so who would I do it...

Not really sure how it can return a string (is it just a pointer into exec_xml?) anyway, you don't provide any information about the calling convention, so maybe its cdecl (system plugin uses stdcall by default) try: System::Call 'SbAdmDll::SBADM_execute(w r1,i 0)i.r0 ? c'

I give up... I think there is something seriously wrong with this dll.. I can't even get the thing to execute in AutoIt.

I'm in touch with the vendor but they are feeding me crap about using the COM DLL instead. (Note the COM DLL requires the DLL I'm trying to call So I would be adding 36K and increasing the complexity exponentially)

I assume you are talking about a Mcafee thing, in that case, I also found references to another function: SBADMDLL_API void SBADM_free_string(LPWSTR str) so then it makes sense how the first call can return a string. Now we just need to know what SBADMDLL_API is defined as. Do you have a header file or something?

Your correct...

I'm trying to get examples from McAfee but all I have at the moment is the PDF Scripting Guide provided by McAfee. Here is an excerpt...

SBADM_execute

“extern C” LPWSTR

SBADM_execute(

    LPCWSTR  exec_xml,

    SBADM_CALLBACK callback

);

Parameters:

SBADM_CALLBACK_CODE_TRACE_A

Return value:

Yeah I saw that already, but like I said, I posted some code that _should_ work, if not, try it without ?c

(Could you maybe upload this dll somewhere so I could take a look at it?)

Originally Posted by Anders View Post

Yeah I saw that already, but like I said, I posted some code that _should_ work, if not, try it without ?c

(Could you maybe upload this dll somewhere so I could take a look at it?)

With and without the ?c option it fails.

Sadly, I don't think I can share the DLL as it is a part of the server installation for the client software I'm managing. I'm pretty sure McAfee's lawyers might grunt in disapproval.

I've ran the DLL through a assembler decompiler in hope I might gleam some insight. I didn't find anything gleamingly obvious but I did get these...
The Undecorated function name is: "?SBADM_execute@@YAPAGPBGP6AHKPAX@Z@Z"
The Address is: 0x0001eab0
The Ordinal is: 0x1

I have no basis except for intuition, but it feels like the issue is with the function name lookup. Can the System plugin call dll functions by the ordinal or address values?Looking at the documentation I'll give GetModuleHandle a try and see if it will work.

For the time being, I'm using the command-line utility in addition to the DLL. I seams stupid that I'm forced to add an exe wrapper to call an added dll that I would call directly if I knew how. 🙁

I'm hoping McAfee will provide me with some actual C usage examples so I can get this properly implemented.


Thank you soo much for your assistance.. this one it turning into a problem I'm going to have to solve on my own but your encouraging nudges do help 😁

Here is an interesting quip I conjured I feel is partially relevant, "Bugs are only introduced when you implement someone else's code." It's not actually true but I thought it funny.

?SBADM_execute@@YAPAGPBGP6AHKPAX@Z@Z == unsigned short * __cdecl SBADM_execute(unsigned short const *,int (__cdecl*)(unsigned long,void *)) 

...it is cdecl so you need ?c

For fun I compiled a simple dll

#include <Windows.h>
__declspec(dllexport) unsigned short * __cdecl SBADM_execute(unsigned short const *xml,int (__cdecl*)(unsigned long,void *)) 
{
    MessageBoxW(0,L"hello world",xml,0);
    return 0;
}
BOOL WINAPI DllMain(HANDLE,DWORD,DWORD) {return true;} 

And I'm able to call it with

outfile test.exe
section
initpluginsdir
setoutpath $pluginsdir
file tempdev.dll
system::call 'kernel32::LoadLibrary(t "$pluginsdir\tempdev.dll")i.r0'
system::call 'kernel32::GetProcAddress(i r0,i 1)i.r2' ;ordinal 1
System::Call '::$2(w "xml w00t",i0)i ?c'
system::call 'kernel32::FreeLibrary(i r0)'
setoutpath $temp
sectionend 

OMG 😐 THANK YOU!!!

It finally works!!!

outfile test.exe
section
setoutpath "C:\Program Files\McAfee\Endpoint Encryption for PC"
system::call 'kernel32::LoadLibrary(t "$OUTDIR\SbAdmDll.dll")i.r0'
system::call 'kernel32::GetProcAddress(i r0,i 1)i.r2' ;ordinal 1
System::Call '::$2(w "<SafeBoot><SbAdminScripting><SbAdminCommand><Command>GetVersion</Command></SbAdminCommand></SbAdminScripting></SafeBoot>",i0)w .r3 ?c'
DetailPrint $3
system::call 'kernel32::FreeLibrary(i r0)'
sectionend 

Output folder: C:\Program Files\McAfee\Endpoint Encryption for PC

?<?xml version="1.0"?>

  <SafeBoot>

    <SbAdminScripting>

      <SbAdminConnectionResult>

        <ResultCode>0x00000000</ResultCode>

        <ResultDescription>The operation completed successfully.

</ResultDescription>

      </SbAdminConnectionResult>

      <SbAdminCommandResult>

        <Command>GetVersion</Command>

        <ResultCode>0x00000000</ResultCode>

        <ResultDescription>The operation completed successfully.

</ResultDescription>

        <Version>5.2.3.5</Version>

      </SbAdminCommandResult>

    </SbAdminScripting>

  </SafeBoot>



Completed

You should probably call SBADM_free_string to free the returned string (If you only call it once, its probably ok just to leak that memory, but if you call it a lot you might want to free the strings) (But when doing that, the return needs to be i to preserve the pointer, then convert the returned string to a "nsis string" with the system plugin struct syntax)

Ok, now that we have it working I need to try and address the issue of extracting the DLL into the client system. We don't want to deploy the DLL, only use it to make the change and then ensure the DLL is removed to attempt to prevent any possible security issues.

I found the SetDLLDirectory api call and it explains exactly what I need but the final call to the SbAdmDll.dll still fails error with "MSVC++ Runtime Library RuntimeError...abnormal program termination".

section
    ;!define UseSetDLLDirectory
    !ifdef UseSetDLLDirectory
        ## Use the SetDLLDirectory API call to temporarily add the client programs path to the DLL search order.
        ##  MSDN: http://msdn.microsoft.com/en-us/library/ms686203%28VS.85%29.aspx
            InitPluginsDir
            SetOutPath $PLUGINSDIR
            
            file "SbAdmDll.dll"
            System::Call 'Kernel32::SetDllDirectory(t "C:\Program Files\McAfee\Endpoint Encryption for PC")i .r0'
            DetailPrint "SetDllDirectory RC=$0" ;<-- Returns 1 so it is a success
    !else
        ## Extract to actual client application path
            SetOutPath "C:\Program Files\McAfee\Endpoint Encryption for PC"
            file "SbAdmDll.dll"
    !endif
    
    System::Call 'kernel32::LoadLibrary(t "$OUTDIR\SbAdmDll.dll")i.r0'
    System::Call 'kernel32::GetProcAddress(i r0,i 1)i.r2' ;ordinal 1
    System::Call '::$2(w "<SafeBoot><SbAdminScripting><SbAdminCommand><Command>GetVersion</Command></SbAdminCommand></SbAdminScripting></SafeBoot>",i0)w .r3 ?c'
    DetailPrint $3
    
    System::Call 'kernel32::FreeLibrary(i r0)'
    
sectionend 

The call works fine when I don't define the UseSetDLLDirectory... any ideas?

I don't see why you need to use SetDllDirectory.

What about:
SetOutPath $PLUGINSDIR
file "SbAdmDll.dll"
SetOutPath c:\programfiles...
System::Call 'kernel32::LoadLibrary(t "$PLUGINSDIR \SbAdmDll.dll")i.r0'
?

... Yeah, that is a lot simpler isn't it?

Working good now 😁

I think I know the answer but I'll ask to verify.

The SBADM_free_string function needed to free the memory doesn't have a return value so I'm uncertain if it succeeded.

void __cdecl SBADM_free_string(unsigned short *}

I tried reading the memory value again so see if it was cleared, gives me an error, or is different. But the value is exactly the same as it was before the call to release the memory.

System::Call `*$3(&w${NSIS_MAX_STRLEN} .r4)` 

I have two possible conclusions....

1. The API call to free the memory is failing.
2. The API call succeeded but the value was left intact at the address.

#2 worries me as it implies that the DLL developers are leaving potentially private data as plain text in the memory space to be easily scanned.

I'm going to try and create a controlled memory and compare the results of the program to see if it is indeed freeing the memory.


UPDATE: I see that the FreeLibrary function is freeing the memory but I think the api call is failing as the memory increases about the same with and without the free_string api call. I'm using....

System::Call 'kernel32::GetProcAddress(i r0,i 2)i.r2' ;ordinal 2
System::Call '::$2(i $3) ?c' 

UPDATE: I fixed a typo in my code and ran the loops again... The API is indeed releasing the memory as I'm seeing almost a 1MB memory size difference when looping 1000 times with and without the free string function. It looks like the memory is indeed left intact. That sucks.. I suppose I could let McAfee know but is it really an issue or is this common practice?

System::Call '::$2(i *$3) ?c' is wrong, no need for *, just "i" is what you need ($3 is a memory address already)

Edit: Did you edit the * out of your post or Icanhazgonecrazy?