Anouther Newbish System Plugin Question

Archive: Anouther Newbish System Plugin Question

Zinthose
15th September 2010 03:18 UTC

Anouther Newbish System Plugin Question
I have a DLL called SbAdmDll.dll.

The documentation has a function defined as follows...

“extern C” LPWSTR 
SBADM_execute( 
   LPCWSTR  exec_xml, 
   SBADM_CALLBACK callback 
);
exec_xml [In] Pointer to a null terminated Unicode string that contains the XML of the command to execute.

callback [In] Optional pointer to a callback function that can be used to receive additional progress and information. This can be NULL if no callback is required. This is primarily intended for use by the command line interface to display more information.

Here is what I interpreted is to be using NSIS....

InitPluginsDir

SetOutPath $PLUGINSDIR

File SbAdmDll.dll


StrCpy$1 `${XMLStuff}`


>ClearErrors

System::Call `SbAdmDll::SBADM_execute(w r1,i 0)w .r0`

>DetailPrint $0 

>

The returned value is "error" which tells me that system::call is choking on something... Any ideas what I may be doing wrong here?

punkomat
15th September 2010 09:57 UTC

Check with Depends that your entypoint is propperly exported.

Zinthose
15th September 2010 13:56 UTC

I think I found the problem...

I'm recreating the NSIS script using AutoIt's DLL calls. AutoIt is telling me that there are missing DLLs. Good to know...

I guess I'll be prototyping all my NSIS scripts with AutoIt from now on.

Zinthose
16th September 2010 22:01 UTC

I checked with Dependency Walker and everything is in place now but now I'm getting a "function not found in the DLL file" error although I'm using the named as supplied by Dependency Walker:

unsigned short * SBADM_execute(unsigned short const *,int (*)(unsigned long,void *))

I have the function entry point address.. Can I use that in place of the name? At least for this instance / version of the dll?
Dose the system plugin allow for this? If so who would I do it...

Anders
16th September 2010 22:53 UTC

Not really sure how it can return a string (is it just a pointer into exec_xml?) anyway, you don't provide any information about the calling convention, so maybe its cdecl (system plugin uses stdcall by default) try: System::Call 'SbAdmDll::SBADM_execute(w r1,i 0)i.r0 ? c'

Zinthose
17th September 2010 16:21 UTC

I give up... I think there is something seriously wrong with this dll.. I can't even get the thing to execute in AutoIt.

I'm in touch with the vendor but they are feeding me crap about using the COM DLL instead. (Note the COM DLL requires the DLL I'm trying to call So I would be adding 36K and increasing the complexity exponentially)

Anders
17th September 2010 18:40 UTC

I assume you are talking about a Mcafee thing, in that case, I also found references to another function: SBADMDLL_API void SBADM_free_string(LPWSTR str) so then it makes sense how the first call can return a string. Now we just need to know what SBADMDLL_API is defined as. Do you have a header file or something?

Zinthose
17th September 2010 22:11 UTC

Your correct...

I'm trying to get examples from McAfee but all I have at the moment is the PDF Scripting Guide provided by McAfee. Here is an excerpt...

SBADM_execute

This is the main function via which all the commands are “executed”.

“extern C” LPWSTR
SBADM_execute(
    LPCWSTR  exec_xml,
    SBADM_CALLBACK callback
);

Parameters:

exec_xml ___[In] Pointer to a null terminated Unicode string that contains the XML of the command to execute (see section 3 for the details of XML format).
callback ___[In] Optional pointer to a callback function that can be used to receive additional progress and information. This can be NULL if no callback is required. This is primarily intended for use by the command line interface to display more information. The supported callback codes are:

SBADM_CALLBACK_CODE_TRACE_A

Return value:

Pointer to a null terminated Unicode string that contains the result XML. If there was an error processing the input XML, then this function will return NULL.
The function will allocate the return buffer. The caller should use the SBADM_free_string() function to free the return buffer when it is no longer required.

Anders
18th September 2010 11:49 UTC

Yeah I saw that already, but like I said, I posted some code that _should_ work, if not, try it without ?c

(Could you maybe upload this dll somewhere so I could take a look at it?)

Zinthose
20th September 2010 15:56 UTC

Originally posted by Anders
Yeah I saw that already, but like I said, I posted some code that _should_ work, if not, try it without ?c

(Could you maybe upload this dll somewhere so I could take a look at it?)

With and without the ?c option it fails.

Sadly, I don't think I can share the DLL as it is a part of the server installation for the client software I'm managing. I'm pretty sure McAfee's lawyers might grunt in disapproval.

I've ran the DLL through a assembler decompiler in hope I might gleam some insight. I didn't find anything gleamingly obvious but I did get these...
The Undecorated function name is: "?SBADM_execute@@YAPAGPBGP6AHKPAX@Z@Z"
The Address is: 0x0001eab0
The Ordinal is: 0x1

I have no basis except for intuition, but it feels like the issue is with the function name lookup. ~~Can the System plugin call dll functions by the ordinal or address values?~~Looking at the documentation I'll give GetModuleHandle a try and see if it will work.

For the time being, I'm using the command-line utility in addition to the DLL. I seams stupid that I'm forced to add an exe wrapper to call an added dll that I would call directly if I knew how. :(

I'm hoping McAfee will provide me with some actual C usage examples so I can get this properly implemented.

Thank you soo much for your assistance.. this one it turning into a problem I'm going to have to solve on my own but your encouraging nudges do help :D

Here is an interesting quip I conjured I feel is partially relevant, "Bugs are only introduced when you implement someone else's code." It's not actually true but I thought it funny.

Anders
20th September 2010 22:11 UTC

*))

...it is cdecl so you need ?c

For fun I compiled a simple dll

#include <Windows.h>
__declspec(dllexport) unsigned short * __cdecl SBADM_execute(unsigned short const *xml,int (__cdecl*)(unsigned long,void *)) 
{
    MessageBoxW(0,L"hello world",xml,0);
    return 0;
}
BOOL WINAPI DllMain(HANDLE,DWORD,DWORD) {return true;}

And I'm able to call it with

outfile test.exe
section
initpluginsdir
setoutpath $pluginsdir
file tempdev.dll
system::call 'kernel32::LoadLibrary(t "$pluginsdir\tempdev.dll")i.r0'
system::call 'kernel32::GetProcAddress(i r0,i 1)i.r2' ;ordinal 1
System::Call '::$2(w "xml w00t",i0)i ?c'
system::call 'kernel32::FreeLibrary(i r0)'
setoutpath $temp
sectionend 

>

Zinthose
20th September 2010 22:57 UTC

OMG :stare: THANK YOU!!!

It finally works!!!


outfile test.exe

section

setoutpath "C:\Program Files\McAfee\Endpoint Encryption for PC"

>system::call 'kernel32::LoadLibrary(t "$OUTDIR\SbAdmDll.dll")i.r0'

>system::call 'kernel32::GetProcAddress(i r0,i 1)i.r2' ;ordinal 1

System::Call '::$2(w "<SafeBoot><SbAdminScripting><SbAdminCommand><Command>GetVersion</Command></SbAdminCommand></SbAdminScripting></SafeBoot>",i0)w .r3 ?c'

>DetailPrint $3

system::call 'kernel32::FreeLibrary(i r0)'

>sectionend 

>

Output folder: C:\Program Files\McAfee\Endpoint Encryption for PC
?<?xml version="1.0"?>
  <SafeBoot>
    <SbAdminScripting>
      <SbAdminConnectionResult>
        <ResultCode>0x00000000</ResultCode>
        <ResultDescription>The operation completed successfully.
</ResultDescription>
      </SbAdminConnectionResult>
      <SbAdminCommandResult>
        <Command>GetVersion</Command>
        <ResultCode>0x00000000</ResultCode>
        <ResultDescription>The operation completed successfully.
</ResultDescription>
        <Version>5.2.3.5</Version>
      </SbAdminCommandResult>
    </SbAdminScripting>
  </SafeBoot>

Completed

http://www.messentools.com/images/em...os-big-179.gif

Anders
20th September 2010 23:02 UTC

You should probably call SBADM_free_string to free the returned string (If you only call it once, its probably ok just to leak that memory, but if you call it a lot you might want to free the strings) (But when doing that, the return needs to be i to preserve the pointer, then convert the returned string to a "nsis string" with the system plugin struct syntax)

Zinthose
21st September 2010 16:51 UTC

Ok, now that we have it working I need to try and address the issue of extracting the DLL into the client system. We don't want to deploy the DLL, only use it to make the change and then ensure the DLL is removed to attempt to prevent any possible security issues.

I found the SetDLLDirectory api call and it explains exactly what I need but the final call to the SbAdmDll.dll still fails error with "MSVC++ Runtime Library RuntimeError...abnormal program termination".


section

   ;!define UseSetDLLDirectory

   !ifdef UseSetDLLDirectory

        ## Use the SetDLLDirectory API call to temporarily add the client programs path to the DLL search order.

        ##  MSDN: http://msdn.microsoft.com/en-us/library/ms686203%28VS.85%29.aspx

            InitPluginsDir

            SetOutPath $PLUGINSDIR

            

            file "SbAdmDll.dll"


           System::Call 'Kernel32::SetDllDirectory(t "C:\Program Files\McAfee\Endpoint Encryption for PC")i .r0'

           DetailPrint "SetDllDirectory RC=$0" ;<-- Returns 1 so it is a success

    !else

        ## Extract to actual client application path

           SetOutPath "C:\Program Files\McAfee\Endpoint Encryption for PC"

           file "SbAdmDll.dll"

    !endif

    

   System::Call 'kernel32::LoadLibrary(t "$OUTDIR\SbAdmDll.dll")i.r0'

   System::Call 'kernel32::GetProcAddress(i r0,i 1)i.r2' ;ordinal 1

    System::Call '::$2(w "<SafeBoot><SbAdminScripting><SbAdminCommand><Command>GetVersion</Command></SbAdminCommand></SbAdminScripting></SafeBoot>",i0)w .r3 ?c'

   DetailPrint $3

    

    System::Call 'kernel32::FreeLibrary(i r0)'

    

>sectionend 

>

The call works fine when I don't define the UseSetDLLDirectory... any ideas?

Anders
21st September 2010 17:01 UTC

I don't see why you need to use SetDllDirectory.

What about:
SetOutPath $PLUGINSDIR
file "SbAdmDll.dll"
SetOutPath c:\programfiles...
System::Call 'kernel32::LoadLibrary(t "$PLUGINSDIR \SbAdmDll.dll")i.r0'
?

Zinthose
21st September 2010 17:40 UTC

... Yeah, that is a lot simpler isn't it?

Working good now :D

Zinthose
21st September 2010 20:35 UTC

I think I know the answer but I'll ask to verify.

The SBADM_free_string function needed to free the memory doesn't have a return value so I'm uncertain if it succeeded.

void __cdecl SBADM_free_string(unsigned short *}

I tried reading the memory value again so see if it was cleared, gives me an error, or is different. But the value is exactly the same as it was before the call to release the memory.


System::Call `*$3(&w${NSIS_MAX_STRLEN} .r4)`

I have two possible conclusions....

The API call to free the memory is failing.
The API call succeeded but the value was left intact at the address.

#2 worries me as it implies that the DLL developers are leaving potentially private data as plain text in the memory space to be easily scanned.

I'm going to try and create a controlled memory and compare the results of the program to see if it is indeed freeing the memory.

UPDATE: I see that the FreeLibrary function is freeing the memory but I think the api call is failing as the memory increases about the same with and without the free_string api call. I'm using....

System::Call 'kernel32::GetProcAddress(i r0,i 2)i.r2' ;ordinal 2
System::Call '::$2(i $3) ?c'

UPDATE: I fixed a typo in my code and ran the loops again... The API is indeed releasing the memory as I'm seeing almost a 1MB memory size difference when looping 1000 times with and without the free string function. It looks like the memory is indeed left intact. That sucks.. I suppose I could let McAfee know but is it really an issue or is this common practice?

Anders
21st September 2010 21:52 UTC

System::Call '::$2(i *$3) ?c' is wrong, no need for *, just "i" is what you need ($3 is a memory address already)

Edit: Did you edit the * out of your post or Icanhazgonecrazy?