What's Wrong?

Function .onInit
IfFileExists "$INSTDIR\conf.dat" exit
exit:
MessageBox MB_ICONINFORMATION|MB_OK "A previous version is installed"
Abort
FunctionEnd

You must jump somewhere else if the file doesn't exist. Use the LogicLib for simpler code:

!include LogicLib.nsh



Function .onInit

${If} ${FileExists} $INSTDIR\conf.dat

MessageBox MB_OK "exists..."

Abort

${EndIf}

FunctionEnd

And for future posts, please at least include what exactly doesn't work as it should for you, instead of letting people guess.