How to Remove Or Replace Characters 11 Through 29 & 1 Through 9 On All Lines?

Archive: How to Remove Or Replace Characters 11 Through 29 & 1 Through 9 On All Lines?

vbgunz
12th April 2005 03:40 UTC

How to Remove Or Replace Characters 11 Through 29 & 1 Through 9 On All Lines?
I've got an extremely long text file and I only need to remove characters at certain positions on all lines. e.g. I am trying to close in on the tenth character so I need to remove characters 11 through 29 & 1 through 9 and I need to do this for all lines.

0002020107001010201010101020230201010201 SHEEP and 7 WOLVES
         ^

In the end, the above code should look like the following.

7 SHEEP and 7 WOLVES

Heres the catch. I've already accomplished this and it works perfectly BUT the process takes a very long 10 minutes on 800+ lines.

The only method I know on how to accomplish this is to:

1. open the file
2. scan the entire file
3. replace each line only once per scan
4. close file
5. repeat steps 1 through 4 800x

Can someone please optimize this savaged attempt with a Function that easily replaces x to n on all lines?

Thank you!

Jnuw
12th April 2005 06:43 UTC

vbgunz, I wrote the following (using Afrow UK's Read from text file line number function), and tested it on a text file (text.txt) that consisted of 800 lines of:

00020201070010102010101010202 SHEEP and 7 WOLVES
00020201070010102010101010202 SHEEP and 7 WOLVES
00020201070010102010101010202 SHEEP and 7 WOLVES
00020201070010102010101010202 SHEEP and 7 WOLVES
etc...

It created a new text file called text1.txt in 15 seconds consisting of 800 lines of:

7 SHEEP and 7 WOLVES
7 SHEEP and 7 WOLVES
7 SHEEP and 7 WOLVES
etc...

Once you have the new text file (text1.txt) you can copy it back over you original, or do with it as you please. Here is the code. Note you should only have to change the text file names and the end number (801). Hope this helps.


!include "MUI.nsh"
!insertmacro MUI_PAGE_INSTFILES
!insertmacro MUI_LANGUAGE "English"

; MUI end ------

OutFile "Setup.exe"
ShowInstDetails show

Section "MainSection" SEC01

StrCpy $1 0
Loop:
Intop $1 $1 + 1   ;Sets line number to be read below
StrCmp $1 801 End

Push $1 ;line number to read from
Push "text.txt" ;text file to read
Call ReadFileLine
Pop $0 ;output string (read from file.txt)

StrCpy $2 $0 "" 9 ;sets $2 = line $1 minus first 9 characters
StrCpy $3 $2 1 ;sets $3 to first character of $2
StrCpy $4 $2 "" 20 ;sets $4 = line $1 minus everything before character 30

FileOpen $5 Text1.txt a
FileSeek $5 0 END
FileWrite $5 $3
FileWrite $5 $4
FileClose $5
Goto Loop

End:
SectionEnd



Function ReadFileLine
Exch $0 ;file
Exch
Exch $1 ;line number
Push $2
Push $3

  FileOpen $2 $0 r
 StrCpy $3 0

Loop:
 IntOp $3 $3 + 1
  ClearErrors
  FileRead $2 $0
  IfErrors +2
 StrCmp $3 $1 0 loop
  FileClose $2

Pop $3
Pop $2
Pop $1
Exch $0
FunctionEnd

vbgunz
12th April 2005 08:11 UTC

First off, I'd like to say, Major Thanks! You've created the code with a lof of room for tweaks and I love it. I am enjoying it's flexibiliy and hope you publish this on the archive for others... (strange no one ever asked for this).

Jnuw, If possible can you please show me a method that will sort of do exactly what it is you do now with the code BUT write something onto the end of every line?

e.g. 7 SHEEP and 7 WOLVES!!!

How would I go about adding exclamation marks onto the end of every line?

Believe this or not you've already jumped me way ahead of myself. Help me with this last piece and the puzzles complete. I'd have nothing but major thanks for you all over the spot :)

Thank you again for your help!

Instructor
12th April 2005 08:12 UTC

Name "Output"
OutFile "Output.exe"

!include "TextFunc.nsh"
!insertmacro LineFind
!insertmacro TrimNewLines

Section
        ${LineFind} "C:\input.txt" "C:\output.txt" "1:-1" "LineFindCallback"

        IfErrors 0 +2
        MessageBox MB_OK "Error"
SectionEnd

Function LineFindCallback
        ${TrimNewLines} '$R9' $R9
        StrCpy $1 $R9 1 9
        StrCpy $R9 $R9 '' 29
        StrCpy $R9 '$1$R9!!!$\r$\n'

        Push $0
FunctionEnd

It created a new text file in 1 second consisting of 8000 lines of :)

Script used header

vbgunz
12th April 2005 08:51 UTC

OMG. That truly is amazing. Jnuw, thank you so much for helping me out, it's always priceless, appreciated and respected! You're trick was way better than mines 100 fold. Thanks again!

Instructor, that really was incredible! I never thought my text files would get processed so fast. man thats good ;) I have just one question which has really nothing to do with what you just provided as it is perfect.

My question is in regards to your "FileReadFromEnd v1.1 part 1.nsi" in which I am trying to use in conjunction with my current project. It seems every time I run that function it tends to append the second to last line onto the end of the last line BEFORE it actually reverses the files contents. I can't seem to work around it... Would you know why this might happen?

Instructor
12th April 2005 09:10 UTC

It seems every time I run that function it tends to append the second to last line onto the end of the last line BEFORE it actually reverses the files contents.

ou man it's so hard to read with my English :) Can you say it with other words (the best way with examples)

vbgunz
12th April 2005 09:27 UTC

I'll try to explain it :)

This is what the file looks like before it gets reversed.

1. a
2. a
3. a
4. a
5. a
6. a
7. a
8. a
9. a

This is what it looks like after it get reversed.

9. a8. a
7. a
6. a
5. a
4. a
3. a
2. a
1. a

This happens to the newly created file. If it helps I am using "example 2" but I have slightly modified it as so

      GetTempFileName $R0
         FileOpen $R1 $R0 w
          ${FileReadFromEnd} "$EXEDIR\test.txt" "WriteReverse"
         FileClose $R1
          Delete $R1
           CopyFiles $R0 "$EXEDIR\test.txt"

Thanks Instructor!

Instructor
12th April 2005 10:11 UTC

Yes, I know about it. It's because last line "9. a" have not $\r or $\n at the and of line. I thought when writing function add these characters if missed but decided to output lines as is. I add some lines in example2. Thanks, vbgunz.

Name "Output"
OutFile "Output.exe"

!include "TextFunc.nsh"
!insertmacro FileReadFromEnd


###Example2 (Reverse text file):
Section
        GetTempFileName $R0
        FileOpen $R1 $R0 w
        ${FileReadFromEnd} "C:\input.txt" "Example2"
        FileClose $R1

        IfErrors 0 +2
        MessageBox MB_OK "Error" IDOK +2
        Exec '"notepad.exe" "$R0"'
SectionEnd

Function Example2
        StrCmp $7 -1 0 +5
        StrCpy $1 $9 1 -1
        StrCmp $1 '$\n' +3
        StrCmp $1 '$\r' +2
        StrCpy $9 '$9$\r$\n'

        FileWrite $R1 "$9"

        Push $0
FunctionEnd

PS:
Why you need this line in code? :)

Delete $R1

try to use /SILENT flag for CopyFiles if you copy just text file (faster)

vbgunz
12th April 2005 10:48 UTC

Major Thanks Instructor! Everything you've written so far is perfect. About my bumble with Delete $R1... Well, I actually forgot why I was using it... I think it belonged to experimental code as I pulled it from a testbed script.

Wow, in the end instructor, I've managed to piece together a perfect parser for my job with 99% thanks to you. Most of the functions and stuff I've scraped together to make it work all have your name on it ;)

Anyhow, I owe you for last time and I promise to try and get your work into the archive when time permits. Right now I am dead beat and am about to go to bed... I got to say though my dreams will be sweeet!

Thanks again!

vbgunz
12th April 2005 11:07 UTC

I just remembered why I was using Delete $R1. If I don't delete $R1 (source file) the copy will fail.

Thanks again!

Jnuw
12th April 2005 14:51 UTC

Originally posted by Instructor
It created a new text file in 1 second consisting of 8000 lines of :)

Very nice! :up:

vbgunz
13th April 2005 05:05 UTC

Instructor, I hope you get this message. Utilizing your work, I've managed to create a perfect runtime parser for my project. Unfortunately, I actually need it to perform during compile time. How would you convert the following runtime code to compiletime code?

Name "Output"
OutFile "Output.exe"

!include "TextFunc.nsh"
!insertmacro LineFind
!insertmacro TrimNewLines

Function FileReadFromEnd
        !define FileReadFromEnd `!insertmacro FileReadFromEndCall`

        !macro FileReadFromEndCall _FILE _FUNC
                Push $0
                Push `${_FILE}`
                GetFunctionAddress $0 `${_FUNC}`
                Push `$0`
                Call FileReadFromEnd
                Pop $0
        !macroend

        Exch $1
        Exch
        Exch $0
        Exch
        Push $7
        Push $8
        Push $9
        ClearErrors

        StrCpy $7 -1
        StrCpy $8 0
        IfFileExists $0 0 error
        FileOpen $0 $0 r
        IfErrors end
        FileRead $0 $9
        IfErrors +4
        Push $9
        IntOp $8 $8 + 1
        goto -4
        FileClose $0

        nextline:
        StrCmp $8 0 end
        Pop $9
        Push $1
        Push $7
        Push $8
        Call $1
        Pop $0
        Pop $8
        Pop $7
        Pop $1
        IntOp $7 $7 - 1
        IntOp $8 $8 - 1
        IfErrors error
        StrCmp $0 'StopFileReadFromEnd' clearstack nextline

        error:
        SetErrors

        clearstack:
        StrCmp $8 0 end
        Pop $9
        IntOp $8 $8 - 1
        StrCmp $8 0 0 -2

        end:
        Pop $9
        Pop $8
        Pop $7
        Pop $1
        Pop $0
FunctionEnd

Function WriteReverse
        StrCmp $7 -1 0 +5
        StrCpy $1 $9 1 -1
        StrCmp $1 '$\n' +3
        StrCmp $1 '$\r' +2
        StrCpy $9 '$9$\r$\n'

        FileWrite $R1 "$9"
        Push $0
FunctionEnd

Section
;--------------------------------------------------------------------------------------------------
        GetTempFileName $R0
         FileOpen $R1 $R0 w
          ${FileReadFromEnd} "$EXEDIR\text.txt" "WriteReverse"
         FileClose $R1
;--------------------------------------------------------------------------------------------------     
        ${LineFind} "$R0" "$EXEDIR\text.txt.log" "1:-1" "LineFindCallback"
SectionEnd

Function LineFindCallback
        ${TrimNewLines} '$R9' $R9
        StrCpy $1 $R9 1 20
        StrCpy $R9 $R9 '' 56
;--------------------------------------------------------------------------------------------------
StrCmp $1 "D" 0 isFile
 StrCpy $R9 'RMDir $\"$$INSTDIR\$R9$\"$\r$\n'

  Goto skipahead
   isFile:
    StrCpy $R9 'Delete /REBOOTOK $\"$$INSTDIR\$R9$\"$\r$\n'
skipahead:
;--------------------------------------------------------------------------------------------------
        Push $0
FunctionEnd

I really don't know how to extend my thanks for your time so I wish to assure you I am very grateful. Thank you again Instructor!

Instructor
13th April 2005 07:35 UTC

It's easy to use:

!include "TextFunc.nsh"
!insertmacro LineFind
!insertmacro TrimNewLines
!insertmacro FileReadFromEnd

In place of big include:

Function FileReadFromEnd
        ;...
FunctionEnd

vbgunz
13th April 2005 11:14 UTC

Man I feel like a bubble head. You say it's easy to get all that runtime code I posted to turn into compiletime code by simply placing an insertmacro... I am running into doozies... e.g.

Error: can't define a macro inside a macro!

It seems your solution at a glance will only work with FileReadFromEnd so I assume I must take the other functions and turn them into !macros. On top of that... The little piece of code I have in the section... I should also place this in a !macro?

Thank you!

Instructor
13th April 2005 14:05 UTC

"TextFunc.nsh" allready contain FileReadFromEnd function. All you need is put function in code "!insertmacro FileReadFromEnd" (if you didn't modifity "TextFunc.nsh" :) )

Name "Output"
OutFile "Output.exe"

!include "TextFunc.nsh"
!insertmacro LineFind
!insertmacro TrimNewLines
!insertmacro FileReadFromEnd

Function WriteReverse
        StrCmp $7 -1 0 +5
        StrCpy $1 $9 1 -1
        StrCmp $1 '$\n' +3
        StrCmp $1 '$\r' +2
        StrCpy $9 '$9$\r$\n'

        FileWrite $R1 "$9"
        Push $0
FunctionEnd

Section
;--------------------------------------------------------------------------------------------------
        GetTempFileName $R0
         FileOpen $R1 $R0 w
          ${FileReadFromEnd} "$EXEDIR\text.txt" "WriteReverse"
         FileClose $R1
;--------------------------------------------------------------------------------------------------     
        ${LineFind} "$R0" "$EXEDIR\text.txt.log" "1:-1" "LineFindCallback"
SectionEnd

Function LineFindCallback
        ${TrimNewLines} '$R9' $R9
        StrCpy $1 $R9 1 20
        StrCpy $R9 $R9 '' 56
;--------------------------------------------------------------------------------------------------
StrCmp $1 "D" 0 isFile
 StrCpy $R9 'RMDir $\"$$INSTDIR\$R9$\"$\r$\n'

  Goto skipahead
   isFile:
    StrCpy $R9 'Delete /REBOOTOK $\"$$INSTDIR\$R9$\"$\r$\n'
skipahead:
;--------------------------------------------------------------------------------------------------
        Push $0
FunctionEnd

vbgunz
13th April 2005 18:15 UTC

I was having some problems and realized I should check for the latest headers. That solved my first problem. My second problem (hopefully the last), is the code is still runtime only.

I am trying to make this code parse a page at compile time. I need to do something with the parsed file before I finish building the installer. This I was hoping you can help me out on.

I have not modified TextFunc in any way. The only code I am modifying is the code in which you've pasted back into this thread... That's about the only code I am working with hands on.

I hope you understand. Thanks Instructor!

Afrow UK
13th April 2005 19:33 UTC

http://nsis.sourceforge.net/archive/...instances=0,44

-Stu

vbgunz
14th April 2005 02:19 UTC

Afrow, I found that link first before ever asking for help on this. The reason I decided to ask, is I have no idea how to implement that script into what I need it for :(

I think I need a better example as I was dazed out and confused just pondering "how do I implement it". Can you provide a different example on something much simpler?

vbgunz
14th April 2005 09:14 UTC

Thanks for the help Afrow, I basically solved it and converted the runtime code into compile time BUT if possible I would still like to see an easier example regarding your code.

Thanks again!

Afrow UK
14th April 2005 11:47 UTC

I added some more notes, but it can't really get much easier than that. What would you suggest?

-Stu

Instructor
14th April 2005 12:39 UTC

I am trying to make this code parse a page at compile time. I need to do something with the parsed file before I finish building the installer. This I was hoping you can help me out on.

The only way I see it's create single exe and hand needfull parameters by CMDLINE (Call this single exe with !execute)

vbgunz
14th April 2005 18:14 UTC

Instructor, that is exactly what I did ;) Afrow, I will take a look at your new example as I did wish to use it :)

Thanks Fellas!