Archive: nsExec question
nsExec question
nsExec is good that it will get the output of an MS-DOS program, but it will only get the ouput and print it after the MS-DOS program has finished running.
Is there any way to get the ouput of the MS-DOS program every second?
I'm suspecting this isn't possible under Windows (but I know it is under Linux)
-Stuart
If you use ExecToLog it gets new output and prints it as long as it gets it from the DOS program. If there is no output it waits 1/10 seconds and tries again. If you use ExecToStack you'll get the output only when the program has finished.
Hmm, well it still only prints the ouput when the program has finished, and prints nothing inbetween.
-Stuart
nsExec takes the output when the program gives it. Maybe it's your program that only outputs when it's finished.
In MS-DOS, there are loads of live output shown on the screen, like % complete, count status, section part etc
None of this shows up with nsExec.
Has nsExec been tested with any 'live' MS-DOS programs before?
-Stuart
That output will indeed not show until the line break appears. I will add that to the todo list. Did I understand right or does it wait for the whole output even though there are line breaks in there?
Output sample...
----------- qvis3 -----------
original code by id Software
Modified by Geoffrey DeWan
Revision 1.02
-----------------------------
fastvis = true
reading C:\Quake2\map-compile\maps\gbr\gbrsnow1.bsp
reading C:\Quake2\map-compile\maps\gbr\gbrsnow1.prt
1890 portalclusters
5804 numportals
0...1...2...3...4...5...6...7...8...9... (54)
WARNING: Leaf portals saw into leaf (938)
WARNING: Leaf portals saw into leaf (1091)
WARNING: Leaf portals saw into leaf (1451)
Average clusters visible: 1091
Building PHS...
Average clusters hearable: 1854
visdatasize:810110 compressed from 907200
writing C:\Quake2\map-compile\maps\gbr\gbrsnow1.bsp
56.0 seconds elapsedYeah the problem with that is the fact that the 0...1... bit doesn't ever print a newline character out so nsExec doesn't do anything with it until the end of the line. This can be solved but isn't brilliantly simple because it would involve repeatedly updating the same line in the details window rather than just printing out one line at a time.
There are probably lots of windows programs that can capture that, it's not hard, just unusual.
I've tried making some changes to nsExec, can you try my changed nsExec.dll please and tell me if it works for you... I don't have time to test this right now.. I'll be back later today some time.
I tried it.
Output still comes up only after it has finished:cry:
-Stuart
I'm having trouble fixing this at the mo, will try again later today.
[edit]I've tried changing all kinds of things but the pipe nsExec reads from is just not receiving the data... the best I've got so far is for nsExec to finally receive some data when the child process fills its output buffer but that takes a while (I'm guessing this is what's happening anyway). The damndest thing is that I don't appear to be able to disable or shrink that buffer. I tried getting the pipe to operate in byte mode but that didn't help either... I'll try some more stuff later.[/edit]
Right, I gave what time I had to this today (not much I'm afraid) but a solution was not forthcoming. I'm not saying I give up, I'm just saying don't hold your breath waiting for me to deliver a solution. I'll continue my investigation as soon as I can.
Would you like me to give you the exe program and a small map to test nsexec on?
-Stuart
I was testing it using pscp which also rewrites a line showing status information about the copy. I simply could not get the nsExec code to receive output from pscp until it finished the line and started a new one.
I know that WorldCraft's compiler window (the map editor I use) only outputs the text after a line break occurs. None of the counting up is shown. This caused a HUGE amount of anoyances from the general web public, but no matter how much the programmers at Valve tried, no one could get it to produce the live counting up output.
I'm not sure if the latest versions of WorldCraft show complete live output, because the old v1.6 version does Quake2 maps.
It seems that it was quite a task to get the output, even for people that created one of the best mapping programs out there.
Please don't give up though!
-Stuart
Maybe you could show me the source?
I'm no C/C++ wizard, but I know of many people who could help extremely.
-Stuart
nsExec
Resuming the topic from before about trying to get nsExec to get the live output from executables, I've just noticed it does get some output from exe files straight away.
e.g. with the Quake2 compilers, it outputs a lot of the information until it reaches the compile % done area. Thats where it stops outputing anything until the compile process has actually finished.
So, nsExec is nearly complete that it gets almost all live output,
however, it still doesn't get all live output.
Other peoples' compile programs do just the same as the current nsExec I have downloaded from SunJammer as I explained above.
-Stu
That's right. nsExec outputs a new line when it gets a line break.
What annoys me is that a DOS window prints characters that change before a line break appears, and no matter how hard I tried I couldn't alter nsExec so that it could do the same. I tried many approaches but it simply never receives the data... :(
I wonder if we could conact M$ :p
-Stu :)
I wonder if we could conact M$ :pand pigs will fly :blah:
nsExec in latest CVS contains the changes that Sunjammer made, plus some other things.
However, the full live output was never achieved.
-Stu
thanks for that, just had a look at the nsExec source but will need to get on my machine to have a full check out of what's going on before i try anything else.
now where's those code backups... ;)
-daz
I'm glad someone is going to have another hack at it!
-Stu
had seen the thing in the todo.txt and had wondered what it was until i finally remembered to have a look at what it was today.
hack at it!i think that's going to be the best way to describe it and i'm a sucker for punishment :blah:
Live Output Test version
attached dll is what i have done so far.
it should capture things correctly when using ExecToLog(). i've checked it using chkdsk on win2k and NT4 (seems to lose the live line from the test just done on NT4 when it starts back on normal output :() otherwise it should be fine. (will have to get worldcraft out again for extra checking)
if there are issues then please post what has gone wrong so i can get an idea of what to fix/change (already got the NT issue to work on).
i'm also working on ExecToStack() to handle live output correctly too (should have that done by tomorrow ;))
-daz
Sorry to be negative, but this version is worse.
The older version actually printed the program name, and then went onto the processing, but this one waits before the processing has finished before printing anything.
-Stu
hmmm, strange that since before i altered things i checked how it worked using the example script and then against what i have done and it seemed to be fine.
what are you testing it with? may have an idea of why the first line could be being ditched. so it's not displaying anything since with chkdsk i've got it to correctly show the when % part was changing
-daz
http://myweb.tiscali.co.uk/imker/test.zip
Run qbsp3.exe on resist.map like so:
nsExec::ExecToLog "$\"$EXEDIR\qbsp3.exe$\" $\"C:\quake2\resist.map$\""
resist.map must go in C:\quake2 for the map compiler program to work.
It's a good idea to try the program in DOS first, to see how it runs, then compare it to nsExec.
-Stu
[off topic]cycling will kill me![/off topic]
at work (again) on a different machine so will give it a go now
-daz
getting No gamedir in c:\quake2\resist.map also i don't have a copy of quake2 to work with at home :(
will just have to crack out worldcraft and half life (hopefully that will work in a similar way - quickly scanned down the page) or it's custom dos program time :eek:
-daz
ok how it's going so far...
managed to get a test system running with worldcraft 3.3 and using Afrow Uk's test map and a smaller one i've generated.
in all tests, the output will not appear until it has all finished (no title even when it starts - checked against direct running via the console) - also checked this with the current cvs code too :(
checking the code and the peeknamedpipe(..) function returns that there is no data to get from the screen even though when running the dos version it can be seen that there is data available (especially when starting!).
why this is the case i'm not too sure :confused:
it almost seems as though the data is being locked from being read and so the pipe can't get at the screen data.
going to check a few more things out about pipes, etc. also going to give my old screen capture code a go as well to see how that fairs with things (readconsolescreen(..) etc and all of those functions, oh the fun :))
will also fix the code i've done to handle certain 'live output' as chkdsk does - is part way there for full live output, just the locked display issue to get around and it should run correctly.
-daz
p.s. i will upload a test exe of what the chkdsk thing does to show what i mean for certain 'live' outputs happening
gamedir is baseq2, so simply move the map to C:\quake2\baseq2\maps
Sorry I should have told you earlier, but haven't been around.
-Stu
don't worry about it ;) and thanks for the info
off to crack on with it again soon so let the fun begin again i guess :eek:
-daz
done some more work on things (using Afrow Uk's example now for testing).
for whatever reason the console does not seem to be recognising that there is data on the screen until processing ends :hang:
checked the working of the code against the standard console redirection and the problems with nsExec also happen with the console
e.g.
qbsp3.exe c:\quake2\baseq2\maps\resist.map > blah.txt
will only put data into the file once the processing has been finished. i also tried this with chkdsk and that will output the screen data as it changes. i even tried control-breaks to see what was output and for the resist.map test there is still no output to the blah.txt file.
so whatever the reason causing this, even the OS can't deal with things :weird:
what i'm working on (as an idea) of outputting a basic string such as nsExec is working... / where the last part ( / ) will go /,-,\,| to indicate that processing is still going - at least gives some indication that things are working.
-daz
here's an example of the indicating processing code when the output can't be read.
it's not complete (only started on it this morning) but it shows the basics of the feature
note: the previous line may disappear when reading occurs (still to track that down)
-daz
[edit]
the test is set to use Afrow Uk's example, see above for the setup of the files and to get them too ;)
[/edit]
Some console based applications do not use the standard handles for their input/output (IO) operations. The Win32 API does not support redirection of these processes.after a little searching my thoughts were confirmed with the above quote from m$ here :(
attached is the modified source code that will put
nsExec is working... /
when the code detects the process is running but it is not able to get the data put to the console (see above as to why).
i'm re-working the code that will display live output that can be detected but the attached modification is good to go for the cases when it is not detectable :)
-daz
p.s. is just the source file (forgot to bring the new dll with me)
finally remembered to bring all the files with me, so here is a working example of the modified nsExec and there is a test program to show it at work when the output is held back until processing is finished (damn you printf(..) :blah: )
just extract the installer and dummycon.exe to the same place and run.
the next update will incorporate earlier code (further up this thread) that will be able to handle the output holding at the same position such as a %increase (or the 0... to 9... in the attached example file).
-daz
off topic i think?!?!
if i'm altering the nsExec code, is it be alright to put my modified version in the archive? i'm asking this since i've added in an abort button (needed it for testing for bits above) and i thought it might be useful for other people.
thanks
-daz
Archive: nsExec question
Of course you can. Where else? :)
It should find its way into CVS too, but I don't know if before b4 or not.
thanks, i just thought it best to check before i did put it in the archive.
if it gets in it's probably best to go after b4 i think since i found a few problems with the code i posted (fixed in the latest version i have ;) )
will put an updated version in the archive as soon as i can
-daz
Well, this thread has been dormant for a few years, but in case anyone else is having this problem, I found that one way to get live output streaming working with nsExec::ExecToLog is to do this in each console app you'll be running:
setvbuf(stdout, NULL, _IONBF, 0);