IonicWind Software

I have come across an inexplicable behavior of the compiler (1.55).

Please download the (well known by now) sensor grid program from

http://paravantis.com/sensor_grid.eba

Compile it WITH DEBUG information and BUILD>DEBUG>START it. Then input (a) any number, e.g. hit ENTER for the default grid size of 4x4 and (b) 2 for debug output option (the second input asked by the user) - this is VERY IMPORTANT: the program will crash ONLY if you INPUT 2 as a debug output option. Then watch the program complete its run without any problems (in DEBUG RUN mode).

Now compile it WITHOUT debug information and run it normally (e.g. BUILD>BUILD SINGLE or COMPILE AND EXECUTE). It will print SOME information in the screen after your inputs and then BOMB!

Since there is NO information provided in debug mode, how on earth am I supposed to find what is going on!

If you REALLY want to look into it, go to line 579

               write f1,"    maxGamma          = "+trimN(maxGamma)

By trial and error, I have found that SOMETIMES the program bombs right before this line; unfortunately this is NOT always so! (trimN is a user defined function).

I may be overlooking something silly so your input would be appreciated. Otherwise, Paul's attention would be of benefit.

John

Patience please, there is no need to both email me and then post here.  I am a very busy person and cannot always devote 100% of my time to debugging users programs for them.

As for your question, yes it is very common for programs to run fine in debug mode, but crash in release mode.  We C programmers run into it all the time.  Has to do with memory corruption usually.  When Windows loads a program for debugging it places the loaded executable at a different memory location, sometimes that is enough to prevent the corruption.

So the answer is to use JIT debugging, which I won't go into a lot of detail here.  After the JIT debugging there seems to be a stack corruption in a call to trimN somewhere, or possibly a bug in rtrim$ which is unlikely but I will look anyway.  Changing the function to:


sub trimN(n as double),string
return ltrim$(str$(n))
endsub


Does the same thing as there is a leading space added by str$, but not a trailing one so rtrim$ is unecessary anyway.  And the program no longer crashes.

Paul.

I posted to the forum precisely because I understood that you presently had no time and wanted to get some feedback from users. No offense was intended.

I did the change that you suggested but the program STILL crashes. It runs ok OCCASIONALLY; if you try running different grid sizes (e.g. 15, 6, 11) and keep inputing 2 at the debug output option prompt, the program will eventually bomb.

I do appreciate that you have no time debugging user's programs "for them". Quite frankly, I am ashamed asking for help all the time and wish eBasic gave more information so that I could debug the program without outside help.

I use BASIC compilers precisely because I don't have the inclination to run into the problems faced by C programmers.

John

Perhaps you'd like to share the source (or at least the offending part) with the rest of us. More eyes will probably find the error more easily.

Barney

Barney. See his first post. He has posted a link with full source code ;)

Lewis

Barney and Lewis,

Thank you for your attention.

Again, the link is

http://paravantis.com/sensor_grid.eba

John

Paul,
I ran in to a similiar problem with the Aurora Help File Manager program, written in IBPro.

Quotesub trim$(a as string),string
ÂÃ,  ÂÃ,  ÂÃ, a=ltrim$(rtrim$(a))
ÂÃ,  ÂÃ,  ÂÃ, return a
endsub

I used it all over the program then it started crashing the program.
I found one place where it was giving me the problem.
I changed the program at that one spot from

Quotetrim$(z)

to

Quotez=ltrim$(z)
z=rtrim$(z)

and the crashing went away.

I was sending it big istrings in that part of the code..

Larry,

That cannot be the case in my code.

I only send SMALL strings, at most up to, say, 80 characters.

In fact, I tried EMPTYING all the trim and trimN SUB code (so that the SUBs do nothing and just RETURN) but the program still bombs.

So far, it looks like a bug.

John

Let's all hang on to our perspective here. Emergence Basic was released for the first time less than three weeks ago. And now we're discussing the possibility that there might be a bug??!

Try reporting a compiler bug to Micro$haft. You're very lucky if you even get an acknowledgement that they received your message. And actually discuss it with you??? Debug your code???

This is exactly why I would always prefer to do my development using one of Paul's languages instead of the Behemoth on the Left Coast!

Quote from: Zen on December 13, 2006, 04:42:07 AM
Barney. See his first post. He has posted a link with full source code ;)

Lewis

Must be my old age. ;)
Now... where's my second pair of glasses...  ;D

Barney

Quote from: Barney on December 13, 2006, 09:49:39 AM
Must be my old age. ;)
Now... where's my second pair of glasses...  ;D

he he. Yet another Dohhhhh moment there :D

Lewis

Well... I just downloaded the source and compiled it. As expected it bombs out but I've found out that it does not matter if it is compiled with or without debug info. So, let's say I compile with the debug info and run it by using the "Debug Start/Continue" button. In that case it will not crash, but when I run the same exe by pressing "Execute" button it will crash.

However, the console screen shows two different values, which may help John in pinpointing the problem.

These are the screen messages when the app runs properly:

Solving 4x4 grid containing 16 cells ...

uncoverage (%) =  0.0

sensors of type 1  = 2
sensors of type 2  = 4

OBJECTIVE FUNCTION = 800

run time (seconds) : 0
run time (minutes) : 0


and these are the messages when it bombs out:


Solving 4x4 grid containing 16 cells ...

uncoverage (%) = 15.6

sensors of type 1  = 0
sensors of type 2  = 3


Perhaps this will help in further debugging. I'll take another look at the code when I find some time.

BTW. John, why don't you use USING instruction when formatting numerical output. It's quite useful and I never had problems with it.

Barney

Dear Barney,

Very many thanks to you for spending time with my stupid code!

The two screens are not in fact different. The "uncoverage" number starts at 100% and is reduced down to 0% when the program ends (and manages to cover the entire grid with sensors).

So what the crash screen displays simply means that the program did not manage to finish, bombing when uncoverage became equal to 15.6%.

I agree with your other comment. In my case, the debug EXE does not crash as long as I run DEBUG>START from within the IDE!

Finally, I do not employ USING because all I want to do is strip leading and trailing spaces.

Thanks again for your kind efforts!
John

John,
Don't always assume it's a problem with Emergence.  Your program is corrupting memory, I just haven't had the time to go through it line by line to find out where.  For one arrays in Emergence are zero based, but you prefer 1's based so are using 1 to N everywhere ao you have to check to make sure every array is at least N+1 in size.

Quote
I use BASIC compilers precisely because I don't have the inclination to run into the problems faced by C programmers.

No you use interpreters for that reason.  Internally the processes that Emergence BASIC goes through is the same as any other true compiler.   Your source is converted into assembly language, and then to 100% machine code.  There is no runtime, the compiler has no idea what the contents of a variable will be when the program is run, it's job is simply to create the assembly language code.  When you are debugging it is the machine code that is being debugged and at this version the only information in the debug executable is the line number from the source file that created the assembly code.  Version 1.6 will have variable display like Aurora does but it still won't protect you from your own errors.  Meaning if you corrupt memory by overwriting a string, or exceed an array dimension, you will still get the occasional hard to track down bug.

I have literally thousands of programs that run fine when compiled with Emergence, many being 40000-50000 lines long.  But of course I do write code much differently than you might.  I never use static arrays, as I prefer to allocate memory and have built in error checking.  I don't write in a top down style, instead I write in a modular style that is easier to maintain.

With that being said I will get back to your problem later this evening.  Right now I have to go dig holes in my back yard looking for my septic tank. 

Paul.

QuoteFor one arrays in Emergence are zero based, but you prefer 1's based so are using 1 to N everywhere ao you have to check to make sure every array is at least N+1 in size.

I did make sure ALL arrays are dimensioned this way.

I am also attempting to solve grids MUCH smaller than N_max (=50).

John

I changed you trimN function code to the following and get no crashes.

Quotesub trimN(n as double),string
   def x as string
   x=str$(n)
   x=ltrim$(x)
   return x
endsub

So it appears there is something that isn't liked when you do a "ltrim$(rtrim$(z))" which is the same sort of problem I was having.

Larry

Larry,

You are a genius!

I confirm that changing the two trim subroutines to

Quotesub trim(s as string),string
   def x as string
   x=rtrim$(s)
   x=ltrim$(x)
   return x
endsub

and

Quotesub trimN(n as double),string
   def x as string
   x=str$(n)
   x=rtrim$(x)
   x=ltrim$(x)
   return x
endsub

enables the program to run without crashing!

So it looks as if nesting ltrim$ and rtrim$ may have created the problem.

Paul may want to look into this after his septic tank adventure.

Thanks!
John

John,ÂÃ,  .. bug finder extraordinaire .. :)

I must have used ltrim$(rtrim$()) lots of times in various programs, and never hit that problem ..

In fact come to think of it, I don't think I've ever found a bug - just rubbish at it I suppose ..ÂÃ,  :)

best wishes, :)

Graham

Graham,

Perhaps you would like to try your hand at finding an alternate explanation of the behavior of my program and why changing the subs did the trick.

The offending code still resides patiently at

http://paravantis.com/sensor_grid.eba

Best luck in your experienced hunts.

John

ooh! -- that's annoying - I'd got it all working the other day - I'd even plugged in the 5D arrays and that worked fine. :)

Unfortunately, I thought -ÂÃ,  oh - John's fixed now, it's all OK - and I deleted everything ..ÂÃ,  ::)

Did you change something from your originally posted code - 'cos that worked for me after Emergence 1.55 was available?
If not, that's a puzzle, 'cos it worked for me the other day ..

Just noticed another funny thing while looking to see if any of the code still remained on my machine - there are two references in c:\windows\prefetch relating to sensor-grid.ÂÃ,  ÂÃ, I've no idea what the 'prefetch' stuff is, but I've got a bunch of it in there amounting to over 5Mb for various things .. more Windows clutter I suppose ..ÂÃ,  ::)

I'll have another go at it ..ÂÃ,  :)

Edit: I've answered my own question - you've made lots of changes .. looks like it's start again time ..ÂÃ,  :)

Graham

Graham,

I don't think that you really looked into it long enough the other days to figure out what is going on. The truth is that the program NEVER worked well in eBasic, be it version 1.5 or 1.55.

Paul was kind enough (as he always is) to enhance version 1.55 pretty much per my wishes; version 1.55 has the far call jump corrected -- now I'd say that was a pretty serious issue, don't you think? I mean, being unable to jump to another location of a long program correctly sounds like a rather fundamental flaw, wouldn't you say? I'd even call it a "bug".

Most importantly, version 1.55 allows arrays to have up to 5 dimensions, but you know what? I thought since types are a part of eBasic, lemme play with them a bit longer and see how they work. So I decided to stick with my user defined types rather than larger arrays. I mean, they should both work well in a finished product.

In the latest version of my program, all I did is what I explain in my earler posting. Really, ALL I DID was untangling ltrim$(rtrim$). Heck you can download the code and try it out for yourself!

Now, Graham, I am NOT interested in a DIFFERENT version of the code, e.g. using arrays of 5 dimensions, that happens to work correctly. I am interesting in finding out what is wrong in the specific version of the program that you may download from my site. If types, for example, don't work, then we should KNOW about it, shouldn't we?

Kindly excuse my forceful style but I am almost sorry I started this thread instead of giving up on eBasic. Some postings, including yours, pretty much make me feel that I have to prove I am not an elephant.

Graham,

Just an addendum to my previous posting.

I am rereading the previous message and thinking what an idiot I can be at times, especially after a full day's work in front of a computer screen!

Will you PLEASE excuse my ...inexcusable writing style that borders on rudeness? I have busted my nerves and, as a result, I am being almost mean to some of the nicest people on the forum, such as yourself!

???
Begging for forgiveness,
John

Just a note:
One of the older IBPro forum members once told me that if I have a program that runs in debug but not in normal mode then I've got a problem with strings somewhere.

My experience has always been in agreement with that statement.

That was what help me find my problem with the nested ltrim$(rtrim$(a)) code.

Hey .. no problem JohnÂÃ,  - I'm not easily offended, and I know how trying it can be getting software do what it's supposed to. :)

Stick with it ;).ÂÃ,  ÂÃ,  I see what you meanÂÃ,  - you want to find out why this particular code won't perform .. that's fineÂÃ,  :)

It's a bit difficult looking for the problem without understanding the algorithm you are coding - and also, Emergence has not yet implemented a debugging facility where variable values can be examined .. the old IBasic interrpreter was really good for that ..

Don't suppose you've got the algorithm written down somewhere have you?ÂÃ,  ÂÃ, :) .. that might help understand what's going on.

You're right of course that we need to check Types are working correctly - mostly though I find it's just some logical oversight on my part that causes my code to blow up .. it's just a matter of finding out what ..ÂÃ,  ;D

I've just run the original November version against the December version using the 'Track Changes' tool in Word, and there's lots of differences and additions - so that probably explains why I got the November code to run, but the December one won't.

Your type array3 for instance just after the 'Coverage Coefficients comment, was def a3[10,10,10] as Int - but is now def a3[Nmax+1,Nmax+1,KK+1] as Word

Any one of the changes could be the culprit ..ÂÃ,  :)

Anyway, on we go ..ÂÃ,  ;D

Graham

John,
I am still debugging the original problem,  line by line, assembly opcode by assembly opcode.  Which is why I asked for patience.  Single stepping through assembly code, when your running a program that normally takes 10 seconds to finish, could take me 10 hours.

I've had exceptions in code that have taken days to solve, some longer.  I may make it look easy to some of the users here but real debugging takes time ;)

The compiler isn't new.  It is the same code base we have been using with Aurora for over a year now and not once has ltrim$ (trimLeft) or rtrim$ (trimRight) caused any problems for us.  And I use combinations of the string functions all of the time.  You are much too quick to judge a problem you don't fully understand yet, which is why I am taking my time discovering exactly what your code is doing when the heap gets corrupted.  Meaning the problem is most likely unrelated to the point in code for which it is failing.

Don't get me wrong.  I appreciate that you've come up with a peice of code that reveiled a potential problem.  It is just the first time any of us have encountered it with Emergence.

Paul.

Paul,

You are, as usually, right on all points.

We do have plenty of patience so worry not.

;D
John

I tried a few variations of the following code to try to make it crash.  I thought that the nested trims caused some trouble but this simple code doesn't crash.


def test as string
test = "  white space before and after  "

test = ""
print "+",test,"+"

test = LTRIM$(RTRIM$(test))
print "+",test,"+"

PRINT "Press Any Key To Close"
DO:UNTIL INKEY$ <> ""



Then I tried:

as string
test = "  white space before and after  "

print "+",test,"+"

test = TRIM(test)
print "+",test,"+"

PRINT "Press Any Key To Close"
DO:UNTIL INKEY$ <> ""

sub trim(s as string),string
return ltrim$(rtrim$(s))
endsub

sub trimN(n as double),string
return trim(str$(n))
endsub



edit:  I could NOT get it to crash.  John, have you tried isolating this area of code to try to duplicate the failure?

There is a problem somewhere in an inner loop.
I'll look at this more when I get home, heading to see Santa..

John S,

After a full day in front of my computer, I now see floaters and have decided to let Paul slay the beast!

He seems to think there is memory corruption somewhere and it seems to me that playing around with code trying to make it crash may not be very fruitful!

Needless to say that I appreciate your kind endeavors!

::)
John

OK found it.  See I only needed a little time.  Don't know if you could call it a bug, just a misunderstanding between me and some assembly code ;)

For years I have been using a fast string copy routine that uses aligned dwords to move data 4 bytes at a time instead of one.  Much faster than other methods.  And it works very fast.  The problem is in some cases it will copy a dword worth of data to the end of the string, causing 1-3 bytes extra needed in the destination memory.   When you are working with defined strings this isn't a problem

string a
a = ltrim$(a)

There are 255 bytes there.

a = ltrim$(rtrim$(a))

The output of rtrim$ is located in heap memory, which is allocated to the exact length of the return plus 1 for the NULL.  ltrim$ uses the fast copy routine to remove the space to another allocated string which is then returned. 

So lets say for example the initial string is " hello ".  Which is 8 bytes long.  rtrim$ allocates a return string that is 7 bytes long and copies up to the space, using that fast string copy routine.  Since it is only 7 bytes the NULL sits on a partial dword boundry and it causes 1 extra byte to overwrite the destination memory.

Now that overwrite may or may not cause an exception, depending on how close to the end of heap block it was overwritten at.

Anyway I have a solution for it, of course, and will have an update this evening.

Your program was tested after the change for all grid sizes from 4 - 20 using debug mode 2 with no more crashes.

Paul.

Do I earn the title of "misunderstanding between Paul and assembly"-hunter extraordinaire or what?

At how many such kills, do I earn a free copy of the next major version of eBasic?

;D
John

Paul, someday you might start handing out T-shirts for people like John........

Steve

John,
At least two more kills ;)

Also to answer your previous question... when you debug an executable windows handles heap memory differently, to catch things like overwrites.  It allocates more than the requested size to place guard bytes around the allocated block.  Unfortunately it never generated a proper exception for me to trace. 

Paul.

Paul,

If you would care to forward me a "beta" version of the next update to fiddle with prior to releasing it, I am game.

This because I seem to have an affinity to ...issues.

Seriously: put me on the beta testers group!

John

We don't have a beta testers group as such. Basically, you find a bug and Paul fixes it. It is already out of beta stages, hence it being released but obvioulsy some bugs don't get found no matter how hard you try to find them.

Lewis

In my last job before retirement I wrote code to process mission critical data being sent to us from our transfer agent overnight.  My assigned task in writing this was to make it perfect.  Yes, my bosses believed in perfection and I was supposed not to disillusion them. :)

Anyway it all worked perfectly for 7 years, processing several million data records each night without fail until it finally found that one combination I hadn't considered.  I was no longer involved in that project and it took a while for them to call me in since that program had been so trouble-free they only considered the possibility it could be at fault when everything else had been eliminated.

When they did call me in I found that very weird combination of data that tripped it up, fixed the program, and everyone was happy again.

In my entire career that is the bug of which I'm most proud. :)

Barry  (who knows there ain't no bug-free code)