Crashes and more crashes, finally solved, I hope

On a non-shared database, I had a procedure that crashed in the same place about 60% of the time. I checked and revised and tried many things to stop the crashing. I had this evening started composing a long email about this asking for help, when I had more idea to try and it seems to have worked! After enough guessing, sometimes you get lucky.

Starting with a form with seven variables with seven text editors, the Next button on the form runs the procedure and a subroutine to add data rows to the database, somewhere between one row and four rows. The procedure crashed, I think, when it reached a loop, addrecord, and then several assignments to populate the newly added row. (When running under Terminal, the last message when the crash occurred was zsh: segmentation fault. /Application/PanoramaX.app/Contents/MacOS/PanoramaX. No idea what that means. No data was there upon re-opening.

The crashing stopped when I added two wait 3 statements, around a console statement and right before the loop statement. The procedure has not crashed since adding these. (Two seconds worked on my computer, but crashed once on another computer-the person who actually uses this database for work).

One issue is that if the user click on another window while this procedure is running, now around 7-8 seconds with the 6 second delay, the procedure may fail. (This is where I thought the disablemouseclicks might work.)

If you want to see the code, I will send it to you.

Unfortunately, I don’t know what that means either, at least in any useful way that would point to a cause or solution.

Yes please. If possible, just use Source>Copy Indented Source and then paste it in here, or in a DM if you don’t want others to see it.

I’m glad you mentioned this. I’ve given some thought to this idea. If I did implement such a statement, it’s effect would certainly not extend across a wait statement. The whole point of the wait statement is that other tasks can proceed during the wait. After reading your message I was thinking that I needed to expand the documentation for the wait statement to emphasize that you can’t rely on a particular window being active after the wait, but I just checked and I think it is clear. Especially if the wait is > 0.

I’m really puzzled as to why adding wait statements would fix a crash, but since it seems to, I have a couple of suggestions. You might try using noshow, then doing the assignments, then showline and endnoshow. If nothing else this will make it faster, since it won’t be redisplaying the line after every assignment. Or you could even do noshow at the beginning of this code, leave it off the entire time, then do showpage at the end (and endnowshow).

Another possibility would be to assemble all the data into a text array, then use importtext to create the new records and bring in the data all in one fell swoop.

You probably don’t want to hear this, but I doubt that adding wait statements fixed the problem, especially if 2 seconds didn’t work but 3 seconds did. Is there a form open for this database? Maybe with text editors or matrixes tied to database data? When data updates those particular types of objects go thru a very complex process to process the update. I don’t know of a bug that fits your description, but if you do have a form open with objects like that I would suggest trying the procedure with the form closed and see if that makes a difference. So I guess that makes three suggestions.

There is nothing visible on the screen being updated or displayed, except the variables that don’t change, during this procedure until the end, when the form window is closed, another window is opened, and a rundialog runs, where the user enters the fees. The process begins with this window where info about the bill is entered, annotated to show the variables:

Here’s the code that starts with clicking the next button.

Next:

if val(fgtfees)<=0  //Checks to confirm the user has added the total fee amount.
	Alertsheet "Please enter the total fees before proceeding"
	rtn
endif
console "about to call post orig.  Soft Costs are "+fgsc
changeobject "Next","Disabled","1" //Disables the Next button so the user does not preess it twice.
fgreduction=1-fgdiscountpercent  //Calculates the adjustment to be made for a discount

Callwithin PostRowData,PostOrigination,fginv,fginvdate,fgmatter,fgtfees,fghc,fgsc,fgreduction

/*. In the crashes, no data ever gets posted and the next console state does not appear */

console "finished post origination"

closewindow
openform "Bill Display Form"
fixwindow  //a custom statement that scrolls and resizes a window
zoomalign "top"
callwithin PostRowData,DialogData  // This runs a dialog to prompt the user to enter the fees for each timekeeper. 

rtn

PostOrigination:
console "starting post origination"


let lvinv=parameter(1)
let lvinvdate=parameter(2)
let lvmat=parameter(3)
let lvtfees=parameter(4)
let lvhc=parameter(5)
let lvsc=parameter(6)
let lvreduction=parameter(7)

console "lvreduvtion is "+lvreduction+"soft costs received are "+lvsc

let lvshortname=lookup("SD Matters",Fileno,lvmat,ShortName,"Missing")
let lvbillnum=stripchar(datepattern(lvinvdate,"YYYY/MM/DD"),"09")+"."+stripchar(fgmatter,"09")  //Creates a format for the billnumber that is added to each row. 

let lvorig=lookup("SD Matters",Fileno,lvmat,Origination,"Missing")  
    if lvinv="Missing" nsnotify "Invoice missing" rtn endif //Stops if an invalid file number has been entered.  This error should be caught back at the first form.

let lvO=arraysize(lvorig,";")/2  //This determines whether the origination is divided among more than one person;  the loop below is used oncce for each originator.
console lvorig
console "about to add origination "+lvO

let a=1
console val(lvtfees)*val(array(lvorig,a+1,";"))*val(lvreduction)

wait 2  //These are the two wait statements that seem to have stopped the crashes.
console "I waited 2 + 2 seconds"
wait 2

loop  //This loop runs either once or twice; that's it.  

    Addrecord
    Initials=array(lvorig,a,";")
    console Initials  //This console statement never appears and no record 
    OrigDebit=val(lvtfees)*val(array(lvorig,a+1,";"))*val(lvreduction)
    console OrigDebit
    a=a+2  
    Invoice=lvinv
    console a+" "+Invoice
    DT="B"
    Matterno=lvmat
    Date=lvinvdate
    Field OrigDebit runfieldcalculations
    ShortName=lvshortname
    DateCreated=today()
    Billno=lvbillnum
    lvO=lvO-1

    fgbilldata=strip(fgbilldata+cr()+Initials+tab()+tab()+tab()+pattern(OrigDebit,"#,.##"))
until lvO=0
console "finished orig, now costs"

if val(lvhc)>0 or val(lvsc)>0

	addrecord
	Invoice=lvinv
	Matterno=lvmat
	Date=lvinvdate
	HCDebit=val(lvhc)
	Field HCDebit runfieldcalculations
	SCDebit=val(lvsc)
	Field SCDebit runfieldcalculations
	Initials="SD"
	Billno=lvbillnum
	DateCreated=today()
	ShortName=lvshortname
	DT="B"

endif
    showvariables fgbilldata
    console "finished adding costs"
rtn

DialogData:

letfileglobal fgtkpr=""
letfileglobal fgfee=""

loop
    rundialog {Form=FeeEntry Height=150 Width=280 Sheet="TRUE" autoedit="TK"}
    TimekeeperChk=""
    showvariables TimekeeperChk
    stoploopif info("trigger")="Dialog.Close"
   
    if info("trigger") contains "Dialog.Initialize"
        objectaction "TK","open"
        changeobject "OKBut", "$PushButtonDefaultButton","0"
        TimekeeperChk=""
    endif
    
endloop

if dlgResult="Cancel"
    nsnotify "You cancelled","Subtitle","No fees were posted."
    rtn
endif

callwithin PostRowData,PostRow,fginv,fginvdate,fgmatter,fgtkpr,fgfee,fgreduction
if error
    goto DialogData
endif

fgbilldata=fgbilldata+cr()+Initials+tab()+fgfee+tab()+GenDebit
showvariables fgbilldata,fgdummy

if aggregate({OrigDebit},"+",{Invoice=fginv and DT="B"})+aggregate({GenDebit},"+" ,{Invoice=fginv and DT="B"})<val(fgtfees)*fgreduction

    Goto DialogData
    
else
    shortcall "BTTI"

endif
noshow
    closewindow
    //wait 1
    openform "BillResults"
    changeobject "AddAnother","$PushButtonDefaultButton","1"
    fixwindow
    zoomalign "top"
endnoshow
rtn


PostRow:
console "started to post a row"

let lvinv=parameter(1)
let lvinvdate=parameter(2)
let lvmat=parameter(3)
let lvx=parameter(4)
let lvy=val(parameter(5))
let lvreduction=parameter(6)
let lvTot=0
let lvshortname=lookup("SD Matters",Fileno,lvmat,ShortName,"Missing")
let lvbillnum=stripchar(datepattern(fginvdate,"YYYY/MM/DD"),"09")+"."+stripchar(fgmatter,"09")

console lvinv+" "+lvinvdate+" "+lvmat+" "+lvx+" "+lvy


console "about to add generation"
/* add generation credit. One row for each fee */

    let lvass=lookup("Timekeepers","Initials",lvx,Associate,"Missing")
    console "lvx is "+lvx+" "+lvass
    if lvass="Missing"
        Alertsheet "bad intiials"+lvx  
        rtn 
    endif
    if val(fgfee)+arraynumerictotal(strip(arraycolumn(fgbilldata,2,cr(),tab())),cr())>val(fgtfees)    
        Alertsheet "You are trying to enter more fees than the total you just entered. No can do."
        stop
    endif


    if lvass="Y"
        let lvassnum=aggregate({Initials},"count",{Initials="ASS" and Invoice=lvinv})
        if lvassnum>0
             Find Invoice=lvinv and Initials="ASS"      
        else
             Addrecord
             Initials="ASS"
        endif
        GenDebit=round(GenDebit+(0.75*lvy)*val(fgreduction),.001)

    else
        Addrecord
        Initials=lvx
        GenDebit=round(0.75*lvy*val(fgreduction),.001)

  
    endif
    
    Invoice=lvinv
    DT="B"
    Matterno=lvmat
    Date=lvinvdate
    Field GenDebit runfieldcalculations
    ShortName=lvshortname
    DateCreated=today()
    Billno=lvbillnum
    Fees=lvy
   
    rtn

So the BP database has no other windows open?

No there are two other windows that I would expect to be open, but neither of them displays any data or would be changed in any way by the procedure. The window I sent would be the active window.
Unfortunately, you are correct that adding the Wait statements would probably not stop the crashing. The crashing has continued. But the procedure worked correctly perhaps 10 times in a row, then crashed and continued to crash. (Is that clue to what is happening?)
BTW, this is the largest database we have, with almost 70,000 records.

Two other windows are open but they don’t display data?? What are they for? Sorry to keep harping on this but it just seems to me like the most likely source of this problem is a form.

It doesn’t matter which window is active. When data is changed, Panorama updates ALL open windows for that database. A window that is not currently active could still cause a problem. But if the windows really don’t display any data, then that probably isn’t it.

When the database is started, the Home window opens and .Initialize closes any other windows. Most day to day activities are initiated with the drop-down menu options on Home, but over time various other procedures accumulated until I finally collected them on another form—Utilities. A button/TDO on Home opens Utilities. The procedure in question is called by the button/TDO “OneStepBillEntry”. This whole effort was intended to speed up and streamline the process of manually adding data from a firm invoice to this system. When it works, it does that. Here are the two forms, Home and Utilities:


The database, btw, is the largest the firm uses, currently 69,773 rows of data 21.2 MB. (Older records are archived once a year.)

How, in general, could a window lead to a crash?

In looking at the images of Home, I now remember that there is a formula on the page that displays the number of invoices in the database and the date of the oldest invoice. I just moved it to another place so it won’t be touched by any procedures.

aggregate("DT","count",{DT="TI"},"",true()+"  Bills")+" Invoices Since "+ 
array(aggregate({Billno},"min"),1,".")[5,6]+"/"+
array(aggregate({Billno},"min"),1,".")[7,8]+"/"+array(aggregate({Billno},"min"),1,".")[1,4]

I had not checked if it gets updated when the problem procedure runs.

I’m not thinking so much of the window as of the objects in the window. For some types of objects, a huge amount of code runs to update the display when a data value changes.

In general, there’s relatively a small amount of code involved in assigning a new value to a field. And this code doesn’t change from database to database. At this point there have probably been billions of field assignments performed with Panorama X, maybe even trillions. If there was a crashing problem, I would think it would likely have become more apparent by now. Of course maybe not, this isn’t absolutely the case, I don’t know what is happening so I could be all wrong.

When updating a complex form object, it might involve 10x or even 50x as much code as the assignment itself. The objects each have formulas that have to be evaluated, which by itself can be very complicated. More code means more chances for a possible bug and/or crash.

When trying to track down a tricky bug an important approach is to cut down the “surface area” of the code that is being run. If we can eliminate any form updates and the crash still occurs, then the problem isn’t form updates. On the other hand, if eliminating form updates fixes the crash, we would know to look more closely at the forms, perhaps start looking at individual form objects.

It definitely references fields that are updated by the code that is causing the crash, but I’m not sure if will actually update the display when the values change. I think probably not. It’s not clear how this information would ever update other than closing and re-opening the window, but I might be missing something.

I notice that this formula runs the same aggregate( formula 3 times in a row. In a 70k record database I would probably want to use the cache( function to get that value once and save it, then use the cache to get each of the three values.

I abandoned the procedures and forms that this thread has been discussing. I have not found a way to stop the frequent crashing. Like many things since I started using PanX, back at the time of the course, this forced me to find another approach and it turns out that I like it better.

The new approach does not save any data to the database until all of the necessary information has been entered. The biggest change, it seems to me, is not using a rundialog for entering the fees. The issue with the fees is that I don’t know how many different timekeepers will appear on a bill, which is why I came up with the strategy of using a rundialog and just repeating it until the user had entered all the fees. (I have recently reviewed past invoices and found 99% have four or fewer timekeepers, so that is how many rows I included for fees.) I have never thought that the rundialog caused the crashes, because they were occurring earlier in the process, I think, before the rundialog was ever reached, although I have learned from past troubleshooting that the code may be way ahead of where you think it is.

In any event, the new approach is working well, although not 100% crash-free; it has had very few crashes. There are no window/form changes until all of the data has been added. Few enough crashes to try using. Even my older method for entering invoice information had occasional crashes, which I have never been able to completely eliminate. The new approach shouldn’t be any worse, and will be significantly more efficient for the user enter data. The final test will be whether the one person who uses this database will prefer the new approach. I don’t think she will be too attached to the older method to switch. The old method, to use a fishing metaphor, looks like a rowboat compared to a bass boat. I will send you via email the instructions I prepared for how this new approach works; you might find it mildly interesting.

BTW, there was not one single problem raised by a user last week. We are adding three more databases and four more users today. I am retiring another 3 Pan6 databases, and I will not miss them.

Well, it’s disappointing that these crashes are occurring, I would of course like zero crashes ever, but I’m glad to hear your new approach is working better.

Fantastic!