Corrupted PDF Problem

[JamesCook]

(This was originally posted under Stray Clicks but has been recognized as a separate problem and reposted)

A LoopArray is generating more than 200 PDF reports. Prior to recent updates of the OS and Panorama, it had been running correctly. This past week it was discovered that a seemingly random number of the PDFs were riddled with field and variable errors. It seems to be somewhere around 10% scattered within the overall output and does not affect the same reports on each run.

Months ago the same printtopdf procedure was randomly having similar issues using the code below:

printtopdf Path+Name,”Database”,XYZ,”onerecord","true","Form”,”Account Report"

In this case, ”Account Report" was not the open form, but too often another open form printed, or the unopened datasheet or an unopened form, and occasionally the procedure itself got printed. Most would be fine, with a smattering being wrong.

So I made sure that ”Account Report" was open and active and the same line of code became far more reliable but still had occasional failures. I made it entirely reliable by adding a wait and showpage:

ShowPage
wait 1
printtopdf Path+Name,”Database”,XYZ,”onerecord","**true**","Form”,”Account Report"

That is where it has been until the noted recent updates of the OS and Panorama. I have it working correctly again (fingers crossed) by adding more waiting:

wait 2
ShowPage
wait 2
printtopdf Path+Name,”Database”,XYZ,”onerecord","**true**","Form”,”Account Report"

The percentage of errors clearly goes up or down depending on the length of the wait. The server running this is an M4 Mini on OS 15.1.1. The same problem occurs on an Intel i7 Mini but requires less wait time to avoid the issue.

My guess is that the screen has to catch up with the process before printing, although watching the screen I never see the errors.

I don’t know if the wait is more effective before or after the ShowPage, if only one of them is making the difference, or if it’s the cumulative value that matters and the ShowPage is unnecessary. I need to experiment more on that but each change requires quite a while to test and then check the results.

Especially since the i7 used for development is more tolerant than the M4, where clients log onto a website to review their accounts, there’s a lot of fear in making any changes. Client confidence runs low when they see garballed reports.

[JamesCook]

Changing the wait times was affecting the number of errors, but when it appeared they were all coming up “clean”, it turned out that many were being saved to the wrong file path even though my log showed everything to be correct.

It does seem to be resolved now and it may be that the root issue is the previously reported problem with the Loop statement although the symptoms gave no clue of it.

Problem with LOOP statement

I want to create an array of comma separated whole numbers from 6 to 99. The following code works except that the number 37 is repeated. let Z="" let n=6 //first number of the array loop Z=Z+","+n n=n+1 stoploopif n=100 endloop Z=Z[2,-1] // gets rid of the leading comma The problem section of the array looks like “…35,36,37,37,38,39…” If I change n to 7, 38 is repeated. If n is 8, 40 is repeated. Can anyone explain this?

My procedure was looping through 50 at a time. In taking several steps in an effort to resolve this issue, one of them was to reduce the LoopArray to processing 25 at a time. Since that change, the procedure has been running correctly. I had also removed the wait statements aside from one Wait 2 at the end of the loop.

This week I’ll test it further on a different machine. I’m not about to mess further with a live server. If it’s not broken, I’m not about to fix it further.

[RAmeeti]

Your perseverance is appreciated by many.

[pcnewble]

However, the bug whereby every 32nd iteration of a loop was repeated was supposedly fixed in 10.2.0 b37. See Panorama X 10.2.0 b37 Build 4442 Release Notes

[JamesCook]

@pcnewble

However, the bug whereby every 32nd iteration of a loop was repeated was supposedly fixed in 10.2.0 b37. See Panorama X 10.2.0 b37 Build 4442 Release Notes

Agreed, that’s why I wasn’t terribly concerned about looping through 50. But if reducing the loop from 50 to 25 proves to be the difference, it could be a relative living in LoopArray.

[JamesCook]

@RAmeeti

Your perseverance is appreciated by many.

Thanks, there is sweat on my brow, unmentionable hours, and quite a few sleepless nights on this one. I’ve been trying to get this vital procedure running properly since May. I sure hope this is finally resolved - aside from vulnerablility to stray clicks.

[admin]

The bug whereby every 32nd iteration of a loop was repeated was not in the looparray statement, it was in the loop statement, and even then it was only when the loop was potentially infinite - i.e. using endloop or while forever. There is not any known problem with the looparray statement, and never has been. In spite of the similar name the code in looparray is completely different and is not susceptible to the same problem that occurred in the loop statement.

@JamesCook

My procedure was looping through 50 at a time. In taking several steps in an effort to resolve this issue, one of them was to reduce the LoopArray to processing 25 at a time.

This makes no sense to me. If you’re using looparray there is no count, it simply handles whatever number of items are passed to it. Since you are explicitly controlling the count, I think you must not be using looparray. At any rate, I can’t comment on why your fix worked since I don’t at all understand what you were doing before or are doing differently now.

[JamesCook]

@admin

@JamesCook

My procedure was looping through 50 at a time. In taking several steps in an effort to resolve this issue, one of them was to reduce the LoopArray to processing 25 at a time.

This makes no sense to me. If you’re using looparray there is no count, it simply handles whatever number of items are passed to it. Since you are explicitly controlling the count, I think you must not be using looparray. At any rate, I can’t comment on why your fix worked since I don’t at all understand what you were doing before or are doing differently now.

I’ve broken my hundreds into smaller arrays and am feeding 25 at a time into a LoopArray. Minutes later my timer sends another 25 to the procedure with the LoopArray. So far my endless variety of errors seems to have cleared up.

It does take a while to rnake each full test run, then about an hour to check all the results, but I have another test running now on my development server. I’ll post onight about those results. I’ll check the live server in the AM.

[admin]

@JamesCook

I’ve broken my hundreds into smaller arrays and am feeding 25 at a time into a LoopArray. Minutes later my timer sends another 25 to the procedure with the LoopArray. So far my endless variety of errors seems to have cleared up.

Ok, I think I understand what you are doing now, at least as far as the loop goes. I wonder if the problem might have something to do with running a bunch of printpdf’s in a row. Maybe there is something internal to Apple’s PDF generation code that can get overwhelmed. I have never tested generating more than one PDF at a time.

[JamesCook]

This morning, a heavy-duty spot check of 296 PDFs on my machine and then too on the live server turned up no errors. The difference is taking smaller bites with the size of the array being processed by LoopArray.

@admin

I wonder if the problem might have something to do with running a bunch of printpdf’s in a row. Maybe there is something internal to Apple’s PDF generation code that can get overwhelmed.

It did seem that something was failing to keep up. Adjacent to the PrinttoPDF line, a log was recording the account number, path and pdf name.

70145 - /70145/activity/70145_Interim_Stmnt_2024-12-15.pdf
70150 - /70150/activity/70150_Interim_Stmnt_2024-12-15.pdf
70165 - /70165/activity/70165_Interim_Stmnt_2024-12-15.pdf
and so on…

In the last error-prone run it recorded them all perfectly. But, once in a while, a series of PDFs were saved with the previous file name and path. So the PDF output for 70150 would be saved with the name and path for 70145.

A short time later, they’d be correct again.

Other times some got the totally garbeled output. I don’t know what it was in all of my throwing things at the wall that eventually cured it but somewhere along the line it stopped.

Re: running sveral PDFs in a row, as I know you realize, there’s other activity between each one…selecting records, creating summary records, doing some math…

My Intel machine seemed far less prone to the issue though it occurred. The M4 needed smaller bites before handling it. Apparently processing lines of code isn’t necessarily proportional.

At this point I’m holding my breath and hoping for no more reports about getting weird reports or someone else’s.