Breakthrough yesterday

I made a breakthrough yesterday, I feel like telling someone about it, even though it is really entirely “inside baseball.” So the three of you on this beta forum have the dubious honor of being on the receiving end of this missive.

When opening a shared database on the client, Panorama performs these steps:

  • load the file from disk into ram
  • contact the server
  • if necessary, load updated new generation from server
  • if necessary, synchronize data from server

Of course, it’s important that these steps happen in the correct order. And they do.

However, there is a somewhat common situation I neglected to think about when developing this. What if the shared database is opened via a procedure, like this?

opendatabase "my shared database"
// more stuff happens
// after database is open

The problem is, up until now Panorama X has performed this in the wrong order, like this:

  • load the file from disk into ram
  • contact the server
  • MORE STUFF (what is this doing here??)
  • if necessary, load updated new generation from server
  • if necessary, synchronize data from server

That is obviously very wrong! What needs to happen is for Panorama to set MORE STUFF aside, and wait until all the server activity is done, then MORE STUFF needs to resume. I already had other areas where code was temporarily set aside and then resumed later, so I was pretty sure it would be possible. Problem was, when I wrote code to do this, it didn’t work. I was setting MORE STUFF aside, but somewhere in the very complicated code to get the server connection established, MORE STUFF was getting corrupted, so that it didn’t run at all at the end.

This proved to be very difficult to untangle. In fact, I was pretty much stuck for six days. There was nearly ten thousand lines of code that were possible suspects. Kind of a needle in a haystack. I wound up building some special tools just to help filter out possible suspects (and if I hadn’t already created the new Debug Instrumentation feature, I would have needed to for this, couldn’t have found the problem without it).

Finally, yesterday I identified the culprit. Once I found the spot, it only took a few lines of code to fix the problem. Now it reliable does everything in order.

  • load the file from disk into ram
  • contact the server
  • if necessary, load updated new generation from server
  • if necessary, synchronize data from server
  • MORE STUFF (yeah!)

What a relief! When people ask me when a release will be ready, this is the sort of issue that makes predictions nearly impossible.

If you’ve read this far, thanks. Since you may be interested, I’ll tell you about a related issue that I solved before this, with less angst. If a single user database has an .Initialize procedure, you might expect it to work like this:

  • load the file from disk into ram
  • run .Initialize
  • MORE STUFF

And in fact, Panorama 6 and earlier do work that way. But Panorama X has note. Instead, it would run in this order:

  • load the file from disk into ram
  • MORE STUFF
  • run .Initialize

But this isn’t really right. So now, starting with 10.2 b7, it will run in the correct order:

  • load the file from disk into ram
  • run .Initialize
  • MORE STUFF

In addition, Panorama X is now more flexible about what kinds of code can be included in the .Initialize procedure.

I did the rearranging of the .Initialize first, because I thought it would be easier. And I was right, I was able to get that working fairly quickly. I thought the same trick would work for the server code, and ultimately it did, but with the extra wrinkle that took six days to figure out. By the way, it all works together now if you have a shared database with an .Initialize procedure, everything runs in the correct order.

  • load the file from disk into ram
  • contact the server
  • if necessary, load updated new generation from server
  • if necessary, synchronize data from server
  • run .Initialize
  • MORE STUFF (yeah!)

I still need to do some more testing, for example to confirm that the .Initialize procedure will work if it contains opendatabase statements that open other shared databases, and to confirm that server variables now work in .Initialize code. I think these will work, but they haven’t been tested yet. Once the testing is finished, the next step will be to finally release b7. So I’ll stop writing now and get on with the testing.

hip hip

1 Like

I agree with Gary.

This GIF is awesome. I looked at in in Preview and it is only 10 frames. Amazing.

Congratulations! Looks like an important milestone. :tada:

Oh wow … that is really an important fix.
And tricky to hunt down … way to go, Jim!

(Plus: I agree with Gary, too!)

Update, after further testing I have confirmed that server variables now work in .Initialize code, and you can use opendatabase and opensecret in .Initialize code, with both single user and shared databases. And it works perfectly even if there is a new generation available and even for secret databases, which was the problem that caused Thomas and I so much frustration a few weeks ago.

A few weeks ago I thought I might not be able to get opendatabase and opensecret working in .Initialize code, so I came up with a new panel in the Database Options dialog that allows you to specify additional databases that will be automatically opened when a database opens, without any code at all. Although this turned out to not be necessary from a bug fix point of view, I am still very pleased with this feature, which I think is much better than the old “file set” feature from previous versions of Panorama.

Note: The auxiliary databases are opened after the .Initialize code runs, so code in .Initialize cannot rely on these databases already being open. If you need that, just use the opendatabase/opensecret statements as before.

So, my inquiring mind’s first thought was if it would be an acceptable scheme to have the auxiliary files stored inside the package of the main file. It would make the main file and the additional files all self contained as a single file to the user but might cause other problems like with a Save As. Ok, just tried a Save As with another .pandb file within the main files package and the new file included the additional file within it. So…

I had a somewhat similar thought. Still pondering.

Cool.

I’ve been doing something similar with Pano6 using an AppleScript application bundle, with the collection of databases in the “Resources” folder (actually, I have a “DBs” folder which contains all the databases – but the idea is the same)

All the AppleScript does is inspect the bundle’s “Resources” folder and tells the Finder to open the Pano6 database. The Finder, in turn, launches Pano6 and opens the selected database. After that, the AppleScript quietly exits and Pano6 is running with the database, and it can, in turn, open any other DBs that it needs in the “.Initialize” procedure.

Sometimes ‘auxiliary’ files are also standalone database files. I use the opensecret… /auxiliary files to improve the user interface-avoid window clutter and to make sure the user has opened needed files without any user intervention.