One user in particular has had this issue two days in a row, including today: and today I cannot get the database to match the server ID. Here are the details:
On opening the database, it gives a notification that it needs updating. After choosing update, a notice appears that the Database ID does not match. This copy of the the database was working yesterday evening, but this morning exhibits this problem.
So I discarded that file, logged on as a Admin, and downloaded the file from the server. Upon opening that file, the same sequence of problems recurred: notice that the database is out of date (on the copy I downloaded a minute earlier!).
I repeated the process of discarding, downloading, and opening without any progress.
This problem so far is appearing only on one computer, which is running OS 10.14.5.
Any ideas on what we can try? Next, I would delete the database from the server, upload a new copy, and distribute the new version. If I can do anything to help figure this out, please let me know.
I assume the files contain sensitive information and you can’t send them. If you can, I would like a copy of the database on the server and of the database on the client.
If not, all I really need is the data.plist file. You can extract that by right clicking on the file and choosing Show Package Contents.
The server file is in ~/Application Support/PanoramaX/Server/Public Databases. You’ll need to Show Package Contents twice, first to open the .panserverdb file, then again to open the .pandb file you find inside. The data.plist file is inside that.
Be sure to identify which file is from the client and which is the server.
I will email the two files, which I have zipped. The database name is CheckRequest, which does not have confidential information, but you should keep it private. I detached it from the server and unlocked it from the account.
In other words, you removed all the information I was looking for! Well, at least that solves one mystery, I got in and looked at the plist files and was very puzzled thinking “why does this look like it is not shared and not attached to the account?” I guess I should have more carefully read your message, well, at least I did so eventually.
If you still have the original file, please send it. I don’t actually want to open the file, just to look at the plist. Also in case it is not clear what I am looking for is the database from the client’s computer, the one that said the database id doesn’t match. If you don’t still have it, maybe you could download it again to the users computer. Of course it’s the weekend now so perhaps you don’t have access.
There is another mystery with the files you sent. Both of these files have an extra file named “Icon?” inside the file. In one file it is zero bytes, in the other it is 3.3 Mb and the data is only 10 Kb, when I think it should be over 3 Mb. I’m wondering if somehow this got messed up in the process of zipping/unzipping. I tried this with one of my own databases and did not run into this problem. How are you making the zip files? Are you using the built in zip (compress) facility in macOS, or some third party program?
I will send the file without detaching. But I also have some significant new information, I think. I created a new account on the problem computer, logged into PanX in the new account, downloaded the relevant databases, including CheckRequest, and it worked normally! So it leads me to think that there is something wrong with that user’s account (that was already a suspicion since CheckRequest was working on other computers.)
I may have changed the file icon at some time on the CheckRequest database; but it now shows the standard PanX icon and that would have been along time ago.
I am using the regular Apple Compress from the file menu (you may recall that I had problems earlier with zipping but that has been resolved.)
The computers are all running 24/7 since they are working remotely 100%.
You are up late. I wasn’t expecting to hear from you until tomorrow.
That’s interesting. Still a VERY weird failure mode. I discussed all this earlier this evening with a programmer friend of mine and he agreed with me that this is really a bizarre failure that you would never expect. I’m about to add some instrumentation to this section of the code so that if you can still get it to happen at the time you get the next version, we can track what’s going on internally. This wasn’t an area of the code I was focused on before, again, it’s an area where I would NEVER expect a failure.
Now I’m even more interested to see the non-detached file. Maybe the problem is in the download process. I might be able to determine that by looking at the plist in the file.
I may have changed the file icon at some time on the CheckRequest database; but it now shows the standard PanX icon and that would have been along time ago.
Perhaps, but I wouldn’t think you would have done that on the server copy of the database. Even if you did it on the client copy, I don’t think there is any way that could affect the server copy.
Also, what’s really weird is that the icon file in the server copy of the file is huge, about the size I would have expected for the data in the file, and the data portion is tiny. It really as like the compression swapped these two components of the file.
Oh, looks like the file you sent just came in, let me take a look.
Ok, that’s interesting. The new file you just sent me definitely has a different database ID than the file you sent me from the server. And I think they really are different files. They have different new generation histories. The file you sent from the client has 7618 records, the one from the server has 44 records. I guess the 44 records would fit in with there only being of 10k of data in that file, though leaving unexplained why there is a 3+ megabyte icon file in there. Which of those record counts sounds correct for the current state of this database? However, they both have the same list of deleted records, which seems super unlikely. That may be a separate problem. Hmm, it definitely is clearing that list when you switch a database to single user.
Is there any chance that the file you are downloading is being put in some other location and you are actually trying to open an old copy of the database that was hanging around on this computer? Perhaps that might explain why creating a new account fixed the problem? What is the process that is being used to open the file? Double click in the Finder? Open Recent? Open File dialog? From program code?
I guess at this point I would be interested to see a copy of this database from one of the clients where the database works.
I guess you are as much of a night person as I am! Isn’t it almost 2am there??
CheckRequest is a small file; 44 records sounds right. 7,600+ records sounds like a different database. I will check in the morning again. I am positive that i was not opening an old file; it is usually opened by a procedure.
If it helps, here are some new generation change notes from the 7600 record database. Do they sound familiar? Perhaps that would help in figuring out which database it is.
- Updated the send conflict memo. Allowed users to use Apple Mail instead of outlook.
- Revised send conflict memo procedure; restored option not to send email to attorneys
The 44 record database has one change note:
- Removed notificaiton emails for new check requests; no longer needed.
Perhaps the server downloaded the wrong database somehow? But weird that that would only happen on one account on one computer. Both of these plists say that the server database name is CheckRequest.
Well one way or the other this is why these databases have ID’s. If not for that, it would have tried to mix these databases, which I suspect would have had suboptimal results.
Here’s something you could try – transfer a copy of the client database from another client a separate way outside of Panorama, via email or whatever method you generally use in house to share files.
I opened the package and looked at the data.archive file with BBEdit from the shared client database that I sent last night. That confirms that this data is from a different shared database (one called SD Matters)! I have previously reported finding one of the shared databases had the wrong filename–the database looked normal except it had the wrong name on it. In those instances, the file actually ran but just showed the wrong name. This had happened perhaps 3 times. (I have been taking more notes and see that this name-switching occurred on May 13 involving two other databases.) As they say in Dutch, tjongejonge (what Charley Brown says).
Regarding your final suggestion, I copied CheckRequest directly from my computer to the problem computer/account. Once again, upon opening it gave this message (at least it’s consistent!):
Interesting.
One more test, I removed SD Matters from the problem account/computer, then repeated the copying the CheckRequest from my computer. (Nowhere to get data now except directly from the server.) So the CheckRequest database opened correctly. SD Matters, the one whose data is getting loaded into CheckRequest, is opened secretly by the .Initialize procedure of CheckRequest.
So next, I commented out the opensecret “CheckRequest” statement in the .Initialize procedure and restored SD Matters to the database folder. Now the CheckRequest database opens normally.
Hypothesis: the data for CheckRequest is being corrupted/substituted with SD Matters data during the .Initialize procedure when it opens SD Matters secretly.
If true, that may help you fix the problem and will provide me a way to work around this problem.
My workaround seems to work: I put the Opensecret “SD Matters” statement in a timer so it would have a delay of a couple of a couple of seconds before running; on the theory that CheckRequest would finish loading then move on to the opensecret step. That seems to be working. (I already had a timer at the end of the .Initialize procedure, so I just added this statement to the code, and confirmed that SD Matters had actually opened secretly.)
Yippeee! I have spent so many hours on this. I hope this workaround continues to work okay. I am going to check all other DBs for opensecret statements.
Also, I wonder if the speed of the individual computer components (processor and storage) affect this failure mode; why is this computer failing and not others? It is a one year old iMac, 16GB ram, 256 GB solid state drive, model iMac19,1, 3 GHz 6-Core Intel Core i5. Looks like a really fast computer to me. My computer at home, a Mac mini, has very similar components, if not identical. This doesn’t seem likely; especially because the failure does not occur on the same computer in a different user account.
That looks like a useful word. I guess I can’t use it, since no one would know what I was talking about. Hearing the spoken version, it sounds like Chinese to me (at least the Wikipedia version). When does Charley Brown say that?
Is that wrong? I thought the problem was that it said that the database ID didn’t match? Isn’t this different?
Huh, that is interesting.
Ok, this sounds promising. Though weird – the .Initialize code already runs in a timer. So you’re starting a second timer.
The code that sets up the connection between a shared database and the server also runs in a different timer. This is the code that checks the database id and whether a database is out of date.
I think perhaps these different timers are getting run out of order. The open secret is happening and the code that sets up the connection is happening to the wrong database, perhaps? I see that ten days ago I actually made a change that may well fix this problem. I wouldn’t have seen these symptoms because I have never tried opening a second database in the initialize of a shared database. Ok, looking at my notes I think as I added instrumentation to the code I noticed that this could be a potential problem, so I made the change even though I hadn’t observed the problem. I put “fingers crossed that this will eliminate possible intermittent problems in opening a shared database.” in the comments. So maybe I have already fixed the problem? Now you have given me an idea for how to stress test it.
I don’t think so. Perhaps there is some element of randomness, depending on the order in which the timers run? Though I think they run in the order submitted, which would not be random, but maybe if there are multiple timers set to run at the same time it is non-deterministic, I have never tested it. Of course the answer is that the code that runs in the timers must not care what the order is. That’s what the “fingers crossed” fix I mentioned above was – I noticed that a particular piece of timer code might not work depending on the order in which it was run. I fixed that one, I will hunt for some more.
I agree, I think we might be finally zeroing in on the solution. This makes much more sense.
Maybe my allusion was too obtuse. Charlie only says it in English as far as I know (Good Grief!)
Sorry, I was just using this as shorthand for the same failure mode; first you get the message that the database is out of date, but upon clicking the update button, the notification says the database ID does not match and you are then dead in the water.
I did not know that .Initialize runs in a timer; I assume there is no problem with using a Timer in the .Initialize procedure, is that right? It seems to be doing what I expect.
Why am I using a Timer in the initialize procedure?
I wanted to give users, including me, some feedback to confirm that they had connected to the server. (I think it is actually redundant of the notification, but that message is potentially ambiguous-it says Already Synchronized.) Originally I just used a formula to do this:
?(dbinfo("connected","")=-1,"<color:107F01>Connected",
"<color:FB0106>Not Connected")
But when that function is evaluated, the connection to the server might not be complete; it takes some noticeable amount of time (a second maybe?), and so it would say Not Connected even though the database did make a successful connection. So I added an ignore function:
ignore(?(dbinfo("connected","")=-1,"<color:107F01>Connected",
"<color:FB0106>Not Connected"),fgdummy)
Then I put the showvariables fgdummy statement into a timer and delayed it long enough so the connection to the server would be established. That seems to work. [The user initially sees the “Not Connected” statement on the Home form, which changes to green and says “Connected” after 1 or 2 seconds.] Finally, as I said, I just moved by opensecret command into the .Initialize procedure into the code in the same timer.
I don’t know what to think about having the two timers (the one that runs .Initialize and the one I add to the .Initialize procedure and how they may interact. I certainly don’t know the order in which things are happening during startup and what to think about that. But it seems to be working right now in the way I want.
I am just hoping for a full week without any failures.
Oh, right, yeah, I guess that’s kind of the same thing.
I actually looked it up on the internet last night, the definition I got was " 1. An exclamation of (mild) annoyance, surprise, wonder or amazement." That sounds like a handy phrase!
Oh – I guess I didn’t understand that. Hmm, is there a slight delay? I think it’s downloading the updated database, but the wrong database. Then it tries to connect to the server and gets the does not match error, because it downloaded the wrong database from the server. Yes, this is actually making sense. Most of the time this just happened in memory, so when you closed and then re-opened it worked the second time, because the timers happened to run in a different order the second time. But on Friday, she must have saved the incorrectly updated database, so that from then on it failed every time. Yes, I think this “one dialog after another” information is an important clue.
It’s not really documented. Panorama 6 did not work that way, of course it didn’t have timers. Panorama relies on a lot of Apple code to handle the process of opening a database, and that Apple code doesn’t really allow for any Panorama code to run in the middle of that process. So instead, it just sets a timer to run the code later. I really should document that, in fact, I don’t think the Panorama X documentation mentions .Initialize at all.
Your assumption is correct, there should be no problem in doing that. In general, there is nothing wrong with code in a timer starting another timer.
The good thing about your idea of putting the opensecret in a separate timer that will run later is that will make sure that the original database is all connected before opening that second database. Well sort of make sure, more on that below. But maybe that will get us to the full week without failures we are both aspiring to.
Earlier versions of Panorama were synchronous, which means they could only do one thing at a time. For example, if code made a request to a server, everything would have to hang until the request was completed one way or another.
Panorama X, however, has the ability to run code asynchronously. It’s possible to set up network requests so that other tasks can proceed while waiting for the server to respond. The programmer setting up the request can also specify code that should run when the request is complete. This is very similar to how timers work. All of this is done with the urltask( function, which I designed with exactly this task in mind.
The Panorama X client uses this capability for any request to the server that could take a significant amount of time, for example synchronizing. The client code requests a synchronization to the server, then allows other tasks to run, for example your .Initialize code. When the data needed to synchronize has been downloaded, additional code runs to complete the synchronization process. This could be a second later, 5 seconds later, 20 seconds, it all depends on the speed of the connection and how much data needs to be downloaded. So this is why I say that your extra timer doesn’t quite guarantee that everything is finished before you run it. There is not really any fixed time you could set – it’s done when it’s done.
What you really need is a way that you could run your own extra code when the synchronization process finishes. And there is a way for you to do that! If you create a procedure in your database called ..PostSynchronize, it will get called when the synchronization is complete. Voila! The only fly in the ointment is that right now, ..PostSynchronize will not get called if the database is already synchronized. One other wrinkle is that ..PostSynchronize will get called every time a synchronization is done, not just when the database opens, but that could be handled. In your case, just check to see if the second database is already open, if not, open it. So ultimately, I think this would be better than your timer, by “better” I mean “more reliable”.
Unlike .Initialize, the ..PostSynchronize procedure is actually documented! (This actually was a feature in Panorama 6 also, though I suspect no one used it. Even I forgot about it, I almost re-invented it slightly differently in Panorama X!)
To make sure I don’t forget, I’ve already made the change to make sure that the ..PostSynchronize procedure is always called, whether synchronization is needed or not.
I’m thinking maybe if there is a ..PostSynchronize procedure present, Panorama should not display a notification. Then you could do that if you want, or not, or you could customize the notification. I haven’t made that change yet, what do you think? Never mind, I went ahead and did it, and the documentation is updated.
Working asynchronously is great because it means Panorama doesn’t ever hang waiting for the server. But it makes programming quite a bit more complicated for me because the asynchronous code can start running in any context. I’m still getting used to this and have made some mistakes, and they are very hard to track down because they aren’t necessarily repeatable. Sound familiar? Hopefully if I do my job you won’t care about any of this, but it looks like maybe there is a loose end here. I wasn’t expecting an opensecret to happen in the middle of my process, but I should have anticipated this. It’s taken me a long time to get this logging feature all worked out but I think it would have caught this. But I’m glad that perhaps we’ve caught it already even without that tool.
There is a lot that I want to say about your post, but first here is another tidbit that I noticed. On the problem computer, I was using console. Upon starting CheckRequest, I get a message in the console saying “Timer _Temp_6125364698 error:dbinfo(“procedures”) is not authorized.” The role at the time was User. That is not my timer, so maybe it’s the timer that runs .Initialize that you taught me about. I have a guess of what that means, but I suspect you know exactly what it means, so I will refrain from guessing.
Now switching the Role to Administrator, the message does not appear upon starting CheckRequest. But here’s the console view showing the message, and it looks like there may be two different timers (the numbers are different):
I returned to the problem computer this morning and put the problem .Initialize code back in so I could check some additional things along the way. But, it wouldn’t fail! Tjongejonge. As I said, it was intermittent.
I think we can be sure that when it was failing, it was starting out with a valid CheckRequest file. I know that because the same failure occurred whether I had downloaded using ServerAdmin or copied it from my computer where it was running correctly. And we can be sure in both cases that the file was synced to the server. So when it opened, it should have determined that it was already synced (I say that based on the notifications I received when a file is synced-it says Already Synced.) There was no reason in that case to download any data.
The failures were all identical: first the out of date message, then the non-matching ID message when the update button was clicked. I do not recall any delay before the ID message appeared–I was going to look at that more carefully this morning. The client computer is an a separate subnet with a permanent VPN connecting them, so I went back and downloaded SD Matters this morning. On Sunday morning it should be at its fastest with little traffic. The SD Matters download was about 1 second. I could not say based on that where the data came from that ended up incorrectly in CheckRequest.
Earlier, the failure occurred several times in a row, between 5 and 10 times. Between each failure, I discarded the problem CheckRequest file, quit Panorama, and started over, first by replacing the CheckRequest file, then opening the CheckRequest file. I did not save it between failures, except on one occasion so I could retrieve a copy, which I sent to you. I did attempt later in my testing to close and reopen CheckRequest to see if that fixed the problem; it did not.
You discovered that the CheckRequest file, after failing, had the data from SD Matters inside, not the CheckRequest data. So that could come from only two places that I can think of: the server or the SD Matters file that was in the same folder and being opened secretly. The opensecret statement was very early in the .Initialize procedure-around line 5. When I commented that out, the failure did not occur. When I moved the opensecret statement to the timer, which was started with the last statement in .Initialize procedure, it was delayed by one second, and then repeated once two seconds later, the failure did not occur.
Based on two facts, I thought the more likely source for the misplaced data was the local file SD Matters being opened secretly. The facts being (1) that the failure does not occur if the opensecret statement is commented out and (2) that there is no reason for the database to download anything-it’s already synced.
Well, I’m going to go over all this code with a fine tooth comb, and also try to set up a pair of databases that work the same as yours. So let me sure I understand exactly.
-
You have a shared file named CheckRequest. It’s opened normally.
-
In CheckRequest’s .Initialize procedure, it uses opensecret to open SD Matters. SD Matters is also a shared database, but you use it with no visible windows.
Does that sound correct?
In thinking about this overnight, I think there may be another problem, which you may or may not have encountered. What if SD Matters is actually out-of-date, i.e. it needs to download a new generation? In that case, it is going to open a dialog sheet giving the user the option to update SD Matters. But SD Matters has no visible windows, so I guess that dialog sheet will attach to whatever window is open, for some other database. This is not a situation I anticipated, and I’ll bet it doesn’t work at all correctly.
The history in one of the files you sent me was for SD Matters, so I can see that you have done several new generations on that file. Perhaps that is the point where file contents have gotten mixed up.
When you first mentioned that you were using databases with no windows, that should have set off red flares for me. I should have realized immediately that I needed to conduct a top-to-bottom audit of how sharing works with secret files, rather than just find the one problem you discovered and fix it. That is what I’ll need to do now. The one bit of good news is the log system I just created will be the perfect tool for that – I think I’ve avoided secret files because you can’t see what is going on with them – very hard for me to work with. But much less so now. I guess this is going to delay the b7 release even further 
But at least now I have a very specific area to focus on.
I am not sure what you mean with par. 1. I have such a shared database, but when the failure that we have been discussing occurs, I would say no, it does not open normally. But most of the time for most users, it was opening normally. With the modifications I made this weekend, I expect it to open normally all the time. In addition, SD Matters, also a shared database, is sometimes used with windows, but in many situations it is just used to look up data, and I have tried to keep it “secret” to avoid window clutter.
I have had situations where a database that is being opened secretly is out of date. I don’t recall any details of how that happens, but I can easily test it. And I don’t recall having a problem/failure as a result. But I will try and see what happens.
By “it’s opened normally”, I mean that the technique you use to open it is “normal”, for example opening it by double clicking, using the Open File dialog, etc. You are not opening it some unusual way like via a .Initialize procedure. I think my understanding is correct, even if my wording was not clear.
Good, that’s what I thought.
Sure, I’d be interested in what you find if you try that. But since everything is invisible, you might not see an apparent problem right away. So whatever your results, I’m going to go over the code involved with a fine tooth comb, at ever step thinking “what if the window is invisible?” I’ve never done that before, and unfortunately the code wasn’t really written with that in mind. It probably won’t take major changes, maybe none at all at this point (some changes already made), but my plan is to be thorough.