Today two users reported problems: one could not edit a record and received a message saying “Cannot edit, Field or Variable [sessionLine] does not exist.” [I don’t I received the same message when I tried to edit the record in this shared database. Upon checking, I found there were some locked records, so I did a forceunlockallrecords. That solved the problem. The user said that he had had a crash yesterday.
The second user reported several crashes in a row. Upon checking the server, there were numerous locked records (9 in one DB and 5 in another that works with the first one.) The crashes stopped after unlocking the records.
Both users in this case are working remotely with a VPN connection to the office; I wonder if a computer going to sleep causes a problem, or losing a VPN connection causes a problem, or closing a laptop while a record is locked. I am going to try to find steps that cause the problem. [I cannot remember what you have written about unlocking records, but I am going to look for your earlier comments. Did you say PanX will unlock a record after some period of time? There is a re-lock time set for the databases involved.
In any event, this has been an ongoing issue that arises periodically.
Both users are working remotely connected with a VPN to the network with the server.
I wonder how the cycle starts.
I suspect that some users working remotely, in some cases with a laptop, close the laptop and leave PanX running. If one does that while a record is being edited, I suspect that the lock may not be removed. I also wonder about the effect of losing the VPN connection or other unknown network issues. (The second user just started working at home with the same computer that had been located in the office; of course, she had crashes in the office also.)
I see that if I start editing a record and then click on a window of a different database, then the record remains locked and the timeout set is ignored. The server shows that it continues to be locked.
Looking back in the Forum, I see discussions about the record unlocking in April. I suggest that a record should lock immediately if the user switches to a different database by whatever means (clicking another window; procedure switches, etc.). I have tried putting my computer to sleep while a record is being edited; in that case, the record remains locked. In one test, I put the computer to sleep while editing, then woke up the computer a minute later and could not resume editing the record and got a message that it was locked by another user, which happened to be me! So I also suggest unlocking the record immediately if the computer is going to sleep; I assume the OS notifies the apps that are running so they can do something before sleeping begins(?)
I suspect that having locked records can lead to crashes, which can lead to more locked records. That’s my hypothesis anyway.
Sorry for the delay in responding, but there are quite a few moving parts in your post and I wanted to give myself time to put together a thorough answer. Let me see if I can untangle this at all.
First of all, locked records should never lead to crashes. Of course nothing should lead to crashes, but locked records in particular are a normal thing, to be expected. So client software needs to be able to handle this.
I did some sleuthing, and there is no sessionLine variable in any client software. There is a sessionLine local variable in the server software however, and it is in fact in the code for locking a record. If an error occurs in this code it is sent back to the client and displayed on the client. If you have enabled the Daily Sharing Error Log in the Preference>Logging panel it will also be logged there. (If not, it doesn’t matter, I don’t think this would really provide much in the way of useful information.)
So we have two issues. First, there seems to be a problem in the server code. Just looking at the code, I don’t see what the problem is. The sessionLine variable is defined with a let statement. The next four lines extract information from this variable by using the tagparameter( function. Then the variable is never used again. There is no conditional logic involved, there is no logic that would skip the let statement. So this variable should always be defined before it is used. Yet somehow it isn’t.
The second issue is that the client is crashing when this error occurs. Or is it? You said the message was displayed, and you mention a crash, but re-reading this, maybe the crash happened earlier? The message is a normal response if the record can’t be locked – normally it displays a message that the record can’t be locked because it is in use by another user, but it will display ANY message the server sends back.
Ok, thinking aloud, I think the sequence is that while the user was editing a record, Panorama crashed, so the record was left locked. When they went back to edit the record, it wouldn’t let them, which is correct, but the wrong message was displayed, and it seems that something wrong happened on the server.
However, then you mention several crashes in a row, and that unlocking the records stopped the crashes. That makes it sound like trying to edit a record caused the crashes. But it sounds like the user didn’t report the exact sequence. Did the crash occur immediately when he or she tried to edit the locked record? Or was it somehow in the “vicinity”?
UPDATE – I take back the part about not seeing how sessionLine could be undefined. There are a couple more lines that access the sessionLine variable about 50 lines down, and there is one possible logic flow that would result in this variable being used without accessing it. So that may solve that mystery. I have now changed the code to ensure that this variable is also defined in this alternate code path. Maybe this will solve the problem??
This is a bit odd, since the message is definitely generated on the server.
That is not odd, but exactly what I would expect. The code in question is code on the server that would only run if a client locked a record, crashed, and then logged on again. If the server sees that a record is locked by a client session that no longer exists, it will automatically unlock it. That is exactly the code where the sessionLine variable is used. So if no records are locked, this code will never run. It will also never run if a record is locked by an active user. It will only run if a record was locked by a client that had then crashed. In other words, it will be quite rare for this code to run. What the code is supposed to do is to detect this, then automatically unlock the record (since no active user has it locked this is ok) and allow the client to edit the record. Hopefully the code change I just made will allow that.
I think it starts with a crash.
This scenarios would leave the record locked. However, the server will think that the record is locked by an active user – it has no way of knowing the client is closed or offline. If another user tries to edit this record, they will be told the record is busy and can’t be edited now.
When a Panorama client first contacts the server, a “session” is started up. This session has a unique id and remains active until all shared databases are closed, then the session is closed down.
If Panorama crashes, the session is left open on the server. However, if that same computer connects to the server again, Panorama will notice that there is already a session open for that old computer, and it will shut down the old session before starting the new one. So the only way the server can realize that a client has crashed is if that client suddenly requests a new session.
Not that I know of. If it does, I highly doubt that the OS would wait around while the client contacted a server. Unlocking is not something that can be done on the client end, it has to be done on the server end. Also, to unlock a record the client must pass the updated record to the server – otherwise you’ll lose your changes.
This really surprises me. I would have expected that to work.
Right now Panorama client has the ability to set a timeout for record locking. If there is no activity, when the timeout occurs it will automatically shut any editing window, send the update to the server and unlock the record. This is set up on a database-by-database basis in the Database Sharing Options>Sharing panel. If you change this after the database is already on multiple computers, you have to manually change it on each computer. If you set it up before sharing, it will be automatically transferred to each computer. However, if you do something like start editing and then close the laptop lid immediately, I don’t think this will kick in. (I’m not sure about that – I’ve discovered that Panorama Server will work on a laptop with a closed lid, as long as it is plugged in.)
There is logic in the server code to implement a timeout on the server, but so far there is no way to turn this on and I have not tested it. This could be super disruptive because someone could think they were in the middle of editing something, and the timeout could kick in and all their changes would be lost if another user decided to start editing. Since I don’t see any way around that, I’ve held off on finishing the implementation of this. I guess I’m hoping that I will think of some miracle solution with no downside, but I don’t think there is one.
A while ago you reported that users could add records even if the server was crashed and the client would complain. I’m in the middle of doing that. It turned out to be a significant project. I’ve got it working for adding and deleting records, and I was working on record locking/unlocking when your message came in. So the way the client handles server errors may change a bit. I’m hoping to wrap up b11 in the next week or so, fingers crossed.
No problem. I trust you to respond if and when needed
Amen.
The crash was earlier, and I hypothesized that the error message was related to an earlier crash. After running forceunlockallrecords I was able to edit the record without receiving the message.
One persistent problem, which I am sure you are well aware of, is that the users almost never report the information that we would like. Not surprising; someone is working away at their job, and not mentally logging the steps that they have taken. I have asked people to tell me every time a crash occurs, but I am sure that is not happening. They just restart and if it works they do not report anything. I don’t suppose there is some way for me to find out if PanX is started directly after a crash?
My fingers are firmly crossed; I always look forward to the updates.
I would not expect a client to go to sleep automatically when a record is locked, assuming an auto-unlock period has been set, which would normally be much shorter than a sleep timer. A user could put the computer to sleep while a record is locked, but then it would not be unreasonable that some changes could be lost, and I don’t see why anyone would do this intentionally but only due to a distraction or something like that.
How long would the server wait for a keep-alive message from the client? Would you want to unlock it on the server before the user restarts, which might only take a minute, or even less. In thinking about this, I realize that I don’t remember what is happening now if Panorama crashes and has left behind a locked record and the client reconnects. Just some thoughts about this topic. I don’t have an opinion now about how this should work. Except that having stranded locked records is not a good thing.
No, I wouldn’t either. This would likely be triggered only if the user locked the record and then soon after shut the lid. Something I personally would never do, but users…
Exactly, not unreasonable.
The server would wait indefinitely – UNLESS another user requested to lock the record. Then the server would check to see how recent the last keep-alive had been. At that point I would think if the last keep alive was more than perhaps 90 seconds ago the server could give the record to the other user.
If the client crashes, they cannot restart and resume the lock. When they do restart, the server can realize that the locked record is not really locked and give it to others (or them). But now, that can’t happen until the user restarts. The keep-away idea would allow the server to realize that a crash has probably happened even if the user doesn’t restart Panorama.
Basically this idea would allow the server to be a lot more proactive in identifying stranded locked records on its own, so that manual intervention wouldn’t be needed.
I was able to duplicate the problem of locking a record, sleeping, then trying to edit again and it saying it couldn’t be edited because the record was already locked by me. Investigating this further, I found that the server code was supposed to detect that the same user was trying to lock the record, and allow the re-lock, but there was a one line bug preventing it from doing this correctly. This bug is now corrected
I have also completed adding the code so that if the server is down or the connection broken, it will definitely notify the user of the problem if they try to add, delete, or edit a record. The Server Admin window now has a hidden option to simulate a broken connection (hold down the Option key while clicking on the server’s icon), which was very handy for getting this working, and for debugging the problem mentioned above (doing this was essentially the same as sleeping, but easier to work with).
