Starting yesterday, it appears that PanX is not syncing clients correctly. That is, one user enters a record in a shared database, a second user opens the database and the new record is not added; even after choosing ‘Synchronize Data’, it says the second user is already up to date. Both clients have build 3581 installed, and the server is build 3581. I suspect the issue arose with the installation of this build.
The record in these cases is added to the server, and a Download Data action will update client #2 and show the newly added record.
I have observed this several times now with several different clients.
Let me know if I can do some logging to help diagnose this. If needed, I could help after the classes today, although It’s my day to cook, which my wife says is my highest priority.
In another test, just now, I added a record from one client machine, then went to another client and spend the shared database, and the new record was present. I then deleted the record on the first machine, but the deletion did not sync to the second machine. The record still appeared on the second machine, but, of course, I could not edit it.
I guess you could try re-installint the older build. But the changes from 3575 to 3581 were really minimal, I didn’t go anywhere near the synchronization code or even touch any code related to sharing at all.
did not sync to the second machine
I assume you mean that the record didn’t disappear after a synchronization.
There is instrumentation that could help track this down but right now my brain is fried.
Sorry you have a fried brain. I know the feeling and will wait until you are ready to return to this issue. But in the meantime, I will add some information that I have just collected.
On computer 1, I opened a shared database. I can see that the data is not complete (I am familiar with records that have been added recently and they are not appearing on computer 1.). There are 7,888 records.
On computer 1, I choose Synchronize Data; a notification says Already synchronized 2577.
On computer 2, I open the same shared database. I can see that it is also missing recently added records. I choose Synchronize Data and get a notification saying already synchronized 2577. But it has one more record than computer 1: 7,889. I see the one additional record.
On computer 2, I choose Download Data. Now there are 7,896 records. The records recently added by other users now appear on computer 2. I choose Synchronize Data, and the notification says Already synchronized 2577. (The 2577 number has not changed.)
Edited 8 records on Computer 2. I choose Synchronize Data, and the notification says Already Synchronized 2581.
On computer 1, I choose Download Data. Now there are 7,896 records. I choose Synchronize Data and the notification says Already Synchronized 2581.
So my conclusion is the the changes are being added to the server when made on a client, but they other clients do not synchronize and download the changes to other clients.
They syncing problems continue today. This morning, another user added a record to a shared database. That database sends an email to others in the company to notify them of the new Information. I opened PanX and the record was present in my copy. That’s good, I thought.
A little while later, another said they could not see the record even after a download data. Now when I try to edit the record an alert says “Cannot lock - record does not exist”. It seems that the record had been on the server but has been deleted. (No user except me has the ability to delete records; that is highly restricted.)
I did a copyrecord/pasterecord to restore the record to the server.
I wonder if there is a way to use Instrumentation log to understand better what is occurring?
In the instrumentation panel, you can enable Shared Database Basic Operation to log all basic sharing operations, which includes record locking, adding records, and synchronizing. If this is enabled on the server computer, it should record this activity for all computers on the network. The clients automatically send their logs to the server and they are all combined into a single log on the server computer.
This problem had been intermittent, occurring when different client computers added the original record and occurring in different databases. But it was not frequent, perhaps once every few days, and has not occurred in over a week. I am prepared to gather a lot more data about it if a case occurs. I have started logging some information that might be useful: the user, timestamp, and server record ID when a record is added to one database, which has had the most problems. So the problem has not occurred since the original post.
But another problem occurred: I discovered to records for the same matter#, which is supposed to be impossible. The records had original time stamps about 15 minutes apart. The user didn’t tell me when it happened, I just found them in reviewing newly added records. He later said that his computer crashed during adding a new record. That is not precisely correct; the procedure adds the record with minimal data, saves, and the the user is prompted to fill out more information. He had added most, but not all, of the required information. That is apparently when the crash occurred. He said that he went to a second computer and could not find the record he had just added, so he added it again. (This is the one step that would clearly violate the rules I have set up, so how this happened, I do not know.) The original record was on the server, so he should not have been able to add another record with the same matter#.
My plan is to try to gather a lot more information about one of these events. I don’t expect you, Jim, to do anything with this report along. I have some questions about logging that I will put into a separate post.
Regarding logging: I had already been running the log enabled in Preferences/Server pane, set to Error level. The instrumentation log you are describing is something in addition, correct?
I assume that I have to enable this on both client computers and the server in order for it to behave as you describe here, is that right?
If so, then I will plan to enable this on the busiest users, where I have the best chance of encountering the problem I have described.
Another question: is there a way to log a crash? When PanX starts, if it has just crashed, that would be useful to log, so I could get better data about the frequency and the database related to the crash. Users do not necessarily tell me a crash has occurred; they just restart and try again.
Yes, that setting only controls logging for the GCDWebServer component. I don’t think that log is of any value for Panorama related problems. Fortunately the GCDWebServer component seems to be very stable and problam free. But it has it’s own logging system built in, so the server panel allows that to be configured in case a problem with that does ever come up.
If I gave the instructions properly, this only needs to be done on the server computer. You don’t have to do anything on the client computers. This only works for favorites that have both a plain and (server) variant. With this setup, the log isn’t actually kept on the clients – the log information is passed to the server each time the client communicates with the server. This has multiple advantages:
• Log only has to be enabled on the server computer, nothing needs to be done on client computers
• Only one log is generated, and it contains server logs and all client logs.
• The combined log allows tracking of individual operations from the client to the server and back to the client. The log automatically records activity for each operation in order, and also indicates what activity happened on what computer.
Thomas, I actually created this system with your organization in mind. It seemed from your reports that your users probably couldn’t be relied on to configure and collect logs reliably. Of course that’s probably true in a majority of organizations. So I invested quite a bit of extra work in making this network coordinated logging system. As it turns out, we haven’t tried to use it until now.
One fly in the ointment – I never got around to documenting this. It shouldn’t be too complicated to do that, but it just hasn’t happened yet. I need to do that and also create a presentation on this for a class session. So the instructions I gave above were from memory, which may be a bit fuzzy after 6 months. Hence my admonition about the possibility of the instructions being incomplete or possibly even wrong. If you have problems with the instructions, I’ll have to accellerate working on the documentation. But the system definitely does work – I used it for some of the later debugging of client/server issues.
Not really at the moment. I’ll have to think about that.
I cannot find anything name ‘Shared Database Basic Operation’ in the Instrumentation Panel. However, I do see in _EnterpriseServerLib many procedures covering the things you list above. Is that what you are referring to?