Seemingly random data written to file

After an error while running a procedure, I again found seemingly random data written to a data cell. My procedure was sending emails via AppleScript to Apple Mail. A timeout error occurred and I Force Quit Mail. In my first record, first field, which is a Name field, there now appeared a series of numbers, separated by commas. ie. 1,2,3,5,10,11

The procedure should only be reading from the First name field to do a mail merge. When it typically runs, all is fine. But this is not the first time that after a crash, that the topmost record, first data cell, will contain odd information. The last time it contained the ServerSharedRecordID or something similar. The file is a shared file but ony used by 1 computer.

Well, I figured out what the seemingly random data was.

After a crash, the 1st cell is being filled with the Info(“ServerRecordID”) of the selected records. Few things in this world are truly random, we usually just do not understand what we are looking at. And now it is for ProVUE tech support to figure out why this is happening.

You say this is a shared file, but I don’t see the Wifi symbol in the tool bar to indicate that it is shared. That makes me wonder if you are using a really old beta version, though I’m not sure if you had access to a beta version old enough that there was no wi-fi version. Of course you could have converted it to single user to make the screen shot.

Assuming this really is a shared file, was the “random” data on just this client, or also on the server? I realize you probably have no way to tell at this point, but if it happens again, it would be useful to know whether this originated on the server or the client.

On your server, is are the Bulk Data Modification and/or Individual Record Detail logging options enabled?

If these options are enabled, the server will generate a comma separated list of id numbers to put in the log for operations like formulafill. Of course this list is supposed to go into the log, not into a cell, and looking at the source code I don’t see how it could go astray. But that is the only place I can think of in the Panorama code where a comma separated list of id numbers is generated. This log is generated on the server, not the client, hence my earlier question.

Oh, I thought of another place – synchronization also can put a comma separated list of record id’s into the log. Again this would be generated on the server, and I don’t see any way that the list could get into a cell (it’s kept in a local variable), but apparently it is getting into a cell somehow.

The only check that I have in the Server Logging (in the upper portion) is the Daily Sharing Erorr Log. The rest are unchecked.

The file was supposed to be shared but you are correct in that the WiFi icon was not there. I found that it was not ‘online’. After sliding the button to take it online, it did not want to ‘Upload New Generation’. I believe it would be better if when a Shared file was opened, there was a notification 'Opening << FileName >> on << ServerName >>. I’ve too many times found that I was using a Shared file that was not online with no ‘in my face’ notice that it was shared.

In the Server Administration:Main, on the Available Servers tab, when I clicked on one of the servers, I got an Error Wizard stating "Window “Server Administration” does not exist, then the Server Administration window did present or refresh.

As for Sychronization, on my shared dbs, it used to be that the Summary Records would be removed during synchronization when I opened a shared file but they are now being retained (which I prefer.) Has this aspect changed? It was explained earlier why they were not kept but apparently that is no longer valid?

The behavior you are describing is all wrong. If a file is shared, there must ALWAYS be an icon in the toolbar. If the file is connected, it will be a WiFi icon. If it is not connected, it will be a lock icon, and the data cannot be edited.

image

But if there is no icon at all, that means the database is single user.

Sliding the button? What button? I don’t know of any sliding button to take a database online.

There is a notification. I just tried it right now to verify that it is still there. Are you sure you have Panorama X notifications enabled?

The name of the window should just be Server Administration, not Server Administration:Main. I’ve seen this name change if I’ve switched the window to graphics mode, but I assume you didn’t do that. If you close and re-open the window it should have the correct name. In any case, as you discovered, this error message doesn’t cause a real problem.

No, nothing has changed. Summary records are always removed during synchronization.

I’m coming to the conclusion that your database is not actually shared on a server, but is single user.

In the File menu, is there a Connect to Server or Disconnect from Server menu item when this database is active?

Sliding the button? What button? I don’t know of any sliding button to take a database online.

The slider button in the above screenshot says that it is being taken offline if you slide it to the right, and it says that you are taking it online if you slide it to the left. When I was needing to debug the situation, I found it slid to the left.

My file had no icon. I found that it had the slider to the left. I slid it to the right and it said that it was taking it online. I then was able to do a new Generation Upload and after a few failures to get this to happen, it did upload and all was apparently fine. I know that I did not turn the file into a Single User as I had just moments before ran a procedure which did an email merge from data in the file. I noticed that the first record had the bad data. That data was only able to be coming from the server. The fact that it was then an offline, unshared file was not my doing. It happened at the same time that the server data got written to the file. This is not the first time that that has happened to me during this procedure.

Yes, I do have Notifications turned on and they work as far as I can see.

A moment ago the Server Administration did just say "Server Administration’ but now when opening it again, it again has :Main

I am not taking any steps but the normal, prescribed steps.

Sometimes my Summary records are removed. Sometimes they remain. I’ve been seeing them removed more often than not and was appreciating the change. O well.

Ok, glad I asked. That button controls the server, it does not affect the client at all. If the button is slid to the left then clients cannot connect to that database. Sliding the button to the right does not cause the local copy of the database to connect. If the local copy is not connected, you would need to use File>Connect to Server, though that won’t work if the database is offline on the server.

When you start a New Database Generation that changes the field structure, Panorama automatically takes the database offline (slides the button to the left). This is to prevent other users from opening the shared database while you are working on the field structure (or doing anything else, for example starting another new database generation). When the final structure is uploaded, the database is automatically brought online again. So I would guess that at some point you started a New Database Generation that was never finished.

My file had no icon. I found that it had the slider to the left.

These are separate issues. If a file has been set up to connect to a server, it will have an icon, whether it is currently connected to the server or not. The icon will be different depending on whether the file is currently connected, but there will always be some icon. If there is no icon, that means the file has not been uploaded to a server. Is there any chance you have two copies of the database in different folders, one that has been uploaded and another that hasn’t?

This is not the first time that that has happened to me during this procedure.

Maybe I need to know more about this procedure.

Server Administration

I notice that your server is listed twice because it is a local server but also defined as a remote server. I don’t think that will cause operation problems, but it should only be listed once. I have opened a bug report for this.

Sometimes my Summary records are removed. Sometimes they remain.

If no synchronizable changes have been made to the database then it won’t sync, in which case the summary records would not be removed. Since you are the only user this might happen sometimes.

I do understand the Offline / Onlne slider button. I did not make use of it in on this file. I did not request a New Generation at a previous moment.

I ran the procedure on the file while it was shared. During the procedure, while shared, the database recceived the Info(“ServerRecordID”) from the server for the currently selected records, wrote that into the top left cell, then altered the file such that it was both Offline, and then not shared.

While all of that may seem preposturous, I know that because of the debug that I did when I saw the odd data. My analalysis was thus…

From the odd data, I repeated the specific selection of the records that were specified by that data. I selected each and every record whose serverrecordID matched that data. Those records had just previously been selected for a uniqued email merge. They were not random records. The first record had successfully been used and its First name field was successfully used in the email merge and an email sent, and the following records First name fields as well. Then later in the process, a error occurred. I believe it was an AppleScript timeout error. The procedure stopped due to this error. I then went to rerun the procedure and selected those records again. Thus the first record was in my view. I saw the ‘random data’ and knew that something was amiss. Unless the local copy already has within each record its ‘ServerRecordID’ in a non displayed manner, then the local copy had to have gotten that from the server while being a shared db, and while connected to the server. Otherwise, a non shared database, could not have obtained that data to then write it into that top left cell. Again, this is not the first time that this has happened to me.

I’ve attached another odd ‘alteration’ of the first record due to server error. After running a procedure, I then found the first record had data written to the first several fields that belonged to fields to the right. In the screenshot attached, data from Address, City Name, and Zip Code, Active and Inactive fields each have been written to the first record. That data belongs to another record that was not currently selected. That address info comes from fields to the right. As I am able to determine specifically which record that data came from, I was then able to see that it was a record that I do not use in my email merge. I do not even use that data in the email merge. I only use the first name field. Note that the Phone field has a Input Pattern of (__) - so the data in that field did not get typed into that field. But yes, it can get pasted. But as I never reference those fields in any of my email merge procedure, that could not have been caused by inadvertently written procedure. I enter that data via the keyboard and then never make use of of again, yet there is was, 5 or more fields, written in proper sequence, from an unselected record, to the top record by the server. Further analysis of the written data shows that the dates entered into the Active & Inactive fields must be some date is specific to some number as those are date fields but there are no records that have that date in that field. That date must be a permutation of another number converted into a date.

The records in that selection were procedurally selected based on the content of the ‘Sail Title’ field. All of the displayed records are correct with the exception that the first record, which now has bad data in it, should have had a person’s First & Last name in those fields along with other data but the data that was inserted into those fields is from a different record in the data set that I have not made use of as that person is inactive and only exists in the database for historical reasons.

I repeat – maybe I need to know more about this procedure. By “more about” I mean source code.

Not much to it. I first do a custom selection, then run one of multiple procedures that differ in the Body but are essentially identical to this…

Local LCrewBody

AlertYesNo "Is this going to be a unique, selected Send?"
If Info("DialogTrigger") = "Yes"
    NOP
Else
Select «Sail Title» Contains "Crew" Or «Last» = "Ibsen"
Field Last SortUp
EndIf

AlertYesNo "Did you remember to set the proper account in Apple Mail?"
If Info("DialogTrigger") contains "No" Stop EndIf

FirstRecord

Loop


LCrewBody = "G'day " + «First» + ", 


Reservation Start: 

Several paragraphs of text here

Oasis Sailing Club Reservation Manager"

SendEmail {FromName="OSC Reservation Manager" 
//     From="OSCReservastionManager@OasisSailingClub.org" 
    To=}+«Email»+{
    subject="⛵️ Oasis Sail Crew - Pre-Registration Email : What you need to know"
    smtp="smtpout.secureserver.net"
    ChannelResult="Yes"
    },LCrewBody

// Message ChannelResult

Email_Log = DatePattern(Today(),"MM/DD/YYYY") + " Pre-Reservation Email" + ¶ + Email_Log

DownRecord

Until Info("Stopped")

Sounds like there may be a weird problem with the Apple Mail channel IF there is a Mail timeout error. I looked at the source code for this channel and didn’t see any immediate problem. This may be a very difficult problem to track down.

I have a couple of suggestions in the meantime. One would be to not use Apple Mail. It’s really not a very good production system, look how you have to prompt to check if they have the correct account set up. It should be pretty simple to set up the Python channel to use smtpout.secureserver.net.

Another suggestion would be to set up a separate, single user database just for sending the email. It wouldn’t even need to have any data in it (and that way, if random data got written to it, it wouldn’t matter). Just collect the email addresses you want to send to into an array, then go to the other database and loop through the array to send the email addresses. Actually in your case I guess the array would need to include the first name and the email address, perhaps tab separated.

If I walk away from this bug, it may not get figured out. In my mind, leaving it as it is (with a slight mod) might give me more information as to when exactly it happens, such that it can be fixed for the next guy.

I am going to procedurally just add a junk first line to the top of the list of records so that it can be a dumping ground for any data that does get written. That way my stuff stays solid yet I can see when something go awry. As it has always been the first record, I will be safe with minimal change.

After the FirstRecord command, I’ll add an InsertRecord and set the values of each of the fields to be a sequential number, ie. First field =1, Second Field = 2, etc so that if data slides it might tell me something. I’ll examine that record at the end and then toss it if all looks good. Small price to pay, minimal code to write, and maybe I can figure out what or when it is happening. Of course, the email field will need to be a junk email address, lest I give Apple Mail something to complain about.

It is interesting that Apple Mail does have a timeout problem (probably actually AppleScript) as that has started happening only recently. Usually around 70-80 records. It didn’t use to happen but is pretty regular lately.

Update on the seemingly random writing of Info(“ServerRecordID”) values for every currently selected record into a single cell…

This just happened to me again and there was no crash involved. I just now ran 2 procedures and the data (1224,1225,1226,1227,1228,1229,1230) was then noticed in a data cell.

Nothing unusual occurred otherwise. The procedures did what they were supposed to do.

The active field when that procedure ran was ‘Crew4’ of a particular record. That record was not the top most record. The data was written into field Crew4 which was the 12th field from the left. The location has no significance other than it was not the first record, and not the first field which apparently was typically the case in prior situations.

Following usage of SendEmail command which interacts with AppleEvents, AppleScript, and Apple Mail, my next procedure line is supposed to write to a field some text for the purpose of logging that email action.

Sometimes, instead of writing log info into the desired field, text from an unknown source gets written to mulitiple fields of the current record which most often causes Panorama to crash, perhaps due to an attempt to force text into a numeric field.

Note that first line which has data from somewhere in this file but I have not found the actual source within the file. The fields First, Last, Cell_Phone, UserName are all obviously corrupted data. That first line was the current record when the procedure was running. It had good data in it prior to the running of the procedure.

In the field titled ‘First’ is the words ‘Res Sys a…’. The words ‘Res Sys’ are words that I would have written as that is my shortening of the ‘Reservation System’ for which this whole process is about. But those words are not in the Procedure that is running and I’ve yet to find where in the file those came from.

The email that that first record should have accomplished has bad data in it such that that email is not able to be sent. So the process is corrupted during the SendEmail action. But interestingly, if I comment out the line following the SendEmail, than that appears to remove the problem entirely and the procedure then appears to not cause problems.

And all of this is a sometimes thing. Sometimes it works just fine. Argh.

Very oddly interesting. Any ideas or thoughts for further debugging of this?

I would try putting a Wait 0 line after the SendEmail line to see if that helps. Just a guess.