Fragile file uploads to Panorama 6 Enterprise Server


#1

Another great big wish for Panorama X Enterprise is a more stable way to upload files to the server. The file upload method used by Pan 6 can be a source of high anxiety and stress to say nothing of the waste of time. I am not a computer guru, so my observations may be without a sound basis or correct understanding. In any event, the Panorama 6 upload can fail at anytime during the upload process particularly if the upload is being done over WIFI or LTE, although the problem can occur over Ethernet. When the failure occurs, Panorama on the client machine performing the upload freezes requiring a Force Quit. (I have learned to disconnect from the network before a Force Quit just incase such might cause corrupt data on the server.) I usually have problems uploading larger files. Some of my files are between 10,000 and 60,000 records. It is highly unusual to be able to upload a large file without a problem, thus there is great reluctance to make changes to those files. I have experienced upload failures since I first purchased and installed Panorama 6 in 2008. I have had various machines and operating systems, and all have suffered mid-upload failures from time to time. I would like to see the process improved on Pan X. Perhaps the file can be quickly uploaded to the server where it can be processed there instead of processing it bit by bit over the internet. My file with 60,000 records is only 16 megabytes in size, but it takes more than 10 minutes to upload to the server. My simple brain wonders why it could not be sent to the server in a quick flash where it could be processed without a long upload which might be vulnerable to failure. By the way, in all of my years of using Pan 6 Enterprise I have never experience a download failure when files are transferred from the server to a client. Also, despite all of Pan 6’s weaknesses, it has been a wonderful workhorse for my business and has helped me make money$$$, so I appreciate all of the work that Jim and his team does.


#2

I’ve built literally dozens of Enterprise systems for sharing and/or web serving. Panorama Enterprise programming is what I do for a living. Along the way I’ve certainly encountered the issues you’re describing but they’re often avoidable. There are some routines that can minimize the likelihood or the degree of trouble if there is an issue.

Before updating, make a backup of the Public Databases and a set of the client files. That makes recovery very quick.

Since you have a seemingly high chance of issues, when replacing a shared file, use the Sharing Wizards to take it offline and then to remove it from the server. If you can reboot Enterprise, that’s good too. The result is that you’re uploading a new shared file versus replacing one. That definitely reduces the rate of incomplete uploads.

That said, one of my clients has some huge shared databases that we reliably update over the internet with only rare issues. (The backups are useful for the times something goes wrong) I use New Generation in the Database Sharing Options to load new versions of the databases and it works very reliably.

Early on we had problems like you’re describing but found two causes. The biggest issue was the RAM on the server. It may have just been cheap RAM, or it may have been damaged by heat or power surges; who knows? Replacing it caused the rate of corrupted files to plummet. Later, a better router was installed so that the transfer rate was increased as well as the “cleanliness” of the signal. The result is that corruption issues are now rare with this set of databases.

Regarding your other post on corruption, Force Quitting on a shared system should be discouraged. Too often impatient users resort to it and it does seem to be a common denominator when files corrupt. RAM on client computers should also be evaluated. Look at the logs and see if a particular user shows up when there are issues or the logs show a recovery process was run.

Beware electro-magnetic fields such as a refrigerator, air conditioner or a machine shop in the vicintiy of server, router of client computers. Older ones especially can contaminate the quality of a wireless signal, or of a wired network if the cables run near the motors.


#3

Panaca, you are not crazy nor alone. Your comments were very much the life that I have lived with Enterprise. When it works, it is wonderful. When it is being temperamental, (which could be due to less than perfect internet connectivity), life can be a real pain. When we have to upload a large (120 MB) file with no burp in the process, things can be beyond unfun. TCP/IP was designed to allow for intermittent burps without being an issue but that has not fallen through to the update process of files. Files should be fully uploaded using standard proven FTP process to the server (which will allow for burps) and then any processing should happen separately from the live data, then as quick as a replacement can happen, the new data should replace the old data through a rename.

Yes, doing backups of everything could make life easier if something goes wrong. But a standard update should not be temperamental and should not be reliant on always having a backup. That is not part of the process of a standard update. Replacing with new versions of the database brings extra, unnecessary steps of then having to move that file to the users, all of which is supposed to be part of the magic.

I’ve gone through multiple servers, multiple exchanges of RAM, etc, with no change.

Fortunately Enterprise has been a version 1.0 product and much has been learned about how it can be done differently. When I’ve made an error and found that my newly 30 minute upload is not going to work, and have had to repeat the process while the office staff all sit on their hands, the next version will not have this issue. I’ll be able to upload a single procedure or form and all will be fine. The entire upload process will be changed that will allow for lousy upload conditions and the to be expected bugs that we all naturally make. While yes, we should still always do a backup cuz that would make it easier in the case of something going wrong, something won’t go wrong so very often that you’re life will be hell without those extra steps and delays for the customer. I’ve had to do my uploads starting at 6am in the hopes that I could be finished before staff gets in at 8 but alas, sometimes they would be waiting for me to have a clean, working upload at 9am.

Our next life with Enterprise will be a better life. I am sure of that.


#4

Thanks, James, for your helpful hints. We are hoping for a more simplified world with Pan X Enterprise. Hope that does not put you out of a job. :slight_smile:


#5

It is good to know that others have experienced similar difficulties and have survived. Incidentally, the LDS church uses a similar database scheme as Panorama Enterprise for it membership records. This program has been in operation for nearly three years with multiple updates. At any given time there can be millions of simultaneous users (clients). Most clients are Android or OIS devices with a resident application on the device, but the same database information can be accessed from desktops or laptops via web browser. Changes to the database can be made from each client as well as searching, sorting and other more complex queries. However, no client receives database changes until there is a manual sync initiated on the client device. It works extremely well. Each device has data from the last sync whether or not the device is connected to the internet.


#6

I’m sure that Pan Enterprise X is going to offer us a lot improvements and I look forward to it - with no fears at all about a loss of work.

Coincidentally, today I was uploading some shared files and Enterprise kept crashing even though I followed all of the steps I outlined yesterday. I ran TechTool on the server and whad’ya know, it claims that there are RAM issues.