Web Serving Observations and Issues

Very helpful is Program>Upload Code to Server. It sure beats using the mouse to get to the New Generation panel with every coding tweak. Any chance of adding a key command to it? Command-^ for instance.

Submitting a form that lacks a method results is an error: Error in CGI code. On a reload after that error, the server crashes.

After a crash of Pan Server I have to restart it via Pan X Preferences. Subsequently, a browser hit to the cgi file usually resulted in an error that the file was not open. It is set to Open as Needed

On hits that work, whether they’re non-forms or forms with a method included, the log shows the cgiExtraParameters and, if appropriate, the cgiParameters. On form submissions lacking a method, nothing is logged. (Yes, forms should always include a method)

From the logs…
Simple hit to the procedure:
02/11/2021 10:05:11 am CGI-POST: WebTest~FormCrunch System=""
Form submission with a method included:
02/11/2021 10:05:22 am CGI-POST: WebTest~FormCrunch Opinion=Y&Red=Red&OS=OS11
Form submission lacking a method and leading to the errors and crashes:
02/11/2021 10:06:10 am CGI-POST: Opinion=Y&Red=Red&OS=OS11

What is “System”?

Server Logging shows my only choice is from a week ago. Nothing for yesterday or today. Reveal in Finder shows those logs do exist. The machine was off overnight, so at least yesterday would be from a different session.

It appears that cgiParameters and cgiExtraParameters are not implemented. What should be their values does show in the log, but trying to access those two variables by name gets errors. They are included in the list obtained via: cgiHTML = info(“globalvariables”)

Some cgi variables do show values: cgiDatabase, cgiContentType, cgiServerDomain, cgiPostParameters (not sure of what that showed though).

At times it feels like Server and Logging belong in Server Administration. I’m jumping from one to another frequently but the crashes certainly increase that need.

Gosh, this is a rather extensive catch all post.

Probably not going to get a keyboard shortcut, but macOS allows you to do that yourself. But did you notice that you can add this command to the tool bar?

So I’m thinking that should probably result in an error but not a crash, is that what you are thinking?

From the information provided, I have no idea.

Shows this where? As far as I remember, there is nothing in Panorama that will show you server logs – you have to go to the Finder to see them.

The cgiParameters variable was used in PanSTAR over 20 years ago. Apparently it was still used internally in Panorama 6 server, but I just checked the documentation and there was no mention of the cgiParameters variable there. Ok, now I checked the old source code – in PanSTAR cgiParameters was used for form items (post parameters). Starting around 2005-2006 this was replaced by the various webformitem statements and functions, and the cgiParameters variable was no longer documented for that purpose. However, internally Panorama still used this variable so I guess prehistoric code that referenced it would still work. Panorama X server does not use this variable internally. You need to use the documented webformitem statements and functions to access this information. (I suppose you could create a subroutine that re-assembled cgiParameters from webformitems, not the approach that I would take but it could be done.)

The cgiExtraParameters variable, on the other hand, is documented, and is supported in Panorama X. I don’t know why you would get an error in trying to access this, it is definitely assigned a value every time a server request is processed.

As best as I can tell there is NO mention of cgiParameters anywhere in the Panorama X source code. If this variable exists, it has probably been defined in your code.

Server Administration is a client tool, it runs on the client. It is not for managing the server itself.

Yes, the crash is obviously wrong. I’m not sure how a lack of method should be handled, but will consider it reported.

In the Server>Logging panel, my choices for logs to view are not up to date. If I choose Reveal… the folder opens and more current logs the were listed are there.

Re: cgiParameters, I don’t recall its history but at some point it replaced the older cgiPostArgs. If it is gone, I will miss it and will need to do a re-assembly. I have many uses for the whole post, including saving it as a text file archive. WebFormItem and others must be referring to something that contains the full post, so…?

Actually, in Panorama X they are not.

In PanSTAR and up to Panorama 6 the raw web request was passed to Panorama, and Panorama code was used to parse the request into individual items (the URL, post parameters, etc.). However this isn’t really an ideal task for Panorama code, even more so now that Unicode is supported. So in Panorama X, all of that parsing is actually done in Objective-C code inside PanoramaX.cgi. The CGI parses all of the information into a dictionary that is passed to Panorama X Server, all ready to go. This is much faster, and also ensures that the encoding issues are handled correctly.

Actually, there is also another reason for this – there are possibly two web servers involved, the built in GCDWebServer, or Apache/Nginx/Abyss. So this system ensures that everything works the same no matter which of these servers is used.

So here is an undocumented secret – the cgiPostParameters variable you discovered does contain all of the parameters, but it is not text, but rather a Panorama dictionary. It contains all of the form items, but as dictionary items rather than embedded into a text string. All the necessary parsing and encoding has already been taken care of. This is where the webformitem functions get the information. If you enable logging of the post parameters, that is done by using the dumpdictionaryquoted( function.

Remember, the entire previous paragraph is an undocumented secret. So I’m not going to promise that it will never change. The proper way to access these items is using the webformitem functions.

As it turns out, Panorama X has a function that takes a dictionary and turns it into encoded http post data. So I think there is a quick hack you could use to get your code working again. Just put this line at the top of any procedure that needs the cgiParameters variable.

letglobal cgiParameters = encodehttppostdata(cgiPostParameters)

Now this is kind of a wasteful solution, performance wise. But I suppose if you are saving it into a text file you won’t care if it takes a couple of extra milliseconds to parse, unparse, and then reparse this data.

I forgot those popup menus were even there! I guess there must be a bug, I’ve logged it.

For what it’s worth, I never use those logs for debugging any more. It’s so much nicer and more useful to use the new instrumentation feature. Especially since then I can enable the output for only the code I am working on at the moment. The new instrumentation feature is way better than sliced bread for working on server code.

:+1:

I have no doubt about that, but the logs have many other uses. Yesterday I used them to simply look back to see what was being received by PanX Server. The log updated dynamically so I could watch every hit as it came in.

Where I’ve used logs the most in the past was in tracking issues that have already occurred on a shared system in use by several people. Seeing who was on and what they were accessing or posting helped solve numerous mysteries, such as a server crash every time a specific user opened a particular file. On web servers you can spot robotic or malicious hits. Or see that someone who complains that the system doesn’t work used every imaginable variation on their password except for the right one.

1 Like

Could you tell me more about this? When you say lacks a method, does that mean that you specified a method in the action tag but that procedure did not exist? Or was nothing specified in the action tag? What was the HTML that caused this?

When you say “reload” after the error, does that mean you did a reload on the browser? What did that reload? Did you have a static file that had the HTML form? Or was it dynamically generated by another procedure, so that the reload did another server request?

In an HTML form with the form action:

<form action="http://rnd.local/cgi-bin/panoramax.cgi?WebTest~FormCrunch" method="post">

submitted to a simple procedure:

cgiHTML = “Hit”
Rtn

It returns “Hit”

If my form incorrectly lacks a method:

<form action="http://rnd.local/cgi-bin/panoramax.cgi?WebTest~FormCrunch">

Or if the method is set to “get”

<form action="http://rnd.local/cgi-bin/panoramax.cgi?WebTest~FormCrunch" method="get">

I get a response:

Error in CGI code

If I refresh the browser window, which essentially resubmits the form, PanX Server crashes. Sometimes it requires a second refresh for the crash.

The browser display on a crash is probably unique to Abyss as the server app:

CGI tunnel request error:The network connection was lost.

Restoring the POST method makes it respond properly again.

And here’s where I tend to turn to the Server Logging in PanoramaX Preferences - where the popup menu for Daily Web Activity is still stuck on showing only “Last Saturday” in spite of use over several days since.

In today’s log it has recorded:

02/13/2021 9:10:09 am Start Panorama X Server
02/13/2021 9:11:09 am CGI-POST: WebTest~FormCrunch System=""
02/13/2021 9:14:15 am CGI-POST: WebTest~FormCrunch System=""
02/13/2021 9:15:11 am CGI-POST: System=
02/13/2021 9:15:42 am CGI-POST: System=
02/13/2021 9:15:44 am CGI-POST: System=
02/13/2021 9:18:51 am Start Panorama X Server
02/13/2021 9:24:02 am CGI-POST: System=
02/13/2021 9:24:21 am CGI-POST: System=
02/13/2021 9:26:15 am Start Panorama X Server
02/13/2021 9:26:50 am CGI-POST: WebTest~FormCrunch System=""
02/13/2021 9:31:20 am CGI-POST: WebTest~FormCrunch System=""

The hits with POST have the parameters recorded. The others do not. Those with post include the “System” that I asked about in my first post of this thread.

In the separate Errors log, it recorded:

02/13/2021 9:15:11 am Error in System=~: Database does not exist (missing info.cfg)
02/13/2021 9:15:42 am Error in System=~: Database does not exist (missing info.cfg)
02/13/2021 9:24:02 am Error in System=~: Database does not exist (missing info.cfg)

And therein is also an example of why I found the logs of Enterprise 6 to be a desirable configuration. The separate Errors log is great for checking and discovering if there are any errors at all, and if so, where and when they occurred. Having them also entered in the full log, in red no less, made it very easy to see where they fit into the overall flow and activity.

If my log for today represented a busy server with thousands of hits today, it would take a lot of back and forth to determine where the errors fit in. Or I can always import both logs into a database and sort them, but gosh, the convenience of that past format.

At least the crashing has provided an opportunity to test my updated ServerMonitor which has promptly relaunched it each time. :slight_smile:

FWIW, one way that I use the full post data, formerly referred to as cgiParameters, is on a site dealing with medical data.

We have literally a few hundred medical forms for all sorts of scenarios for pre-certification submission to insurance companies. Some forms, such as regarding treatment of varicose veins are immense and have many fields with a high percentage that don’t come into play most of the time. All forms begin empty aside from patient, physician and other info that has already been established. Much of the time the forms need to be retrieved for additional info

Creating databases with fields to match all the forms and fields was proving to be a bit on the insane side, especially since some forms, or their variants, might be used frequently as others only under blue moons.

Each form also needs to be converted to a PDF at one point for the actual submission. Building Panorama forms to fit each one was also proving to be an enormous task, especially in the event that even a minor addition to the fields in the HTML form would require modification of the Panorama forms and replacement of the online databases involved.

Instead, I have key elements of common data being saved to a single database to create an index of submitted data. The submitted data is saved as a text file using the POST data as it is supplied in the cgiParameters.

If the form is retrieved, Panorama is more than capable of parsing the text to build a javascript that reloads the form data in the HTML page. ie: document.getElementById(‘city’).value =“Tulsa”;

If a PDF is to be generated, here too, Panorama excels in its ability to parse the saved text using, for instance, TagData(. Hidden tags that have been embedded in the HTML form are utilized to break down and identify the sections. It can determine headings for sections and what pages to include or exclude from a set of Panorama forms that have become somewhat generic. Most of the forms then use variable contents.

As long as the established naming convention is used, new HTML forms can be added readily or existing forms altered and it all just runs without new fields or forms in the online Panorama database. It’s all in the coding.

The text below is the POST data from the simplest of the forms. Some involve more than a hundred fields that are inserted between the !rq and !f tags. (Yes, we can hope no patient ever has all of these issues)

!h=head !Practice=Hesperia Clinic !Logo=813527153.jpg !MName=Blair, Tony S Jr !DOB=02/22/1984 !InsCo=Anthem !GroupNmbr=gr12-9oup !PlanNmbr=545566 !MembrNmbr=652625 posID=6306825821 accountid=813527153 usrid=0975184859230735 reportID=597185084Q Report=Procedure(s) LastName=Blair Nickname=Parse Trial 1 RefNum=321-Go Priority= !d=Diagnosis dgns1=A1812 - Tuberculosis of bladder dgns2=A021 - Salmonella sepsis dgns3=A3211 - Listerial meningitis dgns4= dgns5= !rq=request rqp1=22843 - Spinal Procedures Spinal Fusion rqp2=37765 - Cosmetic Surgery Venous ligation rqp3= rqp4= rqp5= rqp6= rqp7= rqp8= rqp9= rqp10= status=Scheduled explain=No Explanation prcddate=03/10/2021 comments=Comments go Here !f=footer physician=Cook, James tin1=12-2222222 npi1=2136666666 physID=0975184859230735 physician2=Phys 2 tin2=56-5645646 npi2=6216461262 fclty= Facility=Bigger Hospital netstatus=In_Network ein=46-4665656 address=123 Broad Ave\n3rd floor, Room 304 fnpi=6546564564 city=Tulsa state=OK zip=80211 phone=(303) 256-3265 Contact=Helmut Melick cphone=(545) 874-5698 fax=(212) 546-9875 pPayer=Aetna pLogo=…/graphics/logos/aetna.png pPhone=(212) 455-6547 pFax=(212) 685-9568 pEmail=precert@aetna.web !x=end

In PanX, I am able to create a global named cgiParameters that I will load with the entire post - and must do so since I’ve used it so widely in Enterprise 6. I’m even doing more this very day in Enterprise 6 since we’re still building with 6 on a nearly daily basis.

What I have lost is the ability to load saved text to cgiParameters, then use WebFormItem and similar statements that no longer recognize it. In PanX this comes up empty.

Global cgiParameters
cgiParameters = {name=Joe    size=big    gender=M}
LetLocal lvName = ""
WebFormItem "name",lvName
Message lvName

As noted, this is a FWIW, not a grievance. cgiParameters has been very useful and I have the means to continue using it in most cases. In other situations there’s no doubt of finding a way around it.

Jim Rea, Will WebFormItem work in the future? A lot of rewriting would be required if it doesn’t. In fact, I think every web solution I’ve created uses it.

WebFormItem does work on post data. Previously it could be applied to pick apart cgiParameters, but no longer.

Picking apart cgiParameters is exactly what I need it to do. After reading the whole thread, it looks like I can use dictionary statements to retrieve the info from the url’s cgiparameters and extra parameters.