Bug in Panorama LOOKUP

I try hard not to conclude that there are bugs in Panorama 6.0.0 but this seems inescapable. Also hard to reproduce.

I have a situation where all records are selected. A ton of stuff has happened before, and I have no way of determining what state Panorama is in. If I say:

SELECTALL
FIELD «T1»
FILL "asdf"
FORMULAFILL LOOKUP(mSGoalFile,"Date Time UT Quantized",«Date Time UT Quantized»,"Goal","",0)
IF «T1» = "asdf"
    debug
endif

I will never get to the debug statement. The field «T1» always has the correct values from «Goal» and the others are blank.

If, on the other hand, I leave out the initial SELECTALL, I will OFTEN get to the debug statement. All of «T1» will be “asdf”. Further, if I say,

FORMULAFILL LOOKUP(mSGoalFile,“Date Time UT Quantized”,«Date Time UT Quantized»,“Goal”,“z”,0)

I will OFTEN find that «T1» is still “asdf” — no “z” — it’s as if the LOOKUP function was never called.

I’m wondering if somehow the number of records it thinks are selected is different from those it is displaying. INFO(“selected”) gives the correct number of records.

Also, it goes without saying, it performs differently when single stepping than when running.

I don’t expect anybody to have any good answers to this, but if it Reminds You Of A Story — it’s a data point…

I think it is more likely that the FormulaFill was disabled. If you do a select that fails to find any matches, and don’t use info(“empty”) to test to see if that was the case, then some commands like FormulaFill will be disabled to guard against massive data loss. When you do a SelectAll, that is a select that is always successful.

I do believe you’re right. I added these statements:

mSInfoSelected = INFO(“selected”)
mSInfoEmpty = INFO(“empty”)
displaydata "mSInfoEmpty = " + STR(mSInfoEmpty)

  • " mSInfoSelected = " + STR(mSInfoSelected)

and I see this result:

mSInfoEmpty = -1 mSInfoSelected = 96

This leads me to ask, how can the selection be empty when there are 96 records selected? and visible? and, shouldn’t FORMULAFILL at least offer the option of putting up some kind of alert when it decides not to run?

I think I have had this problem before, too. After making a selection, I always test to see if the condition is met before proceeding to operate on the selection; but I need to be more careful about the condition of no items selected in subsequent statements.

oh, you know what’s wacky:

The selection I made was in a DIFFERENT FILE. Inspect this code:

SELECTALL
OPENFILE mSGoalFile
SELECT fieldstyle(«Date Time») = "red"
IF fieldstyle(«Date Time») = "red"
    ARRAYSELECTEDBUILD mARedGoals, ¶, mSGoalFile, «Goal»
    SELECTALL
ELSE
    mARedGoals = ""
ENDIF
WINDOW mSThisFile

mSInfoSelected = INFO(“selected”)
mSInfoEmpty = INFO(“empty”)
displaydata "mSInfoEmpty = " + STR(mSInfoEmpty) + " mSInfoSelected = " + STR(mSInfoSelected)

So you see — I made the selection in one file — it found no “red” records and so set INFO(“empty”) to TRUE — then back to the file the procedure was called from, “mSThisFile” — which did have a selection — but “empty” was set true. Ideally the condition of “empty” selection should be on a file by file basis, don’t you think?

I guess I must think that, because I just checked and that is exactly what Panorama X does. In Panorama 6 the internal variable that keeps track of failed selections is a global variable that is shared across all databases, but in Panorama X each database has its own.

@interface Database : NSObject {
    ...
    ...
    BOOL selectionEmpty;

There are probably quite a few Panorama 6 bugs like this that are fixed in Panorama X. Panorama X has very few internal global variables compared to Panorama 6.

This leads me to ask, how can the selection be empty when there are 96 records selected? and visible? and, shouldn’t FORMULAFILL at least offer the option of putting up some kind of alert when it decides not to run?

This is intentional and documented behavior, see page 558 of the Formulas & Programming book, under the heading *Handling Empty Selections".

The part about switching databases after a selection is a bug, though, though it’s never been reported before. I’m glad it’s fixed in Panorama X.

This is intentional and documented behavior, see page 558 of the Formulas & Programming book, under the heading *Handling Empty Selections".

Can you speak to why this is intentional? We are still using Pan6, but I was hoping that this behavior would be one of the changes in PanX. Coming from an SQL background, running a query that produces no results, and seeing a bunch of records, is unexpected behavior, and my code is littered with if info(“empty”)s.

You might want to check out the checkemptyselection command which is new to Pan X.

You might want to check out the checkemptyselection command which is new to Pan X.

We’re not using PanX (and won’t be able to switch any time soon). But I looked at the documentation for checkemptyselection and it just looks like a slight streamlining of the “if info(“empty”)” code that I’ve been using in Pan6.

I guess my question is less about how to work around this behavior, and more about whether this behavior is intentional. Is the fact that Panorama can’t display zero records a difficult technical problem, or is it by design?

Great question. In my world, I would say that there is no particular reason beyond the desire of the designer/programmer that this would work as it does. I too am used to other databases that give you what you asked for instead of the previous selection. Along these same lines is the Panorama world that you always have to have 1 record which inhibits the desire to sometimes delete that last visible record and you have to play tricks to delete it (select others additionally then delete the record you wanted to delete. 'Tis a bit silly.) FWIW, I would agree that if I do a selection and no records fit that query, then no records should be displayed. That would just make sense. But for historical reasons, it is working as it always has worked and personally, I’ve just gotten used to it. I could easily adapt to something that is more ‘sensible’ but things like this don’t typically change as it would rock too many boats or minds. It will be interesting what we hear from the boss.

Robert Ameeti

These days Panorama is a full fledged programming environment, so it’s easy to forget that originally it really had no programming tools at all. So it was designed around the idea that it would be manually operated. In that situation, it originally seemed liked it made no sense to ever have zero visible records.

There is a lot of code in Panorama that assumes that there is a “current record”. If zero visible records was allowed, every single spot in the code that accesses the current record would have to have special code added to deal with the fact that there is NO current record. I would estimate that there are several hundred places in the code that would have to have additional code added to handle this special case. And it’s not like it would be the same few lines of code in each case, rather, it would have to be decided in each case what special thing needs to happen here if there is no current record. Especially since I’m sure that the fact that there was no current record would have to be handled at a low level in the code, but also would probably ripple up to higher levels. For example, what should a form show if there is no current record? Probably every type of object would need some special handling – maybe you’d want a special default image for an Image Display object, a special “no current record” option for Web Browsers, etc. Maybe pop-up buttons, checkboxes and radio buttons etc. need to be dim if there is no current record? Probably to add this now would take several months of work – and I’m sure many of the decisions I made about what should happen when there is no current record would be second guessed here.

I could have made allowing zero records a goal when starting from scratch with Panorama X, but it still would have added quite a bit of additional complexity and delayed the release of Panorama X further, which I’m sure we can all agree is plenty delayed already. And it certainly has never seemed to be something that there was a lot of demand for. Yes, it has come up occasionally, but my sense is that there is not the demand needed to justify the considerable expense. (And I would guess that if Panorama did ever make this change there would be some people that would object loudly to the change!)

Coming from an SQL background, running a query that produces no results

SQL has no user interface at all, it’s just a language, usually run from a command line shell. So the considerations I described above don’t come into play at all. The equivalent would be more like the arraybuild statement in Panorama X with a query formula. If you use that, and there are no matches, the result is an empty array, just like SQL.

Usually, even in Panorama, you don’t have to worry about this – Panorama does the right thing. If you have a procedure that does a selection and then does some follow on operations, Panorama knows not to perform the follow on operations if the selection result was empty. The original poster found a bug where this didn’t work correctly, but this was a pretty unusual situation involving selecting and then switching to another database. And that bug is fixed in Panorama X.

But the questions might actually still be worthy to be asked…

When Panorama X hits the market, will there be new users that will think Panorama is ‘weird’ in the way it works?

What are the true thoughts of the Panorama programmer users? Perhaps some of the bigger guns might speak up?

While I am anxiously awaiting optimization and Enterprise, and I know that existing procedures could need tweaks and perhaps lots of them, it would not be anything but a bit of relaxation that would come over me to see no records displayed when my query results in no records. I have no desire to delay X but I always am willing to look at the bigger picture.

James, Gary, David, Chris, Jeff, Paul?

Robert Ameeti

I’m not looking for ‘votes’ but rather thoughts from any that look to Panorama a ‘their’ database. And certainly any opinions will be that and that only.

Robert Ameeti

Well, as long as it is just an opinion you are soliciting I’ll drop in my 2¢. I agree that a failed selection would seem more logical producing an empty result. On the other hand, I am so used to the way things have operated all along that it is not personally a major concern if we leave it as it is.

Let me put my earlier detailed response another way – if a group of people want to kick in enough funding for 4-6 months of development to get this done, sounds like a great idea. Maybe we can do a Kickstarter :slight_smile: If you’re asking me to fund that much development out of my pocket, I personally don’t think that is a good investment, nor one I can afford at this time.

I think you’re still looking at this and saying “zero records, Jim can knock that off in an afternoon.” That might even be true, but in doing so I would literally be creating hundreds of bugs all in one fell swoop. Every line of code that assumes that there is always a valid current record would now be a bug, and probably many (most?) of them would be “crash” bugs. Tracking down all of those bugs WOULD absolutely take months, and some might pop up even years later. Plus the user interface issues, in forms especially. This is not a “maybe let’s think about this, flip a coin” question. In fact, I’m not sure if 4-6 months would be enough. Also, the end result would be lots of possible changes required in your programs as well. For example, if your database has a Name field, right now you know that you can put Name in a formula and you’ll always get a value. With this proposed change, that assumption would no longer be true. Every single line of code that accesses or modifies a field value would now have to handle a possible error.

And all of this proposed work is for a rare edge case – a selection that doesn’t match. An edge case where your immediate goal will always to be to get out of that situation by making another selection. Panorama’s current solution is notify you of the failed selection, then go ahead and make another selection immediately. Not the selection you wanted, but it avoids having a special “zero records” mode, and the huge amount of work necessary to support that special mode, a special mode that you will never use for any actual work.

All real products have compromises. I’m not arguing that this isn’t a compromise, but I think it is a very reasonable one.

It’s your program, so sure, it’s your call whether the time investment is worth it. But as you know, Panorama is being used differently by every user. In our company, making a selection that returns no results is not what I would call a “rare edge case.” And the safeguards that are built in to keep you from accidentally acting on records after search returns no results–those safeguards don’t come close to covering all of the ways you can put data at risk (speaking from experience).

I think Robert’s point is valid–this behavior is non-intuitive to me coming from working with other types of databases, and I can tell you that it’s also non-intuitive to new employees in our company who are learning to program in Panorama and have never worked with any databases before. So if attracting new users is a goal, it’s something to consider.

At the risk of sounding patronizing, which is not my intent… As a person who works on websites and databases, when one of my coworkers requests new functionality, I make an effort to set aside what I know about how annoying it will be to implement, and consider whether the end result is worthwhile. I think it’s a real challenge, as one person, to design and implement. Impressive that you do both–I’m sure you are aware of the risks of that approach.

Thanks for your response.

I’m sure you don’t mean to say that something that would take a half man-year of work falls under the category of “annoying”. For ProVUE, that is the level of a strategic investment. If in your business you are able to set aside all considerations of cost, that’s great, I’m afraid I do not operate in that sort of environment.

The question was asked as to why Panorama works the way it does in this area, and I gave a detailed explanation, which I thought the question deserved.

is not what I would call a “rare edge case.”

I meant in terms of using the program interactively.

my code is littered with if info(“empty”)s

I don’t think that this proposed change would eliminate the need for any of those. I suppose they could be replaced with info("selected")=0, but other than that, if you need to use if info("empty") now, you would almost certainly still need to in most cases. In my code, I think the most common reason to use info("empty") is so that I can display a message to the user if there is no selected data, as shown in the example on page 558 of the Formulas and Programming book. I would think you would need a similar test if coding using SQL (of course the test wouldn’t be in SQL itself, but in whatever host language you were using - PHP, Ruby, etc.)

If you want to use Panorama X like SQL, you might consider an entirely different approach. You could use arraybuild( to build a found set, and display it using a text list object. Or, as was discussed here a couple of days ago, you could have the list itself build a found set. That way, there is no way to accidentally modify the original data. It would basically be just like implementing a query in SQL.

If you are coding in SQL, the end-user has no independent access to the database. They can only do what your code allows them to do. So hopefully that’s safer, but a lot more work to do. Most users like Panorama because they can do a lot of stuff without any programming, but they can add programming to automate common tasks. But if you wanted to use Panorama like SQL, you could build a database with all independent access shut down. This is especially possible in Panorama X.

[quote=“admin, post:16, topic:990”]
If you want to use Panorama X like SQL
[/quote]…

It’s not hard to suppress direct access to a Pan X database and use objects, such as a matrix, to display precisely what you care to display, including an empty set.

My business has become almost exclusively one of building cgi apps on Panorama, which also inhibits display and direct access to the database. Whether it’s Panorama or SQL on the server, if nothing matches some selection my programming needs to know it and tell the user.

I’ve been long accustomed to using info(“Empty”) on every selection, but I’d want to evaluate selection results one way or another under just about any circumstances I can imagine.

I think the fact that Panorama’s datasheet, at a minimum, is in front of you may be more of a psychological than programming issue.

How the end user sees it is always a concern. At one time I had many hundreds of customers using a system I built on Panorama and there was routinely a point at which just about every one of them panicked over lost records. They could only see one record or 200 records when there were supposed to be some greater number overall. Comprehending unselected and undisplayed records was difficult for them. I can’t imagine how they would have reacted to see none.

As long as you have a visual display, especially one like the datasheet, it’s hard to imagine there’s any universally satisfactory solution.

Fair enough. From my perspective though, it makes a difference when developing new code, because the consequences of something like…

Select MyField = x
Loop
;; do whatever
Until whenever

…vary depend on how empty selections are handled. I may need to add cases to handle empty selections in either scenario, but with the current system, I may accidentally mess with a bunch of records that I didn’t intend to. I’d rather get an error when trying to act on zero records. I realize that there are all sorts of ways that poorly written code can do unintended things. This just seems like a particularly unintuitive behavior.

I don’t expect that you’ll jump to implement every request that comes your way. I’m offering feedback about an aspect of Panorama that I find frustrating, and just as I’m not going to convince you that it’s worth the time and money to address, you’re not going to convince me that it’s not frustrating :slight_smile:.

There are two aspects to this problem.

(neglecting the original bug, which was that empty selection was a global — that problem has been fixed.)

The first is the whole conundrum of what to do about an empty selection. Conservation of Grunge would predict that problem is not going away. The decision was made to spare the user the Horror of the Blank Page — fair enough.

The second is an instance of the well-known UI problem, that the machine has entered into a state and not indicated so to the user. This problem doesn’t happen when you are using Panorama from the front. If I use the find/select dialog in my address book to select «country» = Utopia, I get a nice little alert saying: “No records selected, reverting to previously selected records.”

But if I attempt this selection in a procedure, I get no such indication. The database is now in the state that no records are selected, but on the screen there are records shown.

It would be helpful, to the programmer writing a procedure, that this situation be flagged. A minimal change to make would be to decide, if the procedure window is open (i.e. if somebody is working on writing it), and a selection made whose result is “empty”, the alert comes up as it would from the front. Or the screen turns gray. Or it beeps. Something.

Select MyField = x
Loop
;; do whatever
Until whenevercode

I wouldn’t write the code like that. I would do this:

Select MyField = x
If info(“empty”)
Do something else
Else
Loop
;; do whatever
Until whenevercode

OR

First record
Loop
If MyField = x
;; do whatever
Downrecord
Until whenevercode

I’m on my iPhone so the code isn’t perfect, but you get the idea.