Clairvoyance and Tab key

There is definitely a problem with clairvoyance and using the tab key when entering data. It works the first time using the tab key, but will not on later entries. When manually clicking into the data field to enter data, clairvoyance always works. Is there a fix to this problem?

That may be the same problem I reported earlier.

There is absolutely no relation between tabbing and clairvoyance. The code that implements clairvoyance is quite small, just a couple dozen lines, and it does not interact with tabbing or anything else.

At this point, Clairvoyance in Panorama X has been working for 5 or 6 years and I am quite confident that it works. In the past, whenever someone has submitted a problem it has always come out that Clairvoyance was working properly for the data contained in the field, once you looked at that data close enough.

If you want me to investigate this further, you must send me the database in question, and the exact sequence of keys that will repeat the problem. Most likely it is really working the way it should once the data is examined closely, as with every earlier report of a problem submitted in the last couple of years, but if you think this is not true and you can submit the problem in a way that you think I will be able to repeat it, I will be happy to investigate.

I have checked this over and over with the file I sent you, and it still does not work. I have given up on expecting it to work.

I was addressing this comment about sending a file to Robert, but I need to apologize to Bruce, as he had already followed exactly these steps back in March, and I had forgotten about that. But I went thru my records and found that I had indeed downloaded the file he sent me back in March. I’m not sure how much time I spent on it at that time.

I wasn’t able to exactly duplicate the problem based on Bruce’s steps, but I did see that there is a problem. It’s a very unusual problem but I think I have it diagnosed, but I haven’t come up with a way to fix it yet.

Clairvoyance works by scanning the entire database each time you type a key. If the database is large the scan can take a while, so Panorama pauses the scan periodically to check to see if another key has been pressed. If another key HAS been pressed, it stops the scan completely and starts over with the new text.

The problem is that Panorama is incorrectly thinking that another key has been pressed, so it is stopping the scan prematurely, resulting in no Clairvoyance match. I’ve narrowed this down to a single API call made to Apple’s code to determine whether another key has been pressed. This API call seems to be always telling Panorama that a key has been pressed, no matter what. The code calling this API was originally written back for OS X 10.7, so I’m not sure if the API has changed or what is going on, I’ll need to do some further research on this.

The problem doesn’t show up all the time because Panorama only infrequently performs this keyboard check. So it’s very dependent on the size of the database, the distribution of data, and exactly what has been typed in. It doesn’t appear to have anything to do with using the tab key or have anything to do with how the record was created. The problem will never occur if the item you are typing occurs less than 100 times in the database (Panorama never performs the keyboard check if there are less than 100 matches). So in Bruce’s database I ran into this problem with the word Transportation, which appeared over 1800 times in his database, but not with Tax, which appeared only 33 times.

When this problem does crop up, the only symptom is that there is no match for Clairvoyance. There is no problem with the database itself.

Is it possible for Panorama to scan those fields that use Clairvoyance upon opening the file, rather than upon editing the field? Then it could refer to those scans when necessary, which would make it faster. Also, it would only have to modify those scans when another item is added to the list.

Although I do not know the inner workings, I mentioned once that Clairvoyance could be implemented by building an array of all the entries in a field, sorting them, and then searching the starts of the array elements as one types. If it worked that way, the array could be modified, bringing back the ability to link to a different field, even in another database. That could be very handy.

Essentially you are suggesting that Panorama fields have an index. That could be good for this but very bad for many other Panorama operations.

There’s no major technical obstacle to bringing back that ability, I just haven’t done it yet.


I’m actually working on this Clairvoyance problem right now. I may be making progress, not sure. I’ll get back to that now.

I realize that I left out one step, which is to make the array of entries, sort it, and then de-duplicate it.

It would not really be an index. It does not need to be saved with the file. It is just a list of previously used entries, and only on certain fields.

I believe I have come up with a fix for this issue. Thank you very much to Bruce for supplying a data set that would fairly consistently trigger the problem, though it did turn out that there was a semi-random component to this, making it extra hard to track down. I can no longer duplicate this problem at all with Bruce’s database, where before it was pretty easy to cause the problem. This fix will be in the next release. Unfortunately, it will be a while before the next release is available.

I think this problem could also have caused the TextEditor object to sometimes not trigger the procedure on keyboard events when using the “Most Keys” option. I wasn’t able to actually make this happen that i could see, but based on the problem I found I think it is possible.

No, not really, you are right. But it would have to be updated every time any data value in that field changed, which would really kill performance on operations like formulafill, plus adding a huge amount of complexity internally.

Normally I wouldn’t talk about internal issues in this level of detail, but this afternoon I was stuck at that point and I wanted to let you know I wasn’t just dropping the issue, though I certainly did for nearly 4 months after you sent me the file.

I have just gone through an issue with my friend who hosts my band’s website (www.oaklandmunicipalband.org) where I had found a way of posting the programs that worked fine on all my computers and IOS devices, but did not work for her, which made it hard for me to understand what was going on. She told me that if she was having the problem, other people might, too, so I had to fix it. I checked with another friend, and he had the same problem. So fix it I did. (Apparently my computers will display pdfs as images using the img tag, but others do not, for reasons I do not understand.) But it is frustrating to try to figure out what does not work when it works for you.

I thought there might have been something on my computers that was interfering with Clairvoyance. Tracking that down would be as difficult as tracking down what enables them to display the pdfs.

I just wanted to chime and say I’m glad to hear progress is being made on this problem and thanks to Jim to being tenacious in tracking it down.

I too have had problems with Clairvoyance working consistently to identify categories in a large file (27,000+ entries) and had kind of given up on it. (And I hadn’t submitted the actual file to Jim for analysis since it’s a financial database with lots of personal information and I just couldn’t figure out how to sufficiently scrub the sensitive data out).

Jim’s potential explanation of the problem is the first that makes sense of the symptoms I have been having (Clarvoyance working sometimes, other times not, seemingly randomly)

I’m glad to hear a fix may be forthcoming, though sorry to hear it may take a while. Still I’ve lived with this issue for a while; I can wait a little longer.

Thanks, Jim, for circling back to this issue.

Peter

l hate to chime in with a “me too” post, but I too have been having problem with clairvoyance as others have described. Sometimes it works…sometimes not. Though I haven’t looked closely at it, it’s possible that its not working on fields that have more than 100 instances of the same item…

Jim - Do you still want a copy of my database? It has over 3900 records.

I think I have fixed the problem, but if you want to send it to me I can check it with your data.

For your future reference and anyone else that sends in a file – please include more information in your email beyond just “Panorama Database”. I get a lot of email. However, I eventually figured out what this was and why it was sent to me. Now that I know exactly what to look for I was able to verify some situations where Clairvoyance does not work with the existing copy of Panorama, and that the fix I made a few days ago does indeed resolve the problem.

P.S. Robert’s database is an index of model railroading magazine articles. I’ve been a model railroading fan since I was a kid, though I haven’t had any time to do anything other than read magazine articles for a couple of decades, and occasionally visit someone else’s layout. Definitely more fun for me than most database that get sent in :slight_smile: