Random data cells are empty after importing a Panorama 6 file

I’ve opened several Panorama 6 databases into Panorama X. Most have opened correctly with no problem. However I have one very basic database with only three Text Fields that is giving me a problem. Random data cells are displaying as empty after the importing the file. When I click on the empty cell to open for editing, the data is visible and editable but when the editing is done the data is again invisible. When I try to do a search on any of the invisible data, the search can’t find the data.

Is it possible that you could email me the file? Probably the best bet would be to send the Panorama 6 file, then I can see what it is supposed to be like and convert it myself.

Jim

I am about to send a lengthy comment on this problem (or one very similar) - I’m willing to bet that Rico’s problem is with data cells that overlay a rectangle.

michael

I guess I was assuming he was talking about the data sheet.

This may be the same as Rico’s problem.

If I select a field in the data sheet, then open a form containing a data cell for every field, no data is shown in the selected field. When I click in the data cell, the value appears:

If some or all of the data cells on the form overlay a rectangle, an additional problem emerges - whenever I click on a data cell which overlaid a rectangle in Pan6, the value disappears. Clicking on another accompanied cell makes the new value vanish and the previous one reappears.

This second problem does not occur with data cells that did not overlay a rectangle in Pan6. Adding a rectangle in Pan X does not cause the same behaviour.

michael

Yes, I am talking about the Data Sheet. See attached screen shots.


Rico sent me his Panorama 6 file and it worked perfectly for me. So we’ve got a bit of a mystery. I’ve sent him the Panorama X file I created so we’ll see whether that works for him or if the problem persists.

I opened the Panorama 6 file that Jim returned to me but unfortunately it’s exhibiting the same behavior in Panorama X… empty cells where data should be.

Here is the database on my computer. As can be seen, all of the data is visible.

So now we have to start thinking about what could be different on your system. What OS version are you using? Do you have Panorama X 0.1.025 (2012)?

Since you’ve already made the data public, I’m going to post the database itself, perhaps some others on the board will try it and see whether the data is visible for them or not.

https://dl.dropboxusercontent.com/u/716345/Support/Guitar%20Repertoire.pan.zip

I’m showing the same results as Rico with invisible data after opening in Panorama X. My system is 10.11.6 and my Panorama version is 0.1.025 (2012).

Same results as Jim, using 10.9.5. Same results as Rico and Gary, using 10.11.5.

If I run a formulafill on each field either in the Pan 6 version before conversion or on the converted Pan X file it makes everything visible. This is the formulafill formula I used:

formulafill stripchar(«»,"!z ")

I’m using latest version of Panorama X, 0.1.025 and OS X El Capitan, 10.11.6

Same as Rico & Gary 10.11.6

Robert Ameeti
(949) 422-6866

Gary has uncovered the vital clue.

Some of the data in the original database is encoded in UTF-16, other data is encoded in ASCII. Where did this data come from, Rico? Clearly it wasn’t typed into Panorama 6, it must have been imported, or perhaps copy/paste (though I think that would automatically convert to ASCII). I’m surprised it worked at all in Panorama 6, and it definitely doesn’t completely work. For example, opening the original database in Panorama 6 and searching for “would” returns zero results, even though “Would” is the very first song. This is because Panorama 6 doesn’t understand UTF-16.

With the data you have, Gary’s formula will convert the UTF-16 data to ASCII (8 bit), while leaving the data that was already ASCII alone. (This formula will not in general work to convert UTF-16 to ASCII, but since the data only contains basic letters and punctuation, it will work in this case.) I think it might be best to do this in Panorama 6, then re-import into Panorama X.

So, the moral of the story is that if a database contains corrupted data, bringing it into Panorama X won’t un-corrupt it!

Could somebody please test my data cell issue ( see above) on an older OS to see if it also is 10.11.6 related?

m

FWIW, I like Rico am looking forward to UTF-16 (its those Czech’s fault) but I’ll settle for UTF-8.

Has the the methodology for searches and such been resolved yet?

i.e. I’d like to be able to enter a search for Soren and have all of Soren and Sören and Søren in the results.

Robert Ameeti
(949) 422-6866

Copy/Paste does not automatically convert it to ASCII. It does act odd though in that it takes 2 presses of the backspace to delete each character.

Robert Ameeti
(949) 422-6866

Well, then you are in luck. Panorama X automatically imports from UTF–8 (the default for OS X applications), Mac OS Roman (the default for OS 9 applications), or UTF–16. You can also explicitly convert from a number of other encodings using the binarytotext( function.

Bringing in a database from Panorama 6 is not importing, though. A Panorama 6 database contains ASCII text (more specifically Mac OS Roman encoding). If for some reason there is some other type of text, that is an error and neither Panorama 6 nor Panorama X will work correctly.

Copy/Paste does not automatically convert it to ASCII

I was thinking the operating system might do that when using a Carbon application like Panorama 6, but I’m not at all sure of that. So Rico may have copied from some UTF-16 data source and pasted into Panorama 6, I’m not sure.

It does act odd though in that it takes 2 presses of the backspace to delete each character

Not odd at all. UTF-16 contains two bytes per character. In this case, one byte is all zeros (which you can’t see, so it is invisible), the other is the same as the ASCII character. So though the text is actually 2 bytes per character, Panorama thinks it is 1 byte per character. So it takes two backspaces to delete the 2 bytes.

I’m thinking that this whole issue is the same one I raised in Bitbucket issue #498. It’s too late at night for me to fool with it now, but I’ll explore it in the next few days and report back.