Weird Import of Large Pan6 - colored bars appear at end

Panorama X
1

For reasons only a crypto-nerd would accept, I needed to generate all the unique ten digit numbers available from the numbers 0 the 9 - with not repeating numbers. If someone knows a more elegant way to doing that - besides creating a 10 deep nested Do Loop with a hellacious IF statement and writing the number only if the ten single digits are unique - I’d like to know.

The generation was done in SimplyFortran and ended up with a 43.5 MB .txt file of 3.6 million records.
I opened the file in Pan6, sorted the numbers (just to see how long it would take) and saved it as a Pan file called UniqueNumbers.

Then I added a .pan extension on it (the Pan icon appeared) and opened it into PanX. Note here that I also opened the original number.txt I had generated from Fortran and it imported perfectly. My question/reprot is about the imported Pan6 file.

One of the numbers had been change to have some alpha characters.
3740512689
37405‹äa
3740298651

and when I scroll the the bottom of the file, I get some jibbish and the rows turn different shades of green.

Sorted Down

0123456789
]a¢πf›]Ì“}M}„Lv$(:|NŒ9ˆ"®,«–Ãk¶î…¸ò6∞ñlj£“£=‹Ùnó{¥‘¥ÙflôW⁄-
≥a5¶V≤ë›áË8ɃȒÒCá¬/ü(∆0˘9πBØlî9Ÿå%4•∆®·Y∫ùl/:ŸÅl∂¬…å„"«™a¯w",E—^ª^Pa
≥a5¶V≤ë›áË8ɃȒÒCá¬/ü(∆0˘9πBØlî9Ÿå%4•∆®·Y∫ùl/:ŸÅl∂¬…å„"«™a¯w",E—^ª^Pa
M∑¬ê≤(}çÎspûB◊+fipE1d:ª>åÜ∑|©ˆ´ƒ«¬û^O‹≈÷f]õ´ïIE’R¶»5◊§É¨y’ßu∑“[b‚(4ù˛fpôl∏¿…‹Ø∫XÏö%.’F«“4√èÚd;⁄≈¡ ÄFîuZÓ©=møx»È"Õn›ï3ıro`W#ˆêK◊◊â¬√ßÆ˘gó!–≈i(üÚ§"™NëÜ“q∑ öª¨d,ôŒ5

About 2M records were missing and not all were visible even thought I haven’t selected anything.

So I’m thinking it might be safer, if possible, to export Pan6 data as .txt and import it into PanX rather than open the Pan6 file itself.

I created another numeric field in Pan6, sequenced it 1, 1 and the final number was the correct count (couldn’t read the font showing record count at bottom). But when I scrolled back to the top, It would only go up to record 11

I have 8GB of RAM in both the Pan6 and PanX machine. There is no problem, so far, with the imported numbers from the .txt file.

I know 3.6M is a lot of records, but Pan6’s memory in use was only 33%
I supposed I could export the file in two smaller sizes but what troubled me was the how it changed when imported to PanX

2

One observation above is incorrect. I thought I was generating a sequence 1, 2, 3, 4 … but when I look at the result, the sequence started at 11 and continued 12, 13, 14 … So it didn’t “loose” the first 10 records.

I’m wondering, with numbers the high, if I didn’t sail over the edge fo the world with respect to a max number size.

Just a curiosity on the Pan6 to PanX thing. Importing the “raw” numbers.txt file works just fine.

3

Does this file still open correctly in Panorama 6? It sure sounds like it is corrupted.

That means that the rows are being interpreted as summary records. Does the Panorama 6 database contain summary records?

You keep talking about “numbers” but this is really a text field, correct?

In any case, I think the most likely explanation for these symptoms is that the file did not correctly import into Panorama 6, and the Panorama 6 file is corrupted. Or, perhaps the data got corrupted in Panorama 6. 43.5 MB is a lot of data so perhaps you just did not see the part that was corrupted in Panorama 6, and the corruption showed up differently in Panorama X.

Did you try exporting the Panorama 6 file to a text file? It would be interesting to see if that came out ok. I’m guessing there will be a problem with the exported text file. When importing a Panorama 6 database into Panorama X, the same algorithm is used that would be used for exporting to text from Panorama 6. So I would not recommend exporting to text to import into Panorama X, the same algorithm is used either way.

4

Looks like it was just that file. When I imported the original numbers file, they came into an “A” text field. I changed that to integer. Interestingly, when the file is on a shared network drive and I look at it on the Mac that has Pan6, it shows the Pan6 Icon. When I look at the same file on the network drive from the Mac that has PanX on it, the file icon is blank (just bend page). When I add .pan as an extension, it changes to the PanX icon. I just did the transfer dance again and it was clean as a whistle.
I move the “green bar” file back to the Mac with Pan6 and it was corrupted there as well - except the color bars at the bottom of the records had changed to blue/purple.

I was going to resequenced in PanX but haven’t found that function yet. PanX is still like the grocery store that moved all the food around. You know it’s in there - just in a different place.

5

As a fellow crypto-nerd, I wrote a procedure to do just that about 5 years ago - you can find it as the PERMUTATIONS procedure in the CustomMaths database in the Panorama Database Exchange.

Sadly, although it’s supposed to cater for up to 28 numbers, it bogs down hopelessly when n > 7. I’m looking at it to see if I can find out why but it could take a while. You’re welcome to try.

Fortran is much better suited to this sort of calculation than Panorama X - I wrote my first Permutation calculator in Fortran almost 50 years ago but for some reason I seem to have forgotten the code - must be getting old.

6

With todays CPU speeds, it didn’t take long for a 10 deep nested loop to generate all the numbers. It took more time to make sure the big IF statement, that only recorded the number if it didn’t equal any of the other nine, was correct.

It will be interesting to see if it is faster to read in 3.6M numbers - processing with each read, VS generating the numbers inside the program with Do Loops and ignoring those with duplicate numbers. Now that I have the numbers, I can see how fast PanX can iterate down the records and process vs Fortran’s Read and process.

7

couldn’t you generate them in sequence and then scramble them? i’d use a 2nd field and fill it with some kind of hash multiplied by the time (or whatever) and sort by that. seems a bit easier.

csw

8

True, but my procedure caters for any value of n up to 28 (well, it does when it works). It’s a general permutations calculator rather than a solution for a specific task.

9

Watts, there is no need to sort the numbers, what was important was generating every possible 10 digit number, with no repeated numbers and no number that had any repeated digits. If I were dealing with 4 digit numbers, 1234 would be accepted but 1233 would not; nor would 4234. So the crux of the effort is making sure none of the numbers had repeating digits.

With today’s speeds generating all possible 10 digit numbers can be done reasonable quickly. But that results in 10 billion numbers (10 to the 10 power). The count of 10 digit numbers were each number in the set of 10 in unique is only 3.6 million(10 factorial). So if I generate those candidates first, and only test them, it should be much faster than generating ALL the numbers each time and selecting only those that qualify (only unique digits in the number) for further testing.

There are puzzles called Cryptarithms (like Arithmetic) where and equation is given with letters in place of numbers. To solve, you need to replace each letter with a number such that the expression is correct. Each letter represents a unique number. One way is to observe the relationships. Like no leading letter can be zero. Or if you have B + K = B then K must be zero. If the goal is the method, then those kinds of observations make it a “higher” mental exercise. Buy often the solution is found by trial and error. You can usually eliminate the candidate numbers down so each letter could be one of a few different numbers and you try them all to discover which assignment works.

In that last case, you are essentially doing a “brute force” exercise. And if the task is reduced to that, I’d rather let the computer do it and I’ll get my mental work out by writing a program that tries each potential assignment until it finds a match that makes everything work. The higher goal is a solution to the problem; not necessarily the method to achieve that goal.

Initially, the program would only work for one puzzle with it’s unique equations - the program would have to be modified for each equation. Later, it can be more clever and I could just import the equations as text and have the program parse them into individual letters, make assignments, and assess if the assignment satisfies the relationships.

It gets a little more interesting when dealing with a base different than 10, like base 11 or base 12. But in those cases, the basic scheme is the same. I’d just have to include a function that changes the candidate base12 numbers to base 10 to see if the equation works. Note that with those higher bases you are now dealing with letters; 0123456789AB. So if I had CAT + DOG = OTGA in base 12, I’d convert the three numbers to base 10 to see if the resulting addition added to the expected sum.

This post started because I initially imported the .txt file of generated candidates into Pan6, then brought it into PanX. Something skipped a beat the first time and the resulting file was corrupted. But subsequent repeated attempts were successful - results as expected.

So the glitch was a “one off”; just a short eyebrow raise on the way to a solution.

10

Paul

If you really want to stretch your abilities, go to Project Euler (https://projecteuler.net) and see how far you can get. Chris Watts introduced me to it some three or four years ago - as the problems get harder, it becomes a real challenge to your logic and programming skills. At times, a solution involves calculations with humungous numbers, in which case my Number Cruncher, available on the Database Exchange, is an essential aid.

I also do cryptic crosswords and, decades ago, I wrote a Word Finder (see screen shot below) which contains a list of over 310,000 words and phrases, including most modern IT terms, and solves anagrams, missing letters and a few other similar tasks. it also has a 180,000 item gazetteer for those problems which involve geographical references. It’s still in Panorama 6.0 mode as I’ve never felt the need to upgrade it to Panorama X. If you or anybody else is interested, you can download it from Dropbox - File Deleted - Simplify your life

Screen Shot 2020-04-19 at 11.01.46 am
Screen Shot 2020-04-19 at 11.01.46 am715×652 11 KB

11

What a resource - Thank you, Michael. When the puzzle is base10 and there is only one word in the solution, it is easy enough to just search all non-pattern (no repeat letters) 10 letter words for one that includes all the letters in the equations. The puzzle designers know this, so for the harder puzzles the solution may result in 2 or 3 words with A or I qualifying as a word. And then they move into different base systems.

Any idea for algorithm example for parsing letter strings into words. so “WEAREINTHISTOGETHER” becomes, “WE ARE IN THIS TOGETHER” Note that we would continue into wear and only after ein would it consider to go back and break at we. And look at together, you have “to" and “get" and “her". If the algorithm three separate words, it’s forgivable. It’s a little for difficult to program in an awareness of context (we are in this together vs we are in this to get her).

12

Bart Ehrman, the bible scholar, talks about how scribes did not separate words by spaces until relatively recently, and gave an example of a sentence that can be interpreted in two ways: “Hesawabundanceonthetable.” One interpretation is mundane, and the other is a miracle.

13

German is still notable for word conflation. My favourite is Betäubungsmittelverschreibungsverordnung which means “a regulation requiring a prescription for an anesthetic”. I suspect that even Kurt might stumble over its pronunciation.

14

No, I am used to that kind of word monsters. :joy:

15

And a true sailor will always first learn to use a sextant before learning to use the available GPS systems. There will always be times when power or antennas are not available.

16

It is amazing that you can just note the angle from your position to the North Star and that’s your Latitude. Longitude is a little tricker. I always have a compass on the deck of my kayak. AND I’ve learned not to store my dry bags with camp cooking (metal) gear under that compass :slight_smile: