Relations with temporary databases?

I’ve found it sometimes useful to generate new databases from text arrays or files, use them for intermediate calculations or lookups and then close them without ever having saved them. Suitably placed info(“windowname”) functions provide names for working with them. I’d like to get relations to work with them. Is that even possible?

Some of the new relational tools allow all the criteria to be specified in code without using the panel. For some only a subset of the criteria are applicable, but the panel remains optional. Some require setting up the panel on one or both sides of the relation. Unless there is some way to set up the relations panel from a previously created blueprint or dictionary I don’t see how to set up a relations panel for a database which only exists during the run of a procedure. And I haven’t gotten what relations I can do with saved databases and empty panels to work with unsaved databases yet.

You cannot set up a relation for a database that has not been saved. That’s what you are asking for, correct? The relation name stored in the current file is the name of the source file, but an unsaved database does not have a stable name.

You can use the join statement without a relation. In that case, you must set up all of the parameters in code. In fact, I actually created the join statement first, then designed the relation panel to make it easier to use, and to eliminate the need to specify all the parameters over and over again. But if you want to use an un-named, un-saved file, that is what you will have to do.

Perhaps a better solution for you would be to create an empty database in advance, and set up the relations using the panel. Then when needed you can open the empty database, import the text array or file into it, perform your operations, then delete the data and close the file.

Yes, that was the question. I suspected the answer was something like that, but hadn’t found it in the documentation. If it’s not there it probably should be. And your solution is what I’d expected to hear in that case. I just need to think through the details.

Is there any way to backup and restore panel settings in a saved database? Perhaps via a dictionary or blueprint saved to a permanent variable? Until I can get all my old code running under 10.2 I periodically need to revert to 10.1.2, which I believe removes those settings. Repeated manual redos of panels across multiple databases is tedious. Even after 10.2 runs fine I suspect toggling between different panel settings for different reasons might sometimes be useful if possible.

You have code that runs in 10.1 but does not run in 10.2? I’d like to know more about that. As far as I know there are no features in 10.1 that have been removed in 10.2. I’ve certainly never had to go back to 10.1 to work with any database.

It’s not so much that it removes settings, as that it doesn’t save settings that it doesn’t know about. Actually, it doesn’t read them in when you open the file either. It just ignores them, so the end result is that they will not be included in the database once it is saved by 10.1.

The Relation panel includes a blueprint button, which displays the blueprint of the currently visible relation and allows you to edit it. You could manually save each blueprint separately, for example in a text editor. Reverse the process to restore it later.

I had lengthy code which ran to completion under 10.1.2 but reliably gave an unexpected quit at the same pair of groupup statements under various betas of 10.2. I’d posted at length in the forum. I finally produced a simplified demo version without confidentiality issues which I emailed to support@provue.com on 4/4/21. I sent it from my icloud email. Searching for my last name should turn it up or I can resend it there or to a better email if you wish. Different coding has resolved the original crash sites and a couple others that appeared later my code. My demo illustrates them. But some code that crashes 10.2 and runs fine under 10.1.2 remains. It might be unique to my hardware, which is going to Applecare as soon as life allows.

Saving and pasting blueprints sound like they’ll suffice for backing up panel info. I just need to be organized and careful because there’s no error checking on changing blueprints.

Replacing older code with relational code, including some dedicated pre-saved ‘empty’ dbs as you’d suggested, as well as using summarytable() instead of groupup and total, has so far cut 20+ minutes from an hour+ procedure with more revisions pending. Thanks for the new tools!

But I now have enough relations set I fear breaking old ones while setting up new ones. b18’s new warnings will help some. But I foresee wanting better records of what I’d done before. I doubt I’ll improve my commenting habits enough to suffice.

Your suggestion of saving Blueprints manually to individual or one big text files suits me. But I’m trying to optimize ‘manual.’ I can selectAll, copy and paste their bodies into text editors, but can’t seem to transfer their (excellent) white on blue texted headers/titles except by manually typing them myself. Is there an easier way for that?

If not, and if it wouldn’t clash with your interface, consider adding a popup option (to the header or to the blueprint icon) to save both to the clipboard or to save them as a text file. Latter might default its filename to that header or to that header plus a timestamp.

I’ve just now changed this dialog so that you can click on the header dialog and select and copy the text.

However, the information in the header is really redundant – you don’t need it. All it is is the name of the current database, and the name of the database that is being linked to. The latter is already in the relation, it’s the DATABASE=... line. And the former is of course simply the current database.

Programmers have long wanted/needed better records of what they’ve done, so they invented what are called source control or version control systems (also known as SVN). There used to be several different SVN systems, but about 15 years ago a program called Git became the industry standard. By the way, Git is a free and open source program that was developed by Linus Torvalds. Linus is the inventor of Linux, and he wrote Git to keep changes to Linux organized (because he didn’t like any of the other SVN tools that were available at the time).

It’s far beyond what I can explain here, but Git can be used with Panorama. In fact, Git is the reason I developed Panorama’s blueprint system. All the other uses for blueprints are happy byproducts. Panorama X itself contains nearly 45,000 lines of Panorama code (in addition to 193,000 lines of Objective-C code), and I definitely needed to be able to keep track of when and how that code has been changed. That’s done by saving a blueprint whenever there’s been significant changes, and then “commiting” those changes into Git. Each change is accompanied by notes describing the change. (Note: When you save an overall database blueprint, with either File>Export>Blueprint or the View Organizer’s Database>Export Blueprints for this Database, ALL of the database structure is included, including field definitions, formulas, AND all relation definitions.)

Keeping a useful Git repository requires some discipline and organization. It’s definitely considerable extra work, though not too bad once you have it set up. For a large project, it’s essential to keep order from descending into chaos. If you ever wondered how I come up with the detail release notes for Panorama, basically I just go thru the commit notes that I put into Git since the last release.

Git is a huge topic, and one I’m not qualified to teach. I’ve learned enough to use it myself on one project. There are several full books on the topic, many YouTube videos, and tons of online material. Git itself is a shell program, but there are several GUI programs available that make it easier to work with. I’ve been using SourceTree for several years, but I’m considering switching to another one.

Most Panorama databases aren’t complicated enough to need to use Git, but it sounds like you might be headed in that direction. Something to consider.

Here’s an example that shows you the type of record keeping that Git gives you. This is part of a recent change I made. The white background is code that didn’t change. Green is new code, and red is deleted code. As you can see, you can see exactly what changes were made as part of this update (this screenshot is from SourceTree).

Blueprints allow even changes that are purely graphical to be tracked. Here’s the change I just made to allow the text in the Blueprint dialog header to be selected and copied. I checked the Allow Text Selection option for the Text Display object, and then saved the blueprint and committed the change.

As I mentioned, links are also included in the blueprint. Here’s part of the blueprint for a database with two links:

I will never do anything that comes close to needing this level of record-keeping, but this post is a wonderful example of Jim’s incredible generosity in being willing to teach us so clearly out of his tremendous depth of knowledge and understanding.

Thank you, Jim! What a remarkable person you are.

Something like Git is essential to a successful developer. You need to maximize your personal productivity and there’s too much to just remember it all. From this and other posts you like Git about as much as we like Panorama. But for an amateur coding for himself it’s probably overkill. But I’ve previously taken some of your other excellent software suggestions so feel free to keep making them.

Open View… and View Search… were excellent advances, but are limited by Panorama’s otherwise key advantage: its focus on open files. I don’t expect you to add searching files on disk. Excellent options for that already exist; the free version of BBEdit works well for me. Before PanX I wrote a generic, fast, procedure saving all of that database’s procedures to a text file in one folder. Come PanX a few statements or functions had syntex changes or weren’t yet implemented. A quick search on that folder gave me a list of every instance in my total codebase needing that change. In PanX I enhanced my procedure to also save any code in fields or form objects. Being able to search all my code, open or not, helps and Time Machine on that provides an extra backup of prior code. In principle the same information could be compiled into a PanX database of one’s code, but this worked well enough I haven’t tried.

For these Relation blueprints I’ve been able to partially automate their collection. If the following procedure window is open when a Relation blueprint from the same database is showing clicking its Run button appends it to a text file hard coded in place of ‘path’. Clutzy, but I was able to collect them faster than one/minute even with moving the procedure db to db via View Organizer…. I added the result to my procedure search folder.

objectaction info("activeobject"), "Open"
objectaction info("activeobject"), "SetSelection",0,-1
objectaction info("activeobject"), "Copy"

let bpText=clipboard()
bpText="Blueprint: Join "+quoted(array(info("windows"),4,cr()))+" with "+array(bpText,2,",")+" "+superdatestr(supernow())+cr()+bpText+cr()+cr()

let current=fileload("path")
current=current+bpText
filesave "path",current

I know it’s hard to keep track of all of Panorama’s features – it turns out Panorama 6 already had a feature that saved all of that database’s procedures to a text file – the Source>Export Source command in the View Organizer wizard.

The Panorama X blueprint feature takes that to the next level, it allows you to export not just the procedures but ALL of the design elements of a database to a folder full of text files. I mentioned using this with Git, but you can certainly also use it with BBEdit’s multi-file search. As new structural elements are added to Panorama (for example relational links) they are also added to the blueprint export.

The blueprint export code is all open source, written in Panorama code (not Objective-C). If you take a look at it, you’ll undoubtably find that it works very similarly to the homegrown code export you’ve created. Here’s the code that exports the relational links as text (from SAVEBLUEPRINT in _PanoramaLib). If for some reason you didn’t want to use the blueprint feature you could adapt this code to your homegrown exporter so that it also includes the link specifications.

let linkDatabases = dbinfo("links",targetDatabase)
if linkDatabases<>""
    looparray linkDatabases,cr(),linkDatabase
        let linkDictionary = dbinfo("link",targetDatabase,linkDatabase)
        let linkSource = crtolf(dumpdictionarysource(linkDictionary))
        blueprintCode = blueprintCode+
            {    "LINK",initializedictionary(}+lf()+
            arrayfilter(linkSource,lf(),{"        "+import()})+"),"+lf()
    endloop
endif

Source>Export Source command

I dimly recall spotting that when Pan6 came out and deciding instead keep doing what I’d already coded. Functions that listed procedure names and getproceduretext were available earlier so I’d looped my own way and was used to it. For someone wanting to start roughly what I’ve been doing by scratch saving the Blueprints is the easier solution, but I paid the costs for my way years ago. I should try it sometime and see which I prefer. It may be I’ll prefer the narrower scope of my home-brewed solution. But if I find the additional information of value rather than distracting I could easily compile an array of paths for all my dbs and looparray the creation of all their blueprint files.

Thanks for the code. I’d been looking for something like that, but the “link” and “links” options weren’t yet documented under dbinfo(.

Good point. I have now corrected that.

A post was split to a new topic: Dbinfo( flash art gallery option