Question about best approach to merge growing csv

Panorama X
#1

Hello,
Been away for awhile, hope everyone is well. I apologize ahead of time if I’m not searching the forum properly for this question, but I don’t think I found the exact solution.

Here’s my situation:
As one of my many “day” jobs I compose music. That music is registered online with a place called ASCAP, and it has all sorts of information important to the individual music cue. When I write a new cue that will be used in a production the first thing I have to do is register it there. It’s now a giant (and growing) list of cues and info that I am able to download as an excel or CSV.

What I need create is a database where, in addition to MOST of the info at ASCAP (not all), I need to add all kinds of additional metadata (song genre, instruments used, cue description, etc.)

What I’d like to do is take the CSV or Excel from ASCAP, automatically import only a few of the attributes (song title, registration number, etc.) so that they’re ready for additional metadata. However, I’m stumped as to how to set this up so that as new cues are added to ASCAP, I can run the process to ONLY import new cues, and of course I don’t want to overwrite the existing cues and lose all the metadata I’ve added.

Is this relatively easy to do? Is this going to be easier to do with the new version of Panorama X that’s been recently announced?

Thank you in advance for any guidance, and I hope everyone is staying sane and healthy!

All the best,
Biagio

#2

Are there multiple cues per record and are the cues saved in a single field. If so, how is one separated from another? If that is so, you’re creating an array. In that case you can import all cues, nit just new. Then run ArrayDeduplicate to remove the dupes and you’ve effectively imported only the new cues.

Or, are the cues all individual records and you’re wanting to just add the new? Do the import and sort by the cues. The new cues should be the second of all duplicated records. Use UnPropagateUp on the cue field to empty the duplicated (second) records. Select those cues that are not empty, delete the unselected and you’ve got it.

1 Like
#3

If I understand what you are asking for, I think the upcoming new version of Panorama will help you greatly with this. With the new version, you could import the csv data into a separate, temporary database. Then you can use the new join feature to bring just the data you want into your main database, without touching any of your existing data. Finally, you can discard the temporary database.

1 Like
#4

Thank you @JamesCook and @admin -

James - yes, cues are single records. Right now they have info like:

-cue name- -registration number- -registration date- -a bunch of other stuff I don’t need-

I want to pull those into a full PanX database where I can keep what I do need and then add things like

  • genre - -instruments- -description-

but ignore a whole bunch of fields that I don’t need, and then most importantly, I don’t want to accidentally replace records since I’ll always be pulling from a source that has duplicates. I think I follow what you’re saying with UnPropagateUp on the cue field. THANK YOU for taking the time.

Jim - if the new version makes that easier that’s AWESOME. I fancy myself a tech guy but I’m not :slight_smile:

I’m more of a “get help, copy and paste code, mess around until it works” person unfortunately! (But at least I know that about myself!)

In peeking at what you have coming, it does seem like it’s going to get easier - certainly from your description above as well. I will probably wait for that and just sort of map things out in Excel in the mean time. Congrats by the way - sounds like exciting stuff is coming to Pan X!

All the best,
Biagio