Why does this not replace the second “111” - the second array element - with “A”? It does replace the 6th element (as it should). Why not the second?
local testreplace
testreplace = “111,111,311,212,211,111,113”
message testreplace
testreplace = arrayreplacevalue(testreplace,“111”,“A”,“,”)
message testreplace
If the value occurs multiple times in the array, every occurence of the value will be replaced, with one exception: if the value occurs in two consecutive array elements, only the first occurence will be replaced.
This is a custom function. Its formula is
ARRAYREPLACEVALUE+4(replace(•4+•1+•4,•4+•2+•4,•4+•3+•4)[length(•4)+1,-(length(•4)+1)]
In order to prevent replacing an element of 1111 with A1, the formula includes the separators that come before and after the element. When there are two matches in a row, the separator between them is used as the separator following the first one, and not as the separator preceding the second one.
Here is a possible alternate formula.
ARRAYREPLACEVALUE+4(replace(replace(•4+replace(•1,•4,•4+•4)+•4,•4+•2+•4,•4+•3+•4)[length(•4)+1,-(length(•4)+1)],•4+•4,•4)
The separators in the interior of the array are doubled up. Then the replacement takes place, Then the separators that were added to the beginning and end of the array are removed, and finally, the separators in the interior are singled up again.
Thank you Dave. I fell for the old, “Read the action, see that its exactly what I want to do, and use it without reading the last sentence in the Description”, routine.
For new procedure writers, the above is an example of what I’ve posted about isolating a problem to make sure what is supposed to happen, does happen (in this case “did not happen”) rather than try to solve the situation inside a much more complicated procedure.
This is part of solving aid for a cipher called a Compressocrat
Instead of trying to figure out if my loop indexes were were incrementing incorrectly, I took the piece of procedure code outside of a more complicated situation just to be sure ArrayReplaceValue was working as expected.
Now that I know about the problem with two consecutive matching elements, I’ll write my own function/procedure and move forward.
Using regexreplace( rather than arrayreplacevalue( to achieve the same thing, I realised I had the same problem, that the consecutive instances wouldn’t be replaced because the final separator of the first was also the initial separator of the next. On that occasion I got round it by doing the regexreplace( operation twice, because after the first time there will be no more consecutive instances and any single ones that remain will be caught the second time. You could do the same with arrayreplacevalue(. That might well be quicker than more convoluted code to do it in one pass.
In other words:
let SourceString="111,111,111,111,311,111,212,111,111,211,111,111,111,113"
let Search="111"
let Replace="A"
let Separator=","
message SourceString+¶+
arrayreplacevalue(
arrayreplacevalue(
SourceString,
Search,
Replace,
Separator
),
Search,
Replace,
Separator
)
→
111,111,111,111,311,111,212,111,111,211,111,111,111,113
A,A,A,A,311,A,212,A,A,211,A,A,A,113
Or simply…
SourceString="111,111,111,111,311,111,212,111,111,211,111,111,111,113"
let Search="111"
let Replace="A"
let Separator=","
message arrayfilter(SourceString,Separator,{?(import()=Search,Replace,import())})
--> A,A,A,A,311,A,212,A,A,211,A,A,A,113
More elegant, true, but comparing the two methods with an array of a million random integers I find it to be of the order of twenty times slower.
arrayfilter( is far more efficient than any of Panorama’s loop programming structures but it still involves executing Panorama code multiple times in a loop, which isn’t as efficient as a single Objective C function, even when executed twice.
let Separator=","
let Search="23" // a random number
let Replace="0" // will not occur in the array to be searched
// My method:
let Array=exportdataarray(generatedataarray(1000000,{randominteger(1,10000)}),Separator)
let Start=info("uptime")
Array=arrayreplacevalue(arrayreplacevalue(Array,Search,Replace,Separator),Search,Replace,Separator)
let ArrayReplaceValueDuration=info("uptime")-Start
// Gary’s method:
Array=exportdataarray(generatedataarray(1000000,{randominteger(1,10000)}),Separator)
Start=info("uptime")
Array=arrayfilter(Array,Separator,{?(import()=Search,Replace,import())})
let ArrayFilterDuration=info("uptime")-Start
message ArrayFilterDuration/ArrayReplaceValueDuration
→ 20-ish (times slower)
In my situation, I’d be dealing with arrays no larger that 200 elements and a routine I might run four times a year.
Reminds me of a “discussion” I had, five lifetimes ago, with a programming supervisor about the effective use of my time. I spent a day reprogramming a routine so it ran in 2.5 seconds instead of 5 seconds - cut the time in half! But that routine was part of a reporting process that was run once a month during nighttime batch processing.
I still think speed is good; a worthy endeavor. But I no longer consider it an alternate deity to be worshiped at all cost. These days, code clarity has a higher priority.
The posting cuts out of some of the posted code but I get that pcnewbie is calling the ArrayReplaceValue twice. So it picks up the changes that aren’t consecutive, and one of the consecutive pair. Then, because of those changes, there will be no more (of the same) consecutive elements so the second pass will get the rest.
I can comment that so it’s pretty clear what is going on - or incorporate it in a function or procedure so the “double dip” is transparent to the main intention - “replacing all these with that”.
Thank you again, Dave, Gary and pcnewbie, for your contributions.
For a quick piggyback - after extensive reworking, much of the procedure code in one routine has turned from black to green - the same color as my //comments
It runs fine, but it would be nice if the actual coder were back to black and the comments stay green. It seems specific to that one procedure; the others comment in green and actual procedure code is black - as expected.
I’ve noticed that if you comment some lines of code and then uncomment those lines they remain green until you close and reopen the procedure.
In my case it usually seems to be the text editor becoming confused about nested quotation marks of different kinds and resorting to all-green for one or more lines. Making a change to an all-green line will restore the colouring for that line and sometimes others too. Closing and reopening the procedure will restore correct colours throughout.
No such luck yet. Open/Close procedure, Close PanX, restart Mac Mini, copy/paste into a new procedure, it still stays as green as Kermit - not that there is anything wrong with that.
I can live with it. Just an eyebrow raise.
I do make use of all the styles: /* … */, // , on its own line, // on the same line as procedure code.
Maybe I’ll select all, paste it into a new procedure, and remove all the comments - see if returns to black.
Ah, looks like it has something to do with my using
//***************
//comment
//comment
//**************
to set off comments. Because of the alternate /* …*/ structure, I can see a potential for confusion. As I remove the //******** lines, the black text returns
My favorite proposed solution is Gary’s, that is very elegant.
However, thanks Peter’s speed testing, I went ahead with Dave’s solution. The next version of Panorama will include this change.
I’m not 100% convinced that Peter’s solution will work in all cases. Probably, but I couldn’t figure out how to prove it, and for Dave’s solution it was straightforward, so I went with that. I think the performance will be similar.
Speaking of performance - a million array items? Seriously?? Panorama Text Arrays are intended for no more than a few hundred items. The performance definitely falls off for larger arrays. If you need a million items, you should probably be storing the information in a database, which was designed for unlimited size.
That said, performance is performance, so thanks for the test results.
At first I thought you meant that the regexreplace( also had a problem, but you didn’t mean that, did you? I looked into it and regexreplace( is completely different, there is no such thing as consecutive instances. So I think what you meant is that if you use regexreplace( the way that replace( was used in arrayreplacevalue(, you’ll get the same problem. That makes sense. So are we in agreement that regexreplace( is fine?
I’m surprised your supervisor didn’t bring up this very famous quote.
The quote “Premature optimization is the root of all evil” is attributed to Donald Knuth, and appears in his book The Art of Computer Programming.
Panorama X has always had a bit of a problem with syntax coloring when editing /* … */ comments. But usually it recovers pretty quickly, definitely if you close and re-open. But maybe your somewhat extreme block comment format confuses it even more than usual.
I am convinced that his approach works, and I like it better than mine. It uses 2 replace( functions, while mine uses 3. If you had several consecutive matches to the search string, the first would be replaced but second would not because they share a separator. The third would be replaced, but the fourth would not, and so on. For every one that did not get replaced, the ones that came before and after would have been, so you would never have two consecutive after the first round of replacements. Here is the formula, using two replacements.
ARRAYREPLACEVALUE+4(replace(replace(•4+•1+•4,•4+•2+•4,•4+•3+•4),•4+•2+•4,•4+•3+•4)[length(•4)+1,-(length(•4)+1)]
Exactly. That was my logic and to replace multiple instances in a text array, including consecutive ones, I have been using two consecutive replaces in this way for some time — just using regexreplace( instead of arrayreplacevalue(, because I hadn’t noticed the existence of the latter.
That was also why I amended Paul’s original source array in my examples to include one, two, three and four consecutive instances of a string. If a solution works in those four cases it should be watertight.
Yes, the regexreplace( function works, it’s just that when using it as an alternative to arrayreplacevalue( it similarly fails to replace consecutive substrings for the same reason:
let Separator=","
let Array="111,111,111,111,311,111,212,111,111,211,111,111,111,113,111,111"
let Search="111"
let Replace="A" // will not occur in the array to be searched
let SearchRegex1="(^|"+regexliteral(Separator)+")"+regexliteral(Search)+"("+regexliteral(Separator)+"|$)"
// in source array, search string will be preceded by either beginning-of-string or Separator, and followed by either separator or end-of-string
let ReplaceRegex1="$1"+Replace+"$2"
message regexreplace(Array,SearchRegex1,ReplaceRegex1)+¶+
arrayreplacevalue(Array,Search,Replace,Separator)
-->
A,111,A,111,311,A,212,A,111,211,A,111,A,113,A,111
A,111,A,111,311,A,212,A,111,211,A,111,A,113,A,111
(I’ve included the regexliteral( functions for the general case, but for this particular combination of Separator, Array, Search and Replace they are not actually needed.)
However adding a second regexreplace( solved the problem for me just like a second arrayreplacevalue( now does. The second pair of search and replace strings for regexreplace( can be slightly simpler than the first because the first element in the array will never need to be replaced the second time round, although the last still might if the original source array ended with an even number of instances of the search string:
let Separator=","
let Array="111,111,111,111,311,111,212,111,111,211,111,111,111,113,111,111"
let Search="111"
let Replace="A" // will not occur in the array to be searched
let SearchRegex1="(^|"+regexliteral(Separator)+")"+regexliteral(Search)+"("+regexliteral(Separator)+"|$)"
let ReplaceRegex1="$1"+Replace+"$2"
let SearchRegex2=regexliteral(Separator)+regexliteral(Search)+"("+regexliteral(Separator)+"|$)"
let ReplaceRegex2=Separator+Replace+"$1"
message regexreplace(regexreplace(Array,SearchRegex1,ReplaceRegex1),SearchRegex2,ReplaceRegex2)+¶+
arrayreplacevalue(arrayreplacevalue(Array,Search,Replace,Separator),Search,Replace,Separator)
-->
A,A,A,A,311,A,212,A,A,211,A,A,A,113,A,A
A,A,A,A,311,A,212,A,A,211,A,A,A,113,A,A
However, double-arrayreplacevalue( is roughly three times quicker than regexreplace( with two sets of arguments, not to mention much more self-explanatory, so I’m very glad that this thread has pointed me in that direction.
Only because the finest granularity of timing offered by Panorama is one millisecond, thus to compare timings meaningfully I wanted something lasting more than a second.
I need to have that tattooed on my mouse hand. I was getting a little lost in the weeds with a simple 2-dimensional array. But the second dimension always only has three elements.
No real need to put something in a loop that will run 3 times. At least not at this stage. Or I can put a Case Structure inside, query the loop index and the appropriate True Case will store values in the appropriate fields instead a 2-dimensional array variable (not that there is anything wrong with that).
The database has all of around 15 fields.Plenty of room to store intermediate results in Step1, Step2, Step3 fields so I can rebuilt (via initialize) where I left left off when I closed the program.
Everything is cleaned up. I took out those //******** markers and all the code has returned to its expected color.