Recursion within ‘looparray’ loop

[pcnewble]

This is an aspect of Panorama X 10.1 (I hesitate to label it either a bug or a feature) which I discovered a year ago, filed away in my mind but then forgot, because I have just spent a long time trying to debug a procedure in 10.2.0.b12 (3581) and eventually remembered I had encountered the same problem before!

Thanks to local variables etc., recursive procedures and subroutines work fine in Panorama X. Previously Michael Kellock has queried how many levels of recursion might be possible, but I’ve had no problems in that regard.

However, if a procedure/subroutine calls itself recursively from within a looparray loop, that stops looping.

The example below assumes a database of heirarchical tree data containing (at least) 3 fields:

• KEY — a unique key for each record created using uniqueid() — H1, H2, H3, etc.
• Name — text to be displayed for each record
• Children — a comma-separated array of KEYs of other records which are children of each record

Thus, if the record for which KEY=H1 is the root of the tree:

• H1’s Children might be H2,H3,H4,H5,H6,H7
• H3’s Children might be H8,H9,H12
• H8’s Children might be H13,H14
• H13’s Children might be H24,H25
• H24’s Children might be H30
• H30 is the end of that particular branch if it has no Children.

Any record may have ≥0 children; none may have >1 parent (and there are no feedback loops or incestuous pairings: these are strictly heirarchical data, such as directories and files).

This procedure and its recursively-called subroutine, List, ought to enumerate all the children of H1 as lines in the log file, with the KEYs in the left-hand column followed by the Name, and with the Name of each generation (level of the tree) indented by one more tab than its parent:

 shortcall List,"H1",tab() // Recursively list all descendents of H1
 stop

 List:
     local ThisKey,Indent,Element,Title
     ThisKey=parameter(1)
     Indent=parameter(2)
     Title=lookup("",KEY,ThisKey,Name,"")
     if Title≠"" zlog ThisKey+Indent+Title endif
     looparray lookupmoredata(Children,""),",",Element
         shortcall List,Element,Indent+tab()
     endloop
 return

However, the result is only the first child of H1, the first child of that child and so on, one entry per level (generation) until the end of that particular branch. The remaining children and their children (etc.) are not listed because, following the first recursive call to List each time, the loop terminates, thus it can have no more than one iteration.

By changing looparray to loopwhile and working through the array manually, it correctly enumerates the whole tree:

shortcall List,"H1",tab() // Recursively list all descendents of H1
stop

List:
    local ThisKey,Indent,Element,Title,Offspring
    ThisKey=parameter(1)
    Indent=parameter(2)
    Title=lookup("",KEY,ThisKey,Name,"")
    if Title≠"" zlog ThisKey+Indent+Title endif
    Offspring=lookupmoredata(Children,"")
    loopwhile Offspring≠""
        Element=arrayfirst(Offspring,",")
        Offspring=arraylefttrim(Offspring,1,",")
        shortcall List,Element,Indent+tab()
    endloop
return

(Perhaps that’s not the most efficient way to loop through array elements without using looparray. However, it works for demonstration purposes and, importantly in this case, like looparray it will execute zero iterations if the array is empty.)

There’s nothing in the documentation for looparray to say it’s not compatible with recursion. It notes that it ‘uses special optimizations to reduce the amount of processing that needs to be done on the array each time through the loop’ and that is expanded upon by David Thompson in this post:

@dave

Looparray makes its own internal copy of the array, and works its way through that. Its copy is guaranteed not to change while the loop progresses, and it remembers where the n-1st element ended, so it doesn’t have to go searching for the beginning of the nth.

However, neither that nor anything else I’ve read suggests it might interfere with the procedure stack and therefore cause recursion to break, but I suppose it might never have been intended to coexist with recursion either, as other loops clearly do. In that case it’s not a bug as such, but neither is it the most desirable of features.

I considered submitting a correction to the documentation for looparray to add a warning that it should not be used within a procedure/subroutine call which calls itself recursively from within looparray . . . endloop — if only to remind me in future! — but I thought the behaviour ought to be noted first.

[admin]

@pcnewble

I considered submitting a correction to the documentation for looparray to add a warning that it should not be used within a procedure/subroutine call which calls itself recursively from within looparray . . . endloop — if only to remind me in future! — but I thought the behaviour ought to be noted first.

Yep, thank you for so carefully laying this out. This makes sense to me. David’s comment that Looparray makes its own internal copy of the array is the explanation for this behavior. The copy is actually attached to the internal representation of the statement. If you call it recursively, it would need multiple internal copies of the array, and keep track of which one to use when. It’s not set up for that, so right now, when the statement runs again, it will trash the earlier internal copy of the array. In other words, it’s not designed for recursion. So yes, perhaps there should be a note about that in the documentation.