LF vs CR Frustration

Panorama X
1

I can’t begin to guess the amount of time I’ve spent resolving issues over invisible line breaks.

A text editor can have a list if items and processing it as an array using cr() as the separator may be working just fine. Then the result may be unexpected. Items may be missing or otherwise malformed.

This can often be because editing the Text Editor contents and using the Return key inserts a lf( instead of a cr(, and the list is now a mixture of separators, even though its appearance has not changed. Suddenly, array processing doesn’t get the expected result.

lftocr( certainly resolves this. I can set my database so that every time a field is edited lftocr( is applied but what an inconvenience, and it’s not foolproof.

It would be a great convenience if there was a consistent separator without my intervention. Maybe through a setting for the field where I can choose which one I want. Or maybe Panorama could ensure that every time I hit the Return key, it applies a cr( or a lf( if that’s better. Then I could be consistent in how I deal with

But the seemingly random application of lf( or cr(, and especially their occasional coexistence in text, has been a headache for me for years.

2

This is the solution I use. I actually think it’s the most simple and reliable.

3

Jim’s original post operates on the assumption that this is fixable. This situation is just like the Tower of Babel - once humans started speaking different languages, the cat was out of the bag.

Over it’s 41 year history Panorama/OverVUE has operated on systems with three different types of line breaks:

  • Carriage Returns (Classic Mac OS)
  • CR-LF (Windows)
  • Line Feed (OS X)

During the Classic Mac OS era, everything was fine, everything used carriage returns. Simple.

When OS X was released, however, Apple decided that instead of maintaining compatibility with Classic Mac OS, it would maintain compatibility with UNIX. UNIX has used line feeds as the line break character going all the way back to the 1970’s. I think I would have made a different choice if it was up to me, but I guess OS X was primarily developed by NeXT developers with a long UNIX background. So that’s what they decided.

So now what? Panorama databases were mostly full of carriage returns. I suppose those could have been converted to linefeeds when converting to Unicode, but then what about the few cases where databases already contained linefeeds? However, maybe that would have been a good solution, because certainly new users that didn’t know anything about Classic Mac OS would expect that line breaks would be line feeds. But that would be problematic for existing Panorama 6 users, because any code they had written would assume that line breaks would be carriage returns.

If someone is new to Panorama X, I would recommend that they just use line feeds and be done with it. Except it’s not that simple, since lots of existing Panorama functions use carriage returns as delimiters, a full text search for “return delimited” in the help turns up 75 entries.

Another option would be to try to turn back time and coerce everything going to/from OS X to carriage returns. In retrospect, maybe I should have attempted that - but it seemed impossible and surely would have taken months of time. In other words, effectively use crtolf( internally when sending data to OS X, and lftocr( internally whenever retrieving data from OS X. As it was the Panorama 6 to Panorama X transition took a decade, and attempting that would have definitely added to that significantly.

Ultimately this only affects a couple of dozen users - with me being at the top of that list. As suggested by @billashw, I just now automatically add lftocr( any time data is being pulled from an external source. I don’t understand why you say it’s not foolproof - it’s absolutely bulletproof.

4

It would seem then, that we should do our best to take cr( out of our vocabulary. As with so many examples in Panorama Help, I use cr( as a separator constantly. As a result, a list like I mentioned in my first post is built using cr(. If I edit the list, I’ve added lf( without necessarily being aware of it.

It’s an issue I’ve run into over and over. And yes, I know how to correct it once I realize what I’m dealing with.

So I’ll begin a purge of cr( in all of my work. But I have to wonder how many others will use the Help examples in something like arraybuild and introduce cr( into their data, only to have hard to find issues later. if lf( gets into it.

5

It would seem then, that we should do our best to take cr( out of our vocabulary.

If you’re asking my opinion, then no, I don’t think this is a good strategy. In fact, I think you’ll actually make life more difficult for yourself with this strategy.

When displaying and editing text, Apple’s code treats carriage returns and line feeds the same. Either one of these characters starts a new line. In fact, the only difference between classic Mac OS and OS X/macOS is what character gets inserted when you press the return key. In classic Mac OS (and when using a Carbon app like Panorama 6) pressing the return key inserts a return character. In OS X/macOS, pressing the return key inserts a line feed.

The bottom line is that you have to assume that any text that is edited by Apple’s code could have carriage returns, line feeds, or a mix of both. In fact, this is not just true for Apple’s code, but also external editors like BBEdit, or text retrieved from the web. In fact, you might also find text that uses CR-LF for line breaks.

So bottom line, you can’t just assume that line breaks are represented by line feeds. Maybe yes, maybe even most of the time, but not 100%. This is not under Panorama’s control.

However, it really doesn’t matter what the line break character is - UNLESS you are planning to use the text as a text array. Then you need to know exactly what the separator is, and my strategy is to use either lftocr( or crtolf( right before I start using text as an array. For example, text from a file:

lines = lftocr(binarytotext(fileload("some path")))
looparray lines,cr(),line
    ... do something with line
endloop

Or text from a database field:

lines = lftocr(SomeField)
looparray lines,cr(),line
    ... do something with line
endloop

Or text from the clipboard:

lines = lftocr(clipboard())
let modifiedLines = arrayfilter(lines,cr(),{... some formula}

In my experience you have to use either lftocr( or crtolf( in these situations, because you just don’t know for sure what separator (or separators) is being used in these external data sources. I usually use lftocr(, because as you say, Panorama has a lot of statements and functions that explicitly work with CR delimited text. For example:

let clipFirstLine = firstline(lftocr(clipboard()))

gets you the first line of the clipboard. Now you could also do it this way:

let clipFirstLine = array(lftocr(clipboard()),1,lf())

but I think the first is easier and clearer. What I would not recommend is this:

let clipFirstLine = array(clipboard(),1,lf())

This last bit of code is assuming there are line feeds being used as line breaks, but I don’t think that is a safe assumption. Best bet is to use lftocr( every time when you transition from an “external” source of text (which includes database fields because they are edited by Apple’s code) to using the text as a text array.

I use cr( as a separator constantly.

So to summarize, I would stick with using cr( as a separator for the most part, and just make sure to use lftocr( before using a text value from an external source as a text array. In fact, not only would I recommend that, that’s exactly what I have done in my coding. Panorama’s own libraries and wizards have several dozen lftocr( functions strategically placed as needed, and also a couple dozen crtolf( functions (I think mostly when I wanted to export a text array back out to something external, like a text file).

1 Like