Parameter appears to be truncated


#1

Having had my fingers burned with recent entries to BitBucket, I’m just putting this item up for comment (and possible destruction).

I have a custom statement, AngleDecToDMS which converts angles expressed as degrees and decimals to the sexagesimal format. The decimal angle is provided as a text string and the core of the conversion is this code, where Alpha1 is the decimal angle:

Alpha1 = "59.82379"
Angle = val(Alpha1)
Deg = int(Angle)
Temp = 60 * (Angle-Deg)
Min = int(Temp)
Sec = round(60 * (Temp-Min),val("1e-"+str(2)))
Result = str(Deg) + "°" + " " + pattern(Min,"##") + chr(39) + " " + pattern(Sec,"##." + rep("#",NumDec)) + chr(34)

Alpha1 is calculated in another custom statement and the AngleDecToDMS statement is invoked by AngleDECtoDMS str(Alpha1),RevAz,"2" where RevAz is the angle in sexagesimal format and the value 2 indicates two decimal places in the seconds component.

The output from the AngleDecToDMS statement is 59° 49’ 25.68’’ which is exactly what you’d get if you set Alpha1 equal to 59.8238, which just happens to be its six-digit default display.

If I invoke the conversion with AngleDECtoDMS pattern(Alpha2,"#.######"),RevAz,"2", I get a result of 59° 49’ 25.64’ which is what you’d get if you did the conversion manually.

So it appears that, at least in this circumstance, when passing a parameter from one procedure to another, it is necessary to use the pattern( function to override the default display protocol if you want your parameter to register as having more than six significant figures. I don’t think this is a good thing.

michael


#2

You say that this is a custom statement, and show an example of invoking this statement with three parameters. And you say there is a problem in passing parameters to this statement. But the code you show us for the statement doesn’t contain any parameter( function at all. You say this is the core of the code, but if there is a problem with parameters, you had better show us the actual code that handles the parameters!

In any case, based on this line:

If I invoke the conversion with AngleDECtoDMS pattern(Alpha2,"#.######"),RevAz,“2”,

It appears that you are passing floating point parameters values as text. That would generally be a poor idea. You should pass floating point values as numbers. That way, there will be no question that the exact value is being passed. This would be the way to do it in both Panorama 6 and Panorama X.

If there is some reason why sometimes you want to pass a parameter as a number, and other times as text, that’s ok. I definitely have code that does this sort of thing. You can use the datatype( function to find out what kind of data was passed, and handle each case appropriately with an if statement.

This function is available in both Panorama 6 and X.


Another minor point, I’m curious why you used this?

val("1e-"+str(2))

Is there some reason you didn’t simply put

1e-2

there? Or even?

0.01

Seems like it would be a lot simpler and clearer. Perhaps this was adapted from somewhere else where the 2 was a variable? In that case, though, I would think you would want to use the ^ operator. But for a constant value, I suggest just using the actual constant value.


#3

[quote=“admin, post:2, topic:435”]
You say that this is a custom statement, and show an example of invoking this statement with three parameters. And you say there is a problem in passing parameters to this statement. But the code you show us for the statement doesn’t contain any parameter( function at all. You say this is the core of the code, but if there is a problem with parameters, you had better show us the actual code that handles the parameters!

I was trying to keep it simple. The parameter is taken in by Ang = replace(parameter(1),"°","") which disposes of any degree sign in the parameter. There’s no reason why it couldn’t be numeric.

It’sstr(NumDec) in the procedure. The variable, NumDec ensures that the output precision matches the input precision so nothing is lost and no false precision is applied.

I’ll post a second reply addressing what I believe to be the root of the problem.


#4

The crux of this problem is that the str( function doesn’t perform as advertised. The Help file says:

“The str( function converts a number into text, using a plain format. If you want to format the number (add commas, specify the number of digits after the decimal point, add leading zeros, etc.) use the pattern( function.”

I don’t want to add commas, specify the number of digits after the decimal point or add leading zeros. Nevertheless, the function doesn’t convert moderately large numbers to strings. Consider this code:

a = 123456.789
message a
b = str(a)
message b
c = val(b)
message c
d = 10*val(b)
message d

In Pan6, the messages are: 123456.789, 123456.789, 123456.789 and 1234567.89, as you’d expect.

In PanX, the messages are: 123457, 123457, 123457, 1234570 which nobody in their right mind would expect, going by the Help file entry.

Is the six-digit limit imposed by Apple? If so, one wonders why but in any case, a work-around is needed. If it’s a Panorama “feature”, it needs to be done away with. If a variable has a value of 123456.789 and I want to display it in a message, I should see 123456.789.


#5

This appears to be a bug. Even the formula
float(12345.678)
is giving the result 12345.7.


#6

The float( function works fine, as you can see if you try this formula.

pattern(float(12345.678),"#.#####")

When converting a floating point value to text without a pattern, Panorama uses the IEEE standard, which is 6 digits precision.

http://pubs.opengroup.org/onlinepubs/009695399/functions/printf.html

The double argument shall be converted to decimal notation in the style “[-]ddd.ddd”, where the number of digits after the radix character is equal to the precision specification. If the precision is missing, it shall be taken as 6;

I have to agree that you’ve presented examples where this is probably what is not expected. However, the C language (and by extension Objective-C) does not provide a formatting option that works “the way you would expect”.

At the moment, I am at a bit of a loss on this topic – forcing it to always output 15 digits of precision would satisfy Michael’s issue, I guess, but I think that would make a lot of other people unhappy.


#7

I tested that formula float(12345.678) in the Formula Workshop, and I indeed don’t expect the result 12345.7 — I would expect it to show the original number with its 3 decimal digits. But I see the rounding happens even with the original number itself:
12345.678
gives the same result 12345.7 .

I had always thought 6 digit precision means 6 digits after the decimal point, but when I try it in the Formula Workshop I see that it seems to mean: Only 6 digits are displayed wherever the decimal point sits:
If I enter 123.45678, the result is 123.457 .
If I enter 1.2345678, the result is 1.23457 .


#8

Unless I’m misreading this, I think this option, with a specified precision of 15, would meet most people’s expectations.

g, G
The double argument shall be converted in the style f or e (or in the style F or E in the case of a G conversion specifier), with the precision specifying the number of significant digits. If an explicit precision is zero, it shall be taken as 1. The style used depends on the value converted; style e (or E ) shall be used only if the exponent resulting from such a conversion is less than -4 or greater than or equal to the precision. Trailing zeros shall be removed from the fractional portion of the result; a radix character shall appear only if it is followed by a digit or a ‘#’ flag is present.


#9

A post was split to a new topic: How do I start a new topic from email?


#10

The problem with using pattern( instead of str( is that you have no idea how many decimal places to cater for (think of 0.000000000000567) and you’ve got to get rid of trailing zeros, so why not just have a second str( function? Here’s a rough bit of code that converts a number with 14 significant digits into a text string. The comments after lines of code give the computed value for that line (saves you having to step through it).

I had to fudge a bit at the end because I lost a tiny bit of precision somewhere along the way and I couldn’t be bothered looking for it. I’m sure it could be done much more efficiently in C++.

; StrAll 26.9.2016

;  A procedure to emulate a Panorama 6.0 style str( function in Pan X

local a,b,c,d,e,StrOut

StrOut = ""

a = 1234568.0934567
b = str(a)                                    ; 1.23457e+06
c = a - val(b)                                ; -1.9065433
d = replace(b[1,"e"],".","")[1,-2]            ; 123457

if c < 0
    e = val(b["e",-1][2,-1])                  ; 6
    d = str(val(d)-1)                         ; 123456
    c = a - val(d[1,1]+"."+d[2,-1])*10^e      ; 8.0934567
endif

StrOut = StrOut + d

b = int(val(c))                               ; 8
StrOut = StrOut + b + "."                     ; 1234568.
c = c - b                                     ; 0.0934567

loop
    if c < 0.1
        c = c*10
        StrOut = StrOut + "0"
        stoploopif c ≥ 0.1
    endif
while forever                                 ; c is now 0.934567
                                              ; StrOut is now 12345678.0

c = c + 0.000000001    ; This is a fudge because I've lost a bit of precision along the way
b = c * 10^6                                  ; 934567
d = int(b)                                    ; 934567
StrOut = StrOut + d                           ; 1234568.0934567

d = b - d                 ; 0.000785602 without the fudge - no idea why, it should be zero.

if d > 0.001
    ; Keep processing
endif

;  The value of StrOut is "1234568.0934567"

It’s rough but you get the idea. If you like it Jim, I’ll sign over the IP if you agree to call it michaelsreallyweirdstr( function :slight_smile:


#11

Dave, as it happens, Panorama X is already using the %g option. However, I’ve not specified the precision, at least not that I recall (this code was written several years ago, so perhaps I tried it once and forgot). I’m not quite sure what the result will be, but seems worth a try to see if it does the trick.