Apparently, under Windows, Python does a UTF-16 word-by-word comparison when comparing two strings:
>>> u'\ud700' < u'\U0001d41a'
True
>>> u'\ue000' < u'\U0001d41a'
False
Fix it by encoding as UTF-32 big endian before comparison, when that happens.
- Remove it from lslextrafuncs, and move all the code to lslbasefuncs.
- Make it behave like SL's more accurately. Denormals return 0 always in SL.
- Use int() for truncation rather then floor/ceil.
- Add test cases.
inf and nan already did the right thing in Python, but just in case that doesn't happen in all platforms, we handle them explicitly. Also, that will make it more immune to bugs in future.
We had a big chaos with type conversion. That caused a bug where passing a key to a function that required a string, or vice versa, crashed the script.
Diminish the chaos by modifying the parameters just prior to invocation (in lslfoldconst). We also remove the, now unnecessary, calls to force floats, either alone or within vectors or quaternions.
The previous commit didn't work as expected. "from module import var" freezes the value at load time; changing it later has no effect. A reference to the module needs to be used.
Fix that and the similar problem with LSO. Also revert some "from lslcommon import *" introduced earlier.
That also revealed another bug about missing 'cond' in the import list of lslextrafuncs. This should fix all functions that return values on null key input.
llBase64ToString doesn't behave like llUnescapeURL wrt invalid UTF-8. First, the last NUL if any is removed, and the remaining NULs are converted to "?". Second, all overlong sequences are converted to a single "?", *including the 5- and 6-byte UTF-8*.
Implement this behavour and the corresponding unit tests.
LSO strings are byte arrays, but our strings are made for Mono which uses Unicode, and invalid UTF-8 sequences can't be stored in Unicode without using a custom representation.
One possible representation is to only use the codepoints 0-255 in the Unicode string, to avoid supporting multiple types for strings. Something to study in future.
Our UTF-8 validity checker failed to recognize that characters in the surrogate range (D800-DFFF) were invalid. Fortunately, Python 2 is happy about that, therefore it doesn't crash (Python 3 fixed that range too). Unfortunately, SL isn't, therefore we fix it.
Added corresponding unit tests.
All three functions have in SL the Intel problem of inaccuracy with high values. Since we work in single precision, it's barely detectable usually, but for large input numbers the difference between the imprecise results of SL and the more accurate results of recent glibc become more obvious.
This change brings back the Intel inaccuracy, even across systems (different versions of Python/C might behave differently otherwise).
Reference:
https://randomascii.wordpress.com/2014/10/09/intel-underestimates-error-bounds-by-1-3-quintillion/
It was designed so that only ints and floats should be accepted. But for safety, and for reuse, make that "any non-float" so that long integers are also truncated to F32.
We were using a very dubious method to distinguish an indeterminate from a NaN, via struct.unpack. Turns out that math.copysign gets the job done and seems more robust.
Besides dividing by zero, any result producing NaN including inf/inf, NaN/anything, anything/NaN causes a math error as well. We only contemplated NaN/anything and neglected the rest, so we generalize it.
- Fix a bunch of bugs found during the debut of the LSL calculator.
- Add infrastructure for functions to be able to produce a result or not depending on arguments. Fixes the llBase64ToInteger/llXorBase64/llXorBase64StringsCorrect cases where they are not deterministic, and allows for the addition of some extra functions whose value can be determined in some cases (e.g. llDetectedType(-1) is always 0). Added several such functions in a new module.
- Add the constant folding option to the help and the default options.
Fix bug: add(Key, Key) is not valid.
Fix bug: llList2CSV was raising an exception always.
Fix bug in test program: llDumpList2String requires Unicode separator.
Patch test program to not output the passed tests.
