The input script must be encoded in UTF-8. We had a check that was only activated when the preprocessor was called. Activate it always, as not doing so triggers an unhandled exception later on.
The reason is they have an embedded delay. A script might rely on it, therefore substituting the call with its value is not equivalent to leaving the call. They were both already excluded from the SEF table for the same reason.
When we reduced the scope of the try block in commit a823158, we introduced a bug because the tree modification was attempted even if no value was assigned (when the exception was triggered). Returning when the function is not computable ensures that this won't happen.
Scaringly, there was no check that caught this.
Apparently, under Windows, Python does a UTF-16 word-by-word comparison when comparing two strings:
>>> u'\ud700' < u'\U0001d41a'
True
>>> u'\ue000' < u'\U0001d41a'
False
Fix it by encoding as UTF-32 big endian before comparison, when that happens.
- Remove it from lslextrafuncs, and move all the code to lslbasefuncs.
- Make it behave like SL's more accurately. Denormals return 0 always in SL.
- Use int() for truncation rather then floor/ceil.
- Add test cases.
inf and nan already did the right thing in Python, but just in case that doesn't happen in all platforms, we handle them explicitly. Also, that will make it more immune to bugs in future.
We had a big chaos with type conversion. That caused a bug where passing a key to a function that required a string, or vice versa, crashed the script.
Diminish the chaos by modifying the parameters just prior to invocation (in lslfoldconst). We also remove the, now unnecessary, calls to force floats, either alone or within vectors or quaternions.
The previous commit didn't work as expected. "from module import var" freezes the value at load time; changing it later has no effect. A reference to the module needs to be used.
Fix that and the similar problem with LSO. Also revert some "from lslcommon import *" introduced earlier.
That also revealed another bug about missing 'cond' in the import list of lslextrafuncs. This should fix all functions that return values on null key input.
Instead of using an option in the command line, use a global in lslcommon, settable by the main program (only the main LSLCalc program, which differs from LSL-PyOptimizer's main, changes it).
Failure to do so caused a regression test to fail. Harmless, because that option is overriden by main, but fixed.
Bug was introduced in commit 397dc89, with the requirement that the 'optimize' option be active for output optimizations to be applied, by forgetting to update the function header to add that default option.
lslcalc is currently derived from a snapshot of PyOptimizer at some point in past, making it difficult to maintain when bug fixes are applied to the optimizer. Solve this by incorporating expression evaluation capabilities.
llBase64ToString doesn't behave like llUnescapeURL wrt invalid UTF-8. First, the last NUL if any is removed, and the remaining NULs are converted to "?". Second, all overlong sequences are converted to a single "?", *including the 5- and 6-byte UTF-8*.
Implement this behavour and the corresponding unit tests.
LSO strings are byte arrays, but our strings are made for Mono which uses Unicode, and invalid UTF-8 sequences can't be stored in Unicode without using a custom representation.
One possible representation is to only use the codepoints 0-255 in the Unicode string, to avoid supporting multiple types for strings. Something to study in future.
Our UTF-8 validity checker failed to recognize that characters in the surrogate range (D800-DFFF) were invalid. Fortunately, Python 2 is happy about that, therefore it doesn't crash (Python 3 fixed that range too). Unfortunately, SL isn't, therefore we fix it.
Added corresponding unit tests.
This option normally takes effect through the base class in lsloptimize.py, which doesn't call the optimization techniques if deactivated. However, lsloutput.py is not called by it, and it applied some optimizations on its own.
Fixed on the reading of the optimization options, by filtering them by whether the optimize option is active.
Literal strings were not conforming to Mono's strict "NUL is end-of-string" rules, so a file with an embedded NUL within a string literal would let the NUL be part of the string. Mostly academic, because it's probably not possible to upload a file with an embedded NUL to SL, but fixed it regardless.
As an enhancement over LSL, we trigger a Type Mismatch when there are void expressions in list constructors, because in LSL, while accepted, they trigger an ugly runtime exception.
This works fine, but then expression lists, where this are checked, are not exclusive of list constructor; they are used in other places. One of these places is the initializer and the iterator of FOR loops. As a consequence, we didn't allow void functions in the FOR initializer or iterator.
Fix by adding another possible value to the parameter 'expected_types' in Parse_expression_list. False means don't allow void either (what Null did before); Null now means allow anything. All callers to Parse_expression_list are changed accordingly. Added corresponding regression test.
