Support for expressions that address any element of an array

SKAO has a use case where support for expressions that address any element of an array would be very useful. A rule like

domain/family/member/my_2d_array > 2.75

expresses then that every element of the array my_2d_array should be checked against the value specified (2.75) and if at least one of them exceeds 2.75, then the expression is true.

The request is clear, but it opens to different scenario. Now every node in the abstract syntax tree representing the formula is a scalar, in the moment I transform them in arrays to implement this feature, the case when both operands are arrays (with size greater then one) needs to be handled. What should be the expected behavior? I see 3 possibilities:

1. an exception is returned, and the alarm is evaluated to the ERROR state
2. the operation is executed element by element only if both operands have the same dimension, exception otherwise as in 1)
3. the operation is executed element by element up to the size of the smaller operand, but possibly the behavior could be different depending on the operation, e.g. '>' could evaluate any left operand element against any right operand element instead

Not sure I understand but let me try. I believe the following two are simple and clear cut:

• foo/bar/dev/the_array > 1.2345: If any of the array elements is > 1.2345, i.e. if the number of array elements in the array is > 0 for which array element > 1.2345 is true, then this evaluates to true.
• foo/bar/dev/the_array > another/great/device/scalar_attribute: Same as the above. If any of the array elements > another/great/device/scalar_attribute then this evaluates to true.

This one, where both parts of the comparison are arrays, is the one your are concerned about, correct? I believe that this can be sorted out. We would have to define what is allowed and the behaviour like you said. I would combine options 1 and 2:

When a rule contains a comparison of arrays then:

• Accept the rule only if both arrays match in dimensions (dim(the_array) === dim(an_array)`. The load command rejects a rule that contains array comparisons for arrays that have different dimensions with an error message. The error message should explain why the rule is rejected.
• When the rule has been accepted, a statement of the form foo/bar/dev/the_array > another/great/device/an_array is true only if every single pair-wise comparison of elements with identical indices is true: the_array[m][n][...] === an_array[m][n][...] for all m, n, ....
• If any of the elements do not match, the result is false.

I would say that the check in the Load command is not meaningful since the size defined for a Spectrum/Image attribute is just the max size, but at every "set_value/push_event", so at every event received, the size could be different. Furthermore we do not have to think only to comparison operations, but there are also math/logical/etc. operations as max(foo/bar/dev/the_array + another/great/device/an_array , result/of/another/expr) != ...

• Mismatch in dimensionality and/or size of each dimension: A very good point. Thanks for raising this. I think we agreed in the call that then at the time of evaluation the AlarmState will be set to ERROR and that Load will not look at the dimensions.
• I believe that the same applies when operations on arrays with different dimensions and/or dimensional size mismatch are in a rule: Set the AlarmState to ERROR.

After some thoughts, I think it should behave in the following way:

with so/me/arr/ay = [1,2,3] the expression (so/me/arr/ay > 2) evaluates to ([FALSE,FALSE,TRUE]), then, if it is the result of the formula, it is implicitly reduced as (FALSE || FALSE || TRUE) ==>TRUE.

So we can be consistent with the expression ([1,2,3] + 1) which evaluates to ([2,3,4]).

If you want to explicitly reduce you array in the formula there are already the functions OR, AND as in (OR(so/me/arr/ay > 2)) which evaluates to (FALSE || FALSE || TRUE) ==> TRUE and (AND(so/me/arr/ay > 2)) which evaluates to (FALSE && FALSE && TRUE) ==> FALSE.

Truethness of numbers is evaluated checking if different from 0, e.g. (OR([0,2.5,0])) ==> ((0!=0)||(2.5!=0)||0!=0).

Finally expressions between arrays evaluates element by element only if same size, ERROR otherwise so: ([1,2,3] >= [3,2,1]) evaluates to ([FALSE,TRUE,TRUE])and ([1,2,3] + [3,2,1]) to ([4,4,4]).

Do you agree?

This is very good!

mentioned in merge request !7 (closed)

mentioned in merge request !8 (merged)

closed with commit 61df5fe1

mentioned in commit 61df5fe1