[SYSTEMDS-3867] Safe Fast Frame Copy

[yj-40-56]

Student project for DIA at TU Berlin: improving copy()

[yj-40-56]

Replaced Locking on ValueType.BOOLEAN with locking based on ArrayType:

• BitSetArray requires locking due to underlying storage in longs. Small Array are safe if they are shorter than 64 rows (using underlying Byte[]) a. Once above that threshold, the array is turned into a BitSetArray

• OptionalArray has an underlying null-mask, indicating the null entries. That null-mask is based on a BitSetArray and is not thread-safe.

Additional ArrayTypes that are locked:

• RaggedArray: internal resizing should cause race conditions. The set()/append() methods all contained a reset() into set() segment which has no guarantee of being atomic.
• DDCArray/ACompressedArray: Similar to RaggedArrays, modifications are not atomic.

I have verified the fix for BitSetSArray and OptionalArray.

I did not include tests for RaggedArray and ACompressedArray. After looking at the implementations, I am convinced they are not thread-safe, but I was not able to create unit tests to show that specific behaviour.

[janniklinde]

Thank you for your analysis and contribution @yj-40-56. It's good to know that you identified OptionalArray, BitSetArray, RaggedArray, and CompressedArray as array types that may require locking. I think it would be safer to allow a whitelist of array types to bypass locking (also for future proofing). I'll leave the PR open to fix some formatting issues and revisit some aspects that could be done more efficiently. Good luck in your exam!

[Baunsgaard]

Just to pitch in, the implementations are indeed not thread safe for some specific variations of calling. However, if the caller does not have overlapping ranges that it sets, it is perfectly safe for OptionalArrays.

The BitSetArray was intentionally fixed by introducing the lock (as the @yj-40-56 is pointing out), however ideally we should (for good performance) avoid using the lock by aligning updates to the nearest 64 values, to allow full parallelism.

The RaggedArray and the DDCArray/ACompressedArray are both not designed to be frequently updated, so should not be considered a valid copy target.

A better design than the proposed would be to specialize each of the fundamental arrays to take from the Source arrays. We have primitives like:

	public abstract void setFromOtherType(int rl, int ru, Array<?> value);

That allows very efficient cloning from another array into the current one. This should just be called in parallel across different input columns.