[enhancement] Memory optimize systematic linear codes

[christakahashi]

I have a proposed enhancement:

I'm using a Reed Solomon code over GF(2**16) and the G matrix gets around 7-8gb. Since its a systematic code, a k sized block of G is the identity matrix, which is typically the majority of that 7-8gb.

I noticed in the source code that optimizations are made to avoid multiplication by the identity for systematic codes but storage of that matrix is still done.

Besides just not being wasteful this has the benefit in enabling faster serialization of a galois.ReedSolomon object. With the newest version (0.4.10) of galois it takes about 20 seconds for me to generate a ReedSolomon object or alternatively about 16 seconds to de-serialize one from pickle. shrinking the memory footprint would greatly speed up de-serialization allowing me to speed code that relies on a predefined long RS code.

(also thanks again for this great library!)

[mhostetter]

Thanks for the report.

I will work to add this in the next release. I think I can refactor things to save a fec._P matrix privately (or maybe expose it) that is the parity generator matrix for systematic codes. That will be much smaller than G. When fec.G is called at runtime, it will np.stack((I, P)) and return it.

Does that sound sufficient to you?

Another option is to not pre-compute G or P at all, but instead encode messages algorithmically on the fly. That may be slightly slower in runtime (not sure), but would definitely be a more complex change to make.

[christakahashi]

Sounds great to me! :)