Add configurable stopwords feature

[yukidukie]

resolve #300

This pr adds a feature that allows users to override the predefined stopwords of Sonic.

The configuration file can include this to configure Sonic's stopwords only to foo and bar.

[channel.search.stopwords]
eng=["foo", "bar"]

I also believe the pr could help #254 and #266 too because you can do this to disable stopwords completely.

[channel.search.stopwords]
eng=[]

[valeriansaliou]

Thank you for this PR. Would it be possible to use macros in order to do some codegen on this?

[yukidukie]

@valeriansaliou
Thank you for the review. I will try macros.

[yukidukie]

Hi, @valeriansaliou
I added a stopwords codegen macro. If I'm misunderstanding something, please let me know. Thank you!

[valeriansaliou]

Thank you, will review soon!

[RemiBardon]

Hello @yukidukie and thank you for your contribution.

I know it’s been silent for a while but we are resuming work on Sonic to make substantial improvements. I will try to integrate your PR in the next releases.

Overlooking your code, I think it could be done in a more flexible way without having to hardcode the struct keys (by using a HashMap and a custom deserializer). I say this because I just had to add a new language in a PR (#342) and it’s the kind of situation where we’d have to remember to update the configuration struct. Mistakes will inevitably happen so I’ll try to find a clever way to mitigate this issue.

Would you be okay if I create my own PR or would you like me to modify yours so you keep co-ownership? In any case, I’ll ping you on my PR whenever I create it and propose to add you as a co-author.

[yukidukie]

Hey @RemiBardon

Thank you for the detailed feedback. That makes sense to me.

Feel free to open your own PR if that’s easier, and thank you for offering co-authorship as well!