refactor(destination): implement shared-filtering

[adleong]

In our testing and in reports from users, we have found that destination controller memory use scales linearly with the total number of concurrent subscribers that it servers. In one load test, we found the scaling rate to be approximately 330 kiB per subscriber. This causes the destination controller to use very large amounts of memory when there are a large number of subscribers. A cluster with 10,000 subscribers might expect the destination controller to use 3.4 GiB of memory.

Memory profiling revealed that the majority of this memory is consumed by the per-subscriber endpoint translator. More specifically, by the endpoint snapshot that it holds in order to calculate filtering and diffs to send to the client.

We reduce the memory scaling factor by noticing that this filtering and diff state can be shared between multiple subscribers if they have the same filtering conditions. In other words, subscribers only need their own state if they have different node-local or zone-local filtering. Therefore, we can group subscribers by a filter key where
all subscribers with the same filtering can share state.

To implement this, we break the unwieldy endpoints_watcher.go into a number of different files, aligned around the major types defined in this file:

• endpoints_watcher.go: the top level EndpointsWatcher type
• service_publisher.go: the servicePublisher type, per service
• port_publisher.go: the portPublisher type, per service port
• filtered_listener_group.go: a new filteredListenerGroup type, per filter key

We move the filtering and diffing logic out of endpointTranlator and into filteredListenerGroup so that it can be shared. In this way, endpoindTranslator becomes a much simpler translation layer.

With these changes, our load test measures up to an 84% reduction in destination controller memory use, depending on the distribution of subscriber filter keys.

[alpeb]

Great changes, the Destination code definitely looks less entagled now 👍
Do you think it would be useful to expose metrics about the different filteredListenerGoups, or is that too much of an implementation detail?