ref(grouping): Log example false positive ip data

[lobsterkatie]

This adds a new parameterization helper, _log_example_data, which will log the given example data up to a given limit of times per deployment. (The default is 100, but a different limit can be passed in.) It then uses the helper to log examples of our IP false positives.

[wedamija]

I don't know off the top of my head how many workers we deploy that would touch this code, but just want to double check it's not some extremely high amount, since it'll happen every time we deploy.

An alternative if we're concerned could be to just use a redis rate limiter

[lobsterkatie]

I don't know off the top of my head how many workers we deploy that would touch this code, but just want to double check it's not some extremely high amount, since it'll happen every time we deploy.

An alternative if we're concerned could be to just use a redis rate limiter

If it helps, we only hit the false positive case we're logging in roughly 1 out of every 1000 cases (which is part of the reason I went with a counter rather than a sample rate). Also, I'm only planning to leave this running long enough to harvest a bunch of examples of the phenomenon (depending on how fast they pile up, possibly only a few hours). So I don't think this should end up being a huge number of logs in total. (IDK if that answers your question, but hopefully?)

[lobsterkatie]

UPDATE: Here's the PR doing the removal, when the time comes: #112522.