Client Reports for PHP

[smeubank]

Client Reports functionality has been added on the JS and python SDKs. We would like to add this to PHP as well.

At the time of writing this, the JS SDK example is the most up to date, with an open request for completion on python.

TLDR: Client Reports allows sentry to capture information about events being lost on client side, since so far it has only been tracked how many were dropped on server side during ingestion.

https://github.com/getsentry/sentry-javascript/milestone/8

https://github.com/getsentry/sentry-python/milestone/1

https://develop.sentry.dev/sdk/client-reports/

[Jean85]

I expressed my doubts in Discord:

with the shared-nothing architecture of PHP, I think client reports would be non-trivial to implement... it would need some backing, so no out-of-the-box experience

Do you think the same? How do you plan to hold onto data in JS for example?

[kamilogorek]

Do you think the same? How do you plan to hold onto data in JS for example?

I'm not quite sure what this question is about. Could you please elaborate @Jean85?

[Jean85]

This feature seems useful to report when events are discarded for some reason. The documentation underlines the importance of not sending too much stuff to Sentry, since this could kick in exactly when you're spamming Sentry, maybe due to a bug, a wrong cycle, a recursion.

To avoid spamming client reports to Sentry servers then, we need to keep track for a minimum amount of time of how many events we dropped; but since PHP is shared-nothing, we cannot do it natively outside of a single request, so... where should we store that data?

[mfb]

Sentry browser sends a client report on a per-request basis if there was a send failure (e.g. turning wi-fi off, triggering an error and switching tabs). In fact it will send multiple client reports per request, if I keep turning wi-fi on and off, triggering JS errors, and switching tabs :) So I guess PHP SDK could do the same as a terminate event or shutdown function, if it kept track of send success/failure.

[Jean85]

IMHO that's completely different. The trigger event in your example is human generated, so it's really hard to generate a flood of client reports that way.

OTOH, you could create an automated flood with a simple bug or a simple action! Like...

• you deploy a patched version of a dependency to fix something urgent
• you start getting a flood of notices on Sentry due to that, risking to go over quota pretty fast
• you deploy a fast fix: something to drop the notices locally, like a before_send filter function
• all good on your side...
• ...except that you're flooding Sentry's server with client reports now!

That's the likely scenario that I'm worried about. It's pretty easy to obtain a scenario like that AND to have it deployed on any app that has >1K RPM... Once you release that feature you're basically planning a DDoS on Sentry!

I hope that I'm wrong, so... please let me know if I'm missing something!

[mfb]

Well my point was that it appears Sentry browser will send a client report whenever there was a send failure, which would include when there are over-quota errors or traces on every single page load, like your example.

I.e. Sentry, for better or worse, wants to keep getting slammed with client reports if it's being slammed with errors/traces/etc.?

[kamilogorek]

Sentry browser sends a client report on a per-request basis if there was a send failure (e.g. turning wi-fi off, triggering an error and switching tabs). In fact it will send multiple client reports per request, if I keep turning wi-fi on and off, triggering JS errors, and switching tabs :) So I guess PHP SDK could do the same as a terminate event or shutdown function, if it kept track of send success/failure.

That's not correct. Right now, browser SDK sends client reports only on visibility change, eg. when the tab is closed or moved into the background. There is no other trigger for client reports in place as of now.

https://github.com/getsentry/sentry-javascript/blob/8a5db68df285fbbbb990d3b7c703f32de90cf104/packages/browser/src/transports/base.ts#L57-L61

[mfb]

Can you clarify what is not correct? I think we're in agreement - client report is sent when visibility changes (e.g. I switch tabs)

[kamilogorek]

Ah right, it makes sense once I read it again. I thought that you mean we send reports for every failure, my bad :)

[Jean85]

So what would be the desired behavior for a server-side Sentry SDK for client reports?

[kamilogorek]

In Python, we send reports whenever there is send_event or send_envelope called and 60s after last client report has passed. We also do that when we call flush, which ignores the time interval. There is one exception to this behavior though. When we send any envelope out, we do check for client reports with 30s interval, and attach it to the very same envelope, which will reuse the request.

Here's implementation PR getsentry/sentry-python#1181

[mfb]

Maybe PHP apps could set this up to run on a scheduled basis if the PHP SDK let implementors specify a listener/processor for storing client reports, and provided a means to send client reports?

[ste93cry]

@kamilogorek I don't think that PHP is comparable to languages like Python because of its request handling mode. In fact, in PHP each request starts and ends in a certain time limit and nothing is shared among them, so unless we store things in third parties like Redis, a DB, APCu or the application is a long-running process, I see little value in this feature and how it's supposed to work. I also think that @Jean85's concerns regarding flooding an already rate-limited Sentry instance are valid and have not been addressed yet in the discussion

[kamilogorek]

@kamilogorek I don't think that PHP is comparable to languages like Python because of its request handling mode.

Sure thing, I do agree with that, I just answered how it currently works in the only server-side language we support :)

I also think that @Jean85's concerns regarding flooding an already rate-limited Sentry instance are valid and have not been addressed yet in the discussion

Rate-limits should be applied per-category, so even when we are flooding Sentry with error events, client reports should still be able to come through. This however, is not easy to do in the current state of JS SDK (and PHP as well I assume), as transports do not prioritize one request types over another.
We are thinking about changing this, so that transports are aware of categories/priorities of delivery. I attempted to implement this in JS SDK, but we didn't go through with this solution just yet - getsentry/sentry-javascript@master...q-transport

Its the easiest to understand how it works by looking at 'Buffer/Queue' tests in base.test.ts file.

[smeubank]

FWIW i understand there are some concerns with risks associated with this feature and flooding sentry, and even abuse. This was also mentioned when we first discussed JS and python implementations, iirc. In the end we agreed that the information/benefits we can gain from this outweigh the risk.

It provides customers and sentry support with a more data driven approach when looking at issues with customer side applications, or even just improvements. We can get a view per project what the sample rate actually is etc

I understand there is technical difference between doing this in different languages or frameworks. But I cannot speak to that so much for python vs PHP.