Sentry's distributed tracing causes missing parent spans in Honeycomb
Forgive me for ranting a bit here: today I burned many hours debugging something stupid. This is a light follow-up to my last rant about how OpenTelemetry feels bad to use.
Also, this is for a very niche audience: people who use OpenTelemetry, Sentry, and Honeycomb.
In short:
Val Town is a TypeScript application: there's a backend, which is based on Fastify, and a frontend, based on React Router 7. They could be combined, but that's besides the point of this. The React Router 7 part also uses tRPC to make queries and mutations nice to do on the frontend.
When I looked into our Honeycomb queries for anything related to tRPC, I saw chaos. There'd be some part of a useful trace, like the tRPC method's name and the database queries in it, and then the GET request above that and then a missing parent span. And since the parent span is missing, I'd get tons of traces all bunched into one. This is a common-enough problem that Honeycomb has debugging documentation for it.
Unfortunately, it's none of the the things Honeycomb mentions. I tried using Claude Code for this too, and it led me through 7 promising solutions, none of which worked.
The problem is really dumb. We use Sentry, and Sentry likes to really get involved in tracing, even though I've tried hard to turn off all of its tracing features. In particular, it loves to muck up OpenTelemetry configuration on the backend as well as tracing on the frontend.
The thing is: by default Sentry also tries to do distributed tracing: it's the default! That sucks!
Which means that every time that there was a fetch() call in our frontend application, Sentry would go in there, instrument the function, and inject a Baggage header that contained a trace ID. And then, on the backend, would take that header and put it on OpenTelemetry traces.
In theory, this lets you connect frontend sessions with backend bugs and see through your whole stack. In practice, it means that if you are:
- Using Honeycomb for backend tracing (but not frontend)
- Using Sentry on both your backend and frontend
Then Honeycomb is broken by default for any backend routes that are on the same domain as the frontend and are requested via fetch().
This is because Sentry creates a trace on the frontend, sends its trace id to the backend along with the fetch request() intending to create a span that spans the frontend and backend. But since that never goes to Honeycomb, it's missing, from Honeycomb's perspective. Which messes up all your queries.
The fix is disabling distributed tracing:
Sentry.init({
dsn: "…",
// Overwrite the defaults to ensure no trace headers are sent
tracePropagationTargets: [],
});And then make sure you don't have SentryPropagator in your OpenTelemetry configuration on the backend, because that's what receives the Baggage header and connects it to traces.
So: I sincerely wish that you find this blog post before you lose a day to this problem.