Description
Describe the bug
OTEL instrumentation seems to stop working for resteasy-reactive resources and JDBC calls when dev mode is reloaded (manually or due to a file change).
The reproducer further down demonstrates that via the logged MDC values, but it is not a mere MDC issue.
I first found this issue on my local Jaeger instance. I was missing some traces and spans and instead I got some unexpected traces that are supposed to be sub-spans of traces.
I also didn't see any JDBC sub-spans in my @Scheduled
traces anymore (see also #28552 (comment)).
Expected behavior
Traces are generated just like before the dev mode reload
Actual behavior
No proper traces are generated anymore after dev mode reload
How to Reproduce?
- start dev mode in the OTEL-quickstart via:
mvn quarkus:dev -Dquarkus.log.console.format='%d{HH:mm:ss} %-5p (%X{traceId},%X{spanId}) [%c{2.}] (%t) %s%e%n'
- call http://localhost:8080/hello
- verify that trace and span ids are logged, e.g. (first bracket):
17:25:09 INFO (f67e99e80328a210c23f58665ade8ffd,e561a98c8dcc9b37) [or.ac.op.TracedResource] (executor-thread-1) hello
- hit
s
in dev mode (or change a file and call/hello
) - call http://localhost:8080/hello
- from this point on only zeros are logged for both ids:
17:25:47 INFO (00000000000000000000000000000000,0000000000000000) [or.ac.op.TracedResource] (executor-thread-0) hello
Note: This quickstart is a bit sparse ATM, see also: quarkusio/quarkus-quickstarts#1120
Output of uname -a
or ver
No response
Output of java -version
No response
GraalVM version (if different from Java)
No response
Quarkus version or git rev
2.14.2
Build tool (ie. output of mvnw --version
or gradlew --version
)
No response
Additional information
Another user had the same issue a few months back: #26804 (comment)
It's worth stressing that bean methods annotated with @WithSpan
are still working after reload, but obviously, if those calls are sub-spans of REST calls they are shown as separate traces in Jaeger as there is no parent trace anymore.