Micrometer to OpenTelemetry Bridge #43831

brunobat · 2024-10-11T14:15:38Z

The implementation uses the opentelemetry-micrometer-1.5 library.

This allows us to create a Micrometer registry implemented with the OpenTelemetry SDK and APIs.

All telemetry data created with both Micrometer and OpenTelemetry will be processed by the same OpenTelemetry SDK instance from the quarkus-opentelemetry extension and will use it's configuration and exporters.

quarkus-micrometer extension configurations should also work, as verified on some tests.

Missing:

Native tests
Tests with exemplars using OTel Tracing.
Documentation updates
Decide what to do with the OTel metrics auto instrumentation
Move the code to the OTel extension? No.
Disable x.histogram in DistributionSummary
Silence io.ope.ins.mic.v1_.OpenTelemetryMeterRegistry] (main) A MeterFilter is being configured ...
Enable otel metrics and disable otel metrics instrumentation

quarkus-bot · 2024-10-11T14:15:59Z

/cc @ebullient (micrometer)

github-actions · 2024-10-11T14:46:14Z

🎊 PR Preview a09ea63 has been successfully built and deployed to https://quarkus-pr-main-43831-preview.surge.sh/version/main/guides/

Images of blog posts older than 3 months are not available.
Newsletters older than 3 months are not available.

brunobat · 2025-01-13T11:51:32Z

.../main/java/io/quarkus/micrometer/opentelemetry/deployment/MicrometerOtelBridgeProcessor.java

+//        // registered to this registry...
+//        // It's unavoidable because of how Quarkus startup works and users cannot do anything about it.
+//        // see: https://github.com/micrometer-metrics/micrometer/issues/4920#issuecomment-2298348202
+//        systemProperty.produce(


@radcortez I wonder why this didn't work here... I had to set these properties in the runtime application.properties

brunobat · 2025-01-16T11:28:30Z

CC @geoand . This one is ready to review.

geoand

Nice work!

I've added some questions and suggestions

docs/src/main/asciidoc/telemetry-micrometer-to-opentelemetry.adoc

geoand · 2025-01-17T12:34:49Z

...quarkus/micrometer/opentelemetry/deployment/MicrometerOTelBridgeConfigBuilderCustomizer.java

+        builder.withSources(
+                new PropertiesConfigSource(Map.of(
+                        "quarkus.otel.metrics.enabled", "true"),
+                        "quarkus-micrometer-opentelemetry", 1),


What is quarkus-micrometer-opentelemetry and why is the value 1?

extensions/micrometer-opentelemetry/deployment/src/test/resources/test-logging.properties

geoand · 2025-01-17T12:36:35Z

.../src/main/java/io/quarkus/micrometer/opentelemetry/runtime/MicrometerOtelBridgeRecorder.java

+            @Override
+            public Object apply(SyntheticCreationalContext<Object> context) {
+                if (otelRuntimeConfig.sdkDisabled()) {
+                    return null;


Are you sure returning null works? cc @mkouba

There is a disabled test and it seems to work.

Very odd. I'd like to hear from @mkouba whether this is expected to work

If you ask me whether it's possible to return null for a @Singleton synthetic bean then I'd say that it could work in theory because it's very similar to a producer method where IllegalProductException is only thrown when a non-Dependent (i.e. normal scoped because CDI happily ignores @Singleton because of @ApplicationScoped) producer returns null (that restriction is there for client proxies).

However, I've never tried that and I think that it's not a good idea in general because then a null could be injected. Which is not what most people expect in CDI and thus may result in unexpected NPEs.

It might make more sense to throw an InjectionException with a detailed error message. Furthermore, you can instruct the consumers to always inject Instance<BEAN> and check Instance#isResolvable() before use.

CC @manovotn @Ladicek

Thanks @mkouba!

However, I've never tried that and I think that it's not a good idea in general because then a null could be injected.

My thoughts exactly!

Furthermore, you can instruct the consumers to always inject Instance and check Instance#isResolvable() before use.

Right, I don't think returning null is ever necessary

@mkouba the point is that I want to return nothing if the sdk is disabled, I don't want to abort with an exception... I imagine I'll need to do that in the processor.

the point is that I want to return nothing if the sdk is disabled

But then the consumers of your bean would need to expect null injected... 🤷

Only the OTel SDK would expect this... and it's disabled. That's why it will not blow up on null.

Better try to handle this on the processor. Will check that.

.../src/main/java/io/quarkus/micrometer/opentelemetry/runtime/MicrometerOtelBridgeRecorder.java

quarkus-bot · 2025-01-17T17:38:41Z

Status for workflow `Quarkus Documentation CI`

This is the status report for running Quarkus Documentation CI on commit 725136a.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

Warning

There are other workflow runs running, you probably need to wait for their status before merging.

quarkus-bot · 2025-01-17T22:29:41Z

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit 725136a.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.

Flaky tests - Develocity

⚙️ JVM Tests - JDK 17

📦 extensions/micrometer-opentelemetry/deployment

✖ io.quarkus.micrometer.opentelemetry.deployment.compatibility.HttpCompatibilityTest.testHttpTimerToHistogramCompatibility - History

Expecting actual: [ImmutableHistogramPointData{getStartEpochNanos=1737139505325063703, getEpochNanos=1737139506429270821, getAttributes={clientName="localhost", method="GET", outcome="SUCCESS", status="200", uri="/pong/{message}"}, getSum=64.44346999999999, getCount=3, hasMin=true, getMin=4.157228, hasMax=true, getMax=52.968073, getBoundaries=[], getCounts=[3], getExemplars=[ImmutableDoubleExemplarData{filteredAttributes={}, epochNanos=1737139506391000000, spanContext=ImmutableSpanContext{traceId=7076a2a29fb65e82885afe524acc1c89, spanId=6393da49e1d36818, traceFlags=01, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=true}, value=7.318169}]}, ImmutableHistogramPointData{getStartEpochNanos=1737139505325063703, getEpochNanos=1737139506429270821, getAttributes={clientName="localhost", method="GET", outcome="CLIENT_ERROR", status="400", uri="/pong/{message}"}, getSum=4.827757, getCount=1, hasMin=true, getMin=4.827757, hasMax=true, getMax=4.827757, getBoundaries=[]... - java.lang.AssertionError

java.lang.AssertionError: 

Expecting actual:
  [ImmutableHistogramPointData{getStartEpochNanos=1737139505325063703, getEpochNanos=1737139506429270821, getAttributes={clientName="localhost", method="GET", outcome="SUCCESS", status="200", uri="/pong/{message}"}, getSum=64.44346999999999, getCount=3, hasMin=true, getMin=4.157228, hasMax=true, getMax=52.968073, getBoundaries=[], getCounts=[3], getExemplars=[ImmutableDoubleExemplarData{filteredAttributes={}, epochNanos=1737139506391000000, spanContext=ImmutableSpanContext{traceId=7076a2a29fb65e82885afe524acc1c89, spanId=6393da49e1d36818, traceFlags=01, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=true}, value=7.318169}]},
    ImmutableHistogramPointData{getStartEpochNanos=1737139505325063703, getEpochNanos=1737139506429270821, getAttributes={clientName="localhost", method="GET", outcome="CLIENT_ERROR", status="400", uri="/pong/{message}"}, getSum=4.827757, getCount=1, hasMin=true, getMin=4.827757, hasMax=true, getMax...

📦 integration-tests/grpc-hibernate

✖ com.example.grpc.hibernate.BlockingRawTest.shouldAdd - History

Condition with Lambda expression in com.example.grpc.hibernate.BlockingRawTestBase was not fulfilled within 30 seconds. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: Condition with Lambda expression in com.example.grpc.hibernate.BlockingRawTestBase was not fulfilled within 30 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:975)
	at com.example.grpc.hibernate.BlockingRawTestBase.shouldAdd(BlockingRawTestBase.java:59)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)

⚙️ JVM Tests - JDK 17 Windows

📦 extensions/micrometer-opentelemetry/deployment

✖ io.quarkus.micrometer.opentelemetry.deployment.compatibility.MicrometerTimedInterceptorTest.testTimeMethod - History

Stream has no elements - java.lang.IllegalArgumentException

java.lang.IllegalArgumentException: Stream has no elements
	at io.quarkus.micrometer.opentelemetry.deployment.common.MetricDataFilter.lambda$lastReading$2(MetricDataFilter.java:213)
	at java.base/java.util.Optional.orElseThrow(Optional.java:403)
	at io.quarkus.micrometer.opentelemetry.deployment.common.MetricDataFilter.lastReading(MetricDataFilter.java:213)
	at io.quarkus.micrometer.opentelemetry.deployment.common.MetricDataFilter.lastReadingDataPoint(MetricDataFilter.java:231)
	at io.quarkus.micrometer.opentelemetry.deployment.compatibility.MicrometerTimedInterceptorTest.testTimeMethod(MicrometerTimedInterceptorTest.java:77)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at io.quarkus.test.QuarkusUnitTest.runExtensionMethod(QuarkusUnitTest.java:513)

⚙️ Gradle Tests - JDK 17

📦 integration-tests/gradle

✖ io.quarkus.gradle.builder.QuarkusModelBuilderTest.shouldLoadSimpleModuleDevModel - History

Could not run build action using connection to Gradle distribution 'https://services.gradle.org/distributions/gradle-8.12-bin.zip'. - org.gradle.tooling.BuildException

org.gradle.tooling.BuildException: Could not run build action using connection to Gradle distribution 'https://services.gradle.org/distributions/gradle-8.12-bin.zip'.
	at app//org.gradle.tooling.internal.consumer.ConnectionExceptionTransformer.transform(ConnectionExceptionTransformer.java:69)
	at app//org.gradle.tooling.internal.consumer.ResultHandlerAdapter.onFailure(ResultHandlerAdapter.java:42)
	at app//org.gradle.tooling.internal.consumer.async.DefaultAsyncConsumerActionExecutor$1$1.run(DefaultAsyncConsumerActionExecutor.java:68)
	at app//org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at app//org.gradle.internal.concurrent.AbstractManagedExecutor$1.run(AbstractManagedExecutor.java:48)
	at java.base@17.0.13/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base@17.0.13/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)

✖ io.quarkus.gradle.builder.QuarkusModelBuilderTest.shouldLoadSimpleModuleTestModel - History

Could not run build action using connection to Gradle distribution 'https://services.gradle.org/distributions/gradle-8.12-bin.zip'. - org.gradle.tooling.BuildException

org.gradle.tooling.BuildException: Could not run build action using connection to Gradle distribution 'https://services.gradle.org/distributions/gradle-8.12-bin.zip'.
	at app//org.gradle.tooling.internal.consumer.ConnectionExceptionTransformer.transform(ConnectionExceptionTransformer.java:69)
	at app//org.gradle.tooling.internal.consumer.ResultHandlerAdapter.onFailure(ResultHandlerAdapter.java:42)
	at app//org.gradle.tooling.internal.consumer.async.DefaultAsyncConsumerActionExecutor$1$1.run(DefaultAsyncConsumerActionExecutor.java:68)
	at app//org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at app//org.gradle.internal.concurrent.AbstractManagedExecutor$1.run(AbstractManagedExecutor.java:48)
	at java.base@17.0.13/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base@17.0.13/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)

geoand · 2025-01-18T10:34:54Z

The CI failure seems related

quarkus-bot bot added the area/metrics label Oct 11, 2024

brunobat mentioned this pull request Oct 11, 2024

Micrometer to OpenTelemetry bridge in Quarkus #43621

Open

brunobat force-pushed the micrometer-to-otel-bridge branch from 7b7cc9f to acd0b25 Compare December 18, 2024 17:23

brunobat force-pushed the micrometer-to-otel-bridge branch from acd0b25 to 35f4d7a Compare January 2, 2025 17:41

quarkus-bot bot added the area/documentation label Jan 2, 2025

brunobat force-pushed the micrometer-to-otel-bridge branch 3 times, most recently from 73b8d37 to b7ae30d Compare January 13, 2025 11:13

quarkus-bot bot added area/infra-automation anything related to CI, bots, etc. that are used to automated our infrastructure area/tracing labels Jan 13, 2025

brunobat force-pushed the micrometer-to-otel-bridge branch 2 times, most recently from d384469 to 7375f46 Compare January 13, 2025 11:49

brunobat marked this pull request as ready for review January 13, 2025 11:49

brunobat requested review from alesj and radcortez January 13, 2025 11:49

brunobat commented Jan 13, 2025

View reviewed changes