57: Seeing clearly with OpenTelemetry

Introduction

David Phillips and Matt Stephenson join us about the support for OpenTelemetry and OpenMetrics in Trino and how it can be useful for troubleshooting.

Video

• Introduction: Episode 57: Seeing clearly with OpenTelemetry (0.0s)
• Releases: Trino 440, Trino 441 and other updates (75.0s)
• Guest introductions: David Phillips and Matt Stephenson (242.0s)
• File system lead and related API and features (5:48 File system lead and related API and featuress)
• Java 22: Upcoming usage with Trino (570.0s)
• Japanese edition of Trino The Definitive Guide: Props to Yuya Ebihara (666.0s)
• Trino Contributor Call (723.0s)
• OpenTelemetry and OpenMetrics introduction and concepts (747.0s)
• Motivation to add to Trino (932.0s)
• Why is tracing useful in Trino? (1043.0s)
• Use in development (1137.0s)
• Implementation details (1199.0s)
• Trace propagation from clients and elsewhere (1435.0s)
• Practical experience and demo (1664.0s)
• OpenMetrics (2274.0s)
• Prometheus sidecar, Airlift, Trino Gateway and more OpenMetrics (2404.0s)
• OpenMetrics vs OpenTelemetry metrics (2723.0s)
• JSON-format logging with TCP socket streaming (2933.0s)
• Learning resources (3015.0s)
• Trino Fest and Trino Contributor Congregation (3170.0s)
• Upcoming Trino Community Broadcast episodes (3295.0s)

Audio

Hosts

• Cole Bowden, Developer Advocate at Starburst
• Manfred Moser, Director of Trino Community Leadership at Starburst, (@simpligility)

Guests

• David Phillips, co-creator of Trino and CTO at Starburst
• Matt Stephenson, Senior Principal Software Engineer at Starburst

Trino releases

Trino 440

• New Snowflake connector
• Support for sub-queries inside UNNEST clauses
• Support for row filtering and column masking with Open Policy Agent
• Improved latency when filesystem caching is enabled in Delta and Iceberg connectors

Trino 441

• Remove the default legacy mode for hive.security

And there is a regression for Iceberg, so wait for 442 potentially. (Update: Trino 442 is released.)

Other Trino news

• Java 22 is coming to Trino
• David Phillips appointed dedicated file system lead
• Trino Contributor Call on the 21st of March
• Japenese edition of Trino: The Definitive Guide is out

OpenTelemetry

OpenTelemetry is a widely-used collection of APIs, SDKs, and tools that instrument, generate, collect, and export telemetry data such as metrics, logs, and traces to help you analyze application performance and behavior.

In our conversation with Matt and David we cover all the following aspects, and a few other topics:

• What is OpenTelemetry?
• Some basic concepts like logs, spans, traces
• How is this related to JMX and system data and other monitoring
• What is OpenMetrics? How is it related to Prometheus?
• What tools can you use with OpenTelemetry? Jaeger, Datadog, …
• Reasoning to add OpenTelemetry to Trino
• Implementation details
• Trino documentation with local example usage with Docker containers for Trino and Jaeger
• Practical experience
• Demo of real world usage with Starburst Galaxy and Datadog
• Bonus topic - JSON-format logging via TCP socket

Rounding out

Trino Fest 2024 and Trino Contributor Congregation are happening in June in Boston. Submit your speaker proposals now, and register for the free event as soon as you can, especially for live attendance.

Check out the upcoming Trino Community Broadcast episodes about Mitzu and jOOQ.

If you want to learn more about Trino, get the definitive guide from O’Reilly. You can get the free PDF from Starburst or buy the English, Polish, Chinese, or Japanese edition online.

Music for the show is from the Megaman 6 Game Play album by Krzysztof Slowikowski.