74: Insights from a Norse god

Introduction

Karsten Jeschkies from Grafana Labs introduces us to the powerful, multi-tenant log management system Loki, some of the use cases, and his work on the Trino Loki connector.

Video

• Introduction with Manfred and Cole (0.0s)
• Trino 475 (94.0s)
• Trino 476 (288.0s)
• Other releases (485.0s)
• Introducing Karsten and Grafana Labs (529.0s)
• Loki presentation (786.0s)
• Trino Loki connector (1512.0s)
• Demo time (1766.0s)
• Next steps and more info (2468.0s)
• Where to learn more and contact Karsten (2758.0s)
• Final words and rounding out with next events (2901.0s)

Hosts

• Manfred Moser, Dev Rel Engineer at Chainguard @simpligility
• Cole Bowden, Developer Advocate at Firebolt

Guest

• Karsten Jeschkies from Grafana Labs

Releases

Following are some highlights of the recent releases:

Trino 475

• Add support for the CORRESPONDING clause in set operations.
• Add support for the AUTO grouping set that includes all non-aggregated columns in the SELECT clause.
• Allow cross-region data retrieval when using the S3 native filesystem.
• Add support for all storage classes when using the S3 native filesystem for writes.
• Numerous improvements on Iceberg, Hive, and Delta Lake connectors.
• SPI - Remove the LazyBlock class.

Trino 476

Another big release with lots of changes:

• Require JDK 24 as runtime.
• Add support for comparing values of geometry type.
• Remove Example HTTP connector from binaries.
• New required JVM config for BigQuery and Snowflake connectors.
• Fix regression with graceful shutdown from Trino 474.
• Improve performance of selective joins for federated queries for nearly all connectors.
• Add columns to the $all_manifests metadata tables for Iceberg tables.
• Add support for user-assigned managed identity authentication for AzureFS for object storage connectors.
• Add support for the FOR TIMESTAMP AS OF clause in Delta Lake connector.

As always, numerous performance improvements, bug fixes, and other features were added as well.

Other releases and announcements:

• Trino Gateway 16 still delayed, but Trino Gateway Helm chart 1.15.2
• Trino Helm chart with 475 -> 1.39.1
• Trino Python client 0.334.0

Introducing Karsten and Grafana Labs

Karsten Jeschkies is an experienced software engineer:

• 2013 - 2016 Engineer at the Core Machine Learning team at Amazon
• 2016 - 2020 Mesosphere and D2IQ, maintainer of Marathon, a container orchestrator for Mesos
• 2020 - now Maintainer of Loki for two years and now Cloud Provider observability engineer at Grafana Labs

Grafana Labs is the home of the well-known Grafana for visualizations and dashboard and other powerful products such as Grafana Tempo, Grafana Mimir, and Grafana Loki. Grafana is also involved in well-known projects such as Prometheus and OpenTelemetry.

Log management with Loki

Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It helps you to drill into petabytes of logging data.

Analytics with Trino

Karsten tells about the motivation to create a Trino connector, how the two tools work together, what features are there, and what his plans are for the future.

Resources

• Presentation
• Demo source code
• Grafana Loki website
• Loki source code repo
• Loki connector documentation

Rounding out

Quick update on where to see Cole or Manfred next, and then join us for the upcoming Trino events:

• Trino Contributor Call - May skipped, June edition to be determined
• Trino Community Broadcast: Visualizing with Logi Symphony and ODBC
• Trino Community Broadcast: One tool to proxy them all (aws-proxy)

Let us know if you want to be a guest in a future broadcast.