Introduction
Manfred Moser and Cole Bowden are joined by Viktor Kessler and Christian Thiel from Vakamo who introduce us to their new metastore solution Lakekeeper. It is an Iceberg REST catalog-compatible lakehouse metastore with big plans for additional features and support for Trino. We dive into the details and have a look at a demo.
Video
- Introduction with Manfred and Cole (0.0s)
- Trino 472 (48.0s)
- Introduction with Manfred and Cole (0.0s)
- Trino 472 (71.0s)
- Trino Gateway 15 (261.0s)
- Java 24 (371.0s)
- Introducing Viktor, Christian, Vakamo, and Lakekeeper (463.0s)
- History and background for metastores/catalogs (906.0s)
- Presentation (1792.0s)
- Demo (2677.0s)
- Questions and mascot discussion (3404.0s)
- Closing out (3527.0s)
Hosts
- Manfred Moser, Dev Rel Engineer at Chainguard @simpligility
- Cole Bowden, Developer Advocate at Firebolt
Guests
- Viktor Kessler, Co-founder at Vakamo
- Christian Thiel, Co-founder at Vakamo
Releases
Following are some highlights of the recent releases:
- Color the server console output for improved readability.
- Fix initialization failure for the DuckDB connector on Docker container.
- Add support for the
rowtype and generate empty values forarray,map, andjsontypes in the Faker connector. - Add the
$partitionhidden column in the Iceberg connector.
As always, numerous performance improvements, bug fixes, and other features were added as well.
- Pop up messages in UI
- Consistent use of
config.yaml - Use of OpenMetrics data from Trino clusters
- Fix query errors when adhoc routing group has no healthy backends.
Introducing Viktor and Christian
We talk with Viktor and Christian about there experience in software engineering and the world of big data, and what led them to start Vakamo together.
Metastores and catalogs
We talk about data lakes, data lakehouses, object storage and the role of metadata. Details we cover include the Hive Metatstore Service, the Thrift protocol, Amazon Glue, and the new wave of catalogs. Specifically we also talk about Apache Iceberg and the Iceberg REST catalog standard as a basis for Lakekeeper, and then learn all the details about Lakekeeper.
Demo time
In their demo Viktor and Christian show a multi-user Trino cluster secured by OAuth 2, Open Policy Agent, and Lakekeeper.
Resources
- Lakekeeper
- Lakekeeper documentation
- Lakekeeper source
- Example project with Trino and OPA
- Iceberg connector documentation
- Iceberg REST catalog documentation
Rounding out
Join us for upcoming events and let us know if you want to a guest:
- Trino Community Broadcast 73: Wrapping Trino packages with a bow