64: Control with Open Policy Agent OPA

Introduction

Sönke Liebau and Sebastian Bernauer from Stackable join us to chat about the Open Policy Agent plugin for Trino and their usage of it. We also talk about their trino-lb project, their Kubernetes operator for Trino, and lessons learned from using Trino on Kubernetes.

Video

• Introduction (0.0s)
• Release update - Trino 454 (121.0s)
• Guest and Stackable introduction (446.0s)
• Start of presentation - Trino and OPA at Stackable (681.0s)
• Stackable stack (855.0s)
• OPA plugin history and usage (926.0s)
• High level architecture and user question (1668.0s)
• Demo overview (2160.0s)
• Demo time (2370.0s)
• trino-lb (2820.0s)
• trino-lb, k8s, and operator (3371.0s)
• Lessons learned (3670.0s)
• Wrapping up Stackable presentation (4125.0s)
• Upcoming events (4192.0s)

Audio

Hosts

• Manfred Moser, Director of Trino Community Leadership at Starburst, (@simpligility)
• Cole Bowden, Developer Advocate at Starburst

Guests

• Sebastian Bernauer, Software Developer at Stackable
• Sönke Liebau, Co-Founder and CPO at Stackable

Releases and news

Trino 454

• Improve performance for queries that contain multiple aggregate functions, including DISTINCT.
• Add Kafka event listener plugin (yet to be documented).
• Add configuration for fetch size with JDBC-based connectors (yet to be documented).
• Add support for writing Deletion Vectors with the Delta Lake connector.
• Add new Resources tab in the web interface with data from the new light-weight query endpoint /v1/query?pruned=true.
• Add new Preview Web UI (help us test and develop!).
• Add S3 security mapping for the native S3 filesystem.

As usual, numerous performance improvements, bug fixes, and other features have been added as well.

Stackable, OPA, and more

We chat with Sönke and Sebastian about the following agenda topics:

• What is Stackable?
• Open Policy Agent (OPA) authorization plugin
  • History
  • Recent development
  • Compatibility layer to Trino’s file-based access control
  • Quick demo on row filtering and column masking
• Auto-scaling Trino clusters using trino-lb
  • Differences between Trino Gateway and trino-lb

Other aspects we discuss include the following:

• Performance considerations
• Aspects of Trino on Kubernetes such as graceful shutdown, PodDisruptionBudgets, and anti-affinity
• Plans for next steps

Other resources

• Presentation slide deck
• Video for Trino OPA Authorizer - Stackable and Bloomberg at Trino Summit 2023 presented by Sönke from Stackable and Pablo Arteaga from Bloomberg
• Source code repo for compatibility layer between Trino classic file-based access control JSON and OPA/Trino
• Longer demo video for row filtering and column masking

Rounding out

• Trino Summit 2024 is coming on the 11th and 12th of December, and registration, call for speakers, and sponsorship opportunities are open.
• Next Trino Community Broadcast 65 about the new Exasol connector.

If you want to learn more about Trino, check out the definitive guide from O’Reilly. You can get the free PDF from Starburst or buy the English, Polish, Chinese, or Japanese edition.