Episodes

68: Year of the Snake - Python UDFs

Manfred and Cole are joined by David Phillips to talk about the new support of user-defined functions written in Python. We discuss motivation, development history, dive into implementation details, and explore some examples.

• Introduction with Cole, Manfred, and David: (0.0s)
• Releases 465-468: (70.0s)
• Trino 469 progress and Trino Gateway: (647.0s)
• Trino Summit recap: (780.0s)
• Python UDF history, architecture, and details: (1005.0s)

67: Extra speed with Exasol and Trino

Manfred and Cole are joined by Thomas Bestfleisch from the Exasol team to talk about Exasol, the new Trino Exasol connector, and some related use cases.

• Introduction: (0.0s)
• Release 461-464: (87.0s)
• Trino 463 including HTTP/2: (144.0s)
• Trino 464 with Java 23 and connector news: (415.0s)
• Other news: (685.0s)

66: Chat with Trino and Wren AI

Manfred is joined by Wren AI team members and contributors to talk about the new AI-powered, text to SQL tool and its great support for Trino.

• Introduction: (0.0s)
• Trino releases 458, 459, and 460: (53.0s)
• Contributor guideline updates: (420.0s)
• Shout out to maintainers: (494.0s)
• Guests introductions: (570.0s)

65: Performance boosts

Manfred and Cole dive into the details of some changes in recent and upcoming Trino releases. They cover file compression and decompression improvements, the updated file system support, and other performance boosters.

• Introduction: (0.0s)
• Releases 455, 456, and 457: (107.0s)
• JDBC driver JAR: (404.0s)
• Concurrent write improvement with Delta Lake: (473.0s)
• Adaptive plan optimization and join ordering: (604.0s)

64: Control with Open Policy Agent OPA

Sönke Liebau and Sebastian Bernauer from Stackable join us to chat about the Open Policy Agent plugin for Trino and their usage of it. We also talk about their trino-lb project, their Kubernetes operator for Trino, and lessons learned from using Trino on Kubernetes.

• Introduction: (0.0s)
• Release update - Trino 454: (121.0s)
• Guest and Stackable introduction: (446.0s)
• Start of presentation - Trino and OPA at Stackable: (681.0s)
• Stackable stack: (855.0s)

63: Querying with JS

Emily Sunaryo from Starburst joins us to talk about her journey to learn about Trino and develop a web application that queries Trino with the trino-js-client.

• Introduction: (0.0s)
• Trino 452 and 453: (150.0s)
• Trino Gateway 10: and Trino Contributor Call (515.0s)
• Guest Emily Sunaryo: (700.0s)
• Data application with JavaScript and Trino: (795.0s)

62: A lakehouse that simply works at Prezi

Vincenzo Cassaro from Prezi presents the usage of Trino, in the form of Starburst Galaxy, in the data platform for Prezi.

• Introduction: (0.0s)
• Trino 451: (106.0s)
• Trino Japan meetup and Star Poon: (390.0s)
• Other announcements: (481.0s)
• Guest Vincenzo Cassaro: (600.0s)

61: Trino powers business intelligence

Patrick Pichler from Creative Data joins us to talk about his open source driver for the interactive data visualization software product suite Power BI from Microsoft.

• Introduction and Trino Fest recap: (0.0s)
• Trino releases 449 and 450: (122.0s)
• Guest Patrick Pichler: (399.0s)
• Creative Data overview: (552.0s)
• Power BI overview: (689.0s)

60: Trino calling AI

Isa Inalcik from BestSecret chats with us about his exploration to create a Trino connector that interacts with generative AI tools.

• Welcome from Manfred and Cole: (0.0s)
• Trino releases: Trino 446, 447, and 448 (104.0s)
• Trino Gateway 8 and 9: (320.0s)
• Jan Was a subproject maintainer: (459.0s)
• Cole's recaptures Iceberg Summit: (519.0s)

59: Querying Trino with Java and jOOQ

Lukas Eder from the jOOQ Object Oriented Querying project joins us to talk about Trino support for jOOQ, Java, and SQL.

• Introduction and Manfred's Seattle trip: (0.0s)
• Trino 445: (106.0s)
• Our guest - Lukas Eder: Java expert, Java Champion and creator of jOOQ (206.0s)
• High-level intro to jOOQ: (452.0s)
• data geekery history and more jOOQ architecture: (572.0s)

58: Understanding your users with Trino and Mitzu

István Mészáros from Mitzu joins us to talk about using Mitzu with Trino. Mitzu is a warehouse-native product analytics platform that revolutionizes how companies leverage their product usage data in the data lake.

• Introduction: Episode 58: Understanding your users with Trino and Mitzu (0.0s)
• Releases: Trino 442, 443, and 444 (76.0s)
• Trino Gateway: Version 7 and Starburst Trino Champion Peng Wei (323.0s)
• Starburst Trino Tamas Cservenak: For all the Apache Maven help (369.0s)
• Introduction of István Mészáros: Founder and CEO of Mitzu (492.0s)

57: Seeing clearly with OpenTelemetry

David Phillips and Matt Stephenson join us about the support for OpenTelemetry and OpenMetrics in Trino and how it can be useful for troubleshooting.

• Introduction: Episode 57: Seeing clearly with OpenTelemetry (0.0s)
• Releases: Trino 440, Trino 441 and other updates (75.0s)
• Guest introductions: David Phillips and Matt Stephenson (242.0s)
• File system lead and related API and features: (5:48 File system lead and related API and featuress)
• Java 22: Upcoming usage with Trino (570.0s)

56: The vast possibilities of VAST and Trino

Colleen Tartow and Roman Zeyde from VAST join us to introduce the VAST data platform. We talk about how it includes and uses Trino and the Trino VAST database connector.

• Introduction: Episode 56: The vast possibilities of VAST and Trino (0s)
• Releases: Trino 439 and Trino Gateway 6 (112.0s)
• Guest introductions: Colleen Tartow and Roman Zeyde (285.0s)
• VAST database and VAST data platform: An introduction and overview (415.0s)
• Data lake, on-prem, hardware, ...: (534.0s)

55: Commander Bun Bun peeks at Peaka

Mustafa Sakalsiz and Ali Tekin from Peaka join to demo and talk about their Trino-based data platform.

• Intro: Episode 55: Commander Bun Bun peeks at Peaka (0s)
• Trino releases: 437-438 (96s)
• Concept of the episode: Introducing Peaka (252s)
• An overview of Peaka: What it does and how it works (487s)
• Engineering of Peaka: Architecture and under the hood (962s)

54: Trino 2023 wrapped

• Intro: Episode 54: Trino 2023 wrapped (0s)
• Trino releases 434-436: including json_table, OpenSearch connector, Java 21 and more (81s)
• Trino 2023 wrapped: Statistics (623s)
• Events: Trino Fest and Trino Summit (789s)
• More events: Seoul, Tokyo, Bengaluru, ... (1049s)

53: Understanding your data with Coginiti and Trino

• Intro: Episode 53: Understanding your data with Coginiti and Trino (0s)
• Trino releases: 431-433 (140s)
• Feature: Dynamic catalogs (270s)
• Competition: Share your best SQL routine (380s)
• Java 21: Upgrading Trino (465s)

52: Commander Bun Bun takes a bite out of Yugabyte

• Intro: Episode 52: Commander Bun Bun takes a bite out of Yugabyte (0s)
• Trino releases: 428-430 (108s)
• Concept of the episode: Trino, JDBC, and PostgreSQL (390s)
• Concept of the episode: Introducing YugabyteDB (848s)
• Demo of the episode: Using PostgreSQL with Trino (1293s)

51: Trino cools off with PopSQL

• Intro: Episode 51: Trino cools off with PopSQL (0s)
• Trino releases: 423-427 (108s)
• Concept of the episode: Introducing PopSQL (480s)
• Concept of the episode: Building a new Node adapter for Trino (682s)
• Demo of the episode: Demoing PopSQL (1464s)

50: Celebrating 50 episodes of Trino Community Broadcast

• Intro: Episode 50! (0s)
• Trino releases: 421-422 (114s)
• Introducing our guests: Brian Olsen and Dain Sundstrom (240s)
• Storytime with Brian: The incoming Snowflake connector (678s)
• Questions for Dain: Setting an open source project up for success (889s)

49: Trino, Ibis, and wrangling Python in the SQL ecosystem

• Intro: Trino, Ibis, and wrangling Python in the SQL ecosystem (0s)
• Trino releases: 419-420 (83s)
• Concept of the episode: Introducing Philip Cloud and Ibis (309s)
• Concept of the episode: Why use Ibis? (769s)
• Concept of the episode: Scale vs. convenience (1640s)

48: What is Trino?

• Trino releases: 417-418 (120s)
• Question of the episode: What is Trino? (337s)
• Question of the episode: How do you run Trino? (1667s)
• Demo of the episode: Trino in the Docker container (1800s)
• Demo of the episode: The Trino tarball (2747s)

47: Meet the new Trino maintainers

• Trino releases: 411-416 (105s)
• Concept of the episode: Meet the new maintainers! (443s)
• Concept of the episode: What is a maintainer? (550s)
• Concept of the episode: The path to becoming a maintainer (1061s)
• PR of the episode: Improve TopN row number / rank performance (1973s)

46: Trino heats up with Ignite

• Trino releases: 408-410 (122s)
• Concept of the episode: Introducing the Ignite connector to Trino (330s)
• Concept of the episode: What is Ignite? (450s)
• Concept of the episode: Contributing the Ignite connector (610s)
• PR of the episode: Add support for `migrate` procedure in Iceberg (3138s)

45: Trino swimming with the DolphinScheduler

• Trino Release: 407 (802s)
• Concept of the episode: What is workflow orchestration? (1272s)
• Concept of the episode: Why do we need a workflow orchestration tool for building a data lake? (1867s)
• Concept of the episode: What is Apache DolphinScheduler? (2255s)
• Concept of the episode: Does DolphinScheduler have any computing engine or storage layer? (3191s)

44: Seeing clearly with Metabase

• Introduction: Manfred and guests (60s)
• Recap of Trino in 2022: A summary of highlights from a recent blog post (240s)
• Trino Summit recap: Our fun video from Brian (320s)
• Release update: Trino releases from 2022 and the new 405 and 406 releases (530s)
• Concept: What is Metabase (660s)

43: Trino saves trips with Alluxio

• Release update: 404 Not Found (639s)
• Concept of the episode: Data caching and orchestration (998s)
• Concept of the episode: Trino without caching (1573s)
• Concept of the episode: What is Alluxio? (1785s)
• Concept of the episode: How does Alluxio work with table formats other than Hive? (2478s)

42: Trino Summit 2022 recap

• Trino Summit intro: 1:46 (106s)
• Why the Pokémon theme?: 3:56 (236s)
• Overview of Trino Summit and what stood out: 10:58 (658s)
• Bringing Trino to the masses: 33:57 (2037s)
• Trino Contributor Congregation recap: 43:52 (2632s)

41: Trino puts on its Hudi

• Releases: 396 to 401 (883s)
• Concept of the episode: Intro to Hudi and the Hudi connector (1349s)
• Concept of the episode: Merge on read and copy on write tables (1349s)
• Concept of the episode: Hudi metadata table (2364s)
• Concept of the episode: Hudi data layout (2799s)

40: Trino's cold as Iceberg!

• Releases: 394 to 395 (387s)
• Concept of the episode: What is Iceberg? (687s)
• Concept: Why Iceberg over other formats? (1010s)
• Concept: Metadata catalogs (2140s)
• Concept: Branching, tagging, and auditing, oh my! (2634s)

39: Raft floats on Trino to federate silos

• Releases 392 and 393: Details about features and fixes in the new releases (314s)
• Concept of the episode: Trino at Raft (728s)
• Concept of the episode: Software factory (801s)
• Concept of the episode: Standards and anatomy of a stack (984s)
• Concept of the episode: Data Fabric at Raft (1138s)

38: Trino tacks on polymorphic table functions

• Trino is turning 10: Space Quest League and t-shirt giveaways (55s)
• Releases 387 to 391: Details about features and fixes in the new releases (245s)
• Concept of the episode: Polymorphic table functions (550s)
• Demo of the episode: The many ways you can use polymorphic table functions (1640s)
• Pull request of the episode: PR 12325: Support query pass-through for JDBC-based connectors (2840s)

37: Trino powers up the community support

• Releases 382 to 386: Details about features and fixes in the new releases (577s)
• Concept of the episode: How to strengthen the Trino community (907s)
• Concept of the episode: Pull request releases (1833s)
• Concept of the episode: Impact of community and developer experience (1987s)
• Concept of the episode: Community metrics for better decision making (2640s)

36: Trino plans to jump to Java 17

• Releases 379 to 381: Details about features and fixes in the new releases (497s)
• Question of the episode: Will Trino be making a vectorized C++ version of Trino workers? (1162s)
• Concept of the episode: Java 17 and rearchitecting Trino (2199s)
• Java 17 updates: Performance (2410s)
• Java 17 updates: Garbage collectors (2805s)

35: Packaging and modernizing Trino

• Releases 375 to 378: Details about features and fixes in the new releases (262s)
• Concept of the episode: Packaging Trino (1288s)
• Additional topic of the episode: Modernizing Trino with Java 17 (2809s)
• Pull requests of the episode: Worker stats in the Web UI (3325s)
• Demo of the episode: Tarball installation and new Web UI feature (3958s)

34: A big delta for Trino

• Releases 372, 373, and 374: Details about features and fixes in the new releases. (125s)
• Project Tardigrade update: Learn about our progress for fault-tolerant query processing. (561s)
• Concept of the episode: A new connector for Delta Lake object storage. (1117s)
• Pull requests of the episode: Add Delta Lake connector and documentation. (1570s)
• Demo of the episode: Delta Lake connector in action. (1754s)

33: Trino becomes highly available for high demand

• Concept of the month: High availability with Trino (1223s)
• PR of the month: PR 8956 Add support for external db for schema management in MongoDB connector (3849s)
• Bonus PR of the month: PR 8202 Metadata for alias in elasticsearch connector only uses the first mapping (4515s)
• Demo of the month: Trino Fiddle: A tool for easy online testing and sharing of Trino SQL problems and their solutions (5528s)
• Question of the month: Does Trino support CarbonData? (5889s)

32: Trino Tardigrade: Try, try, and never die

• Concept of the month: Introducing Project Tardigrade (1226s)
• Concept of the month: Why ETL in Trino? (1377s)
• Concept of the month: Why are people reluctant to do their ETL in Trino? (2128s)
• Concept of the month: What are the limitations of the current architecture? (2605s)
• Concept of the month: Trino engine improvements with Project Tardigrade (3179s)

31: Trinites II: Trino on AWS Kubernetes Service

• Concept of the month: ReplicaSets, Deployments, and Services (1658s)
• Demo of the month: Deploy Trino k8s to Amazon EKS (4891s)
• PR of the month: PR 8921: Support TRUNCATE TABLE statement (5681s)
• Question of the month: How do I run `system.sync_partition_metadata` with different catalogs? (5901s)

30: Trino and dbt, a hot data mesh

• Concept of the week: Trino and dbt, a hot data mesh (935s)
• PR of the week: Partitioned table tests and fixed PR 9757 (5055s)
• Question of the week: What’s the difference between location and external_location? (5298s)

29: What is Trino and the Hive connector

• Concept of the week: What is Trino? (1023s)
• PR of the week: PR 8821 Add HTTP/S query event logger (3515s)
• Question of the week: Does the Hive connector depend on the Hive runtime? (3763s)

28: Autoscaling streaming ingestion to Trino with Pravega

• Concept of the week: Event stream abstractions and Pravega (915s)
• Demo of the week: Querying Pravega from Trino (4260s)
• PR of the week: Pravega presto-connector PR 49 (4851s)
• Question of the week: What is the point of Trino Forum, and what is the relationship to Trino Slack? (5167s)

27: Trino gits to wade in the data LakeFS

• Concept of the week: LakeFS and Git on Object Storage (546s)
• Demo of the week: Running Trino on LakeFS (2445s)
• PR of the week: PR 8762 Add query error info to cluster overview page in web UI (4271s)
• Question of the week: Why are deletes so limited in Trino? (4454s)

26: Trino discovers data catalogs with Amundsen

• Concept of the week: Data discovery and Amundsen (724s)
• Concept of the week: Amundsen architecture (1450s)
• Concept of the week: Amundsen as a subcomponent to data mesh (2720s)
• PR of the week: Index Trino views (3606s)
• Question of the week: Can I add a UDF without restarting Trino? (4374s)

25: Trino going through changes

• Concept of the week: Change Data Capture (934s)
• Concept of the week: Debezium (1652s)
• Concept of the week: Debezium + Trino at Zomato (2747s)
• PR of the week: PR 4140 Implement aggregation pushdown in Pinot (3865s)
• Question of the week: Is there an array function that flattens a row into three rows? (4265s)

24: Trinetes I: Trino on Kubernetes

• Concept of the week: K8s architecture: Containers, Pods, and kubelets (867s)
• PR of the week: PR 11 Merge contributor version of k8s charts with the community version (3320s)
• Demo: Running the Trino charts with kubectl (3462s)

23: Trino looking for patterns

• Concept of the week: Row pattern matching and MATCH_RECOGNIZE (866s)
• PR of the week: PR 8348 Document row pattern recognition in window (3136s)
• Demo: Showing MATCH_RECOGNIZE functionality by example (3433s)
• Question of the week: How do you tag a list of rows with custom periodic rules? (4371s)

22: TrinkedIn: LinkedIn gets a Trino promotion

• Concept of the week: Trino usage at LinkedIn (955s)
• Concept of the week: Trino hardware and operational scale (1403s)
• Concept of the week: Challenges operating at scale (2649s)
• Concept of the week: Open source at LinkedIn (2916s)
• Concept of the week: PrestoSQL to Trino upgrade challenges (3491s)

21: Trino + dbt = a match made in SQL heaven?

• Question of the week: Can dbt connect to different databases in the same project? (1098s)
• Concept of the week: Trino + dbt = a match made in SQL heaven? (1288s)
• Demo: Querying Trino from a dbt project (2841s)
• PR of the week: PR 8283 Externalised destination table cache expiry duration for BigQuery Connector (4873s)

20: Trino for the Trinewbie

• Concept of the week: Trino for the Trinewbie (1152s)
• PR of the week: PR 8135 Set default time zone for the current session (3816s)
• Demo: Contributing to Trino (4309s)
• Question of the week: How do I remove nulls from an array in Trino? (5064s)

19: Data Ingestion to Iceberg and Trino

• Concept of the week: Ingesting into Iceberg with Pulsar and Flink at BlueCat (1050s)
• PR of the week: PR 1905 Add format_number function (3715s)
• Demo: Showing the format_number functionality (3878s)
• Question of the week: How do I search nested objects in Elasticsearch from Trino? (4134s)

18: Trino enjoying the view

• Concept of the week: Trino Views, Hive Views and Materialized Views (297s)
• PR of the week: PR 4832 Add Iceberg support for materialized views (3544s)
• Demo: Showing the different views in Trino (3685s)
• Question of the week: Are JDBC drivers backwards compatible with older Trino versions? (4862s)

17: Trino connector resurfaces API calls

• Concept of the week: Resurface and the Resurface connector (538s)
• PR of the week: PR 4022 Add Soundex function (4097s)
• Demo: Using the soundex function (4227s)
• Question of the week: Question of the week: How to export query results into a file (e.g. CTAS, but into a single file)? (4726s)

16: Make data fluid with Apache Druid

• Concept of the week: Apache Druid and realtime analytics (891s)
• PR of the week: PR 3522 Add Druid connector (2015s)
• Demo: Using the Druid Web UI to create an ingestion spec querying via Trino (3689s)
• Question of the week: Does the Druid connector? (4220s)

15: Iceberg right ahead!

• Concept of the week: Apache Iceberg and the Iceberg spec (812s)
• PR of the week: PR 7233 Fix queries on tables without snapshot id (4064s)
• Demo: Creating tables with Iceberg and reading the data in Trino (4239s)
• Question of the week: What do I do to restart the test pipeline if it fails on me? (5015s)

14: Iceberg: March of the Trinos

• Concept of the week: Apache Iceberg and the table format (1034s)
• PR of the week: PR 1067 Add Iceberg connector (3562s)
• Demo: Creating tables with Iceberg and reading the data in Trino (3844s)
• Question of the week: Why does Trino still depend on the Hive metastore if metadata for Iceberg saves to the filesystem? (4719s)

13: Trino takes a sip of Pinot!

• Concept of the week: Data cubes and Apache Pinot (840s)
• Interview: Apache Pinot (840s)
• PR of the week: PR 2028 Add Pinot connector (3231s)
• Question of the week: Why does my passthrough query not work in the Pinot connector? (3983s)
• Demo: Pinot batch insertion and query using Trino Pinot connector (4338s)

12: Trino gets super visual with Apache Superset!

• Concept of the week: Trino clients, Python, and Apache Superset (463s)
• PR of the week: Superset PR 13105 feat: first step native support Trino (1076s)
• Demo: Superset querying Trino to create visualization dashboard (2523s)
• Question of the week: How do I use the Trino REST api? (2523s)

11: Dynamic filtering and dynamic partition pruning

• Concept of the week: Dynamic filtering and Recap (463s)
• PR of the week: PR 1072 Implement dynamic partition pruning (1076s)
• PR Demo: PR Demo: PR 1072 Implement dynamic partition pruning (2523s)

10: Naming the bunny!

• Where did the bunny come from?: Link (538s)
• Bunny names in the running...: Link (1018s)
• And our bunny's name is..: Link (1215s)
• Release 352 sneak peek: Link (1426s)
• Community status update: Link (2094s)

9: Distributed hash-joins, and how to migrate to Trino

• Question of the week: How do I migrate to Trino from PrestoSQL? (715s)
• Concept of the week: Distributed hash-join (963s)
• Quick Discussion: Contributing Documents and Testimonials (3485s)

8: Trino: A ludicrously fast query engine: past, present, and future

• Martin Traverso Introduction: Link (105s)
• Dain Sundstrom Introduction: Link (270s)
• David Phillips Introduction: Link (506s)
• Eric Hwang Introduction: Link (744s)
• Past: Presto name, Facebook projects, and open source: Link (1010s)

7: Cost Based Optimizer, Decorrelate subqueries, and does Presto make my RDBMS faster?

• Concept of the week: Cost Based Optimizer (1008s)
• PR of the week: PR 1415 Decorrelate subqueries with Limit or TopN (2589s)
• PR Demo: EXPLAIN Decorrelate subqueries with Limit or TopN (3216s)
• Question of the week: Will running Presto on my relational database make processing faster? (3744s)

6: Query Planning, Remove duplicate predicates, and Memory settings

• Concept of the week: Query Planning (1231s)
• PR of the week: PR 730 Remove duplicate predicates (1795s)
• PR Demo: EXPLAIN LOGICAL and DISTRIBUTED demo (2027s)
• Question of the week: How should I allocate memory properties? (3289s)

5: Hive Partitions, sync_partition_metadata, and Query Exceeded Max Columns!

• Concept of the week: Hive Partitioning (1152s)
• PR of the week: PR 223 Add system.sync_partition_metadata procedure to sync Hive table partitions (1760s)
• PR Demo: system.sync_partition_metadata procedure demo (2097s)
• Question of the week: Why am I getting, 'Query exceeded maximum columns.' error? (3656s)

4: Presto on ACID, row-level INSERT/DELETE, and why JDK11?

• Concept of the week: Presto on ACID (1175s)
• PR of the week: PR 5402 Hive ACID row-level INSERT and DELETE (1644s)
• PR Demo: Hive ACID row-level INSERT and DELETE demo (2546s)
• Question of the week: Why is JDK 11 required to run Presto and how can I revert to JDK8? (3259s)

3: Running two Presto distributions and Kafka headers as Presto columns

• Interview: Using multiple Presto distributions (1810s)
• PR of the week: PR 4462 Add Kafka headers as columns (3095s)

2: Kubernetes, arrays on Elasticsearch, and security breaks the UI

• Presto Event: Presto for any company size (889s)
• Concept of the week: Presto on Kubernetes (1699s)
• PR of the week: PR 2441 Add array support using the _meta mapping in Elasticsearch (2484s)
• Question of the week: Why does my Presto UI say web interface disabled? (3306s)

1: What is Presto, WITH RECURSIVE, and Hive connector

• Concept of the week: What is Presto? (1477s)
• PR of the week: PR 5163 WITH RECURSIVE (2345s)
• Question of the week: Does the Hive connector depend on the Hive runtime? (2918s)