Apache Ranger has arrived! With the new Trino 466 you all get another jam-packed release of Trino awesomeness. One of the goodies is a new plugin for access control for your data with Apache Ranger, and it has gone through a long story to get here.
Apache Ranger has a long history and wide adoption as an access control system for data lakes using Hadoop and Hive. Since Trino brings fast analytics to this space, and also supports modern data lakehouses and other data sources, Apache Ranger is a natural fit for access control on a Trino-powered data platform.
The beginnings #
Apache Ranger has been in use with Trino for a long time - in fact there are early, rudimentary pull requests from 2019 that implemented some support. And even before then, various hacks existed. In 2020, a plugin for PrestoSQL was added to Apache Ranger. Aakash Nand blogged about Integrating Trino and Apache Ranger in 2021 to adjust for the changes to Trino. Jeff Xu followed up with Integrating Trino and Apache Ranger in a Kerberos-secured enterprise environment in 2022, followed quickly by the addition of the Trino support to the Apache Ranger repository.
Testing and container images #
However that was only half of the needed support. The Trino project moves very fast with nearly weekly releases, so the best approach is to have the supporting plugin in Trino directly so every release includes the relevant updates. Erik Anderson created a more mature plugin that was in production use for quite a while for Trino. His pull request from July 2022 included great background reasoning for having the plugin in Trino. One of the issues that Erik solved for the Trino project is testing. Trino plugins require the availability of a container image for testing whatever integration. Apache Ranger did still not ship a container in 2022, but thanks to the lobbying efforts of Erik this changed and a container image became available over the months.
A long sprint #
Unfortunately, focus changed and while the PR from Erik existed and was useful, it never made it to merge due to waning priorities. That changed when Madhan Neethiraj from the Apache Ranger project stepped up and created new PR in July 2024.
We knew this could be another shot at it, and it would require a lot of work to get it done, since we put a high focus on quality so that we can maintain the Trino codebase for the long run. Monitoring all PRs regularly I (Manfred Moser) noticed it and jumped in with first help.
Erik and other interested users chimed in. lozbrown and Manfred helped with documentation and getting other developers interested. The heavy technical reviews and lots of guidance came from Krzysztof Sobolewski and Grzegorz Kokosiński.
During the whole process, Madhan had to react to comments, update the code, and also regularly rebase his PR to adjust for the constantly changing Trino codebase in the master branch. Starburst recognized Madhan’s effort and featured him as Starburst Trino Champion. Interestingly, the container image ended up not being used for testing, however it will be crucially important for many users deploying Apache Ranger on Kubernetes anyway. Nearly 400 comments and over four months later we all got to celebrate. The Trino maintainer Grzegorz took on the responsibility and merged the PR. Yuya Ebihara and Martin Traverso followed up with minor cleanups, and we finally shipped the plugin as part of Trino 466.
A huge congratulations and thank you goes out to everyone involved.
Now it is your turn to have a look at the documentation, learn more about Trino and Apache Ranger, and maybe even proceed to help us improve the integration.
Next steps #
Beyond our celebration, more tasks are waiting for all of us:
- Test it out in your usage and migrate from any old or custom versions.
- Help us improve the documentation significantly to allow easier adoption.
- Work with lozbrown on adding support to the Helm chart.
- Check out the codebase and help us fix bugs and add features.
And last, but not least - join us all to celebrate Trino at the upcoming Trino Summit 2024 for two days of amazing sessions and interaction with your peers from the Trino community and the Trino Contributor Call for more open community chat and discussion.