Apache Kyuubi(Incubating) 1.3.0-incubating Release Notes

To download Apache Kyuubi (Incubating) 1.3.0-incubating, visit the downloads page.

Highlight

• Fully verified with the latest Spark 3.0/3.1

• Support build on JDK8(default), JDK11

• Provide enhanced AQE in Kyuubi Spark Extensions (for Spark 3.1)

• Provide several auxiliary SQL functions as a supplement to Spark’s built-in Functions

• Major improvements of Spark on Kubernetes, include documents, integration tests, tools, etc.

• Support engine/session level configurable initialize SQL statements

• Support custom authentication implementation

• Support stop Spark engine through Web UI

• Support single session mode

• Several improvements of observability

• Fully tested with the all data lake platforms Delta, Iceberg

• Fully tested with MiniYARNCluster

Functionality

• [KYUUBI #631] Add kyuubi-spark-extensions-3-1 module

• [KYUUBI #657][KYUUBI #659] Provides auxiliary SQL functions as supplement to Spark’s built-in Functions

• [KYUUBI #897] Support stop Spark engine through Web UI

• [KYUUBI #762] Add Kyuubi Hive JDBC Module

• [KYUUBI #926] Support add repartition for OptimizedCreateHiveTableAsSelectCommand

• [KYUUBI #670] KyuubiSQLException support sqlState and errorCode

• [KYUUBI #707] Respect spark.sql.catalogImplementation in engine

• [KYUUBI #641] Support custom authentication

• [KYUUBI #751] Support single session mode

• [1.2][KYUUBI #457] Support configurable initialize sql statement for engine startup

• [KYUUBI #857] Support configurable initialize SQL statement for each session

Integration

• [KYUUBI #755] Default build against Spark 3.1

• [KYUUBI #693][KYUUBI #760] Support build against JDK11

• [KYUUBI #768][K8S] Add module shuffle-files-cleaner

• [KYUUBI #844][K8S] Exposing Kyuubi as a NodePort service

• [KYUUBI #859][K8S] Add kubernetes configmap

• [KYUUBI #656][K8S] Document how to deploy Kyuubi engines on Kubernetes

• [KYUUBI #776][K8S] Run kyuubi on Kubernetes - make basic dockerfile and example

• [KYUUBI #444][KYUUBI #854][TEST][K8S] Add minikube for K8s integration test

• [KYUUBI #712][LDAP] Add a new config property authentication.ldap.guidKey

• [KYUUBI #643][TEST] Test Detla 1.0.0 with Spark 3.1

• [KYUUBI #948][TEST] Test Iceberg 0.12.0 with Spark 3.0/3.1

• [KYUUBI #527][TEST][YARN] Test with MiniYARNCluster

Observability

• [KYUUBI #737] Add monitor module

• [KYUUBI #727] Add the listener for each SQL

• [KYUUBI #767] Add statement event tracking

Performance

• [1.2][KYUUBI #629] Refine distributed lock to lock only engine bootstrap

Others

• [KYUUBI #416] Detected Dependency List Change for better dependency management

• [KYUUBI #487][KYUUBI #566] Add TPC-DS DDL test cases

• [KYUUBI #871] Change module name kyuubi-main to kyuubi-server