大家好,
在这个版本中,Kyuubi 支持最新的 Spark 3.2,Z-Order 优化 sql 查询,无需提供 principal 和 keytab 的长时间运行的 SQL 引擎,支持 mysql 协议前端和 mysql 客户端。 其他主要更新包括 GROUP 引擎共享级别的支持、支持异步打开会话、用户友好的 kyuubi beeline 和 zookeeper 身份验证增强。
要下载 Apache Kyuubi (Incubating) 1.4.0-incubating,请访问 下载页面。
使用最新的 Spark 3.2 进行了全面验证
支持Z-Order优化SQL
支持长时间运行的 SQL 引擎,无需提供 principal 和 keytab
支持 GROUP 引擎共享级别
增强 USER 引擎共享级别,支持引擎池
增强 zookeeper 身份验证和 ACL
添加更多 Kyuubi 自定义的函数
对所有数据湖平台 Delta Lake、Iceberg 和 Hudi 进行全面测试
同时支持 KERBEROS 和 PLAIN 认证
支持 MySQL 协议前端和 MySQL 客户端(实验性质)
支持在打开会话期间异步启动查询引擎和用户友好的 Kyuubi Hive JDBC 和 BeeLine
[KYUUBI #1225] [KYUUBI #1228] Add kyuubi-extension-spark-3-2 module and profile to support latest spark-3.2
[KYUUBI #939] Add Z-Order extensions to support optimize SQL
[KYUUBI #913] Support long running SQL engine without principal and keytab provided
[KYUUBI #1219] Support MySQL Protocol Frontend
[KYUUBI #1349] Add new conf kyuubi.frontend.protocols
[KYUUBI #1206] Support GROUP engine share level
[KYUUBI #1018] Set spark.sql.execution.topKSortFallbackThreshold to 10000
[KYUUBI #1059] Add Plan Only Operations
[KYUUBI #1039] Application stop log redirect append to engineLog
[KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page
[KYUUBI #1198] Support to incremental collect statement result
[KYUUBI #1248] Change kyuubi.engine.session.initialize.sql default value to Nil
[KYUUBI #1286] Increase kyuubi.session.engine.initialize.timeout default to 3min
[KYUUBI #1323] Introduce EngineType to distinguish multiple engines
[KYUUBI #1262] Support both KERBEROS and PLAIN authentication at the same time
[KYUUBI #1160] Support to config operation log root dir both for kyuubi server and engine sides
[KYUUBI #1204] Enhance zookeeper authentication and ACLs
[KYUUBI #1039] Application stop log redirect append to engineLog
[KYUUBI #703] Integrate test with Hudi 0.9.0
[KYUUBI #1338] Integrate test with Iceberg 0.12.1
[KYUUBI #1131] Rename KyuubiDriver to KyuubiHiveDriver
[KYUUBI #1346] Support launch query engine asynchronously during opening session
[KYUUBI #1377] Support to get launch engine log asynchronously with KyuubiConnection
[KYUUBI #1414] Add kyuubi-hive-beeline module to support KyuubiConnection and KyuubiStatement
[KYUUBI #1047] Use deployment to run kyuubi server HA on K8s
[KYUUBI #1223] Remove dead module kyuubi-spark-monitor
[KYUUBI #1277] Add UDF engine_id
[KYUUBI #660] Add UDF session_user
最后但同样重要的是,如果没有以下贡献者,该版本是不可能发布的: AnybodyHome, Brian Yue, Cheng Pan, Fei Wang, Fu Chen, Jagadesh Adireddi, Kent Yao, Min Zhao, Guoqing Yang, h, happytofly, hongdongdong, jiaoqingbo, qiuliang, qphien, senmiaoliu, simon, sunfangbin, sychen, timothy65535, ulysses-you, wFoget, xiebo03, xiongyinke, yanghua, yanyu34946, zhang1002, zhenjiaguo, zhouyifan279, zwangsheng, 张宇翔
2021 12月 10