Releases Archive
- [KYUUBI #5451] Ignore NoSuchFileException during OperationLog.close()
- [KYUUBI #6291] Avoid NPE in MySQLErrPacket
- [KYUUBI #2782] Add missed UNION_TYPE back
- [KYUUBI #6204] Fix kyuubi session limiter leak when opening session failed
- [KYUUBI #6061] Delete svn staging dir before checking out
- [KYUUBI #6047] Free up disk space
- [KYUUBI #5953] Update NOTICE
- [KYUUBI #5951] Improve doc for Session Conf Advisor
- [KYUUBI #5933] Happy New Year 2024
- [KYUUBI #5922] Enhance YARN app state judgment
- [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak
- [KYUUBI #5857] Dependencies CI check when
dev/dependencyList
changes - [KYUUBI #5849] Incorrectly parse JDBC URL while variable includes colon
- [KYUUBI #5822] Fix the usage msg of KyuubiBeeLine
- [KYUUBI #5819] Fix rst syntax of missing blank line after code-block
- [KYUUBI #5785] Fix flaky test - JpsApplicationOperation with spark local mode
- [KYUUBI #5750] Fix the inaccurate issue of obtaining COLUMN_SIZE in the decimal field jdbc of spark engine
- [KYUUBI #5754] Fix engine id of the engine probe failures logging messages
- [KYUUBI #5317] Hive Connector throws NotSerializableException on reading Hive Avro partitioned table
- [KYUUBI #5720] Fix flaky test in TPCHQuerySuite
- [KYUUBI #5713] Backport HIVE-27271: Client connection to HS2 fails when transportMode=http, ssl=true, sslTrustStore specified without trustStorePassword in the JDBC URL
- [KYUUBI #5697] Fix incorrect url reference in code and docs
- [KYUUBI #5599] Correct the parameter name of fetching op log in RESTful API
- [KYUUBI #5499]Catch any exception when closing idle session
- [KYUUBI #5414] KSHC reader should not pollut the global hiveConf instance
- [KYUUBI #5432] Fix typo in README.md
- [KYUUBI #5347] Fix java code errors in quick_start_with_jdbc
- [KYUUBI #5333] Aaptive call fetchLaunchEngineResult
- [KYUUBI #5308] Ensure release using Java 8
- Shaoyun Chen
- Chao Chen
- Flyangz
- Pengqi Li
- Bowen Liang
- Omkar Makhare
- Drake Youngkun Min
- Cheng Pan
- Gianluca Principini
- Zeyu Wang
- Kang Wang
- Fei Wang
- Zhen Wang
- Kaifei Yi
- He Zhao
- mrtisttt
- [KYUUBI #6320] Fix terminated application pods not deleted issue when kyuubi server restarted
- [KYUUBI #6298] Make ServiceMonitor use selector labels
- [KYUUBI #6291] Avoid NPE in MySQLErrPacket
- [KYUUBI #6283] Improve the message of distribution built without enabling web ui
- [KYUUBI #6287] Avoid missing matche
- [KYUUBI #6268] Specify logDir for RollingFile filePattern
- [KYUUBI #6212] Added audit handler shutdown to the shutdown hook
- [KYUUBI #6271] Upgrade kafka-clients from 3.5.1 to 3.5.2
- [KYUUBI #5767] Fix spark batch conf not convert issue
- [KYUUBI #6223] Fix Scala interpreter can not access spark.jars issue
- [KYUUBI #6234] Fix invalid code-block syntax in docs
- [KYUUBI #6215] Improve DropIgnoreNonexistent rule for Spark 3.5
- [KYUUBI #6221] Fix parameter replacement issue caused by incorrect sql split
- [KYUUBI #2782] Add missed UNION_TYPE back
- [KYUUBI #6207] Support to retrieve Spark UserDefinedType result
- [KYUUBI #6206] Update comments for job group id of Spark broadcast jobs
- [KYUUBI #6204] Fix kyuubi session limiter leak when opening session failed
- [KYUUBI #6182] JDBC engine using the overlay conf as session conf
- [KYUUBI #6193] Add known_transkations
- [KYUUBI #6183] KyuubiConf.getKubernetesConf should set namespace even if the Kubernetes context is empty
- [KYUUBI #6175] Refactor copy path about SPARK_HOME in docker
- [KYUUBI #6166] Fix wrong copy path about SPARK_HOME in docker
- [KYUUBI #6170] Merge JDBC Trino TPC jobs and Style Check jobs in GitHub Actions workflow
- [KYUUBI #6164] Limit max parallel for each GitHub Actions workflow
- [KYUUBI #6156] Remove
flink.
prefix for create session configurations - [KYUUBI #6173] Bump maven-download-plugin from 1.7.1 to 1.8.1
- [KYUUBI #6168] Check if forcedMaxOutputRows is negative
- [KYUUBI #6167] Drop JDK 11 on GitHub Actions
- [KYUUBI #6158] Remove confusing variable
- [KYUUBI #6152] Remove useless variable
- [KYUUBI #6130] Stop engine immediately after close session for
CONNECTION
level FlinkSQLEngine - [KYUUBI #6147] Use the overlay conf as session conf
- [KYUUBI #6078] KSHC should handle the commit of the partitioned table as dynamic partition at write path
- [KYUUBI #6139] Expose env vars and sys props to JUnit tests too
- [KYUUBI #6126] Filter blank engine home env
- [KYUUBI #6114] Properly handle KyuubiStatement#getMoreResults(CLOSE_CURRENT_RESULT)
- [KYUUBI #5902] Bump doc build dependencies
- [KYUUBI #6081] RESTful client should retry on UnknownHostException
- [KYUUBI #6071] Add .java-version into git ignore
- [KYUUBI #6069] TFrontendServiceSuite - close expired operations
- [KYUUBI #5104] BatchesResourceSuite - expose the metrics with operation type and current state
- [KYUUBI #6061] Delete svn staging dir before checking out
- Shaoyun Chen
- Lennon Chin
- Fei Wang
- Bowen Liang
- Xiao Liu
- Senmiao Liu
- Tigran Manasyan
- Cheng Pan
- TakawaAkirayo
- Zhen Wang
- Binjie Yang
- Vino Yang
- Kaifei Yi
- He Zhao
- amanraj2520
- dupen01
- sudohainguyen
- Fully verified with Java 8, 11, and 17, Scala 2.12 and 2.13
- Fully verified with Apache Spark 3.1 to 3.5, support for Spark 3.1 is deprecated and will be removed in the next release
- Fully verified with Apache Flink 1.16 to 1.19, support for Flink 1.16 is deprecated and will be removed in the next release
- Fully verified with Apache Hive 2.3, 3.1, and Cloudera Hive 2.1.1-cdh6, support running Hive engine on YARN (experimental)
- Spark engine supports save large query result to file before retrieving
- JDBC engine supports more dialects, including Doris, MySQL, PostgreSQL, Phoenix, StarRocks and Impala
- Kyuubis server metastore supports PostgreSQL as backend storage
- Introduce a basic SQL query editor in Kyuubi Web UI
- Continuously Spark AuthZ plugin improvements, including enhanced Iceberg support and new introduced Apache Hudi and Delta Lake support
- Continuously Kyuubi & Spark on Kubernetes improvements
- [KYUUBI #5423] Support chaining SessionConfAdvisors
- [KYUUBI #5631][K8S][HELM] Set session affinity if needed in helm chart
- [KYUUBI #5674] Add PostgreSQL as backend database for kyuubi metadata
- [KYUUBI #5683] Add
kyuubi.session.proxy.user
as an alternative tohive.server2.proxy.user
- [KYUUBI #5704] Add unit tests for
kyuubi.session.proxy.user
in RESTful API - [KYUUBI #5729] Use G1GC as Java option example in kyuubi-env template
- [KYUUBI #5797] Support to describe engine with command in current session
- [KYUUBI #5797][FOLLOWUP] Desc engine command support show engine registered attributes
- [KYUUBI #5873] Support to get operation progress with RESTful API
- [KYUUBI #5912] Cleanup unused config
kyuubi.frontend.thrift.http.allow.user.substitution
- [KYUUBI #5921]
EngineRef#deregister
should return the value - [KYUUBI #5934][K8S][HELM] Add Spark configuration support
- [KYUUBI #6003] Allow disabling user impersonation on launching engine
- [KYUUBI #6006][HELM] Support additional labels for service monitor
- [KYUUBI #6066] Fix PostgreSQL metastore migration script failure if indexes already exist
- [KYUUBI #6070] Improve perf on assembling row-based TRowSet
- [KYUUBI #6080] Improve instruction for deploy Kyuubi on K8s
- [KYUUBI #6081] RESTful client should retry on UnknownHostException
- [KYUUBI #6099] Remove support of Derby for Kyuubi metastore
- [KYUUBI #6126] Filter blank engine home env
- [KYUUBI #6140] Move getCodeSourceLocation to kyuubi-util
- [KYUUBI #5377] Spark engine query result save to file
- [KYUUBI #5377][FOLLOWUP] Spark engine query result save to file
- [KYUUBI #5377][FOLLOWUP] Always try to cleanup session result path
- [KYUUBI #5377][FOLLOWUP] Get limit from more spark plan and regard result max rows
- [KYUUBI #5870] Directly mapping engine’s data type to Java type for TRowSet generation
- [KYUUBI #5877] Support Python magic syntax for notebook usage
- [KYUUBI #5877][FOLLOWUP] Add spark output mode to better support PySpark notebook
- [KYUUBI #5894] Separate closed and online sessions/statements in the SparkUI’s engine tab
- [KYUUBI #5899] Refine the python/scala interpreter error prompt
- [KYUUBI #6053][K8S] Show more details for pod/container status in application error
- [KYUUBI #6083] Bump Spark 3.3.3 to 3.3.4
- [KYUUBI #6091] Deprecate and remove building support for Spark 3.1
- [KYUUBI #6092] Enable DeltaTest for Spark 3.5
- [KYUUBI #6094] Bump Spark from 3.5.0 to 3.5.1
- [KYUUBI #6095] Enable PaimonTest for Spark 3.5
- [KYUUBI #6096] Enable PySpark tests for Spark 3.5
- [KYUUBI #6107][Spark] Collect and summarize the
executorRunTime
andexecutorCpuTime
of the statement - [KYUUBI #6107][FOLLOWUP] Add comments for the newly added parameters of the Spark event
- [KYUUBI #6108] Display the CPU time consumed by the statement in the Spark Engine tab
- [KYUUBI #6156] Remove
flink.
prefix for create session configurations - [KYUUBI #6163] Set default Spark version to 3.5
- [KYUUBI #6166][MINOR] Fix wrong copy path about SPARK_HOME in docker
- [KYUUBI #6175][MINOR] Refactor copy path about SPARK_HOME in docker
- [KYUUBI #5248][AUTHZ] Check previleges for Iceberg’s CALL to RewriteDataFilesProcedure procedure
- [KYUUBI #5255] Add an optional comment field to the authz specs for better recognition
- [KYUUBI #5284] Support Hudi Alter Table Command in Authz
- [KYUUBI #5323][AUTHZ] Drop Hive and Iceberg tables with PURGE option in tests
- [KYUUBI #5359][AUTHZ] Support Create Table Commands for Hudi
- [KYUUBI #5360] Support Hudi InsertIntoHoodieTableCommand
- [KYUUBI #5361][AUTHZ] Support Drop/Truncate Table Commands for Hudi
- [KYUUBI #5362] Remove Spark 3.0 support for Authz
- [KYUUBI #5399][AUTHZ] Cleanup Spark 3.0 specific implementation
- [KYUUBI #5407][AUTHZ] Tests for Iceberg system procedures of snapshot management
- [KYUUBI #5417] should not check in-subquery in permanent view
- [KYUUBI #5417] Authz should not check dependent subquery plan privilege
- [KYUUBI #5427][AUTHZ] Shade spark authz plugin
- [KYUUBI #5446][AUTHZ] Support Create/Drop/Show/Reresh index command for Hudi
- [KYUUBI #5447][AUTHZ] Support Hudi DeleteHoodieTableCommand/UpdateHoodieTableCommand/MergeIntoHoodieTableCommand
- [KYUUBI #5447][FOLLOWUP] Remove unrelated debug prints in TableIdentifierTableExtractor
- [KYUUBI #5452][AUTHZ] Support Compaction table commands for Hudi
- [KYUUBI #5455][AUTHZ] Support hudi CompactionHoodiePathCommand & CompactionShowHoodiePathCommand
- [KYUUBI #5457][AUTHZ] Support RepairTable Commands for Hudi
- [KYUUBI #5462][AUTHZ] Support create table command for Paimon
- [KYUUBI #5472] Permanent View should pass column when child plan no output
- [KYUUBI #5472][AUTHZ][FOLLOWUP] Check permanent view also need support merge projection
- [KYUUBI #5475][FOLLOWUP] Authz check permanent view’s subquery should check view’s correct privilege
- [KYUUBI #5478][AUTHZ] Support Hudi ShowHoodieTablePartitionsCommand
- [KYUUBI #5479][AUTHZ] Support Hudi CallProcedureHoodieCommand for stored procedures
- [KYUUBI #5492][AUTHZ] saveAsTable create DataSource table miss db info
- [KYUUBI #5496][AUTHZ] Authz shaded include the transitive jackson libs
- [KYUUBI #5497][AuthZ] Simplify debug message for missing field/method in ReflectUtils
- [KYUUBI #5506][AUTHZ][MINOR] Reuse loaded table extractor in Hudi table extractors
- [KYUUBI #5512][AuthZ] Remove the non-existent query specs in Deletes and Updates
- [KYUUBI #5529][AUTHZ] Support create table command for Delta Lake
- [KYUUBI #5503][AUTHZ] Check plan auth checked should not set tag to all child nodes
- [KYUUBI #5503][FOLLOWUP][AUTHZ] Authz should skip inner plan that have been verified
- [KYUUBI #5529][AUTHZ][FOLLOWUP] Fix Create table ut for Delta Lake does not trigger a sql operation
- [KYUUBI #5529][AUTHZ][FOLLOWUP] Remove useless org.apache.spark.sql.delta.commands.CreateDeltaTableCommand
- [KYUUBI #5532][AUTHZ] Support Alter table commands of updating table schema for Delta Lake
- [KYUUBI #5533][AUTHZ] Support merge into table command for Delta Lake
- [KYUUBI #5534][AUTHZ] Support optimize table command for Delta Lake
- [KYUUBI #5535][AUTHZ] Support vacuum table command for Delta Lake
- [KYUUBI #5537][AuthZ] Order Command Spec by Command Class [group, classname]
- [KYUUBI #5543][AUTHZ] Generalize command specs and check duplicated class names of command spec
- [KYUUBI #5546][AUTHZ] Reorgnize the package names for rules
- [KYUUBI #5555][AUTHZ] Extractor common authorization rule class
- [KYUUBI #5556][AUTHZ] Support Alter table commands of set table properties
- [KYUUBI #5557][AUTHZ] Refactor code about handle PermanentViewMarker
- [KYUUBI #5565][AUTHZ] Support Delete/Insert/Update table command for Delta Lake
- [KYUUBI #5575][AUTHZ] Support InsertIntoDataSourceDir/InsertIntoHiveDirCommand path check
- [KYUUBI #5575][KYUUBI #5611][AUTHZ] Check path privilege should use correct action type
- [KYUUBI #5576][Bug] Fix wrong code in test case of dir command
- [KYUUBI #5579][AUTHZ] Support LogicalRelation don’t have CatalogTable but have HadoopFsRelation
- [KYUUBI #5594][AUTHZ] BuildQuery should respect normal node’s input
- [KYUUBI #5611][AUTHZ] Authz support path privilege for SaveIntoDataSourceCommand
- [KYUUBI #5628][AUTHZ] Support path privilege check for resource command
- [KYUUBI #5630][Authz] Support path check of LoadDataCommand
- [KYUUBI #5638][AUTHZ] URI related privileges shall be READ/WRITE
- [KYUUBI #5638][FOLLOWUP] Remove unnecessary actionType in uri desc
- [KYUUBI #5642][AUTHZ] Support path privilege of CreateDatabaseCommand/AlterDatabaseSetLocationCommand/CreateNamespace/SetNamespaceLocation
- [KYUUBI #5642][FOLLOWUP] Fix unit test
- [KYUUBI #5652][AUTHZ] CreateTable related command support path privilege
- [KYUUBI #5657][AUTHZ] Support path privilege of AlterTableSetLocationCommand/AlterTableAddPartitionCommand
- [KYUUBI #5664][AUTHZ] Support create path-based table for Delta Lake
- [KYUUBI #5677][AUTHZ] Typeof expression miss column information
- [KYUUBI #5684][AUTHZ] Support delete/update path-based table for Delta Lake
- [KYUUBI #5690][AUTHZ] Support insert into/overwrite path-based table for Delta Lake in Authz
- [KYUUBI #5692][Bug] Authz not skip explain command
- [KYUUBI #5707][AUTHZ] Support merge into path-based table for Delta Lake in Authz
- [KYUUBI #5715][MINOR] Replace with .isDefined and .isEmpty
- [KYUUBI #5726][AUTHZ] Support optimize path-based table for Delta Lake in Authz
- [KYUUBI #5735][AUTHZ] Support vacuum path-based table for Delta Lake
- [KYUUBI #5743][AUTHZ] Improve AccessControlException verification of RangerSparkExtensionSuite
- [KYUUBI #5768][AUTHZ] Authz internal place holder should skip privilege check
- [KYUUBI #5780][AUTHZ] Treating PermanentViewMarker as LeafNode make code simple and got correct privilege object
- [KYUUBI #5780][AUTHZ][FOLLOWUP] Format PermanentViewMarker tree string
- [KYUUBI #5793][AUTHZ][BUG] PVM with nested scalar-subquery should not check src table privilege
- [KYUUBI #5884] PVM should inherit MultiInstance and wrap with new exprId
- [KYUUBI #5903] PVM should override computeStats method
- [KYUUBI #5913][Bug] After resolve PVM should mark all nodes as checked
- [KYUUBI #5937] PVM cause cache table not work
- [KYUUBI #5964][BUG] Avoid check not fully optimized query for InsertIntoDataSourceDirCommand and InsertIntoDataSourceCommand
- [KYUUBI #5965][AUTHZ] Supports check hoodie procedures show_commits resource privileges
- [KYUUBI #5985][AUTHZ][MINOR] Remove incorrect getUri method
- [KYUUBI #5997][AUTHZ] Avoid unnecessary loop of RuleEliminateTypeOf
- [KYUUBI #6007] AuthZ should check hoodie procedures path resource privileges
- [KYUUBI #5580] Support generate and window operators for InferRebalanceAndSortOrders
- [KYUUBI #5786] Disable spark script transformation
- [KYUUBI #5816] Change spark rule class to object or case class
- [KYUUBI #6125] Drop Kyuubi extension for Spark 3.1
- [KYUUBI #6168] Check if forcedMaxOutputRows is negative
- [KYUUBI #5264][FLINK] Force disable Flink’s session timeout
- [KYUUBI #5489] Adjust shuffle partitions dynamically
- [KYUUBI #5505][FLINK] Support HELP command
- [KYUUBI #6097][KYUUBI #6098] Upgrade Flink 1.16.3 and 1.18.1
- [KYUUBI #6130] Stop engine immediately after close session for
CONNECTION
level FlinkSQLEngine - [KYUUBI #6134] Support Flink 1.19
- [KYUUBI #6142] Deprecate Flink 1.16
- [KYUUBI #6152] Remove useless variable
- [KYUUBI #6158][MINOR] Remove confusing variable
- [KYUUBI #6171] Drop building support for Flink 1.16
- [KYUUBI #6171][FOLLOWUP] Restore flink-1.16 profile to recover Flink IT
- [KYUUBI #5830] Test Hive engine with CDH Hive 2.1.1-cdh6.3.2
- [KYUUBI #5865] Support get SQL keywords from Hive engine through API
- [KYUUBI #5867] HiveEngine support run on YARN mode
- [KYUUBI #5867][FOLLOWUP] Remove unused config
- [KYUUBI #5867][FOLLOWUP] Fix generated configuration docs and typo
- [KYUUBI #5968] Support set authentication user for Trino engine
- [KYUUBI #6147][TRINO] Use the overlay conf as session conf
- [KYUUBI #5375] JDBC Engine supports PostgreSQL
- [KYUUBI #5382][JDBC] Duplication cleanup improvement in JdbcDialect and schema helpers
- [KYUUBI #5404] JDBC Engine supports StarRocks
- [KYUUBI #5464] JDBC Engine supports MySQL
- [KYUUBI #5509] Add Apache Impala JDBC engine dialect
- [KYUUBI #5582] JDBC Engine supports configurable default fetchSize
- [KYUUBI #5851] Generalize TRowSet generators
- [KYUUBI #5861] Generalize TRowSet generator for JDBC engine with dialects
- [KYUUBI #5862] Use TestContainerForAll for testing JDBC engine with testcontainers
- [KYUUBI #5906][JDBC] Rebase Doris dialect implementation
- [KYUUBI #6037] Fix the JDBC engine provider name
- [KYUUBI #5384][KSCH] Hive connector supports Spark 3.5
- [KYUUBI #5709] Avoid exposing client keystore and truststore passwords in the JDBC URL
- [KYUUBI #5783] Switch to
kyuubi-relocated-hive-service-rpc
- [KYUUBI #5828][CHAT] Kyuubi chat engine supports ernie bot(文心一言)
- [KYUUBI #6078] KSHC should handle the commit of the partitioned table as dynamic partition at write path
- [KYUUBI #6114] Properly handle KyuubiStatement#getMoreResults(CLOSE_CURRENT_RESULT)
- [KYUUBI #5366][UI] Add submit SQL page in Kyuubi Web UI
- [KYUUBI #5517][UI] Initial implement the SQL Lab page
- [KYUUBI #5958][WEBUI] Bump follow-redirects from 1.15.1 to 1.15.4
- [KYUUBI #6002] Bump vite from 4.3.9 to 4.5.2 in web UI
- [KYUUBI #5265][DOCS][FLINK] Add notice about Jobmanager HA to Flink deployment docs
- [KYUUBI #5500] Add Kyuubi Code Program to Doc
- [KYUUBI #5688][DOC] Update supported spark 3.5.x in authz doc
- [KYUUBI #5762][DOC] Introduce Kyuubi Web UI Engine Page
- [KYUUBI #5893][DOC] Update logos in layer picture on welcome page
- [KYUUBI #5905][DOC] Workaround for long inlined code sample in config page
- [KYUUBI #6010] Improve Gluten usage documentation
- [KYUUBI #6119][DOC] Update doc for HA/Zookeeper Configuration
- [KYUUBI #4453][FOLLOW] Delete LICENSE for removed kubernetes client dependencies
- [KYUUBI #5104][TEST] BatchesResourceSuite - expose the metrics with operation type and current state
- [KYUUBI #5241] Bump Commons Lang3 from 3.12.0 to 3.13.0
- [KYUUBI #5245][INFRA] Daily deploy snapshot version on branch-1.8
- [KYUUBI #5266] Upgrade playground to Kyuubi 1.7.1
- [KYUUBI #5322][INFRA] Enable GitHub Projects
- [KYUUBI #5324] Bump latest 1.7.3 in playground and helm charts
- [KYUUBI #5342] Add label hacktoberfest to project
- [KYUUBI #5356][INFRA] GitHub Issue template for 2023 Kyuubi Code Contribution Program
- [KYUUBI #5395] Bump netty from 4.1.93.Final to 4.1.100.Final
- [KYUUBI #5449] Bump Delta Lake 3.0.0
- [KYUUBI #5501] Update codecov token and fix codecov reporting on PRs
- [KYUUBI #5509][FOLLOWUP] JDBC IT should always depends on kyuubi-hive-jdbc-shaded
- [KYUUBI #5600] Fix flaky test SessionsResourceSuite
- [KYUUBI #5602] Improve issue template for 2023 Kyuubi Code Contribution Program
- [KYUUBI #5660][TEST] Ensure SPI provider file content sorted in alphabet order
- [KYUUBI #5671] Bump axios from 0.27.2 to 1.6.0 in /kyuubi-server/web-ui
- [KYUUBI #5674][LICENSE][FOLLOWUP] Update license files
- [KYUUBI #5676] Bump Apache parent pom from 30 to 31
- [KYUUBI #5701] Enhance the PR Template to make PR desc informative
- [KYUUBI #5716][TEST] Fix match may not be exhaustive in Scala 2.13
- [KYUUBI #5774][CI] Bump actions/checkout from v3 to v4
- [KYUUBI #5779][CI] Upgrade Github actions for Docker
- [KYUUBI #5800][KYUUBI#5467] Integrate Intel Gluten with Spark engine
- [KYUUBI #5835] Remove issue template for 2023 Kyuubi Code Contribution Task
- [KYUUBI #5838][BUILD] Bump actions/setup-python from v4 to v5
- [KYUUBI #5839][BUILD] Bump actions/setup-java from v3 to v4
- [KYUUBI #5841][BUILD] Bump actions/stale from v7 to v9
- [KYUUBI #5842][BUILD] Bump actions/setup-node from v3 to v4
- [KYUUBI #5874][INFRA] Bump github labeler action to v5
- [KYUUBI #5879] Bump dropwizard metrics from 4.2.8 to 4.2.23
- [KYUUBI #5902] Bump doc build dependencies
- [KYUUBI #5914] Update layer diagram on welcome page
- [KYUUBI #5925] Kyuubi TPC-DS support running benchmark with skipping some queries
- [KYUUBI #5939] Bump Gluten version to recover Gluten IT
- [KYUUBI #5941] Drop Kubernetes Block Cleaner Tool from Kyuubi
- [KYUUBI #5944] Introduce Prometheus and Grafana for Kyuubi Playground
- [KYUUBI #5955][TEST][GLUTEN] Disable gluten ui when running integration test
- [KYUUBI #5973] Bump gRPC from 1.53.0 to 1.60.1
- [KYUUBI #5974] Bump jetcd 0.7.7
- [KYUUBI #5975] Bump trino-client 411
- [KYUUBI #5975][FOLLOWUP][TEST] Fix trino - get type info
- [KYUUBI #6043][TEST][GLUTEN] Gluten-it gluten package add arch suffix
- [KYUUBI #6061] Delete svn staging dir before checking out
- [KYUUBI #6063] Bump iceberg 1.4.3
- [KYUUBI #6067] Bump org.postgresql:postgresql from 42.6.0 to 42.7.2
- [KYUUBI #6068] Remove community section from user docs
- [KYUUBI #6069][TEST] TFrontendServiceSuite - close expired operations
- [KYUUBI #6071] Add .java-version into git ignore
- [KYUUBI #6074] Add a script to simplify the process of creating release notes
- [KYUUBI #6075] Update known_translations
- [KYUUBI #6076] Update Helm Chart and playground to use 1.8.1
- [KYUUBI #6085] Bump gRPC from 1.60.1 to 1.60.2
- [KYUUBI #6089] Bump Paimon from 0.5.0-incubating to 0.7.0-incubating
- [KYUUBI #6090] Bump Delta from 3.0.0 to 3.1.0 for Spark 3.5
- [KYUUBI #6101] Bump Jackson from 2.15.0 to 2.15.4
- [KYUUBI #6109] Fork beeline module from Apache Hive 3.1.3
- [KYUUBI #6117] Bump Kyuubi Shaded 0.3.0
- [KYUUBI #6118] Migrate to shaded HMS client for getting delegation token on server
- [KYUUBI #6129][BUILD] Remove the extension spark jars before build to avoid include unspecified version
- [KYUUBI #6131] Simplify Maven dependency management after dropping building support for Spark 3.1
- [KYUUBI #6139] Expose env vars and sys props to JUnit tests too
- [KYUUBI #6143] Add Dockerfile.HMS for testing
- [KYUUBI #6162] Cut out hive-common deps in beeline module
- [KYUUBI #6164] Limit max parallel for each GitHub Actions workflow
- [KYUUBI #6145] Use HiveMetaStore Docker for testing
- [KYUUBI #6145][FOLLOWUP] Remove unused dependency commons-lang
- [KYUUBI #6151] Remove invalid matrix spark 3.1 in GA
- [KYUUBI #6153] Cut out Hive JDBC and Hive Service deps in beeline module
- [KYUUBI #6160] Fix beeline test KyuubiBeeLineTest.testKyuubiBeelineComment
- [KYUUBI #6167] Drop JDK 11 on GitHub Actions
- [KYUUBI #6170] Merge JDBC Trino TPC jobs and Style Check jobs in GitHub Actions workflow
- [KYUUBI #6173] Bump maven-download-plugin from 1.7.1 to 1.8.1
- [KYUUBI #6174] Remove unused dependency management from POM
- [KYUUBI #6177] Bump BouncyCastle from 1.67 to 1.77
- [KYUUBI #6178] Improve Gluten integration test
- GitHub Bot
- Shaoyun Chen
- GGHWS
- Deepali Gupta
- Baokun Han
- Denis Krivenko
- Kwafoor
- Pengqi Li
- Bowen Liang
- Paul Lin
- Senmiao Liu
- Chenliang Lu
- Tigran Manasyan
- Hai Nguyen
- Cheng Pan
- XH Qian
- Zhen Wang
- Fei Wang
- Kang Wang
- Junjie Xu
- Binjie Yang
- Kent Yao
- Xianxun Ye
- Kaifei Yi
- David Yuan
- He Zhao
- Yangming Zhao
- Yifan Zhou
- Mingliang Zhu
- Yi Zhu
- Yaogai Zhu
- huangxiaopingRD
- kevinjmh
- mrtisttt
- BingKun Pan
- rhh777
- silencily
- sincewhen
- Vino Yang
- [KYUUBI #6058] Make Jetty server stop timeout configurable
- [KYUUBI #6048] Assign serviceNode and add volatile for variables
- [KYUUBI #6045] Sync the AdminRestApi with the AdminResource Apis
- [KYUUBI #6028] Exited spark-submit process should not block batch submit queue
- [KYUUBI #6001] Fix RESTful protocol security enabled evaluation
- [KYUUBI #5999] loads the interval of sessionConfCache from configurable file in FileSessionConfAdvisor
- [KYUUBI #5990] Always take the first declared SASL/PLAIN auth type
- [KYUUBI #5922] Enhance YARN app state judgment
- [KYUUBI #5900] Get the engine service node to delete instead of delete engine space
- [KYUUBI #5900] Support to deregister the engine on open failure
- [KYUUBI #5892] Metadata store JDBC url supports <KYUUBI_HOME> variable
- [KYUUBI #5869] Add volatile to variables in KyuubiSessionImpl that may be operated by multiple threads
- [KYUUBI #5244] Make engineAliveMaxFailCount configurable
- [KYUUBI #5795] Using correct time unit for pod cleanup executor
- [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak
- [KYUUBI #5853] Ensure closing the operation log for batch submission in fast-failed case
- [KYUUBI #5833] Rename service registered endpoint key from serviceUri to serverUri
- [KYUUBI #5795] Support to cleanup the spark driver pod periodically
- [KYUUBI #5680] Optimize Spark engine pod name generation
- [KYUUBI #5807] Simple norm the use of
kyuubi.operation.result.arrow.timestampAsString
andkyuubi.operation.result.format
- [KYUUBI #5767] Extract common utils for assembling key value pairs with config option prefix in processbuilder
- [KYUUBI #5765] Avoid array copy in generating process builder commands
- [KYUUBI #5771] Cleanup the terminated app info trigger before clear kubernetes clients
- [KYUUBI #5711] Fix typo for container states audit
- [KYUUBI #5711] Fix typo and audit pod and container states
- [KYUUBI #4847] Check engine alive for KyuubiSessionImpl only
- [KYUUBI #5761] Pretty ProcBuilder generated command
- [KYUUBI #5754] Fix engine id of the engine probe failures logging messages
- [KYUUBI #5730] Tolerate execeptions for periodical scheduled tasks
- [KYUUBI #5210] Close batch operation log even batch is in cancel state
- [KYUUBI #5731] Support to cleanup the spark driver pod with specified cleanup strategy
- [KYUUBI #5725] Handle the session exception for all kyuubi operations during running
- [KYUUBI #2503] Catch all exceptions for sessions timeout check task
- [KYUUBI #5717] Infer the proxy user automatically for delete batch operation
- [KYUUBI #5711] Support to build application state and error from pod container
- [KYUUBI #5714] Fix NPE when deleting terminated application pod
- [KYUUBI #5714] Support to cleanup the spark driver pod after application terminates for retain period
- [KYUUBI #5696] Return remote session id and remote operation id
- [KYUUBI #5654] Introduce new config to forcibly rewrite pod name
- [KYUUBI #5438] Add common method to get session level config
- [KYUUBI #5952] Disconnect connections without running operations after engine maxlife time graceful period
- [KYUUBI #6018] Speed up GetTables operation for Spark session catalog
- [KYUUBI #5966] Error occurs on clicking on the Spark UI Kyuubi Query Engine tab
- [KYUUBI #5831] Pre-allocate array list capacity for TColumns and TColumnValues in TRowSet generation
- [KYUUBI #5811] Reuse time formatters instance in value serialization of TRowSet generation
- [KYUUBI #5766] Default
spark.yarn.maxAppAttempts
to 1 for spark engine - [KYUUBI #5750] Fix the inaccurate issue of obtaining COLUMN_SIZE in the decimal field jdbc of spark engine
- [KYUUBI #5331] Spark engine should throw an exception when it fails to start
- [KYUUBI #5991] Error on reading Atlas properties composed of multi values
- [KYUUBI #5977] Fix argument position on constructing Trino ClientSession
- [KYUUBI #5969] Trino engine add default schema
- [KYUUBI #5878] Hive engine should recognize
hiveserver2-site.xml
- [KYUUBI #5756] Introduce specified initialized SQL to every engine
- [KYUUBI #5814] Skip the max-rows limitation if the statement is not DQL in Flink engine
- [KYUUBI #5799] Fix fetch timeout in session conf doesn’t support ISO-8601 in Flink engine
- [KYUUBI #5789] Flink engine kyuubi.session.engine.flink.fetch.timeout parameter on the server is not effective
- [KYUUBI #5784] Implement HiveTBinaryFrontendService#RenewDelegationToken
- [KYUUBI #5782] Flink Engine GetInfo support CLI_ODBC_KEYWORDS
- [KYUUBI #5778] Hive engine shall respect
hive.server2.enable.doAs
- [KYUUBI #5777] Include
hive-service-rpc
in Hive engine jar - [KYUUBI #5507] Support Initialize SQL in Flink Engine
- [KYUUBI #5280] Hive engine compatible with Hive 2.3
- [KYUUBI #6036] JDBC driver conditional sets fetchSize on opening session
- [KYUUBI #6016] Conditional call GetInfo CLI_ODBC_KEYWORDS to restore compatible with lower version Kyuubi and HS2
- [KYUUBI #5961] Prevent NPE when checking ticket cache exists
- [KYUUBI #5961] Support to specify client kerberosAuthType as fromTicketCache
- [KYUUBI #5849] Incorrectly parse JDBC URL while variable includes colon
- [KYUUBI #5822] Fix the usage msg of KyuubiBeeLine
- [KYUUBI #5752] JDBC driver implements getDate, getTimestamp and getTime with Calendar in ResultSet
- [KYUUBI #5713] Backport HIVE-27271: Client connection to HS2 fails when transportMode=http, ssl=true, sslTrustStore specified without trustStorePassword in the JDBC URL
- [KYUUBI #5981] Deploy Spark Hive connector with Scala 2.13 to Maven Central
- [KYUUBI #6047] Free up disk space
- [KYUUBI #5987] Always export KYUUBI_HOME in
load-kyuubi-env.sh
- [KYUUBI #5987] Enhance KYUUBI_HOME detection in shell script to handle softlink cases
- [KYUUBI #6000] Modify the incorrect configuration file in the trino-cli documentation
- [KYUUBI #5978] Canonicalize Trino IT in GitHub Action workflow
- [KYUUBI #5962] Bump Flink 1.17.1 to 1.17.2
- [KYUUBI #5953] Update NOTICE
- [KYUUBI #5951] Improve doc for Session Conf Advisor
- [KYUUBI #5933] Happy New Year 2024
- [KYUUBI #5244] Corrected ENGINE_ALIVE_MAX_FAILURES conf version
- [KYUUBI #5856] Bump Spark 3.4.2
- [KYUUBI #5857] Dependencies CI check when
dev/dependencyList
changes - [KYUUBI #5837] Only enable Jacoco with explicit profile
- [KYUUBI #5827] Fix wrong test code about directory lineage
- [KYUUBI #5819] Fix rst syntax of missing blank line after code-block
- [KYUUBI #5802] Add scala-2.13 as supported Scala profile to the docs of building from source
- [KYUUBI #5801] Add spark-3.4 and spark-3.5 as supported Spark profiles to the docs of building from source
- [KYUUBI #5785] Fix flaky test - JpsApplicationOperation with spark local mode
- [KYUUBI #5737] Correct JDBC URL demo in Flink docs
- [KYUUBI #5617] Make flaky test
spnego batch rest client
reliable - [KYUUBI #5720] Fix flaky test in TPCHQuerySuite
- [KYUUBI #5697] Fix incorrect url reference in code and docs
- [KYUUBI #5599] Correct the parameter name of fetching op log in RESTful API
- [KYUUBI #5641] Bump Kyuubi 1.8.0 in Helm charts
- [KYUUBI #5640] Upgrade playground to Kyuubi 1.8.0 and Spark 3.4.1
- [KYUUBI #5637] Add known contributor translation
- [KYUUBI #5624] Remove npm run build warning about NODE_ENV=production
- [KYUUBI #5634] Smoother the icon animation on collapsing sidebar
- [KYUUBI #5626] Fix sytle linting violation in web ui
- Shaoyun Chen
- Chao Chen
- Flyangz
- Pengqi Li
- Bowen Liang
- Paul Lin
- Senmiao Liu
- Xiao Liu
- Peiyue Liu
- Junjie Ma
- Ocean22
- Cheng Pan
- Gianluca Principini
- Zhiming She
- SwordyZhao
- William Tong
- Fei Wang
- Zhen Wang
- Tao Wang
- Zeyu Wang
- Kang Wang
- Binjie Yang
- Kaifei Yi
- David Yuan
- Yuwei Zhan
- He Zhao
- Yifan Zhou
- Mingliang Zhu
- Yi Zhu
- liaoyt
- mrtisttt
- 基于 Java 8, 11 和 17,Scala 2.12 做了充分验证,初步支持 Scala 2.13
- 基于 Spark 3.1, 3.2, 3.3 和 3.4 做了充分验证, 初步支持 Spark 3.5
- 基于 Flink 1.16, 1.17 和 1.18 做了充分验证
- 全新 Web UI (实验特性)
- 新增以 OpenAI ChatGPT 作为后端的 Chat 引擎
- Spark Batch API 重大改进,针对大规模作业提交进行了优化 (实验特性)
- Spark AuthZ 插件重大改进,包括增强了对 Iceberg 和 DSv2 算子的支持
- Flink 引擎重大改进,包括 YARN Application Mode 的支持
- Spark 血缘插件引入 Atlas hook 支持
- 持续改进 Kyuubi Helm Chart 和 Spark on K8s
- [KYUUBI #1265]
OPTIMIZE
where clause expression support - [KYUUBI #3420] Expose UI url on registering engine service
- [KYUUBI #3444] Support the planOnly mode of kyuubi spark engine support SQL lineage
- [KYUUBI #3929] Refactor lineage plugin to add LineageDispatcher
- [KYUUBI #4167] [AUTHZ] Introduce function support in PrivilegeBuilder with Serde layers
- [KYUUBI #4171] Support skip retrieving table’s properties to speed up GetTables operation
- [KYUUBI #4186] Spark showProgress with JobInfo
- [KYUUBI #4330] Non-temporary views do not resolve to a specific real table
- [KYUUBI #4331] [KYUUBI #4431] Lineage supports
Union
andMergeInto
- [KYUUBI #4415] Align the operation handle in server/engine for ExecuteScala, ExecutePython and PlanOnlyStatement
- [KYUUBI #4430] Lineage supports the
lateral view explode
statement - [KYUUBI #4433] Fix the case with
create a view from view
- [KYUUBI #4443] Do not set engine session init sql for alive probe session
- [KYUUBI #4450] Ignore unknown fields
policyPriority
when reading policy json file - [KYUUBI #4488] [KSHC] Keep object original name defined in HiveBridgeHelper
- [KYUUBI #4493] [AUTHZ] Enable Tests for Spark 3.1 with iceberg tables
- [KYUUBI #4497] Add RowFilteringTestBase to improve the test coverage
- [KYUUBI #4504] [AUTHZ] Fix source table privilege requirement when querying permanent view in Spark 3.1 and below
- [KYUUBI #4525] [KSHC] Partitioning predicates should take effect to filter data
- [KYUUBI #4530] [AUTHZ] Support non-English chars for MASK, MASK_SHOW_FIRST_4, and MASK_SHOW_FIRST_4
- [KYUUBI #4532] [AUTHZ] Displays the columns involved in extracting the aggregation operator
- [KYUUBI #4541] Support to customize the attributes to expose for Spark engine
- [KYUUBI #4560] [KSHC] Support Kerberized HMS in cluster mode w/o keytab
- [KYUUBI #4585] [AUTHZ] Authz policy file generation
- [KYUUBI #4587] Take care of table cache query stage
- [KYUUBI #4592] Support eagerly kill redundant executors
- [KYUUBI #4615] Support stage level schedule for final write stage
- [KYUUBI #4617] [AUTHZ] Collect results for filtered show objects ahead to prevent holding unserializable spark plan
- [KYUUBI #4620] [KSHC] Cut off transitive dependencies
- [KYUUBI #4636] Improve eagerly kill redundant executors
- [KYUUBI #4641] Add MaxFileSizeStrategy to limit max scan file size
- [KYUUBI #4644] Manually terminate the Py4JServer during engine shutdown
- [KYUUBI #4658] [AUTHZ] Fix InsertIntoHiveDirCommand classname so that we can extract the query in it when authorization.
- [KYUUBI #4662] [ARROW] Arrow serialization should not introduce extra shuffle for outermost limit
- [KYUUBI #4664] Fix empty relation when kill executors
- [KYUUBI #4672] [AUTHZ] Remove Support for Apache Ranger 0.6.0
- [KYUUBI #4676] [AUTHZ] Reuse users and namespaces in both tests and policy file generation
- [KYUUBI #4677] [AUTHZ] Check generated policy file in test suite
- [KYUUBI #4678] Improve FinalStageResourceManager kill executors
- [KYUUBI #4681] Set thread
CreateSparkTimeoutChecker
daemon - [KYUUBI #4683] Update
spark.sql.finalWriteStage.resourceIsolation.enabled
version - [KYUUBI #4702] [ARROW] CommandResultExec should not trigger job
- [KYUUBI #4710] [ARROW] LocalTableScanExec should not trigger job
- [KYUUBI #4710] [ARROW] Post driver-side metrics for LocalTableScanExec/CommandResultExec
- [KYUUBI #4716] [KYUUBI #4715][AUTHZ] Fix the incorrect class name of InsertIntoHiveDirCommand in table spec generator
- [KYUUBI #4717] [AUTHZ] Check Authz plugin’s spec json files in UT
- [KYUUBI #4720] [ARROW] Fix java.lang.NoSuchFieldError: IpcOption.DEFAULT for Spark-3.1/3.2
- [KYUUBI #4741] Kyuubi Spark Engine/TPC connectors support Spark 3.4
- [KYUUBI #4754] [ARROW] Use
KyuubiArrowConverters#toBatchIterator
instead ofArrowConverters#toBatchIterator
- [KYUUBI #4797] [ARROW] Reflective calls to the function
ArrowUtils#toArrowSchema
- [KYUUBI #4812] Generalize case transformation method for string type config entry
- [KYUUBI #4834] Reduce the scope of method references in Authz plugin cleanup shutdown hook
- [KYUUBI #4838] Fix spark operation exception leak in
withLocalProperties
method - [KYUUBI #4866] Add annotation for Iceberg tests in Authz plugin
- [KYUUBI #4869] [AUTHZ] Introduce table extractor for ResolvedIdentifier in Spark 3.4
- [KYUUBI #4871] [AUTHZ] Adapt plan changes for CreateNamespace and SetCatalogAndNamespace in Spark 3.4
- [KYUUBI #4873] [AUTHZ] Refactor Authz reflection with kyuubi-util’s DynMethods
- [KYUUBI #4874] [AUTHZ] Improve methods in AuthzUtils
- [KYUUBI #4875] [AUTHZ] Remove checking Spark v2 in tests since Spark v2 not supported
- [KYUUBI #4878] Delta support for Spark-3.4
- [KYUUBI #4880] [BUILD] Skip compiling useless files when making distribution
- [KYUUBI #4888] [AUTHZ] Remove filtering results for ShowDatabasesCommand in Spark 2.x
- [KYUUBI #4892] [AUTHZ] Make identifier part name comparison case insensitive in tests of PrivilegeBuilder
- [KYUUBI #4895] [ARROW] Reflective calls to the function
ArrowConverters#fromBatchIterator
- [KYUUBI #4899] [AUTHZ] Extract function from FunctionIdentifier for CreateFunction and DropFunction in Spark 3.4
- [KYUUBI #4903] [AUTHZ] Fix NoSuchElementException when listing database in CatalogImpl in Spark 3.4
- [KYUUBI #4909] [AUTHZ] Enable authz plugin tests for Spark 3.4
- [KYUUBI #4910] Extract table from ResolvedIdentifier for DropTable in Spark 3.4
- [KYUUBI #4913] Eliminate unnecessary output in ut “union an unmasked table”
- [KYUUBI #4914] [AUTHZ] Reuse extractor singleton instance with generalized getter for supported extractor types
- [KYUUBI #4915] Adapt database not exist error message change for Spark 3.4
- [KYUUBI #4916] [AUTHZ] Support ReplaceData and compatible Spark 3.4 and 3.5
- [KYUUBI #4916] Fix Iceberg merge into UT
- [KYUUBI #4917] [AUTHZ] Table owner undefined in Iceberg 1.3.0 on Spark 3.4
- [KYUUBI #4923] [ARROW] Update arguments of
ArrowUtils#toArrowSchema
function - [KYUUBI #4925] Add default catalog using
spark_catalog
with the lineage result - [KYUUBI #4935] More than target num of executors may survive after FinalStageResourceManager did kill
- [KYUUBI #4937] Cleanup spark catalog shim and renamed to catalog utils
- [KYUUBI #4952] Enhance AWS Glue default database does not exist error message on opening session
- [KYUUBI #4995] Use hadoop conf and hive conf from catalog options
- [KYUUBI #4999] [KSHC] Kyuubi-Spark-Hive-Connector support Apache Spark 3.4
- [KYUUBI #5002] Fail the engine fast when no incoming connection in CONNECTION mode
- [KYUUBI #5005] Remove default settings
spark.sql.execution.topKSortFallbackThreshold
- [KYUUBI #5007] Remove blank SparkSimpleStatsReportListener
- [KYUUBI #5009] Pass Spark Engine Log Path to Spark Conf
- [KYUUBI #5017] [KSHC] Support Parquet/Orc provider is splittable
- [KYUUBI #5018] Make kyuubi spark extension compatible with Spark 3.4
- [KYUUBI #5022] [KSHC] CreateTable should use the correct provider
- [KYUUBI #5023] [KSHC] TableIdentify don’t attach catalog
- [KYUUBI #5025] Use formatDuration instead of formatDurationVerbose in Spark UI’s engine tab
- [KYUUBI #5028] Update session hadoop conf to catalog hadoop conf
- [KYUUBI #5030] Support get query id in Spark engine
- [KYUUBI #5035] Spark engine session page display session end time and duration
- [KYUUBI #5045] Initialize EXECUTOR_POD_NAME_PREFIX_MAX_LENGTH before using
- [KYUUBI #5055] [AUTHZ] Support building function privileges in Spark 3.4
- [KYUUBI #5073] Correct the method name in SparkSQLLineageParserHelperSuite
- [KYUUBI #5075] Refine tests to adapt Spark 4.0
- [KYUUBI #5076] Add KDF engine_url
- [KYUUBI #5105] GetColumns gets duplicate column names when Spark integrates with Iceberg
- [KYUUBI #5136] Spark App may hang forever if FinalStageResourceManager killed all executors
- [KYUUBI #5148] Improve spark.driver.host assignment in Spark on K8s client mode
- [KYUUBI #5170] Identifier compatible with Spark 3.4
- [KYUUBI #5172] [AUTHZ] Check USE permissions for DESCRIBE FUNCTION
- [KYUUBI #5180] Replace deprecated classes by equivalent in scala.tools.nsc.interpreter package
- [KYUUBI #5184] Rename Kyuubi’s StageInfo to SparkStageInfo to fix class mismatch
- [KYUUBI #5188] Make server module and Spark sql engine module compilable on Scala 2.13
- [KYUUBI #5189] [AUTHZ] Make spark authz module compilable on Scala 2.13
- [KYUUBI #5191] Make Spark extension plugin compilable on Scala 2.13
- [KYUUBI #5192] Make Spark sql lineage plugin compilable on Scala 2.13
- [KYUUBI #5193] Make Spark hive connector plugin compilable on Scala 2.13
- [KYUUBI #5195] Make Spark TPC-H connector plugin compilable on Scala 2.13
- [KYUUBI #5196] Enable CI Test on Scala 2.13 and support custom or spark-core extracted Scala version for Spark’s engine
- [KYUUBI #5196] Extract spark core scala version lazily and respect engine env
- [KYUUBI #5199] Read all columns of metadata to prevent column missing
- [KYUUBI #5202] Switch to Spark 3.4 by default
- [KYUUBI #5224] Modify the README file of authz to make spark 3.4 the default version
- [KYUUBI #5225] [KSHC] Unify the exception handling of v1 and v2 during dropDatabase
- [KYUUBI #5232] In SparkOperation#cleanup always calls cancelJobGroup even though it’s in the completed state
- [KYUUBI #5257] [LINEAGE] Adapt
CreateTableAsSelect
plan to Spark 3.5 changes - [KYUUBI #5268] [AUTHZ] Extract method for enabling authorization in single call mode
- [KYUUBI #5269] Duplicated AnalyzesTablesCommand in kyuubi authz
- [KYUUBI #5271] AnalyzeTableCommand should also add table write privilege
- [KYUUBI #5291] Verify Spark engine is compatible with Spark 3.5.0
- [KYUUBI #5317] [KSHC] Hive Connector throws NotSerializableException on reading Hive Avro partitioned table
- [KYUUBI #5336] Spark extension supports Spark 3.5
- [KYUUBI #5339] Avoid using
kyuubi.session.engine.initialize.timeout
string - [KYUUBI #5414] [KSHC] Reader should not pollute the global hiveConf instance
- [KYUUBI #5426] [KSHC] Avoid use class.newInstance directly
- [KYUUBI #5550] Optimizing TPC-DS dataset generation for 10x speedup
- [KYUUBI #1652] Support Flink YARN application mode
- [KYUUBI #4367] Support Flink 1.17
- [KYUUBI #4387] Remove support for Flink 1.14
- [KYUUBI #4446] Fix connections blocked by Flink insert statements
- [KYUUBI #4495] Support Flink job management statements
- [KYUUBI #4517] Fix multiple executions lead to abnormal results on Flink 1.14
- [KYUUBI #4635] Support flink time type in query operation
- [KYUUBI #4666] Support flink varbinary type in query operation
- [KYUUBI #4745] Support Flink’s LocalZonedTimestamp DataType
- [KYUUBI #4806] Support time-out incremental result fetch for Flink engine
- [KYUUBI #4861] Support Flink session idleness
- [KYUUBI #4938] Implement Kyuubi UDF in Flink engine
- [KYUUBI #5080] Fix EmbeddedExecutorFactory not thread-safe during bootstrap
- [KYUUBI #5106] Improve logs for fatal errors
- [KYUUBI #5108] Fix FileNotFoundException during Flink engine bootstrap
- [KYUUBI #5160] Refactor getNextRowSetInternal to support fetch streaming data
- [KYUUBI #5190] Explicitly name Flink bootstrap SQL in application mode
- [KYUUBI #5200] Optimize Flink application name generating
- [KYUUBI #5222] Remove unused provided dependencies in Flink SQL engine
- [KYUUBI #5238] Fix credentials may break Flink engine launch command
- [KYUUBI #5315] Propagate HIVE_CONF_DIR on launching Flink engine
- [KYUUBI #5405] Support Flink 1.18
- [KYUUBI #5531] Fix flaky FlinkOperationOnYarnSuite by enlarging the max rows setting
- [KYUUBI #3374] Support password authentication for Trino engine
- [KYUUBI #4412] Align the server/engine session handle for flink/hive/trino/jdbc engines
- [KYUUBI #4491] Fix Trino typo
- [KYUUBI #4522]
use:catalog
should execute before thanuse:database
- [KYUUBI #4544] Initial implement Kyuubi Chat Engine
- [KYUUBI #4548] Kyuubi Chat Engine supports Chinese questions and HTTP proxy
- [KYUUBI #4554] Code improvement in ChatGPTProvider
- [KYUUBI #4556] Refactor ChatGPTProvider to use
openai-java
client - [KYUUBI #4558] Make ChatGPT model ID configurable
- [KYUUBI #4559] Pretty chat engine launch command
- [KYUUBI #4711] JDBC client should catch task failed exception instead of NPE in the incremental mode
- [KYUUBI #4809] Set session user in ChatGPT request
- [KYUUBI #4810] Request and use a single choice for chat completion
- [KYUUBI #4881] JDBCEngine performs initialization sql
- [KYUUBI #4940] Implement Kyuubi UDF in Hive engine
- [KYUUBI #5120] Allow using session’s user and password for connecting database in JDBC engine
- [KYUUBI #5282] Support configure Trino session conf in
kyuubi-default.conf
- [KYUUBI #5333] Adaptive call fetchLaunchEngineResult
- [KYUUBI #5570] Fix memory leak when using incremental collect mode in JDBC engine
- [KYUUBI #3653] AdminResource add list Kyuubi server api
- [KYUUBI #3739] Remove unused parameters in SessionsResource rest docs
- [KYUUBI #3872] Skip setting proxy user when keytab is provided on batch session
- [KYUUBI #3876] Update the rest api docs for open session response
- [KYUUBI #3920] Fix flaky test - distribute lock
- [KYUUBI #3957] Rename the config prefix from kyuubi.server.batch.limit to kyuubi.server.limit.batch
- [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
- [KYUUBI #4344] Expose exec pool work queue size metrics
- [KYUUBI #4364] Add metrics for user opened connections with session type
- [KYUUBI #4376] Support to config the Kyuubi service administrator with Kyuubi conf
- [KYUUBI #4388] Limit the max rows for get nextRowSet api
- [KYUUBI #4404] Support to list/close sessions in AdminResource
- [KYUUBI #4407] Adapt SLF4J2
- [KYUUBI #4424] Catch No Node Exception, when list kyuubi engines
- [KYUUBI #4439] Add list/close operation method for AdminResource
- [KYUUBI #4462] Fix variable usage issue in
SessionManager#stop
- [KYUUBI #4464] Simplify and improve log for JDBCMetadataStore
- [KYUUBI #4467] Tolerate Driver Pod ephemerally invisible after submitting
- [KYUUBI #4472] add session/operation methods in AdminRestApi
- [KYUUBI #4480] Engine alive probe should close thrift connection on engine lost
- [KYUUBI #4482] Support to disable batch resource upload function
- [KYUUBI #4483] Enable REST frontend protocol by default
- [KYUUBI #4496] Call
super.clusterManager()
whenbatchConf
does not havespark.master
- [KYUUBI #4498] Support to list sessions/operations with conditions in AdminRestApi
- [KYUUBI #4501] Allow administrator to specify the user for engine admin
- [KYUUBI #4506] Return sessionType in session data
- [KYUUBI #4516] Return kyuubi instance in SessionData and OperationData
- [KYUUBI #4518] Return engine id in SessionData and post kyuubi instance in KyuubiOperationEvent
- [KYUUBI #4519] Update metadata when batch application state changed
- [KYUUBI #4523] support close engine session gracefully
- [KYUUBI #4526] Defer evaluation for GroupProvider
- [KYUUBI #4540] Support to deny some users to make connection
- [KYUUBI #4540] Create session limiter if user unlimited list or deny list nonempty
- [KYUUBI #4563] Format error log output in case of metrics json file not found
- [KYUUBI #4575] Fix the empty last line may causes the session to exit directly
- [KYUUBI #4579] Fix the unstable situation of test cases for Trino-Fronted-Service
- [KYUUBI #4584] Post the session exception to EventBus in time
- [KYUUBI #4594] Support PUT method in REST client
- [KYUUBI #4611] support all sessions api in SessionRestApi
- [KYUUBI #4618] Admin Resource list operations with sessionHandle filter
- [KYUUBI #4623] KubernetesApplicationOperation uses Informer instead of list
- [KYUUBI #4639] Support to specify confOverlay when executing statement with RESTful API
- [KYUUBI #4652] Fix JaasConfiguration ClassNotFoundException for Hadoop 3.3.4 and previous
- [KYUUBI #4653] [KYUUBI 4650] LogDivertAppender supports reading RollingFileAppender pattern
- [KYUUBI #4669] LDAP authentication allows auth user contains domain when bind.dn/pw enabled
- [KYUUBI #4688] Fix the failure to read the operation log after executing catalog and database operation
- [KYUUBI #4691] Configure FAIL_ON_UNKNOWN_PROPERTIES to false for KyuubiScalaObjectMapper
- [KYUUBI #4713] Fix false positive result in SchedulerPoolSuite
- [KYUUBI #4719] Support submission timeout for yarn application manager and get the applicationInfo in-memory
- [KYUUBI #4731] Support batch session conf advisor
- [KYUUBI #4733] Introduce Kafka event logger for server events
- [KYUUBI #4738]
AdminResource.getEngineSpace
should use primary group name on GROUP share level - [KYUUBI #4739] Add operation lock instead of locking state Enumeration
- [KYUUBI #4746] Do not recreate async request executor if has been shutdown
- [KYUUBI #4749] Fix flaky test issues in SchedulerPoolSuite
- [KYUUBI #4753] KyuubiConf.unset should not log deprecation warning
- [KYUUBI #4767] Correct the submit time for BatchJobSubmission and check applicationInfo if submitted application
- [KYUUBI #4777] Deregister event handlers when stopping server with event handler made auto-closeable
- [KYUUBI #4780] Get engine application info with interval to prevent frequent call to resource manager
- [KYUUBI #4786] Support yarn-client and yarn-cluster for YarnApplicationOperation
- [KYUUBI #4790] Initial implement Batch V2
- [KYUUBI #4791] Add helper method to simplify REST enabled judgment
- [KYUUBI #4792] Enhance hardcode session keywords and remove unused code
- [KYUUBI #4796] Expose JVM attributes to metrics
- [KYUUBI #4798] Allows BatchJobSubmission to run in sync mode
- [KYUUBI #4801] Using different engine submit timeout config for kubernetes and yarn
- [KYUUBI #4807] Fix flaky test “basic batch rest client”
- [KYUUBI #4807] Retry more times to fix flaky test “spnego batch rest client”
- [KYUUBI #4811] Do not update app info after batch or application terminated
- [KYUUBI #4813] Add the fetch orientation parameter to the /v1/operations/:operationId/log interface
- [KYUUBI #4814] Introduce Apache Atlas hook support in lineage plugin
- [KYUUBI #4816] Correct the implementation of cleanup K8s terminated appInfo
- [KYUUBI #4829] Support to expose operation metrics
- [KYUUBI #4830] Fix flaky test “support to interrupt the thrift request if remote engine is broken”
- [KYUUBI #4835] Using hive conf to check whether to apply HIVE_DELEGATION_TOKEN
- [KYUUBI #4836] Set UncaughtExceptionHandler for thread to log exception
- [KYUUBI #4840] Return cached appInfo iif both op and app are terminated
- [KYUUBI #4841] Log more info for closing session
- [KYUUBI #4842] Fix typo in etcd client config
- [KYUUBI #4843] Support multiple kubernetes contexts and namespaces
- [KYUUBI #4847] Close the session immediately when engine corrupt
- [KYUUBI #4847] Close the session immediately when engine connection closed
- [KYUUBI #4850] Change default EC spec to secp521r1 for internal session variable signature
- [KYUUBI #4855] Change socket “connection refused” message match pattern
- [KYUUBI #4857] Fix flaky test TFrontendServiceSuite
- [KYUUBI #4858] Relax test message assertion in “sync query causes engine crash”
- [KYUUBI #4859] HttpException message should not be null
- [KYUUBI #4865] Fix flaky test BackendServiceMetricSuite
- [KYUUBI #4876] fix Flaky test BackendServiceMetricSuite (#4924)
- [KYUUBI #4889] Admin command line supports list server command
- [KYUUBI #4890] Return operation metrics in OperationData and fix typo
- [KYUUBI #4893] Prevent null collection for rest dto
- [KYUUBI #4898] Fix logOperation multiple read with missing line
- [KYUUBI #4919] Fix the flaky tests in the “open batch session” functionality within the “BatchesResourceSuite”
- [KYUUBI #4920] Add OperationRestApi
- [KYUUBI #4946] Alter the order of initLoggerEventHandler
- [KYUUBI #4949] For operation getNextRowSet method, with operation lock required
- [KYUUBI #4950] Migrate Kyuubi embedded database from Derby to SQLite
- [KYUUBI #4950] Misc improvement for usage of SQLite
- [KYUUBI #4950] SQLite schema files should start from 1.8.0
- [KYUUBI #4951] Add reload4j support
- [KYUUBI #4977] Log error message when REST API invocation error occurs
- [KYUUBI #4978] Fix flaky test: close expired operations
- [KYUUBI #4978] Fix flaky test: close expired operations
- [KYUUBI #4979] Fix flaky test: support to interrupt the thrift request if remote engine is broken
- [KYUUBI #4981] Refactor code of closeBatchSession
- [KYUUBI #4985] Fix log level restore behavior of
KyuubiFunSuite#withLogAppender
- [KYUUBI #4986] Always use Files#deleteIfExists
- [KYUUBI #4994] Respect the engine type and share level for listing all engines
- [KYUUBI #4994] Support listing all engines
- [KYUUBI #4996] Support to refresh kubernetes configs dynamically
- [KYUUBI #5004] Fix typo for kubernetes allowed context and namespace check
- [KYUUBI #5010] Make Kyuubi server’s connection URL configurable
- [KYUUBI #5029] Close the alive probe session after engine session closed
- [KYUUBI #5036] Fix Operation.close not update complete timestamp
- [KYUUBI #5043] Destroy the build process when waitCompletion is false and the engine is running in cluster mode
- [KYUUBI #5050] Add admin checking in
getProxyUser
- [KYUUBI #5056] Return more helpful error message to http client
- [KYUUBI #5063] Support to filter batch with batch name
- [KYUUBI #5065] Call destroy first on killing Spark startup process to allows it release temp files
- [KYUUBI #5065] Graceful close the process when launch engine timeout
- [KYUUBI #5078] Make
kyuubi_instance
nullable in metadata table schema - [KYUUBI #5081] Minor refactor JDBCMetadataStore
- [KYUUBI #5089] Limit maximum engine startup concurrency of kyuubi server
- [KYUUBI #5130] Allow setting Zookeeper SPN using Kyuubi configuration
- [KYUUBI #5131] Create index on
metastore
.create_time
- [KYUUBI #5138] Fix flaky test BatchesResourceSuite - get batch session list
- [KYUUBI #5145] Change embedded Zookeeper server to method-local variable
- [KYUUBI #5158] Allow embedded Zookeeper binding IP address
- [KYUUBI #5165] Build Spark Driver/Executor Pod Name(Prefix) in process
- [KYUUBI #5185] Use Set collection for order-insensitive configs
- [KYUUBI #5187] Fix info message in UserFilter
- [KYUUBI #5196] Extract spark core scala version lazily and respect engine env
- [KYUUBI #5201] Allow disabling batch impl v2 on globally
- [KYUUBI #5204] Thrift HTTP FE checks req conf nullable before evaluating proxy user
- [KYUUBI #5206] Try to kill pod with label if no ApplicationInfo found to prevent pod leak
- [KYUUBI #5210] Cancel operation will cause the log file node to leak
- [KYUUBI #5213] Check config value by enum values
- [KYUUBI #5216] Workaround for negative counter in SessionLimiter
- [KYUUBI #5218] Improve logs of KubernetesApplicationOperation
- [KYUUBI #5220] Batch submitter should only block submitting stage
- [KYUUBI #5220] Batch submitted considers application state
- [KYUUBI #5243] Distinguish metadata between batch impl v2 and recovery
- [KYUUBI #5289] RESTful API should always print audit log
- [KYUUBI #5290] Batch impl v2 should pick batch jobs in FIFO
- [KYUUBI #5300] Batch v2 should not update kyuubi_instance when open session
- [KYUUBI #5305] Return dummy log when batch job before being added to session manager
- [KYUUBI #5306] YarnApplicationOperation supports proxy user
- [KYUUBI #5310] Batch session recovery should start after HTTP server getting started
- [KYUUBI #5327] Add priority field in batch session metadata table definition
- [KYUUBI #5328] Batch supports priority scheduling
- [KYUUBI #5340] AdminResourceSuite make engine idle time 3min
- [KYUUBI #5380] Create PySpark batch jobs tests for RESTful API
- [KYUUBI #5381] Change the default metrics reporter to Prometheus
- [KYUUBI #5391] Close batch session after batch operation completion for idle timeout
- [KYUUBI #5392] Add query timeout monitor on server-side in ExecuteStatement
- [KYUUBI #5408] MetadataManager tries MySQL 8 driver class first
- [KYUUBI #5412] Resolve the relative zk configuration dir based on KYUUBI_HOME
- [KYUUBI #5441] Make the configuration kyuubi.zookeeper.embedded.data.log.dir effective
- [KYUUBI #5451] Ignore NoSuchFileException during OperationLog.close()
- [KYUUBI #5499] [KYUUBI #2503] Catch any exception when closing idle session
- [KYUUBI #5513] Always redirect delete batch request to Kyuubi instance that owns batch session
- [KYUUBI #5522] Ignore main class for PySpark batch job submission
- [KYUUBI #5528] Remove duplicate properties tag and respect KYUUBI_LOG_DIR
- [KYUUBI #5566] InternalRestClient respects
kyuubi.engine.security.enabled
to add HTTP auth header - [KYUUBI #5566] Check InternalSecurityAccessor is initialized only when
kyuubi.engine.security.enabled
is true - [KYUUBI #5568] Check administrator only when security is enabled
- [KYUUBI #5568] Fix security enabled judgement
- [KYUUBI #5606] Handle engine listing request properly for users who have not created engine
- [KYUUBI #3887] Fix kyuubiServerPrincipal logic in KyuubiCommands
- [KYUUBI #4305] Backport HIVE-15820: comment at the head of beeline -e
- [KYUUBI #4438] Render the batch info map in kyuubi-ctl
- [KYUUBI #4439] Add dto class for operation data
- [KYUUBI #4452] Strip the redundant leading and tailing slash of getZooKeeperNamespace.
- [KYUUBI #4479] Restore JDBC Kerberos authentication behavior for UGI.doAs
- [KYUUBI #4489] Fix missing
org.apache.commons.logging
by includingjcl-over-slf4j
dependency in REST client - [KYUUBI #4492] Correct engine subdomain calculation in
kyuubi-ctl
- [KYUUBI #4508] Beeline should reset stdin after consuming init SQL file
- [KYUUBI #4581] Add KyuubiCommands parse python snippets unit test
- [KYUUBI #4609] get engineRefId in KyuubiConnection
- [KYUUBI #4619] Fix beeline with -e when there are other SQL statements before the source statement, the source statement cannot be executed normally
- [KYUUBI #4657] Building rest client to kyuubi instance including original host urls
- [KYUUBI #4962] Backport HIVE-19048: init script errors are ignored
- [KYUUBI #4965] Beeline supports
--python-mode
option and remove comments for non-python mode - [KYUUBI #4969] Run JUnit tests on beeline module
- [KYUUBI #5054] Refine the kyuubi client version properties file to prevent conflicts
- [KYUUBI #5100] Fix unpaired quote in message pattern
- [KYUUBI #5101] Remove unused StringBuilders in ResultSets
- [KYUUBI #5129] KyuubiBeeline should redirect JDK logging
- [KYUUBI #5179] Use Iterable instead of Seq in kyuubi-ctl commands
- [KYUUBI #5297] New RetryableClient get http://null server uri from metadata.kyuubiInstance
- [KYUUBI #5480] Support setting kyuubi hive jdbc client protocol version
- [KYUUBI #5564] Support to config ticket cache location in JDBC URL
- [KYUUBI #3420] Kyuubi Server Proxy Engine UI
- [KYUUBI #3646] Front-end style should bracket same line
- [KYUUBI #3646] Init Session Statistic Page
- [KYUUBI #3648] Add Session Detail Page
- [KYUUBI #3650] Add Operation Statistics Page
- [KYUUBI #3652] Add Kyuubi Server Management page
- [KYUUBI #3654] Add Engine Manager Page
- [KYUUBI #4537] Enable Vite proxy server for web ui development
- [KYUUBI #4565] Fix out-date
README.md
aboutDevelopment Project
- [KYUUBI #4622] Session api use admin api for security
- [KYUUBI #4626] Fix Web UI CI check fail with pnpm the lockfile needs updates
- [KYUUBI #4757] Move Statistics to Management Menu
- [KYUUBI #4982] Add query string to forward requests to Engine UI
- [KYUUBI #5259] Web UI supports Swagger pages
- [KYUUBI #5463] Adjust the file directory structure
- [KYUUBI #5484] Remove legacy Web UI
- [KYUUBI #5573] Delete parts of the Kyuubi Web UI that are not useful
- [KYUUBI #5586] Kyuubi Web UI show current version
- [KYUUBI #5608] Kyuubi version should always use the light color
- [KYUUBI #5620] Remove dummy information card from Overview page
- [KYUUBI #4308] Make README more welcoming
- [KYUUBI #4312] Fix over-matched README pattern for markdown styling
- [KYUUBI #4367] Kyuubi requires Flink 1.16/1.17
- [KYUUBI #4387] Update Flink version requirements
- [KYUUBI #4568] Make kyuubi-ctl doc enable variable ‘release’ automatic substitution
- [KYUUBI #4583] Rename Flink Table Store to Apache Paimon (Incubating)
- [KYUUBI #4593] Keep promote the release step update-to-date
- [KYUUBI #4599] Simplify project version evaluation
- [KYUUBI #4607] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Hive SQL Query Engine
- [KYUUBI #4608] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Trino SQL Query Engine
- [KYUUBI #4614] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Flink SQL Query Engine
- [KYUUBI #4624] Fix table headers in kyuubi_vs_hive.md
- [KYUUBI #4627] Fix a typo in rest_api.md.
- [KYUUBI #4655] Enrich docs for Kyuubi Hive JDBC driver
- [KYUUBI #4655] Recover the missing JDBC URL
- [KYUUBI #4722] Kyuubi requires Spark 3.1 and above now
- [KYUUBI #4727] kyuubi-spark-lineage has no transitive deps
- [KYUUBI #4734] Fix typo in docs of custom event handler
- [KYUUBI #4742] Add docs for Flink application mode
- [KYUUBI #4763] Fix the Kyuubi JDBC kerberos parameters
- [KYUUBI #4800] Update readthedocs.yaml
- [KYUUBI #4864] Fix grammar in HA section
- [KYUUBI #4933] Mark
spark.sql.optimizer.insertRepartitionNum
config for Spark 3.1 only - [KYUUBI #4954] Fix variable reference in quick_start_with_jdbc
- [KYUUBI #4960] Fix doc for pyhive client.
- [KYUUBI #5049] PyHive Kerberos usage doc
- [KYUUBI #5083] Add LDAP document
- [KYUUBI #5085] Update config section of deploy on kubernetes
- [KYUUBI #5094] Add Python JayDeBeApi doc
- [KYUUBI #5099] Document link failure
- [KYUUBI #5118] Add Flink KDF doc
- [KYUUBI #5122] Hive KDF usage
- [KYUUBI #5127] Improvements for markdown builder
- [KYUUBI #5146] Fix link of IntelliJ IDEA Setup Guide
- [KYUUBI #5153] Minor change w/ to with
- [KYUUBI #5154] Move configuration docs to the top level
- [KYUUBI #5162] Add statements about contributions co-authored with AIGC in developer guide page
- [KYUUBI #5178] Make MarkdownBuilder not extending Growable
- [KYUUBI #5227] Added deploy mode info
- [KYUUBI #5247] Correct building command for Spark authz/lineage plugins
- [KYUUBI #5273] Update release docs
- [KYUUBI #5275] Improve and fix comparison and regeneration for golden files
- [KYUUBI #5294] Update supported dialects for JDBC engine
- [KYUUBI #5335] Set markdown file EOL
- [KYUUBI #5345] Fix non-kerberized code description in quick_start_with_jdbc
- [KYUUBI #5347] Fix java code errors in quick_start_with_jdbc
- [KYUUBI #5409] Update config docs regeneration hints in developer guide
- [KYUUBI #5432] Fix typo in README.md
- [KYUUBI #5523] Update the Kyuubi supported components version
- [KYUUBI #4078] Fix shellcheck violations in scripts of /bin folder
- [KYUUBI #4187] Windows build-info script supports revision time
- [KYUUBI #4278] Use new Apache ‘closer.lua’ syntax to obtain Maven
- [KYUUBI #4295] Introduce
super-linter
action for linting JSON, XML, ENV files and bash_exec - [KYUUBI #4299] Bump JUnit to 4.13.2
- [KYUUBI #4299] Bump MySQL JDBC driver to 8.0.32
- [KYUUBI #4315] Check modules available before build step in dependency workflow
- [KYUUBI #4347] Bump maven download plugin from 1.6.6 to 1.6.8
- [KYUUBI #4348] Cache engine archives in CI jobs for maven download plugin
- [KYUUBI #4397]
build/dist
supports--web-ui
- [KYUUBI #4413] Build docker image should include web UI
- [KYUUBI #4447] Bump log4j from 2.19.0 to 2.20.0
- [KYUUBI #4448] Bump Netty from 4.1.87.Final to 4.1.89.Final
- [KYUUBI #4451] Skip etcd test if no docker env
- [KYUUBI #4453] Bump Kubernetes Client Version to 6.4.1
- [KYUUBI #4461] Add
Kyuubi Kubernetes IT
Module to CI style check - [KYUUBI #4466] Use bitnami/minio docker image
- [KYUUBI #4471] Prompted command for docs generation should skip Java tests
- [KYUUBI #4484] [K8S][HELM] Set specific Kyuubi version to the chart appVersion
- [KYUUBI #4494]
bin/kyuubi
should useexec
to run Kyuubi server - [KYUUBI #4513] Bump Kyuubi 1.7.0 in Playground
- [KYUUBI #4546] Fully exclude
metrics
dir from git - [KYUUBI #4557] [K8S][HELM] Kyuubi server should bind Pod IP by default
- [KYUUBI #4572] Bump Iceberg from 1.1.0 to 1.2.0
- [KYUUBI #4576] Multi arch build for ARM support in Docker Hub
- [KYUUBI #4589] [K8S][HELM] Update template on port-forward usage guide NOTES.txt
- [KYUUBI #4590] Bump delta from 2.2.0 to 2.3.0
- [KYUUBI #4598] Bump Flink-1.15 from 1.15.3 to 1.15.4
- [KYUUBI #4600] Bump Iceberg 1.2.0 for playground
- [KYUUBI #4601] Bump Hadoop 3.3.5 for playground
- [KYUUBI #4605] [K8S][HELM] Set IfNotPresent pullPolicy by default
- [KYUUBI #4606] [K8S][HELM] Add command and args configuration support
- [KYUUBI #4615] Bump Ranger from 2.3.0 to 2.4.0
- [KYUUBI #4628] Add Dependency management ISSUE_TEMPLATE
- [KYUUBI #4632] Add rule for labeler to tag
module:ui
- [KYUUBI #4638] Update dependency.yml as label in not permitted in markdown block
- [KYUUBI #4652] Upgrade Hadoop from 3.3.4 to 3.3.5
- [KYUUBI #4695] Bump super-linter action from v4 to v5
- [KYUUBI #4696] Upgrade scalafmt from 3.7.1 to 3.7.3
- [KYUUBI #4697] [K8S][HELM] Add template rendering info to README
- [KYUUBI #4698] [K8S][HELM] Centralize Kyuubi labels definition
- [KYUUBI #4706] [K8S][HELM] Use template comments for the chart license header
- [KYUUBI #4707] [K8S][HELM] Fix README typo and minor reformatting
- [KYUUBI #4712] Bump Spark from 3.2.3 to 3.2.4
- [KYUUBI #4737] Restore Project & Community Status in README.md
- [KYUUBI #4744] Remove Hudi integration tests
- [KYUUBI #4747] Install and cache build/mvn in CI builds
- [KYUUBI #4768] Guidelines for Document and Code contributions
- [KYUUBI #4772] Bump Jersey from 2.39 to 2.39.1
- [KYUUBI #4783] Use pnpm v8 and update pnpm lockfile to v6 schema
- [KYUUBI #4784] Use nodejs v18 in CI builds and maven web-ui builds
- [KYUUBI #4788] [K8S][HELM] Use StatefulSet instead of Deployment
- [KYUUBI #4805] Bump openai-java from 0.11.1 to 0.12.0
- [KYUUBI #4808] Bump Jetty from 9.4.50 to 9.4.51
- [KYUUBI #4818] [K8S][HELM] Update default Kyuubi version to 1.7.1
- [KYUUBI #4819] [K8S][HELM] Refactor liveliness and readiness probes
- [KYUUBI #4824] Bump Jackson from 2.14.2 to 2.15.0
- [KYUUBI #4827] Bump Apache Arrow from 11.0.0 to 12.0.0
- [KYUUBI #4828] Exclude macOS tar extended metadata in build/dist
- [KYUUBI #4849] Open modules to enable JDK 17 support
- [KYUUBI #4851] Bump Maven 3.8.8
- [KYUUBI #4852] Switch to Kyuubi Shaded Zookeeper
- [KYUUBI #4856] Add
restore-keys
foraction/cache
- [KYUUBI #4870] Add kyuubi-util and kyuubi-util-scala modules
- [KYUUBI #4877] Bump Flink 1.17 to 1.17.1
- [KYUUBI #4879] Refactor and promote reflection utils and cleanup similar reflection methods
- [KYUUBI #4883] Bump Iceberg 1.3.0
- [KYUUBI #4887] Refactor and add ut for ClassUtils
- [KYUUBI #4891] Bump Flink 1.16.2
- [KYUUBI #4894] Bump Mockito from 4.9.0 to 4.11.0 and ScalaTest from 3.2.15 to 3.2.16
- [KYUUBI #4904] Move AssertionUtils to kyuubi-util-scala module
- [KYUUBI #4905] Generalize util method for loading class from service loader
- [KYUUBI #4907] Add a maven profile ‘remote-debug’
- [KYUUBI #4908] Bump scalafmt from 3.7.3 to 3.7.4
- [KYUUBI #4911] Remove duplicate test annotations
org.apache.kyuubi.tags.*
in kyuubi-common - [KYUUBI #4912] Replace Scala’s assert in tests with scalatest’s for prettified error message
- [KYUUBI #4931] Bump vite from 3.0.0 to 4.2.3 and vitest from 0.22.0 to 0.32.0
- [KYUUBI #4937] Remove redundant quoteIfNeeded method
- [KYUUBI #4943] Bump Guava from 31.1 to 32.0.1
- [KYUUBI #4944] Code improvement for Java
- [KYUUBI #4947] Bump pnpm version in maven’s frontend plugin to 8.6.1 for lockfileVersion 6.1
- [KYUUBI #4948] Reuse versions of NodeJS and pnpm from maven properties
- [KYUUBI #4959] Code improvements for Scala
- [KYUUBI #4966] Bump Apache parent pom from 29 to 30
- [KYUUBI #4968] Simplify Option conversion
- [KYUUBI #4970] Unified reflection methods invokeAs and getField
- [KYUUBI #4974] Eliminate raw-types compilation warning in KyuubiBeeLine
- [KYUUBI #4975] Fixed run test link in PR template
- [KYUUBI #4990] Bump Spark 3.4.1
- [KYUUBI #4992] Bump Netty 4.1.93.Final
- [KYUUBI #5001] Bump Scala from 2.12.17 to 2.12.18
- [KYUUBI #5003] [K8S][HELM] Add doc strings to the chart values.yaml
- [KYUUBI #5007] Bump scalafmt from 3.7.4 to 3.7.5
- [KYUUBI #5019] Shows details of compilation info in SparkUI’s engine tab
- [KYUUBI #5024] Bump grpc-protobuf from 1.48.0 to 1.53.0
- [KYUUBI #5034] Bump tough-cookie from 4.0.0 to 4.1.3 in /kyuubi-server/web-ui
- [KYUUBI #5039] Use semantic versions and remove redundant version comparison methods
- [KYUUBI #5058] [K8S][HELM] Allow the user to set the priorityClass used by Kyuubi server Pod
- [KYUUBI #5059] Upgrade kubernetes client version to 6.7.2
- [KYUUBI #5064] Audit kubernetes application state change in separate log files
- [KYUUBI #5068] Bump Spark in spark-master profile from 3.5.0-SNAPSHOT to 4.0.0-SNAPSHOT
- [KYUUBI #5070] Bump word-wrap from 1.2.3 to 1.2.4 in web UI
- [KYUUBI #5072] Fix KyuubiOperationWithEngineSecuritySuite and related issues
- [KYUUBI #5090] Fix AllKyuubiConfiguration to generate redundant blank lines in Windows
- [KYUUBI #5113] Remove dummy Spark Kudu connector module
- [KYUUBI #5116] Bump Hadoop 3.3.6
- [KYUUBI #5117] Code improvements for Java and Scala code
- [KYUUBI #5121] Extract dev scripts for regenerating the golden files
- [KYUUBI #5132] [K8S][HELM] Kyuubi add server metrics monitor
- [KYUUBI #5135] Exclude web-ui coverage reports in rat plugins
- [KYUUBI #5149] Correct error message of ReflectUtils’s invokeAs when method not found
- [KYUUBI #5150] Bump Spark 3.3.3
- [KYUUBI #5152] Check milestone and assignees when merging pull request
- [KYUUBI #5177] Use Scala binary version placeholder in Maven module’s artifactId suffix
- [KYUUBI #5183] Introducing profile scala-2.13 for development use
- [KYUUBI #5196] Enable CI Test on Scala 2.13 and support custom or spark-core extracted Scala version for Spark’s engine
- [KYUUBI #5202] Update upload_nexus_staging
- [KYUUBI #5205] Upgrade
kubernetes-client
to 6.8.1 - [KYUUBI #5209] Reformat Scala source code crossing versions
- [KYUUBI #5212] Fix configuration errors causing by helm charts of prometheus services
- [KYUUBI #5214] Correct file path for scala tests in util-scala module
- [KYUUBI #5234] Upgrade Jetty version from 9.4.51.v20230217 to 9.4.52.v20230823
- [KYUUBI #5235] Daily deploy snapshot version on master and branch-1.7
- [KYUUBI #5250] Bump Iceberg from 1.3.0 to 1.3.1
- [KYUUBI #5252] Remove incubator link
- [KYUUBI #5253] Upgrade testcontainers-scala from 0.40.12 to 0.41.0
- [KYUUBI #5253] Supply testcontainers-scala-scalatest deps
- [KYUUBI #5293] Upgrade snakeyaml from 1.33 to 2.2
- [KYUUBI #5305] Fix get local log UT
- [KYUUBI #5308] Ensure release using Java 8
- [KYUUBI #5318] Pin maven-source-plugin 3.2.1
- [KYUUBI #5358] Bump iceberg 1.4.0
- [KYUUBI #5365] Don’t use Log4j2’s extended throwable conversion pattern in default logging configurations
- [KYUUBI #5425] Add thread name and ID to log4j2-defaults.xml
- [KYUUBI #5435] Improve Kyuubi On Kubernetes IT
- [KYUUBI #5443] Add kyuubi-kubernetes-it module to reformat script
- [KYUUBI #5444] Check style with profiles of Spark 3.4 and 3.5
- [KYUUBI #5483] Fix deploy command for Scala 2.13 TPC-DS/H connectors
- [KYUUBI #5483] Release Spark TPC-H/DS Connectors with Scala 2.13
- [KYUUBI #5486] Bump Kafka client version from 3.4.0 to 3.5.1
- [KYUUBI #5508] Upgrade download-maven-plugin from 1.6.8 to 1.7.1
- [KYUUBI #5544] Release contains Spark 3.5 extension
- [KYUUBI #5561] Binary distribution artifact ships database schema scripts
- [KYUUBI #5561] Rename folder to db-scripts
- [KYUUBI #5569] Fix pnpm-lock file version
- Deng An
- GitHub Bot
- CavemanIV
- Fu Chen
- Shaoyun Chen
- Chao Chen
- Derick Cypho
- Yang Du
- FourSpaces
- Xu Guo
- Karsonnel
- Denis Krivenko
- Kyofin
- Pengqi Li
- Xieming Li
- Guanhua Li
- Bowen Liang
- Tianlin Liao
- Hanna Liashchuk
- Paul Lin
- Senmiao Liu
- Peiyue Liu
- Naijie Liu
- Chenliang Lu
- Ruilei Ma
- Omkar Makhare
- Drake Youngkun Min
- Sidhant Nagpal
- Cheng Pan
- QianyongY
- Anurag Rajawat
- Rojin Raju
- Fei Wang
- Zhen Wang
- Kang Wang
- Miao Wang
- Bruce Wong
- XorSum
- Junjie Xu
- Binjie Yang
- Remzi Yang
- Kent Yao
- Xianxun Ye
- Kaifei Yi
- Xiduo You
- Ruguo Yu
- David Yuan
- Qiang Zhang
- Yaodong Zhang
- Yangming Zhao
- He Zhao
- Min Zhao
- Yifan Zhou
- Yi Zhu
- Yaogai Zhu
- Alex Zou
- biangjuang
- camper42
- fantasticKe
- huangzhir
- huapan123456
- liaoyt
- mans2singh
- mrtisttt
- phyyou
- rhh777
- runzhliu
- touchida
- uicosp
- wangyepeng2
- weixi62961
- yehere
- 修复 Cancel 操作导致文件句柄泄漏的问题
- [KYUUBI #5210] Cancel operation will cause the log file node to leak
- [KYUUBI #5269] [Bug] Dupllicated AnalyzesTablesCommand in kyuubi authz
- AngersZhuuuu
- ASiegeLion
- 增强JDBC驱动和Beeline客户端:Beeline支持Python模式;修复HTTP认证异常消息缺失;修复HIVE-19048:正确传递初始化脚本错误;修复HIVE-15820:beeline -e无法执行注释开头的SQL
- 增强与引擎启动相关的代码:在YarnApplicationOperation中支持yarn-client/yarn-cluster模式;支持为多种调度器独立配置超时时间;当waitCompletion禁用时,仅在cluster模式下销毁spark-submit进程
- 修复大量不稳定的测试用例和文档错误
- [KYUUBI #4786] Support yarn-client and yarn-cluster for YarnApplicationOperation
- [KYUUBI #4801] Using different engine submit timeout config for kubernetes and yarn
- [KYUUBI #4811] Do not update app info after batch or application terminated
- [KYUUBI #4816] [K8S] Correct the implementation of cleanup terminated appInfo
- [KYUUBI #4830] [TEST] Fix flaky test “support to interrupt the thrift request if remote engine is broken”
- [KYUUBI #4840] Return cached appInfo iif both op and app are terminated
- [KYUUBI #4841] [MINOR] Log more info for closing session
- [KYUUBI #4807] [TEST] Retry more times to fix flaky test “spnego batch rest client”
- [KYUUBI #4807] [FOLLOWUP] Fix flaky test “basic batch rest client”
- [KYUUBI #4855] Change socket “connection refused” message match pattern
- [KYUUBI #4858] Relax test message assertion in “sync query causes engine crash”
- [KYUUBI #4919] Fix the flaky tests in the “open batch session” functionality within the “BatchesResourceSuite”
- [KYUUBI #4876] fix Flaky test BackendServiceMetricSuite
- [KYUUBI #4979] Fix flaky test: support to interrupt the thrift request if remote engine is broken
- [KYUUBI #4977] Log error message when REST API invocation error occurs
- [KYUUBI #4946] Alter the order of initLoggerEventHandler
- [KYUUBI #5072] [TEST] Fix KyuubiOperationWithEngineSecuritySuite and related issues
- [KYUUBI #5043] [1.7] Destroy the build process when waitCompletion is false and the engine is running in cluster mode
- [KYUUBI #3920] [TEST] Fix flaky test - distribute lock
- [KYUUBI #5204] Thrift HTTP FE checks req conf nullable before evaluating proxy user
- [KYUUBI #4838] Fix spark operation exception leak in
withLocalProperties
method - [KYUUBI #4835] [K8S] Using hive conf to check whether to apply HIVE_DELEGATION_TOKEN
- [KYUUBI #4915] Adapt database not exist error message change for Spark 3.4
- [KYUUBI #4952] Enhance AWS Glue default database does not exist error message on opening session
- [KYUUBI #5005] Remove default settings
spark.sql.execution.topKSortFallbackThreshold
- [KYUUBI #5025] Use formatDuration instead of formatDurationVerbose in SparkUI’s engine tab
- [KYUUBI #5019] Shows details of compilation info in SparkUI’s engine tab
- [KYUUBI #5045] Initialize EXECUTOR_POD_NAME_PREFIX_MAX_LENGTH before using
- [KYUUBI #5105] [Bug] getColumns gets duplicate column names when Spark integrates with Iceberg
- [KYUUBI #5148] Improve spark.driver.host assignment in Spark on K8s client mode
- [KYUUBI #5232] In SparkOperation#cleanup always calls cancelJobGroup even though it’s in the completed state
- [KYUUBI #4995] Use hadoop conf and hive conf from catalog options
- [KYUUBI #5028] Update session hadoop conf to catalog hadoop conf
- [KYUUBI #4859] HttpException message should not be null
- [KYUUBI #4969] [TEST] Run JUnit tests on beeline module
- [KYUUBI #4962] [1.7] Backport HIVE-19048: Initscript errors are ignored
- [KYUUBI #4965] [BEELINE] Support –python-mode option and remove comments for non-python mode
- [KYUUBI #4305] Backport HIVE-15820: comment at the head of beeline -e
- [KYUUBI #5100] [Minor] Fix unpaired quote in message pattern
- [KYUUBI #5101] [MINOR] Remove unused StringBuilders in ResultSets
- [KYUUBI #5129] KyuubiBeeline should redirect JDK logging
- [KYUUBI #4808] Bump Jetty from 9.4.50 to 9.4.51
- [KYUUBI #4836] Set UncaughtExceptionHandler for thread to log exception
- [KYUUBI #4857] Fix flaky test TFrontendServiceSuite
- [KYUUBI #4898] fix logOperation multiple read with missing line
- [KYUUBI #4949] For operation getNextRowSet method, with operation lock required
- [KYUUBI #4951] Add reload4j support
- [KYUUBI #4978] Fix flaky test: close expired operations
- [KYUUBI #4986] Always use Files#deleteIfExists
- [KYUUBI #4978] [FOLLOWUP] Fix flaky test: close expired operations
- [KYUUBI #4985] [TEST] Fix loglevel restore behavior of
KyuubiFunSuite#withLogAppender
- [KYUUBI #5036] Fix Operation.close not update complete timestamp
- [KYUUBI #5090] Fix AllKyuubiConfiguration to generate redundant blank lines in Windows
- [KYUUBI #5158] Allow embedded Zookeeper binding IP address
- [KYUUBI #5187] [Minor] Fix info message in UserFilter
- [KYUUBI #4818] [K8S] [HELM] Update default Kyuubi version to 1.7.1
- [KYUUBI #4828] [BUILD] Exclude macOS tar extended metadata in build/dist
- [KYUUBI #4851] [BUILD] Bump Maven 3.8.8
- [KYUUBI #4864] [DOCS] Fix grammar in HA section
- [KYUUBI #4954] [DOCS] Fix variable reference in quick_start_with_jdbc
- [KYUUBI #4960] [DOC] Fix doc for pyhive client.
- [KYUUBI #5049] [DOCS] PyHive Kerberos usage doc
- [KYUUBI #5085] Update config section of deploy on kubernetes
- [KYUUBI #5094] Add Python JayDeBeApi doc
- [KYUUBI #5099] Document link failure
- [KYUUBI #5083] [DOC] Add LDAP document
- [KYUUBI #5146] [DOC] Fix link of IntelliJ IDEA Setup Guide
- BruceWong96
- CavemanIV
- QianyongY
- bowenliang123
- cxzl25
- dev-lpq
- dnskr
- huangzhir
- lightning-L
- liunaijie
- lsm1
- mans2singh
- pan3793
- risyomei
- touchida
- turboFei
- uicosp
- ulysses-you
- wForget
- yaooqinn
- zhaohehuhu
- zhaomin1423
- Kyuubi 支持 Spark 3.1 及以上版本
- 依赖的 Hadoop 从 3.3.4 升级为 3.3.5
- 对 Arrow 序列化传输的一系列改进
- 对 Spark on K8s/YARN 批任务的一些列改进
- 修正 “beeline -e” 语句中 source 语句前SQL未执行的执行问题
- Spark Authz 权限插件修正Saprk3.1中永久视图下的源表权限检查
- [KYUUBI #4652][FOLLOWUP] Fix JaasConfiguration ClassNotFoundException for Hadoop 3.3.4 and previous
- [KYUUBI #3887][FOLLOWUP] Fix kyuubiServerPrincipal logic in KyuubiCommands
- [KYUUBI #4763][DOCS] Fix the Kyuubi JDBC kerberos parameters
- [KYUUBI #4738]
AdminResource.getEngineSpace
should use primary group name on GROUP share level - [KYUUBI #4655][DOCS][FOLLOWUP] Recover the missing JDBC URL
- [KYUUBI #4753][KYUUBI 4752][Improvement] KyuubiConf.unset should not log deprecation warning
- [KYUUBI #4407][FOLLOWUP] Adapt SLF4J2
- [KYUUBI #4746] Do not recreate async request executor if has been shutdown
- [KYUUBI #4739] Add operation lock instead of locking state Enumeration
- [KYUUBI #4719] Support submission timeout for yarn application manager and get the applicationInfo in-memory
- [KYUUBI #4731] Support batch session conf advisor
- [KYUUBI #4734][Docs] Fix typo in docs of custom event handler
- [KYUUBI #4711] JDBC client should catch task failed exception instead of NPE in the incremental mode
- [KYUUBI #4688] Fix the failure to read the operation log after executing catalog and database operation
- [KYUUBI #4652] Upgrade Hadoop from 3.3.4 to 3.3.5
- [KYUUBI #4691][REST] Configure FAIL_ON_UNKNOWN_PROPERTIES to false for KyuubiScalaObjectMapper
- [KYUUBI #4658][FOLLOWUP] Improve unit tests
- [KYUUBI #4669] LDAP authentication allows auth user contains domain when bind.dn/pw enabled
- [KYUUBI #4653][KYUUBI 4650][Improvement] LogDivertAppender supports reading RollingFileAppender pattern
- [KYUUBI #4623][FOLLOWUP] Fix malformed docs
- [KYUUBI #4522]
use:catalog
should execute before thanuse:database
- [KYUUBI #4655][DOCS] Enrich docs for Kyuubi Hive JDBC driver
- [KYUUBI #4644] Manually terminate the Py4JServer during engine shutdown
- [KYUUBI #4623][FOLLOWUP] Remove unused import
- [KYUUBI #4623][K8S] KubernetesApplicationOperation uses Informer instead of list
- [KYUUBI #3872][BATCH] Skip setting proxy user when keytab is provided
- [KYUUBI #4639] Support to specify confOverlay when executing statement with RESTful API
- [KYUUBI #4627][Docs] Fix a typo in rest_api.md.
- [KYUUBI #4626][UI][CI] Fix Web UI Ci check fail with pnpm the lockfile needs updates
- [KYUUBI #4624][Docs] Fix table headers in kyuubi_vs_hive.md
- [KYUUBI #4620][KSHC] Cut off transitive dependencies
- [KYUUBI #4599][DOCS] Simplify project version evaluation
- [KYUUBI #4480] Engine alive probe should close thrift connection on engine lost
- [KYUUBI #4593][DOCS] Keep promote the release step update-to-date
- [KYUUBI #4579] Fix the unstable situation of test cases for Trino-Fronted-Service
- [KYUUBI #4584] Post the session exception to EventBus in time
- [KYUUBI #4563][Improvement] Format error log output in case of metrics json file not found
- [KYUUBI #3739][FOLLOWUP][REST] Remove unused parameters in SessionsResource rest docs (#4542)
- [KYUUBI #4581][TEST] Add KyuubiCommands parse python snippets unit test
- [KYUUBI #4575] Fix the empty last line may causes the session to exit directly
- [KYUUBI #4568] Make kyuubi-ctl doc enable variable ‘release’ automatic substitution
- [KYUUBI #4526][FOLLOWUP] Defer evaluation for GroupProvider
- [KYUUBI #4519] Update metadata when batch application state changed
- [KYUUBI #4467][K8S][BATCH] Tolerate Driver Pod ephemerally invisible after submitting
- [KYUUBI #4496] Call
super.clusterManager()
whenbatchConf
does not havespark.master
- [KYUUBI #4494]
bin/kyuubi
should useexec
to run Kyuubi server - [KYUUBI #4491] Fix Trino typo
- [KYUUBI #4489][REST] Fix missing
org.apache.commons.logging
by includingjcl-over-slf4j
dependency in REST client - [KYUUBI #4488][KSHC] Keep object original name defined in HiveBridgeHelper
- [KYUUBI #4471] Prompted command for docs generation should skip Java tests
- [KYUUBI #4482] Support to disable batch resource upload function
- [KYUUBI #4452] Strip the redundant leading and tailing slash of getZooKeeperNamespace.
- [KYUUBI #4451] Skip etcd test if no docker env
- [KYUUBI #4433] Fix the case with
create a view from view
- [KYUUBI #4464] Simplify and improve log for JDBCMetadataStore
- [KYUUBI #4462] Fix variable usage issue in
SessionManager#stop
- [KYUUBI #4443] Do not set engine session init sql for alive probe session
- [KYUUBI #4754][ARROW] Use
KyuubiArrowConveters#toBatchIterator
instead ofArrowConveters#toBatchIterator
- [KYUUBI #4749] Fix flaky test issues in SchedulerPoolSuite
- [KYUUBI #4722][DOCS] Kyuubi requires Spark 3.1 and above now
- [KYUUBI #4720][ARROW] Fix java.lang.NoSuchFieldError: IpcOption.DEFAULT for Spark-3.1/3.2
- [KYUUBI #4713][TEST] Fix false positive result in SchedulerPoolSuite (#4714)
- [KYUUBI #4713][TEST] Fix false positive result in SchedulerPoolSuite
- [KYUUBI #4710][ARROW] LocalTableScanExec should not trigger job
- [KYUUBI #4702][ARROW] CommandResultExec should not trigger job
- [KYUUBI #4662][ARROW] Arrow serialization should not introduce extra shuffle for outermost limit
- [KYUUBI #4587][SPARK][EXT] Take care of table cache query stage
- [KYUUBI #4585] Authz policy file generation
- [KYUUBI #4658][Authz][Bug] Fix InsertIntoHiveDirCommand classname so that we can extract the query in it when authorization.
- [KYUUBI #4617][AUTHZ] Collect results for filtered show objects ahead to prevent holding unserializable spark plan
- [KYUUBI #4504][Authz][Bug] Fix source table privilege requirement when querying permanent view in Spark 3.1 and below
- [KYUUBI #4532][AUTHZ] Displays the columns involved in extracting the aggregation operator
- [KYUUBI #4450] Ignore unknown fields
policyPriority
when reading policy json file - [KYUUBI #4727][DOC] kyuubi-spark-lineage has no transitive deps
- [KYUUBI #4586][TEST][1.7] Fix to clean up views in lineage unit tests
- [KYUUBI #4525][KSHC] Partitioning predicates should take effect to filter data
- [KYUUBI #4331][KYUUBI #4431] Lineage supports
Union
andMergeInto
- [KYUUBI #4666] Support Flink varbinary type in query operation
- [KYUUBI #4635] Support Flink time type in query operation
- [KYUUBI #4614][DOCS] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Flink SQL Query Engine
- [KYUUBI #4607][DOCS] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Hive SQL Query Engine
- [KYUUBI #4608][DOCS] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Trino SQL Query Engine
- [KYUUBI #4598] Bump Flink-1.15 from 1.15.3 to 1.15.4
- [KYUUBI #4583][DOCS] Rename Flink Table Store to Apache Paimon (Incubating)
- [KYUUBI #4517][FLINK] Fix multiple executions lead to abnormal results on Flink 1.1
- [KYUUBI #4446] Fix connections blocked by Flink insert statements
- [KYUUBI #4619] Fix beeline with -e When there are other SQL statements before the source statement, the source statement cannot be executed normally
- [KYUUBI #4508][BEELINE] Beeline should reset stdin after consuming init SQL file
- [KYUUBI #4492] Correct engine subdomain calculation in
kyuubi-ctl
- [KYUUBI #4479] Restore JDBC Kerberos authentication behavior for UGI.doAs
- [KYUUBI #4438] Render the batch info map in kyuubi-ctl
- [KYUUBI #4706][K8S][HELM] Use template comments for the chart license header
- [KYUUBI #4707][K8S][HELM] Fix README typo and minor reformatting
- [KYUUBI #4606][K8S][HELM] Add command and args configuration support
- [KYUUBI #4605][K8S][HELM] Set IfNotPresent pullPolicy by default
- [KYUUBI #4589][K8S][HELM] Update template on port-foward usage guide NOTES.txt
- [KYUUBI #4557][K8S][HELM] Kyuubi server should bind Pod IP by default
- [KYUUBI #4484][K8S][HELM] Set specific Kyuubi version to the chart appVersion
- Alex
- Bowen Liang
- Chao Chen
- Cheng Pan
- Derick Cypho
- Fei Wang
- Fu Chen
- Karsonnel
- Paul Lin
- Ruguo Yu
- Tianlin Liao
- Xieming LI
- Yikf
- dnskr
- fwang12
- guanhua.lgh
- huangzhir
- maruilei
- miaowang
- minyk
- odone
- pengqli
- phyyou
- remzi
- senmiaoliu
- sychen
- ulysses-you
- wangyepeng2
- xu.guo
- zwangsheng
- 基于 Spark 3.1, 3.2 和 3.3 做了充分验证, 初步支持即将发布的 3.4
- 基于 Flink 1.14, 1.15 和 1.16 做了充分验证
- 新增 Trino 接入协议 (实验特性)
- 新增 Spark 血缘插件 (实验特性)
- 新增 Spark Hive 连接器 (实验特性)
- Spark 引擎支持基于 Arrow 序列化的高性能结果集传输
- Spark 引擎支持执行 Python/PySpark 脚本 (实验特性)
- 诸多 Kyuubi Helm Chart 和 Spark on K8s 改进
- REST 和 Batch 接口在功能、客户端和稳定性上进一步增强
- Spark 权限插件重大改进,并包含了对 Apache Iceberg 的支持
- 包括LDAP认证、内部通信加密在内的诸多安全改进
- [KYUUBI #2887] Add a POLLING balance policy for engine pool
- [KYUUBI #3089] Support custom event handler
- [KYUUBI #3449] Change default server info provider to ENGINE
- [KYUUBI #3545] [KYUUBI #3563] Support restrict spark configurations
- [KYUUBI #3554] REST API functions/command-line tool enhancements
- [KYUUBI #3577] Transfer connection url when opening session
- [KYUUBI #3615] Retry opening the engine when encountering a special error
- [KYUUBI #3658] Support SSL for Kyuubi thrift binary connection
- [KYUUBI #3659] Support alternative keys in ConfigBuilder
- [KYUUBI #3663] Support auto build Kubernetes client from env when Kyuubi running in Pod
- [KYUUBI #3742] Add FileSessionConfAdvisor to manage session level configuration
- [KYUUBI #3766] Support real user for KyuubiSession
- [KYUUBI #3835] Allow to use spark-internal as resource for batch job
- [KYUUBI #3839] Introduce signature mechanism to protect session variable on engine side
- [KYUUBI #3844] Forward the server ip in openSession
- [KYUUBI #3847] Add jdbc-shaded profile to support IDE debug
- [KYUUBI #3863] Arrow-based results serialization
- [KYUUBI #3867] Init SQL scripts should create table if table does not exist
- [KYUUBI #3887] Provide kyuubiServerPrincipal as alias for principal in JDBC parameter
- [KYUUBI #3897] Supplying pluggable GroupProvider
- [KYUUBI #3901] Introduce Trino frontend (experimental)
- [KYUUBI #3922] Only the ApplicationInfo with non-empty id is valid for BatchJobSubmission
- [KYUUBI #3923] Add dedicated batch session idle timeout
- [KYUUBI #3926] Introduce antlr4 to parse query statement
- [KYUUBI #3928] Application state mapping considers FinalApplicationStatus
- [KYUUBI #3950] Fix the batch metadata in-consistent issue on open batch session failure
- [KYUUBI #3975] Support to post batch session/operation event
- [KYUUBI #3983] [KYUUBI #3982] Introduce refreshing user defaults configs
- [KYUUBI #4021] Fix async start engine failure resulting in inaccurate operation metrics
- [KYUUBI #4106] Introduce resource file uploading in batch creation via REST API
- [KYUUBI #4119] Return app submission time for batch
- [KYUUBI #4145] Change lock and polling seq_num path on service discovery
- [KYUUBI #4151] Support to ignore subdomain when engine pool conditions are met
- [KYUUBI #4152] Enhance LDAP authentication
- [KYUUBI #4241] Only force close engine ref when open session failed
- [KYUUBI #4322] MySQL URL configuration joiner should use
&
instead of;
- [KYUUBI #4352] Support System.gc() with periodic GC interval
- [KYUUBI #4360] Support to refresh the unlimited users for session limiter
- [KYUUBI #4372] Support to return null value for OperationsResource RowSet
- [KYUUBI #4390] Allow user to provide batch id on submitting batch job
- [KYUUBI #4418] Allow disable async retry and fail fast on unrecoverable DB error
- [KYUUBI #4419] Implement simple EngineSecuritySecretProvider
- [KYUUBI #3973] Fix conf fallback for
kyuubi-ctl list server
- [KYUUBI #4006] Fix the incorrect execution of the source command
- [KYUUBI #4067] Reset the operation log before fetching new one
- [KYUUBI #4216] Support to transfer client version for JDBC and REST client
- [KYUUBI #4221] REST client for creating batch with uploading resource file
- [KYUUBI #4311] Fix the wrong parsing of JVM parameters in JDBC url
- [KYUUBI #4334] REST client should catch
NoHttpResponse Exception
and retry - [KYUUBI #3128] Support CostMode for PlanOnlyStatement
- [KYUUBI #3385] Set executor Pod name prefix if missing in Spark on K8s case
- [KYUUBI #3441] Change default Spark version to 3.3.1
- [KYUUBI #3487] Support Kyuubi as Spark dialect
- [KYUUBI #3776] Revise Kyuubi Spark engine shaded
- [KYUUBI #3780] Add support for executing Python/PySpark scripts (experimental)
- [KYUUBI #3792] Engine UI support grace stop
- [KYUUBI #3801] Correctly calculate active stages in SQLOperationListener
- [KYUUBI #3851] Support auto set up spark.master when Kyuubi running inside K8s Pod
- [KYUUBI #3885] Fix memory leak when using incremental collect mode
- [KYUUBI #3892] Remove sensitive kyuubi.engine.credentials from spark conf
- [KYUUBI #4035] Post SparkOperationEvent and show sessionId for statements
- [KYUUBI #4150] Support to execute Scala statement synchronized to prevent conflicts
- [KYUUBI #4316] Fix returned Timestamp values may lose precision
- [KYUUBI #4336] Avoid listing all schemas for Spark session catalog on schema pruning
- [KYUUBI #4407] Adapt SLF4J 2.x
- [KYUUBI #4412] Align the session handle between server and engine for Spark engine
- [KYUUBI #3424] Access privilege checks for namespaces and tables of DatasourceV2
- [KYUUBI #3515] Support Iceberg commands and skip apply row-filter to output tables
- [KYUUBI #3325] Privilege checks for permanent views and skipping shadowed tables
- [KYUUBI #3300] Support user group based policies from Ranger
- [KYUUBI #3371] Ranger client support sending requests in batch
- [KYUUBI #3581] Support row filter and data masking on permanent views
- [KYUUBI #3607] Support {OWNER} variable defined in Ranger policy
- [KYUUBI #3904] New Authz Plan Serde Layer
- [KYUUBI #4076] Modified query plan should correctly report stats
- [KYUUBI #4270] Register shutdown hook for plugin cleanup
- [KYUUBI #4255] Handle describe relation for V2 relations
- [KYUUBI #4262] Rename table requires ALTER privilege of the source table
- [KYUUBI #4202] Fix reference resolution when data masking enabled for V2 relations
- [KYUUBI #4437] Fix dependencies conflict by replacing
jersey-bundle
withjersey-client
- [KYUUBI #2282] Introduce Spark Lineage plugin (experimental)
- [KYUUBI #3365] Introduce Spark Hive connector (experimental)
- [KYUUBI #3601] Support infer columns for rebalance and sort
- [KYUUBI #3962] Add two conditions to decide if add shuffle before writing
- [KYUUBI #3988] Final stage config isolation support write only
- [KYUUBI #3893] UnspecifiedDistribution does not have default partitioning
- [KYUUBI #3514] Support Flink 1.16
- [KYUUBI #3604] Propagate HADOOP_PROXY_USER on launching Flink engine
- [KYUUBI #3717] Support Flink engine see primary keys
- [KYUUBI #3473] Add Docker Compose based Kyuubi playground
- [KYUUBI #3903] Support generate
kyuubi-version-info.properties
on Windows - [KYUUBI #4000] JDBC engine supports Apache Phoenix
- [KYUUBI #4288] Use eclipse-temurin:8-jdk-focal as default base image
- [KYUUBI #4320] Trino engine supports GetPrimaryKeys
- [KYUUBI #4473] Helm Chart improvements
- AdithyaPadmashali
- Alex Wiss-Wolferding
- Bowen Liang
- Brandon Grams
- CHzxp
- Cheng Pan
- Deng An
- Fei Wang
- Fu Chen
- HYBG
- Hanna Liashchuk
- Hyukjin Kwon
- John Caveman
- Kent Yao
- Luning Wang
- Min Zhao
- Nietzsche_w
- Paul Lin
- Ruguo Yu
- SteNicholas
- Thomas Prelle
- Tianlin Liao
- Wang Zhen
- William Tong
- Xuedong Luan
- Yikf
- asagjj
- df_liu
- dnskr
- dongkelun
- edddddy
- firefox
- hongdongdong
- hongkunyoo
- jiaoqingbo
- jinsilei
- lcy999
- liangbowen
- liaopeng
- maming
- odone
- pengqli
- runzhliu
- senmiaoliu
- sundy xiong
- sychen
- ulysses-you
- wForget
- wangjunbo
- waywtdcc
- wxmimperio
- xuefeimiaoao
- yehere
- yexianxun
- yikf
- yongqian
- yuqi
- zhouyifan279
- zwangsheng
- 毕博
- 佘志铭
- 胡安忠
- 袁福元
- 如果引擎应用程序已终止,则快速使 kyuubi 连接失败
- 将 TGetInfoReq 传输到 kyuubi 引擎端以检查连接是否有效并保持连接有效
- GetInfo 支持返回服务器/引擎信息
- 处理 zookeeper watch 事件
- 在 Kyuubi JDBC 驱动实现了 ResultSetMetaData 的 isSigned 方法
- 扩展 JDBC URL 以支持 catalog
- [SPARK] 将非本地 jar URI 添加到类加载器时修复 scala NPE 问题
- [SPARK] Kyuubi 应该为 K8s deployment 设置 env SPARK_USER_NAME
- [SPARK] 运行在 K8s 的 Spark 引擎在服务发现注册优先使用 IP
- [FLINK] Flink SQL 引擎修复 GetTables 操作
- [FLINK] Flink SQL 引擎实现 GetColumns 操作
- [FLINK] Flink SQL 引擎支持跨 Flink 版本运行 DDL SQL
- Bowen Liang
- Cheng Pan
- Fei Wang
- Fu Chen
- HYBG
- Hanna Liashchuk
- Luning Wang
- Min Zhao
- Paul Lin
- SteNicholas
- Wang Zhen
- Yikf
- asagjj
- df_liu
- hongdongdong
- jiaoqingbo
- senmiaoliu
- sychen
- ulysses-you
- zhouyifan279
- zwangsheng
发布记录 - 1.7.4
2024 4月 26
Changlogs
Credits
Last but not least, this release would not have been possible without the following contributors:
发布记录 - 1.8.2
2024 4月 26
Changelogs
Credits
Last but not least, this release would not have been possible without the following contributors:
发布记录 - 1.9.0
2024 3月 19
Highlight
Server
Spark Engine
Spark AuthZ Plugin
Other Spark Plugins
Flink Engine
Hive Engine
Trino Engine
JDBC Engine
Other Engine
Web UI
Documentation
Building, Infrastructure and Dependency
Credits
Last but not least, this release would not have been possible without the following contributors:
发布记录 - 1.8.1
2024 2月 21
Server
Spark Engine, Connectors and Plugins
Other Engines
Client
Build, Test and Misc
Credits
Last but not least, this release would not have been possible without the following contributors:
发布记录 - 1.8.0
2023 11月 7
重要变更
Spark Engine, Connectors and Plugins
Flink Engine
Other Engines
Server
Client
Web UI
Documentation
Building, Infrastructure and Dependency
Credits
Last but not least, this release would not have been possible without the following contributors:
发布记录 - 1.7.3
2023 9月 25
重要变更
Server
Spark Authz Plugin
Credits
Last but not least, this release would not have been possible without the following contributors:
发布记录 - 1.7.2
2023 9月 18
重要变更
Server
Spark Engine
Spark Extensions
Client
Common
Deployment
Documentation
Credits
Last but not least, this release would not have been possible without the following contributors:
发布记录 - 1.7.1
2023 五月 5
重要变更
Server
Spark Engine
Spark Authz Plugin
Other Spark Extensions
Flink Engine
Client
Deployment
Credits
Last but not least, this release would not have been possible without the following contributors:
发布记录 - 1.7.0
2023 3月 8
重大更新
Server
Client
Spark Engine
Spark Authz Plugin
Other Spark Extensions
Flink Engine
Other Notable Changes
Credits
Last but not least, this release would not have been possible without the following contributors:
发布记录 - 1.6.1-incubating
2022 11月 16
重大更新
服务端
客户端
引擎
更改列表
[KYUUBI #3744][Bug] InvalidACL appears in engine after upgrade to Kyuubi 1.6.0
[KYUUBI #3774] Output OpenSessionCount when Engine is waiting for exit because it reaches the Max lifetime
[KYUUBI #3746] Support to specify multiple config items with --conf
in kyuubi command line
[KYUUBI #3748] Adding metrics kyuubi.metadata.request.opened and fix flaky test - MetadataManagerSuite
[KYUUBI #3745] Closing existing seekable reader when adding extra log
Revert “[KYUUBI #3385] Set spark.kubernetes.executor.podNamePrefix if missing”
[KYUUBI #3730] Change the parameter annotation of the runRemoveJarOperation method to Remove-jar operation instead of Add-jar operation
[KYUUBI #3726] [DOCS] Place comments above line to avoid mislead users
[KYUUBI #3720][FOLLOWUP] Using correct config version for CTL_BATCH_LOG_ON_FAILURE_TIMEOUT
[KYUUBI #3720] Support to show remaining logs with timeout if the batch failed
[KYUUBI #3728] [CI] Store kyuubi-spark-batch-submit.log for IT spark on kubernetes batch test
[KYUUBI #3714] Use ServicesResourceTransformer to concatenating service entries
[KYUUBI #3711] PlanOnly should output engine logs to log files
[KYUUBI #3710] Delete the swagger UI related resource files from the source folder
[KYUUBI #3703][DOCS] Fix typo in localLog REST API
[KYUUBI #3715] Remove unexpected mysql connector jar
[KYUUBI #3701] Allow JDBC authencation to connect auth db with no user name
[KYUUBI #3705] Add docs for JDBC authentication usage with in-memory database
[KYUUBI #3685] Fix flaky test BatchRestApiSuite - basic batch rest client with invalid user
[KYUUBI #3385] Set spark.kubernetes.executor.podNamePrefix if missing
[KYUUBI #3664] [SPARK][K8S][BATCH] Kyuubi should set env SPARK_USER_NAME on submitting Spark batch job to K8s
[KYUUBI #3670] [SPARK][K8S] Fix Spark master match rule on tagging Spark application
[KYUUBI #3624][REST] Fix description typo of session REST open api
[KYUUBI #3627] Support vanilla Jetty for Spark packaged by sbt
[KYUUBI #3543][FOLLOWUP][1.6] Handle zookeeper watch events
[KYUUBI #3543] Handle zookeeper watch events
[KYUUBI #3619] Enhance JAVA_HOME detection to support RPM
[KYUUBI #3600] [SPARK][K8S] Respect default static port for web UI on Spark K8s cluster mode
[KYUUBI #3597] Engine should prefer ip for registering on K8s cluster mode
[KYUUBI #3555] [FEATURE] Helm install help set kyuubi pod global env
[KYUUBI #3598] Beeline doesn’t work with -e option when started in background
[KYUUBI #3504] Extend JDBC URL to support catalog
[KYUUBI #3527][SPARK][FOLLOWUP] spark.kubernetes.executorEnv should be spark.executorEnv
[KYUUBI #3568] [Bug] [Doris Engine] Fix return decimal, date, timestamp using string
[KYUUBI #3579] Use strict path in dist script to avoid unexpected cp behavior
[KYUUBI #3372] [BUILD] Enable -x on build/dist
[KYUUBI #3571] Fix NPE for batch command line
[KYUUBI #3566] Shade Zookeeper into spark engine jar
[KYUUBI #3519] Flink SQL Engine - GetColumns Operation
[KYUUBI #3539] [FEATURE][TPCDS] Add white list help run the specified queries
[KYUUBI #3562] Fix the outdate docs index
[KYUUBI #3560] Flink SQL engine supports run DDL across versions
[KYUUBI #3521] [TEST] Fix Flink flaky test - select count
[KYUUBI #3549][FOLLOWUP] Simplify test
[KYUUBI #3549] Support query id in Flink engine
[KYUUBI #3544] Fix bin/kyuubi stop check
[KYUUBI #3406] [FOLLOWUP] Add create datasource table DDL usage to Pyspark docs
[KYUUBI #3547] Fix Flink statements results validation
[KYUUBI #3406] [Subtask] [Doc] Add PySpark client docs
[KYUUBI #3527] [SPARK] Kyuubi should set env SPARK_USER_NAME for K8s deployment
[KYUUBI #3451] Implement GetInfo for JDBC engine
[KYUUBI #3510] Reading an uninitialized log should return empty rowSet
[KYUUBI #3522][FOLLOWUP] Fix benchmark results-dir
[KYUUBI #3522] [BUG][TPCDS] Fix benchmark –results-dir
[KYUUBI #3498] Fix GetTables operation in Flink
[KYUUBI #3502] [DOCS] Fix typo and use beeline provided by Kyuubi
[KYUUBI #3484] Implement isSigned method of ResultSetMetaData for Kyuubi JDBC driver
[KYUUBI #3452] Implement GetInfo for Trino engine
[KYUUBI #3459] [BUILD] Share jars to reduce binary release tarball size
[KYUUBI #3051][FOLLOWUP] Remove redundant serverIpAddress method in SparkSessionImpl
[KYUUBI #3453] [MINOR] Remove unused spark.version property from the pom file of kyuubi-extension-spark-common module
[KYUUBI #3122] GetInfo supports return server/engine info
[KYUUBI #3318] Transfer the TGetInfoReq to kyuubi engine side to check the connection valid and keep connection alive
[KYUUBI #3064] Fix scala NPE issue when adding non-local jar URI to class loader
[KYUUBI #3408] [TEST] Remove the unstable flink process builder test case
[KYUUBI #3208] Fix Flaky Test - MetadataManagerSuite: metadata request metrics
[KYUUBI #3315] Fast fail the kyuubi connection if the engine application has been terminated
致谢
最后但同样重要的是,如果没有以下贡献者,该版本是不可能发布的: