Releases Archive
- Fully verified with Java 8, 11, and 17, Scala 2.12 and 2.13
- Fully verified with Apache Spark 3.1 to 3.5, support for Spark 3.1 is deprecated and will be removed in the next release
- Fully verified with Apache Flink 1.16 to 1.19, support for Flink 1.16 is deprecated and will be removed in the next release
- Fully verified with Apache Hive 2.3, 3.1, and Cloudera Hive 2.1.1-cdh6, support running Hive engine on YARN (experimental)
- Spark engine supports save large query result to file before retrieving
- JDBC engine supports more dialects, including Doris, MySQL, PostgreSQL, Phoenix, StarRocks and Impala
- Kyuubis server metastore supports PostgreSQL as backend storage
- Introduce a basic SQL query editor in Kyuubi Web UI
- Continuously Spark AuthZ plugin improvements, including enhanced Iceberg support and new introduced Apache Hudi and Delta Lake support
- Continuously Kyuubi & Spark on Kubernetes improvements
- [KYUUBI #5423] Support chaining SessionConfAdvisors
- [KYUUBI #5631][K8S][HELM] Set session affinity if needed in helm chart
- [KYUUBI #5674] Add PostgreSQL as backend database for kyuubi metadata
- [KYUUBI #5683] Add
kyuubi.session.proxy.user
as an alternative tohive.server2.proxy.user
- [KYUUBI #5704] Add unit tests for
kyuubi.session.proxy.user
in RESTful API - [KYUUBI #5729] Use G1GC as Java option example in kyuubi-env template
- [KYUUBI #5797] Support to describe engine with command in current session
- [KYUUBI #5797][FOLLOWUP] Desc engine command support show engine registered attributes
- [KYUUBI #5873] Support to get operation progress with RESTful API
- [KYUUBI #5912] Cleanup unused config
kyuubi.frontend.thrift.http.allow.user.substitution
- [KYUUBI #5921]
EngineRef#deregister
should return the value - [KYUUBI #5934][K8S][HELM] Add Spark configuration support
- [KYUUBI #6003] Allow disabling user impersonation on launching engine
- [KYUUBI #6006][HELM] Support additional labels for service monitor
- [KYUUBI #6066] Fix PostgreSQL metastore migration script failure if indexes already exist
- [KYUUBI #6070] Improve perf on assembling row-based TRowSet
- [KYUUBI #6080] Improve instruction for deploy Kyuubi on K8s
- [KYUUBI #6081] RESTful client should retry on UnknownHostException
- [KYUUBI #6099] Remove support of Derby for Kyuubi metastore
- [KYUUBI #6126] Filter blank engine home env
- [KYUUBI #6140] Move getCodeSourceLocation to kyuubi-util
- [KYUUBI #5377] Spark engine query result save to file
- [KYUUBI #5377][FOLLOWUP] Spark engine query result save to file
- [KYUUBI #5377][FOLLOWUP] Always try to cleanup session result path
- [KYUUBI #5377][FOLLOWUP] Get limit from more spark plan and regard result max rows
- [KYUUBI #5870] Directly mapping engine’s data type to Java type for TRowSet generation
- [KYUUBI #5877] Support Python magic syntax for notebook usage
- [KYUUBI #5877][FOLLOWUP] Add spark output mode to better support PySpark notebook
- [KYUUBI #5894] Separate closed and online sessions/statements in the SparkUI’s engine tab
- [KYUUBI #5899] Refine the python/scala interpreter error prompt
- [KYUUBI #6053][K8S] Show more details for pod/container status in application error
- [KYUUBI #6083] Bump Spark 3.3.3 to 3.3.4
- [KYUUBI #6091] Deprecate and remove building support for Spark 3.1
- [KYUUBI #6092] Enable DeltaTest for Spark 3.5
- [KYUUBI #6094] Bump Spark from 3.5.0 to 3.5.1
- [KYUUBI #6095] Enable PaimonTest for Spark 3.5
- [KYUUBI #6096] Enable PySpark tests for Spark 3.5
- [KYUUBI #6107][Spark] Collect and summarize the
executorRunTime
andexecutorCpuTime
of the statement - [KYUUBI #6107][FOLLOWUP] Add comments for the newly added parameters of the Spark event
- [KYUUBI #6108] Display the CPU time consumed by the statement in the Spark Engine tab
- [KYUUBI #6156] Remove
flink.
prefix for create session configurations - [KYUUBI #6163] Set default Spark version to 3.5
- [KYUUBI #6166][MINOR] Fix wrong copy path about SPARK_HOME in docker
- [KYUUBI #6175][MINOR] Refactor copy path about SPARK_HOME in docker
- [KYUUBI #5248][AUTHZ] Check previleges for Iceberg’s CALL to RewriteDataFilesProcedure procedure
- [KYUUBI #5255] Add an optional comment field to the authz specs for better recognition
- [KYUUBI #5284] Support Hudi Alter Table Command in Authz
- [KYUUBI #5323][AUTHZ] Drop Hive and Iceberg tables with PURGE option in tests
- [KYUUBI #5359][AUTHZ] Support Create Table Commands for Hudi
- [KYUUBI #5360] Support Hudi InsertIntoHoodieTableCommand
- [KYUUBI #5361][AUTHZ] Support Drop/Truncate Table Commands for Hudi
- [KYUUBI #5362] Remove Spark 3.0 support for Authz
- [KYUUBI #5399][AUTHZ] Cleanup Spark 3.0 specific implementation
- [KYUUBI #5407][AUTHZ] Tests for Iceberg system procedures of snapshot management
- [KYUUBI #5417] should not check in-subquery in permanent view
- [KYUUBI #5417] Authz should not check dependent subquery plan privilege
- [KYUUBI #5427][AUTHZ] Shade spark authz plugin
- [KYUUBI #5446][AUTHZ] Support Create/Drop/Show/Reresh index command for Hudi
- [KYUUBI #5447][AUTHZ] Support Hudi DeleteHoodieTableCommand/UpdateHoodieTableCommand/MergeIntoHoodieTableCommand
- [KYUUBI #5447][FOLLOWUP] Remove unrelated debug prints in TableIdentifierTableExtractor
- [KYUUBI #5452][AUTHZ] Support Compaction table commands for Hudi
- [KYUUBI #5455][AUTHZ] Support hudi CompactionHoodiePathCommand & CompactionShowHoodiePathCommand
- [KYUUBI #5457][AUTHZ] Support RepairTable Commands for Hudi
- [KYUUBI #5462][AUTHZ] Support create table command for Paimon
- [KYUUBI #5472] Permanent View should pass column when child plan no output
- [KYUUBI #5472][AUTHZ][FOLLOWUP] Check permanent view also need support merge projection
- [KYUUBI #5475][FOLLOWUP] Authz check permanent view’s subquery should check view’s correct privilege
- [KYUUBI #5478][AUTHZ] Support Hudi ShowHoodieTablePartitionsCommand
- [KYUUBI #5479][AUTHZ] Support Hudi CallProcedureHoodieCommand for stored procedures
- [KYUUBI #5492][AUTHZ] saveAsTable create DataSource table miss db info
- [KYUUBI #5496][AUTHZ] Authz shaded include the transitive jackson libs
- [KYUUBI #5497][AuthZ] Simplify debug message for missing field/method in ReflectUtils
- [KYUUBI #5506][AUTHZ][MINOR] Reuse loaded table extractor in Hudi table extractors
- [KYUUBI #5512][AuthZ] Remove the non-existent query specs in Deletes and Updates
- [KYUUBI #5529][AUTHZ] Support create table command for Delta Lake
- [KYUUBI #5503][AUTHZ] Check plan auth checked should not set tag to all child nodes
- [KYUUBI #5503][FOLLOWUP][AUTHZ] Authz should skip inner plan that have been verified
- [KYUUBI #5529][AUTHZ][FOLLOWUP] Fix Create table ut for Delta Lake does not trigger a sql operation
- [KYUUBI #5529][AUTHZ][FOLLOWUP] Remove useless org.apache.spark.sql.delta.commands.CreateDeltaTableCommand
- [KYUUBI #5532][AUTHZ] Support Alter table commands of updating table schema for Delta Lake
- [KYUUBI #5533][AUTHZ] Support merge into table command for Delta Lake
- [KYUUBI #5534][AUTHZ] Support optimize table command for Delta Lake
- [KYUUBI #5535][AUTHZ] Support vacuum table command for Delta Lake
- [KYUUBI #5537][AuthZ] Order Command Spec by Command Class [group, classname]
- [KYUUBI #5543][AUTHZ] Generalize command specs and check duplicated class names of command spec
- [KYUUBI #5546][AUTHZ] Reorgnize the package names for rules
- [KYUUBI #5555][AUTHZ] Extractor common authorization rule class
- [KYUUBI #5556][AUTHZ] Support Alter table commands of set table properties
- [KYUUBI #5557][AUTHZ] Refactor code about handle PermanentViewMarker
- [KYUUBI #5565][AUTHZ] Support Delete/Insert/Update table command for Delta Lake
- [KYUUBI #5575][AUTHZ] Support InsertIntoDataSourceDir/InsertIntoHiveDirCommand path check
- [KYUUBI #5575][KYUUBI #5611][AUTHZ] Check path privilege should use correct action type
- [KYUUBI #5576][Bug] Fix wrong code in test case of dir command
- [KYUUBI #5579][AUTHZ] Support LogicalRelation don’t have CatalogTable but have HadoopFsRelation
- [KYUUBI #5594][AUTHZ] BuildQuery should respect normal node’s input
- [KYUUBI #5611][AUTHZ] Authz support path privilege for SaveIntoDataSourceCommand
- [KYUUBI #5628][AUTHZ] Support path privilege check for resource command
- [KYUUBI #5630][Authz] Support path check of LoadDataCommand
- [KYUUBI #5638][AUTHZ] URI related privileges shall be READ/WRITE
- [KYUUBI #5638][FOLLOWUP] Remove unnecessary actionType in uri desc
- [KYUUBI #5642][AUTHZ] Support path privilege of CreateDatabaseCommand/AlterDatabaseSetLocationCommand/CreateNamespace/SetNamespaceLocation
- [KYUUBI #5642][FOLLOWUP] Fix unit test
- [KYUUBI #5652][AUTHZ] CreateTable related command support path privilege
- [KYUUBI #5657][AUTHZ] Support path privilege of AlterTableSetLocationCommand/AlterTableAddPartitionCommand
- [KYUUBI #5664][AUTHZ] Support create path-based table for Delta Lake
- [KYUUBI #5677][AUTHZ] Typeof expression miss column information
- [KYUUBI #5684][AUTHZ] Support delete/update path-based table for Delta Lake
- [KYUUBI #5690][AUTHZ] Support insert into/overwrite path-based table for Delta Lake in Authz
- [KYUUBI #5692][Bug] Authz not skip explain command
- [KYUUBI #5707][AUTHZ] Support merge into path-based table for Delta Lake in Authz
- [KYUUBI #5715][MINOR] Replace with .isDefined and .isEmpty
- [KYUUBI #5726][AUTHZ] Support optimize path-based table for Delta Lake in Authz
- [KYUUBI #5735][AUTHZ] Support vacuum path-based table for Delta Lake
- [KYUUBI #5743][AUTHZ] Improve AccessControlException verification of RangerSparkExtensionSuite
- [KYUUBI #5768][AUTHZ] Authz internal place holder should skip privilege check
- [KYUUBI #5780][AUTHZ] Treating PermanentViewMarker as LeafNode make code simple and got correct privilege object
- [KYUUBI #5780][AUTHZ][FOLLOWUP] Format PermanentViewMarker tree string
- [KYUUBI #5793][AUTHZ][BUG] PVM with nested scalar-subquery should not check src table privilege
- [KYUUBI #5884] PVM should inherit MultiInstance and wrap with new exprId
- [KYUUBI #5903] PVM should override computeStats method
- [KYUUBI #5913][Bug] After resolve PVM should mark all nodes as checked
- [KYUUBI #5937] PVM cause cache table not work
- [KYUUBI #5964][BUG] Avoid check not fully optimized query for InsertIntoDataSourceDirCommand and InsertIntoDataSourceCommand
- [KYUUBI #5965][AUTHZ] Supports check hoodie procedures show_commits resource privileges
- [KYUUBI #5985][AUTHZ][MINOR] Remove incorrect getUri method
- [KYUUBI #5997][AUTHZ] Avoid unnecessary loop of RuleEliminateTypeOf
- [KYUUBI #6007] AuthZ should check hoodie procedures path resource privileges
- [KYUUBI #5580] Support generate and window operators for InferRebalanceAndSortOrders
- [KYUUBI #5786] Disable spark script transformation
- [KYUUBI #5816] Change spark rule class to object or case class
- [KYUUBI #6125] Drop Kyuubi extension for Spark 3.1
- [KYUUBI #6168] Check if forcedMaxOutputRows is negative
- [KYUUBI #5264][FLINK] Force disable Flink’s session timeout
- [KYUUBI #5489] Adjust shuffle partitions dynamically
- [KYUUBI #5505][FLINK] Support HELP command
- [KYUUBI #6097][KYUUBI #6098] Upgrade Flink 1.16.3 and 1.18.1
- [KYUUBI #6130] Stop engine immediately after close session for
CONNECTION
level FlinkSQLEngine - [KYUUBI #6134] Support Flink 1.19
- [KYUUBI #6142] Deprecate Flink 1.16
- [KYUUBI #6152] Remove useless variable
- [KYUUBI #6158][MINOR] Remove confusing variable
- [KYUUBI #6171] Drop building support for Flink 1.16
- [KYUUBI #6171][FOLLOWUP] Restore flink-1.16 profile to recover Flink IT
- [KYUUBI #5830] Test Hive engine with CDH Hive 2.1.1-cdh6.3.2
- [KYUUBI #5865] Support get SQL keywords from Hive engine through API
- [KYUUBI #5867] HiveEngine support run on YARN mode
- [KYUUBI #5867][FOLLOWUP] Remove unused config
- [KYUUBI #5867][FOLLOWUP] Fix generated configuration docs and typo
- [KYUUBI #5968] Support set authentication user for Trino engine
- [KYUUBI #6147][TRINO] Use the overlay conf as session conf
- [KYUUBI #5375] JDBC Engine supports PostgreSQL
- [KYUUBI #5382][JDBC] Duplication cleanup improvement in JdbcDialect and schema helpers
- [KYUUBI #5404] JDBC Engine supports StarRocks
- [KYUUBI #5464] JDBC Engine supports MySQL
- [KYUUBI #5509] Add Apache Impala JDBC engine dialect
- [KYUUBI #5582] JDBC Engine supports configurable default fetchSize
- [KYUUBI #5851] Generalize TRowSet generators
- [KYUUBI #5861] Generalize TRowSet generator for JDBC engine with dialects
- [KYUUBI #5862] Use TestContainerForAll for testing JDBC engine with testcontainers
- [KYUUBI #5906][JDBC] Rebase Doris dialect implementation
- [KYUUBI #6037] Fix the JDBC engine provider name
- [KYUUBI #5384][KSCH] Hive connector supports Spark 3.5
- [KYUUBI #5709] Avoid exposing client keystore and truststore passwords in the JDBC URL
- [KYUUBI #5783] Switch to
kyuubi-relocated-hive-service-rpc
- [KYUUBI #5828][CHAT] Kyuubi chat engine supports ernie bot(文心一言)
- [KYUUBI #6078] KSHC should handle the commit of the partitioned table as dynamic partition at write path
- [KYUUBI #6114] Properly handle KyuubiStatement#getMoreResults(CLOSE_CURRENT_RESULT)
- [KYUUBI #5366][UI] Add submit SQL page in Kyuubi Web UI
- [KYUUBI #5517][UI] Initial implement the SQL Lab page
- [KYUUBI #5958][WEBUI] Bump follow-redirects from 1.15.1 to 1.15.4
- [KYUUBI #6002] Bump vite from 4.3.9 to 4.5.2 in web UI
- [KYUUBI #5265][DOCS][FLINK] Add notice about Jobmanager HA to Flink deployment docs
- [KYUUBI #5500] Add Kyuubi Code Program to Doc
- [KYUUBI #5688][DOC] Update supported spark 3.5.x in authz doc
- [KYUUBI #5762][DOC] Introduce Kyuubi Web UI Engine Page
- [KYUUBI #5893][DOC] Update logos in layer picture on welcome page
- [KYUUBI #5905][DOC] Workaround for long inlined code sample in config page
- [KYUUBI #6010] Improve Gluten usage documentation
- [KYUUBI #6119][DOC] Update doc for HA/Zookeeper Configuration
- [KYUUBI #4453][FOLLOW] Delete LICENSE for removed kubernetes client dependencies
- [KYUUBI #5104][TEST] BatchesResourceSuite - expose the metrics with operation type and current state
- [KYUUBI #5241] Bump Commons Lang3 from 3.12.0 to 3.13.0
- [KYUUBI #5245][INFRA] Daily deploy snapshot version on branch-1.8
- [KYUUBI #5266] Upgrade playground to Kyuubi 1.7.1
- [KYUUBI #5322][INFRA] Enable GitHub Projects
- [KYUUBI #5324] Bump latest 1.7.3 in playground and helm charts
- [KYUUBI #5342] Add label hacktoberfest to project
- [KYUUBI #5356][INFRA] GitHub Issue template for 2023 Kyuubi Code Contribution Program
- [KYUUBI #5395] Bump netty from 4.1.93.Final to 4.1.100.Final
- [KYUUBI #5449] Bump Delta Lake 3.0.0
- [KYUUBI #5501] Update codecov token and fix codecov reporting on PRs
- [KYUUBI #5509][FOLLOWUP] JDBC IT should always depends on kyuubi-hive-jdbc-shaded
- [KYUUBI #5600] Fix flaky test SessionsResourceSuite
- [KYUUBI #5602] Improve issue template for 2023 Kyuubi Code Contribution Program
- [KYUUBI #5660][TEST] Ensure SPI provider file content sorted in alphabet order
- [KYUUBI #5671] Bump axios from 0.27.2 to 1.6.0 in /kyuubi-server/web-ui
- [KYUUBI #5674][LICENSE][FOLLOWUP] Update license files
- [KYUUBI #5676] Bump Apache parent pom from 30 to 31
- [KYUUBI #5701] Enhance the PR Template to make PR desc informative
- [KYUUBI #5716][TEST] Fix match may not be exhaustive in Scala 2.13
- [KYUUBI #5774][CI] Bump actions/checkout from v3 to v4
- [KYUUBI #5779][CI] Upgrade Github actions for Docker
- [KYUUBI #5800][KYUUBI#5467] Integrate Intel Gluten with Spark engine
- [KYUUBI #5835] Remove issue template for 2023 Kyuubi Code Contribution Task
- [KYUUBI #5838][BUILD] Bump actions/setup-python from v4 to v5
- [KYUUBI #5839][BUILD] Bump actions/setup-java from v3 to v4
- [KYUUBI #5841][BUILD] Bump actions/stale from v7 to v9
- [KYUUBI #5842][BUILD] Bump actions/setup-node from v3 to v4
- [KYUUBI #5874][INFRA] Bump github labeler action to v5
- [KYUUBI #5879] Bump dropwizard metrics from 4.2.8 to 4.2.23
- [KYUUBI #5902] Bump doc build dependencies
- [KYUUBI #5914] Update layer diagram on welcome page
- [KYUUBI #5925] Kyuubi TPC-DS support running benchmark with skipping some queries
- [KYUUBI #5939] Bump Gluten version to recover Gluten IT
- [KYUUBI #5941] Drop Kubernetes Block Cleaner Tool from Kyuubi
- [KYUUBI #5944] Introduce Prometheus and Grafana for Kyuubi Playground
- [KYUUBI #5955][TEST][GLUTEN] Disable gluten ui when running integration test
- [KYUUBI #5973] Bump gRPC from 1.53.0 to 1.60.1
- [KYUUBI #5974] Bump jetcd 0.7.7
- [KYUUBI #5975] Bump trino-client 411
- [KYUUBI #5975][FOLLOWUP][TEST] Fix trino - get type info
- [KYUUBI #6043][TEST][GLUTEN] Gluten-it gluten package add arch suffix
- [KYUUBI #6061] Delete svn staging dir before checking out
- [KYUUBI #6063] Bump iceberg 1.4.3
- [KYUUBI #6067] Bump org.postgresql:postgresql from 42.6.0 to 42.7.2
- [KYUUBI #6068] Remove community section from user docs
- [KYUUBI #6069][TEST] TFrontendServiceSuite - close expired operations
- [KYUUBI #6071] Add .java-version into git ignore
- [KYUUBI #6074] Add a script to simplify the process of creating release notes
- [KYUUBI #6075] Update known_translations
- [KYUUBI #6076] Update Helm Chart and playground to use 1.8.1
- [KYUUBI #6085] Bump gRPC from 1.60.1 to 1.60.2
- [KYUUBI #6089] Bump Paimon from 0.5.0-incubating to 0.7.0-incubating
- [KYUUBI #6090] Bump Delta from 3.0.0 to 3.1.0 for Spark 3.5
- [KYUUBI #6101] Bump Jackson from 2.15.0 to 2.15.4
- [KYUUBI #6109] Fork beeline module from Apache Hive 3.1.3
- [KYUUBI #6117] Bump Kyuubi Shaded 0.3.0
- [KYUUBI #6118] Migrate to shaded HMS client for getting delegation token on server
- [KYUUBI #6129][BUILD] Remove the extension spark jars before build to avoid include unspecified version
- [KYUUBI #6131] Simplify Maven dependency management after dropping building support for Spark 3.1
- [KYUUBI #6139] Expose env vars and sys props to JUnit tests too
- [KYUUBI #6143] Add Dockerfile.HMS for testing
- [KYUUBI #6162] Cut out hive-common deps in beeline module
- [KYUUBI #6164] Limit max parallel for each GitHub Actions workflow
- [KYUUBI #6145] Use HiveMetaStore Docker for testing
- [KYUUBI #6145][FOLLOWUP] Remove unused dependency commons-lang
- [KYUUBI #6151] Remove invalid matrix spark 3.1 in GA
- [KYUUBI #6153] Cut out Hive JDBC and Hive Service deps in beeline module
- [KYUUBI #6160] Fix beeline test KyuubiBeeLineTest.testKyuubiBeelineComment
- [KYUUBI #6167] Drop JDK 11 on GitHub Actions
- [KYUUBI #6170] Merge JDBC Trino TPC jobs and Style Check jobs in GitHub Actions workflow
- [KYUUBI #6173] Bump maven-download-plugin from 1.7.1 to 1.8.1
- [KYUUBI #6174] Remove unused dependency management from POM
- [KYUUBI #6177] Bump BouncyCastle from 1.67 to 1.77
- [KYUUBI #6178] Improve Gluten integration test
- GitHub Bot
- Shaoyun Chen
- GGHWS
- Deepali Gupta
- Baokun Han
- Denis Krivenko
- Kwafoor
- Pengqi Li
- Bowen Liang
- Paul Lin
- Senmiao Liu
- Chenliang Lu
- Tigran Manasyan
- Hai Nguyen
- Cheng Pan
- XH Qian
- Zhen Wang
- Fei Wang
- Kang Wang
- Junjie Xu
- Binjie Yang
- Kent Yao
- Xianxun Ye
- Kaifei Yi
- David Yuan
- He Zhao
- Yangming Zhao
- Yifan Zhou
- Mingliang Zhu
- Yi Zhu
- Yaogai Zhu
- huangxiaopingRD
- kevinjmh
- mrtisttt
- BingKun Pan
- rhh777
- silencily
- sincewhen
- Vino Yang
- [KYUUBI #6058] Make Jetty server stop timeout configurable
- [KYUUBI #6048] Assign serviceNode and add volatile for variables
- [KYUUBI #6045] Sync the AdminRestApi with the AdminResource Apis
- [KYUUBI #6028] Exited spark-submit process should not block batch submit queue
- [KYUUBI #6001] Fix RESTful protocol security enabled evaluation
- [KYUUBI #5999] loads the interval of sessionConfCache from configurable file in FileSessionConfAdvisor
- [KYUUBI #5990] Always take the first declared SASL/PLAIN auth type
- [KYUUBI #5922] Enhance YARN app state judgment
- [KYUUBI #5900] Get the engine service node to delete instead of delete engine space
- [KYUUBI #5900] Support to deregister the engine on open failure
- [KYUUBI #5892] Metadata store JDBC url supports <KYUUBI_HOME> variable
- [KYUUBI #5869] Add volatile to variables in KyuubiSessionImpl that may be operated by multiple threads
- [KYUUBI #5244] Make engineAliveMaxFailCount configurable
- [KYUUBI #5795] Using correct time unit for pod cleanup executor
- [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak
- [KYUUBI #5853] Ensure closing the operation log for batch submission in fast-failed case
- [KYUUBI #5833] Rename service registered endpoint key from serviceUri to serverUri
- [KYUUBI #5795] Support to cleanup the spark driver pod periodically
- [KYUUBI #5680] Optimize Spark engine pod name generation
- [KYUUBI #5807] Simple norm the use of
kyuubi.operation.result.arrow.timestampAsString
andkyuubi.operation.result.format
- [KYUUBI #5767] Extract common utils for assembling key value pairs with config option prefix in processbuilder
- [KYUUBI #5765] Avoid array copy in generating process builder commands
- [KYUUBI #5771] Cleanup the terminated app info trigger before clear kubernetes clients
- [KYUUBI #5711] Fix typo for container states audit
- [KYUUBI #5711] Fix typo and audit pod and container states
- [KYUUBI #4847] Check engine alive for KyuubiSessionImpl only
- [KYUUBI #5761] Pretty ProcBuilder generated command
- [KYUUBI #5754] Fix engine id of the engine probe failures logging messages
- [KYUUBI #5730] Tolerate execeptions for periodical scheduled tasks
- [KYUUBI #5210] Close batch operation log even batch is in cancel state
- [KYUUBI #5731] Support to cleanup the spark driver pod with specified cleanup strategy
- [KYUUBI #5725] Handle the session exception for all kyuubi operations during running
- [KYUUBI #2503] Catch all exceptions for sessions timeout check task
- [KYUUBI #5717] Infer the proxy user automatically for delete batch operation
- [KYUUBI #5711] Support to build application state and error from pod container
- [KYUUBI #5714] Fix NPE when deleting terminated application pod
- [KYUUBI #5714] Support to cleanup the spark driver pod after application terminates for retain period
- [KYUUBI #5696] Return remote session id and remote operation id
- [KYUUBI #5654] Introduce new config to forcibly rewrite pod name
- [KYUUBI #5438] Add common method to get session level config
- [KYUUBI #5952] Disconnect connections without running operations after engine maxlife time graceful period
- [KYUUBI #6018] Speed up GetTables operation for Spark session catalog
- [KYUUBI #5966] Error occurs on clicking on the Spark UI Kyuubi Query Engine tab
- [KYUUBI #5831] Pre-allocate array list capacity for TColumns and TColumnValues in TRowSet generation
- [KYUUBI #5811] Reuse time formatters instance in value serialization of TRowSet generation
- [KYUUBI #5766] Default
spark.yarn.maxAppAttempts
to 1 for spark engine - [KYUUBI #5750] Fix the inaccurate issue of obtaining COLUMN_SIZE in the decimal field jdbc of spark engine
- [KYUUBI #5331] Spark engine should throw an exception when it fails to start
- [KYUUBI #5991] Error on reading Atlas properties composed of multi values
- [KYUUBI #5977] Fix argument position on constructing Trino ClientSession
- [KYUUBI #5969] Trino engine add default schema
- [KYUUBI #5878] Hive engine should recognize
hiveserver2-site.xml
- [KYUUBI #5756] Introduce specified initialized SQL to every engine
- [KYUUBI #5814] Skip the max-rows limitation if the statement is not DQL in Flink engine
- [KYUUBI #5799] Fix fetch timeout in session conf doesn’t support ISO-8601 in Flink engine
- [KYUUBI #5789] Flink engine kyuubi.session.engine.flink.fetch.timeout parameter on the server is not effective
- [KYUUBI #5784] Implement HiveTBinaryFrontendService#RenewDelegationToken
- [KYUUBI #5782] Flink Engine GetInfo support CLI_ODBC_KEYWORDS
- [KYUUBI #5778] Hive engine shall respect
hive.server2.enable.doAs
- [KYUUBI #5777] Include
hive-service-rpc
in Hive engine jar - [KYUUBI #5507] Support Initialize SQL in Flink Engine
- [KYUUBI #5280] Hive engine compatible with Hive 2.3
- [KYUUBI #6036] JDBC driver conditional sets fetchSize on opening session
- [KYUUBI #6016] Conditional call GetInfo CLI_ODBC_KEYWORDS to restore compatible with lower version Kyuubi and HS2
- [KYUUBI #5961] Prevent NPE when checking ticket cache exists
- [KYUUBI #5961] Support to specify client kerberosAuthType as fromTicketCache
- [KYUUBI #5849] Incorrectly parse JDBC URL while variable includes colon
- [KYUUBI #5822] Fix the usage msg of KyuubiBeeLine
- [KYUUBI #5752] JDBC driver implements getDate, getTimestamp and getTime with Calendar in ResultSet
- [KYUUBI #5713] Backport HIVE-27271: Client connection to HS2 fails when transportMode=http, ssl=true, sslTrustStore specified without trustStorePassword in the JDBC URL
- [KYUUBI #5981] Deploy Spark Hive connector with Scala 2.13 to Maven Central
- [KYUUBI #6047] Free up disk space
- [KYUUBI #5987] Always export KYUUBI_HOME in
load-kyuubi-env.sh
- [KYUUBI #5987] Enhance KYUUBI_HOME detection in shell script to handle softlink cases
- [KYUUBI #6000] Modify the incorrect configuration file in the trino-cli documentation
- [KYUUBI #5978] Canonicalize Trino IT in GitHub Action workflow
- [KYUUBI #5962] Bump Flink 1.17.1 to 1.17.2
- [KYUUBI #5953] Update NOTICE
- [KYUUBI #5951] Improve doc for Session Conf Advisor
- [KYUUBI #5933] Happy New Year 2024
- [KYUUBI #5244] Corrected ENGINE_ALIVE_MAX_FAILURES conf version
- [KYUUBI #5856] Bump Spark 3.4.2
- [KYUUBI #5857] Dependencies CI check when
dev/dependencyList
changes - [KYUUBI #5837] Only enable Jacoco with explicit profile
- [KYUUBI #5827] Fix wrong test code about directory lineage
- [KYUUBI #5819] Fix rst syntax of missing blank line after code-block
- [KYUUBI #5802] Add scala-2.13 as supported Scala profile to the docs of building from source
- [KYUUBI #5801] Add spark-3.4 and spark-3.5 as supported Spark profiles to the docs of building from source
- [KYUUBI #5785] Fix flaky test - JpsApplicationOperation with spark local mode
- [KYUUBI #5737] Correct JDBC URL demo in Flink docs
- [KYUUBI #5617] Make flaky test
spnego batch rest client
reliable - [KYUUBI #5720] Fix flaky test in TPCHQuerySuite
- [KYUUBI #5697] Fix incorrect url reference in code and docs
- [KYUUBI #5599] Correct the parameter name of fetching op log in RESTful API
- [KYUUBI #5641] Bump Kyuubi 1.8.0 in Helm charts
- [KYUUBI #5640] Upgrade playground to Kyuubi 1.8.0 and Spark 3.4.1
- [KYUUBI #5637] Add known contributor translation
- [KYUUBI #5624] Remove npm run build warning about NODE_ENV=production
- [KYUUBI #5634] Smoother the icon animation on collapsing sidebar
- [KYUUBI #5626] Fix sytle linting violation in web ui
- Shaoyun Chen
- Chao Chen
- Flyangz
- Pengqi Li
- Bowen Liang
- Paul Lin
- Senmiao Liu
- Xiao Liu
- Peiyue Liu
- Junjie Ma
- Ocean22
- Cheng Pan
- Gianluca Principini
- Zhiming She
- SwordyZhao
- William Tong
- Fei Wang
- Zhen Wang
- Tao Wang
- Zeyu Wang
- Kang Wang
- Binjie Yang
- Kaifei Yi
- David Yuan
- Yuwei Zhan
- He Zhao
- Yifan Zhou
- Mingliang Zhu
- Yi Zhu
- liaoyt
- mrtisttt
- Fully verified with Java 8, 11, and 17, Scala 2.12, initial support Scala 2.13
- Fully verified with Spark 3.1, 3.2, 3.3, and 3.4, initial support Spark 3.5
- Fully verified with Flink 1.16, 1.17, and 1.18
- New Web UI (experimental)
- New Chat engine with OpenAI ChatGPT backend
- Major improvements of Spark Batch API, optimized for large-scale job submission (experimental)
- Major improvements of Spark AuthZ plugin, including enhanced Iceberg and DSv2 operators support
- Major improvements of Flink engine, including YARN Application Mode support
- Introducing Atlas hook support in Spark Lineage plugin
- Continuously Kyuubi Helm Chart & Spark on K8s improvements
- [KYUUBI #1265]
OPTIMIZE
where clause expression support - [KYUUBI #3420] Expose UI url on registering engine service
- [KYUUBI #3444] Support the planOnly mode of kyuubi spark engine support SQL lineage
- [KYUUBI #3929] Refactor lineage plugin to add LineageDispatcher
- [KYUUBI #4167] [AUTHZ] Introduce function support in PrivilegeBuilder with Serde layers
- [KYUUBI #4171] Support skip retrieving table’s properties to speed up GetTables operation
- [KYUUBI #4186] Spark showProgress with JobInfo
- [KYUUBI #4330] Non-temporary views do not resolve to a specific real table
- [KYUUBI #4331] [KYUUBI #4431] Lineage supports
Union
andMergeInto
- [KYUUBI #4415] Align the operation handle in server/engine for ExecuteScala, ExecutePython and PlanOnlyStatement
- [KYUUBI #4430] Lineage supports the
lateral view explode
statement - [KYUUBI #4433] Fix the case with
create a view from view
- [KYUUBI #4443] Do not set engine session init sql for alive probe session
- [KYUUBI #4450] Ignore unknown fields
policyPriority
when reading policy json file - [KYUUBI #4488] [KSHC] Keep object original name defined in HiveBridgeHelper
- [KYUUBI #4493] [AUTHZ] Enable Tests for Spark 3.1 with iceberg tables
- [KYUUBI #4497] Add RowFilteringTestBase to improve the test coverage
- [KYUUBI #4504] [AUTHZ] Fix source table privilege requirement when querying permanent view in Spark 3.1 and below
- [KYUUBI #4525] [KSHC] Partitioning predicates should take effect to filter data
- [KYUUBI #4530] [AUTHZ] Support non-English chars for MASK, MASK_SHOW_FIRST_4, and MASK_SHOW_FIRST_4
- [KYUUBI #4532] [AUTHZ] Displays the columns involved in extracting the aggregation operator
- [KYUUBI #4541] Support to customize the attributes to expose for Spark engine
- [KYUUBI #4560] [KSHC] Support Kerberized HMS in cluster mode w/o keytab
- [KYUUBI #4585] [AUTHZ] Authz policy file generation
- [KYUUBI #4587] Take care of table cache query stage
- [KYUUBI #4592] Support eagerly kill redundant executors
- [KYUUBI #4615] Support stage level schedule for final write stage
- [KYUUBI #4617] [AUTHZ] Collect results for filtered show objects ahead to prevent holding unserializable spark plan
- [KYUUBI #4620] [KSHC] Cut off transitive dependencies
- [KYUUBI #4636] Improve eagerly kill redundant executors
- [KYUUBI #4641] Add MaxFileSizeStrategy to limit max scan file size
- [KYUUBI #4644] Manually terminate the Py4JServer during engine shutdown
- [KYUUBI #4658] [AUTHZ] Fix InsertIntoHiveDirCommand classname so that we can extract the query in it when authorization.
- [KYUUBI #4662] [ARROW] Arrow serialization should not introduce extra shuffle for outermost limit
- [KYUUBI #4664] Fix empty relation when kill executors
- [KYUUBI #4672] [AUTHZ] Remove Support for Apache Ranger 0.6.0
- [KYUUBI #4676] [AUTHZ] Reuse users and namespaces in both tests and policy file generation
- [KYUUBI #4677] [AUTHZ] Check generated policy file in test suite
- [KYUUBI #4678] Improve FinalStageResourceManager kill executors
- [KYUUBI #4681] Set thread
CreateSparkTimeoutChecker
daemon - [KYUUBI #4683] Update
spark.sql.finalWriteStage.resourceIsolation.enabled
version - [KYUUBI #4702] [ARROW] CommandResultExec should not trigger job
- [KYUUBI #4710] [ARROW] LocalTableScanExec should not trigger job
- [KYUUBI #4710] [ARROW] Post driver-side metrics for LocalTableScanExec/CommandResultExec
- [KYUUBI #4716] [KYUUBI #4715][AUTHZ] Fix the incorrect class name of InsertIntoHiveDirCommand in table spec generator
- [KYUUBI #4717] [AUTHZ] Check Authz plugin’s spec json files in UT
- [KYUUBI #4720] [ARROW] Fix java.lang.NoSuchFieldError: IpcOption.DEFAULT for Spark-3.1/3.2
- [KYUUBI #4741] Kyuubi Spark Engine/TPC connectors support Spark 3.4
- [KYUUBI #4754] [ARROW] Use
KyuubiArrowConverters#toBatchIterator
instead ofArrowConverters#toBatchIterator
- [KYUUBI #4797] [ARROW] Reflective calls to the function
ArrowUtils#toArrowSchema
- [KYUUBI #4812] Generalize case transformation method for string type config entry
- [KYUUBI #4834] Reduce the scope of method references in Authz plugin cleanup shutdown hook
- [KYUUBI #4838] Fix spark operation exception leak in
withLocalProperties
method - [KYUUBI #4866] Add annotation for Iceberg tests in Authz plugin
- [KYUUBI #4869] [AUTHZ] Introduce table extractor for ResolvedIdentifier in Spark 3.4
- [KYUUBI #4871] [AUTHZ] Adapt plan changes for CreateNamespace and SetCatalogAndNamespace in Spark 3.4
- [KYUUBI #4873] [AUTHZ] Refactor Authz reflection with kyuubi-util’s DynMethods
- [KYUUBI #4874] [AUTHZ] Improve methods in AuthzUtils
- [KYUUBI #4875] [AUTHZ] Remove checking Spark v2 in tests since Spark v2 not supported
- [KYUUBI #4878] Delta support for Spark-3.4
- [KYUUBI #4880] [BUILD] Skip compiling useless files when making distribution
- [KYUUBI #4888] [AUTHZ] Remove filtering results for ShowDatabasesCommand in Spark 2.x
- [KYUUBI #4892] [AUTHZ] Make identifier part name comparison case insensitive in tests of PrivilegeBuilder
- [KYUUBI #4895] [ARROW] Reflective calls to the function
ArrowConverters#fromBatchIterator
- [KYUUBI #4899] [AUTHZ] Extract function from FunctionIdentifier for CreateFunction and DropFunction in Spark 3.4
- [KYUUBI #4903] [AUTHZ] Fix NoSuchElementException when listing database in CatalogImpl in Spark 3.4
- [KYUUBI #4909] [AUTHZ] Enable authz plugin tests for Spark 3.4
- [KYUUBI #4910] Extract table from ResolvedIdentifier for DropTable in Spark 3.4
- [KYUUBI #4913] Eliminate unnecessary output in ut “union an unmasked table”
- [KYUUBI #4914] [AUTHZ] Reuse extractor singleton instance with generalized getter for supported extractor types
- [KYUUBI #4915] Adapt database not exist error message change for Spark 3.4
- [KYUUBI #4916] [AUTHZ] Support ReplaceData and compatible Spark 3.4 and 3.5
- [KYUUBI #4916] Fix Iceberg merge into UT
- [KYUUBI #4917] [AUTHZ] Table owner undefined in Iceberg 1.3.0 on Spark 3.4
- [KYUUBI #4923] [ARROW] Update arguments of
ArrowUtils#toArrowSchema
function - [KYUUBI #4925] Add default catalog using
spark_catalog
with the lineage result - [KYUUBI #4935] More than target num of executors may survive after FinalStageResourceManager did kill
- [KYUUBI #4937] Cleanup spark catalog shim and renamed to catalog utils
- [KYUUBI #4952] Enhance AWS Glue default database does not exist error message on opening session
- [KYUUBI #4995] Use hadoop conf and hive conf from catalog options
- [KYUUBI #4999] [KSHC] Kyuubi-Spark-Hive-Connector support Apache Spark 3.4
- [KYUUBI #5002] Fail the engine fast when no incoming connection in CONNECTION mode
- [KYUUBI #5005] Remove default settings
spark.sql.execution.topKSortFallbackThreshold
- [KYUUBI #5007] Remove blank SparkSimpleStatsReportListener
- [KYUUBI #5009] Pass Spark Engine Log Path to Spark Conf
- [KYUUBI #5017] [KSHC] Support Parquet/Orc provider is splittable
- [KYUUBI #5018] Make kyuubi spark extension compatible with Spark 3.4
- [KYUUBI #5022] [KSHC] CreateTable should use the correct provider
- [KYUUBI #5023] [KSHC] TableIdentify don’t attach catalog
- [KYUUBI #5025] Use formatDuration instead of formatDurationVerbose in Spark UI’s engine tab
- [KYUUBI #5028] Update session hadoop conf to catalog hadoop conf
- [KYUUBI #5030] Support get query id in Spark engine
- [KYUUBI #5035] Spark engine session page display session end time and duration
- [KYUUBI #5045] Initialize EXECUTOR_POD_NAME_PREFIX_MAX_LENGTH before using
- [KYUUBI #5055] [AUTHZ] Support building function privileges in Spark 3.4
- [KYUUBI #5073] Correct the method name in SparkSQLLineageParserHelperSuite
- [KYUUBI #5075] Refine tests to adapt Spark 4.0
- [KYUUBI #5076] Add KDF engine_url
- [KYUUBI #5105] GetColumns gets duplicate column names when Spark integrates with Iceberg
- [KYUUBI #5136] Spark App may hang forever if FinalStageResourceManager killed all executors
- [KYUUBI #5148] Improve spark.driver.host assignment in Spark on K8s client mode
- [KYUUBI #5170] Identifier compatible with Spark 3.4
- [KYUUBI #5172] [AUTHZ] Check USE permissions for DESCRIBE FUNCTION
- [KYUUBI #5180] Replace deprecated classes by equivalent in scala.tools.nsc.interpreter package
- [KYUUBI #5184] Rename Kyuubi’s StageInfo to SparkStageInfo to fix class mismatch
- [KYUUBI #5188] Make server module and Spark sql engine module compilable on Scala 2.13
- [KYUUBI #5189] [AUTHZ] Make spark authz module compilable on Scala 2.13
- [KYUUBI #5191] Make Spark extension plugin compilable on Scala 2.13
- [KYUUBI #5192] Make Spark sql lineage plugin compilable on Scala 2.13
- [KYUUBI #5193] Make Spark hive connector plugin compilable on Scala 2.13
- [KYUUBI #5195] Make Spark TPC-H connector plugin compilable on Scala 2.13
- [KYUUBI #5196] Enable CI Test on Scala 2.13 and support custom or spark-core extracted Scala version for Spark’s engine
- [KYUUBI #5196] Extract spark core scala version lazily and respect engine env
- [KYUUBI #5199] Read all columns of metadata to prevent column missing
- [KYUUBI #5202] Switch to Spark 3.4 by default
- [KYUUBI #5224] Modify the README file of authz to make spark 3.4 the default version
- [KYUUBI #5225] [KSHC] Unify the exception handling of v1 and v2 during dropDatabase
- [KYUUBI #5232] In SparkOperation#cleanup always calls cancelJobGroup even though it’s in the completed state
- [KYUUBI #5257] [LINEAGE] Adapt
CreateTableAsSelect
plan to Spark 3.5 changes - [KYUUBI #5268] [AUTHZ] Extract method for enabling authorization in single call mode
- [KYUUBI #5269] Duplicated AnalyzesTablesCommand in kyuubi authz
- [KYUUBI #5271] AnalyzeTableCommand should also add table write privilege
- [KYUUBI #5291] Verify Spark engine is compatible with Spark 3.5.0
- [KYUUBI #5317] [KSHC] Hive Connector throws NotSerializableException on reading Hive Avro partitioned table
- [KYUUBI #5336] Spark extension supports Spark 3.5
- [KYUUBI #5339] Avoid using
kyuubi.session.engine.initialize.timeout
string - [KYUUBI #5414] [KSHC] Reader should not pollute the global hiveConf instance
- [KYUUBI #5426] [KSHC] Avoid use class.newInstance directly
- [KYUUBI #5550] Optimizing TPC-DS dataset generation for 10x speedup
- [KYUUBI #1652] Support Flink YARN application mode
- [KYUUBI #4367] Support Flink 1.17
- [KYUUBI #4387] Remove support for Flink 1.14
- [KYUUBI #4446] Fix connections blocked by Flink insert statements
- [KYUUBI #4495] Support Flink job management statements
- [KYUUBI #4517] Fix multiple executions lead to abnormal results on Flink 1.14
- [KYUUBI #4635] Support flink time type in query operation
- [KYUUBI #4666] Support flink varbinary type in query operation
- [KYUUBI #4745] Support Flink’s LocalZonedTimestamp DataType
- [KYUUBI #4806] Support time-out incremental result fetch for Flink engine
- [KYUUBI #4861] Support Flink session idleness
- [KYUUBI #4938] Implement Kyuubi UDF in Flink engine
- [KYUUBI #5080] Fix EmbeddedExecutorFactory not thread-safe during bootstrap
- [KYUUBI #5106] Improve logs for fatal errors
- [KYUUBI #5108] Fix FileNotFoundException during Flink engine bootstrap
- [KYUUBI #5160] Refactor getNextRowSetInternal to support fetch streaming data
- [KYUUBI #5190] Explicitly name Flink bootstrap SQL in application mode
- [KYUUBI #5200] Optimize Flink application name generating
- [KYUUBI #5222] Remove unused provided dependencies in Flink SQL engine
- [KYUUBI #5238] Fix credentials may break Flink engine launch command
- [KYUUBI #5315] Propagate HIVE_CONF_DIR on launching Flink engine
- [KYUUBI #5405] Support Flink 1.18
- [KYUUBI #5531] Fix flaky FlinkOperationOnYarnSuite by enlarging the max rows setting
- [KYUUBI #3374] Support password authentication for Trino engine
- [KYUUBI #4412] Align the server/engine session handle for flink/hive/trino/jdbc engines
- [KYUUBI #4491] Fix Trino typo
- [KYUUBI #4522]
use:catalog
should execute before thanuse:database
- [KYUUBI #4544] Initial implement Kyuubi Chat Engine
- [KYUUBI #4548] Kyuubi Chat Engine supports Chinese questions and HTTP proxy
- [KYUUBI #4554] Code improvement in ChatGPTProvider
- [KYUUBI #4556] Refactor ChatGPTProvider to use
openai-java
client - [KYUUBI #4558] Make ChatGPT model ID configurable
- [KYUUBI #4559] Pretty chat engine launch command
- [KYUUBI #4711] JDBC client should catch task failed exception instead of NPE in the incremental mode
- [KYUUBI #4809] Set session user in ChatGPT request
- [KYUUBI #4810] Request and use a single choice for chat completion
- [KYUUBI #4881] JDBCEngine performs initialization sql
- [KYUUBI #4940] Implement Kyuubi UDF in Hive engine
- [KYUUBI #5120] Allow using session’s user and password for connecting database in JDBC engine
- [KYUUBI #5282] Support configure Trino session conf in
kyuubi-default.conf
- [KYUUBI #5333] Adaptive call fetchLaunchEngineResult
- [KYUUBI #5570] Fix memory leak when using incremental collect mode in JDBC engine
- [KYUUBI #3653] AdminResource add list Kyuubi server api
- [KYUUBI #3739] Remove unused parameters in SessionsResource rest docs
- [KYUUBI #3872] Skip setting proxy user when keytab is provided on batch session
- [KYUUBI #3876] Update the rest api docs for open session response
- [KYUUBI #3920] Fix flaky test - distribute lock
- [KYUUBI #3957] Rename the config prefix from kyuubi.server.batch.limit to kyuubi.server.limit.batch
- [KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
- [KYUUBI #4344] Expose exec pool work queue size metrics
- [KYUUBI #4364] Add metrics for user opened connections with session type
- [KYUUBI #4376] Support to config the Kyuubi service administrator with Kyuubi conf
- [KYUUBI #4388] Limit the max rows for get nextRowSet api
- [KYUUBI #4404] Support to list/close sessions in AdminResource
- [KYUUBI #4407] Adapt SLF4J2
- [KYUUBI #4424] Catch No Node Exception, when list kyuubi engines
- [KYUUBI #4439] Add list/close operation method for AdminResource
- [KYUUBI #4462] Fix variable usage issue in
SessionManager#stop
- [KYUUBI #4464] Simplify and improve log for JDBCMetadataStore
- [KYUUBI #4467] Tolerate Driver Pod ephemerally invisible after submitting
- [KYUUBI #4472] add session/operation methods in AdminRestApi
- [KYUUBI #4480] Engine alive probe should close thrift connection on engine lost
- [KYUUBI #4482] Support to disable batch resource upload function
- [KYUUBI #4483] Enable REST frontend protocol by default
- [KYUUBI #4496] Call
super.clusterManager()
whenbatchConf
does not havespark.master
- [KYUUBI #4498] Support to list sessions/operations with conditions in AdminRestApi
- [KYUUBI #4501] Allow administrator to specify the user for engine admin
- [KYUUBI #4506] Return sessionType in session data
- [KYUUBI #4516] Return kyuubi instance in SessionData and OperationData
- [KYUUBI #4518] Return engine id in SessionData and post kyuubi instance in KyuubiOperationEvent
- [KYUUBI #4519] Update metadata when batch application state changed
- [KYUUBI #4523] support close engine session gracefully
- [KYUUBI #4526] Defer evaluation for GroupProvider
- [KYUUBI #4540] Support to deny some users to make connection
- [KYUUBI #4540] Create session limiter if user unlimited list or deny list nonempty
- [KYUUBI #4563] Format error log output in case of metrics json file not found
- [KYUUBI #4575] Fix the empty last line may causes the session to exit directly
- [KYUUBI #4579] Fix the unstable situation of test cases for Trino-Fronted-Service
- [KYUUBI #4584] Post the session exception to EventBus in time
- [KYUUBI #4594] Support PUT method in REST client
- [KYUUBI #4611] support all sessions api in SessionRestApi
- [KYUUBI #4618] Admin Resource list operations with sessionHandle filter
- [KYUUBI #4623] KubernetesApplicationOperation uses Informer instead of list
- [KYUUBI #4639] Support to specify confOverlay when executing statement with RESTful API
- [KYUUBI #4652] Fix JaasConfiguration ClassNotFoundException for Hadoop 3.3.4 and previous
- [KYUUBI #4653] [KYUUBI 4650] LogDivertAppender supports reading RollingFileAppender pattern
- [KYUUBI #4669] LDAP authentication allows auth user contains domain when bind.dn/pw enabled
- [KYUUBI #4688] Fix the failure to read the operation log after executing catalog and database operation
- [KYUUBI #4691] Configure FAIL_ON_UNKNOWN_PROPERTIES to false for KyuubiScalaObjectMapper
- [KYUUBI #4713] Fix false positive result in SchedulerPoolSuite
- [KYUUBI #4719] Support submission timeout for yarn application manager and get the applicationInfo in-memory
- [KYUUBI #4731] Support batch session conf advisor
- [KYUUBI #4733] Introduce Kafka event logger for server events
- [KYUUBI #4738]
AdminResource.getEngineSpace
should use primary group name on GROUP share level - [KYUUBI #4739] Add operation lock instead of locking state Enumeration
- [KYUUBI #4746] Do not recreate async request executor if has been shutdown
- [KYUUBI #4749] Fix flaky test issues in SchedulerPoolSuite
- [KYUUBI #4753] KyuubiConf.unset should not log deprecation warning
- [KYUUBI #4767] Correct the submit time for BatchJobSubmission and check applicationInfo if submitted application
- [KYUUBI #4777] Deregister event handlers when stopping server with event handler made auto-closeable
- [KYUUBI #4780] Get engine application info with interval to prevent frequent call to resource manager
- [KYUUBI #4786] Support yarn-client and yarn-cluster for YarnApplicationOperation
- [KYUUBI #4790] Initial implement Batch V2
- [KYUUBI #4791] Add helper method to simplify REST enabled judgment
- [KYUUBI #4792] Enhance hardcode session keywords and remove unused code
- [KYUUBI #4796] Expose JVM attributes to metrics
- [KYUUBI #4798] Allows BatchJobSubmission to run in sync mode
- [KYUUBI #4801] Using different engine submit timeout config for kubernetes and yarn
- [KYUUBI #4807] Fix flaky test “basic batch rest client”
- [KYUUBI #4807] Retry more times to fix flaky test “spnego batch rest client”
- [KYUUBI #4811] Do not update app info after batch or application terminated
- [KYUUBI #4813] Add the fetch orientation parameter to the /v1/operations/:operationId/log interface
- [KYUUBI #4814] Introduce Apache Atlas hook support in lineage plugin
- [KYUUBI #4816] Correct the implementation of cleanup K8s terminated appInfo
- [KYUUBI #4829] Support to expose operation metrics
- [KYUUBI #4830] Fix flaky test “support to interrupt the thrift request if remote engine is broken”
- [KYUUBI #4835] Using hive conf to check whether to apply HIVE_DELEGATION_TOKEN
- [KYUUBI #4836] Set UncaughtExceptionHandler for thread to log exception
- [KYUUBI #4840] Return cached appInfo iif both op and app are terminated
- [KYUUBI #4841] Log more info for closing session
- [KYUUBI #4842] Fix typo in etcd client config
- [KYUUBI #4843] Support multiple kubernetes contexts and namespaces
- [KYUUBI #4847] Close the session immediately when engine corrupt
- [KYUUBI #4847] Close the session immediately when engine connection closed
- [KYUUBI #4850] Change default EC spec to secp521r1 for internal session variable signature
- [KYUUBI #4855] Change socket “connection refused” message match pattern
- [KYUUBI #4857] Fix flaky test TFrontendServiceSuite
- [KYUUBI #4858] Relax test message assertion in “sync query causes engine crash”
- [KYUUBI #4859] HttpException message should not be null
- [KYUUBI #4865] Fix flaky test BackendServiceMetricSuite
- [KYUUBI #4876] fix Flaky test BackendServiceMetricSuite (#4924)
- [KYUUBI #4889] Admin command line supports list server command
- [KYUUBI #4890] Return operation metrics in OperationData and fix typo
- [KYUUBI #4893] Prevent null collection for rest dto
- [KYUUBI #4898] Fix logOperation multiple read with missing line
- [KYUUBI #4919] Fix the flaky tests in the “open batch session” functionality within the “BatchesResourceSuite”
- [KYUUBI #4920] Add OperationRestApi
- [KYUUBI #4946] Alter the order of initLoggerEventHandler
- [KYUUBI #4949] For operation getNextRowSet method, with operation lock required
- [KYUUBI #4950] Migrate Kyuubi embedded database from Derby to SQLite
- [KYUUBI #4950] Misc improvement for usage of SQLite
- [KYUUBI #4950] SQLite schema files should start from 1.8.0
- [KYUUBI #4951] Add reload4j support
- [KYUUBI #4977] Log error message when REST API invocation error occurs
- [KYUUBI #4978] Fix flaky test: close expired operations
- [KYUUBI #4978] Fix flaky test: close expired operations
- [KYUUBI #4979] Fix flaky test: support to interrupt the thrift request if remote engine is broken
- [KYUUBI #4981] Refactor code of closeBatchSession
- [KYUUBI #4985] Fix log level restore behavior of
KyuubiFunSuite#withLogAppender
- [KYUUBI #4986] Always use Files#deleteIfExists
- [KYUUBI #4994] Respect the engine type and share level for listing all engines
- [KYUUBI #4994] Support listing all engines
- [KYUUBI #4996] Support to refresh kubernetes configs dynamically
- [KYUUBI #5004] Fix typo for kubernetes allowed context and namespace check
- [KYUUBI #5010] Make Kyuubi server’s connection URL configurable
- [KYUUBI #5029] Close the alive probe session after engine session closed
- [KYUUBI #5036] Fix Operation.close not update complete timestamp
- [KYUUBI #5043] Destroy the build process when waitCompletion is false and the engine is running in cluster mode
- [KYUUBI #5050] Add admin checking in
getProxyUser
- [KYUUBI #5056] Return more helpful error message to http client
- [KYUUBI #5063] Support to filter batch with batch name
- [KYUUBI #5065] Call destroy first on killing Spark startup process to allows it release temp files
- [KYUUBI #5065] Graceful close the process when launch engine timeout
- [KYUUBI #5078] Make
kyuubi_instance
nullable in metadata table schema - [KYUUBI #5081] Minor refactor JDBCMetadataStore
- [KYUUBI #5089] Limit maximum engine startup concurrency of kyuubi server
- [KYUUBI #5130] Allow setting Zookeeper SPN using Kyuubi configuration
- [KYUUBI #5131] Create index on
metastore
.create_time
- [KYUUBI #5138] Fix flaky test BatchesResourceSuite - get batch session list
- [KYUUBI #5145] Change embedded Zookeeper server to method-local variable
- [KYUUBI #5158] Allow embedded Zookeeper binding IP address
- [KYUUBI #5165] Build Spark Driver/Executor Pod Name(Prefix) in process
- [KYUUBI #5185] Use Set collection for order-insensitive configs
- [KYUUBI #5187] Fix info message in UserFilter
- [KYUUBI #5196] Extract spark core scala version lazily and respect engine env
- [KYUUBI #5201] Allow disabling batch impl v2 on globally
- [KYUUBI #5204] Thrift HTTP FE checks req conf nullable before evaluating proxy user
- [KYUUBI #5206] Try to kill pod with label if no ApplicationInfo found to prevent pod leak
- [KYUUBI #5210] Cancel operation will cause the log file node to leak
- [KYUUBI #5213] Check config value by enum values
- [KYUUBI #5216] Workaround for negative counter in SessionLimiter
- [KYUUBI #5218] Improve logs of KubernetesApplicationOperation
- [KYUUBI #5220] Batch submitter should only block submitting stage
- [KYUUBI #5220] Batch submitted considers application state
- [KYUUBI #5243] Distinguish metadata between batch impl v2 and recovery
- [KYUUBI #5289] RESTful API should always print audit log
- [KYUUBI #5290] Batch impl v2 should pick batch jobs in FIFO
- [KYUUBI #5300] Batch v2 should not update kyuubi_instance when open session
- [KYUUBI #5305] Return dummy log when batch job before being added to session manager
- [KYUUBI #5306] YarnApplicationOperation supports proxy user
- [KYUUBI #5310] Batch session recovery should start after HTTP server getting started
- [KYUUBI #5327] Add priority field in batch session metadata table definition
- [KYUUBI #5328] Batch supports priority scheduling
- [KYUUBI #5340] AdminResourceSuite make engine idle time 3min
- [KYUUBI #5380] Create PySpark batch jobs tests for RESTful API
- [KYUUBI #5381] Change the default metrics reporter to Prometheus
- [KYUUBI #5391] Close batch session after batch operation completion for idle timeout
- [KYUUBI #5392] Add query timeout monitor on server-side in ExecuteStatement
- [KYUUBI #5408] MetadataManager tries MySQL 8 driver class first
- [KYUUBI #5412] Resolve the relative zk configuration dir based on KYUUBI_HOME
- [KYUUBI #5441] Make the configuration kyuubi.zookeeper.embedded.data.log.dir effective
- [KYUUBI #5451] Ignore NoSuchFileException during OperationLog.close()
- [KYUUBI #5499] [KYUUBI #2503] Catch any exception when closing idle session
- [KYUUBI #5513] Always redirect delete batch request to Kyuubi instance that owns batch session
- [KYUUBI #5522] Ignore main class for PySpark batch job submission
- [KYUUBI #5528] Remove duplicate properties tag and respect KYUUBI_LOG_DIR
- [KYUUBI #5566] InternalRestClient respects
kyuubi.engine.security.enabled
to add HTTP auth header - [KYUUBI #5566] Check InternalSecurityAccessor is initialized only when
kyuubi.engine.security.enabled
is true - [KYUUBI #5568] Check administrator only when security is enabled
- [KYUUBI #5568] Fix security enabled judgement
- [KYUUBI #5606] Handle engine listing request properly for users who have not created engine
- [KYUUBI #3887] Fix kyuubiServerPrincipal logic in KyuubiCommands
- [KYUUBI #4305] Backport HIVE-15820: comment at the head of beeline -e
- [KYUUBI #4438] Render the batch info map in kyuubi-ctl
- [KYUUBI #4439] Add dto class for operation data
- [KYUUBI #4452] Strip the redundant leading and tailing slash of getZooKeeperNamespace.
- [KYUUBI #4479] Restore JDBC Kerberos authentication behavior for UGI.doAs
- [KYUUBI #4489] Fix missing
org.apache.commons.logging
by includingjcl-over-slf4j
dependency in REST client - [KYUUBI #4492] Correct engine subdomain calculation in
kyuubi-ctl
- [KYUUBI #4508] Beeline should reset stdin after consuming init SQL file
- [KYUUBI #4581] Add KyuubiCommands parse python snippets unit test
- [KYUUBI #4609] get engineRefId in KyuubiConnection
- [KYUUBI #4619] Fix beeline with -e when there are other SQL statements before the source statement, the source statement cannot be executed normally
- [KYUUBI #4657] Building rest client to kyuubi instance including original host urls
- [KYUUBI #4962] Backport HIVE-19048: init script errors are ignored
- [KYUUBI #4965] Beeline supports
--python-mode
option and remove comments for non-python mode - [KYUUBI #4969] Run JUnit tests on beeline module
- [KYUUBI #5054] Refine the kyuubi client version properties file to prevent conflicts
- [KYUUBI #5100] Fix unpaired quote in message pattern
- [KYUUBI #5101] Remove unused StringBuilders in ResultSets
- [KYUUBI #5129] KyuubiBeeline should redirect JDK logging
- [KYUUBI #5179] Use Iterable instead of Seq in kyuubi-ctl commands
- [KYUUBI #5297] New RetryableClient get http://null server uri from metadata.kyuubiInstance
- [KYUUBI #5480] Support setting kyuubi hive jdbc client protocol version
- [KYUUBI #5564] Support to config ticket cache location in JDBC URL
- [KYUUBI #3420] Kyuubi Server Proxy Engine UI
- [KYUUBI #3646] Front-end style should bracket same line
- [KYUUBI #3646] Init Session Statistic Page
- [KYUUBI #3648] Add Session Detail Page
- [KYUUBI #3650] Add Operation Statistics Page
- [KYUUBI #3652] Add Kyuubi Server Management page
- [KYUUBI #3654] Add Engine Manager Page
- [KYUUBI #4537] Enable Vite proxy server for web ui development
- [KYUUBI #4565] Fix out-date
README.md
aboutDevelopment Project
- [KYUUBI #4622] Session api use admin api for security
- [KYUUBI #4626] Fix Web UI CI check fail with pnpm the lockfile needs updates
- [KYUUBI #4757] Move Statistics to Management Menu
- [KYUUBI #4982] Add query string to forward requests to Engine UI
- [KYUUBI #5259] Web UI supports Swagger pages
- [KYUUBI #5463] Adjust the file directory structure
- [KYUUBI #5484] Remove legacy Web UI
- [KYUUBI #5573] Delete parts of the Kyuubi Web UI that are not useful
- [KYUUBI #5586] Kyuubi Web UI show current version
- [KYUUBI #5608] Kyuubi version should always use the light color
- [KYUUBI #5620] Remove dummy information card from Overview page
- [KYUUBI #4308] Make README more welcoming
- [KYUUBI #4312] Fix over-matched README pattern for markdown styling
- [KYUUBI #4367] Kyuubi requires Flink 1.16/1.17
- [KYUUBI #4387] Update Flink version requirements
- [KYUUBI #4568] Make kyuubi-ctl doc enable variable ‘release’ automatic substitution
- [KYUUBI #4583] Rename Flink Table Store to Apache Paimon (Incubating)
- [KYUUBI #4593] Keep promote the release step update-to-date
- [KYUUBI #4599] Simplify project version evaluation
- [KYUUBI #4607] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Hive SQL Query Engine
- [KYUUBI #4608] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Trino SQL Query Engine
- [KYUUBI #4614] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Flink SQL Query Engine
- [KYUUBI #4624] Fix table headers in kyuubi_vs_hive.md
- [KYUUBI #4627] Fix a typo in rest_api.md.
- [KYUUBI #4655] Enrich docs for Kyuubi Hive JDBC driver
- [KYUUBI #4655] Recover the missing JDBC URL
- [KYUUBI #4722] Kyuubi requires Spark 3.1 and above now
- [KYUUBI #4727] kyuubi-spark-lineage has no transitive deps
- [KYUUBI #4734] Fix typo in docs of custom event handler
- [KYUUBI #4742] Add docs for Flink application mode
- [KYUUBI #4763] Fix the Kyuubi JDBC kerberos parameters
- [KYUUBI #4800] Update readthedocs.yaml
- [KYUUBI #4864] Fix grammar in HA section
- [KYUUBI #4933] Mark
spark.sql.optimizer.insertRepartitionNum
config for Spark 3.1 only - [KYUUBI #4954] Fix variable reference in quick_start_with_jdbc
- [KYUUBI #4960] Fix doc for pyhive client.
- [KYUUBI #5049] PyHive Kerberos usage doc
- [KYUUBI #5083] Add LDAP document
- [KYUUBI #5085] Update config section of deploy on kubernetes
- [KYUUBI #5094] Add Python JayDeBeApi doc
- [KYUUBI #5099] Document link failure
- [KYUUBI #5118] Add Flink KDF doc
- [KYUUBI #5122] Hive KDF usage
- [KYUUBI #5127] Improvements for markdown builder
- [KYUUBI #5146] Fix link of IntelliJ IDEA Setup Guide
- [KYUUBI #5153] Minor change w/ to with
- [KYUUBI #5154] Move configuration docs to the top level
- [KYUUBI #5162] Add statements about contributions co-authored with AIGC in developer guide page
- [KYUUBI #5178] Make MarkdownBuilder not extending Growable
- [KYUUBI #5227] Added deploy mode info
- [KYUUBI #5247] Correct building command for Spark authz/lineage plugins
- [KYUUBI #5273] Update release docs
- [KYUUBI #5275] Improve and fix comparison and regeneration for golden files
- [KYUUBI #5294] Update supported dialects for JDBC engine
- [KYUUBI #5335] Set markdown file EOL
- [KYUUBI #5345] Fix non-kerberized code description in quick_start_with_jdbc
- [KYUUBI #5347] Fix java code errors in quick_start_with_jdbc
- [KYUUBI #5409] Update config docs regeneration hints in developer guide
- [KYUUBI #5432] Fix typo in README.md
- [KYUUBI #5523] Update the Kyuubi supported components version
- [KYUUBI #4078] Fix shellcheck violations in scripts of /bin folder
- [KYUUBI #4187] Windows build-info script supports revision time
- [KYUUBI #4278] Use new Apache ‘closer.lua’ syntax to obtain Maven
- [KYUUBI #4295] Introduce
super-linter
action for linting JSON, XML, ENV files and bash_exec - [KYUUBI #4299] Bump JUnit to 4.13.2
- [KYUUBI #4299] Bump MySQL JDBC driver to 8.0.32
- [KYUUBI #4315] Check modules available before build step in dependency workflow
- [KYUUBI #4347] Bump maven download plugin from 1.6.6 to 1.6.8
- [KYUUBI #4348] Cache engine archives in CI jobs for maven download plugin
- [KYUUBI #4397]
build/dist
supports--web-ui
- [KYUUBI #4413] Build docker image should include web UI
- [KYUUBI #4447] Bump log4j from 2.19.0 to 2.20.0
- [KYUUBI #4448] Bump Netty from 4.1.87.Final to 4.1.89.Final
- [KYUUBI #4451] Skip etcd test if no docker env
- [KYUUBI #4453] Bump Kubernetes Client Version to 6.4.1
- [KYUUBI #4461] Add
Kyuubi Kubernetes IT
Module to CI style check - [KYUUBI #4466] Use bitnami/minio docker image
- [KYUUBI #4471] Prompted command for docs generation should skip Java tests
- [KYUUBI #4484] [K8S][HELM] Set specific Kyuubi version to the chart appVersion
- [KYUUBI #4494]
bin/kyuubi
should useexec
to run Kyuubi server - [KYUUBI #4513] Bump Kyuubi 1.7.0 in Playground
- [KYUUBI #4546] Fully exclude
metrics
dir from git - [KYUUBI #4557] [K8S][HELM] Kyuubi server should bind Pod IP by default
- [KYUUBI #4572] Bump Iceberg from 1.1.0 to 1.2.0
- [KYUUBI #4576] Multi arch build for ARM support in Docker Hub
- [KYUUBI #4589] [K8S][HELM] Update template on port-forward usage guide NOTES.txt
- [KYUUBI #4590] Bump delta from 2.2.0 to 2.3.0
- [KYUUBI #4598] Bump Flink-1.15 from 1.15.3 to 1.15.4
- [KYUUBI #4600] Bump Iceberg 1.2.0 for playground
- [KYUUBI #4601] Bump Hadoop 3.3.5 for playground
- [KYUUBI #4605] [K8S][HELM] Set IfNotPresent pullPolicy by default
- [KYUUBI #4606] [K8S][HELM] Add command and args configuration support
- [KYUUBI #4615] Bump Ranger from 2.3.0 to 2.4.0
- [KYUUBI #4628] Add Dependency management ISSUE_TEMPLATE
- [KYUUBI #4632] Add rule for labeler to tag
module:ui
- [KYUUBI #4638] Update dependency.yml as label in not permitted in markdown block
- [KYUUBI #4652] Upgrade Hadoop from 3.3.4 to 3.3.5
- [KYUUBI #4695] Bump super-linter action from v4 to v5
- [KYUUBI #4696] Upgrade scalafmt from 3.7.1 to 3.7.3
- [KYUUBI #4697] [K8S][HELM] Add template rendering info to README
- [KYUUBI #4698] [K8S][HELM] Centralize Kyuubi labels definition
- [KYUUBI #4706] [K8S][HELM] Use template comments for the chart license header
- [KYUUBI #4707] [K8S][HELM] Fix README typo and minor reformatting
- [KYUUBI #4712] Bump Spark from 3.2.3 to 3.2.4
- [KYUUBI #4737] Restore Project & Community Status in README.md
- [KYUUBI #4744] Remove Hudi integration tests
- [KYUUBI #4747] Install and cache build/mvn in CI builds
- [KYUUBI #4768] Guidelines for Document and Code contributions
- [KYUUBI #4772] Bump Jersey from 2.39 to 2.39.1
- [KYUUBI #4783] Use pnpm v8 and update pnpm lockfile to v6 schema
- [KYUUBI #4784] Use nodejs v18 in CI builds and maven web-ui builds
- [KYUUBI #4788] [K8S][HELM] Use StatefulSet instead of Deployment
- [KYUUBI #4805] Bump openai-java from 0.11.1 to 0.12.0
- [KYUUBI #4808] Bump Jetty from 9.4.50 to 9.4.51
- [KYUUBI #4818] [K8S][HELM] Update default Kyuubi version to 1.7.1
- [KYUUBI #4819] [K8S][HELM] Refactor liveliness and readiness probes
- [KYUUBI #4824] Bump Jackson from 2.14.2 to 2.15.0
- [KYUUBI #4827] Bump Apache Arrow from 11.0.0 to 12.0.0
- [KYUUBI #4828] Exclude macOS tar extended metadata in build/dist
- [KYUUBI #4849] Open modules to enable JDK 17 support
- [KYUUBI #4851] Bump Maven 3.8.8
- [KYUUBI #4852] Switch to Kyuubi Shaded Zookeeper
- [KYUUBI #4856] Add
restore-keys
foraction/cache
- [KYUUBI #4870] Add kyuubi-util and kyuubi-util-scala modules
- [KYUUBI #4877] Bump Flink 1.17 to 1.17.1
- [KYUUBI #4879] Refactor and promote reflection utils and cleanup similar reflection methods
- [KYUUBI #4883] Bump Iceberg 1.3.0
- [KYUUBI #4887] Refactor and add ut for ClassUtils
- [KYUUBI #4891] Bump Flink 1.16.2
- [KYUUBI #4894] Bump Mockito from 4.9.0 to 4.11.0 and ScalaTest from 3.2.15 to 3.2.16
- [KYUUBI #4904] Move AssertionUtils to kyuubi-util-scala module
- [KYUUBI #4905] Generalize util method for loading class from service loader
- [KYUUBI #4907] Add a maven profile ‘remote-debug’
- [KYUUBI #4908] Bump scalafmt from 3.7.3 to 3.7.4
- [KYUUBI #4911] Remove duplicate test annotations
org.apache.kyuubi.tags.*
in kyuubi-common - [KYUUBI #4912] Replace Scala’s assert in tests with scalatest’s for prettified error message
- [KYUUBI #4931] Bump vite from 3.0.0 to 4.2.3 and vitest from 0.22.0 to 0.32.0
- [KYUUBI #4937] Remove redundant quoteIfNeeded method
- [KYUUBI #4943] Bump Guava from 31.1 to 32.0.1
- [KYUUBI #4944] Code improvement for Java
- [KYUUBI #4947] Bump pnpm version in maven’s frontend plugin to 8.6.1 for lockfileVersion 6.1
- [KYUUBI #4948] Reuse versions of NodeJS and pnpm from maven properties
- [KYUUBI #4959] Code improvements for Scala
- [KYUUBI #4966] Bump Apache parent pom from 29 to 30
- [KYUUBI #4968] Simplify Option conversion
- [KYUUBI #4970] Unified reflection methods invokeAs and getField
- [KYUUBI #4974] Eliminate raw-types compilation warning in KyuubiBeeLine
- [KYUUBI #4975] Fixed run test link in PR template
- [KYUUBI #4990] Bump Spark 3.4.1
- [KYUUBI #4992] Bump Netty 4.1.93.Final
- [KYUUBI #5001] Bump Scala from 2.12.17 to 2.12.18
- [KYUUBI #5003] [K8S][HELM] Add doc strings to the chart values.yaml
- [KYUUBI #5007] Bump scalafmt from 3.7.4 to 3.7.5
- [KYUUBI #5019] Shows details of compilation info in SparkUI’s engine tab
- [KYUUBI #5024] Bump grpc-protobuf from 1.48.0 to 1.53.0
- [KYUUBI #5034] Bump tough-cookie from 4.0.0 to 4.1.3 in /kyuubi-server/web-ui
- [KYUUBI #5039] Use semantic versions and remove redundant version comparison methods
- [KYUUBI #5058] [K8S][HELM] Allow the user to set the priorityClass used by Kyuubi server Pod
- [KYUUBI #5059] Upgrade kubernetes client version to 6.7.2
- [KYUUBI #5064] Audit kubernetes application state change in separate log files
- [KYUUBI #5068] Bump Spark in spark-master profile from 3.5.0-SNAPSHOT to 4.0.0-SNAPSHOT
- [KYUUBI #5070] Bump word-wrap from 1.2.3 to 1.2.4 in web UI
- [KYUUBI #5072] Fix KyuubiOperationWithEngineSecuritySuite and related issues
- [KYUUBI #5090] Fix AllKyuubiConfiguration to generate redundant blank lines in Windows
- [KYUUBI #5113] Remove dummy Spark Kudu connector module
- [KYUUBI #5116] Bump Hadoop 3.3.6
- [KYUUBI #5117] Code improvements for Java and Scala code
- [KYUUBI #5121] Extract dev scripts for regenerating the golden files
- [KYUUBI #5132] [K8S][HELM] Kyuubi add server metrics monitor
- [KYUUBI #5135] Exclude web-ui coverage reports in rat plugins
- [KYUUBI #5149] Correct error message of ReflectUtils’s invokeAs when method not found
- [KYUUBI #5150] Bump Spark 3.3.3
- [KYUUBI #5152] Check milestone and assignees when merging pull request
- [KYUUBI #5177] Use Scala binary version placeholder in Maven module’s artifactId suffix
- [KYUUBI #5183] Introducing profile scala-2.13 for development use
- [KYUUBI #5196] Enable CI Test on Scala 2.13 and support custom or spark-core extracted Scala version for Spark’s engine
- [KYUUBI #5202] Update upload_nexus_staging
- [KYUUBI #5205] Upgrade
kubernetes-client
to 6.8.1 - [KYUUBI #5209] Reformat Scala source code crossing versions
- [KYUUBI #5212] Fix configuration errors causing by helm charts of prometheus services
- [KYUUBI #5214] Correct file path for scala tests in util-scala module
- [KYUUBI #5234] Upgrade Jetty version from 9.4.51.v20230217 to 9.4.52.v20230823
- [KYUUBI #5235] Daily deploy snapshot version on master and branch-1.7
- [KYUUBI #5250] Bump Iceberg from 1.3.0 to 1.3.1
- [KYUUBI #5252] Remove incubator link
- [KYUUBI #5253] Upgrade testcontainers-scala from 0.40.12 to 0.41.0
- [KYUUBI #5253] Supply testcontainers-scala-scalatest deps
- [KYUUBI #5293] Upgrade snakeyaml from 1.33 to 2.2
- [KYUUBI #5305] Fix get local log UT
- [KYUUBI #5308] Ensure release using Java 8
- [KYUUBI #5318] Pin maven-source-plugin 3.2.1
- [KYUUBI #5358] Bump iceberg 1.4.0
- [KYUUBI #5365] Don’t use Log4j2’s extended throwable conversion pattern in default logging configurations
- [KYUUBI #5425] Add thread name and ID to log4j2-defaults.xml
- [KYUUBI #5435] Improve Kyuubi On Kubernetes IT
- [KYUUBI #5443] Add kyuubi-kubernetes-it module to reformat script
- [KYUUBI #5444] Check style with profiles of Spark 3.4 and 3.5
- [KYUUBI #5483] Fix deploy command for Scala 2.13 TPC-DS/H connectors
- [KYUUBI #5483] Release Spark TPC-H/DS Connectors with Scala 2.13
- [KYUUBI #5486] Bump Kafka client version from 3.4.0 to 3.5.1
- [KYUUBI #5508] Upgrade download-maven-plugin from 1.6.8 to 1.7.1
- [KYUUBI #5544] Release contains Spark 3.5 extension
- [KYUUBI #5561] Binary distribution artifact ships database schema scripts
- [KYUUBI #5561] Rename folder to db-scripts
- [KYUUBI #5569] Fix pnpm-lock file version
- Deng An
- GitHub Bot
- CavemanIV
- Fu Chen
- Shaoyun Chen
- Chao Chen
- Derick Cypho
- Yang Du
- FourSpaces
- Xu Guo
- Karsonnel
- Denis Krivenko
- Kyofin
- Pengqi Li
- Xieming Li
- Guanhua Li
- Bowen Liang
- Tianlin Liao
- Hanna Liashchuk
- Paul Lin
- Senmiao Liu
- Peiyue Liu
- Naijie Liu
- Chenliang Lu
- Ruilei Ma
- Omkar Makhare
- Drake Youngkun Min
- Sidhant Nagpal
- Cheng Pan
- QianyongY
- Anurag Rajawat
- Rojin Raju
- Fei Wang
- Zhen Wang
- Kang Wang
- Miao Wang
- Bruce Wong
- XorSum
- Junjie Xu
- Binjie Yang
- Remzi Yang
- Kent Yao
- Xianxun Ye
- Kaifei Yi
- Xiduo You
- Ruguo Yu
- David Yuan
- Qiang Zhang
- Yaodong Zhang
- Yangming Zhao
- He Zhao
- Min Zhao
- Yifan Zhou
- Yi Zhu
- Yaogai Zhu
- Alex Zou
- Junjie Zu
- biangjuang
- camper42
- fantasticKe
- huangzhir
- huapan123456
- liaoyt
- mans2singh
- mrtisttt
- phyyou
- rhh777
- runzhliu
- touchida
- uicosp
- wangyepeng2
- weixi62961
- yehere
- Fix file handle leak caused by cancel operation
- [KYUUBI #5210] Cancel operation will cause the log file node to leak
- [KYUUBI #5269] [Bug] Dupllicated AnalyzesTablesCommand in kyuubi authz
- AngersZhuuuu
- ASiegeLion
- Enhance JDBC Client and Beeline Client, like: Support running Python scripts and codes in Beeline, fix Http authentication exception message is empty, backport HIVE-19048: Initscript errors are ignored, backport HIVE-15820: comment at the head of beeline -e.
- Enhance submission engine related code, like: support yarn-client and yarn-cluster for YarnApplicationOperation, different engine submit timeout config, destroy build process when waitCompletion is false and engine is running in cluster mode
- Fix many flaky tests and incorrect documentations
- [KYUUBI #4786] Support yarn-client and yarn-cluster for YarnApplicationOperation
- [KYUUBI #4801] Using different engine submit timeout config for kubernetes and yarn
- [KYUUBI #4811] Do not update app info after batch or application terminated
- [KYUUBI #4816] [K8S] Correct the implementation of cleanup terminated appInfo
- [KYUUBI #4830] [TEST] Fix flaky test “support to interrupt the thrift request if remote engine is broken”
- [KYUUBI #4840] Return cached appInfo iif both op and app are terminated
- [KYUUBI #4841] [MINOR] Log more info for closing session
- [KYUUBI #4807] [TEST] Retry more times to fix flaky test “spnego batch rest client”
- [KYUUBI #4807] [FOLLOWUP] Fix flaky test “basic batch rest client”
- [KYUUBI #4855] Change socket “connection refused” message match pattern
- [KYUUBI #4858] Relax test message assertion in “sync query causes engine crash”
- [KYUUBI #4919] Fix the flaky tests in the “open batch session” functionality within the “BatchesResourceSuite”
- [KYUUBI #4876] fix Flaky test BackendServiceMetricSuite
- [KYUUBI #4979] Fix flaky test: support to interrupt the thrift request if remote engine is broken
- [KYUUBI #4977] Log error message when REST API invocation error occurs
- [KYUUBI #4946] Alter the order of initLoggerEventHandler
- [KYUUBI #5072] [TEST] Fix KyuubiOperationWithEngineSecuritySuite and related issues
- [KYUUBI #5043] [1.7] Destroy the build process when waitCompletion is false and the engine is running in cluster mode
- [KYUUBI #3920] [TEST] Fix flaky test - distribute lock
- [KYUUBI #5204] Thrift HTTP FE checks req conf nullable before evaluating proxy user
- [KYUUBI #4838] Fix spark operation exception leak in
withLocalProperties
method - [KYUUBI #4835] [K8S] Using hive conf to check whether to apply HIVE_DELEGATION_TOKEN
- [KYUUBI #4915] Adapt database not exist error message change for Spark 3.4
- [KYUUBI #4952] Enhance AWS Glue default database does not exist error message on opening session
- [KYUUBI #5005] Remove default settings
spark.sql.execution.topKSortFallbackThreshold
- [KYUUBI #5025] Use formatDuration instead of formatDurationVerbose in SparkUI’s engine tab
- [KYUUBI #5019] Shows details of compilation info in SparkUI’s engine tab
- [KYUUBI #5045] Initialize EXECUTOR_POD_NAME_PREFIX_MAX_LENGTH before using
- [KYUUBI #5105] [Bug] getColumns gets duplicate column names when Spark integrates with Iceberg
- [KYUUBI #5148] Improve spark.driver.host assignment in Spark on K8s client mode
- [KYUUBI #5232] In SparkOperation#cleanup always calls cancelJobGroup even though it’s in the completed state
- [KYUUBI #4995] Use hadoop conf and hive conf from catalog options
- [KYUUBI #5028] Update session hadoop conf to catalog hadoop conf
- [KYUUBI #4859] HttpException message should not be null
- [KYUUBI #4969] [TEST] Run JUnit tests on beeline module
- [KYUUBI #4962] [1.7] Backport HIVE-19048: Initscript errors are ignored
- [KYUUBI #4965] [BEELINE] Support –python-mode option and remove comments for non-python mode
- [KYUUBI #4305] Backport HIVE-15820: comment at the head of beeline -e
- [KYUUBI #5100] [Minor] Fix unpaired quote in message pattern
- [KYUUBI #5101] [MINOR] Remove unused StringBuilders in ResultSets
- [KYUUBI #5129] KyuubiBeeline should redirect JDK logging
- [KYUUBI #4808] Bump Jetty from 9.4.50 to 9.4.51
- [KYUUBI #4836] Set UncaughtExceptionHandler for thread to log exception
- [KYUUBI #4857] Fix flaky test TFrontendServiceSuite
- [KYUUBI #4898] fix logOperation multiple read with missing line
- [KYUUBI #4949] For operation getNextRowSet method, with operation lock required
- [KYUUBI #4951] Add reload4j support
- [KYUUBI #4978] Fix flaky test: close expired operations
- [KYUUBI #4986] Always use Files#deleteIfExists
- [KYUUBI #4978] [FOLLOWUP] Fix flaky test: close expired operations
- [KYUUBI #4985] [TEST] Fix loglevel restore behavior of
KyuubiFunSuite#withLogAppender
- [KYUUBI #5036] Fix Operation.close not update complete timestamp
- [KYUUBI #5090] Fix AllKyuubiConfiguration to generate redundant blank lines in Windows
- [KYUUBI #5158] Allow embedded Zookeeper binding IP address
- [KYUUBI #5187] [Minor] Fix info message in UserFilter
- [KYUUBI #4818] [K8S] [HELM] Update default Kyuubi version to 1.7.1
- [KYUUBI #4828] [BUILD] Exclude macOS tar extended metadata in build/dist
- [KYUUBI #4851] [BUILD] Bump Maven 3.8.8
- [KYUUBI #4864] [DOCS] Fix grammar in HA section
- [KYUUBI #4954] [DOCS] Fix variable reference in quick_start_with_jdbc
- [KYUUBI #4960] [DOC] Fix doc for pyhive client.
- [KYUUBI #5049] [DOCS] PyHive Kerberos usage doc
- [KYUUBI #5085] Update config section of deploy on kubernetes
- [KYUUBI #5094] Add Python JayDeBeApi doc
- [KYUUBI #5099] Document link failure
- [KYUUBI #5083] [DOC] Add LDAP document
- [KYUUBI #5146] [DOC] Fix link of IntelliJ IDEA Setup Guide
- BruceWong96
- CavemanIV
- QianyongY
- bowenliang123
- cxzl25
- dev-lpq
- dnskr
- huangzhir
- lightning-L
- liunaijie
- lsm1
- mans2singh
- pan3793
- risyomei
- touchida
- turboFei
- uicosp
- ulysses-you
- wForget
- yaooqinn
- zhaohehuhu
- zhaomin1423
- Kyuubi requires Spark 3.1 and above
- Upgrade Hadoop from 3.3.4 to 3.3.5
- A series of improvements for Arrow based serialization
- A series of improvements for batch job of Spark on K8s/YARN
- Bug fix for ‘beeline -e’ with SQLs before ‘source’ statement
- Spark Authz plugin fixes privilege checks on source tables of the permanent view in Spark 3.1
- [KYUUBI #4652][FOLLOWUP] Fix JaasConfiguration ClassNotFoundException for Hadoop 3.3.4 and previous
- [KYUUBI #3887][FOLLOWUP] Fix kyuubiServerPrincipal logic in KyuubiCommands
- [KYUUBI #4763][DOCS] Fix the Kyuubi JDBC kerberos parameters
- [KYUUBI #4738]
AdminResource.getEngineSpace
should use primary group name on GROUP share level - [KYUUBI #4655][DOCS][FOLLOWUP] Recover the missing JDBC URL
- [KYUUBI #4753][KYUUBI 4752][Improvement] KyuubiConf.unset should not log deprecation warning
- [KYUUBI #4407][FOLLOWUP] Adapt SLF4J2
- [KYUUBI #4746] Do not recreate async request executor if has been shutdown
- [KYUUBI #4739] Add operation lock instead of locking state Enumeration
- [KYUUBI #4719] Support submission timeout for yarn application manager and get the applicationInfo in-memory
- [KYUUBI #4731] Support batch session conf advisor
- [KYUUBI #4734][Docs] Fix typo in docs of custom event handler
- [KYUUBI #4711] JDBC client should catch task failed exception instead of NPE in the incremental mode
- [KYUUBI #4688] Fix the failure to read the operation log after executing catalog and database operation
- [KYUUBI #4652] Upgrade Hadoop from 3.3.4 to 3.3.5
- [KYUUBI #4691][REST] Configure FAIL_ON_UNKNOWN_PROPERTIES to false for KyuubiScalaObjectMapper
- [KYUUBI #4658][FOLLOWUP] Improve unit tests
- [KYUUBI #4669] LDAP authentication allows auth user contains domain when bind.dn/pw enabled
- [KYUUBI #4653][KYUUBI 4650][Improvement] LogDivertAppender supports reading RollingFileAppender pattern
- [KYUUBI #4623][FOLLOWUP] Fix malformed docs
- [KYUUBI #4522]
use:catalog
should execute before thanuse:database
- [KYUUBI #4655][DOCS] Enrich docs for Kyuubi Hive JDBC driver
- [KYUUBI #4644] Manually terminate the Py4JServer during engine shutdown
- [KYUUBI #4623][FOLLOWUP] Remove unused import
- [KYUUBI #4623][K8S] KubernetesApplicationOperation uses Informer instead of list
- [KYUUBI #3872][BATCH] Skip setting proxy user when keytab is provided
- [KYUUBI #4639] Support to specify confOverlay when executing statement with RESTful API
- [KYUUBI #4627][Docs] Fix a typo in rest_api.md.
- [KYUUBI #4626][UI][CI] Fix Web UI Ci check fail with pnpm the lockfile needs updates
- [KYUUBI #4624][Docs] Fix table headers in kyuubi_vs_hive.md
- [KYUUBI #4620][KSHC] Cut off transitive dependencies
- [KYUUBI #4599][DOCS] Simplify project version evaluation
- [KYUUBI #4480] Engine alive probe should close thrift connection on engine lost
- [KYUUBI #4593][DOCS] Keep promote the release step update-to-date
- [KYUUBI #4579] Fix the unstable situation of test cases for Trino-Fronted-Service
- [KYUUBI #4584] Post the session exception to EventBus in time
- [KYUUBI #4563][Improvement] Format error log output in case of metrics json file not found
- [KYUUBI #3739][FOLLOWUP][REST] Remove unused parameters in SessionsResource rest docs (#4542)
- [KYUUBI #4581][TEST] Add KyuubiCommands parse python snippets unit test
- [KYUUBI #4575] Fix the empty last line may causes the session to exit directly
- [KYUUBI #4568] Make kyuubi-ctl doc enable variable ‘release’ automatic substitution
- [KYUUBI #4526][FOLLOWUP] Defer evaluation for GroupProvider
- [KYUUBI #4519] Update metadata when batch application state changed
- [KYUUBI #4467][K8S][BATCH] Tolerate Driver Pod ephemerally invisible after submitting
- [KYUUBI #4496] Call
super.clusterManager()
whenbatchConf
does not havespark.master
- [KYUUBI #4494]
bin/kyuubi
should useexec
to run Kyuubi server - [KYUUBI #4491] Fix Trino typo
- [KYUUBI #4489][REST] Fix missing
org.apache.commons.logging
by includingjcl-over-slf4j
dependency in REST client - [KYUUBI #4488][KSHC] Keep object original name defined in HiveBridgeHelper
- [KYUUBI #4471] Prompted command for docs generation should skip Java tests
- [KYUUBI #4482] Support to disable batch resource upload function
- [KYUUBI #4452] Strip the redundant leading and tailing slash of getZooKeeperNamespace.
- [KYUUBI #4451] Skip etcd test if no docker env
- [KYUUBI #4433] Fix the case with
create a view from view
- [KYUUBI #4464] Simplify and improve log for JDBCMetadataStore
- [KYUUBI #4462] Fix variable usage issue in
SessionManager#stop
- [KYUUBI #4443] Do not set engine session init sql for alive probe session
- [KYUUBI #4754][ARROW] Use
KyuubiArrowConveters#toBatchIterator
instead ofArrowConveters#toBatchIterator
- [KYUUBI #4749] Fix flaky test issues in SchedulerPoolSuite
- [KYUUBI #4722][DOCS] Kyuubi requires Spark 3.1 and above now
- [KYUUBI #4720][ARROW] Fix java.lang.NoSuchFieldError: IpcOption.DEFAULT for Spark-3.1/3.2
- [KYUUBI #4713][TEST] Fix false positive result in SchedulerPoolSuite (#4714)
- [KYUUBI #4713][TEST] Fix false positive result in SchedulerPoolSuite
- [KYUUBI #4710][ARROW] LocalTableScanExec should not trigger job
- [KYUUBI #4702][ARROW] CommandResultExec should not trigger job
- [KYUUBI #4662][ARROW] Arrow serialization should not introduce extra shuffle for outermost limit
- [KYUUBI #4587][SPARK][EXT] Take care of table cache query stage
- [KYUUBI #4585] Authz policy file generation
- [KYUUBI #4658][Authz][Bug] Fix InsertIntoHiveDirCommand classname so that we can extract the query in it when authorization.
- [KYUUBI #4617][AUTHZ] Collect results for filtered show objects ahead to prevent holding unserializable spark plan
- [KYUUBI #4504][Authz][Bug] Fix source table privilege requirement when querying permanent view in Spark 3.1 and below
- [KYUUBI #4532][AUTHZ] Displays the columns involved in extracting the aggregation operator
- [KYUUBI #4450] Ignore unknown fields
policyPriority
when reading policy json file - [KYUUBI #4727][DOC] kyuubi-spark-lineage has no transitive deps
- [KYUUBI #4586][TEST][1.7] Fix to clean up views in lineage unit tests
- [KYUUBI #4525][KSHC] Partitioning predicates should take effect to filter data
- [KYUUBI #4331][KYUUBI #4431] Lineage supports
Union
andMergeInto
- [KYUUBI #4666] Support Flink varbinary type in query operation
- [KYUUBI #4635] Support Flink time type in query operation
- [KYUUBI #4614][DOCS] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Flink SQL Query Engine
- [KYUUBI #4607][DOCS] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Hive SQL Query Engine
- [KYUUBI #4608][DOCS] Rename Flink Table Store to Apache Paimon (Incubating) in docs
Connectors for Trino SQL Query Engine
- [KYUUBI #4598] Bump Flink-1.15 from 1.15.3 to 1.15.4
- [KYUUBI #4583][DOCS] Rename Flink Table Store to Apache Paimon (Incubating)
- [KYUUBI #4517][FLINK] Fix multiple executions lead to abnormal results on Flink 1.1
- [KYUUBI #4446] Fix connections blocked by Flink insert statements
- [KYUUBI #4619] Fix beeline with -e When there are other SQL statements before the source statement, the source statement cannot be executed normally
- [KYUUBI #4508][BEELINE] Beeline should reset stdin after consuming init SQL file
- [KYUUBI #4492] Correct engine subdomain calculation in
kyuubi-ctl
- [KYUUBI #4479] Restore JDBC Kerberos authentication behavior for UGI.doAs
- [KYUUBI #4438] Render the batch info map in kyuubi-ctl
- [KYUUBI #4706][K8S][HELM] Use template comments for the chart license header
- [KYUUBI #4707][K8S][HELM] Fix README typo and minor reformatting
- [KYUUBI #4606][K8S][HELM] Add command and args configuration support
- [KYUUBI #4605][K8S][HELM] Set IfNotPresent pullPolicy by default
- [KYUUBI #4589][K8S][HELM] Update template on port-foward usage guide NOTES.txt
- [KYUUBI #4557][K8S][HELM] Kyuubi server should bind Pod IP by default
- [KYUUBI #4484][K8S][HELM] Set specific Kyuubi version to the chart appVersion
- Alex
- Bowen Liang
- Chao Chen
- Cheng Pan
- Derick Cypho
- Fei Wang
- Fu Chen
- Karsonnel
- Paul Lin
- Ruguo Yu
- Tianlin Liao
- Xieming LI
- Yikf
- dnskr
- fwang12
- guanhua.lgh
- huangzhir
- maruilei
- miaowang
- minyk
- odone
- pengqli
- phyyou
- remzi
- senmiaoliu
- sychen
- ulysses-you
- wangyepeng2
- xu.guo
- zwangsheng
- Fully tested w/ Spark 3.1, 3.2 and 3.3, initial support oncoming 3.4
- Fully tested w/ Flink 1.14, 1.15 and 1.16
- Introduce Trino frontend (experimental)
- Introduce Spark Lineage plugin (experimental)
- Introduce Spark Hive connector (experimental)
- Spark engine supports arrow-based results serialization
- Spark engine supports executing Python/PySpark scripts(experimental)
- Several Kyuubi Helm Chart & Spark on K8s improvements
- REST and Batch API functions, clients and stability enhancements
- Major Spark Authz plugin improvements of including Apache Iceberg support
- Security enhancements including LDAP authentication, internal encrypted transports, etc.
- [KYUUBI #2887] Add a POLLING balance policy for engine pool
- [KYUUBI #3089] Support custom event handler
- [KYUUBI #3449] Change default server info provider to ENGINE
- [KYUUBI #3545] [KYUUBI #3563] Support restrict spark configurations
- [KYUUBI #3554] REST API functions/command-line tool enhancements
- [KYUUBI #3577] Transfer connection url when opening session
- [KYUUBI #3615] Retry opening the engine when encountering a special error
- [KYUUBI #3658] Support SSL for Kyuubi thrift binary connection
- [KYUUBI #3659] Support alternative keys in ConfigBuilder
- [KYUUBI #3663] Support auto build Kubernetes client from env when Kyuubi running in Pod
- [KYUUBI #3742] Add FileSessionConfAdvisor to manage session level configuration
- [KYUUBI #3766] Support real user for KyuubiSession
- [KYUUBI #3835] Allow to use spark-internal as resource for batch job
- [KYUUBI #3839] Introduce signature mechanism to protect session variable on engine side
- [KYUUBI #3844] Forward the server ip in openSession
- [KYUUBI #3847] Add jdbc-shaded profile to support IDE debug
- [KYUUBI #3863] Arrow-based results serialization
- [KYUUBI #3867] Init SQL scripts should create table if table does not exist
- [KYUUBI #3887] Provide kyuubiServerPrincipal as alias for principal in JDBC parameter
- [KYUUBI #3897] Supplying pluggable GroupProvider
- [KYUUBI #3901] Introduce Trino frontend (experimental)
- [KYUUBI #3922] Only the ApplicationInfo with non-empty id is valid for BatchJobSubmission
- [KYUUBI #3923] Add dedicated batch session idle timeout
- [KYUUBI #3926] Introduce antlr4 to parse query statement
- [KYUUBI #3928] Application state mapping considers FinalApplicationStatus
- [KYUUBI #3950] Fix the batch metadata in-consistent issue on open batch session failure
- [KYUUBI #3975] Support to post batch session/operation event
- [KYUUBI #3983] [KYUUBI #3982] Introduce refreshing user defaults configs
- [KYUUBI #4021] Fix async start engine failure resulting in inaccurate operation metrics
- [KYUUBI #4106] Introduce resource file uploading in batch creation via REST API
- [KYUUBI #4119] Return app submission time for batch
- [KYUUBI #4145] Change lock and polling seq_num path on service discovery
- [KYUUBI #4151] Support to ignore subdomain when engine pool conditions are met
- [KYUUBI #4152] Enhance LDAP authentication
- [KYUUBI #4241] Only force close engine ref when open session failed
- [KYUUBI #4322] MySQL URL configuration joiner should use
&
instead of;
- [KYUUBI #4352] Support System.gc() with periodic GC interval
- [KYUUBI #4360] Support to refresh the unlimited users for session limiter
- [KYUUBI #4372] Support to return null value for OperationsResource RowSet
- [KYUUBI #4390] Allow user to provide batch id on submitting batch job
- [KYUUBI #4418] Allow disable async retry and fail fast on unrecoverable DB error
- [KYUUBI #4419] Implement simple EngineSecuritySecretProvider
- [KYUUBI #3973] Fix conf fallback for
kyuubi-ctl list server
- [KYUUBI #4006] Fix the incorrect execution of the source command
- [KYUUBI #4067] Reset the operation log before fetching new one
- [KYUUBI #4216] Support to transfer client version for JDBC and REST client
- [KYUUBI #4221] REST client for creating batch with uploading resource file
- [KYUUBI #4311] Fix the wrong parsing of JVM parameters in JDBC url
- [KYUUBI #4334] REST client should catch
NoHttpResponse Exception
and retry - [KYUUBI #3128] Support CostMode for PlanOnlyStatement
- [KYUUBI #3385] Set executor Pod name prefix if missing in Spark on K8s case
- [KYUUBI #3441] Change default Spark version to 3.3.1
- [KYUUBI #3487] Support Kyuubi as Spark dialect
- [KYUUBI #3776] Revise Kyuubi Spark engine shaded
- [KYUUBI #3780] Add support for executing Python/PySpark scripts (experimental)
- [KYUUBI #3792] Engine UI support grace stop
- [KYUUBI #3801] Correctly calculate active stages in SQLOperationListener
- [KYUUBI #3851] Support auto set up spark.master when Kyuubi running inside K8s Pod
- [KYUUBI #3885] Fix memory leak when using incremental collect mode
- [KYUUBI #3892] Remove sensitive kyuubi.engine.credentials from spark conf
- [KYUUBI #4035] Post SparkOperationEvent and show sessionId for statements
- [KYUUBI #4150] Support to execute Scala statement synchronized to prevent conflicts
- [KYUUBI #4316] Fix returned Timestamp values may lose precision
- [KYUUBI #4336] Avoid listing all schemas for Spark session catalog on schema pruning
- [KYUUBI #4407] Adapt SLF4J 2.x
- [KYUUBI #4412] Align the session handle between server and engine for Spark engine
- [KYUUBI #3424] Access privilege checks for namespaces and tables of DatasourceV2
- [KYUUBI #3515] Support Iceberg commands and skip apply row-filter to output tables
- [KYUUBI #3325] Privilege checks for permanent views and skipping shadowed tables
- [KYUUBI #3300] Support user group based policies from Ranger
- [KYUUBI #3371] Ranger client support sending requests in batch
- [KYUUBI #3581] Support row filter and data masking on permanent views
- [KYUUBI #3607] Support {OWNER} variable defined in Ranger policy
- [KYUUBI #3904] New Authz Plan Serde Layer
- [KYUUBI #4076] Modified query plan should correctly report stats
- [KYUUBI #4270] Register shutdown hook for plugin cleanup
- [KYUUBI #4255] Handle describe relation for V2 relations
- [KYUUBI #4262] Rename table requires ALTER privilege of the source table
- [KYUUBI #4202] Fix reference resolution when data masking enabled for V2 relations
- [KYUUBI #4437] Fix dependencies conflict by replacing
jersey-bundle
withjersey-client
- [KYUUBI #2282] Introduce Spark Lineage plugin (experimental)
- [KYUUBI #3365] Introduce Spark Hive connector (experimental)
- [KYUUBI #3601] Support infer columns for rebalance and sort
- [KYUUBI #3962] Add two conditions to decide if add shuffle before writing
- [KYUUBI #3988] Final stage config isolation support write only
- [KYUUBI #3893] UnspecifiedDistribution does not have default partitioning
- [KYUUBI #3514] Support Flink 1.16
- [KYUUBI #3604] Propagate HADOOP_PROXY_USER on launching Flink engine
- [KYUUBI #3717] Support Flink engine see primary keys
- [KYUUBI #3473] Add Docker Compose based Kyuubi playground
- [KYUUBI #3903] Support generate
kyuubi-version-info.properties
on Windows - [KYUUBI #4000] JDBC engine supports Apache Phoenix
- [KYUUBI #4288] Use eclipse-temurin:8-jdk-focal as default base image
- [KYUUBI #4320] Trino engine supports GetPrimaryKeys
- [KYUUBI #4473] Helm Chart improvements
- AdithyaPadmashali
- Alex Wiss-Wolferding
- Bowen Liang
- Brandon Grams
- CHzxp
- Cheng Pan
- Deng An
- Fei Wang
- Fu Chen
- HYBG
- Hanna Liashchuk
- Hyukjin Kwon
- John Caveman
- Kent Yao
- Luning Wang
- Min Zhao
- Nietzsche_w
- Paul Lin
- Ruguo Yu
- SteNicholas
- Thomas Prelle
- Tianlin Liao
- Wang Zhen
- William Tong
- Xuedong Luan
- Yikf
- asagjj
- df_liu
- dnskr
- dongkelun
- edddddy
- firefox
- hongdongdong
- hongkunyoo
- jiaoqingbo
- jinsilei
- lcy999
- liangbowen
- liaopeng
- maming
- odone
- pengqli
- runzhliu
- senmiaoliu
- sundy xiong
- sychen
- ulysses-you
- wForget
- wangjunbo
- waywtdcc
- wxmimperio
- xuefeimiaoao
- yehere
- yexianxun
- yikf
- yongqian
- yuqi
- zhouyifan279
- zwangsheng
- 毕博
- 佘志铭
- 胡安忠
- 袁福元
- Fast fail the kyuubi connection if the engine application has been terminated
- Transfer the TGetInfoReq to kyuubi engine side to check the connection valid and keep connection alive
- GetInfo supports returning server/engine information
- Handle zookeeper watch events
- Implement isSigned method of ResultSetMetaData for Kyuubi JDBC driver
- Extend JDBC URL to support catalog
- [SPARK] Fix scala NPE issue when adding non-local jar URI to class loader
- [SPARK] Kyuubi should set env SPARK_USER_NAME for K8s deployment
- [SPARK] Spark engines running on K8s give preference to IP for service discovery registration
- [FLINK] Flink SQL engine fixes GetTables operation
- [FLINK] Flink SQL engine supports getColumns operation
- [FLINK] Flink SQL engine supports run DDL across versions
- Bowen Liang
- Cheng Pan
- Fei Wang
- Fu Chen
- HYBG
- Hanna Liashchuk
- Luning Wang
- Min Zhao
- Paul Lin
- SteNicholas
- Wang Zhen
- Yikf
- asagjj
- df_liu
- hongdongdong
- jiaoqingbo
- senmiaoliu
- sychen
- ulysses-you
- zhouyifan279
- zwangsheng
- Support Batch(Jar) Job submission
- Support Thrift HTTP mode
- Support Etcd as Discovery Service
- Support JDBC authentication mechanism
- Unify the authentication for Thrift and REST v1 API
- Enhance metrics system
- Introduce REST client for Batch(Jar) Job submission
- Decouple Hive and Hadoop dependencies from builtin JDBC driver
- Builtin JDBC driver supports using keytab for Kerberos authentication
- Enhance beeline to support Spark console progress bar
- Fully verified with the latest Spark 3.1/3.2/3.3
- Fully verified with the latest Flink 1.14/1.15
- Fully verified with the Trino 363
- Support Hive backend Engine (Beta)
- Support JDBC backend Engine (Beta)
- Introduce Spark TPC-DS connector
- Introduce Spark TPC-H connector
- Introduce Spark Authz (Apache Ranger) plugin
- User Documentation refactoring and improvement
- [KYUUBI #3363] Kyuubi Beeline requires commons-lang
- [KYUUBI #3353] Backport HIVE-21899 - Utils.getCanonicalHostName() may return IP address depending on DNS infra
- [KYUUBI #3355] Backport HIVE-20583 - Use canonical hostname only for kerberos auth in HiveConnection
- [KYUUBI #3348] Specifies none checks to be performed on Javadoc comments for maven-javadoc-plugin
- [KYUUBI #3347] [MINOR] Relocation
AccessControlException
- [KYUUBI #3336] Use StageAttempt instead of StageId in SQLOperationListener
- [KYUUBI #3023] [FOLLOWUP] Kyuubi Hive JDBC: Replace UGI-based Kerberos authentication w/ JAAS
- [KYUUBI #3316] [BUILD] Enable spark-3.3 profile for license check on GA
- [KYUUBI #3254] Supplement the licenses of support etcd discovery
- [KYUUBI #3301] Construct lifetimeTerminatingChecker only when needed
- [KYUUBI #3272] Synchronize graceful shutdown with main stop sequence
- [KYUUBI #3281] [MINOR] Use AccessControlException instead of RuntimeException if check privilege failed
- [KYUUBI #3222] [FOLLOWUP] Fixing placeholder and config of user in JDBC Authentication Provider
- [KYUUBI #3275] [KYUUBI 3269] [DOCS] Doc for JDBC authentication provider
- [KYUUBI #3217] [DOCS] Doc for using Marcos in row-level filter in Authz
- [KYUUBI #3297] [MINOR] Null is replaced by KyuubiSQLException.featureNotSupported()
- [KYUUBI #3244] Bump Hudi 0.12.0
- [KYUUBI #3287] Exclude reload4j from hadoop-minikdc
- [KYUUBI #3222] [FOLLOWUP] Introdude JdbcUtils to simplify code
- [KYUUBI #3020] [FOLLOWUP] Refactor the code style
- [KYUUBI #3020] kyuubi ldap add new config property kyuubi.authentication.ldap.bindpw and kyuubi.authentication.ldap.attrs
- [KYUUBI #3226] Privileges should be checked only once in
RuleAuthorization
- [KYUUBI #3156] Expose REST frontend connection metrics
- [KYUUBI #3241] [DOCS] Update
Develop Tools / Building a Runnable Distribution
- [KYUUBI #3255] Add miss engine type config docs
- [KYUUBI #3214] Plan only mode should unset when mode value is incorrect
- [KYUUBI #3239] [SUBTASK] DorisSQLEngine - Add integration tests
- [KYUUBI #3252] Fix the problem that |release| in the document was not replaced correctly
- [KYUUBI #3247] Minor clean up Kyuubi JDBC code
- [KYUUBI #3222] JDBC Authentication Provider for server
- [KYUUBI #3243] Move trait Logging#initializeLogging to object Logging
- [KYUUBI #3245] Add spark-3.3 profile in building.md
- [KYUUBI #3184] OperationResource rowset api should have default values for maxrows and fetchorientation
- [KYUUBI #3220] Make kyuubi.engine.ui.stop.enabled false in HistoryServer
- [KYUUBI #1776] [FOLLOWUP] Fill empty td tag for
Failure Reason
column in EngineTable - [KYUUBI #3157] [DOC] Modify logging doc due to using log4j2 instead of log4j
- [KYUUBI #3228] [SUBTASK] Connectors for Spark SQL Query Engine -> TPC-DS
- [KYUUBI #3138] [SUBTASK] DorisSQLEngine - Add jdbc engine to dist
- [KYUUBI #3230] Flink SQL engine supports run across versions
- [KYUUBI #3072] [DOC] Add a doc of Flink Table Store for Flink SQL engine
- [KYUUBI #3170] Expose thrift binary connection metrics
- [KYUUBI #3227] SparkConfParser supports parse bytes and time
- [KYUUBI #3219] Error renew delegation tokens: Unknown version of delegation token 8
- [KYUUBI #3080] [DOC] Add a doc of the Flink Table Store for the Trino SQL Engine
- [KYUUBI #3107] [SUBTASK] DorisSQLEngine - Add process builder (#3123)
- [KYUUBI #3211] [SUBTASK] Connectors for Spark SQL Query Engine -> TPC-H
- [KYUUBI #3206] Change Flink default version to 1.15
- [KYUUBI #833] Check if
spark.kubernetes.executor.podNamePrefix
is invalid - [KYUUBI #3098] Unify the event log code path
- [KYUUBI #3210] [DOCS] Mention Kyuubi Spark SQL extension supports Spark 3.3
- [KYUUBI #3204] Fix duplicated ldapServer#close in LdapAuthenticationProviderImplSuite
- [KYUUBI #3209] Support configure TPC-H connector in runtime
- [KYUUBI #3200] Make KyuubiSessionEvent.sessionId clear
- [KYUUBI #3186] Support applying Row-level Filter and Data Masking policies for DatasourceV2 in Authz module
- [KYUUBI #2584] [INFRA] Migrate CI to Ubuntu 22.04
- [KYUUBI #3172] [FLINK] Fix failed test cases in Flink 1.15
- [KYUUBI #3203] [DOCS] Fix typo
- [KYUUBI #3192] Refactor TPCDSConf
- [KYUUBI #3180] Add component version util
- [KYUUBI #3162] Bump Hadoop 3.3.4
- [KYUUBI #3191] [DOCS] Add missing binary scala version in engine jar name
- [KYUUBI #3199] [BUILD] Fix travis JAVA_HOME
- [KYUUBI #3194] [Scala-2.13] Refine deprecated config
- [KYUUBI #3198] [DOCS] Fix index of Hudi Flink connector
- [KYUUBI #3189] [BUILD] Bump jetcd 0.7.3 and pin Netty dependencies
- [KYUUBI #3190] [BUILD] Use jdk_switcher to setup JAVA_HOME
- [KYUUBI #3145] Bump log4j from 2.17.2 to 2.18.0
- [KYUUBI #3135] Bump gRPC from 1.47.0 to 1.48.0
- [KYUUBI #3178] Add application operation docs
- [KYUUBI #3174] Update MATURITY for C30, RE50, CO20, CO40, CO50, CS10, IN10
- [KYUUBI #3175] Add session conf advisor docs
- [KYUUBI #3141] Trino engine etcd support
- [KYUUBI #3150] Expose metadata request metrics
- [KYUUBI #3070] [DOC] Add a doc of the Hudi connector for the Flink SQL Engine
- [KYUUBI #3104] Support SSL for Etcd
- [KYUUBI #3152] Introduce JDBC parameters to control connection timeout
- [KYUUBI #3136] Change Map to a case class ApplicationInfo as the application info holder
- [KYUUBI #2240] [SUBTASK] Skip add metadata manager if frontend does not support rest
- [KYUUBI #3131] Improve operation state change logging
- [KYUUBI #3158] Fix npe issue when formatting the kyuubi-ctl output
- [KYUUBI #3160] [DOCS]
Dependencies
links inConnectors for Spark SQL Query Engine
pages jump to wrong place #3160 (#3161) - [KYUUBI #3154] [SUBTASK] Connectors for Spark SQL Query Engine -> TiDB/TiKV
- [KYUUBI #3082] Add iceberg connector doc for Trino SQL Engine
- [KYUUBI #3071] [DOC] Add iceberg connector for Flink SQL Engine
- [KYUUBI #3153] Move batch util class to kyuubi rest sdk for programing friendly
- [KYUUBI #3067] [DOC] Add Flink Table Store connector doc for Spark SQL Engine
- [KYUUBI #3148] Change etcd docker image to recover arm64 CI
- [KYUUBI #3119] [TEST] Using more light-weight SparkPi for batch related tests
- [KYUUBI #3023] Kyuubi Hive JDBC: Replace UGI-based Kerberos authentication w/ JAAS
- [KYUUBI #3144] Remove deprecated KyuubiDriver in services manifest
- [KYUUBI #3143] Check class loadable before applying SLF4JBridgeHandler
- [KYUUBI #3139] Override toString method for rest dto classes
- [KYUUBI #3087] Convert the kyuubi batch conf with
spark.
prefix so that spark could identify - [KYUUBI #3133] Always run Flink statement in sync mode
- [KYUUBI #3121] [CI] Fix GA oom issue
- [KYUUBI #3126] Using markdown 3.3.7 for kyuubi document build
- [KYUUBI #3106] Correct
RelMetadataProvider
used in flink-sql-engine - [KYUUBI #3069] [DOC] Add Iceberg connector doc for Spark SQL Engine
- [KYUUBI #3068] [DOC] Add the Hudi connector doc for Spark SQL Query Engine
- [KYUUBI #3108] [DOC] Fix path errors in the build document
- [KYUUBI #3111] Replace HashMap with singletonMap
- [KYUUBI #3113] Bump up delta lake version from 2.0.0rc1 to 2.0.0
- [KYUUBI #3101] [SUBTASK][#3100] Build the content for extension points documentation
- [KYUUBI #3102] Fix multi endpoints for etcd
- [KYUUBI #3008] Bump prometheus from 0.14.1 to 0.16.0
- [KYUUBI #3095] Move TPC-DS/TPC-H queries to unique folder
- [KYUUBI #3094] Code refactor on Kyuubi Hive JDBC driver
- [KYUUBI #3092] Replace apache commons Base64 w/ JDK
- [KYUUBI #3093] Fix Kyuubi Hive JDBC driver SPNEGO header
- [KYUUBI #3050] Bump Apache Iceberg 0.14.0
- [KYUUBI #3044] Bump Spark 3.2.2
- [KYUUBI #3052] [FOLLOWUP] Do not use the ip in proxy http header for authentication to prevent CVE
- [KYUUBI #3051] Support to get the real client ip address for thrift connection when using VIP as kyuubi server load balancer
- [KYUUBI #3046] [Metrics] Add meter metrics for recording the rate of the operation state for each kyuubi operation
- [KYUUBI #3045] [FOLLOWUP] Correct the common options and add docs for kyuubi-admin command
- [KYUUBI #3076] [SUBTASK][#3039] Add the docs for rest api - Batch Resource
- [KYUUBI #3077] Remove meaningless statement override in LaunchEngine
- [KYUUBI #3018] [SUBTASK] DorisSQLEngine - GetColumns Operation
- [KYUUBI #3073] CredentialsManager should use appUser to renew credential
- [KYUUBI #3065] Support to retry the killApplicationByTag for JpsApplicationOperation
- [KYUUBI #3054] Add description of the discovery client in the conf doc
- [KYUUBI #3043] [FOLLOWUP] Restore accidentally removed public APIs of kyuubi-hive-jdbc module
- [KYUUBI #3060] [SUBTASK][#3059] Build content of the connector document section
- [KYUUBI #3045] Support to do admin rest request with kyuubi-adminctl
- [KYUUBI #3055] Expose client ip address into batch request conf
- [KYUUBI #3052] Support to get the real client ip address for http connection when using VIP as kyuubi server load balancer
- [KYUUBI #3043] Clean up Kyuubi Hive JDBC client
- [KYUUBI #3047] Fallback krb5 conf to OS if not configured
- [KYUUBI #2644] Add etcd discovery client for HA
- [KYUUBI #3040] [SUBTASK][#3039] Build the skeleton of client side documentation
- [KYUUBI #2974] [FEATURE] EOL Support for Spark 3.0
- [KYUUBI #3042] Kyuubi Hive JDBC should throw KyuubiSQLException
- [KYUUBI #3037] Handles configuring the JUL -> SLF4J bridge
- [KYUUBI #3033] [Bug] Kyuubi failed to start due to PID directory not exists
- [KYUUBI #3028] [FLINK] Bump Flink versions to 1.14.5 and 1.15.1
- [KYUUBI #2478] [FOLLOWUP] Fix bin/beeline without -u exits unexpectedly
- [KYUUBI #3025] Fix the kyuubi restful href link format issue
- [KYUUBI #3007] Bump scopt from 4.0.1 to 4.1.0
- [KYUUBI #3019] Backport HIVE-21538 - Beeline: password source though the console reader did not pass to connection param
- [KYUUBI #3017] kyuubi-ctl should print error message to right place
- [KYUUBI #3010] Bump Jetty from 9.4.41.v20210516 to 9.4.48.v20220622
- [KYUUBI #3011] Bump swagger from 2.1.11 to 2.2.1
- [KYUUBI #3009] Bump Jersey from 2.35 to 2.36
- [KYUUBI #2801] [FOLLOWUP] Also check whether the batch main resource path is in local dir allow list
- [KYUUBI #3012] Remove unused thrift request max attempts and related ut
- [KYUUBI #3005] [DOCS] Correct spelling errors and optimizations in ‘Building Kyuubi Documentation’ part
- [KYUUBI #3004] Clean up JDBC shaded client pom and license
- [KYUUBI #2895] Show final info in trino engine
- [KYUUBI #2984] Refactor TPCDS configurations using SparkConfParser
- [KYUUBI #2996] Remove Hive storage-api dependencies from Kyuubi Hive JDBC
- [KYUUBI #2997] Use spark shim set current namespace
- [KYUUBI #2994] Remove Hive common dependencies from Kyuubi Hive JDBC
- [KYUUBI #2850] [FOLLOWUP] Replace log4j2.properties by log4j2.xml
- [KYUUBI #2801] Add local dir allow list and check the application access path URI
- [KYUUBI #2953] Support to interrupt the thrift request if remote engine is broken
- [KYUUBI #2999] Fix Kyuubi Hive Beeline dependencies
- [KYUUBI #2850] [FOLLOWUP] Fix default log4j2 configuration
- [KYUUBI #2977] [BATCH] Using KyuubiApplicationManger#tagApplication help tag batch application
- [KYUUBI #2993] Fix typo in KyuubiConf and mark more config entries server only
- [KYUUBI #2987] Remove Hive shims-common and shims-0.23 dependencies from Kyuubi Hive JDBC
- [KYUUBI #2985] Prompt configuration when starting engine timeout
- [KYUUBI #2989] Remove HS2 active-passive support in Kyuubi Hive JDBC client
- [KYUUBI #2983] Remove Hive llap-client dependencies from Kyuubi Hive JDBC
- [KYUUBI #2981] Improve TPC-DS scan performance
- [KYUUBI #2917] Remove Hive service dependencies from Kyuubi Hive JDBC
- [KYUUBI #2868] [K8S] Add KubernetesApplicationOperation
- [KYUUBI #2979] Fix helm icon url
- [KYUUBI #2978] [SUBTASK][KPIP-4] If batch app status not found from cluster manager, fall back to metadata store
- [KYUUBI #2975] Code improvement in rest client
- [KYUUBI #2964] [SUBTASK][KPIP-4] Refine the batch response and render
- [KYUUBI #2976] Expose session name into kyuubi engine tab
- [KYUUBI #2850] [FOLLOWUP] Provide log4j2.xml.template in binary and use log4j2-defaults.xml
- [KYUUBI #2963] Bump Delta 2.0.0rc1
- [KYUUBI #2918] [Bug] Kyuubi integrated Ranger failed to query: table stats must be specified
- [KYUUBI #2966] Remove TProtocolVersion from SessionHandle/OperationHandle
- [KYUUBI #2972] Using stdout for the output of kyuubi-ctl
- [KYUUBI #2973] Decorate LOG in the RetryableRestClient with static final
- [KYUUBI #2962] [SUBTASK][KPIP-4] Throw exception if the metadata update count is zero
- [KYUUBI #2956] Support to config the connect/socket timeout of rest client for kyuubi-ctl
- [KYUUBI #2960] TFrontendService.SERVER_VERSION shall be HIVE_CLI_SERVICE_PROTOCOL_V11
- [KYUUBI #2957] [SUBTASK][KPIP-4] Use canonical host name for kyuubi instance
- [KYUUBI #2952] Remove OperationType from OperationHandle for simplification
- [KYUUBI #2955] BatchRequest args fix: need toString operation for different data types
- [KYUUBI #2949] Flaky test: execute statement - analysis exception
- [KYUUBI #2951] No need to extend CompositeService for MetadataManager
- [KYUUBI #2948] Remove thrift request timeout for KyuubiSyncThriftClient
- [KYUUBI #2943] [Bug][K8S] Remove Start Local Kyuubi Server For Kyuubi On K8S Test
- [KYUUBI #2924] Correct the frontend server start state
- [KYUUBI #886] [FOLLOWUP] Support to reload hadoop conf for KyuubiTHttpFrontendService
- [KYUUBI #2929] Kyuubi integrated Ranger does not support the CTAS syntax
- [KYUUBI #2935] Support spnego authentication for thrift http transport mode
- [KYUUBI #2894] Add synchronized for the ciphers of internal security accessor
- [KYUUBI #2927] Fix the thread in ScheduleThreadExecutorPool can’t be shutdown immediately
- [KYUUBI #2876] Bump Hudi 0.11.1
- [KYUUBI #2922] Clean up SparkConsoleProgressBar when SQL execution fails
- [KYUUBI #2919] Fix typo and wording for JDBCMetadataStoreConf
- [KYUUBI #2920] Fix typo for mysql metadata schema
- [KYUUBI #2890] Get the db From Sparksession When TableIdentifier’s Database Field Is Empty
- [KYUUBI #2911] [SUBTASK][KPIP-4] If the kyuubi instance unreachable, support to backfill state from resource manager and mark batch closed by remote kyuubi instance
- [KYUUBI #2912] [INFRA][DOCS] Improve release md
- [KYUUBI #2905] [DOCS] Update the number of new committers in MATURITY.md
- [KYUUBI #2745] [SUBTASK] DorisSQLEngine - GetTables Operation
- [KYUUBI #2628] [FOLLOWUP] Support waitCompletion for submit batch
- [KYUUBI #2827] [BUILD][TEST] Decouple integration tests from kyuubi-server
- [KYUUBI #2898] Bump maven-surefire-plugin 3.0.0-M7
- [KYUUBI #2628] [FOLLOWUP] Reuse the kyuubi-ctl batch commands for SubmitBatchCommand
- [KYUUBI #2897] Remove Hive metastore dependencies from Kyuubi Hive JDBC
- [KYUUBI #2782] Decouple Kyuubi Hive JDBC from Hive Serde
- [KYUUBI #2854] Add exception field in KyuubiSessionEvent
- [KYUUBI #2873] [INFRA][DOCS] Improve release template script
- [KYUUBI #2761] Flaky Test: engine.jdbc.doris.StatementSuite - test select
- [KYUUBI #2834] [SUBTASK][KPIP-4] Support to retry the metadata requests on transient issue and unblock main thread
- [KYUUBI #2543] Add
maxPartitionBytes
configuration for TPC-DS connecter - [KYUUBI #886] Add HTTP transport mode support to KYUUBI - no Kerberos support
- [KYUUBI #2628] [FOLLOWUP] Refine kyuubi-ctl batch commands
- [KYUUBI #2859] [SUBTASK][KPIP-4] Support
--conf
for kyuubi-ctl - [KYUUBI #2888] Bump Spark-3.3.0
- [KYUUBI #2708] Open engine session and renew engine credentials in the one
- [KYUUBI #2668] [FOLLOWUP] Add log4j for rest client test
- [KYUUBI #2881] [SUBTASK][KPIP-4] Rest client supports retry request if catch net exception
- [KYUUBI #2883] [Bug] java.lang.NoClassDefFoundError: org/apache/hadoop/hive/common/ValidWriteIdList in HiveDelegationTokenProvider#initialize
- [KYUUBI #2813] Bump Iceberg 0.13.2
- [KYUUBI #2628] [SUBTASK][KPIP-4] Implement kyuubi-ctl for batch job operation
- [KYUUBI #2872] Catch the exception for the iterator job when incremental collect is enabled
- [KYUUBI #2870] Fix sf0 query error in TPCH
- [KYUUBI #2861] [FOLLOWUP][GA] Daily publish snapshot with profile spark-3.3
- [KYUUBI #2865] Bump Spark 3.3.0-rc6
- [KYUUBI #2863] Unify the logic of tpch and tpcds to generate golden file
- [KYUUBI #2862] [BUILD] Release script supports Spark 3.3
- [KYUUBI #2861] [GA] Daily publish snapshot with profile spark-3.3
- [KYUUBI #2624] Support isExtended for FilteredShowTablesCommand in AuthZ module.
- [KYUUBI #2858] Support skipTests for kyuubi rest client module
- [KYUUBI #2848] Global temp view should only exist in session catalog
- [KYUUBI #2849] Close the engine alive pool gracefully
- [KYUUBI #2851] Log session name when opening/closing session
- [KYUUBI #2247] Change log4j2 properties to xml
- [KYUUBI #2704] verify TPC-DS query output
- [KYUUBI #2846] Add v1.5.2-incubating changelog
- [KYUUBI #2842] [TEST] Optimize the output of ExceptionThrowingDelegationTokenProvider in the Test
- [KYUUBI #2829] Make secret id static and remove thrift protocol from RPC handles
- [KYUUBI #2820] [FOLLOWUP] Fix duplicate SPNEGO typo
- [KYUUBI #2839] Refactor changelog
- [KYUUBI #2781] Fix KyuubiDataSource#getConnection to set user and password
- [KYUUBI #2845] [GA] Stop daily publish on branch-1.3
- [KYUUBI #2805] Add TPC-H queries verification
- [KYUUBI #2837] [BUILD] Support publish to private repo
- [KYUUBI #2820] [SUBTASK][KPIP-4] Support to redirect getLocalLog and closeBatchSession requests across kyuubi instances
- [KYUUBI #2830] Imporve Z-Order with Spark3.3
- [KYUUBI #2746] [INFRA] Improve NOTICE of binary release
- [KYUUBI #2825] [BUILD] Remove kyuubi-flink-sql-engine from kyuubi-server dependencies
- [KYUUBI #2211] [IMPROVEMENT] Add CHANGELOG.md to codebase for maintaining release notes
- [KYUUBI #2643] [FOLLOWUP] Using javax AuthenticationException instead of hadoop AuthenticationException
- [KYUUBI #2373] [SUBTASK][KPIP-4] Support to recovery batch session on Kyuubi instances restart
- [KYUUBI #2824] [TEST] Replace test tag ExtendedSQLTest by Slow
- [KYUUBI #2822] [GA] Set log level to info
- [KYUUBI #2817] Bump Spark 3.3.0-rc5
- [KYUUBI #2676] Flaky Test: SparkOperationProgressSuite: test operation progress
- [KYUUBI #2812] [SUBTASK][KPIP-4] Refine the batch info response
- [KYUUBI #2469] Support RangerDefaultAuditHandler for AuthZ module
- [KYUUBI #2807] Trino, Hive and JDBC Engine support session conf in newExecuteStatementOperation
- [KYUUBI #2814] Set JAVA_HOME in travis via javac
- [KYUUBI #2804] Add flaky test report template
- [KYUUBI #2800] [FOLLOWUP] Return CloseBatchReponse for kyuubi rest client deleteBatch
- [KYUUBI #2800] Refine batch mode code path
- [KYUUBI #2802] Retry opening the TSocket in KyuubiSyncThriftClient
- [KYUUBI #2742] Introduce admin resource for service admin - refresh frontend hadoop conf without restart
- [KYUUBI #2794] Change KyuubiRestException to extend RuntimeException
- [KYUUBI #2793] [DOCS] Add debugging engine
- [KYUUBI #2788] Add excludeDatabases for TPC-H catalogs
- [KYUUBI #2780] Refine stylecheck
- [KYUUBI #2789] Kyuubi Spark TPC-H Connector - Add tiny scale
- [KYUUBI #2765] [SUBTASK][KPIP-4] Refactor current kyuubi-ctl
- [KYUUBI #2717] [FOLLOWUP] Fix BatchRestApiSuite due to jdbc state store UPPER the batch type
- [KYUUBI #2717] [SUBTASK][KPIP-4] Introduce jdbc session state store for batch session multiple HA
- [KYUUBI #2643] [FOLLOWUP] Generate spnego auth token dynamically per request
- [KYUUBI #2741] Add kyuubi-spark-connector-common module
- [KYUUBI #2775] Add excludeDatabases for TPC-DS catalogs
- [KYUUBI #2643] [FOLLOWUP] Refine the rest sdk
- [KYUUBI #2553] Kyuubi Spark TPC-DS Connector - Add tiny scale
- [KYUUBI #2772] Kyuubi Spark TPC-H Connector - use log4j1
- [KYUUBI #2643] [SUBTASK][KPIP-4] Implement kyuubi rest sdk for batch job operation
- [KYUUBI #2763] Expected error code for invalid basic/spnego authentication should be SC_FORBIDDEN
- [KYUUBI #2768] Use the default DB passed in by session in Flink
- [KYUUBI #2764] [DOCS] Fix tables in docs being coverd by right toc sidebar
- [KYUUBI #2760] Add adapter layer in Kyuubi Hive JDBC module
- [KYUUBI #2751] [DOC] Replace sphinx_rtd_theme with sphinx_book_theme
- [KYUUBI #2754] [GA] Separate log archive name
- [KYUUBI #2755] [SUBTASK] DorisSQLEngine - add jdbc label
- [KYUUBI #2752] Kyuubi Spark TPC-DS Connector - configurable catalog’s name by initialize method
- [KYUUBI #2721] Implement dedicated set/get catalog/database operators
- [KYUUBI #2664] Kyuubi Spark TPC-H Connector - SupportsReportStatistics
- [KYUUBI #2471] [FOLLOWUP] Remove unexpected test-function.jar
- [KYUUBI #2743] colorfully kyuubi logo support
- [KYUUBI #2736] Upgrade Jackson 2.13.3
- [KYUUBI #2735] Test Spark 3.3.0-rc3
- [KYUUBI #2665] Kyuubi Spark TPC-H Connector - SupportsNamespaces
- [KYUUBI #2543] Add TPCDSTable generate benchmark
- [KYUUBI #2658] [SUBTASK] DorisSQLEngine with execute statement support
- [KYUUBI #2730] [WIP][KYUUBI #2238] Support Flink 1.15
- [KYUUBI #2663] Kyuubi Spark TPC-H Connector - Initial implementation
- [KYUUBI #2631] Rename high availability config key to support multi discovery client
- [KYUUBI #2733] [CI] Cross version verification for spark-3.3
- [KYUUBI #2285] trino’s result fetching method is changed to a streaming iterator mode to avoid hold data at server side
- [KYUUBI #2718] [KYUUBI#2405] Support Flink StringData Data Type
- [KYUUBI #2719] [SUBTASK][KPIP-4] Support internal rest request authentication to enable http request redirection across kyuubi instances
- [KYUUBI #2720] Fix KyuubiDatabaseMetaData#supportsCatalogs*
- [KYUUBI #2706] Spark extensions support Spark-3.3
- [KYUUBI #2714] Log4j2 layout pattern add date
- [KYUUBI #2686] [FOLLOWUP] Avoid potential flaky test
- [KYUUBI #2594] [FOLLOWUP] Fix flaky Test - support engine alive probe to fast fail on engine broken
- [KYUUBI #2701] Kyuubi Spark TPC-DS Connector - Rework SupportsReportStatistics and code refactor
- [KYUUBI #2712] Bump Spark master to 3.4.0-SNAPSHOT
- [KYUUBI #2541] Set nullable in table schema
- [KYUUBI #2709] Improve TPCDSTable display in Spark Web UI
- [KYUUBI #2619] Add profile spark-3.3
- [KYUUBI #2702] Fix TPC-DS columns name and add TPC-DS queries verification
- [KYUUBI #2348] Add it test for trino engine
- [KYUUBI #2686] Fix lock bug if engine initialization timeout
- [KYUUBI #2690] Make ProcessBuilder.commands immutable
- [KYUUBI #2696] [TEST] Stop NoopServer should not throw exception
- [KYUUBI #2700] Handle SPARK-37929 breaking change in TPCDSCatalog
- [KYUUBI #2683] Add INFO log in ServiceDiscovery.stopGracefully
- [KYUUBI #2694] EngineEvent.toString outputs application tags
- [KYUUBI #2594] Fix flaky Test - support engine alive probe to fast fail on engine broken
- [KYUUBI #2668] [FOLLOWUP] Remove unused Option because the collection is never null
- [KYUUBI #2675] Fix compatibility for spark authz with spark v3.3
- [KYUUBI #2680] Remove SwaggerScalaModelConverter after rest dto classes rewritten in Java
- [KYUUBI #2642] Fix flaky test - JpsApplicationOperation with spark local mode
- [KYUUBI #2668] [SUBTASK][KPIP-4] Rewrite the rest DTO classes in java
- [KYUUBI #2670] Delete the useless judgment in the extractURLComponents method of Utils.java
- [KYUUBI #2672] Check if the table exists
- [KYUUBI #2641] Client should not assume launch engine has completed on exception
- [KYUUBI #2666] Backport HIVE-24694 to Kyuubi Hive JDBC
- [KYUUBI #2661] [SUBTASK][KPIP-4] Rename GET /batches/$batchId/log to GET /batches/$batchId/localLog
- [KYUUBI #2650] Add FilteredShowColumnsCommand to AuthZ module
- [KYUUBI #2576] [FOLLOWUP] Bump Hudi 0.11.0
- [KYUUBI #2540] Kyuubi Spark TPC-DS Connector - SupportsNamespaces
- [KYUUBI #2655] Using the defined app keys for JpsApplicationOperation
- [KYUUBI #2640] Implement TGetInfoType CLI_ODBC_KEYWORDS
- [KYUUBI #2601] Add a config to support different service discovery client class implementation
- [KYUUBI #2471] Fix the bug of dynamically loading external packages
- [KYUUBI #2636] Refine BatchesResourceSuite
- [KYUUBI #2634] [SUBTASK][KPIP-4] Enhance the response error msg
- [KYUUBI #2616] Remove embedded mode support in Kyuubi Hive JDBC driver
- [KYUUBI #2605] Make SQLOperationListener configurable
- [KYUUBI #2474] [IMPROVEMENT] Add FilteredShowFunctionsCommand to Authz module
- [KYUUBI #2604] Hive Backend Engine - Multi tenancy support
- [KYUUBI #2576] Bump Hudi 0.11.0
- [KYUUBI #2473] [FOLLOWUP] Simplify FilteredShowNamespaceExec
- [KYUUBI #2621] Always use Hadoop shaded client
- [KYUUBI #2615] Add support HIVE_CLI_SERVICE_PROTOCOL_V11
- [KYUUBI #2473] [IMPROVEMENT] Add FilteredShowDatabasesCommand to AuthZ module
- [KYUUBI #2626] Replace literal by FetchType.LOG
- [KYUUBI #2614] Add commons-io to beeline module since jdbc upgraded to 3.1.3
- [KYUUBI #2539] [SUBTASK] Kyuubi Spark TPC-DS Connector - SupportsReportStatistics
- [KYUUBI #2591] Redact secret information from ProcBuilder log
- [KYUUBI #2542] [SUBTASK] Kyuubi Spark TPC-DS Connector - Make useAnsiStringType configurable
- [KYUUBI #2607] Introduce new module and setup testcontainers-based Kudu service for testing
- [KYUUBI #2333] Configuring Flink Engine heap memory and java opts
- [KYUUBI #2029] Hive Backend Engine - Operation Logs
- [KYUUBI #2609] Set Kyuubi server thrift client socket timeout to inf
- [KYUUBI #2560] Upgrade kyuubi-hive-jdbc hive version to 3.1.3
- [KYUUBI #2602] Bump testcontainers-scala 0.40.7
- [KYUUBI #2565] Variable substitution should work in plan only mode
- [KYUUBI #2493] [FOLLOWUP] Fix the exception that occurred when beeline rendered spark progress
- [KYUUBI #2378] Implement BatchesResource GET /batches/${batchId}/log
- [KYUUBI #2599] Bump scala-maven-plugin 4.6.1
- [KYUUBI #2493] Implement the progress of statement for spark sql engine
- [KYUUBI #2375] [FOLLOWUP] Implement BatchesResource GET /batches
- [KYUUBI #2588] Reformat kyuubi-hive-sql-engine/pom.xml
- [KYUUBI #2558] Fix warn message
- [KYUUBI #2427] [FOLLOWUP] Flaky test: deregister when meeting specified exception
- [KYUUBI #2582] Minimize Travis build and test
- [KYUUBI #2500] [FOLLOWUP] Resolve flink conf at engine side
- [KYUUBI #2571] Minimize YARN tests overhead
- [KYUUBI #2573] [KPIP-4][SUBTASK] Add a seekable buffered reader for random access operation log
- [KYUUBI #2375] [SUBTASK][KPIP-4] Implement BatchesResource GET /batches
- [KYUUBI #2571] Release connection to prevent the engine leak
- [KYUUBI #2522] Even the process exit code is zero, also check the application state from resource manager
- [KYUUBI #2569] Change the acquisition method of flinkHome to keep it consistent with other engines
- [KYUUBI #2550] Fix swagger does not show the request/response schema issue
- [KYUUBI #2500] Command OptionParser for launching Flink Backend Engine
- [KYUUBI #2379] [SUBTASK][KPIP-4] Implement BatchesResource DELETE /batches/${batchId}
- [KYUUBI #2513] Support NULL type in trino engine and add QueryTests
- [KYUUBI #2403] [IMPROVEMENT] move addTimeoutMonitor to AbstractOperation because it was used in multiple engines
- [KYUUBI #2531] [SUBTASK] Kyuubi Spark TPC-DS Connector - Initial implementation
- [KYUUBI #2523] Flaky Test: KyuubiBatchYarnClusterSuite - open batch session
- [KYUUBI #2376] [SUBTASK][KPIP-4] Implement BatchesResource GET /batches/${batchId}
- [KYUUBI #2547] Support jdbc url prefix jdbc:kyuubi://
- [KYUUBI #2549] Do not auth the request to load OpenApiConf
- [KYUUBI #2548] Prevent dead loop if the batch job submission process it not alive
- [KYUUBI #2533] Make Utils.parseURL public to remove unnecessary reflection
- [KYUUBI #2524] [DOCS] Update metrics.md
- [KYUUBI #2532] Avoid NPE in KyuubiHiveDriver.acceptsURL
- [KYUUBI #2478] [FOLLOWUP] Invoke getOpts method instead of Reflection
- [KYUUBI #2490] [FOLLOWUP] Fix and move set command test case
- [KYUUBI #2517] Rename ZorderSqlAstBuilder to KyuubiSparkSQLAstBuilder
- [KYUUBI #2025] [HIVE] Add a Hive on Yarn doc
- [KYUUBI #2032] [SUBTASK] Hive Backend Engine - new APIs with hive-service-rpc 3.1.2 - SetClientInfo
- [KYUUBI #2490] Fix NPE in getOperationStatus
- [KYUUBI #2516] [DOCS] Add Contributor over time in README.md
- [KYUUBI #2346] [IMPROVEMENT] Simplify FlinkProcessBuilder with java executable
- [KYUUBI #2472] Support FilteredShowTablesCommand for AuthZ module
- [KYUUBI #2309] [SUBTASK][KPIP-4] Implement BatchesResource POST /batches
- [KYUUBI #2028] [FOLLOWUP] add engine stop event and fix the partition of initialized event
- [KYUUBI #2512] Fix broken link of IntelliJ IDEA Setup Guide
- [KYUUBI #2450] [FOLLOWUP] Remove opHandle from opHandleSet when exception occurs
- [KYUUBI #2510] Fix NPE when invoking YarnApplicationOperation::getApplicationInfoByTag
- [KYUUBI #2496] Prevent empty auth user when anonymous is allowed
- [KYUUBI #2498] Upgrade Delta version to 1.2.1
- [KYUUBI #2419] Release engine during closing kyuubi server session if share level is connection
- [KYUUBI #2487] Fix test command to make it runnable
- [KYUUBI #2457] Fix flaky test: engine log truncation
- [KYUUBI #2478] Backport HIVE-19018 to Kyuubi Beeline
- [KYUUBI #2020] [SUBTASK] Hive Backend Engine - new APIs with hive-service-rpc 3.1.2 - TGetQueryId
- [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
- [KYUUBI #2433] HiveSQLEngine load required jars from HIVE_HADOOP_CLASSPATH
- [KYUUBI #2477] Change state early on stopping
- [KYUUBI #2451] Support isWrapperFor and unwrap
- [KYUUBI #2453] [IMPROVEMENT] checkValue of TypedConfigBuilder shall also print the config name
- [KYUUBI #2427] Flaky test: deregister when meeting specified exception
- [KYUUBI #2456] Supports managing engines of different share level in kyuubi-ctl
- [KYUUBI #1987] Support preserve user context in group/server share level
- [KYUUBI #2467] Remove close launchEngineOp
- [KYUUBI #2440] [IMPROVEMENT] spark engine event add endTime when it is stopped
- [KYUUBI #2461] Use the original host argument
- [KYUUBI #2463] Redact
kyuubi.ha.zookeeper.auth.digest
in Spark engine - [KYUUBI #2445] Implement ApplicationManager and Yarn/ JPS-local Application Operation
- [KYUUBI #1936] [FOLLOWUP] Stop updating credentials when credentials are expired
- [KYUUBI #2450] Update lastAccessTime in getStatus and add opHandle to opHandleSet before run
- [KYUUBI #2439] Using Pure Java TPC-DS generator
- [KYUUBI #2448] Log the engine id when opening Kyuubi connection
- [KYUUBI #2436] Add AlterTableRecoverPartitionsCommand for Spark Sql Authz PrivilegesBuilder
- [KYUUBI #2432] [DOCS] button “Download” is invalid
- [[KYUUBI #2429] KYUUBI #2416] Increase Test Coverage For Privileges Builder](https://github.com/apache/incubator-kyuubi/commit/14f675d2)
- [KYUUBI #2344] [IMPROVEMENT] Add Kyuubi Server on Kubernetes with Spark Cluster mode integration test
- [KYUUBI #2201] Show ExecutionId when running status on query engine page
- [KYUUBI #2424] [IMPROVEMENT] add Flink compile version and Trino client compile version to KyuubiServer Log
- [KYUUBI #2253] [IMPROVEMENT] Trino Engine - Events support
- [KYUUBI #2426] Return complete error stack trace information
- [KYUUBI #2410] [IMPROVEMENT] Fix docker-image-tool.sh example version to 1.4.0
- [KYUUBI #2351] Fix Hive Engine terminating blocked by non-daemon threads
- [KYUUBI #2422] Wrap close session with try-finally
- [KYUUBI #2420] Fix outdate .gitignore for dependency-reduced-pom.xml
- [KYUUBI #2368] [IMPROVEMENT] Command OptionParser for launching Trino Backend Engine
- [KYUUBI #2021] [FOLLOWUP] Move derby workaround to test code
- [KYUUBI #2301] Limit the maximum number of concurrent connections per user and ipaddress
- [KYUUBI #2323] Separate events to a submodule - kyuubi-event
- [KYUUBI #2289] Use unique tag to kill applications
- [KYUUBI #2021] Command OptionParser for launching Hive Backend Engine
- [KYUUBI #2414] [KYUUBI apache#2413 ] Fix InsertIntoHiveTableCommand case in PrivilegesBuilder#buildCommand()
- [KYUUBI #2406] Add Flink environments to template
- [KYUUBI #2355] Bump Delta Lake 1.2.0
- [KYUUBI #2349] [DOCS] Usage docs for kyuubi-ctl
- [KYUUBI #2381] [Test] Add Kyuubi on k8s With Spark on k8s client deploy-mode unit test
- [KYUUBI #2390] RuleEliminateMarker stays in analyze phase for data masking
- [KYUUBI #2395] [DOC] Add Documentation for Spark AuthZ Extension
- [KYUUBI #2397] Supports managing engines of different versions in kyuubi-ctl
- [KYUUBI #2402] [IMPROVEMENT] addTimeoutMonitor for trino engine when it run query async
- [KYUUBI #2360] [SUBTASK] Configuring Hive engine heap memory and java opts
- [KYUUBI #2399] Fix PrivilegesBuilder Build Wrong PrivilegeObjets When Query Without Project But With OrderBy/PartitionBy
- [KYUUBI #2308] [SUBTASK][KPIP-4] Batch job configuration ignore list and pre-defined configuration in server-side
- [KYUUBI #2369] [IMPROVEMENT] update developer.md to describe what append descriptions of new configurations to settings.md
- [KYUUBI #2391] Fix privileges builder return wrong result when there is no project but has filter/join
- [KYUUBI #2361] [IMPROVEMENT] Configuring Trino Engine heap memory and java opts
- [KYUUBI #2338] [DOCS] Upgrade sphinx dependencies for documentation build
- [KYUUBI #2385] Export JAVA_HOME. It seems TravisCI stopped doing it recently
- [KYUUBI #2353] [SUBTASK][KPIP-4] Implement BatchJobSubmission operation and basic KyuubiBatchSessionImpl
- [KYUUBI #2359] [Test] Build WithKyuubiServerOnKuberntes
- [KYUUBI #2330] [SUBTASK] Hive Backend Engine - GetTypeInfo Operation
- [KYUUBI #2257] Replace replace conf vars with strings in HiveSQLEngine
- [KYUUBI #2248] [DOCS] Add a flink on yarn kerberos doc
- [KYUUBI #1451] Support Data Column Masking
- [KYUUBI #2357] [IMPROVEMENT] Add warn log and check in class of HiveProcessBuilder
- [KYUUBI #2328] Support getting mainResource in the module target directory of KYUUBI_HOME
- [KYUUBI #2354] Fix NPE in process builder log capture thread
- [KYUUBI #2337] [DOCS] Access Kyuubi with Kyuubi JDBC Driver
- [KYUUBI #2331] Add createSession method to further abstract openSession
- [KYUUBI #2347] Output trino query id within query execute
- [KYUUBI #2345] [DOC] Hot Upgrade Kyuubi Server
- [KYUUBI #2336] Simplify TrinoProcessBuilder with java executable
- [KYUUBI #2343] [KYUUBI#2297] HiveEngineEvent toString is not pretty
- [KYUUBI #1989] Decouple curator from other modules
- [KYUUBI #2324] [SUBTASK][KPIP-4] Implement SparkBatchProcessBuilder to submit spark batch job
- [KYUUBI #2329] [KYUUBI#2214][FOLLOWUP] Cleanup kubernetes-deployment-it
- [KYUUBI #2216] [Test] [K8s] Add Spark Cluster mode on Kubernetes integration test
- [KYUUBI #2024] [FOLLOWUP] Hive Backend Engine - ProcBuilder for HiveEngine
- [KYUUBI #2281] The RenewDelegationToken method of TFrontendService should return SUCCESS_STATUS by default
- [KYUUBI #2300] Add http UGIAssuming handler wrapper for kerberos enabled restful frontend service
- [KYUUBI #2292] [FOLLOWUP] Unify kyuubi server plugin location
- [KYUUBI #2320] Make the CodeSource location correctly obtained on Windows
- [KYUUBI #2316] [DOCS] Fix typo in GitHub issue template
- [KYUUBI #2317] [BUILD] Bump hive-service-rpc 3.1.3 version
- [KYUUBI #2311] Refine github label
- [KYUUBI #2310] Make ranger extension work with mac m1
- [KYUUBI #2312] Spark data type TimestampNTZ supported version changes as 3.4.0
- [KYUUBI #2296] Fix operation log file handler leak
- [KYUUBI #2250] Support to limit the spark engine max running time
- [KYUUBI #2299] Fix flaky test: support engine alive probe to fast fail on engine broken
- [KYUUBI #2292] Unify kyuubi server plugin location
- [KYUUBI #2292] Unify spark extension location
- [KYUUBI #2084] [FOLLOWUP] Support arbitrary parameters for KyuubiConf
- [KYUUBI #2287] Revamp Flink IT by random port and merge tests
- [KYUUBI #1451] Add Row-level filtering support
- [KYUUBI #2280] [INFRA] Replace BSD 3-clause with ASF License v2 for scala binaries
- [KYUUBI #2277] Inline kyuubi prefix in KyuubiConf
- [KYUUBI #2266] The default value of frontend.connection.url.use.hostname should be set to true to be consistent with previous versions
- [KYUUBI #2260] The running query will not update the duration of the page
- [KYUUBI #2272] Fix incorrect doc link
- [KYUUBI #2268] Flaky test: submit spark app timeout with last log output
- [KYUUBI #2275] [DOCS] Fix missing prefix in trino engine quick start
- [KYUUBI #2243] [DOCS] Add quick start for trino engine
- [KYUUBI #2255] The engine state of Spark’s EngineEvent is hardcoded with 0
- [KYUUBI #2263] [KYUUBI #2262]Kyuubi Spark Nightly failed - select timestamp_ntz *** FAILED ***
- [KYUUBI #2209] Add detail usage documents of Flink engine
- [KYUUBI #2246] [BUILD] Pick Commons-Logging dependence out of Hudi-Common
- [KYUUBI #2028] Hive Backend Engine - Events support
- [KYUUBI #2207] Fix DayTimeIntervalType/YearMonthIntervalType Column Size
- [KYUUBI #1451] Introduce Kyuubi Spark AuthZ Module with column-level fine-grained authorization
- [KYUUBI #1798] Add EventBus module to unify the distribution and subscription of Kyuubi’s events
- [KYUUBI #2207] Support newly added spark data types: TimestampNTZType
- [KYUUBI #1021] Expire CredentialsRef in a proper time to reduce memor…
- [KYUUBI #2241] Remove unused deployment documents of Spark
- [KYUUBI #2244] load-kyuubi-env.sh should print SPARK_ENGINE_HOME for consistent
- [KYUUBI #2023] Hive Backend Engine - Shade HiveSQLEngine runtime
- [KYUUBI #1498] Support operation log for ExecuteScala
- [KYUUBI #2087] Add issue template for documentation improvement
- [KYUUBI #1962] Add timeout check for createSpark
- [KYUUBI #2218] Fix maven options about hive-provided
- [KYUUBI #2225] Support to set result max rows for spark engine
- [KYUUBI #2008] [FOLLOWUP] Support engine type and subdomain in kyuubi-ctl
- [KYUUBI #2231] Close action and default sparksession before
createSpark
. - [KYUUBI #2222] Refactor the log when failing to get hadoop fs delegation token
- [KYUUBI #2227] Fix operation log dir not deleted issue
- [KYUUBI #2223] Return the last rows of log for prompts even exception detected
- [KYUUBI #2221] Shade hive-service-rpc and thrift in Spark engine
- [KYUUBI #2102] Support to retry the internal thrift request call and add engine liveness probe to enable fast fail before retry
- [KYUUBI #2207] Support newly added spark data types: DayTimeIntervalType/YearMonthIntervalType
- [KYUUBI #2085] [FOLLOWUP] Fix kyuubi-common wrong label
- [KYUUBI #2208] Fixed session close operator log session dir not deleted
- [KYUUBI #2214] Add Spark Engine on Kubernetes integration test
- [KYUUBI #2203] [FLINK] Support flink conf set by kyuubi conf file
- [KYUUBI #2197] [BUILD] Bump jersey 2.35 version
- [KYUUBI #2204] Make comments consistent with code in EngineRef
- [KYUUBI #2186] Manage test failures with kyuubi spark nightly build - execute statement - select interval
- [KYUUBI #2195] Using while-loop or for-loop instead of map/range to improve performance in RowSet
- [KYUUBI #2033] Hive Backend Engine - GetCrossReference
- [KYUUBI #2135] Build and test all modules on Linux ARM64
- [KYUUBI #2035] Hive Backend Engine -
build/dist
support - [KYUUBI #2189] Manage test failures with kyuubi spark nightly build - deregister when meeting specified exception *** FAILED ***
- [KYUUBI #2119] [FOLLOWUP] Support output progress bar in Spark engine
- [KYUUBI #2187] Manage test failures with kyuubi spark nightly build - execute simple scala code *** FAILED ***
- [KYUUBI #2184] Manage test failures with kyuubi spark nightly build - operation listener *** FAILED ***
- [KYUUBI #2034] Hive Backend Engine - GetPrimaryKeys
- [KYUUBI #2163] [K8s] copy beeline-jars into docker image
- [KYUUBI #2175] Improve CI with cancel & concurrency & paths filter
- [KYUUBI #2172] [BUILD] Bump Flink 1.14.4 version
- [KYUUBI #2159] [FLINK] Prevent loss of exception message line separator
- [KYUUBI #2024] Hive Backend Engine - ProcBuilder for HiveEngine
- [KYUUBI #2017] Hive Backend Engine - GetColumns Operation
- [KYUUBI #2019] Hive Backend Engine - GetTableTypes Operation
- [KYUUBI #2018] Hive Backend Engine - GetFunctions Operation
- [KYUUBI #2156] [FOLLOWUP] Fix configuration format in document
- [KYUUBI #2119] Support output progress bar in Spark engine
- [KYUUBI #2016] Hive Backend Engine - GetTables Operation
- [KYUUBI #2156] Change log to reflect exactly why getting token failed
- [KYUUBI #2148] [DOCS] Add dev/reformat usage
- [KYUUBI #2150] [DOCS] Fix Getting Started With Kyuubi on Kubernetes
- [KYUUBI #2143] [KYUBBI #2142][DOCS] Add IDEA setup guide
- [KYUUBI #2015] Hive Backend Engine - GetSchemas Operation
- [KYUUBI #1936] [FOLLOWUP] Send credentials when opening session and wait for completion
- [KYUUBI #2085] [FOLLOWUP] Fix the wrong path of
module:hive
- [KYUUBI #2134] Respect Spark bundled log4j in extension modules
- [KYUUBI #2014] Hive Backend Engine - GetCatalog Operation
- [KYUUBI #2129] FlinkEngine throws UnsupportedOperationException in GetColumns
- [KYUUBI #2022] Hive Backend Engine - maven-google-downloader plugin support for hive distribution
- [KYUUBI #2084] Support arbitrary parameters for KyuubiConf
- [KYUUBI #2112] Improve the compatibility of queryTimeout in more version clients
- [KYUUBI #2115] Update license and enhance collect_licenses script
- [KYUUBI #2085] Add a labeler github action to triage PRs
- [KYUUBI #1866] [FOLLOWUP] Add Deploy Kyuubi Flink engine on Yarn
- [KYUUBI #2120] Optimize RenewDelegationToken logs in Spark engine
- [KYUUBI #2104] Kill yarn job using yarn client API when kyuubi engine …
- [KYUUBI #2118] [SUBTASK][KPIP-2] Support session jars management
- [KYUUBI #2127] avoid to set HA_ZK_NAMESPACE and HA_ZK_ENGINE_REF_ID repetitively when create flink sql engine
- [KYUUBI #2123] Output engine information after openEngineSession call fails
- [KYUUBI #2125] closeSession should avoid sending RPC after openSession fails
- [KYUUBI #2116] move toString() to ProcBuilder trait from its implements
- [KYUUBI #1866] [FOLLOWUP] Add logging of Flink SQL Engine
- [KYUUBI #1866] [DOCS] Add flink sql engine quick start
- [KYUUBI #2108] Add description about trino in the config of engine.type
- [KYUUBI #2071] Using while-loop instead of map/range to improve performance in RowSet
- [KYUUBI #2070] [FLINK] Support Flink job submission on yarn-session mode
- [KYUUBI #2089] Add debugging instructions for Flink engine
- [KYUUBI #2097] [CI] Upload Test Log for CI failure shall contain trino engine log #2094
- [KYUUBI #2095] Remove useless logic about add conf when create a new engine
- [KYUUBI #1948] [FOLLOWUP] Relocate fb303 classes
- [KYUUBI #1978] Support NEGOTIATE/BASIC authorization for restful frontend service
- [KYUUBI #2081] YARN_CONF_DIR shall be added to kyuubi server classpath as HADOOP_CONF_DIR
- [KYUUBI #2078]
logCaptureThread
does not catch sparksubmit exception - [KYUUBI #1936] Send credentials when opening session and wait for completion
- [KYUUBI #2079] Update kyuubi layer source file to add flink and trino…
- [KYUUBI #1563] Fix broken link and add new link in
CONTRIBUTION.md
- [KYUUBI #2072] Improve rest server behavior
- [KYUUBI #2075] Using thread-safe FastDateFormat instead of SimpleDateFormat
- [KYUUBI #2066] Fix spelling mistake and appropriate naming
- [KYUUBI #2063] Fix engine idle timeout lose efficacy for Flink Engine
- [KYUUBI #2061] Implementation of the very basic UI on current Jetty server
- [KYUUBI #2011] Introduce to very basic hive engine
- [KYUUBI #1215] [DOC] Document incremental collection
- [KYUUBI #2043] Upgrade log4j/2.x/ to 2.17.2
- [KYUUBI #2060] Clear job group for init SQL
- [KYUUBI #2044] Remove authentication thread local objects to prevent memory leak
- [KYUUBI #1955] Add CI for branch-1.5 & 1.4 SNAPSHOTS
- [KYUUBI #2054] [KYUUBI-1819] Support closing Flink SQL engine process
- [KYUUBI #2055] correct the log service name
- [KYUUBI #2047] Support more MySQL JDBC driver versions
- Ada Wong
- BoYang
- Brandon Grams
- Cheng Pan
- Deng An
- Fei Wang
- Fu Chen
- Kent Yao
- Luning Wang
- Mahmoud Bahaa
- Martin Tzvetanov Grigorov
- Min Zhao
- Nick Song
- ParisaTork
- Paul Lin
- Shengkai Fang
- Nicholas Jiang
- Tianlin Liao
- Wang Zhen
- Xiao Zhao
- benjobs
- gabrywu
- guanhua.lgh
- hongdongdong
- jason
- jiadongdong
- jiaoqingbo
- liangbowen
- liaoyt
- liuzy04
- lxorc
- maming
- odone
- packyan
- pengqli
- senmiaoliu
- sychen
- taokelu
- Xiduo You
- wangjunbo
- Zhen Wang
- winfys
- xsun4
- Vino Yang
- yangrong688
- yikf
- zhangliang
- zhangrenhua
- zhanqian
- zhouyifan279
- zwangsheng
- [KYUUBI #2841] [1.5] Revert “[KYUUBI #2211] [Improvement] Add CHANGELOG.md to codebase for maintaining release notes”
- [KYUUBI #2746][INFRA][1.5] Improve NOTICE of binary release
- [KYUUBI-2422] Wrap close session with try-finally (#2836)
- [KYUUBI #2227] Fix operation log dir not deleted issue
- [KYUUBI #2208] Fixed session close operator log session dir not deleted
- [KYUUBI #2211] [Improvement] Add CHANGELOG.md to codebase for maintaining release notes
- [KYUUBI #2736] Upgrade Jackson 2.13.3
- [KYUUBI #2720] Fix KyuubiDatabaseMetaData#supportsCatalogs*
- [KYUUBI #2686][1.5] Fix lock bug if engine initialization timeout
- [KYUUBI #2640] Implement TGetInfoType CLI_ODBC_KEYWORDS
- [KYUUBI #2450][FOLLOWUP] Remove opHandle from opHandleSet when exception occurs
- [KYUUBI #2478] Backport HIVE-19018 to Kyuubi Beeline
- [KYUUBI #2484] Add conf to SessionEvent and display it in EngineSessionPage
- [KYUUBI #2450] Update lastAccessTime in getStatus and add opHandle to opHandleSet before run
- Cheng Pan
- Fei Wang
- jiaoqingbo
- winfys
- Xiduo You
- yangrong688
- Zhen Wang
Release Notes - 1.9.0
2024 Mar 19
Highlight
Server
Spark Engine
Spark AuthZ Plugin
Other Spark Plugins
Flink Engine
Hive Engine
Trino Engine
JDBC Engine
Other Engine
Web UI
Documentation
Building, Infrastructure and Dependency
Credits
Last but not least, this release would not have been possible without the following contributors:
Release Notes - 1.8.1
2024 Feb 21
Server
Spark Engine, Connectors and Plugins
Other Engines
Client
Build, Test and Misc
Credits
Last but not least, this release would not have been possible without the following contributors:
Release Notes - 1.8.0
2023 Nov 7
Highlights
Spark Engine, Connectors and Plugins
Flink Engine
Other Engines
Server
Client
Web UI
Documentation
Building, Infrastructure and Dependency
Credits
Last but not least, this release would not have been possible without the following contributors:
Release Notes - 1.7.3
2023 Sep 25
Highlight
Server
Spark Authz Plugin
Credits
Last but not least, this release would not have been possible without the following contributors:
Release Notes - 1.7.2
2023 Sep 18
Highlight
Server
Spark Engine
Spark Extensions
Client
Common
Deployment
Documentation
Credits
Last but not least, this release would not have been possible without the following contributors:
Release Notes - 1.7.1
2023 May 5
Highlight
Server
Spark Engine
Spark Authz Plugin
Other Spark Extensions
Flink Engine
Client
Deployment
Credits
Last but not least, this release would not have been possible without the following contributors:
Release Notes - 1.7.0
2023 Mar 8
Highlight
Server
Client
Spark Engine
Spark Authz Plugin
Other Spark Extensions
Flink Engine
Other Notable Changes
Credits
Last but not least, this release would not have been possible without the following contributors:
Release Notes - 1.6.1-incubating
2022 Nov 16
Highlight
Server
Client
Engine
ChangeList
[KYUUBI #3744][Bug] InvalidACL appears in engine after upgrade to Kyuubi 1.6.0
[KYUUBI #3774] Output OpenSessionCount when Engine is waiting for exit because it reaches the Max lifetime
[KYUUBI #3746] Support to specify multiple config items with --conf
in kyuubi command line
[KYUUBI #3748] Adding metrics kyuubi.metadata.request.opened and fix flaky test - MetadataManagerSuite
[KYUUBI #3745] Closing existing seekable reader when adding extra log
Revert “[KYUUBI #3385] Set spark.kubernetes.executor.podNamePrefix if missing”
[KYUUBI #3730] Change the parameter annotation of the runRemoveJarOperation method to Remove-jar operation instead of Add-jar operation
[KYUUBI #3726] [DOCS] Place comments above line to avoid mislead users
[KYUUBI #3720][FOLLOWUP] Using correct config version for CTL_BATCH_LOG_ON_FAILURE_TIMEOUT
[KYUUBI #3720] Support to show remaining logs with timeout if the batch failed
[KYUUBI #3728] [CI] Store kyuubi-spark-batch-submit.log for IT spark on kubernetes batch test
[KYUUBI #3714] Use ServicesResourceTransformer to concatenating service entries
[KYUUBI #3711] PlanOnly should output engine logs to log files
[KYUUBI #3710] Delete the swagger UI related resource files from the source folder
[KYUUBI #3703][DOCS] Fix typo in localLog REST API
[KYUUBI #3715] Remove unexpected mysql connector jar
[KYUUBI #3701] Allow JDBC authencation to connect auth db with no user name
[KYUUBI #3705] Add docs for JDBC authentication usage with in-memory database
[KYUUBI #3685] Fix flaky test BatchRestApiSuite - basic batch rest client with invalid user
[KYUUBI #3385] Set spark.kubernetes.executor.podNamePrefix if missing
[KYUUBI #3664] [SPARK][K8S][BATCH] Kyuubi should set env SPARK_USER_NAME on submitting Spark batch job to K8s
[KYUUBI #3670] [SPARK][K8S] Fix Spark master match rule on tagging Spark application
[KYUUBI #3624][REST] Fix description typo of session REST open api
[KYUUBI #3627] Support vanilla Jetty for Spark packaged by sbt
[KYUUBI #3543][FOLLOWUP][1.6] Handle zookeeper watch events
[KYUUBI #3543] Handle zookeeper watch events
[KYUUBI #3619] Enhance JAVA_HOME detection to support RPM
[KYUUBI #3600] [SPARK][K8S] Respect default static port for web UI on Spark K8s cluster mode
[KYUUBI #3597] Engine should prefer ip for registering on K8s cluster mode
[KYUUBI #3555] [FEATURE] Helm install help set kyuubi pod global env
[KYUUBI #3598] Beeline doesn’t work with -e option when started in background
[KYUUBI #3504] Extend JDBC URL to support catalog
[KYUUBI #3527][SPARK][FOLLOWUP] spark.kubernetes.executorEnv should be spark.executorEnv
[KYUUBI #3568] [Bug] [Doris Engine] Fix return decimal, date, timestamp using string
[KYUUBI #3579] Use strict path in dist script to avoid unexpected cp behavior
[KYUUBI #3372] [BUILD] Enable -x on build/dist
[KYUUBI #3571] Fix NPE for batch command line
[KYUUBI #3566] Shade Zookeeper into spark engine jar
[KYUUBI #3519] Flink SQL Engine - GetColumns Operation
[KYUUBI #3539] [FEATURE][TPCDS] Add white list help run the specified queries
[KYUUBI #3562] Fix the outdate docs index
[KYUUBI #3560] Flink SQL engine supports run DDL across versions
[KYUUBI #3521] [TEST] Fix Flink flaky test - select count
[KYUUBI #3549][FOLLOWUP] Simplify test
[KYUUBI #3549] Support query id in Flink engine
[KYUUBI #3544] Fix bin/kyuubi stop check
[KYUUBI #3406] [FOLLOWUP] Add create datasource table DDL usage to Pyspark docs
[KYUUBI #3547] Fix Flink statements results validation
[KYUUBI #3406] [Subtask] [Doc] Add PySpark client docs
[KYUUBI #3527] [SPARK] Kyuubi should set env SPARK_USER_NAME for K8s deployment
[KYUUBI #3451] Implement GetInfo for JDBC engine
[KYUUBI #3510] Reading an uninitialized log should return empty rowSet
[KYUUBI #3522][FOLLOWUP] Fix benchmark results-dir
[KYUUBI #3522] [BUG][TPCDS] Fix benchmark –results-dir
[KYUUBI #3498] Fix GetTables operation in Flink
[KYUUBI #3502] [DOCS] Fix typo and use beeline provided by Kyuubi
[KYUUBI #3484] Implement isSigned method of ResultSetMetaData for Kyuubi JDBC driver
[KYUUBI #3452] Implement GetInfo for Trino engine
[KYUUBI #3459] [BUILD] Share jars to reduce binary release tarball size
[KYUUBI #3051][FOLLOWUP] Remove redundant serverIpAddress method in SparkSessionImpl
[KYUUBI #3453] [MINOR] Remove unused spark.version property from the pom file of kyuubi-extension-spark-common module
[KYUUBI #3122] GetInfo supports return server/engine info
[KYUUBI #3318] Transfer the TGetInfoReq to kyuubi engine side to check the connection valid and keep connection alive
[KYUUBI #3064] Fix scala NPE issue when adding non-local jar URI to class loader
[KYUUBI #3408] [TEST] Remove the unstable flink process builder test case
[KYUUBI #3208] Fix Flaky Test - MetadataManagerSuite: metadata request metrics
[KYUUBI #3315] Fast fail the kyuubi connection if the engine application has been terminated
Credits
Last but not least, this release would not have been possible without the following contributors:
Release Notes - 1.6.0-incubating
2022 Sep 6
Highlight
Server
Client
Engine
Plugin
Docs
ChangeList
Credits
Last but not least, this release would not have been possible without the following contributors:
Release Notes - 1.5.2-incubating
2022 Jun 18
ChangeList
Credits
Last but not least, this release would not have been possible without the following contributors: