Releases Archive

    Release Notes - 1.10.2

    2025 May 27

    Highlight

    • Support Spark 4.0

    Changelogs

    • [KYUUBI #7055] Support to filter out server only configs with prefixes
    • [KYUUBI #7051] Fix usage without sslTrustStore of JDBC driver
    • [KYUUBI #7054] Add server only flag for more server/credentials/frontend/metadata configs
    • [KYUUBI #7046] Bump dropwizard metrics version to 4.2.30
    • [KYUUBI #7045] Expose jetty metrics
    • [KYUUBI #7044] Bump jetty version to 9.4.57.v20241219
    • [KYUUBI #7035] Close the operation by operation manager to prevent operation leak
    • [KYUUBI #7041] Fix NPE when getting metadtamanager in KubernetesApplicationOperation
    • [KYUUBI #6828] Fix NPE in KyuubiBaseResultSet::getBigDecimal
    • [KYUUBI #7036] Improve docs for kyuubi-extension-spark-jdbc-dialect
    • [KYUUBI #7034] Prefer to use pod spark-app-name label as application name than pod name
    • [KYUUBI #7025] [KYUUBI #6686] Prefer terminated container app state than terminated pod state
    • [KYUUBI #7026] Audit the kubernetes pod event type and fix DELETE event process logical
    • [KYUUBI #7027] Support to initialize kubernetes clients on kyuubi server startup
    • [KYUUBI #7022] Update announcement mail template to contain download links
    • [KYUUBI #7015] Record the session disconnected info into kyuubi session event
    • [KYUUBI #7017] Using mutable JettyServer uri to prevent batch kyuubi instance mismatch
    • [KYUUBI #7011] Set kyuubi session engine client after opening engine session successfully
    • [KYUUBI #7009] Backport HIVE-26723: Configurable canonical name checking.
    • [KYUUBI #7008] Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1
    • [KYUUBI #7004] Include FastXML Jackson into authZ shaded jar
    • [KYUUBI #7003] Cut out JNA dependencies for authZ plugin
    • [KYUUBI #7000] Exclude aws-java-sdk-logs from kyuubi-spark-authz-shaded
    • [KYUUBI #6998] Harness SparkProcessBuilderSuite
    • [KYUUBI #6997] Get the latest batch app info after submit process terminated to prevent batch ERROR due to engine submit timeout
    • [KYUUBI #6785] Shutdown the executor service in KubernetesApplicationOperation and prevent NPE
    • [KYUUBI #6984] Fix ValueError when rendering MapType data
    • [KYUUBI #6965] Bump Scala 2.13.16
    • [KYUUBI #6939] KSHC supports Spark 3.5.5
    • [KYUUBI #6921] Test CTAS for Paimon
    • [KYUUBI #6925] Only run Paimon authz tests with Scala 2.12
    • [KYUUBI #6920] Spark SQL engine supports Spark 4.0
    • [KYUUBI #6918] Cache client ipAddress in kyuubi jdbc connection
    • [KYUUBI #6912] Properly handle empty attribute set on mergeRelationColumnLineage
    • [KYUUBI #6917] Bump Hudi 1.0.1
    • [KYUUBI #6915] Fix ClickHouse integration tests
    • [KYUUBI #6814] Bump cross-spawn from 7.0.3 to 7.0.6
    • [KYUUBI #6900] Correct spelling errors in ’large_query_results’ part
    • [KYUUBI #6891] Fix get existing gauge issue
    • [KYUUBI #6883] Using withOauthTokenProvider instead of withOauthToken to support token refresh
    • [KYUUBI #6843] Fix ‘query-timeout-thread’ thread leak
    • [KYUUBI #6866] Add metrics for SSL keystore expiration time
    • [KYUUBI #6864] Support to return prometheus metrics with instance label
    • [KYUUBI #6862] Spark 3.3: MaxScanStrategy supports DSv2
    • [KYUUBI #6859] Exclude log4j12 from hive engine module classpath
    • [KYUUBI #6857] Spark 3.4: MaxScanStrategy supports DSv2

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Igor Khrol
    • Fei Wang
    • Reese Feng
    • Senmiao Liu
    • David Yuan
    • Xu Guang Lv
    • Zhen Wang
    • Peiyue Liu
    • shuyouZZ
    • GitHub Bot
    • He Zhao
    • Cheng Pan

    Release Notes - 1.9.4

    2025 May 27

    Highlight

    • Support Spark 4.0

    Changelogs

    • [KYUUBI #7073] Retry 3 times on deploying to nexus
    • [KYUUBI #7051] Fix usage without sslTrustStore of JDBC driver
    • [KYUUBI #7035] Close the operation by operation manager to prevent operation leak
    • [KYUUBI #7041] Fix NPE when getting metadtamanager in KubernetesApplicationOperation
    • [KYUUBI #6828] Fix NPE in KyuubiBaseResultSet::getBigDecimal
    • [KYUUBI #7011] Set kyuubi session engine client after opening engine session successfully
    • [KYUUBI #7008] Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1
    • [KYUUBI #6997] Get the latest batch app info after submit process terminated to prevent batch ERROR due to engine submit timeout
    • [KYUUBI #6785] Shutdown the executor service in KubernetesApplicationOperation and prevent NPE
    • [KYUUBI #6984] Fix ValueError when rendering MapType data
    • [KYUUBI #6939] KSHC supports Spark 3.5.5
    • [KYUUBI #6925] Only run Paimon authz tests with Scala 2.12
    • [KYUUBI #6920] Spark SQL engine supports Spark 4.0
    • [KYUUBI #6912] Properly handle empty attribute set on mergeRelationColumnLineage
    • [KYUUBI #6883] Using withOauthTokenProvider instead of withOauthToken to support token refresh
    • [KYUUBI #6843] Fix ‘query-timeout-thread’ thread leak
    • [KYUUBI #6859] Exclude log4j12 from hive engine module classpath
    • [KYUUBI #6838] Update archive mailing list address
    • [KYUUBI #6837] Disable K8s CI image GHA cache

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Peiyue Liu
    • Reese Feng
    • Senmiao Liu
    • Cheng Pan
    • Fei Wang
    • Igor Khrol
    • Xu Guang Lv

    Release Notes - 1.10.1

    2024 Dec 26

    Changelogs

    • [KYUUBI #6840] Fix PodMonitor pods selection
    • [KYUUBI #6829] Add metrics for batch pending max elapse time
    • [KYUUBI #6836] Ship kafka-clients in binary distribution tarball without compression libs
    • [KYUUBI #6838] [INFRA] Update archive mailing list address
    • [KYUUBI #6837] [INFRA] Disable K8s CI image GHA cache
    • [KYUUBI #6828] Clean up and improve error message for KyuubiBaseResultSet
    • Revert “[KYUUBI #6639] Port HIVE-27815: Support update numModifiedRows”
    • [KYUUBI #6722] Fix AppState when Engine connection is terminated
    • [KYUUBI #6820] Explicitly disable attach-scaladocs for pure Java modules
    • [KYUUBI #6813] [BUILD] Replace java.version 1.8 with 8
    • [KYUUBI #6773] Bump commons-codec from 1.15 to 1.17.1
    • [KYUUBI #6808] [BUILD]Replace systemProperties with systemPropertyVariables in maven-surefire-plugin
    • [KYUUBI #6805] [K8S][HELM] Update default Kyuubi version to 1.10.0
    • [KYUUBI #6778] [BUILD] allow frontend plugin not to inherits Maven’s http proxy config
    • [KYUUBI #6786] Skip repeated checks on convert function in TColumnGenerator
    • [KYUUBI #6790] Fix engine cannot exit when gracefully stopped

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Bowen Liang
    • pionCham
    • Denis Krivenko
    • Fei Wang
    • Zhen Wang
    • Cheng Pan
    • Joao Amaral
    • zhifanggao

    Release Notes - 1.9.3

    2024 Dec 2

    Changelogs

    • [KYUUBI #6828] Clean up and improve error message for KyuubiBaseResultSet
    • [KYUUBI #6741] [CI] update GHA actions/upload-artifact from v3 to v4
    • [KYUUBI #6722] Fix AppState when Engine connection is terminated
    • [KYUUBI #6820] Explicitly disable attach-scaladocs for pure Java modules
    • [KYUUBI #6813] [BUILD] Replace java.version 1.8 with 8
    • [KYUUBI #6790] Fix engine cannot exit when gracefully stopped
    • [KYUUBI #6772] Fix ProcessBuilder to properly handle Java opts as a list
    • [KYUUBI #6757] Pass build args to extra spark engine build command
    • [KYUUBI #6750] [REST] Using ForbiddenException instead of NotAllowedException
    • [KYUUBI #6728] [DOC] update Authz plugin docs of build command with -am option
    • [KYUUBI #6688] [SPARK] Avoid trigger execution when getting result schema
    • [KYUUBI #6638] Authz shaded should include jsr311-api
    • [KYUUBI #6720] K8s pod OOM Killed should be identified as Application failed state
    • [KYUUBI #6709] Fix seq and set equals check bug while check auth type in AuthenticationFilter.initAuthHandlers
    • [KYUUBI #6708] spark-sql-engine’s classes should not be visible in server module
    • [KYUUBI #6699] Bump Spark 4.0.0-preview2
    • [KYUUBI #6648] Support to show launch engine log for thrift http mode
    • [KYUUBI #6698] Exclude transitive deps of paimon
    • [KYUUBI #6680] Format the kyuubi-ctl batch command output
    • [KYUUBI #6668] Fix kyuubi batch state abnormal
    • [KYUUBI #6661] Improve perf for column-based TRowSet generation
    • [KYUUBI #6658] [DOCS] Fixed typo in REST API docs
    • [KYUUBI #6594] Port HIVE-26633: Make thrift client maxMessageSize configurable
    • [KYUUBI #6638] Shade jsr311-api in Authz
    • [KYUUBI #6635] Fix Hive 2.3.10 integration tests
    • [KYUUBI #6583] Support to cancel Spark python operation
    • [KYUUBI #6627] Using concurrent collections for session operation handle set
    • [KYUUBI #6626] Fix operation never expired issue with periodical listOperations api calls
    • [KYUUBI #6620] [K8S][HELM] Update default Kyuubi version to 1.9.2
    • [KYUUBI #6615] Make Jetty sending server version in response configurable
    • [KYUUBI #6609] Bump Spark 3.5.2
    • [KYUUBI #6606] Restore shell compatibility for build/mvn
    • [KYUUBI #6606] build/mvn checks project local downloaded binary ahead
    • [KYUUBI #6600] Correct usage of maven.scaladoc.skip
    • [KYUUBI #6598] Flink engine module supports building with Scala 2.13
    • [KYUUBI #6599] Disable Javadoc/Scaladoc by default
    • [KYUUBI #6596] Fix typos in architecture page
    • [KYUUBI #6591] Support authorization on swagger UI
    • [KYUUBI #6592] Add uploadWorkDir into local dir allow list and do not skip check for resource uploaded use case
    • [KYUUBI #6588] Get effective USER if env var is unavailable
    • [KYUUBI #6571] Bump apache parent pom from 31 to 33
    • [KYUUBI #6586] Bump commons-io to 2.16.1
    • [KYUUBI #6564] Insert into table check the privilege of table
    • [KYUUBI #6549] Correctly handle empty Java options for engines
    • [KYUUBI #6579] Show kyuubi batch app details on app state change
    • [KYUUBI #6364] Bump scala-maven-plugin from 4.8.0 to 4.9.2
    • [KYUUBI #6566] Fix typos in KyuubiConf
    • [KYUUBI #6541] [AUTHZ] Fix DataSourceV2RelationTableExtractor can’t get the ‘database’ attribute if it’s a Paimon plan.
    • [KYUUBI #6550] Delete updateStrategy.rollingUpdate.maxUnavailable because it is alpha feature
    • [KYUUBI #6543] Bump Kyuubi Shaded 0.4.1

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • promising-forever
    • Cheng Pan
    • Bowen Liang
    • lifulong
    • be-hase
    • DenineLu
    • Baokun Han
    • hh-cn
    • LiJie20190102
    • Senmiao Liu
    • Zhen Wang
    • Joao Amaral
    • Fei Wang
    • Mike Wilson
    • Madhukar525722
    • liujiayi771
    • paul8263
    • pionCham

    Release Notes - 1.8.3

    2024 Nov 27

    Changelogs

    [KYUUBI #6417] Remove scala compile args -Yno-adapted-args [KYUUBI #6820] Explicitly disable attach-scaladocs for pure Java modules [KYUUBI #6790] Fix engine cannot exit when gracefully stopped [KYUUBI #6709] Fix seq and set equals check bug while check auth type in AuthenticationFilter.initAuthHandlers [KYUUBI #6708] spark-sql-engine’s classes should not be visible in server module [KYUUBI #6658] Fixed typo in REST API docs [KYUUBI #6606] Restore shell compatibility for build/mvn [KYUUBI #6606] build/mvn checks project local downloaded binary ahead [KYUUBI #6600] Correct usage of maven.scaladoc.skip [KYUUBI #6591] Support authorization on swagger UI [KYUUBI #6592] Add uploadWorkDir into local dir allow list and do not skip check for resource uploaded use case [KYUUBI #6588] Get effective USER if env var is unavailable [KYUUBI #6564] Insert into table check the privilege of table [KYUUBI #6549] Correctly handle empty Java options for engines [KYUUBI #6542] KyuubiBatchService should wait for HTTP server started before picking jobs [KYUUBI #6508] Add the key-value pairs in optimizedConf to session conf [KYUUBI #6466] Bump minikube and kubernetes version of integration test [KYUUBI #6463] Release semaphore immediately after startup process exit [KYUUBI #5957] Flink engine should not load kyuubi-defaults.conf [KYUUBI #6440] Fix casing in kubectl commands for serviceaccount creation [KYUUBI #6430] livenessProbe uses absolute path based on KYUUBI_HOME [KYUUBI #6422] Merge standard error to output [KYUUBI #6410] Check if asyncRequestExecutor is initialized before shutdown [KYUUBI #6394] Fix example SQL in Spark TPC-DS connector docs [KYUUBI #4847] Exclude the alive probe sessions in terminating checker [KYUUBI #6172] Support to interrupt the thrift request immediately after marking the engine not alive [KYUUBI #6373] Avoid NPE in get operation log api [KYUUBI #6366] Improve log message for YARN getApplicationInfoByTag

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Fei Wang
    • Zhen Wang
    • ic4y
    • lifulong
    • pionCham
    • liujiayi771
    • LiJie20190102
    • beryllw
    • huangzhir
    • Fengyu Cao
    • lllong33
    • Cheng Pan
    • Fu Chen
    • Qingbo Jiao

    Release Notes - 1.10.0

    2024 Oct 27

    Highlight

    • Fully verified with Java 8, 11, and 17, on Scala 2.12 and 2.13
    • Fully verified with Apache Spark 3.2 to 3.5, initial support for 4.0-preview, support for Spark 3.2 is deprecated and will be removed in the next release
    • Fully verified with Apache Flink 1.17 to 1.20
    • Support creating batch job with uploading extra resources via REST API
    • Support server choosing strategy for HA mode
    • Introduce Spark JVM quake plugin
    • Periodically cleanup expired temp files and operation logs on server
    • Performance improvement for Ranger access requests deduplication in AuthZ plugin

    Server

    • [KYUUBI #6034] Kyuubi Server HA&ZK get server from serverHosts support more strategy
    • [KYUUBI #6756] [REST] Check max file size of uploaded resource and extra resources in batch creation
    • [KYUUBI #6750] [REST] Using ForbiddenException instead of NotAllowedException
    • [KYUUBI #6731] [REST] Check all required extra resource files uploaded in creating batch request
    • [KYUUBI #6720] K8s pod OOM Killed should be identified as Application failed state
    • [KYUUBI #6681][FOLLOWUP] Log the kill batch request before closeSession and during closeOperation
    • [KYUUBI #6772] Fix ProcessBuilder to properly handle Java opts as a list
    • [KYUUBI #6753] Set hadoop fs delegation token renewer to empty
    • [KYUUBI #6704] Disable periodic gc if set interval to 0
    • [KYUUBI #6680] Format the kyuubi-ctl batch command output
    • [KYUUBI #6668] Fix kyuubi batch state abnormal
    • [KYUUBI #6681] Log the delete batch request in batch operation log
    • [KYUUBI #6669] Track the client ip for internal kyuubi RESTful requests
    • [KYUUBI #6645] Size based eviction for server-side temp files cleanup
    • [KYUUBI #6661] Improve perf for column-based TRowSet generation
    • [KYUUBI #6652] Support to list batches in descending order
    • [KYUUBI #6648] Support to show launch engine log for thrift http mode
    • [KYUUBI #6646] Fix RESTful API NPE when restarting Kyuubi server
    • [KYUUBI #6587] Periodically expire temp files and operation logs on server to avoid memeory leak by Files.deleteOnExit
    • [KYUUBI #6335][FOLLOWUP] Using sessionId for sessionUploadFolderPath
    • [KYUUBI #6627] Using concurrent collections for session operation handle set
    • [KYUUBI #6626] Fix operation never expired issue with periodical listOperations api calls
    • [KYUUBI #6615] Make Jetty sending server version in response configurable
    • [KYUUBI #6585] Support Multiple Configuration Files in FileSessionConfAdvisor
    • [KYUUBI #6592] Add uploadWorkDir into local dir allow list and do not skip check for resource uploaded use case
    • [KYUUBI #6588] Get effective USER if env var is unavailable
    • [KYUUBI #6335] [REST] Support uploading extra resources in creating batch jobs via REST API
    • [KYUUBI #6582] [KYUUBI-6581] Zorder clause syntax does not support special characters
    • [KYUUBI #6554] Delete redundant code related to zorder
    • [KYUUBI #6551] Allow insert zorder when global sort is false and the plan is Repartition or RepartitionByExpression.
    • [KYUUBI #6542] KyuubiBatchService should wait for HTTP server started before picking jobs
    • [KYUUBI #6534] Fix checkEngineConnectionAlive may throw NPE
    • [KYUUBI #6524] Trino engine supports insecure configuration
    • [KYUUBI #6008] RESTful API supports killing engine forcibly
    • [KYUUBI #6508] Add the key-value pairs in optimizedConf to session conf
    • [KYUUBI #6500] Remove service loader descriptor for relocated HMS DelegationTokenIdentifier
    • [KYUUBI #6499] Rewrite some utility methods in Java
    • [KYUUBI #6469] Lazily initialize RecordReaderIterator to avoid driver oom when fetching big result set
    • [KYUUBI #6476] Fix incomplete app events deserialization in SHS
    • [KYUUBI #6302] Call cancelJobGroup immediately after statement execution finished
    • [KYUUBI #4847][FOLLOWUP] Fix engine session never idle issue
    • [KYUUBI #6422] Merge standard error to output
    • [KYUUBI #6423] Fix Scalatest exclusion tags
    • [KYUUBI #6396][FOLLOWUP] Avoid NPE
    • [KYUUBI #6415] Manage Jersey in kyuubi-server module
    • [KYUUBI #6410] Check if asyncRequestExecutor is initialized before shutdown
    • [KYUUBI #6408] Change to Base64 RFC4648 for handle guid and credential
    • [KYUUBI #6400] Fix memory leak when using saveToFile
    • [KYUUBI #6376] Improve closeEngine method names and logs due to closeSession
    • [KYUUBI #6379] Return engine name and engine url within KyuubiSessionEvent/SessionData
    • [KYUUBI #4847][FOLLOWUP] Exclude the alive probe sessions in terminating checker
    • [KYUUBI #6172][TASK][EASY] Support to interrupt the thrift request immediately after marking the engine not alive
    • [KYUUBI #6373] Avoid NPE in get operation log api
    • [KYUUBI #6359] [REST] Return more fields within session/batch data
    • [KYUUBI #6366] Improve log message for YARN getApplicationInfoByTag
    • [KYUUBI #6244][TEST] Fix test logs upload error
    • [KYUUBI #6322] Expose more config items of server internal rest client to users
    • [KYUUBI #6049] Support to filter sessions/operations with session type
    • [KYUUBI #6320] Fix terminated application pods not deleted issue when kyuubi server restarted
    • [KYUUBI #6278] Update DB state when the query fallback to resource manager and the batch app is terminal (#6284)
    • [KYUUBI #6291] Avoid NPE in MySQLErrPacket
    • [KYUUBI #6216] Support to deny some client ips to make connection
    • [KYUUBI #6199] Support to run HiveSQLEngine on kerberized YARN
    • [KYUUBI #6204] Fix kyuubi session limiter leak when opening session failed
    • [KYUUBI #6165] Add some HttpClient options for EngineUIProxyServlet

    Spark Engine

    • [KYUUBI #6688] [SPARK] Avoid trigger execution when getting result schema
    • [KYUUBI #6708] spark-sql-engine’s classes should not be visible in server module
    • [KYUUBI #6699] Bump Spark 4.0.0-preview2
    • [KYUUBI #6698] Exclude transitive deps of paimon
    • [KYUUBI #6686] Ignore Spark pod container state if pod is terminated
    • [KYUUBI #6653] Persist spark pod name conf into batch metadata
    • [KYUUBI #6583] Support to cancel Spark python operation
    • [KYUUBI #6609] Bump Spark 3.5.2
    • [KYUUBI #6601] Bump Delta Lake 4.0.0rc1 and enable Delta Lake test for Spark 4.0
    • [KYUUBI #6574] Skip eagerly execute command in PlanOnly mode of Spark Engine
    • [KYUUBI #6549] Correctly handle empty Java options for engines
    • [KYUUBI #6579] Show kyuubi batch app details on app state change
    • [KYUUBI #6545] Deprecate and remove building support for Spark 3.2
    • [KYUUBI #6546] Update incorrect descriptions in Zorder related configurations
    • [KYUUBI #6531] Fix SPARK-EngineTab stop/gracefulstop not work
    • [KYUUBI #6446] Add tests for Spark saveToFile function
    • [KYUUBI #6518] Support extracting URL for Spark 4 on YARN
    • [KYUUBI #6516] Fix KyuubiSparkUtil.buildURI
    • [KYUUBI #6489] [PYTHON] PyKyuubi get_table_names also supports Spark SQL dialect
    • [KYUUBI #6302][FOLLOWUP] Skip spark job group cancellation on incremental collect mode
    • [KYUUBI #6453] Make KSHC support Spark 4.0 and enable CI for Spark 4.0
    • [KYUUBI #6452] Add cross test for Spark 4.0
    • [KYUUBI #6447] Use static regex Pattern instances in JavaUtils.timeStringAs and JavaUtils.byteStringAs
    • [KYUUBI #6437] Fix Spark engine query result save to HDFS
    • [KYUUBI #6441] Kyuubi Spark TPC-DS/H Connector cross version test
    • [KYUUBI #6247] Make KSHC binary compatible with multiple Spark versions
    • [KYUUBI #6425] Fix tests in spark engine and kyuubi server modules with Spark 4.0
    • [KYUUBI #6424] TPC-H/DS connector support Spark 4.0
    • [KYUUBI #6413] SPARK_HOME detection supports Spark 4
    • [KYUUBI #6404] Fix HiveResult.toHiveString compatibility for Spark 4.0
    • [KYUUBI #6405] Spark engine supports both javax and jakarta ws.rs namespaces
    • [KYUUBI #6399] Spark Kyuubi UI supports both javax and jakarta servlet namespaces
    • Revert “[KYUUBI #6390] Temporarily disable UI Tab for Spark 4.0 and above”
    • [KYUUBI #6390] Temporarily disable UI Tab for Spark 4.0 and above
    • [KYUUBI #6305][FOLLOWUP] Improve package Spark SQL engine both Scala 2.12 and 2.13
    • [KYUUBI #6377] Fix isCommand check and set min rows threshold for saveToFile
    • [KYUUBI #6370] For saveToFile, checking idx to prevent IndexOutOfBoundsException
    • [KYUUBI #6323] Upgrade Spark 3.4.3
    • [KYUUBI #6321] Support to get Spark Kubernetes app URL
    • [KYUUBI #6315] Spark 3.5: MaxScanStrategy supports DSv2
    • [KYUUBI #6297] Package Spark SQL engine both Scala 2.12 and 2.13
    • [KYUUBI #6290] Add custom exception serialization for SparkOperationEvent
    • [KYUUBI #6287] Avoid missing matche
    • [KYUUBI #6250][FOLLOWUP] Simplify code after dropping Spark 3.1
    • [KYUUBI #6250] Drop support for Spark 3.1
    • [KYUUBI #6260] Clean up and improve comments for spark extensions
    • [KYUUBI #5767][FOLLOWUP] Fix spark batch conf not convert issue
    • [KYUUBI #6223] Fix Scala interpreter can not access spark.jars issue
    • [KYUUBI #6215] Improve DropIgnoreNonexistent rule for Spark 3.5
    • [KYUUBI #6211] Check memory offHeap enabled for CustomResourceProfileExec
    • [KYUUBI #6207] Support to retrieve Spark UserDefinedType result
    • [KYUUBI #6205] Backport SPARK-47300: quoteIfNeeded should quote identifier starts with digits
    • [KYUUBI #6206] Update comments for job group id of Spark broadcast jobs
    • [KYUUBI #6180] Kyuubi Spark nightly testing should use JDK 17
    • [KYUUBI #6180] Kyuubi Spark nightly testing should use JDK 17

    Spark AuthZ Plugin

    • [KYUUBI #6754][AUTHZ] Improve the performance of Ranger access requests deduplication
    • [KYUUBI #6638][FOLLOWUP] Authz shaded should include jsr311-api
    • [KYUUBI #6709] Fix seq and set equals check bug while check auth type in AuthenticationFilter.initAuthHandlers
    • [KYUUBI #6666][AUTHZ]Upgrade Ranger plugin to 2.5.0
    • [KYUUBI #5402] Introduce Spark JVM quake plugin
    • [KYUUBI #6619][FOLLOWUP] Do not log auth audit log for thrift http cookie auth
    • [KYUUBI #6640] [AUTHZ] Adapt Derby 10.16 new JDBC driver package name
    • [KYUUBI #6638] Shade jsr311-api in Authz
    • [KYUUBI #6619] Support http bearer authentication for ThriftHttp protocol
    • [KYUUBI #6623] Support bearer token authentication on swagger UI
    • [KYUUBI #6618] Support http bearer token authentication for REST protocol
    • [KYUUBI #6564] Insert into table check the privilege of table
    • [KYUUBI #6541] [AUTHZ] Fix DataSourceV2RelationTableExtractor can’t get the ‘database’ attribute if it’s a Paimon plan.
    • [KYUUBI #6197] Revise dependency management of Spark authZ plugin
    • [KYUUBI #6194] AuthZ shaded should include ranger-plugins-cred

    Other Spark Plugins

    • [KYUUBI #6398] Fix lineage plugin UT for Spark 4.0
    • [KYUUBI #6368] Flink engine supports user impersonation
    • [KYUUBI #6598] Flink engine module supports building with Scala 2.13
    • [KYUUBI #6557] Support Flink 1.20
    • [KYUUBI #6537] Upgrade Flink from 1.19.0 to 1.19.1
    • [KYUUBI #5957] Flink engine should not load kyuubi-defaults.conf
    • [KYUUBI #6353] Catch exception for closing flink internal session
    • [KYUUBI #6367] Flink SQL engine supports RenewDelegationToken
    • [KYUUBI #6344] FlinkProcessBuilder prioritizes user configurations
    • [KYUUBI #6249] Drop support for Flink 1.16.
    • [KYUUBI #6134][FOLLOWUP] Add profile and CI for Flink 1.19

    Trino Engine

    • [KYUUBI #6696] Fix Trino Status Printer to Prevent Thread Leak
    • [KYUUBI #6485] Fix the Presto TABLE NOT FOUND error message that failed to match
    • [KYUUBI #6524] Trino engine supports insecure configuration
    • [KYUUBI #6387] [TRINO] Trino engine improve operation log

    JDBC Engine

    • Revert “[KYUUBI #6526] Kyuubi BeeLine wrongly process JDBC URL that contains --”
    • [KYUUBI #6526] Kyuubi BeeLine wrongly process JDBC URL that contains --
    • [KYUUBI #6339] kyuubi-beeline auto constructs JDBC URL from kyuubi-defaults.conf
    • [KYUUBI #6396] Add caching for KerberosAuthentication using ticketCache key
    • [KYUUBI #6346] Change Kyuubi JDBC Driver default port and make it effective
    • [KYUUBI #6338] Support connecting Kyuubi using Hive JDBC driver 4.0
    • [KYUUBI #6316] Support switching jdbc dialect based on session conf
    • [KYUUBI #6253] Support running JDBC engine on YARN AM
    • [KYUUBI #5374][FOLLOWUP] Fix JDBC ClickHouse TRowSet Generator
    • [KYUUBI #6243] Add missing Impala alias for JDBC engine
    • [KYUUBI #5374] JDBC Engine supports ClickHouse
    • [KYUUBI #6221] Fix parameter replacement issue caused by incorrect sql split
    • [KYUUBI #2782][FOLLOWUP] Add missed UNION_TYPE back
    • [KYUUBI #6182] JDBC engine using the overlay conf as session conf

    Other Engine

    • [KYUUBI #6594] Port HIVE-26633: Make thrift client maxMessageSize configurable
    • [KYUUBI #6639] Port HIVE-27815: Support update numModifiedRows
    • [KYUUBI #6635] Fix Hive 2.3.10 integration tests
    • [KYUUBI #6201] Distinguish the collect mode of different engines

    Python client

    • [KYUUBI #6281][PY] Enable hive test in python client
    • [KYUUBI #6281][PY] Speed up testing with xdist plugin
    • [KYUUBI #6281][PY] Initialize github action for python unit testing
    • [KYUUBI #6567] Fix compatibility of pyhive with setuptools==72.0.0
    • [KYUUBI #6434] Add footnote about pyhive origin

    Web UI

    • [KYUUBI #6727] replace immutable empty list and map in BatchRequest initialization
    • ⬆️ Bump rollup from 3.29.4 to 3.29.5 in /kyuubi-server/web-ui (#6717)
    • [KYUUBI #6712] [UI] Bump pnpm to v9 and nodejs to v20 LTS
    • [KYUUBI #6660][UI] Improve the display of sql editor and query result
    • [KYUUBI #6591] Support authorization on swagger UI
    • [KYUUBI #6463] Release semaphore immediately after startup process exit
    • ⬆️ Bump braces from 3.0.2 to 3.0.3 in /kyuubi-server/web-ui (#6461)
    • [KYUUBI #6428] kyuubi-server/web-ui should not to set a timeout for axios requests
    • [KYUUBI #6299] Support disabling Web UI
    • [KYUUBI #6079] Web UI support Basic authN
    • [KYUUBI #6283] Improve the message of distribution built without enabling web ui
    • [KYUUBI #6248] Web SQL Editor supports run SQL snippets
    • [KYUUBI #6257] Bump vite from 4.5.2 to 4.5.3 in /kyuubi-server/web-ui
    • [KYUUBI #6228] Dropdown Selection and Parameter Parsing Issues in Kyuubi SQL Editor
    • [KYUUBI #6188] Bump follow-redirects from 1.15.4 to 1.15.6 in web-ui

    Documentation

    • [KYUUBI #6752] [DOC] Bump doc build requirements
    • [KYUUBI #6728] [DOC] update Authz plugin docs of build command with -am option
    • ⬆️ Bump get-func-name in /kyuubi-server/web-ui (#6735)
    • [KYUUBI #6734] [DOC] add authentication example in REST API docs
    • ⬆️ Bump micromatch from 4.0.5 to 4.0.8 in /kyuubi-server/web-ui (#6729)
    • [KYUUBI #6719] [DOC] Fix a couple of typos
    • [KYUUBI #6673] [DOC] Fix typos in logging.md
    • [KYUUBI #6671] [DOC] Fix typo in ENGINE SHARE LEVEL docs
    • [KYUUBI #6658] [DOCS] Fixed typo in REST API docs
    • [KYUUBI #6628] [DOCS] Improve docs for GROUP Share Level
    • [KYUUBI #6596] Fix typos in architecture page
    • [KYUUBI #6566] Fix typos in KyuubiConf
    • [KYUUBI #6512] Improve docs for KSHC
    • [KYUUBI #6514] Add link to Github Issues on Contributing Documentation page
    • [KYUUBI #6507] Correct Kyuubi BeeLine help message
    • [KYUUBI #6504] Improve docs for Solution for Large Query Results
    • [KYUUBI #6440] Fix casing in kubectl commands for serviceaccount creation
    • [KYUUBI #6432] Mention support for Spark 4.0 in docs
    • [KYUUBI #6394] Fix example SQL in Spark TPC-DS connector docs
    • [KYUUBI #6242] Remove block cleaner docs
    • [KYUUBI #6234] Fix invalid code-block syntax in docs
    • [KYUUBI #6191] Update docs to mention support of Flink 1.19

    Building, Infrastructure and Dependency

    • [KYUUBI #6775] Bump Github actions action/cache and actions/dependency-review-action v4
    • [KYUUBI #6768] [CI] skip explicitly setting check-latest to false
    • [KYUUBI #6762] [BUILD] cleanup dockerfile warnings
    • [KYUUBI #6763] [CI] Specify JDK 11 for spark engine on kubernetes IT
    • [KYUUBI #6741] [CI] update GHA actions/upload-artifact from v3 to v4
    • [KYUUBI #6739] [CI] Bump runner image from Ubuntu 22.04 to 24.04
    • [KYUUBI #6687] [BUILD] Use MAVEN_ARGS for passing Maven CLI arguments
    • [KYUUBI #6757] Pass build args to extra spark engine build command
    • [KYUUBI #6740] [CI] update GHA docker/build-push-action from v5 to v6 to support build summary
    • [KYUUBI #6755] Upgrade default flink version to 1.20.0
    • [KYUUBI #6742] Bump Jetty from 9.4.54 to 9.4.56
    • [KYUUBI #6743] Bump protobuf from 3.25.4 to 3.25.5
    • [KYUUBI #6744] Bump vite from 4.5.3 to 4.5.5
    • [KYUUBI #6745] Bump testcontainers-scala from 0.41.0 to 0.41.4
    • [KYUUBI #6746] Bump commons-lang3 from 3.13.0 to 3.17.0
    • [KYUUBI #6747] Bump Guava from 32.1.3 to 33.3.1
    • [KYUUBI #6737] Bump vertx-grpc and vertx-core from 4.5.1 to 4.5.3
    • ⬆️ Bump semver in /kyuubi-server/web-ui (#6736)
    • ⬆️ Bump axios from 1.6.0 to 1.7.4 in /kyuubi-server/web-ui (#6617)
    • [KYUUBI #6733] Bump SQLlite from 3.42 to 3.46
    • [KYUUBI #6682] Bump super-linter to v7
    • [KYUUBI #6677] Bump scala-collection-compat to 2.12.0
    • [KYUUBI #6676] Bump Scala to 2.12.19
    • [KYUUBI #6672] Cleanup unused Commons Lang 2 dependency
    • [KYUUBI #6664] [BUILD] Bump Maven from 3.8.8 to 3.9.9
    • [KYUUBI #6659] Bump Iceberg to 1.6.1
    • [KYUUBI #6644] Bump Paimon to 0.8
    • [KYUUBI #5314] Support JDK 21
    • [KYUUBI #6603] Upgrade Scala version from 2.13.8 to 2.13.14
    • [KYUUBI #6620] [K8S][HELM] Update default Kyuubi version to 1.9.2
    • [KYUUBI #6606][FOLLOWUP] Restore shell compatibility for build/mvn
    • [KYUUBI #6584] Upgrade flip-tables from 1.0.2 to 1.1.1
    • [KYUUBI #6593] bin/kyuubi supports kill command
    • [KYUUBI #6607] Bump gRPC 1.65.1 Protobuf 3.25.4 Guava 32.1.3
    • [KYUUBI #6606] build/mvn checks project local downloaded binary ahead
    • [KYUUBI #6600] Correct usage of maven.scaladoc.skip
    • [KYUUBI #6599] Disable Javadoc/Scaladoc by default
    • [KYUUBI #6571] Bump apache parent pom from 31 to 33
    • [KYUUBI #6580] [K8S][HELM] Improve metrics configuration
    • [KYUUBI #6586] Bump commons-io to 2.16.1
    • [KYUUBI #6364] Bump scala-maven-plugin from 4.8.0 to 4.9.2
    • [KYUUBI #6558] Bump Iceberg 1.6.0
    • [KYUUBI #6559] Various improvments for release scripts
    • [KYUUBI #6550] Delete updateStrategy.rollingUpdate.maxUnavailable because it is alpha feature
    • [KYUUBI #6543] Bump Kyuubi Shaded 0.4.1
    • [KYUUBI #6548] Bump enforcer extra mojo rules to 1.8.0 for recognizing higher Java bytecode versions
    • [KYUUBI #6523] Support configuration with --conf options
    • [KYUUBI #6529] Upgrade kubernetes-client from 6.8.1 to 6.13.1
    • [KYUUBI #6528] Bump Netty 4.1.108.Final
    • [KYUUBI #6515] Bumping up dropwizard metrics to 4.2.26
    • [KYUUBI #6506] kyuubi-beeline supports --conf
    • [KYUUBI #6493] [K8S][HELM] Update default Kyuubi version to 1.9.1
    • [KYUUBI #6484] Bump ws from 8.8.1 to 8.17.1 in web-ui
    • [KYUUBI #6466] Bump minikube and kubernetes version of integration test
    • [KYUUBI #6458] Remove commons-logging from binary release
    • [KYUUBI #6457] Strip dependencies from REST client
    • [KYUUBI #6456] Opens jdk.internal.ref
    • [KYUUBI #6445] Normalize extra name for optional Python distribution dependencies
    • [KYUUBI #6427] Extract data lake artifact names as maven properties
    • [KYUUBI #6451] Bump Hudi 0.15.0 and enable Hudi authZ test for Spark 3.5
    • [KYUUBI #6439] kyuubi-util-scala test jar leaked to compile scope
    • [KYUUBI #6430] livenessProbe uses absolute path based on KYUUBI_HOME
    • [KYUUBI #6419] Bump Jetty 9.4.54.v20240208
    • [KYUUBI #6418] Bump Jersey to 2.40
    • [KYUUBI #6416] Generate flattened POM
    • [KYUUBI #6417] Remove scala compile args -Yno-adapted-args
    • [KYUUBI #6397] Align Antlr version with Spark for Spark 4.0
    • [KYUUBI #6392] Support javax.servlet and jakarta.servlet co-exist
    • [KYUUBI #6391] Bump Arrow from 15.0.2 to 16.0.0
    • [KYUUBI #6382] Bump Delta from 3.1.0 to 3.2.0 for Spark 3.5
    • [KYUUBI #6348] Upgrade iceberg from 1.5.0 to 1.5.2
    • [KYUUBI #6328] Bump Hive 2.3.10
    • [KYUUBI #6357] Bump org.bouncycastle:bcprov-jdk18on from 1.77 to 1.78
    • [KYUUBI #6363] [INFRA] Increase CI concurrency
    • [KYUUBI #6294] Simplify Netty and gRPC dependency management
    • [KYUUBI #6312][HELM] Allow attaching annotations to service account
    • [KYUUBI #6293] Upgrade Arrow from 12.0.0 to 15.0.2
    • [KYUUBI #6298][HELM] Make ServiceMonitor use selector labels
    • [KYUUBI #6251] Improve kyuubi-beeline help message
    • [KYUUBI #6252] Upgrade hive-service-rpc 4.0.0
    • [KYUUBI #6266] Kyuubi BeeLine supports JSON output format.
    • [KYUUBI #5686][FOLLOWUP] Rename pyhive to python
    • [KYUUBI #6180][FOLLOWUP] Update the setup JDK name in nightly.yml
    • [KYUUBI #5686] Donate PyHive to Apache Kyuubi
    • [KYUUBI #6268] Specify logDir for RollingFile filePattern
    • [KYUUBI #6212] Added audit handler shutdown to the shutdown hook
    • [KYUUBI #6267] Remove unused dependency management in POM
    • [KYUUBI #6271] Upgrade kafka-clients from 3.5.1 to 3.5.2
    • [KYUUBI #6239] Rename beeline to kyuubi-beeline
    • [KYUUBI #6240] [INFRA] Add issue template for 2024H1 Kyuubi Code Contribution Task
    • [KYUUBI #6185] Audit kyuubi operation state change log
    • [KYUUBI #6196] Add log4j2 template for kyuubi repl commands
    • [KYUUBI #6195] Update Helm Chart and playground to use 1.9.0
    • [KYUUBI #6193] [INFRA] Add known_transkations
    • [KYUUBI #6183] [K8S] KyuubiConf.getKubernetesConf should set namespace even if the Kubernetes context is empty
    • [KYUUBI #6179] Bump iceberg from 1.4.3 to 1.5.0

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Alexander Rashed
    • Binjie Yang
    • Bogdan
    • Bowen Liang
    • Bruce Wong
    • Cheng Pan
    • Fei Wang
    • Fu Chen
    • George314159
    • Harry
    • Lennon Chin
    • Liu Xiao
    • Lucas Resch
    • Mike Wilson
    • PorterZhang2021
    • Shilun Fan
    • SteNicholas
    • Wang, Fei
    • Zhigang
    • amanraj2520
    • artilexx
    • avishnus
    • axiangzheng
    • binjie yang
    • camper42
    • caoyu
    • chengpeiming
    • dependabot[bot]
    • dnskr
    • dupeng
    • felixzh
    • futureltl
    • hang.huang
    • hezhao2
    • huangxiaoping
    • huangzhir
    • ic4y
    • jeanlyn
    • jialiang
    • jiaoqingbo
    • joey.ljy
    • lifulong
    • lijie0203
    • liuxiao
    • lllong33
    • ludejiu
    • madlnu
    • minyk
    • pengbei
    • promising-forever
    • ryosuke-hasebe
    • senmiaoliu
    • tatian
    • taylor.fan
    • vinoyang
    • wangjunbo
    • wankunde
    • wenjie.wang01
    • wforget
    • xorsum
    • yanghua
    • yikaifei
    • zRain
    • zhang_yao
    • zhouyifan279
    • zml1206
    • 吴梓溢
    • 王龙

    Release Notes - 1.9.2

    2024 Jul 23

    Changelogs

    • [KYUUBI #6542] KyuubiBatchService should wait for HTTP server started before picking jobs
    • [KYUUBI #6534] Fix checkEngineConnectionAlive may throw NPE
    • [KYUUBI #6529] Upgrade kubernetes-client from 6.8.1 to 6.13.1
    • [KYUUBI #6531] Fix SPARK-EngineTab stop/gracefulstop not work
    • [KYUUBI #6518] Support extracting URL for Spark 4 on YARN
    • [KYUUBI #6524] Trino engine supports insecure configuration
    • [KYUUBI #6515] Bumping up dropwizard metrics to 4.2.26
    • [KYUUBI #6516] Fix KyuubiSparkUtil.buildURI
    • [KYUUBI #6508] Add the key-value pairs in optimizedConf to session conf
    • [KYUUBI #6512] Improve docs for KSHC
    • [KYUUBI #6507] Correct Kyuubi BeeLine help message
    • [KYUUBI #6500] Remove service loader descriptor for relocated HMS DelegationTokenIdentifier
    • [KYUUBI #6469] Lazily initialize RecordReaderIterator to avoid driver oom when fetching big result set
    • [KYUUBI #6476] Fix incomplete app events deserialization in SHS
    • [KYUUBI #6302] Skip spark job group cancellation on incremental collect mode
    • [KYUUBI #6302] Call cancelJobGroup immediately after statement execution finished
    • [KYUUBI #4847] Fix engine session never idle issue
    • [KYUUBI #6466] Bump minikube and kubernetes version of integration test
    • [KYUUBI #6463] Release semaphore immediately after startup process exit
    • [KYUUBI #6458] Remove commons-logging from binary release
    • [KYUUBI #6457] Strip dependencies from REST client
    • [KYUUBI #5957] Flink engine should not load kyuubi-defaults.conf
    • [KYUUBI #6353] Catch exception for closing flink internal session
    • [KYUUBI #6251] Replace kyuubi-beeline with beeline in beeline help message
    • [KYUUBI #6439] kyuubi-util-scala test jar leaked to compile scope
    • [KYUUBI #6437] Fix Spark engine query result save to HDFS
    • [KYUUBI #6440] Fix casing in kubectl commands for serviceaccount creation
    • [KYUUBI #6441] Kyuubi Spark TPC-DS/H Connector cross version test
    • [KYUUBI #6247] Make KSHC binary compatible with multiple Spark versions
    • [KYUUBI #6493] [K8S][HELM] Update default Kyuubi version to 1.9.1

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Fu Chen
    • Fei Wang
    • Baokun Han
    • Madhukar525722
    • Cheng Pan
    • Zhen Wang
    • Z1Wu
    • Yifan Zhou
    • avishnus
    • camper42
    • felixzh2020
    • huangzhir
    • ic4y
    • jeanlyn
    • jiaoqingbo
    • promising-forever
    • zcx513566

    Release Notes - 1.9.1

    2024 Jun 3

    Highlights

    • Experimental support Spark 4.0-preview
    • Web UI supports Basic Authentication

    Features and Improvements

    • [KYUUBI #6424] TPC-H/DS connector support Spark 4.0
    • [KYUUBI #6413] SPARK_HOME detection supports Spark 4
    • [KYUUBI #6404] HiveResult.toHiveString compatible to Spark 4.0
    • [KYUUBI #6405] Spark engine supports both javax and jakarta ws.rs namespaces
    • [KYUUBI #6399] Spark engine supports both javax and jakarta servlet namespaces
    • [KYUUBI #6392] Support javax.servlet and jakarta.servlet co-exist
    • [KYUUBI #6297] Package Spark SQL engine both Scala 2.12 and 2.13
    • [KYUUBI #4847] Exclude the alive probe sessions in terminating checker
    • [KYUUBI #6172] Support to interrupt the thrift request immediately after marking the engine not alive
    • [KYUUBI #6315] Spark 3.5: MaxScanStrategy supports DSv2
    • [KYUUBI #6049] Support to filter sessions/operations with session type
    • [KYUUBI #6366] Improve log message for YARN getApplicationInfoByTag
    • [KYUUBI #6298] Make ServiceMonitor use selector labels in Helm charts
    • [KYUUBI #6251] Improve kyuubi-beeline help message
    • [KYUUBI #6290] Add custom exception serialization for SparkOperationEvent
    • [KYUUBI #6079] Web UI supports Basic authN
    • [KYUUBI #6338] Support connecting Kyuubi using Hive JDBC driver 4.0
    • [KYUUBI #6248] Web SQL Editor supports run SQL snippets
    • [KYUUBI #6216] Support to deny some client ips to make connection
    • [KYUUBI #6283] Improve the message of distribution built without enabling web ui
    • [KYUUBI #6207] Support to retrieve Spark UserDefinedType result
    • [KYUUBI #6205] Backport SPARK-47300: quoteIfNeeded should quote identifier starts with digits
    • [KYUUBI #6215] Improve DropIgnoreNonexistent rule for Spark 3.5
    • [KYUUBI #6211] Check memory offHeap enabled for CustomResourceProfileExec

    Bug Fixes

    • [KYUUBI #6430] livenessProbe uses absolute path based on KYUUBI_HOME
    • [KYUUBI #6428] kyuubi-server/web-ui should not to set a timeout for axios requests
    • [KYUUBI #6422] Merge standard error to output
    • [KYUUBI #6396] Add caching for KerberosAuthentication using ticketCache key
    • [KYUUBI #6410] Check if asyncRequestExecutor is initialized before shutdown
    • [KYUUBI #6394] Fix example SQL in Spark TPC-DS connector docs
    • [KYUUBI #6400] Fix memory leak when using saveToFile
    • [KYUUBI #6344] FlinkProcessBuilder prioritizes user configurations
    • [KYUUBI #6346] Change Kyuubi JDBC Driver default port and make it effective
    • [KYUUBI #6212] Added audit handler shutdown to the shutdown hook
    • [KYUUBI #6370] For saveToFile, checking idx to prevent IndexOutOfBoundsException
    • [KYUUBI #6320] Fix terminated application pods not deleted issue when kyuubi server restarted
    • [KYUUBI #6291] Avoid NPE in MySQLErrPacket
    • [KYUUBI #5767] Fix spark batch conf not convert issue
    • [KYUUBI #6223] Fix Scala interpreter can not access spark.jars issue
    • [KYUUBI #6287] Avoid missing match
    • [KYUUBI #6183] KyuubiConf.getKubernetesConf should set namespace even if the Kubernetes context is empty
    • [KYUUBI #6204] Fix kyuubi session limiter leak when opening session failed
    • [KYUUBI #6182] JDBC engine using the overlay conf as session conf
    • [KYUUBI #6221] Fix parameter replacement issue caused by incorrect sql split
    • [KYUUBI #6228] Dropdown Selection and Parameter Parsing Issues in Kyuubi SQL Editor

    Build, Test and Misc

    • [KYUUBI #6432] Mention support for Spark 4.0 in docs
    • [KYUUBI #6423] Fix Scalatest exclusion tags
    • [KYUUBI #6419] Bump Jetty 9.4.54.v20240208
    • [KYUUBI #6418] Bump Jersey to 2.40
    • [KYUUBI #6416] Generate flattened POM
    • [KYUUBI #6415] Manage Jersey in kyuubi-server module
    • [KYUUBI #6417] Remove scala compile args -Yno-adapted-args
    • [KYUUBI #6382] Bump Delta from 3.1.0 to 3.2.0 for Spark 3.5
    • [KYUUBI #6348] Upgrade iceberg from 1.5.0 to 1.5.2
    • [KYUUBI #6328] Bump Hive 2.3.10
    • [KYUUBI #6373] Avoid NPE in get operation log api
    • [KYUUBI #6377] Fix isCommand check and set min rows threshold for saveToFile
    • [KYUUBI #6357] Bump org.bouncycastle:bcprov-jdk18on from 1.77 to 1.78
    • [KYUUBI #6363] Increase CI concurrency
    • [KYUUBI #6244] Fix test logs upload error
    • [KYUUBI #6294] Simplify Netty and gRPC dependency management
    • [KYUUBI #6252] Upgrade hive-service-rpc 4.0.0
    • [KYUUBI #6268] Specify logDir for RollingFile filePattern
    • [KYUUBI #6267] Remove unused dependency management in POM
    • [KYUUBI #6271] Upgrade kafka-clients from 3.5.1 to 3.5.2
    • [KYUUBI #6260] Clean up and improve comments for spark extensions
    • [KYUUBI #6257] Bump vite from 4.5.2 to 4.5.3 in /kyuubi-server/web-ui
    • [KYUUBI #6243] Add missing Impala alias for JDBC engine
    • [KYUUBI #6234] Fix invalid code-block syntax in docs
    • [KYUUBI #2782] Add missed UNION_TYPE back
    • [KYUUBI #6206] Update comments for job group id of Spark broadcast jobs
    • [KYUUBI #6197] Revise dependency management of Spark authZ plugin
    • [KYUUBI #6196] Add log4j2 template for kyuubi repl commands
    • [KYUUBI #6194] AuthZ shaded should include ranger-plugins-cred
    • [KYUUBI #6188] Bump follow-redirects from 1.15.4 to 1.15.6 in web-ui
    • [KYUUBI #6191] Update docs to mention support of Flink 1.19
    • [KYUUBI #6134] Add profile and CI for Flink 1.19
    • [KYUUBI #6193] Add known_transkations
    • [KYUUBI #6179] Bump iceberg from 1.4.3 to 1.5.0

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Lennon Chin
    • JiaLiangC
    • Senmiao Liu
    • Xiao Liu
    • Cheng Pan
    • PorterZhang2021
    • TakawaAkirayo
    • Zhen Wang
    • Fei Wang
    • Binjie Yang
    • He Zhao
    • Mingliang Zhu
    • amanraj2520
    • beryllw
    • camper42
    • dupen01
    • lllong33
    • mrpengbei
    • sudohainguyen
    • zRains

    Release Notes - 1.7.4

    2024 Apr 26

    Changlogs

    • [KYUUBI #5451] Ignore NoSuchFileException during OperationLog.close()
    • [KYUUBI #6291] Avoid NPE in MySQLErrPacket
    • [KYUUBI #2782] Add missed UNION_TYPE back
    • [KYUUBI #6204] Fix kyuubi session limiter leak when opening session failed
    • [KYUUBI #6061] Delete svn staging dir before checking out
    • [KYUUBI #6047] Free up disk space
    • [KYUUBI #5953] Update NOTICE
    • [KYUUBI #5951] Improve doc for Session Conf Advisor
    • [KYUUBI #5933] Happy New Year 2024
    • [KYUUBI #5922] Enhance YARN app state judgment
    • [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak
    • [KYUUBI #5857] Dependencies CI check when dev/dependencyList changes
    • [KYUUBI #5849] Incorrectly parse JDBC URL while variable includes colon
    • [KYUUBI #5822] Fix the usage msg of KyuubiBeeLine
    • [KYUUBI #5819] Fix rst syntax of missing blank line after code-block
    • [KYUUBI #5785] Fix flaky test - JpsApplicationOperation with spark local mode
    • [KYUUBI #5750] Fix the inaccurate issue of obtaining COLUMN_SIZE in the decimal field jdbc of spark engine
    • [KYUUBI #5754] Fix engine id of the engine probe failures logging messages
    • [KYUUBI #5317] Hive Connector throws NotSerializableException on reading Hive Avro partitioned table
    • [KYUUBI #5720] Fix flaky test in TPCHQuerySuite
    • [KYUUBI #5713] Backport HIVE-27271: Client connection to HS2 fails when transportMode=http, ssl=true, sslTrustStore specified without trustStorePassword in the JDBC URL
    • [KYUUBI #5697] Fix incorrect url reference in code and docs
    • [KYUUBI #5599] Correct the parameter name of fetching op log in RESTful API
    • [KYUUBI #5499]Catch any exception when closing idle session
    • [KYUUBI #5414] KSHC reader should not pollut the global hiveConf instance
    • [KYUUBI #5432] Fix typo in README.md
    • [KYUUBI #5347] Fix java code errors in quick_start_with_jdbc
    • [KYUUBI #5333] Aaptive call fetchLaunchEngineResult
    • [KYUUBI #5308] Ensure release using Java 8

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Shaoyun Chen
    • Chao Chen
    • Flyangz
    • Pengqi Li
    • Bowen Liang
    • Omkar Makhare
    • Drake Youngkun Min
    • Cheng Pan
    • Gianluca Principini
    • Zeyu Wang
    • Kang Wang
    • Fei Wang
    • Zhen Wang
    • Kaifei Yi
    • He Zhao
    • mrtisttt

    Release Notes - 1.8.2

    2024 Apr 26

    Changelogs

    • [KYUUBI #6320] Fix terminated application pods not deleted issue when kyuubi server restarted
    • [KYUUBI #6298] Make ServiceMonitor use selector labels
    • [KYUUBI #6291] Avoid NPE in MySQLErrPacket
    • [KYUUBI #6283] Improve the message of distribution built without enabling web ui
    • [KYUUBI #6287] Avoid missing matche
    • [KYUUBI #6268] Specify logDir for RollingFile filePattern
    • [KYUUBI #6212] Added audit handler shutdown to the shutdown hook
    • [KYUUBI #6271] Upgrade kafka-clients from 3.5.1 to 3.5.2
    • [KYUUBI #5767] Fix spark batch conf not convert issue
    • [KYUUBI #6223] Fix Scala interpreter can not access spark.jars issue
    • [KYUUBI #6234] Fix invalid code-block syntax in docs
    • [KYUUBI #6215] Improve DropIgnoreNonexistent rule for Spark 3.5
    • [KYUUBI #6221] Fix parameter replacement issue caused by incorrect sql split
    • [KYUUBI #2782] Add missed UNION_TYPE back
    • [KYUUBI #6207] Support to retrieve Spark UserDefinedType result
    • [KYUUBI #6206] Update comments for job group id of Spark broadcast jobs
    • [KYUUBI #6204] Fix kyuubi session limiter leak when opening session failed
    • [KYUUBI #6182] JDBC engine using the overlay conf as session conf
    • [KYUUBI #6193] Add known_transkations
    • [KYUUBI #6183] KyuubiConf.getKubernetesConf should set namespace even if the Kubernetes context is empty
    • [KYUUBI #6175] Refactor copy path about SPARK_HOME in docker
    • [KYUUBI #6166] Fix wrong copy path about SPARK_HOME in docker
    • [KYUUBI #6170] Merge JDBC Trino TPC jobs and Style Check jobs in GitHub Actions workflow
    • [KYUUBI #6164] Limit max parallel for each GitHub Actions workflow
    • [KYUUBI #6156] Remove flink. prefix for create session configurations
    • [KYUUBI #6173] Bump maven-download-plugin from 1.7.1 to 1.8.1
    • [KYUUBI #6168] Check if forcedMaxOutputRows is negative
    • [KYUUBI #6167] Drop JDK 11 on GitHub Actions
    • [KYUUBI #6158] Remove confusing variable
    • [KYUUBI #6152] Remove useless variable
    • [KYUUBI #6130] Stop engine immediately after close session for CONNECTION level FlinkSQLEngine
    • [KYUUBI #6147] Use the overlay conf as session conf
    • [KYUUBI #6078] KSHC should handle the commit of the partitioned table as dynamic partition at write path
    • [KYUUBI #6139] Expose env vars and sys props to JUnit tests too
    • [KYUUBI #6126] Filter blank engine home env
    • [KYUUBI #6114] Properly handle KyuubiStatement#getMoreResults(CLOSE_CURRENT_RESULT)
    • [KYUUBI #5902] Bump doc build dependencies
    • [KYUUBI #6081] RESTful client should retry on UnknownHostException
    • [KYUUBI #6071] Add .java-version into git ignore
    • [KYUUBI #6069] TFrontendServiceSuite - close expired operations
    • [KYUUBI #5104] BatchesResourceSuite - expose the metrics with operation type and current state
    • [KYUUBI #6061] Delete svn staging dir before checking out

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Shaoyun Chen
    • Lennon Chin
    • Fei Wang
    • Bowen Liang
    • Xiao Liu
    • Senmiao Liu
    • Tigran Manasyan
    • Cheng Pan
    • TakawaAkirayo
    • Zhen Wang
    • Binjie Yang
    • Vino Yang
    • Kaifei Yi
    • He Zhao
    • amanraj2520
    • dupen01
    • sudohainguyen