Releases Archive

    Release Notes - 1.10.3

    2026 Jan 2

    Changelogs

    • [KYUUBI #7245] Fix arrow batch converter error
    • [KYUUBI #7272] Improve build/dependency.sh script
    • [KYUUBI #7226] Support to wait the batch recovery appliction submission to throttle the load on the system
    • [KYUUBI #7266] Upgrade jersey version to 2.47
    • [KYUUBI #7265] Recommend to configure kyuubi.session.local.dir.allowlist for multi-tenant clusters
    • [KYUUBI #7261] Bump extra-enforcer-rules 1.11.0
    • [KYUUBI #7260] Suppress rat-plugin deprecate warnings
    • [KYUUBI #7256] Enable authZ compile support for Spark 4.0 and refactor some test methods
    • [KYUUBI #7248] JDBC engine should cancel the statement when receive cancel operation
    • [KYUUBI #7251] Use RustFS instead of MinIO in Playground
    • [KYUUBI #7241] Transfer engine pool name to lower case
    • [KYUUBI #7237] Replace recommonmark with myst
    • [KYUUBI #7224] [DOC] Fix “duplicate label usage” warnings
    • [KYUUBI #7225] [CI] Skip Python Client unit tests when only documentation changes are made
    • [KYUUBI #7222] [DOC] Fix “document isn’t included in any toctree” warnings
    • [KYUUBI #7223] [CI] Skip running tests for documentation only changes
    • [KYUUBI #7219] Fix endless swagger openapi.json security items
    • [KYUUBI #7214] Fix kubernetes container state
    • [KYUUBI #7171] Fix empty list engine result when etcd is used as the service registry
    • [KYUUBI #7205] Use Apache closer.lua syntax for kyuubi-download to obtain engine archives
    • [KYUUBI #7211] [DOC] Remove unsupported ’logo_only’ and ’extra_navbar’ theme options
    • [KYUUBI #7202] [AUTHZ] make FilteredShowObjectsExec extends V2CommandExec to avoid running executor side task
    • [KYUUBI #7197] Bump Spark 4.0.1
    • [KYUUBI #7196] Bump Maven 3.9.11
    • [KYUUBI #7194] Speed up K8s integration tests in CI
    • [KYUUBI #7153] Share JAAS configuration for Zookeeper client to avoid server OOM
    • [KYUUBI #7163] [SPARK] Check whether engine context stopped in engine terminating checker
    • [KYUUBI #7165] Remove banned action-shellcheck
    • [KYUUBI #7157] Bump form-data from 4.0.0 to 4.0.4 in /kyuubi-server/web-ui
    • [KYUUBI #7162] Bump kafka-clients 3.9.1
    • [KYUUBI #6928] Bump Spark 4.0.0
    • [KYUUBI #7158] Spark engine respects session-level idle timeout threshold
    • [KYUUBI #7151] Bump commons-lang3 3.18.0
    • [KYUUBI #7139] Fix Spark extension rules to support RebalancePartitions
    • [KYUUBI #7135] Fix cannot access /tmp/engine-archives: No such file or directory
    • [KYUUBI #7131] Print cached engine archives
    • [KYUUBI #7109] Ignore the ? in backticks
    • [KYUUBI #7112] Enhance test ‘capture error from spark process builder’ for Spark 4.0
    • [KYUUBI #7113] Skip Hadoop classpath check if flink-shaded-hadoop jar exists in Flink lib directory
    • [KYUUBI #7101] [1.10][FOLLOWUP] Fix code broken
    • [KYUUBI #7110] Fix serverOnlyPrefixConfigKeys is iterator issue
    • [KYUUBI #7076] Update known_translations
    • [KYUUBI #7101] Load the existing pods when initializing kubernetes client to cleanup terminated app pods
    • [KYUUBI #7104] Bump Maven 3.9.10
    • [KYUUBI #7095] Respect terminated app state when building batch info from metadata
    • [KYUUBI #7077] Spark 3.5: Enhance MaxScanStrategy for DSv2
    • [KYUUBI #7073] Retry 3 times on deploying to nexus

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Wenjun Ruan
    • Fei Wang
    • GitHub Bot
    • moelhoussein
    • 1358035421
    • Akira Ajisaka
    • Lennon Chin
    • echo567
    • Ziyi Wu
    • Denis Krivenko
    • cutiechi
    • He Zhao
    • ruanwenjun
    • Cheng Pan

    Release Notes - 1.11.0

    2025 Dec 31

    Highlight

    • Fully verified with Java 8, 11, 17, and 21, on Scala 2.12 and 2.13
    • Fully verified with Apache Spark 3.3 to 4.0, initial support Spark 4.1, support for Spark 3.3 and 3.4 is deprecated.
    • Fully verified with Apache Flink 1.17 to 1.20, support for Flink 1.17 to 1.19 is deprecated.

    Changelogs

    • [KYUUBI #7289] Release requires Java 17 or 21
    • [KYUUBI #7288] Bump Maven 3.9.12
    • [KYUUBI #7284] Upgrade Flink and Spark to latest patched version
    • [KYUUBI #7287] Fix checkstyle workflow
    • [KYUUBI #7245] Fix arrow batch converter error
    • [KYUUBI #7281] Support to enable asynchronous post event
    • [KYUUBI #7277] Add UUID v7 generator
    • [KYUUBI #7275] Kyuubi Spark extension supports Spark 4.1
    • [KYUUBI #7254] Initial support Spark 4.1
    • [KYUUBI #7274] Bump Scala 2.13.17
    • [KYUUBI #7273] Bump Netty 4.2.7
    • [KYUUBI #7259] Bump Netty 4.1.128 gRPC 1.76.2 Protobuf 3.25.8 Guava 33.4.8-jre
    • [KYUUBI #7272] Improve build/dependency.sh script
    • [KYUUBI #6884][FOLLOWUP] Support reassign batch with BatchRestApi
    • [KYUUBI #6884][FOLLOWUP] Fix internal kyuubi instance ping failure
    • [KYUUBI #7226] Support to wait the batch recovery appliction submission to throttle the load on the system
    • [KYUUBI #7186] Introduce RuleFunctionAuthorization for persistent function calls authorization
    • [KYUUBI #7266] Upgrade jersey version to 2.47
    • [KYUUBI #7265] Recommend to configure kyuubi.session.local.dir.allowlist for multi-tenant clusters
    • [KYUUBI #7261] Bump extra-enforcer-rules 1.11.0
    • [KYUUBI #7260] Suppress rat-plugin deprecate warnings
    • [KYUUBI #7257] Bump Jackson 2.20.1 and kubernetes-client 6.14.0
    • [KYUUBI #7256] Enable authZ compile support for Spark 4.0 and refactor some test methods
    • [KYUUBI #7248] JDBC engine should cancel the statement when receive cancel operation
    • [KYUUBI #7251] Use RustFS instead of MinIO in Playground
    • [KYUUBI #7250] Fix JDBCMetadataStore Scala 2.13 compile error
    • [KYUUBI #7241] Transfer engine pool name to lower case
    • [KYUUBI #7237] Replace recommonmark with myst
    • [KYUUBI #7238] [DOC] Replace ‘Getting Started with Hive JDBC’ markdown with rst
    • [KYUUBI #7224] [DOC] Fix “duplicate label usage” warnings
    • [KYUUBI #7225] [CI] Skip Python Client unit tests when only documentation changes are made
    • [KYUUBI #7229] Fix countMetadata if the params is empty
    • [KYUUBI #7209] Bump mysql image to v8 for authZ plugin test suite on iceberg
    • [KYUUBI #7222] [DOC] Fix “document isn’t included in any toctree” warnings
    • [KYUUBI #7221] [DOC] Fix nonexisting document ‘client/advanced/features/engine_pool’
    • [KYUUBI #7223] [CI] Skip running tests for documentation only changes
    • [KYUUBI #7220] Proactive cleanup kubernetes terminated application info
    • [KYUUBI #7219] Fix endless swagger openapi.json security items
    • [KYUUBI #7217] Support customized session protocol version to support binary type
    • [KYUUBI #7215] Make all BatchUtils variables to be final
    • [KYUUBI #7214] Fix kubernetes container state
    • [KYUUBI #7213] Enable test of Spark lineage plugin for Spark 4.0
    • [KYUUBI #7190] Fix Presto SQLAlchemy dialect did not implement get_view_names
    • [KYUUBI #7171] Fix empty list engine result when etcd is used as the service registry
    • [KYUUBI #7205] Use Apache closer.lua syntax for kyuubi-download to obtain engine archives
    • [KYUUBI #7211] [DOC] Remove unsupported ’logo_only’ and ’extra_navbar’ theme options
    • [KYUUBI #7172][FOLLOWUP] kyuubi-spark-lineage-shaded artifactId should include scala.binary.version
    • [KYUUBI #7212] [DOC] Use unordered list for Appendixes page
    • [KYUUBI #7208] Pyhive remove thrift results columns assert
    • [KYUUBI #7202] [AUTHZ] make FilteredShowObjectsExec extends V2CommandExec to avoid running executor side task
    • [KYUUBI #7203] Bump Iceberg 1.10.0
    • [KYUUBI #7156] Support update HBase delegation token for Spark SQL Engine
    • [KYUUBI #7176] Cleanup metadata with batch size and interval
    • [KYUUBI #7198] Remove support for Spark 3.2
    • [KYUUBI #7199] Deprecate support for Flink 1.17 and 1.18
    • [KYUUBI #7183][LINEAGE] Support collect all input tables by the plan
    • [KYUUBI #7197] Bump Spark 4.0.1
    • [KYUUBI #7196] Bump Maven 3.9.11
    • [KYUUBI #7194] Speed up K8s integration tests in CI
    • [KYUUBI #7174] Kyuubi Spark Extention supports Spark 4.0
    • [KYUUBI #7175] Enhance the performance for metadata cleanup by leveraging create_time index
    • [KYUUBI #7172] Shade kyuubi spark lineage module
    • [KYUUBI #7153] Share JAAS configuration for Zookeeper client to avoid server OOM
    • [KYUUBI #7126][LINEAGE] Support merge into syntax in row level catalog
    • [KYUUBI #7168] Adapt PermanentViewMarker introduced by authz plugin in lineage plugin
    • [KYUUBI #7164] Bump Byte Buddy 1.17.6
    • [KYUUBI #7163][SPARK] Check whether engine context stopped in engine terminating checker
    • [KYUUBI #7165] Remove banned action-shellcheck
    • [KYUUBI #7157] Bump form-data from 4.0.0 to 4.0.4 in /kyuubi-server/web-ui
    • [KYUUBI #7162] Bump kafka-clients 3.9.1
    • [KYUUBI #7138] Respect kyuubi.session.engine.spark.initialize.sql set by client in shared engine mode
    • [KYUUBI #7158] Spark engine respects session-level idle timeout threshold
    • [KYUUBI #7160] Use single try-with-resources blocks in docs example
    • [KYUUBI #7144] Remove direct usage of sun.misc.Signal
    • [KYUUBI #7147] Enforce to use Kyuubi’s PlainSASLServer for PLAIN mechanism
    • [KYUUBI #7151] Bump commons-lang3 3.18.0
    • [KYUUBI #7148] Fix spark.kubernetes.file.upload.path permission
    • [KYUUBI #7141] Support to get spark app url with pattern http://{{SPARK_DRIVER_POD_IP}}:{{SPARK_UI_PORT}}
    • [KYUUBI #7139] Fix Spark extension rules to support RebalancePartitions
    • [KYUUBI #7129] Support PARQUET hive table pushdown filter
    • [KYUUBI #7132] Respect kyuubi.session.engine.startup.waitCompletion for wait engine completion
    • [KYUUBI #7135] Fix cannot access /tmp/engine-archives: No such file or directory
    • [KYUUBI #7131] Print cached engine archives
    • [KYUUBI #7118] docs(client): Add comprehensive docs for Engine Pool feature
    • [KYUUBI #7122] Support ORC hive table pushdown filter
    • [KYUUBI #7121] Improve operation timeout management with configurable executors
    • [KYUUBI #6876] Fix hadoopConf for autoCreateFileUploadPath
    • [KYUUBI #7109] Ignore the ? in backticks
    • [KYUUBI #7112] Enhance test ‘capture error from spark process builder’ for Spark 4.0
    • [KYUUBI #7113] Skip Hadoop classpath check if flink-shaded-hadoop jar exists in Flink lib directory
    • [KYUUBI #6924] Upgrade Spark Ranger plugin to 2.6.0
    • [KYUUBI #7100] [#7099] Ranger Support Check Iceberg Alter Table Command & Change Iceberg Test Use JDBC Catalog
    • [KYUUBI #7110] Fix serverOnlyPrefixConfigKeys is iterator issue
    • [KYUUBI #7105] [K8S][HELM] Support additional labels for PrometheusRule
    • [KYUUBI #7076] Update known_translations
    • [KYUUBI #7106] Make response.results.columns optional
    • [KYUUBI #6884] [FEATURE] Support to reassign the batches to alternative kyuubi instance in case kyuubi instance lost
    • [KYUUBI #7101] Load the existing pods when initializing kubernetes client to cleanup terminated app pods
    • [KYUUBI #7103] Bump Delta 4.0.0 and enable Delta tests for Spark 4.0
    • [KYUUBI #7104] Bump Maven 3.9.10
    • [KYUUBI #7098] Add ability to annotate pods and headless service in Kyuubi helm chart
    • [KYUUBI #7072][FOLLOWUP] Fix engine startup permit Grafana panel unit
    • [KYUUBI #7095] Respect terminated app state when building batch info from metadata
    • [KYUUBI #7094] Add serverOnly flag for metrics config items
    • [KYUUBI #7097] [INFRA] Write sorted authors for release contributors
    • [KYUUBI #7093] Log the metadata cleanup count
    • [KYUUBI #7072] Expose metrics of engine startup permit state
    • [KYUUBI #7077] Spark 3.5: Enhance MaxScanStrategy for DSv2
    • [KYUUBI #7068] Iceberg ranger support check branch and tag ddl
    • [KYUUBI #6870][FOLLOWUP] Correct file name of grafana/README.md
    • [KYUUBI #6928] Bump Spark 4.0.0
    • [KYUUBI #7073] Retry 3 times on deploying to nexus
    • [KYUUBI #7063] Bump Kyuubi Shaded 0.5.0
    • [KYUUBI #7065] [#7066] Iceberg Support add partition field check
    • [KYUUBI #6920][FOLLOWUP] Spark SQL engine supports Spark 4.0
    • [KYUUBI #7062] Bump Delta Lake 3.3.1
    • [KYUUBI #7055] Support to filter out server only configs with prefixes
    • [KYUUBI #7051] Fix usage without sslTrustStore of JDBC driver
    • [KYUUBI #7054] Add server only flag for more server/credentials/frontend/metadata configs
    • [KYUUBI #6926] Add SERVER_LOCAL engine share level
    • [KYUUBI #7048] Fix KeyError when parsing unknown Hive type_id in schema inspection
    • [KYUUBI #7033] Treat YARN/Kubernetes application NOT_FOUND as failed to prevent data quality issue
    • [KYUUBI #7028] Persist the kubernetes application terminate state into metastore for app info store fallback
    • [KYUUBI #7046] Bump Dropwizard metrics version to 4.2.30
    • [KYUUBI #7045] Expose jetty metrics
    • [KYUUBI #7044] Bump jetty version to 9.4.57.v20241219
    • [KYUUBI #7043] Support to construct the batch info from metadata directly
    • [KYUUBI #7034][FOLLOWUP] Decouple the kubernetes pod name and app name
    • [KYUUBI #7035] Close the operation by operation manager to prevent operation leak
    • [KYUUBI #7041][FOLLOWUP] Fix build for SparkOnKubernetesTestsSuite
    • [KYUUBI #7041] Fix NPE when getting metadtamanager in KubernetesApplicationOperation
    • [KYUUBI #6828][FOLLOWUP] Fix NPE in KyuubiBaseResultSet::getBigDecimal
    • [KYUUBI #7036] [DOCS] Improve docs for kyuubi-extension-spark-jdbc-dialect
    • [KYUUBI #7034] [KUBERNETES] Prefer to use pod spark-app-name label as application name than pod name
    • [KYUUBI #7025] [KYUUBI #6686][FOLLOWUP] Prefer terminated container app state than terminated pod state
    • [KYUUBI #7032] Remove Umbrella/Subtask issue template
    • [KYUUBI #7026] Audit the kubernetes pod event type and fix DELETE event process logical
    • [KYUUBI #7027] Support to initialize kubernetes clients on kyuubi server startup
    • [KYUUBI #7023] Upgrade kubernetes client version to 6.13.5
    • [KYUUBI #7022] Update announcement mail template to contain download links
    • [KYUUBI #7015] Record the session disconnected info into kyuubi session event
    • [KYUUBI #7017] Using mutable JettyServer uri to prevent batch kyuubi instance mismatch
    • [KYUUBI #7011] Set kyuubi session engine client after opening engine session successfully
    • [KYUUBI #7009] Backport HIVE-26723: Configurable canonical name checking.
    • [KYUUBI #7008] Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1
    • [KYUUBI #7005] [DOC] Remove empty page “Getting Started with Jupyter Lap”
    • [KYUUBI #7004] Include FastXML Jackson into authZ shaded jar
    • [KYUUBI #6979] Support check paimon system producers
    • [KYUUBI #7003] Cut out JNA dependencies for authZ plugin
    • [KYUUBI #7000] Exclude aws-java-sdk-logs from kyuubi-spark-authz-shaded
    • [KYUUBI #6999] Keep JNA in authz-shaded with Scala 2.13
    • [KYUUBI #6998] [TEST] Harness SparkProcessBuilderSuite
    • [KYUUBI #6997] Get the latest batch app info after submit process terminated to prevent batch ERROR due to engine submit timeout
    • [KYUUBI #6990] Add rebalance before InsertIntoHiveDirCommand and InsertIntoDataSourceDirCommand to align with behaviors of hive
    • [KYUUBI #6785] Shutdown the executor service in KubernetesApplicationOperation and prevent NPE
    • [KYUUBI #6984] Fix ValueError when rendering MapType data
    • [KYUUBI #6989] Calculate expected join partitions based on scanned table size
    • [KYUUBI #6988] [INFRA] Foward GitHub discussions to ASF mailing list
    • [KYUUBI #6941] Test Add new Column for paimon
    • [KYUUBI #6951] Test changing column type
    • [KYUUBI #6986] [DOC] Fix multiple Pygments lexer name issues
    • [KYUUBI #6987] [DOC] Fix Unknown target name issues
    • [KYUUBI #6983] Remove support for spark.sql.watchdog.forcedMaxOutputRows
    • [KYUUBI #6981] [DOC] Fix nested lists
    • [KYUUBI #6964] Fix typos in serveral docs
    • [KYUUBI #6973][AUTHZ] Support Paimon DELETE FROM / UPDATE / MERGE INTO commands
    • [KYUUBI #6975] Clean up code for Spark 3.5 extension
    • [KYUUBI #6977] [DOC] Remove empty note block
    • [KYUUBI #6947] Test Dropping Columns
    • [KYUUBI #6972] [KYUUBI 6971]Test Table Query&Batch Time Travel
    • [KYUUBI #6970] [DOC] Fix “nonexisting document” issues
    • [KYUUBI #6969] [DOC] Fix “Title underline too short” issues
    • [KYUUBI #6965] Bump Scala 2.13.16
    • [KYUUBI #6948] Test Changing Column Comment
    • [KYUUBI #6958] Test INSERT TABLE
    • [KYUUBI #6960] Bump log4j version to 2.24.3 to fix ConcurrentModificationException in LoggerContext
    • [KYUUBI #6936] Test RenameTable command
    • [KYUUBI #6949] Test adding column position
    • [KYUUBI #6940] Test Unset Table Properties Command
    • [KYUUBI #6950] Test changing column position
    • [KYUUBI #6939] Bump Spark 3.5.5
    • [KYUUBI #6942] Test Rename Column Name for paimon
    • [KYUUBI #6908] Connection class ssl context object paramater
    • [KYUUBI #6938] [DOC] Refine monitoring docs
    • [KYUUBI #6923] Test Create Partitioned Table for Paimon
    • [KYUUBI #6932] Test ALTER TBLPROPERTIES for Paimon
    • [KYUUBI #6674] Bump Scalafmt to 3.9.x
    • [KYUUBI #6921][AUTHZ] Test CTAS for Paimon
    • [KYUUBI #6925] Only run Paimon authz tests with Scala 2.12
    • [KYUUBI #6920] Spark SQL engine supports Spark 4.0
    • [KYUUBI #6918] Cache client ipAddress in kyuubi jdbc connection
    • [KYUUBI #6914] Bump Iceberg 1.8.0
    • [KYUUBI #6905] PyHive HTTP/HTTPS dialect to use the database name from url
    • [KYUUBI #6912][LINEAGE] Properly handle empty attribute set on mergeRelationColumnLineage
    • [KYUUBI #6916] Bump Delta 3.3.0 for Spark 3.5
    • [KYUUBI #6917] Bump Hudi 1.0.1
    • [KYUUBI #6915] Fix ClickHouse integration tests
    • [KYUUBI #6901] Default policy for spark
    • [KYUUBI #6814] [UI] Bump cross-spawn from 7.0.3 to 7.0.6
    • [KYUUBI #6900] [DOCS] Correct spelling errors in ’large_query_results’ part
    • [KYUUBI #6891] Fix get existing gauge issue
    • [KYUUBI #6875] [K8S][HELM] Add Hadoop configuration files support
    • [KYUUBI #6876] Support rolling spark.kubernetes.file.upload.path
    • [KYUUBI #6883] Using withOauthTokenProvider instead of withOauthToken to support token refresh
    • [KYUUBI #6843] [FOLLOWUP] Fix ‘query-timeout-thread’ thread leak
    • [KYUUBI #6843] Fix ‘query-timeout-thread’ thread leak
    • [KYUUBI #6870] Add more metrics for grafana dashboard
    • [KYUUBI #6866][FOLLOWUP] Prevent register gauge conflicts if both thrift binary SSL and thrift http SSL enabled
    • [KYUUBI #6866] Add metrics for SSL keystore expiration time
    • [KYUUBI #6867] Refactor the grafana dashboard template
    • [KYUUBI #6864][FOLLOWUP] Support to apply instance with existing labels
    • [KYUUBI #6864] Support to return prometheus metrics with instance label
    • [KYUUBI #6861] Configuration guide of structured logging for Kyuubi server
    • [KYUUBI #6862] Spark 3.3: MaxScanStrategy supports DSv2
    • [KYUUBI #6859] Exclude log4j12 from hive engine module classpath
    • [KYUUBI #6615] Make Jetty sending server version in response configurable
    • [KYUUBI #6857] Spark 3.4: MaxScanStrategy supports DSv2
    • [KYUUBI #5834] Add Grafana dashboard template
    • [KYUUBI #6842] Bump Spark 3.5.4
    • [KYUUBI #6848] Fix Gluten CI
    • [KYUUBI #6845] Support additional labels for PodMonitor to be discovered by Prometheus
    • [KYUUBI #6840] Fix PodMonitor pods selection
    • [KYUUBI #6726][FOLLOWUP] Fix compilation on scala-2.13
    • [KYUUBI #6846] Enable gluten CI for kyuubi apache repo
    • [KYUUBI #6839] Add example for service monitor in Helm chart
    • [KYUUBI #6829] Add metrics for batch pending max elapse time
    • [KYUUBI #6836] Ship kafka-clients in binary distribution tarball without compression libs
    • [KYUUBI #6838] [INFRA] Update archive mailing list address
    • [KYUUBI #6837] [INFRA] Disable K8s CI image GHA cache
    • [KYUUBI #6815] JDBC Engine supports Oracle
    • [KYUUBI #6828] Clean up and improve error message for KyuubiBaseResultSet
    • [KYUUBI #6826] Bump log4j from 2.24.1 to 2.24.2
    • [KYUUBI #6722] Fix AppState when Engine connection is terminated
    • [KYUUBI #6521] [K8S][HELM] Implement new configuration approach
    • [KYUUBI #6726] Support trino stage progress
    • [KYUUBI #6818] Upgrade mysql jdbc version to 8.4.0
    • [KYUUBI #6821] Restore the plain PULL_REQUEST_TEMPLATE
    • [KYUUBI #6820] Explicitly disable attach-scaladocs for pure Java modules
    • [KYUUBI #6813] [BUILD] Replace java.version 1.8 with 8
    • [KYUUBI #6807] [BUILD] Bump maven download plugin from 1.8.1 to 1.11.3
    • [KYUUBI #6773] Bump commons-codec from 1.15 to 1.17.1
    • [KYUUBI #6765] [CI] speed up the script for freeing disk space by batching removals
    • [KYUUBI #6806] [K8S][HELM] Headless service renders ports for enabled protocols only
    • [KYUUBI #6804] Bump Iceberg from 1.6.1 to 1.7.0
    • [KYUUBI #6774] Bump log4j from 2.20.0 to 2.24.1
    • [KYUUBI #6808] [BUILD]Replace systemProperties with systemPropertyVariables in maven-surefire-plugin
    • [KYUUBI #6805] [K8S][HELM] Update default Kyuubi version to 1.10.0
    • [KYUUBI #6778] [BUILD] allow frontend plugin not to inherits Maven’s http proxy config
    • [KYUUBI #6786] Skip repeated checks on convert function in TColumnGenerator
    • [KYUUBI #6790] Fix engine cannot exit when gracefully stopped
    • [KYUUBI #6787] Improve the compatibility of queryTimeout in more version clients
    • Normalize local path in Kyuubi server
    • [KYUUBI #7001] Bump nanoid from 3.3.6 to 3.3.11 in /kyuubi-server/web-ui
    • [KYUUBI #6967] Bump axios from 1.7.4 to 1.8.2 in /kyuubi-server/web-ui

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • 1358035421
    • Akira Ajisaka
    • Alex Wojtowicz
    • Bowen Liang
    • Cheng Pan
    • Chenliang Lu
    • cutiechi
    • David Yuan
    • Denis Krivenko
    • echo567
    • eric9204
    • Fei Wang
    • Frank Bertsch
    • gabry-lab
    • GitHub Bot
    • He Zhao
    • Igor Khrol
    • Jan Willem
    • jasonj
    • Joao Amaral
    • John Zhang
    • Kent Yao
    • Lennon Chin
    • moelhoussein
    • naive-zhang
    • namanagraw
    • Octavian Ciubotaru
    • Pat Buxton
    • paul8263
    • Peiyue Liu
    • pionCham
    • Reese Feng
    • ruanwenjun
    • Senmiao Liu
    • shuyouZZ
    • taylor12805
    • Tian Bao
    • wangzhigang1999
    • Wenjun Ruan
    • wenxin-cn
    • Xu Guang Lv
    • Yaguang Jia
    • z1131392774
    • Zhen Wang
    • zhifanggao
    • Ziyi Wu

    Release Notes - 1.10.2

    2025 May 27

    Highlight

    • Support Spark 4.0

    Changelogs

    • [KYUUBI #7055] Support to filter out server only configs with prefixes
    • [KYUUBI #7051] Fix usage without sslTrustStore of JDBC driver
    • [KYUUBI #7054] Add server only flag for more server/credentials/frontend/metadata configs
    • [KYUUBI #7046] Bump dropwizard metrics version to 4.2.30
    • [KYUUBI #7045] Expose jetty metrics
    • [KYUUBI #7044] Bump jetty version to 9.4.57.v20241219
    • [KYUUBI #7035] Close the operation by operation manager to prevent operation leak
    • [KYUUBI #7041] Fix NPE when getting metadtamanager in KubernetesApplicationOperation
    • [KYUUBI #6828] Fix NPE in KyuubiBaseResultSet::getBigDecimal
    • [KYUUBI #7036] Improve docs for kyuubi-extension-spark-jdbc-dialect
    • [KYUUBI #7034] Prefer to use pod spark-app-name label as application name than pod name
    • [KYUUBI #7025] [KYUUBI #6686] Prefer terminated container app state than terminated pod state
    • [KYUUBI #7026] Audit the kubernetes pod event type and fix DELETE event process logical
    • [KYUUBI #7027] Support to initialize kubernetes clients on kyuubi server startup
    • [KYUUBI #7022] Update announcement mail template to contain download links
    • [KYUUBI #7015] Record the session disconnected info into kyuubi session event
    • [KYUUBI #7017] Using mutable JettyServer uri to prevent batch kyuubi instance mismatch
    • [KYUUBI #7011] Set kyuubi session engine client after opening engine session successfully
    • [KYUUBI #7009] Backport HIVE-26723: Configurable canonical name checking.
    • [KYUUBI #7008] Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1
    • [KYUUBI #7004] Include FastXML Jackson into authZ shaded jar
    • [KYUUBI #7003] Cut out JNA dependencies for authZ plugin
    • [KYUUBI #7000] Exclude aws-java-sdk-logs from kyuubi-spark-authz-shaded
    • [KYUUBI #6998] Harness SparkProcessBuilderSuite
    • [KYUUBI #6997] Get the latest batch app info after submit process terminated to prevent batch ERROR due to engine submit timeout
    • [KYUUBI #6785] Shutdown the executor service in KubernetesApplicationOperation and prevent NPE
    • [KYUUBI #6984] Fix ValueError when rendering MapType data
    • [KYUUBI #6965] Bump Scala 2.13.16
    • [KYUUBI #6939] KSHC supports Spark 3.5.5
    • [KYUUBI #6921] Test CTAS for Paimon
    • [KYUUBI #6925] Only run Paimon authz tests with Scala 2.12
    • [KYUUBI #6920] Spark SQL engine supports Spark 4.0
    • [KYUUBI #6918] Cache client ipAddress in kyuubi jdbc connection
    • [KYUUBI #6912] Properly handle empty attribute set on mergeRelationColumnLineage
    • [KYUUBI #6917] Bump Hudi 1.0.1
    • [KYUUBI #6915] Fix ClickHouse integration tests
    • [KYUUBI #6814] Bump cross-spawn from 7.0.3 to 7.0.6
    • [KYUUBI #6900] Correct spelling errors in ’large_query_results’ part
    • [KYUUBI #6891] Fix get existing gauge issue
    • [KYUUBI #6883] Using withOauthTokenProvider instead of withOauthToken to support token refresh
    • [KYUUBI #6843] Fix ‘query-timeout-thread’ thread leak
    • [KYUUBI #6866] Add metrics for SSL keystore expiration time
    • [KYUUBI #6864] Support to return prometheus metrics with instance label
    • [KYUUBI #6862] Spark 3.3: MaxScanStrategy supports DSv2
    • [KYUUBI #6859] Exclude log4j12 from hive engine module classpath
    • [KYUUBI #6857] Spark 3.4: MaxScanStrategy supports DSv2

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Igor Khrol
    • Fei Wang
    • Reese Feng
    • Senmiao Liu
    • David Yuan
    • Xu Guang Lv
    • Zhen Wang
    • Peiyue Liu
    • shuyouZZ
    • GitHub Bot
    • He Zhao
    • Cheng Pan

    Release Notes - 1.9.4

    2025 May 27

    Highlight

    • Support Spark 4.0

    Changelogs

    • [KYUUBI #7073] Retry 3 times on deploying to nexus
    • [KYUUBI #7051] Fix usage without sslTrustStore of JDBC driver
    • [KYUUBI #7035] Close the operation by operation manager to prevent operation leak
    • [KYUUBI #7041] Fix NPE when getting metadtamanager in KubernetesApplicationOperation
    • [KYUUBI #6828] Fix NPE in KyuubiBaseResultSet::getBigDecimal
    • [KYUUBI #7011] Set kyuubi session engine client after opening engine session successfully
    • [KYUUBI #7008] Backport HIVE-27817: Disable ssl hostname verification for 127.0.0.1
    • [KYUUBI #6997] Get the latest batch app info after submit process terminated to prevent batch ERROR due to engine submit timeout
    • [KYUUBI #6785] Shutdown the executor service in KubernetesApplicationOperation and prevent NPE
    • [KYUUBI #6984] Fix ValueError when rendering MapType data
    • [KYUUBI #6939] KSHC supports Spark 3.5.5
    • [KYUUBI #6925] Only run Paimon authz tests with Scala 2.12
    • [KYUUBI #6920] Spark SQL engine supports Spark 4.0
    • [KYUUBI #6912] Properly handle empty attribute set on mergeRelationColumnLineage
    • [KYUUBI #6883] Using withOauthTokenProvider instead of withOauthToken to support token refresh
    • [KYUUBI #6843] Fix ‘query-timeout-thread’ thread leak
    • [KYUUBI #6859] Exclude log4j12 from hive engine module classpath
    • [KYUUBI #6838] Update archive mailing list address
    • [KYUUBI #6837] Disable K8s CI image GHA cache

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Peiyue Liu
    • Reese Feng
    • Senmiao Liu
    • Cheng Pan
    • Fei Wang
    • Igor Khrol
    • Xu Guang Lv

    Release Notes - 1.10.1

    2024 Dec 26

    Changelogs

    • [KYUUBI #6840] Fix PodMonitor pods selection
    • [KYUUBI #6829] Add metrics for batch pending max elapse time
    • [KYUUBI #6836] Ship kafka-clients in binary distribution tarball without compression libs
    • [KYUUBI #6838] [INFRA] Update archive mailing list address
    • [KYUUBI #6837] [INFRA] Disable K8s CI image GHA cache
    • [KYUUBI #6828] Clean up and improve error message for KyuubiBaseResultSet
    • Revert “[KYUUBI #6639] Port HIVE-27815: Support update numModifiedRows”
    • [KYUUBI #6722] Fix AppState when Engine connection is terminated
    • [KYUUBI #6820] Explicitly disable attach-scaladocs for pure Java modules
    • [KYUUBI #6813] [BUILD] Replace java.version 1.8 with 8
    • [KYUUBI #6773] Bump commons-codec from 1.15 to 1.17.1
    • [KYUUBI #6808] [BUILD]Replace systemProperties with systemPropertyVariables in maven-surefire-plugin
    • [KYUUBI #6805] [K8S][HELM] Update default Kyuubi version to 1.10.0
    • [KYUUBI #6778] [BUILD] allow frontend plugin not to inherits Maven’s http proxy config
    • [KYUUBI #6786] Skip repeated checks on convert function in TColumnGenerator
    • [KYUUBI #6790] Fix engine cannot exit when gracefully stopped

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Bowen Liang
    • pionCham
    • Denis Krivenko
    • Fei Wang
    • Zhen Wang
    • Cheng Pan
    • Joao Amaral
    • zhifanggao

    Release Notes - 1.9.3

    2024 Dec 2

    Changelogs

    • [KYUUBI #6828] Clean up and improve error message for KyuubiBaseResultSet
    • [KYUUBI #6741] [CI] update GHA actions/upload-artifact from v3 to v4
    • [KYUUBI #6722] Fix AppState when Engine connection is terminated
    • [KYUUBI #6820] Explicitly disable attach-scaladocs for pure Java modules
    • [KYUUBI #6813] [BUILD] Replace java.version 1.8 with 8
    • [KYUUBI #6790] Fix engine cannot exit when gracefully stopped
    • [KYUUBI #6772] Fix ProcessBuilder to properly handle Java opts as a list
    • [KYUUBI #6757] Pass build args to extra spark engine build command
    • [KYUUBI #6750] [REST] Using ForbiddenException instead of NotAllowedException
    • [KYUUBI #6728] [DOC] update Authz plugin docs of build command with -am option
    • [KYUUBI #6688] [SPARK] Avoid trigger execution when getting result schema
    • [KYUUBI #6638] Authz shaded should include jsr311-api
    • [KYUUBI #6720] K8s pod OOM Killed should be identified as Application failed state
    • [KYUUBI #6709] Fix seq and set equals check bug while check auth type in AuthenticationFilter.initAuthHandlers
    • [KYUUBI #6708] spark-sql-engine’s classes should not be visible in server module
    • [KYUUBI #6699] Bump Spark 4.0.0-preview2
    • [KYUUBI #6648] Support to show launch engine log for thrift http mode
    • [KYUUBI #6698] Exclude transitive deps of paimon
    • [KYUUBI #6680] Format the kyuubi-ctl batch command output
    • [KYUUBI #6668] Fix kyuubi batch state abnormal
    • [KYUUBI #6661] Improve perf for column-based TRowSet generation
    • [KYUUBI #6658] [DOCS] Fixed typo in REST API docs
    • [KYUUBI #6594] Port HIVE-26633: Make thrift client maxMessageSize configurable
    • [KYUUBI #6638] Shade jsr311-api in Authz
    • [KYUUBI #6635] Fix Hive 2.3.10 integration tests
    • [KYUUBI #6583] Support to cancel Spark python operation
    • [KYUUBI #6627] Using concurrent collections for session operation handle set
    • [KYUUBI #6626] Fix operation never expired issue with periodical listOperations api calls
    • [KYUUBI #6620] [K8S][HELM] Update default Kyuubi version to 1.9.2
    • [KYUUBI #6615] Make Jetty sending server version in response configurable
    • [KYUUBI #6609] Bump Spark 3.5.2
    • [KYUUBI #6606] Restore shell compatibility for build/mvn
    • [KYUUBI #6606] build/mvn checks project local downloaded binary ahead
    • [KYUUBI #6600] Correct usage of maven.scaladoc.skip
    • [KYUUBI #6598] Flink engine module supports building with Scala 2.13
    • [KYUUBI #6599] Disable Javadoc/Scaladoc by default
    • [KYUUBI #6596] Fix typos in architecture page
    • [KYUUBI #6591] Support authorization on swagger UI
    • [KYUUBI #6592] Add uploadWorkDir into local dir allow list and do not skip check for resource uploaded use case
    • [KYUUBI #6588] Get effective USER if env var is unavailable
    • [KYUUBI #6571] Bump apache parent pom from 31 to 33
    • [KYUUBI #6586] Bump commons-io to 2.16.1
    • [KYUUBI #6564] Insert into table check the privilege of table
    • [KYUUBI #6549] Correctly handle empty Java options for engines
    • [KYUUBI #6579] Show kyuubi batch app details on app state change
    • [KYUUBI #6364] Bump scala-maven-plugin from 4.8.0 to 4.9.2
    • [KYUUBI #6566] Fix typos in KyuubiConf
    • [KYUUBI #6541] [AUTHZ] Fix DataSourceV2RelationTableExtractor can’t get the ‘database’ attribute if it’s a Paimon plan.
    • [KYUUBI #6550] Delete updateStrategy.rollingUpdate.maxUnavailable because it is alpha feature
    • [KYUUBI #6543] Bump Kyuubi Shaded 0.4.1

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • promising-forever
    • Cheng Pan
    • Bowen Liang
    • lifulong
    • be-hase
    • DenineLu
    • Baokun Han
    • hh-cn
    • LiJie20190102
    • Senmiao Liu
    • Zhen Wang
    • Joao Amaral
    • Fei Wang
    • Mike Wilson
    • Madhukar525722
    • liujiayi771
    • paul8263
    • pionCham

    Release Notes - 1.8.3

    2024 Nov 27

    Changelogs

    [KYUUBI #6417] Remove scala compile args -Yno-adapted-args [KYUUBI #6820] Explicitly disable attach-scaladocs for pure Java modules [KYUUBI #6790] Fix engine cannot exit when gracefully stopped [KYUUBI #6709] Fix seq and set equals check bug while check auth type in AuthenticationFilter.initAuthHandlers [KYUUBI #6708] spark-sql-engine’s classes should not be visible in server module [KYUUBI #6658] Fixed typo in REST API docs [KYUUBI #6606] Restore shell compatibility for build/mvn [KYUUBI #6606] build/mvn checks project local downloaded binary ahead [KYUUBI #6600] Correct usage of maven.scaladoc.skip [KYUUBI #6591] Support authorization on swagger UI [KYUUBI #6592] Add uploadWorkDir into local dir allow list and do not skip check for resource uploaded use case [KYUUBI #6588] Get effective USER if env var is unavailable [KYUUBI #6564] Insert into table check the privilege of table [KYUUBI #6549] Correctly handle empty Java options for engines [KYUUBI #6542] KyuubiBatchService should wait for HTTP server started before picking jobs [KYUUBI #6508] Add the key-value pairs in optimizedConf to session conf [KYUUBI #6466] Bump minikube and kubernetes version of integration test [KYUUBI #6463] Release semaphore immediately after startup process exit [KYUUBI #5957] Flink engine should not load kyuubi-defaults.conf [KYUUBI #6440] Fix casing in kubectl commands for serviceaccount creation [KYUUBI #6430] livenessProbe uses absolute path based on KYUUBI_HOME [KYUUBI #6422] Merge standard error to output [KYUUBI #6410] Check if asyncRequestExecutor is initialized before shutdown [KYUUBI #6394] Fix example SQL in Spark TPC-DS connector docs [KYUUBI #4847] Exclude the alive probe sessions in terminating checker [KYUUBI #6172] Support to interrupt the thrift request immediately after marking the engine not alive [KYUUBI #6373] Avoid NPE in get operation log api [KYUUBI #6366] Improve log message for YARN getApplicationInfoByTag

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Fei Wang
    • Zhen Wang
    • ic4y
    • lifulong
    • pionCham
    • liujiayi771
    • LiJie20190102
    • beryllw
    • huangzhir
    • Fengyu Cao
    • lllong33
    • Cheng Pan
    • Fu Chen
    • Qingbo Jiao

    Release Notes - 1.10.0

    2024 Oct 27

    Highlight

    • Fully verified with Java 8, 11, and 17, on Scala 2.12 and 2.13
    • Fully verified with Apache Spark 3.2 to 3.5, initial support for 4.0-preview, support for Spark 3.2 is deprecated and will be removed in the next release
    • Fully verified with Apache Flink 1.17 to 1.20
    • Support creating batch job with uploading extra resources via REST API
    • Support server choosing strategy for HA mode
    • Introduce Spark JVM quake plugin
    • Periodically cleanup expired temp files and operation logs on server
    • Performance improvement for Ranger access requests deduplication in AuthZ plugin

    Server

    • [KYUUBI #6034] Kyuubi Server HA&ZK get server from serverHosts support more strategy
    • [KYUUBI #6756] [REST] Check max file size of uploaded resource and extra resources in batch creation
    • [KYUUBI #6750] [REST] Using ForbiddenException instead of NotAllowedException
    • [KYUUBI #6731] [REST] Check all required extra resource files uploaded in creating batch request
    • [KYUUBI #6720] K8s pod OOM Killed should be identified as Application failed state
    • [KYUUBI #6681][FOLLOWUP] Log the kill batch request before closeSession and during closeOperation
    • [KYUUBI #6772] Fix ProcessBuilder to properly handle Java opts as a list
    • [KYUUBI #6753] Set hadoop fs delegation token renewer to empty
    • [KYUUBI #6704] Disable periodic gc if set interval to 0
    • [KYUUBI #6680] Format the kyuubi-ctl batch command output
    • [KYUUBI #6668] Fix kyuubi batch state abnormal
    • [KYUUBI #6681] Log the delete batch request in batch operation log
    • [KYUUBI #6669] Track the client ip for internal kyuubi RESTful requests
    • [KYUUBI #6645] Size based eviction for server-side temp files cleanup
    • [KYUUBI #6661] Improve perf for column-based TRowSet generation
    • [KYUUBI #6652] Support to list batches in descending order
    • [KYUUBI #6648] Support to show launch engine log for thrift http mode
    • [KYUUBI #6646] Fix RESTful API NPE when restarting Kyuubi server
    • [KYUUBI #6587] Periodically expire temp files and operation logs on server to avoid memeory leak by Files.deleteOnExit
    • [KYUUBI #6335][FOLLOWUP] Using sessionId for sessionUploadFolderPath
    • [KYUUBI #6627] Using concurrent collections for session operation handle set
    • [KYUUBI #6626] Fix operation never expired issue with periodical listOperations api calls
    • [KYUUBI #6615] Make Jetty sending server version in response configurable
    • [KYUUBI #6585] Support Multiple Configuration Files in FileSessionConfAdvisor
    • [KYUUBI #6592] Add uploadWorkDir into local dir allow list and do not skip check for resource uploaded use case
    • [KYUUBI #6588] Get effective USER if env var is unavailable
    • [KYUUBI #6335] [REST] Support uploading extra resources in creating batch jobs via REST API
    • [KYUUBI #6582] [KYUUBI-6581] Zorder clause syntax does not support special characters
    • [KYUUBI #6554] Delete redundant code related to zorder
    • [KYUUBI #6551] Allow insert zorder when global sort is false and the plan is Repartition or RepartitionByExpression.
    • [KYUUBI #6542] KyuubiBatchService should wait for HTTP server started before picking jobs
    • [KYUUBI #6534] Fix checkEngineConnectionAlive may throw NPE
    • [KYUUBI #6524] Trino engine supports insecure configuration
    • [KYUUBI #6008] RESTful API supports killing engine forcibly
    • [KYUUBI #6508] Add the key-value pairs in optimizedConf to session conf
    • [KYUUBI #6500] Remove service loader descriptor for relocated HMS DelegationTokenIdentifier
    • [KYUUBI #6499] Rewrite some utility methods in Java
    • [KYUUBI #6469] Lazily initialize RecordReaderIterator to avoid driver oom when fetching big result set
    • [KYUUBI #6476] Fix incomplete app events deserialization in SHS
    • [KYUUBI #6302] Call cancelJobGroup immediately after statement execution finished
    • [KYUUBI #4847][FOLLOWUP] Fix engine session never idle issue
    • [KYUUBI #6422] Merge standard error to output
    • [KYUUBI #6423] Fix Scalatest exclusion tags
    • [KYUUBI #6396][FOLLOWUP] Avoid NPE
    • [KYUUBI #6415] Manage Jersey in kyuubi-server module
    • [KYUUBI #6410] Check if asyncRequestExecutor is initialized before shutdown
    • [KYUUBI #6408] Change to Base64 RFC4648 for handle guid and credential
    • [KYUUBI #6400] Fix memory leak when using saveToFile
    • [KYUUBI #6376] Improve closeEngine method names and logs due to closeSession
    • [KYUUBI #6379] Return engine name and engine url within KyuubiSessionEvent/SessionData
    • [KYUUBI #4847][FOLLOWUP] Exclude the alive probe sessions in terminating checker
    • [KYUUBI #6172][TASK][EASY] Support to interrupt the thrift request immediately after marking the engine not alive
    • [KYUUBI #6373] Avoid NPE in get operation log api
    • [KYUUBI #6359] [REST] Return more fields within session/batch data
    • [KYUUBI #6366] Improve log message for YARN getApplicationInfoByTag
    • [KYUUBI #6244][TEST] Fix test logs upload error
    • [KYUUBI #6322] Expose more config items of server internal rest client to users
    • [KYUUBI #6049] Support to filter sessions/operations with session type
    • [KYUUBI #6320] Fix terminated application pods not deleted issue when kyuubi server restarted
    • [KYUUBI #6278] Update DB state when the query fallback to resource manager and the batch app is terminal (#6284)
    • [KYUUBI #6291] Avoid NPE in MySQLErrPacket
    • [KYUUBI #6216] Support to deny some client ips to make connection
    • [KYUUBI #6199] Support to run HiveSQLEngine on kerberized YARN
    • [KYUUBI #6204] Fix kyuubi session limiter leak when opening session failed
    • [KYUUBI #6165] Add some HttpClient options for EngineUIProxyServlet

    Spark Engine

    • [KYUUBI #6688] [SPARK] Avoid trigger execution when getting result schema
    • [KYUUBI #6708] spark-sql-engine’s classes should not be visible in server module
    • [KYUUBI #6699] Bump Spark 4.0.0-preview2
    • [KYUUBI #6698] Exclude transitive deps of paimon
    • [KYUUBI #6686] Ignore Spark pod container state if pod is terminated
    • [KYUUBI #6653] Persist spark pod name conf into batch metadata
    • [KYUUBI #6583] Support to cancel Spark python operation
    • [KYUUBI #6609] Bump Spark 3.5.2
    • [KYUUBI #6601] Bump Delta Lake 4.0.0rc1 and enable Delta Lake test for Spark 4.0
    • [KYUUBI #6574] Skip eagerly execute command in PlanOnly mode of Spark Engine
    • [KYUUBI #6549] Correctly handle empty Java options for engines
    • [KYUUBI #6579] Show kyuubi batch app details on app state change
    • [KYUUBI #6545] Deprecate and remove building support for Spark 3.2
    • [KYUUBI #6546] Update incorrect descriptions in Zorder related configurations
    • [KYUUBI #6531] Fix SPARK-EngineTab stop/gracefulstop not work
    • [KYUUBI #6446] Add tests for Spark saveToFile function
    • [KYUUBI #6518] Support extracting URL for Spark 4 on YARN
    • [KYUUBI #6516] Fix KyuubiSparkUtil.buildURI
    • [KYUUBI #6489] [PYTHON] PyKyuubi get_table_names also supports Spark SQL dialect
    • [KYUUBI #6302][FOLLOWUP] Skip spark job group cancellation on incremental collect mode
    • [KYUUBI #6453] Make KSHC support Spark 4.0 and enable CI for Spark 4.0
    • [KYUUBI #6452] Add cross test for Spark 4.0
    • [KYUUBI #6447] Use static regex Pattern instances in JavaUtils.timeStringAs and JavaUtils.byteStringAs
    • [KYUUBI #6437] Fix Spark engine query result save to HDFS
    • [KYUUBI #6441] Kyuubi Spark TPC-DS/H Connector cross version test
    • [KYUUBI #6247] Make KSHC binary compatible with multiple Spark versions
    • [KYUUBI #6425] Fix tests in spark engine and kyuubi server modules with Spark 4.0
    • [KYUUBI #6424] TPC-H/DS connector support Spark 4.0
    • [KYUUBI #6413] SPARK_HOME detection supports Spark 4
    • [KYUUBI #6404] Fix HiveResult.toHiveString compatibility for Spark 4.0
    • [KYUUBI #6405] Spark engine supports both javax and jakarta ws.rs namespaces
    • [KYUUBI #6399] Spark Kyuubi UI supports both javax and jakarta servlet namespaces
    • Revert “[KYUUBI #6390] Temporarily disable UI Tab for Spark 4.0 and above”
    • [KYUUBI #6390] Temporarily disable UI Tab for Spark 4.0 and above
    • [KYUUBI #6305][FOLLOWUP] Improve package Spark SQL engine both Scala 2.12 and 2.13
    • [KYUUBI #6377] Fix isCommand check and set min rows threshold for saveToFile
    • [KYUUBI #6370] For saveToFile, checking idx to prevent IndexOutOfBoundsException
    • [KYUUBI #6323] Upgrade Spark 3.4.3
    • [KYUUBI #6321] Support to get Spark Kubernetes app URL
    • [KYUUBI #6315] Spark 3.5: MaxScanStrategy supports DSv2
    • [KYUUBI #6297] Package Spark SQL engine both Scala 2.12 and 2.13
    • [KYUUBI #6290] Add custom exception serialization for SparkOperationEvent
    • [KYUUBI #6287] Avoid missing matche
    • [KYUUBI #6250][FOLLOWUP] Simplify code after dropping Spark 3.1
    • [KYUUBI #6250] Drop support for Spark 3.1
    • [KYUUBI #6260] Clean up and improve comments for spark extensions
    • [KYUUBI #5767][FOLLOWUP] Fix spark batch conf not convert issue
    • [KYUUBI #6223] Fix Scala interpreter can not access spark.jars issue
    • [KYUUBI #6215] Improve DropIgnoreNonexistent rule for Spark 3.5
    • [KYUUBI #6211] Check memory offHeap enabled for CustomResourceProfileExec
    • [KYUUBI #6207] Support to retrieve Spark UserDefinedType result
    • [KYUUBI #6205] Backport SPARK-47300: quoteIfNeeded should quote identifier starts with digits
    • [KYUUBI #6206] Update comments for job group id of Spark broadcast jobs
    • [KYUUBI #6180] Kyuubi Spark nightly testing should use JDK 17
    • [KYUUBI #6180] Kyuubi Spark nightly testing should use JDK 17

    Spark AuthZ Plugin

    • [KYUUBI #6754][AUTHZ] Improve the performance of Ranger access requests deduplication
    • [KYUUBI #6638][FOLLOWUP] Authz shaded should include jsr311-api
    • [KYUUBI #6709] Fix seq and set equals check bug while check auth type in AuthenticationFilter.initAuthHandlers
    • [KYUUBI #6666][AUTHZ]Upgrade Ranger plugin to 2.5.0
    • [KYUUBI #5402] Introduce Spark JVM quake plugin
    • [KYUUBI #6619][FOLLOWUP] Do not log auth audit log for thrift http cookie auth
    • [KYUUBI #6640] [AUTHZ] Adapt Derby 10.16 new JDBC driver package name
    • [KYUUBI #6638] Shade jsr311-api in Authz
    • [KYUUBI #6619] Support http bearer authentication for ThriftHttp protocol
    • [KYUUBI #6623] Support bearer token authentication on swagger UI
    • [KYUUBI #6618] Support http bearer token authentication for REST protocol
    • [KYUUBI #6564] Insert into table check the privilege of table
    • [KYUUBI #6541] [AUTHZ] Fix DataSourceV2RelationTableExtractor can’t get the ‘database’ attribute if it’s a Paimon plan.
    • [KYUUBI #6197] Revise dependency management of Spark authZ plugin
    • [KYUUBI #6194] AuthZ shaded should include ranger-plugins-cred

    Other Spark Plugins

    • [KYUUBI #6398] Fix lineage plugin UT for Spark 4.0
    • [KYUUBI #6368] Flink engine supports user impersonation
    • [KYUUBI #6598] Flink engine module supports building with Scala 2.13
    • [KYUUBI #6557] Support Flink 1.20
    • [KYUUBI #6537] Upgrade Flink from 1.19.0 to 1.19.1
    • [KYUUBI #5957] Flink engine should not load kyuubi-defaults.conf
    • [KYUUBI #6353] Catch exception for closing flink internal session
    • [KYUUBI #6367] Flink SQL engine supports RenewDelegationToken
    • [KYUUBI #6344] FlinkProcessBuilder prioritizes user configurations
    • [KYUUBI #6249] Drop support for Flink 1.16.
    • [KYUUBI #6134][FOLLOWUP] Add profile and CI for Flink 1.19

    Trino Engine

    • [KYUUBI #6696] Fix Trino Status Printer to Prevent Thread Leak
    • [KYUUBI #6485] Fix the Presto TABLE NOT FOUND error message that failed to match
    • [KYUUBI #6524] Trino engine supports insecure configuration
    • [KYUUBI #6387] [TRINO] Trino engine improve operation log

    JDBC Engine

    • Revert “[KYUUBI #6526] Kyuubi BeeLine wrongly process JDBC URL that contains --”
    • [KYUUBI #6526] Kyuubi BeeLine wrongly process JDBC URL that contains --
    • [KYUUBI #6339] kyuubi-beeline auto constructs JDBC URL from kyuubi-defaults.conf
    • [KYUUBI #6396] Add caching for KerberosAuthentication using ticketCache key
    • [KYUUBI #6346] Change Kyuubi JDBC Driver default port and make it effective
    • [KYUUBI #6338] Support connecting Kyuubi using Hive JDBC driver 4.0
    • [KYUUBI #6316] Support switching jdbc dialect based on session conf
    • [KYUUBI #6253] Support running JDBC engine on YARN AM
    • [KYUUBI #5374][FOLLOWUP] Fix JDBC ClickHouse TRowSet Generator
    • [KYUUBI #6243] Add missing Impala alias for JDBC engine
    • [KYUUBI #5374] JDBC Engine supports ClickHouse
    • [KYUUBI #6221] Fix parameter replacement issue caused by incorrect sql split
    • [KYUUBI #2782][FOLLOWUP] Add missed UNION_TYPE back
    • [KYUUBI #6182] JDBC engine using the overlay conf as session conf

    Other Engine

    • [KYUUBI #6594] Port HIVE-26633: Make thrift client maxMessageSize configurable
    • [KYUUBI #6639] Port HIVE-27815: Support update numModifiedRows
    • [KYUUBI #6635] Fix Hive 2.3.10 integration tests
    • [KYUUBI #6201] Distinguish the collect mode of different engines

    Python client

    • [KYUUBI #6281][PY] Enable hive test in python client
    • [KYUUBI #6281][PY] Speed up testing with xdist plugin
    • [KYUUBI #6281][PY] Initialize github action for python unit testing
    • [KYUUBI #6567] Fix compatibility of pyhive with setuptools==72.0.0
    • [KYUUBI #6434] Add footnote about pyhive origin

    Web UI

    • [KYUUBI #6727] replace immutable empty list and map in BatchRequest initialization
    • ⬆️ Bump rollup from 3.29.4 to 3.29.5 in /kyuubi-server/web-ui (#6717)
    • [KYUUBI #6712] [UI] Bump pnpm to v9 and nodejs to v20 LTS
    • [KYUUBI #6660][UI] Improve the display of sql editor and query result
    • [KYUUBI #6591] Support authorization on swagger UI
    • [KYUUBI #6463] Release semaphore immediately after startup process exit
    • ⬆️ Bump braces from 3.0.2 to 3.0.3 in /kyuubi-server/web-ui (#6461)
    • [KYUUBI #6428] kyuubi-server/web-ui should not to set a timeout for axios requests
    • [KYUUBI #6299] Support disabling Web UI
    • [KYUUBI #6079] Web UI support Basic authN
    • [KYUUBI #6283] Improve the message of distribution built without enabling web ui
    • [KYUUBI #6248] Web SQL Editor supports run SQL snippets
    • [KYUUBI #6257] Bump vite from 4.5.2 to 4.5.3 in /kyuubi-server/web-ui
    • [KYUUBI #6228] Dropdown Selection and Parameter Parsing Issues in Kyuubi SQL Editor
    • [KYUUBI #6188] Bump follow-redirects from 1.15.4 to 1.15.6 in web-ui

    Documentation

    • [KYUUBI #6752] [DOC] Bump doc build requirements
    • [KYUUBI #6728] [DOC] update Authz plugin docs of build command with -am option
    • ⬆️ Bump get-func-name in /kyuubi-server/web-ui (#6735)
    • [KYUUBI #6734] [DOC] add authentication example in REST API docs
    • ⬆️ Bump micromatch from 4.0.5 to 4.0.8 in /kyuubi-server/web-ui (#6729)
    • [KYUUBI #6719] [DOC] Fix a couple of typos
    • [KYUUBI #6673] [DOC] Fix typos in logging.md
    • [KYUUBI #6671] [DOC] Fix typo in ENGINE SHARE LEVEL docs
    • [KYUUBI #6658] [DOCS] Fixed typo in REST API docs
    • [KYUUBI #6628] [DOCS] Improve docs for GROUP Share Level
    • [KYUUBI #6596] Fix typos in architecture page
    • [KYUUBI #6566] Fix typos in KyuubiConf
    • [KYUUBI #6512] Improve docs for KSHC
    • [KYUUBI #6514] Add link to Github Issues on Contributing Documentation page
    • [KYUUBI #6507] Correct Kyuubi BeeLine help message
    • [KYUUBI #6504] Improve docs for Solution for Large Query Results
    • [KYUUBI #6440] Fix casing in kubectl commands for serviceaccount creation
    • [KYUUBI #6432] Mention support for Spark 4.0 in docs
    • [KYUUBI #6394] Fix example SQL in Spark TPC-DS connector docs
    • [KYUUBI #6242] Remove block cleaner docs
    • [KYUUBI #6234] Fix invalid code-block syntax in docs
    • [KYUUBI #6191] Update docs to mention support of Flink 1.19

    Building, Infrastructure and Dependency

    • [KYUUBI #6775] Bump Github actions action/cache and actions/dependency-review-action v4
    • [KYUUBI #6768] [CI] skip explicitly setting check-latest to false
    • [KYUUBI #6762] [BUILD] cleanup dockerfile warnings
    • [KYUUBI #6763] [CI] Specify JDK 11 for spark engine on kubernetes IT
    • [KYUUBI #6741] [CI] update GHA actions/upload-artifact from v3 to v4
    • [KYUUBI #6739] [CI] Bump runner image from Ubuntu 22.04 to 24.04
    • [KYUUBI #6687] [BUILD] Use MAVEN_ARGS for passing Maven CLI arguments
    • [KYUUBI #6757] Pass build args to extra spark engine build command
    • [KYUUBI #6740] [CI] update GHA docker/build-push-action from v5 to v6 to support build summary
    • [KYUUBI #6755] Upgrade default flink version to 1.20.0
    • [KYUUBI #6742] Bump Jetty from 9.4.54 to 9.4.56
    • [KYUUBI #6743] Bump protobuf from 3.25.4 to 3.25.5
    • [KYUUBI #6744] Bump vite from 4.5.3 to 4.5.5
    • [KYUUBI #6745] Bump testcontainers-scala from 0.41.0 to 0.41.4
    • [KYUUBI #6746] Bump commons-lang3 from 3.13.0 to 3.17.0
    • [KYUUBI #6747] Bump Guava from 32.1.3 to 33.3.1
    • [KYUUBI #6737] Bump vertx-grpc and vertx-core from 4.5.1 to 4.5.3
    • ⬆️ Bump semver in /kyuubi-server/web-ui (#6736)
    • ⬆️ Bump axios from 1.6.0 to 1.7.4 in /kyuubi-server/web-ui (#6617)
    • [KYUUBI #6733] Bump SQLlite from 3.42 to 3.46
    • [KYUUBI #6682] Bump super-linter to v7
    • [KYUUBI #6677] Bump scala-collection-compat to 2.12.0
    • [KYUUBI #6676] Bump Scala to 2.12.19
    • [KYUUBI #6672] Cleanup unused Commons Lang 2 dependency
    • [KYUUBI #6664] [BUILD] Bump Maven from 3.8.8 to 3.9.9
    • [KYUUBI #6659] Bump Iceberg to 1.6.1
    • [KYUUBI #6644] Bump Paimon to 0.8
    • [KYUUBI #5314] Support JDK 21
    • [KYUUBI #6603] Upgrade Scala version from 2.13.8 to 2.13.14
    • [KYUUBI #6620] [K8S][HELM] Update default Kyuubi version to 1.9.2
    • [KYUUBI #6606][FOLLOWUP] Restore shell compatibility for build/mvn
    • [KYUUBI #6584] Upgrade flip-tables from 1.0.2 to 1.1.1
    • [KYUUBI #6593] bin/kyuubi supports kill command
    • [KYUUBI #6607] Bump gRPC 1.65.1 Protobuf 3.25.4 Guava 32.1.3
    • [KYUUBI #6606] build/mvn checks project local downloaded binary ahead
    • [KYUUBI #6600] Correct usage of maven.scaladoc.skip
    • [KYUUBI #6599] Disable Javadoc/Scaladoc by default
    • [KYUUBI #6571] Bump apache parent pom from 31 to 33
    • [KYUUBI #6580] [K8S][HELM] Improve metrics configuration
    • [KYUUBI #6586] Bump commons-io to 2.16.1
    • [KYUUBI #6364] Bump scala-maven-plugin from 4.8.0 to 4.9.2
    • [KYUUBI #6558] Bump Iceberg 1.6.0
    • [KYUUBI #6559] Various improvments for release scripts
    • [KYUUBI #6550] Delete updateStrategy.rollingUpdate.maxUnavailable because it is alpha feature
    • [KYUUBI #6543] Bump Kyuubi Shaded 0.4.1
    • [KYUUBI #6548] Bump enforcer extra mojo rules to 1.8.0 for recognizing higher Java bytecode versions
    • [KYUUBI #6523] Support configuration with --conf options
    • [KYUUBI #6529] Upgrade kubernetes-client from 6.8.1 to 6.13.1
    • [KYUUBI #6528] Bump Netty 4.1.108.Final
    • [KYUUBI #6515] Bumping up dropwizard metrics to 4.2.26
    • [KYUUBI #6506] kyuubi-beeline supports --conf
    • [KYUUBI #6493] [K8S][HELM] Update default Kyuubi version to 1.9.1
    • [KYUUBI #6484] Bump ws from 8.8.1 to 8.17.1 in web-ui
    • [KYUUBI #6466] Bump minikube and kubernetes version of integration test
    • [KYUUBI #6458] Remove commons-logging from binary release
    • [KYUUBI #6457] Strip dependencies from REST client
    • [KYUUBI #6456] Opens jdk.internal.ref
    • [KYUUBI #6445] Normalize extra name for optional Python distribution dependencies
    • [KYUUBI #6427] Extract data lake artifact names as maven properties
    • [KYUUBI #6451] Bump Hudi 0.15.0 and enable Hudi authZ test for Spark 3.5
    • [KYUUBI #6439] kyuubi-util-scala test jar leaked to compile scope
    • [KYUUBI #6430] livenessProbe uses absolute path based on KYUUBI_HOME
    • [KYUUBI #6419] Bump Jetty 9.4.54.v20240208
    • [KYUUBI #6418] Bump Jersey to 2.40
    • [KYUUBI #6416] Generate flattened POM
    • [KYUUBI #6417] Remove scala compile args -Yno-adapted-args
    • [KYUUBI #6397] Align Antlr version with Spark for Spark 4.0
    • [KYUUBI #6392] Support javax.servlet and jakarta.servlet co-exist
    • [KYUUBI #6391] Bump Arrow from 15.0.2 to 16.0.0
    • [KYUUBI #6382] Bump Delta from 3.1.0 to 3.2.0 for Spark 3.5
    • [KYUUBI #6348] Upgrade iceberg from 1.5.0 to 1.5.2
    • [KYUUBI #6328] Bump Hive 2.3.10
    • [KYUUBI #6357] Bump org.bouncycastle:bcprov-jdk18on from 1.77 to 1.78
    • [KYUUBI #6363] [INFRA] Increase CI concurrency
    • [KYUUBI #6294] Simplify Netty and gRPC dependency management
    • [KYUUBI #6312][HELM] Allow attaching annotations to service account
    • [KYUUBI #6293] Upgrade Arrow from 12.0.0 to 15.0.2
    • [KYUUBI #6298][HELM] Make ServiceMonitor use selector labels
    • [KYUUBI #6251] Improve kyuubi-beeline help message
    • [KYUUBI #6252] Upgrade hive-service-rpc 4.0.0
    • [KYUUBI #6266] Kyuubi BeeLine supports JSON output format.
    • [KYUUBI #5686][FOLLOWUP] Rename pyhive to python
    • [KYUUBI #6180][FOLLOWUP] Update the setup JDK name in nightly.yml
    • [KYUUBI #5686] Donate PyHive to Apache Kyuubi
    • [KYUUBI #6268] Specify logDir for RollingFile filePattern
    • [KYUUBI #6212] Added audit handler shutdown to the shutdown hook
    • [KYUUBI #6267] Remove unused dependency management in POM
    • [KYUUBI #6271] Upgrade kafka-clients from 3.5.1 to 3.5.2
    • [KYUUBI #6239] Rename beeline to kyuubi-beeline
    • [KYUUBI #6240] [INFRA] Add issue template for 2024H1 Kyuubi Code Contribution Task
    • [KYUUBI #6185] Audit kyuubi operation state change log
    • [KYUUBI #6196] Add log4j2 template for kyuubi repl commands
    • [KYUUBI #6195] Update Helm Chart and playground to use 1.9.0
    • [KYUUBI #6193] [INFRA] Add known_transkations
    • [KYUUBI #6183] [K8S] KyuubiConf.getKubernetesConf should set namespace even if the Kubernetes context is empty
    • [KYUUBI #6179] Bump iceberg from 1.4.3 to 1.5.0

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Alexander Rashed
    • Binjie Yang
    • Bogdan
    • Bowen Liang
    • Bruce Wong
    • Cheng Pan
    • Fei Wang
    • Fu Chen
    • George314159
    • Harry
    • Lennon Chin
    • Liu Xiao
    • Lucas Resch
    • Mike Wilson
    • PorterZhang2021
    • Shilun Fan
    • SteNicholas
    • Wang, Fei
    • Zhigang
    • amanraj2520
    • artilexx
    • avishnus
    • axiangzheng
    • binjie yang
    • camper42
    • caoyu
    • chengpeiming
    • dependabot[bot]
    • dnskr
    • dupeng
    • felixzh
    • futureltl
    • hang.huang
    • hezhao2
    • huangxiaoping
    • huangzhir
    • ic4y
    • jeanlyn
    • jialiang
    • jiaoqingbo
    • joey.ljy
    • lifulong
    • lijie0203
    • liuxiao
    • lllong33
    • ludejiu
    • madlnu
    • minyk
    • pengbei
    • promising-forever
    • ryosuke-hasebe
    • senmiaoliu
    • tatian
    • taylor.fan
    • vinoyang
    • wangjunbo
    • wankunde
    • wenjie.wang01
    • wforget
    • xorsum
    • yanghua
    • yikaifei
    • zRain
    • zhang_yao
    • zhouyifan279
    • zml1206
    • 吴梓溢
    • 王龙

    Release Notes - 1.9.2

    2024 Jul 23

    Changelogs

    • [KYUUBI #6542] KyuubiBatchService should wait for HTTP server started before picking jobs
    • [KYUUBI #6534] Fix checkEngineConnectionAlive may throw NPE
    • [KYUUBI #6529] Upgrade kubernetes-client from 6.8.1 to 6.13.1
    • [KYUUBI #6531] Fix SPARK-EngineTab stop/gracefulstop not work
    • [KYUUBI #6518] Support extracting URL for Spark 4 on YARN
    • [KYUUBI #6524] Trino engine supports insecure configuration
    • [KYUUBI #6515] Bumping up dropwizard metrics to 4.2.26
    • [KYUUBI #6516] Fix KyuubiSparkUtil.buildURI
    • [KYUUBI #6508] Add the key-value pairs in optimizedConf to session conf
    • [KYUUBI #6512] Improve docs for KSHC
    • [KYUUBI #6507] Correct Kyuubi BeeLine help message
    • [KYUUBI #6500] Remove service loader descriptor for relocated HMS DelegationTokenIdentifier
    • [KYUUBI #6469] Lazily initialize RecordReaderIterator to avoid driver oom when fetching big result set
    • [KYUUBI #6476] Fix incomplete app events deserialization in SHS
    • [KYUUBI #6302] Skip spark job group cancellation on incremental collect mode
    • [KYUUBI #6302] Call cancelJobGroup immediately after statement execution finished
    • [KYUUBI #4847] Fix engine session never idle issue
    • [KYUUBI #6466] Bump minikube and kubernetes version of integration test
    • [KYUUBI #6463] Release semaphore immediately after startup process exit
    • [KYUUBI #6458] Remove commons-logging from binary release
    • [KYUUBI #6457] Strip dependencies from REST client
    • [KYUUBI #5957] Flink engine should not load kyuubi-defaults.conf
    • [KYUUBI #6353] Catch exception for closing flink internal session
    • [KYUUBI #6251] Replace kyuubi-beeline with beeline in beeline help message
    • [KYUUBI #6439] kyuubi-util-scala test jar leaked to compile scope
    • [KYUUBI #6437] Fix Spark engine query result save to HDFS
    • [KYUUBI #6440] Fix casing in kubectl commands for serviceaccount creation
    • [KYUUBI #6441] Kyuubi Spark TPC-DS/H Connector cross version test
    • [KYUUBI #6247] Make KSHC binary compatible with multiple Spark versions
    • [KYUUBI #6493] [K8S][HELM] Update default Kyuubi version to 1.9.1

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Fu Chen
    • Fei Wang
    • Baokun Han
    • Madhukar525722
    • Cheng Pan
    • Zhen Wang
    • Z1Wu
    • Yifan Zhou
    • avishnus
    • camper42
    • felixzh2020
    • huangzhir
    • ic4y
    • jeanlyn
    • jiaoqingbo
    • promising-forever
    • zcx513566

    Release Notes - 1.9.1

    2024 Jun 3

    Highlights

    • Experimental support Spark 4.0-preview
    • Web UI supports Basic Authentication

    Features and Improvements

    • [KYUUBI #6424] TPC-H/DS connector support Spark 4.0
    • [KYUUBI #6413] SPARK_HOME detection supports Spark 4
    • [KYUUBI #6404] HiveResult.toHiveString compatible to Spark 4.0
    • [KYUUBI #6405] Spark engine supports both javax and jakarta ws.rs namespaces
    • [KYUUBI #6399] Spark engine supports both javax and jakarta servlet namespaces
    • [KYUUBI #6392] Support javax.servlet and jakarta.servlet co-exist
    • [KYUUBI #6297] Package Spark SQL engine both Scala 2.12 and 2.13
    • [KYUUBI #4847] Exclude the alive probe sessions in terminating checker
    • [KYUUBI #6172] Support to interrupt the thrift request immediately after marking the engine not alive
    • [KYUUBI #6315] Spark 3.5: MaxScanStrategy supports DSv2
    • [KYUUBI #6049] Support to filter sessions/operations with session type
    • [KYUUBI #6366] Improve log message for YARN getApplicationInfoByTag
    • [KYUUBI #6298] Make ServiceMonitor use selector labels in Helm charts
    • [KYUUBI #6251] Improve kyuubi-beeline help message
    • [KYUUBI #6290] Add custom exception serialization for SparkOperationEvent
    • [KYUUBI #6079] Web UI supports Basic authN
    • [KYUUBI #6338] Support connecting Kyuubi using Hive JDBC driver 4.0
    • [KYUUBI #6248] Web SQL Editor supports run SQL snippets
    • [KYUUBI #6216] Support to deny some client ips to make connection
    • [KYUUBI #6283] Improve the message of distribution built without enabling web ui
    • [KYUUBI #6207] Support to retrieve Spark UserDefinedType result
    • [KYUUBI #6205] Backport SPARK-47300: quoteIfNeeded should quote identifier starts with digits
    • [KYUUBI #6215] Improve DropIgnoreNonexistent rule for Spark 3.5
    • [KYUUBI #6211] Check memory offHeap enabled for CustomResourceProfileExec

    Bug Fixes

    • [KYUUBI #6430] livenessProbe uses absolute path based on KYUUBI_HOME
    • [KYUUBI #6428] kyuubi-server/web-ui should not to set a timeout for axios requests
    • [KYUUBI #6422] Merge standard error to output
    • [KYUUBI #6396] Add caching for KerberosAuthentication using ticketCache key
    • [KYUUBI #6410] Check if asyncRequestExecutor is initialized before shutdown
    • [KYUUBI #6394] Fix example SQL in Spark TPC-DS connector docs
    • [KYUUBI #6400] Fix memory leak when using saveToFile
    • [KYUUBI #6344] FlinkProcessBuilder prioritizes user configurations
    • [KYUUBI #6346] Change Kyuubi JDBC Driver default port and make it effective
    • [KYUUBI #6212] Added audit handler shutdown to the shutdown hook
    • [KYUUBI #6370] For saveToFile, checking idx to prevent IndexOutOfBoundsException
    • [KYUUBI #6320] Fix terminated application pods not deleted issue when kyuubi server restarted
    • [KYUUBI #6291] Avoid NPE in MySQLErrPacket
    • [KYUUBI #5767] Fix spark batch conf not convert issue
    • [KYUUBI #6223] Fix Scala interpreter can not access spark.jars issue
    • [KYUUBI #6287] Avoid missing match
    • [KYUUBI #6183] KyuubiConf.getKubernetesConf should set namespace even if the Kubernetes context is empty
    • [KYUUBI #6204] Fix kyuubi session limiter leak when opening session failed
    • [KYUUBI #6182] JDBC engine using the overlay conf as session conf
    • [KYUUBI #6221] Fix parameter replacement issue caused by incorrect sql split
    • [KYUUBI #6228] Dropdown Selection and Parameter Parsing Issues in Kyuubi SQL Editor

    Build, Test and Misc

    • [KYUUBI #6432] Mention support for Spark 4.0 in docs
    • [KYUUBI #6423] Fix Scalatest exclusion tags
    • [KYUUBI #6419] Bump Jetty 9.4.54.v20240208
    • [KYUUBI #6418] Bump Jersey to 2.40
    • [KYUUBI #6416] Generate flattened POM
    • [KYUUBI #6415] Manage Jersey in kyuubi-server module
    • [KYUUBI #6417] Remove scala compile args -Yno-adapted-args
    • [KYUUBI #6382] Bump Delta from 3.1.0 to 3.2.0 for Spark 3.5
    • [KYUUBI #6348] Upgrade iceberg from 1.5.0 to 1.5.2
    • [KYUUBI #6328] Bump Hive 2.3.10
    • [KYUUBI #6373] Avoid NPE in get operation log api
    • [KYUUBI #6377] Fix isCommand check and set min rows threshold for saveToFile
    • [KYUUBI #6357] Bump org.bouncycastle:bcprov-jdk18on from 1.77 to 1.78
    • [KYUUBI #6363] Increase CI concurrency
    • [KYUUBI #6244] Fix test logs upload error
    • [KYUUBI #6294] Simplify Netty and gRPC dependency management
    • [KYUUBI #6252] Upgrade hive-service-rpc 4.0.0
    • [KYUUBI #6268] Specify logDir for RollingFile filePattern
    • [KYUUBI #6267] Remove unused dependency management in POM
    • [KYUUBI #6271] Upgrade kafka-clients from 3.5.1 to 3.5.2
    • [KYUUBI #6260] Clean up and improve comments for spark extensions
    • [KYUUBI #6257] Bump vite from 4.5.2 to 4.5.3 in /kyuubi-server/web-ui
    • [KYUUBI #6243] Add missing Impala alias for JDBC engine
    • [KYUUBI #6234] Fix invalid code-block syntax in docs
    • [KYUUBI #2782] Add missed UNION_TYPE back
    • [KYUUBI #6206] Update comments for job group id of Spark broadcast jobs
    • [KYUUBI #6197] Revise dependency management of Spark authZ plugin
    • [KYUUBI #6196] Add log4j2 template for kyuubi repl commands
    • [KYUUBI #6194] AuthZ shaded should include ranger-plugins-cred
    • [KYUUBI #6188] Bump follow-redirects from 1.15.4 to 1.15.6 in web-ui
    • [KYUUBI #6191] Update docs to mention support of Flink 1.19
    • [KYUUBI #6134] Add profile and CI for Flink 1.19
    • [KYUUBI #6193] Add known_transkations
    • [KYUUBI #6179] Bump iceberg from 1.4.3 to 1.5.0

    Credits

    Last but not least, this release would not have been possible without the following contributors:

    • Lennon Chin
    • JiaLiangC
    • Senmiao Liu
    • Xiao Liu
    • Cheng Pan
    • PorterZhang2021
    • TakawaAkirayo
    • Zhen Wang
    • Fei Wang
    • Binjie Yang
    • He Zhao
    • Mingliang Zhu
    • amanraj2520
    • beryllw
    • camper42
    • dupen01
    • lllong33
    • mrpengbei
    • sudohainguyen
    • zRains