Release Notes - 1.12.0

Highlight

• Fully verified with Java 8, 11, 17, and 21, on Scala 2.12 and 2.13.
• Fully verified with Apache Spark 3.3 to 4.2, support for Spark 3.3 and 3.4 is deprecated.
• Fully verified with Apache Flink 1.17 to 1.20, support for Flink 1.17 to 1.19 is deprecated.
• Introduce Data Agent Engine, with Web UI improvement.
• Introduce Spark YARN connector for reading YARN aggregated logs.
• Remove Chat engine, an experimental feature that never reached production-ready status.
• Remove MySQL frontend protocol, an experimental feature that never got improved.

Server

• [KYUUBI #7387] Replace serverOnly with audience and immutable flags on ConfigEntry
• [KYUUBI #7575] Improve access path validation
• [KYUUBI #7509] Add engine startup time metric
• [KYUUBI #7567] Improve Engine UI proxy 403 response
• [KYUUBI #7542] Validate filenames in Utils
• [KYUUBI #7435] Honor server-level kyuubi.server.info.provider in getInfo
• [KYUUBI #7528] Remove MySQL frontend protocol
• [KYUUBI #7503] Document that kyuubi.session.local.dir.allow.list is only for Spark engine
• [KYUUBI #7488] Fix canceled batch reporting stale RUNNING application state
• [KYUUBI #7443] Fix NullPointerException when SPARK_HOME is misconfigured
• [KYUUBI #7438] Make JSON event logger permission management configurable
• [KYUUBI #7418] Rename kyuubi.engine.security.* to kyuubi.internal.security.* and document them
• [KYUUBI #7430] Remove Chat engine
• [KYUUBI #7407] Free STGroup to avoid OOM Kill
• [KYUUBI #7415] Allow including additional Spark keys as path validation targets
• [KYUUBI #7403] Fix password parsing bug when password contains a colon
• [KYUUBI #7391] Improve session REST API security
• [KYUUBI #7389] Fix parameter binding order in JDBCMetadataStore#transformMetadataState
• [KYUUBI #7374] Fix static zero IV in InternalSecurityAccessor
• [KYUUBI #7348] Separate principal and keytab for ZooKeeper authentication between server and engine
• [KYUUBI #7319] Return 404 for non-existent sessions
• [KYUUBI #7313] Audit pending Kubernetes application status
• [KYUUBI #7290] Prevent engine session leak after Kyuubi session closes
• [KYUUBI #7295] Upgrade playground components to recent versions

Spark Engine, Connectors and Plugins

• [KYUUBI #7551] Optimize InferRebalanceAndSortOrders to skip non-cheap keys and support local sort
• [KYUUBI #7539] Remove rebalance shuffle before writing at AQE time when not worthwhile
• [KYUUBI #7540] Spark Hive Connector supports Spark 4.1
• [KYUUBI #7531] Add advisoryPartitionSizeInBytes config for rebalance
• [KYUUBI #7433] Preserve Hive delegation tokens with non-empty service
• [KYUUBI #7515] Handle Spark variant type in metadata
• [KYUUBI #7495] Avoid overlong Kubernetes pod log directory names
• [KYUUBI #6832] Initial Spark DSv2 YARN connector for reading YARN aggregated logs
• [KYUUBI #6943] Support DPP in HiveScan
• [KYUUBI #7422] Fix FileWriterFactory reusing the same TaskAttemptId across task attempts in Spark Hive Connector
• [KYUUBI #7335] Sort before write in DynamicPartitionDataSingleWriter
• [KYUUBI #7406] Remove redundant table properties when creating tables in Spark Hive Connector
• [KYUUBI #7398] Include SERDEPROPERTIES when creating tables in Spark Hive Connector
• [KYUUBI #7377] Respect catalog-level warehouse when creating databases in Spark Hive Connector
• [KYUUBI #7369] Support PURGE TABLE in Spark Hive Connector
• [KYUUBI #7350] Make TPC-H/DS connectors compatible with Spark 4.2
• [KYUUBI #7304] Fix NumberFormatException when parsing ISO 8601 duration for session idle timeout

Spark AuthZ Plugin

• [KYUUBI #2470] Add Ranger performance tracing to rule authorization
• [KYUUBI #7230] Skip privilege check for DataSourceV2Relation without catalog and identifier
• [KYUUBI #7576] Fix data masking failure on a UNION-ALL view over a masked view
• [KYUUBI #7570] Support Spark 4.2
• [KYUUBI #7568] Eliminate TypeOfPlaceHolder across the whole plan
• [KYUUBI #7552] Support Spark 4.0 and 4.1
• [KYUUBI #7299] Remove redundant concatenation with the list of conditions

Spark Lineage Plugin

• [KYUUBI #7180] Drill down subqueries in the project to resolve lineage relationships

Flink Engine

• [KYUUBI #7310] Support variable substitution in JSON metrics report path

Data Agent Engine

• [KYUUBI #7379] Introduce Data Agent Engine with tool system, agent runtime, OpenAI provider, REST API, SSE streaming, and Web UI

Other Engines

• [KYUUBI #7527] Respect interrupt while waiting for engine launch in JDBC engine
• [KYUUBI #7549] Fix Subject.getSubject() in Kyuubi JDBC driver for JDK 25 compatibility
• [KYUUBI #7305] Route JDBC engine metadata calls through DatabaseMetaData
• [KYUUBI #7317] Fix parsePropertyFromUrl to strip query string from parsed values

Client

• [KYUUBI #7428] Support fetchOrientation parameter in OperationRestApi.getOperationLog

Web UI

• [KYUUBI #7538] Upgrade ESLint to 9 (flat config) and clear remaining dev-dependency advisories
• [KYUUBI #7533] Upgrade web-ui build toolchain and fix dependency security advisories
• [KYUUBI #7504] Add a Batch management page to the Web UI
• [KYUUBI #7483] Restrict Engine UI proxy targets
• [KYUUBI #7386] Replace axios with native fetch in web-ui

Documentation

• [KYUUBI #7513] Remove broken status badges from README
• [KYUUBI #7502] Align Python version and style
• [KYUUBI #7484] Add CLAUDE.md that references AGENTS.md
• [KYUUBI #7445] Add AGENTS.md to guide AI coding agents
• [KYUUBI #7441] Update Apache Paimon references after graduation
• [KYUUBI #7365] Fix documentation links
• [KYUUBI #7344] Bump markdown to 3.8.1
• [KYUUBI #7339] Fix non-consecutive header level increases
• [KYUUBI #7292] Add instructions to archive older releases

Building, Infrastructure and Dependency

• [KYUUBI #7578] Update default Kyuubi version to 1.12.0 in Helm chart
• [KYUUBI #7574] Fix typo in kyuubi-extension-spark-jdbc-dialect pom.xml
• [KYUUBI #7571] Redirect build/mvn fallback message to stderr to avoid corrupting captures
• [KYUUBI #7564] Bump GitHub Actions to latest versions
• [KYUUBI #7561] Cache CI engine archives per URL with a reusable action
• [KYUUBI #7562] Remove Gluten integration test module
• [KYUUBI #7554] Bump Log4j to 2.26.1
• [KYUUBI #7553] Bump Flink to 1.20.5
• [KYUUBI #7548] Bump docker/setup-buildx-action to 4.2.0
• [KYUUBI #7547] Bump docker/build-push-action to 7.3.0
• [KYUUBI #7546] Bump docker/setup-qemu-action to 4.2.0
• [KYUUBI #7545] Bump docker/login-action to 4.4.0
• [KYUUBI #7544] Bump Iceberg to 1.11.0
• [KYUUBI #7543] Bump Spark to 3.5.8, 4.0.3, and 4.1.2
• [KYUUBI #7535] Bump actions/cache to 6.1.0
• [KYUUBI #7536] Bump actions/setup-java to 5.4.0
• [KYUUBI #7537] Bump actions/setup-python to 6.3.0
• [KYUUBI #7526] Bump actions/checkout to 7.0.0
• [KYUUBI #7525] Bump actions/cache to 5.0.5
• [KYUUBI #7524] Bump actions/setup-python to 6.2.0
• [KYUUBI #7523] Bump actions/setup-java to 5.3.0
• [KYUUBI #7517] Bump actions/setup-java to 5.2.0
• [KYUUBI #7508] Bump codecov/codecov-action to 7.0.0
• [KYUUBI #7499] Use ubuntu-slim runner for lightweight workflows
• [KYUUBI #7501] Bump actions/labeler to 6
• [KYUUBI #7500] Bump actions/checkout to 6.0.3
• [KYUUBI #7494] Bump docker/build-push-action to 7.2.0
• [KYUUBI #7491] Bump docker/setup-qemu-action to 4.1.0
• [KYUUBI #7493] Bump docker/login-action to 4.2.0
• [KYUUBI #7492] Bump actions/checkout to 6.0.2
• [KYUUBI #7490] Bump actions/stale to 10.3.0
• [KYUUBI #7485] Run Docs workflow only for documentation changes
• [KYUUBI #7475] Add ASF allowlist check workflow
• [KYUUBI #7480] Bump actions/dependency-review-action to 5
• [KYUUBI #7479] Bump actions/upload-artifact to 7
• [KYUUBI #7478] Bump codecov/codecov-action to 6
• [KYUUBI #7477] Bump actions/setup-java to 5
• [KYUUBI #7481] Bump docker/setup-buildx-action to 4.1.0
• [KYUUBI #7473] Migrate Java tests from JUnit 4 to JUnit 5
• [KYUUBI #7465] Bump docker/build-push-action to 7.1.0
• [KYUUBI #7466] Bump actions/setup-python to 6
• [KYUUBI #7464] Bump actions/stale to 10.2.0
• [KYUUBI #7467] Bump actions/cache to 5
• [KYUUBI #7468] Bump actions/setup-node to 6
• [KYUUBI #7457] Bump Log4j to 2.26.0
• [KYUUBI #7460] Enable Dependabot for GitHub Actions
• [KYUUBI #7461] Set default max-parallel value for all matrices
• [KYUUBI #7446] Upgrade actions/checkout to v6
• [KYUUBI #7440] Update Apache Gluten references after graduation
• [KYUUBI #7444] Upgrade actions/first-interaction to v3
• [KYUUBI #7432] Bump PostgreSQL JDBC driver to 42.7.11
• [KYUUBI #7396] Bump Log4j to 2.25.4
• [KYUUBI #7372] Pin GitHub Actions versions to satisfy ASF policy
• [KYUUBI #7357] Bump Maven to 3.9.14
• [KYUUBI #7356] Bump Hadoop to 3.4.3
• [KYUUBI #7355] Bump Kafka to 3.9.2
• [KYUUBI #7351] Ignore GitHub Copilot and Claude folders in Git
• [KYUUBI #7352] Skip Java 8 tests for Spark engine
• [KYUUBI #7331] Bump Log4j core to 2.25.3
• [KYUUBI #7330] Bump Bouncy Castle bcpkix-jdk18on to 1.79
• [KYUUBI #7327] Bump spark-master profile to 4.2.0-SNAPSHOT
• [KYUUBI #7328] Bump testcontainers-scala to 0.44.1
• [KYUUBI #7318] Allow specifying executionMode when running the TPC-DS benchmark
• [KYUUBI #7303] Bump Spark to 4.1.1
• [KYUUBI #7243] Bump js-yaml to 4.1.1
• [KYUUBI #7291] Update NOTICE files for 2026
• [KYUUBI #7371] Bump brace-expansion to 1.1.13
• [KYUUBI #7342] Bump immutable to 4.3.8
• [KYUUBI #7367] Bump picomatch to 2.3.2
• [KYUUBI #7402] Bump follow-redirects to 1.16.0
• [KYUUBI #7401] Bump pytest to 9.0.3
• [KYUUBI #7411] Bump Bouncy Castle bcprov-jdk18on to 1.84

Credits

Last but not least, this release would not have been possible without the following contributors:

• Akira Ajisaka
• anandnalya
• attilapiros
• Cheng Pan
• Chenliang Lu
• Denis Krivenko
• Eric Liu
• Fei Wang
• GitHub Bot
• Han, Yutong
• Hiroki Egawa
• Hoon Oh
• hutiefang76
• Jiwon Park
• kongjiexin888-source
• maomaodev
• ruanwenjun
• Sunwoo-Shin
• wangzhigang1999
• Xiduo You
• ychris78
• Yuming Wang
• Yutong-1424
• Zhigang Wang