Releases Archive

发布记录 - 1.5.1-incubating

2022 4月 21

更改列表

• [KYUUBI #2354] Fix NPE in process builder log capture thread
• [KYUUBI #2296] Fix operation log file handler leak
• [KYUUBI #2266] The default value of frontend.connection.url.use.hostname should be set to true to be consistent with previous versions
• [KYUUBI #2255]The engine state of Spark’s EngineEvent is hardcoded with 0
• [KYUUBI #2008][FOLLOWUP] Support engine type and subdomain in kyuubi-ctl
• [KYUUBI #2156][FOLLOWUP] Fix configuration format in document
• [KYUUBI #2156] Change log to reflect exactly why getting token failed
• [KYUUBI #2134] Respect Spark bundled log4j in extension modules
• [KYUUBI #2150] [DOCS] Fix Getting Started With Kyuubi on Kubernetes

致谢

最后但同样重要的是，如果没有以下贡献者，该版本是不可能发布的：

• Cheng Pan
• Fei Wang
• HYBG
• Nick-0723
• cxzl25
• jiaoqingbo
• ulysses

发布记录 - 1.5.0-incubating

2022 3月 25

重大更新

• 支持 Flink SQL 引擎 (Beta)
• 支持 Trino/ Presto 引擎 (Beta)
• 支持为 Spark SQL 引擎运行 Scala
• 将 Hive 服务 RPC 协议升级到 3.1.2
• 引擎启动过程默认切换到异步
• 将 Session Conf Provider 添加为 Developer API
• 正确性 - 修复 Spark SQL 的浮点精度损失
• 从 log4j1 迁移到 log4j2

更改列表

• [KYUUBI #1936][FOLLOWUP] Send credentials when opening session and wait for completion
• [KYUUBI #2115][1.5] Update license and enhance collect_licenses script
• [KYUUBI #2125] closeSession should avoid sending RPC after openSession fails
• [KYUUBI #2103] [1.5] Revert “[KYUUBI #1948] Upgrade thrift version to 0.16.0”
• [KYUUBI #1866][FOLLOWUP] Add logging of Flink SQL Engine
• [KYUUBI #1866][DOCS] Add flink sql engine quick start
• [KYUUBI #2070][FLINK] Support Flink job submission on yarn-session mode
• [KYUUBI #2089] Add debugging instructions for Flink engine
• [KYUUBI #1948][FOLLOWUP] Relocate fb303 classes
• [KYUUBI #2078] logCaptureThread does not catch sparksubmit exception
• [KYUUBI #1936] Send credentials when opening session and wait for completion
• [KYUUBI #2079] Update kyuubi layer source file to add flink and trino…
• [KYUUBI #2075] Using thread-safe FastDateFormat instead of SimpleDateFormat
• [KYUUBI #2063] Fix engine idle timeout lose efficacy for Flink Engine
• [KYUUBI #1215][DOC] Document incremental collection
• [KYUUBI #2043] Upgrade log4j/2.x/ to 2.17.2
• [KYUUBI #2060] Clear job group for init SQL
• [KYUUBI #2054] [KYUUBI-1819] Support closing Flink SQL engine process
• [KYUUBI #2055] correct the log service name
• [KYUUBI #2047] Support more MySQL JDBC driver versions
• [KYUUBI #2012] rename EventLoggingService and creates an correct directory
• [KYUUBI #2042] [KYUUI #2036] Redirect Issues/PR Notifications to notifications@kyuubi.apache.org
• [KYUUBI #2039] Remove LegacyFlinkOperationSuite
• [KYUUBI #1961] Upgrade hive-service-rpc to 3.1.2
• [KYUUBI #2008] Support engine type and subdomain in kyuubi-ctl
• [KYUUBI #2006] Improve the readability of ServiceControl list/get operations
• [KYUUBI #1999] Update quick_start_with_helm.md
• [KYUUBI #2000][DEPS] Bump Hadoop 3.3.2
• [KYUUBI #2004] Sync contents for CONTRIBUTING & COMMUNITY between web and main repo
• [KYUUBI #1996] Clean up log4j2 conf and enhance surefire plugin conf
• [KYUUBI #1986] Apache Kyuubi(Incubating) Maturity Self Assessment
• [KYUUBI #1988] Tune initialization of vars depend on system level confs
• [KYUUBI #1992] Remove SPARK_HOME predication.
• [KYUUBI #1960] Skip creating sparksession and starting engine if the max initialization time exceeds
• [KYUUBI #1958] Fix flaky test: statement redact for engine tab
• [[KYUUBI #1983] [KYUUBI #1806][FOLLOWUP] Fix LaunchEngine may be closed multiple times
• [KYUUBI #1981] Remove unused artifacts for common and parent module
• [KYUUBI #1972] Fix flaky test: SchedulerPoolSuite: Scheduler pool
• [KYUUBI #1969] Fix race on some service during start and stop phase
• [KYUUBI #1979] Simplify Kyuubi Event
• [KYUUBI #1975] [BUILD] Run TPC-DS test using the default Spark version
• [KYUUBI #1974] Support merge small files in multi insert statement
• [KYUUBI #1963] Make plan only mode skippable for configuable plans
• [KYUUBI #1970] Replace mirror-cn profile with Apache officially suggested dlcdn
• [KYUUBI #1948][FOLLOWUP] Remove TFramedTransport in extension-spark-3-1 and extension-spark-3-2
• [KYUUBI #1965] [DOCS] Use the newly added official logo at asf svn in README
• [KYUUBI #1948] Upgrade thrift version to 0.16.0
• [KYUUBI #1964] [BUILD] Update scalatest.exclude.tags
• [KYUUBI #1883] Support max result rows for Flink queries
• [KYUUBI #1920] Skip the Plan parsing of UseStatement
• [KYUUBI #1944] Upgrade Spark 3.1.3
• [KYUUBI #1957] Skip html comments in merge commit test body from PR desc
• [KYUUBI #1947] Revamp Github Action settings for CI
• [KYUUBI #1950] Remove ambiguous SPARK_HADOOP_VERSION
• [KYUUBI #1949] [INFA] Add homepage for every module in maven central
• [KYUUBI #1946] Close the kyuubi connection on launch engine operation failure
• [KYUUBI #1918] Build and test on Linux ARM64
• [KYUUBI #1945] [BUILD] Remove spark-hadoop from matrix to simplify workflow
• [KYUUBI #1940] [TEST] Fix EngineSecurityAccessorSuite
• [KYUUBI #1876][FOLLOWUP] Update iceberg.name
• [KYUUBI #1937] Build: Use Spark 3.2 for cross version verification
• [KYUUBI #1939] Build: Bump Jacoco maven plugin 0.8.7
• [KYUUBI #1796] Destroy the process after the engine starts
• [KYUUBI #1931] AbstractSession._lastIdleTime should be reset in acquire method
• [KYUUBI #1919] Add more enum values for OperationModes
• [KYUUBI #1930] Export TRINO_ENGINE_HOME in load-kyuubi-env script
• [KYUUBI #1926] Remove useless constants in Constants
• [KYUUBI #1923] Add GetFunctions for trino engine
• [KYUUBI #1906] Add GetColumns for trino engine
• [KYUUBI #1914] Update metric doc
• [KYUUBI #1867] Support PlanOnlyStatement operation like Spark SQL engine
• [KYUUBI #1911] Add GetTypeInfo for trino engine
• [KYUUBI #1876] Set Spark 3.2.1 as default
• [KYUUBI #1536] Add session conf advisor
• [KYUUBI #1899] Bump Iceberg 0.13.1
• [KYUUBI #1887] Add GetTables for trino engine
• [KYUUBI #1909] [DOCS] Remove unnecessary docs link in readme
• [KYUUBI #1886] Add GetSchemas for trino engine
• [KYUUBI #1905] log4j2-defaults.properties is not working since log4j 2 is always intialized by default
• [KYUUBI #1908] Codecov for flink/trino/beeline/jdbc modules
• [KYUUBI #1903] Add GetTableTypes for trino engine
• [KYUUBI #1901] [GA] Fix checkout version
• [KYUUBI #1898] Enhance release guide add more information about how to…
• [KYUUBI #1896] Make Query Details and Failure Reason sortable on the Kyuubi Query Engine page
• [KYUUBI #1811] TPC-DS benchmark expose cli args breakdown and results-dir
• [KYUUBI #1893] Bump Delta 1.0.1
• [KYUUBI #1880] Fixed: forcedMaxOutputRows extension for join
• [KYUUBI #1889] Improve the user experience of the configuration parameters of the document
• [KYUUBI #1885] Add GetCatalogs for trino engine
• [KYUUBI #1821] Add trino ExecuteStatement
• [KYUUBI #1884] Fix compile SQL measurement
• [KYUUBI #1816] Implement KyuubiHistoryServerPlugin
• [KYUUBI #1873] Fix incorrect applicationId
• [KYUUBI #1878] Fix typo in log4j2-test.properties
• [KYUUBI #1838] Clean up query results after query operations finish
• [KYUUBI #1868] Upgrate netty from 4.1.68 to 4.1.73
• [KYUUBI #1872] Make ProcBuilder decouple with YARN
• [KYUUBI #1875] Upgrade codahale.metrics.version to 4.2.8 and its deps
• [KYUUBI #1870] Upgrate k8s client from 5.5.0 to 5.12.1
• [KYUUBI #1871] Upgrade minikube action from 2.4.2 to 2.4.3
• [KYUUBI #1804][FOLLOWUP][GA] Avoid push docker image for forked repos
• [KYUUBI #1844] Add step for generating release notes
• [KYUUBI #1849] Bump Iceberg 0.13.0 and enable Iceberg test with Spark 3.2
• [KYUUBI #1861] Migrate bind-api and activation-api from javax to jakarta
• [KYUUBI #1848] Bump Spark 3.2.1
• [KYUUBI #1769][FOLLOWUP] Exclude log4j 12 deps from spark-hive and fix detect deps change workflow
• [KYUUBI #1769][FOLLOWUP] Revert the dependency change for kyuubi-hive-jdbc(shaded) modules
• [KYUUBI #1846] Add improvement issue template
• [KYUUBI #1858] Rename test log4j2 properties to log4j2-test.properties
• [KYUUBI #1850] Bump Prometheus 0.14.1
• [KYUUBI #1855] Suppress logging to console in scalatest
• [KYUUBI #1847] Bump Flink 1.14.3
• [KYUUBI #1829] Support executing statement async for Flink SQL engine
• [KYUUBI #1852] Fix UT property of log4j2 conf
• [KYUUBI #1769][FOLLOWUP] Some cleanups for log4j properties
• [KYUUBI #1839] Minor refactor OperationUtil
• [KYUUBI #1813] Enable the internal secure authentication between Kyuubi server and engine
• [KYUUBI #1769] [BUILD] Migrate from log4j1 to log4j2
• [KYUUBI #1832] Fixed: forcedMaxOutputRows extension for subquery
• [KYUUBI #1836] [RELEASE] Enhance release guide
• [KYUUBI #1840] Minor fix for general_incubator_vote script
• [KYUUBI #1834] [BUILD] Add support of external mvn with default mvn jvm opts when making distribution
• [KYUUBI #1835] Fix error to helm manifest
• [KYUUBI #1786] Fix the Spark sql engine logger level changed to WARN issue
• [KYUUBI #1685] Support jar and lib start option
• [KYUUBI #1778] Support Flink Set/Reset Operations
• [KYUUBI #1825] Generate appName for Flink applications
• [KYUUBI #1831] [DOCS] Fix the jdbc connection url format doc
• [KYUUBI #1610] Rest frontend service jetty server has started at null
• [KYUUBI #1806] Fix LaunchEngine may be closed multiple times
• [KYUUBI #1698] Add kyuubi server operation state rate
• [KYUUBI #1808] Update the link in the PULL_REQUEST_TEMPLATE
• [KYUUBI #1800] Remove oom hook
• [KYUUBI #1780] Log error message on REST endpoints
• [KYUUBI #1784][FOLLOWUP][TRINO] Fix float types lose precision
• [KYUUBI #1784][FOLLOWUP][FLINK] Support float type
• [KYUUBI #1797] Only exposing jdbc url for server side
• [KYUUBI #1804]Avoid nightly build for forked repos
• [KYUUBI #1764] Reuse the workingDir in ProcBuilder
• [KYUUBI #1699] Improve codahale metric
• [KYUUBI #1784] Fix float types lose precision
• [KYUUBI #1787] Export FLINK_ENGINE_HOME in load-kyuubi-env script
• [KYUUBI #1774] Export FLINK_HOME in load-kyuubi-env script
• [KYUUBI #1775] Add kyuubi.server.name configuration
• [KYUUBI #1785] Stop Discovery services properly
• [KYUUBI #1643][FOLLOWUP] Fix Flink GetFunctions result schema
• [KYUUBI #1788] Update greetings.yml
• [KYUUBI #1776] Improve EnginePage
• [KYUUBI #1648] Implement Flink GetTypeInfo operation
• [KYUUBI #1771] Change rate unit in JsonReporterService
• [KYUUBI #1696] Add fetch logs and results rows rate
• [KYUUBI #1704] Complete Flink-Hive data type mapping
• [KYUUBI #1645] Implement Flink engine GetSchemas operation
• [KYUUBI #1714] Add executeScala api for KyuubiStatement
• [KYUUBI #1756] Clean up code of FlinkOperation
• [KYUUBI #1766] [TEST] Remove SessionManagerSuite to reduce duplicated tests
• [KYUUBI #1763] [INFRA] Remove scala installation inbuild/mvn and using warn level for the ci
• [KYUUBI #1643] Implement GetFunctions operation
• [KYUUBI #1761] [SubTask][THTTP] Make abstract TFrontendService for thrift http protocol support
• [KYUUBI #1757] Extract workingDir from engine specific process builde…
• [KYUUBI #1510] Add TRINO engine type and TrinoProcessBuilder
• [KYUUBI #1748] bin/kyuubi-zk-cli should has execute permission
• [KYUUBI #1759] Remove duplicate close operation api
• [KYUUBI #1731][FEATURE][FOLLOWUP] Use validated HadoopFs URI in HadoopFsDelegationTokenProvider#obtainDelegationTokens
• [KYUUBI #1738] Unify kubernetes integration test
• [KYUUBI #1660] Add restart option for bin/kyuubi
• [KYUUBI #1746] Move sparkEngineMajorMinorVersion into SparkQueryTests to make HiveJDBCTestHelper spark free
• [KYUUBI #1743] Fix parallelism of DataGenerator and other enhancements
• [KYUUBI #1740] Use testcontainers scalatest in trino module
• [KYUUBI #1745] [KYUBBI #1744] Fix trino client flaky test
• [KYUUBI #1737] [WIP][KYUUBI #1731][FEATURE] Kyuubi server should not fail to start when Hadoop FileSystem class not found
• [KYUUBI #1716] Unify Hive deps for server and client
• [KYUUBI #1729] Extract java lint workflow to a single yml and disable it in other jobs
• [KYUUBI #1735] [CI] Add current ip address, long hostname and short hostname to /etc/hosts
• [KYUUBI #1736] Bump up GA actions/checkout to latest (v2.3.5)
• [KYUUBI #1732] Reduce spark driver memory configuration to protect fa…
• [KYUUBI #1730] Use getHostName instead of getCanonicalHostName
• [KYUUBI #1672] Add VersionInfo in REST API
• [KYUUBI #1726] Extract dep workflow to a single yml and disable it in other jobs
• [KYUUBI #1719] javax.activation:activation:jar duplicates
• [KYUUBI #1720] Extract scalastyle workflow to a single yml and disable it in other jobs
• [KYUUBI #1646] support flink get tables
• [KYUUBI #1723] Create greetings.yml
• [KYUUBI #1722] [NOTICE] Update NOTICE for 2022
• [KYUUBI #1721] Recover maven deplop plugin in all module
• [KYUUBI #1709] Introduce a IT module for Kyuubi and FlinkSQL Engine t…
• [KYUUBI #1718] Extract rat workflow to a single yml and disable it in other jobs
• [KYUUBI #1717] Extract rat configurations from pom to file
• [KYUUBI #1715] For kyuubi-beeline, using kyuubi-hive-jdbc instead of kyuubi-hive-jdbc-shaded
• [KYUUBI #1712] Bump up maven.version to 3.8.4
• [KYUUBI #1710] Support to specify OPERATION_LANGUAGE with TExecuteStatementReq confOverlay
• [KYUUBI #1647] Change GetCatalog returned schema key name to support JDBC specification
• [KYUUBI #1690] Checking whether launchEngineOpHandle is null for KyuubiConnection:: getExecLog
• [KYUUBI #1697] BackendService method avg call times and rate
• [KYUUBI #1681] Extend Flink ExecuteStatement to support more operations
• [KYUUBI #1705] Support decimal type for Flink SQL Engine
• [KYUUBI #1691] Implement api: /${version}/operations/${operation_identifier}/rowset
• [KYUUBI #1641] Add Trino client
• [KYUUBI #1647] Implement GetTableTypes operation
• [KYUUBI #1477] Use KyuubiHadoopUtils.newHadoopConf instead of new Con…
• [KYUUBI #1678] Fixed: The value of hasResultSet is always set to true in Kyuubi Server
• [KYUUBI #1679] Remove invalid column and field number check
• [KYUUBI #1673] [BUILD] Improve log4j configurations
• [KYUUBI #1568] [FOLLOWUP] Add getSessionEvent in session
• [KYUUBI #1675][FEATURE] Redact secret information from SparkSQLEngine log
• [KYUUBI #1674] Uncache cached tables when session closed
• [KYUUBI #1654][Bug] Flaky OperationsResourceSuite
• [KYUUBI #1665][FOLLOWUP] Rename FlinkEngineProcessBuilder to FlinkProcessBuilder to align with SparkProcessBuilder
• [KYUUBI #1594] SparkProcessBuild submit add –conf spark.driver.host=
• [KYUUBI #1665] Rename FlinkEngineProcessBuilder to FlinkProcessBuilder to align with SparkProcessBuilder
• [KYUUBI #1656] Fix dist failed with –spark-provided
• [KYUUBI #1658] Revamp swagger ui dependencies
• [KYUUBI #1662] Turn up maven options in mvn script
• [KYUUBI #1597] fix row value in Query Execution should be queryExecution
• [KYUUBI #1649] Determine the yarn command in stop-application.sh.
• [KYUUBI #1634][FEATURE] Redact secret information from SparkSQLEngine UI
• [KYUUBI #1639] Make ApiRequestContext provide KyuubiRestFrontendService directly
• [KYUUBI #1629] Flink backend implementation
• [KYUUBI #1631] Migrating existing rest fe tests to real cases
• [KYUUBI #1556] Upgrade swagger-ui version from 4.1.0 to 4.1.3
• [KYUUBI #1614] Fix hardcoded version in OpenAPIDefinition
• [KYUUBI #1623] KyuubiSessionEvent shall always have Id
• [KYUUBI #1618] Correct hadoop version in RELEASE file
• [KYUUBI #1581] Add SchemaHelper for trino engine
• [KYUUBI #1620] Implement api: /${version}/operations/${operation_identifier}/log
• [KYUUBI #1613] Record the engine’s applicationId in KyuubiSessionEvent
• [KYUUBI #1611] Fix swagger-ui redirect wrong to other service address
• [KYUUBI #1601] Align SparkStatementEvent to KyuubiOperationEvent
• [KYUUBI #1592] [TEST][ICEBERG][DELTA] Introduce row level operation test for data lake format
• [KYUUBI #1582] Use ClientTypeSignature to further analysis of trino column type
• [KYUUBI #1607] [DOC] Add operation and backend service metrics in metrics doc
• [KYUUBI #1579] Implement basic ability of executing statement in Flink engine
• [KYUUBI #1575] Implement api: /${version}/operations/${operation_identifier}/resultsetmetadata
• [KYUUBI #1593] use user set host or ip instead of read hostname from user set
• [KYUUBI #1591] Watchdog support for Spark-3.2
• [KYUUBI #1587] [SUB-TASK][KPIP-2] Bump Flink 1.14.2
• [KYUUBI #1577] Add DropIgnoreNonexistent Rule.
• [KYUUBI #1586] Add time metric on each KyuubiBackendService method
• [KYUUBI #1578] Introduce flink-sql-client dependency to reduce some duplicated code
• [KYUUBI #1499] Introduce DataFrameHolder for cli result fetching
• [KYUUBI #1550] Provide a specific user guide about connecting to kerberized kyuubi
• [KYUUBI #1555][FLINK] Bump flink version to 1.14.0
• [KYUUBI #1554] Add kyuubi.engine.pool.name configuration
• [KYUUBI #1460] Bump Hudi 0.10.0
• [KYUUBI #1559] Bump Jersey 2.34
• [KYUUBI #1568] Replace sessionDetail with sessionEvent
• [KYUUBI #1565] Move time functions to RowSetUtils and move schema to spark package
• [KYUUBI #1567] [Bug] Expose HADOOP_CONF_DIR to Kyuubi beeline to fix “GSS initiate failed” error
• [KYUUBI #1571] [BUILD] Correct build command in build/dist
• [KYUUBI #1558] Bump Jackson 2.12.5
• [KYUUBI #1556] Shorten kyuubi.session.engine.check.interval to one minute
• [KYUUBI #1560] Align all Jackson libs version
• [KYUUBI #1539] Replace operationDetail with KyuubiEvent
• [KYUUBI #1557] [DOC] The TTL Of Kyuubi Engines
• [KYUUBI #1174][FOLLOWUP] change configuration(ha.zookeeper.publish.configs) default value to false
• [KYUUBI #1515] Add RowSet for trino engine
• [KYUUBI #1525][FLINK] Implement GetCatalogs operation and do the further initialization
• [KYUUBI #1548] Get docs version from project.version
• [KYUUBI #1544] Update the mailing list to send announcement
• [KYUUBI #1542] Move FetchIterator into kyuubi-common for multi-engine support
• [KYUUBI #1541] Remove keys sync step for Kyuubi finalize release
• [KYUUBI #1533] Add documentation about Kyuubi docker-image-tool using Spark image as BASE_IMAGE
• [KYUUBI #1535] Remove duplicate hive client version property
• [KYUUBI #1531] Add kyuubi version in engineSpace
• [KYUUBI #1528] Record the kyuubi server ip address in event log
• [KYUUBI #1527] [DOC] Improve High Availability Guide
• [KYUUBI #1516] Implement api: /${version}/operations/${operation_identifier}
• [KYUUBI #1480] Fix bug in docker-image-tool & Provider image way to config SPARK instead of local copy
• [KYUUBI #1393][TASK-2] Add operation number metric
• [KYUUBI #1523] Refine usage about option ‘–namespace’ in kyuubi-ctl
• [KYUUBI #1522] Pass the client ip to engine
• [KYUUBI #1526] [DOC] Update kyuubi_layers.drawio to add driver layer
• [KYUUBI #1521] [BUILD] Migrate to actions/cache@v2
• [KYUUBI #1489] Support operation log for ExecuteScala
• [KYUUBI #1520] [BUILD] Switch to setup-java@v2
• [KYUUBI #1509] Make KYUUBI_WORK_DIR_ROOT as the default root path.
• [KYUUBI #1511] Initialize kyuubi trino engine module
• [KYUUBI #1513] Added support liveness and readiness for kyuubi pod
• [KYUUBI #1501] Introduce operationsResource
• [KYUUBI #1503] Introduce FlinkThriftBinaryFrontendService to do the further initialization
• [KYUUBI #1487] Correct the KEYS link and add script to generate vote and announcement
• [KYUUBI #1496] Support tpcds benchmark
• [KYUUBI #1508] Add Kyuubi version in UI
• [KYUUBI #1504] Support to capture console out/error into interpreter output for ExecuteScala
• [KYUUBI #1505] Move ServiceNodeInfo to a separate file
• [KYUUBI #1492] Initialize kyuubi flink sql engine module
• [KYUUBI #1445] [Kyuubi #1444] Implement api: /${version}/sessions/${session_identifier}/operations/${operation_identifier}
• [KYUUBI #1490] Introduce the basic framework for running scala
• [KYUUBI #1493] Make a reasonable thread pool size for listing file statuses
• [KYUUBI #1379] Make SESSION_ENGINE_LAUNCH_ASYNC true by default
• [KYUUBI #1486] Enable delta test on Spark 3.2 with delta 1.1.0
• [KYUUBI #1484] Add a dependency describe for kyuubi on azure delta-lake doc.
• [KYUUBI #1483] Revamp KyuubiSessionImpl and KyuubiSyncThriftClient
• [KYUUBI #1458] Delta lake table columns won’t show up in DBeaver.
• [KYUUBI #1382] Sync the flink-provided profile in docs and testing suite comments
• [KYUUBI #1481] Add z-order docs
• [KYUUBI #1449] Fix when KyuubiServer initialize fail but not exit
• [KYUUBI #1478] Docker-image-tool.sh awareness of incorrectly configured SPARK_HOME
• [KYUUBI #1472] allow the engine not to self-terminate.
• [KYUUBI #1473] Exit gracefully when engine idle
• [KYUUBI #1471] Improve log if user specify the spark principal and keytab but not same with proxy user
• [KYUUBI #1446] Decouple zookeeper from abstract ServiceDiscovery
• [KYUUBI #1470] Git ignore kyuubi-server metrics
• [KYUUBI #1467] Add The Doc of Intergrate Azure Blob Storage Based Delta Lake with Kyuubi
• [KYUUBI #1469] Add cancel vote template and add release notes into vote template
• [KYUUBI #1462] [DOC] Use relative paths instead of absolute paths
• [KYUUBI #1466] Unify the guava version for kyuubi server and kyuubi beeline
• [KYUUBI #1463] Enabled to preserve symbolic links when copying dist dir
• [KYUUBI #1454] Add Doc About Deploy Kyuubi On Kubernetes
• [KYUUBI #1459] Kyuubi Server Event Test change
• [KYUUBI #1461] [DOC] Remove deprecated dist repo directory for deprecated release candidates
• [KYUUBI #1396][K8S] Add docker image tools and replace Dockerfile
• [KYUUBI #1443] MaxPartitionStrategy support file source based table.
• [KYUUBI #1050] Add KyuubiServerInfoEvent
• [KYUUBI #1439] Add KYUUBI_CTL_JAVA_OPTS to configure java opts of kyuubi-ctl.
• [KYUUBI #1442] Fix encode MySQLHandshakePacket UT

致谢

最后但同样重要的是，如果没有以下贡献者，该版本是不可能发布的：

• Ada Wong
• Brian Yue
• Cheng Pan
• Fei Wang
• Fu Chen
• HYBG-1126
• I521853
• Jeff Zhang
• Kent Yao
• Martin Tzvetanov Grigorov
• Muru Muthusamy
• Paul Lam
• Paul Lin
• Rishi Kumar Ray
• Simon
• SteNicholas
• Tianlin Liao
• Wang Zhen
• fwang12
• hongdongdong
• hwk42
• jiaoqingbo
• jing10.gao
• mask
• minyk
• odone
• rayliu
• senmiaoliu
• shenbing
• simon
• sundapeng
• sunfangbin
• sychen
• ulysses-you
• wForget
• xifeng yang
• yanghua
• zhenjiaguo
• zhichaoleo
• zhouyifan279
• zwangsheng

Apache Kyuubi(Incubating) 1.4.1-incubating 发布记录

2022 1月 31

大家好，

Apache Kyuubi (Incubating) 1.4.1-incubating 现已发布。它是一个维护版本，包含基于 branch-1.4 的稳定性修复。主要修复事件日志的一些问题。

Apache Kyuubi (Incubating) 是一个用于大规模数据处理和分析的分布式多租户 JDBC 服务，构建在 Apache Spark 之上，旨在支持更多引擎（如 Apache Flink）。请参考 Apache Kyuubi (Incubating) 了解更多。

要下载 Apache Kyuubi (Incubating) 1.4.1-incubating，请访问 downloads page.

重大更新

• [KYUUBI #1449] Fix when KyuubiServer initialize fail but not exit

• [KYUUBI #1458] Delta lake table columns won’t show up in DBeaver.

• [KYUUBI #1473] Exit gracefully when engine idle

• [KYUUBI #1567] Expose HADOOP_CONF_DIR to Kyuubi beeline to fix “GSS initiate failed” error

• [KYUUBI #1439] Add KYUUBI_CTL_JAVA_OPTS to configure java opts of kyuubi-ctl

• [KYUUBI #1806] Fix LaunchEngine may be closed multiple times

• [KYUUBI #1731] Kyuubi server should not fail to start when Hadoop FileSystem class not found

• [KYUUBI #1509] Make KYUUBI_WORK_DIR_ROOT as the default root path.

其他

• [KYUUBI #1800] [1.4] Remove oom hook

• [KYUUBI #1784] Fix float types lose precision

• [KYUUBI #1748] bin/kyuubi-zk-cli should has execute permission

• [KYUUBI #1722] [NOTICE] Update NOTICE for 2022

• [KYUUBI #1611] Fix swagger-ui redirect wrong to other service address

致谢

感谢以下贡献者帮助审查并提交代码至 Apache Kyuubi (Incubating) 1.4.1-incubating 版本，排序以字典顺序为准：

Cheng Pan, Fei Wang, Fu Chen, Muru Muthusamy, Wang Zhen, sunfangbin, sychen, ulysses-you, yanghua, zhenjiaguo, zhouyifan279, zwangsheng

Apache Kyuubi(Incubating) 1.4.0-incubating 发布记录

2021 12月 10

大家好，

在这个版本中，Kyuubi 支持最新的 Spark 3.2，Z-Order 优化 sql 查询，无需提供 principal 和 keytab 的长时间运行的 SQL 引擎，支持 mysql 协议前端和 mysql 客户端。 其他主要更新包括 GROUP 引擎共享级别的支持、支持异步打开会话、用户友好的 kyuubi beeline 和 zookeeper 身份验证增强。

要下载 Apache Kyuubi (Incubating) 1.4.0-incubating，请访问 下载页面。

重大更新

使用最新的 Spark 3.2 进行了全面验证

支持Z-Order优化SQL

支持长时间运行的 SQL 引擎，无需提供 principal 和 keytab

支持 GROUP 引擎共享级别

增强 USER 引擎共享级别，支持引擎池

增强 zookeeper 身份验证和 ACL

添加更多 Kyuubi 自定义的函数

对所有数据湖平台 Delta Lake、Iceberg 和 Hudi 进行全面测试

同时支持 KERBEROS 和 PLAIN 认证

支持 MySQL 协议前端和 MySQL 客户端（实验性质）

支持在打开会话期间异步启动查询引擎和用户友好的 Kyuubi Hive JDBC 和 BeeLine

功能性更新

• [KYUUBI #1225] [KYUUBI #1228] Add kyuubi-extension-spark-3-2 module and profile to support latest spark-3.2

• [KYUUBI #939] Add Z-Order extensions to support optimize SQL

• [KYUUBI #913] Support long running SQL engine without principal and keytab provided

• [KYUUBI #1219] Support MySQL Protocol Frontend

• [KYUUBI #1349] Add new conf kyuubi.frontend.protocols

• [KYUUBI #1206] Support GROUP engine share level

• [KYUUBI #1018] Set spark.sql.execution.topKSortFallbackThreshold to 10000

• [KYUUBI #1059] Add Plan Only Operations

• [KYUUBI #1039] Application stop log redirect append to engineLog

• [KYUUBI #1159] Add Session stats on Kyuubi Query Engine Page

• [KYUUBI #1198] Support to incremental collect statement result

• [KYUUBI #1248] Change kyuubi.engine.session.initialize.sql default value to Nil

• [KYUUBI #1286] Increase kyuubi.session.engine.initialize.timeout default to 3min

• [KYUUBI #1323] Introduce EngineType to distinguish multiple engines

• [KYUUBI #1262] Support both KERBEROS and PLAIN authentication at the same time

• [KYUUBI #1160] Support to config operation log root dir both for kyuubi server and engine sides

• [KYUUBI #1204] Enhance zookeeper authentication and ACLs

• [KYUUBI #1039] Application stop log redirect append to engineLog

集成

• [KYUUBI #703] Integrate test with Hudi 0.9.0

• [KYUUBI #1338] Integrate test with Iceberg 0.12.1

Hive JDBC

• [KYUUBI #1131] Rename KyuubiDriver to KyuubiHiveDriver

• [KYUUBI #1346] Support launch query engine asynchronously during opening session

• [KYUUBI #1377] Support to get launch engine log asynchronously with KyuubiConnection

• [KYUUBI #1414] Add kyuubi-hive-beeline module to support KyuubiConnection and KyuubiStatement

其他

• [KYUUBI #1047] Use deployment to run kyuubi server HA on K8s

• [KYUUBI #1223] Remove dead module kyuubi-spark-monitor

• [KYUUBI #1277] Add UDF engine_id

• [KYUUBI #660] Add UDF session_user

致谢

最后但同样重要的是，如果没有以下贡献者，该版本是不可能发布的： AnybodyHome, Brian Yue, Cheng Pan, Fei Wang, Fu Chen, Jagadesh Adireddi, Kent Yao, Min Zhao, Guoqing Yang, h, happytofly, hongdongdong, jiaoqingbo, qiuliang, qphien, senmiaoliu, simon, sunfangbin, sychen, timothy65535, ulysses-you, wFoget, xiebo03, xiongyinke, yanghua, yanyu34946, zhang1002, zhenjiaguo, zhouyifan279, zwangsheng, 张宇翔

Apache Kyuubi(Incubating) 1.3.1-incubating 发布记录

2021 10月 25

大家好，

Apache Kyuubi (Incubating) 1.3.1-incubating 现已发布。这是一个包含基于 branch-1.3 的稳定性修复的维护版本。主要修复事件日志的一些问题。

Apache Kyuubi (Incubating) 是一个用于大规模数据处理和分析的分布式多租户 JDBC 服务，构建在 Apache Spark 之上，旨在支持更多引擎（如 Apache Flink）。请参考 Apache Kyuubi (Incubating) 了解更多。

要下载 Apache Kyuubi (Incubating) 1.3.1-incubating，请访问 下载页面。

重大更新

• [KYUUBI #1190] Enhance error log detection in case lost the exception log

• [KYUUBI #1067] Catch exception if logRoot is not exist

• [KYUUBI #884] [FOLLOWUP] Fix catalog in KyuubiDatabaseMetaData#getTables

• [KYUUBI #1108] Register KyuubiDriver in ServiceLoader

• [KYUUBI #1094] [BUGFIX #1068]Support upload file to HDFS

• [KYUUBI #1034] Engine may deadlock when close operationLog

• [KYUUBI #1057] [KYUUBI#1056] Output error log when currentEngine is None

其他

• [KYUUBI #1158] [INFRA] Update ANNOUNCE mail template

• [KYUUBI #1107] [INFRA] Add release mail templates

• [KYUUBI #1141] Remove redundant unit

• [KYUUBI #1134] [TEST] DataLake suites should clean up withKyuubiConf instead of extraConfigs

• [KYUUBI #1130] [DOCS] Restore Kyuubi logo image size

• [KYUUBI #1125] [FOLLOWUP] Compress and remove duplicated new logo and keep legacy logo

• [KYUUBI #1125] A new version of logo flie, apache information included.

• [KYUUBI #1104] Fix release script finalize_svn

• [KYUUBI #1096] Remove never expired declaration in NOTICE

已知问题

• [KYUUBI #1263] Kyuubi auxiliary UDF failed on Spark Yarn mode

• [KYUUBI #1267] Modify comments for the golden file test suite and fix it

致谢

感谢以下贡献者帮助审查并提交代码至 Apache Kyuubi (Incubating) 1.3.1-incubating 版本，以提交时间排序： qiuliang, hongdd, Yuxiang Zhang, Cheng Pan, happytofly, timothy, cxzl25, Xiduo You, Simon

Apache Kyuubi(Incubating) 1.3.0-incubating 发布记录

2021 9月 14

要下载 Apache Kyuubi (Incubating) 1.3.0-incubating，请访问 下载页面。

重大更新

• 完全验证，支持最新的 Spark 3.0/3.1

• 支持在 JDK8（默认）、JDK11 上构建

• 在 Kyuubi Spark Extensions 中提供增强的 AQE（适用于 Spark 3.1）

• 提供几个辅助 SQL 函数作为 Spark 内置函数的补充

• Spark on Kubernetes 的主要改进，包括文档、集成测试、工具等

• 支持引擎/会话级别可配置的初始化 SQL 语句

• 支持自定义认证实现

• 支持通过 Web UI 停止 Spark 引擎

• 支持单会话模式

• 可观察性的几项改进

• 对数据湖平台 Delta、Iceberg 进行全面测试

• 对 MiniYARNCluster 进行全面测试

功能性

• [KYUUBI #631] Add kyuubi-spark-extensions-3-1 module

• [KYUUBI #657][KYUUBI #659] Provides auxiliary SQL functions as supplement to Spark’s built-in Functions

• [KYUUBI #897] Support stop Spark engine through Web UI

• [KYUUBI #762] Add Kyuubi Hive JDBC Module

• [KYUUBI #926] Support add repartition for OptimizedCreateHiveTableAsSelectCommand

• [KYUUBI #670] KyuubiSQLException support sqlState and errorCode

• [KYUUBI #707] Respect spark.sql.catalogImplementation in engine

• [KYUUBI #641] Support custom authentication

• [KYUUBI #751] Support single session mode

• [1.2][KYUUBI #457] Support configurable initialize sql statement for engine startup

• [KYUUBI #857] Support configurable initialize SQL statement for each session

集成

• [KYUUBI #755] Default build against Spark 3.1

• [KYUUBI #693][KYUUBI #760] Support build against JDK11

• [KYUUBI #768][K8S] Add module shuffle-files-cleaner

• [KYUUBI #844][K8S] Exposing Kyuubi as a NodePort service

• [KYUUBI #859][K8S] Add kubernetes configmap

• [KYUUBI #656][K8S] Document how to deploy Kyuubi engines on Kubernetes

• [KYUUBI #776][K8S] Run kyuubi on Kubernetes - make basic dockerfile and example

• [KYUUBI #444][KYUUBI #854][TEST][K8S] Add minikube for K8s integration test

• [KYUUBI #712][LDAP] Add a new config property authentication.ldap.guidKey

• [KYUUBI #643][TEST] Test Detla 1.0.0 with Spark 3.1

• [KYUUBI #948][TEST] Test Iceberg 0.12.0 with Spark 3.0/3.1

• [KYUUBI #527][TEST][YARN] Test with MiniYARNCluster

可观察性

• [KYUUBI #737] Add monitor module

• [KYUUBI #727] Add the listener for each SQL

• [KYUUBI #767] Add statement event tracking

性能

• [1.2][KYUUBI #629] Refine distributed lock to lock only engine bootstrap

其他

• [KYUUBI #416] Detected Dependency List Change for better dependency management

• [KYUUBI #487][KYUUBI #566] Add TPC-DS DDL test cases

• [KYUUBI #871] Change module name kyuubi-main to kyuubi-server