Releases Archive

Release Notes - 1.8.0

2023 Nov 7

Highlights

Fully verified with Java 8, 11, and 17, Scala 2.12, initial support Scala 2.13
Fully verified with Spark 3.1, 3.2, 3.3, and 3.4, initial support Spark 3.5
Fully verified with Flink 1.16, 1.17, and 1.18
New Web UI (experimental)
New Chat engine with OpenAI ChatGPT backend
Major improvements of Spark Batch API, optimized for large-scale job submission (experimental)
Major improvements of Spark AuthZ plugin, including enhanced Iceberg and DSv2 operators support
Major improvements of Flink engine, including YARN Application Mode support
Introducing Atlas hook support in Spark Lineage plugin
Continuously Kyuubi Helm Chart & Spark on K8s improvements

Spark Engine, Connectors and Plugins

[KYUUBI #1265] OPTIMIZE where clause expression support
[KYUUBI #3420] Expose UI url on registering engine service
[KYUUBI #3444] Support the planOnly mode of kyuubi spark engine support SQL lineage
[KYUUBI #3929] Refactor lineage plugin to add LineageDispatcher
[KYUUBI #4167] [AUTHZ] Introduce function support in PrivilegeBuilder with Serde layers
[KYUUBI #4171] Support skip retrieving table’s properties to speed up GetTables operation
[KYUUBI #4186] Spark showProgress with JobInfo
[KYUUBI #4330] Non-temporary views do not resolve to a specific real table
[KYUUBI #4331] [KYUUBI #4431] Lineage supports Union and MergeInto
[KYUUBI #4415] Align the operation handle in server/engine for ExecuteScala, ExecutePython and PlanOnlyStatement
[KYUUBI #4430] Lineage supports the lateral view explode statement
[KYUUBI #4433] Fix the case with create a view from view
[KYUUBI #4443] Do not set engine session init sql for alive probe session
[KYUUBI #4450] Ignore unknown fields policyPriority when reading policy json file
[KYUUBI #4488] [KSHC] Keep object original name defined in HiveBridgeHelper
[KYUUBI #4493] [AUTHZ] Enable Tests for Spark 3.1 with iceberg tables
[KYUUBI #4497] Add RowFilteringTestBase to improve the test coverage
[KYUUBI #4504] [AUTHZ] Fix source table privilege requirement when querying permanent view in Spark 3.1 and below
[KYUUBI #4525] [KSHC] Partitioning predicates should take effect to filter data
[KYUUBI #4530] [AUTHZ] Support non-English chars for MASK, MASK_SHOW_FIRST_4, and MASK_SHOW_FIRST_4
[KYUUBI #4532] [AUTHZ] Displays the columns involved in extracting the aggregation operator
[KYUUBI #4541] Support to customize the attributes to expose for Spark engine
[KYUUBI #4560] [KSHC] Support Kerberized HMS in cluster mode w/o keytab
[KYUUBI #4585] [AUTHZ] Authz policy file generation
[KYUUBI #4587] Take care of table cache query stage
[KYUUBI #4592] Support eagerly kill redundant executors
[KYUUBI #4615] Support stage level schedule for final write stage
[KYUUBI #4617] [AUTHZ] Collect results for filtered show objects ahead to prevent holding unserializable spark plan
[KYUUBI #4620] [KSHC] Cut off transitive dependencies
[KYUUBI #4636] Improve eagerly kill redundant executors
[KYUUBI #4641] Add MaxFileSizeStrategy to limit max scan file size
[KYUUBI #4644] Manually terminate the Py4JServer during engine shutdown
[KYUUBI #4658] [AUTHZ] Fix InsertIntoHiveDirCommand classname so that we can extract the query in it when authorization.
[KYUUBI #4662] [ARROW] Arrow serialization should not introduce extra shuffle for outermost limit
[KYUUBI #4664] Fix empty relation when kill executors
[KYUUBI #4672] [AUTHZ] Remove Support for Apache Ranger 0.6.0
[KYUUBI #4676] [AUTHZ] Reuse users and namespaces in both tests and policy file generation
[KYUUBI #4677] [AUTHZ] Check generated policy file in test suite
[KYUUBI #4678] Improve FinalStageResourceManager kill executors
[KYUUBI #4681] Set thread CreateSparkTimeoutChecker daemon
[KYUUBI #4683] Update spark.sql.finalWriteStage.resourceIsolation.enabled version
[KYUUBI #4702] [ARROW] CommandResultExec should not trigger job
[KYUUBI #4710] [ARROW] LocalTableScanExec should not trigger job
[KYUUBI #4710] [ARROW] Post driver-side metrics for LocalTableScanExec/CommandResultExec
[KYUUBI #4716] [KYUUBI #4715][AUTHZ] Fix the incorrect class name of InsertIntoHiveDirCommand in table spec generator
[KYUUBI #4717] [AUTHZ] Check Authz plugin’s spec json files in UT
[KYUUBI #4720] [ARROW] Fix java.lang.NoSuchFieldError: IpcOption.DEFAULT for Spark-3.1/3.2
[KYUUBI #4741] Kyuubi Spark Engine/TPC connectors support Spark 3.4
[KYUUBI #4754] [ARROW] Use KyuubiArrowConverters#toBatchIterator instead of ArrowConverters#toBatchIterator
[KYUUBI #4797] [ARROW] Reflective calls to the function ArrowUtils#toArrowSchema
[KYUUBI #4812] Generalize case transformation method for string type config entry
[KYUUBI #4834] Reduce the scope of method references in Authz plugin cleanup shutdown hook
[KYUUBI #4838] Fix spark operation exception leak in withLocalProperties method
[KYUUBI #4866] Add annotation for Iceberg tests in Authz plugin
[KYUUBI #4869] [AUTHZ] Introduce table extractor for ResolvedIdentifier in Spark 3.4
[KYUUBI #4871] [AUTHZ] Adapt plan changes for CreateNamespace and SetCatalogAndNamespace in Spark 3.4
[KYUUBI #4873] [AUTHZ] Refactor Authz reflection with kyuubi-util’s DynMethods
[KYUUBI #4874] [AUTHZ] Improve methods in AuthzUtils
[KYUUBI #4875] [AUTHZ] Remove checking Spark v2 in tests since Spark v2 not supported
[KYUUBI #4878] Delta support for Spark-3.4
[KYUUBI #4880] [BUILD] Skip compiling useless files when making distribution
[KYUUBI #4888] [AUTHZ] Remove filtering results for ShowDatabasesCommand in Spark 2.x
[KYUUBI #4892] [AUTHZ] Make identifier part name comparison case insensitive in tests of PrivilegeBuilder
[KYUUBI #4895] [ARROW] Reflective calls to the function ArrowConverters#fromBatchIterator
[KYUUBI #4899] [AUTHZ] Extract function from FunctionIdentifier for CreateFunction and DropFunction in Spark 3.4
[KYUUBI #4903] [AUTHZ] Fix NoSuchElementException when listing database in CatalogImpl in Spark 3.4
[KYUUBI #4909] [AUTHZ] Enable authz plugin tests for Spark 3.4
[KYUUBI #4910] Extract table from ResolvedIdentifier for DropTable in Spark 3.4
[KYUUBI #4913] Eliminate unnecessary output in ut “union an unmasked table”
[KYUUBI #4914] [AUTHZ] Reuse extractor singleton instance with generalized getter for supported extractor types
[KYUUBI #4915] Adapt database not exist error message change for Spark 3.4
[KYUUBI #4916] [AUTHZ] Support ReplaceData and compatible Spark 3.4 and 3.5
[KYUUBI #4916] Fix Iceberg merge into UT
[KYUUBI #4917] [AUTHZ] Table owner undefined in Iceberg 1.3.0 on Spark 3.4
[KYUUBI #4923] [ARROW] Update arguments of ArrowUtils#toArrowSchema function
[KYUUBI #4925] Add default catalog using spark_catalog with the lineage result
[KYUUBI #4935] More than target num of executors may survive after FinalStageResourceManager did kill
[KYUUBI #4937] Cleanup spark catalog shim and renamed to catalog utils
[KYUUBI #4952] Enhance AWS Glue default database does not exist error message on opening session
[KYUUBI #4995] Use hadoop conf and hive conf from catalog options
[KYUUBI #4999] [KSHC] Kyuubi-Spark-Hive-Connector support Apache Spark 3.4
[KYUUBI #5002] Fail the engine fast when no incoming connection in CONNECTION mode
[KYUUBI #5005] Remove default settings spark.sql.execution.topKSortFallbackThreshold
[KYUUBI #5007] Remove blank SparkSimpleStatsReportListener
[KYUUBI #5009] Pass Spark Engine Log Path to Spark Conf
[KYUUBI #5017] [KSHC] Support Parquet/Orc provider is splittable
[KYUUBI #5018] Make kyuubi spark extension compatible with Spark 3.4
[KYUUBI #5022] [KSHC] CreateTable should use the correct provider
[KYUUBI #5023] [KSHC] TableIdentify don’t attach catalog
[KYUUBI #5025] Use formatDuration instead of formatDurationVerbose in Spark UI’s engine tab
[KYUUBI #5028] Update session hadoop conf to catalog hadoop conf
[KYUUBI #5030] Support get query id in Spark engine
[KYUUBI #5035] Spark engine session page display session end time and duration
[KYUUBI #5045] Initialize EXECUTOR_POD_NAME_PREFIX_MAX_LENGTH before using
[KYUUBI #5055] [AUTHZ] Support building function privileges in Spark 3.4
[KYUUBI #5073] Correct the method name in SparkSQLLineageParserHelperSuite
[KYUUBI #5075] Refine tests to adapt Spark 4.0
[KYUUBI #5076] Add KDF engine_url
[KYUUBI #5105] GetColumns gets duplicate column names when Spark integrates with Iceberg
[KYUUBI #5136] Spark App may hang forever if FinalStageResourceManager killed all executors
[KYUUBI #5148] Improve spark.driver.host assignment in Spark on K8s client mode
[KYUUBI #5170] Identifier compatible with Spark 3.4
[KYUUBI #5172] [AUTHZ] Check USE permissions for DESCRIBE FUNCTION
[KYUUBI #5180] Replace deprecated classes by equivalent in scala.tools.nsc.interpreter package
[KYUUBI #5184] Rename Kyuubi’s StageInfo to SparkStageInfo to fix class mismatch
[KYUUBI #5188] Make server module and Spark sql engine module compilable on Scala 2.13
[KYUUBI #5189] [AUTHZ] Make spark authz module compilable on Scala 2.13
[KYUUBI #5191] Make Spark extension plugin compilable on Scala 2.13
[KYUUBI #5192] Make Spark sql lineage plugin compilable on Scala 2.13
[KYUUBI #5193] Make Spark hive connector plugin compilable on Scala 2.13
[KYUUBI #5195] Make Spark TPC-H connector plugin compilable on Scala 2.13
[KYUUBI #5196] Enable CI Test on Scala 2.13 and support custom or spark-core extracted Scala version for Spark’s engine
[KYUUBI #5196] Extract spark core scala version lazily and respect engine env
[KYUUBI #5199] Read all columns of metadata to prevent column missing
[KYUUBI #5202] Switch to Spark 3.4 by default
[KYUUBI #5224] Modify the README file of authz to make spark 3.4 the default version
[KYUUBI #5225] [KSHC] Unify the exception handling of v1 and v2 during dropDatabase
[KYUUBI #5232] In SparkOperation#cleanup always calls cancelJobGroup even though it’s in the completed state
[KYUUBI #5257] [LINEAGE] Adapt CreateTableAsSelect plan to Spark 3.5 changes
[KYUUBI #5268] [AUTHZ] Extract method for enabling authorization in single call mode
[KYUUBI #5269] Duplicated AnalyzesTablesCommand in kyuubi authz
[KYUUBI #5271] AnalyzeTableCommand should also add table write privilege
[KYUUBI #5291] Verify Spark engine is compatible with Spark 3.5.0
[KYUUBI #5317] [KSHC] Hive Connector throws NotSerializableException on reading Hive Avro partitioned table
[KYUUBI #5336] Spark extension supports Spark 3.5
[KYUUBI #5339] Avoid using kyuubi.session.engine.initialize.timeout string
[KYUUBI #5414] [KSHC] Reader should not pollute the global hiveConf instance
[KYUUBI #5426] [KSHC] Avoid use class.newInstance directly
[KYUUBI #5550] Optimizing TPC-DS dataset generation for 10x speedup

Flink Engine

[KYUUBI #1652] Support Flink YARN application mode
[KYUUBI #4367] Support Flink 1.17
[KYUUBI #4387] Remove support for Flink 1.14
[KYUUBI #4446] Fix connections blocked by Flink insert statements
[KYUUBI #4495] Support Flink job management statements
[KYUUBI #4517] Fix multiple executions lead to abnormal results on Flink 1.14
[KYUUBI #4635] Support flink time type in query operation
[KYUUBI #4666] Support flink varbinary type in query operation
[KYUUBI #4745] Support Flink’s LocalZonedTimestamp DataType
[KYUUBI #4806] Support time-out incremental result fetch for Flink engine
[KYUUBI #4861] Support Flink session idleness
[KYUUBI #4938] Implement Kyuubi UDF in Flink engine
[KYUUBI #5080] Fix EmbeddedExecutorFactory not thread-safe during bootstrap
[KYUUBI #5106] Improve logs for fatal errors
[KYUUBI #5108] Fix FileNotFoundException during Flink engine bootstrap
[KYUUBI #5160] Refactor getNextRowSetInternal to support fetch streaming data
[KYUUBI #5190] Explicitly name Flink bootstrap SQL in application mode
[KYUUBI #5200] Optimize Flink application name generating
[KYUUBI #5222] Remove unused provided dependencies in Flink SQL engine
[KYUUBI #5238] Fix credentials may break Flink engine launch command
[KYUUBI #5315] Propagate HIVE_CONF_DIR on launching Flink engine
[KYUUBI #5405] Support Flink 1.18
[KYUUBI #5531] Fix flaky FlinkOperationOnYarnSuite by enlarging the max rows setting

Other Engines

[KYUUBI #3374] Support password authentication for Trino engine
[KYUUBI #4412] Align the server/engine session handle for flink/hive/trino/jdbc engines
[KYUUBI #4491] Fix Trino typo
[KYUUBI #4522] use:catalog should execute before than use:database
[KYUUBI #4544] Initial implement Kyuubi Chat Engine
[KYUUBI #4548] Kyuubi Chat Engine supports Chinese questions and HTTP proxy
[KYUUBI #4554] Code improvement in ChatGPTProvider
[KYUUBI #4556] Refactor ChatGPTProvider to use openai-java client
[KYUUBI #4558] Make ChatGPT model ID configurable
[KYUUBI #4559] Pretty chat engine launch command
[KYUUBI #4711] JDBC client should catch task failed exception instead of NPE in the incremental mode
[KYUUBI #4809] Set session user in ChatGPT request
[KYUUBI #4810] Request and use a single choice for chat completion
[KYUUBI #4881] JDBCEngine performs initialization sql
[KYUUBI #4940] Implement Kyuubi UDF in Hive engine
[KYUUBI #5120] Allow using session’s user and password for connecting database in JDBC engine
[KYUUBI #5282] Support configure Trino session conf in kyuubi-default.conf
[KYUUBI #5333] Adaptive call fetchLaunchEngineResult
[KYUUBI #5570] Fix memory leak when using incremental collect mode in JDBC engine

Server

[KYUUBI #3653] AdminResource add list Kyuubi server api
[KYUUBI #3739] Remove unused parameters in SessionsResource rest docs
[KYUUBI #3872] Skip setting proxy user when keytab is provided on batch session
[KYUUBI #3876] Update the rest api docs for open session response
[KYUUBI #3920] Fix flaky test - distribute lock
[KYUUBI #3957] Rename the config prefix from kyuubi.server.batch.limit to kyuubi.server.limit.batch
[KYUUBI #4325] Support replace preparedStatement for Trino-jdbc
[KYUUBI #4344] Expose exec pool work queue size metrics
[KYUUBI #4364] Add metrics for user opened connections with session type
[KYUUBI #4376] Support to config the Kyuubi service administrator with Kyuubi conf
[KYUUBI #4388] Limit the max rows for get nextRowSet api
[KYUUBI #4404] Support to list/close sessions in AdminResource
[KYUUBI #4407] Adapt SLF4J2
[KYUUBI #4424] Catch No Node Exception, when list kyuubi engines
[KYUUBI #4439] Add list/close operation method for AdminResource
[KYUUBI #4462] Fix variable usage issue in SessionManager#stop
[KYUUBI #4464] Simplify and improve log for JDBCMetadataStore
[KYUUBI #4467] Tolerate Driver Pod ephemerally invisible after submitting
[KYUUBI #4472] add session/operation methods in AdminRestApi
[KYUUBI #4480] Engine alive probe should close thrift connection on engine lost
[KYUUBI #4482] Support to disable batch resource upload function
[KYUUBI #4483] Enable REST frontend protocol by default
[KYUUBI #4496] Call super.clusterManager() when batchConf does not have spark.master
[KYUUBI #4498] Support to list sessions/operations with conditions in AdminRestApi
[KYUUBI #4501] Allow administrator to specify the user for engine admin
[KYUUBI #4506] Return sessionType in session data
[KYUUBI #4516] Return kyuubi instance in SessionData and OperationData
[KYUUBI #4518] Return engine id in SessionData and post kyuubi instance in KyuubiOperationEvent
[KYUUBI #4519] Update metadata when batch application state changed
[KYUUBI #4523] support close engine session gracefully
[KYUUBI #4526] Defer evaluation for GroupProvider
[KYUUBI #4540] Support to deny some users to make connection
[KYUUBI #4540] Create session limiter if user unlimited list or deny list nonempty
[KYUUBI #4563] Format error log output in case of metrics json file not found
[KYUUBI #4575] Fix the empty last line may causes the session to exit directly
[KYUUBI #4579] Fix the unstable situation of test cases for Trino-Fronted-Service
[KYUUBI #4584] Post the session exception to EventBus in time
[KYUUBI #4594] Support PUT method in REST client
[KYUUBI #4611] support all sessions api in SessionRestApi
[KYUUBI #4618] Admin Resource list operations with sessionHandle filter
[KYUUBI #4623] KubernetesApplicationOperation uses Informer instead of list
[KYUUBI #4639] Support to specify confOverlay when executing statement with RESTful API
[KYUUBI #4652] Fix JaasConfiguration ClassNotFoundException for Hadoop 3.3.4 and previous
[KYUUBI #4653] [KYUUBI 4650] LogDivertAppender supports reading RollingFileAppender pattern
[KYUUBI #4669] LDAP authentication allows auth user contains domain when bind.dn/pw enabled
[KYUUBI #4688] Fix the failure to read the operation log after executing catalog and database operation
[KYUUBI #4691] Configure FAIL_ON_UNKNOWN_PROPERTIES to false for KyuubiScalaObjectMapper
[KYUUBI #4713] Fix false positive result in SchedulerPoolSuite
[KYUUBI #4719] Support submission timeout for yarn application manager and get the applicationInfo in-memory
[KYUUBI #4731] Support batch session conf advisor
[KYUUBI #4733] Introduce Kafka event logger for server events
[KYUUBI #4738] AdminResource.getEngineSpace should use primary group name on GROUP share level
[KYUUBI #4739] Add operation lock instead of locking state Enumeration
[KYUUBI #4746] Do not recreate async request executor if has been shutdown
[KYUUBI #4749] Fix flaky test issues in SchedulerPoolSuite
[KYUUBI #4753] KyuubiConf.unset should not log deprecation warning
[KYUUBI #4767] Correct the submit time for BatchJobSubmission and check applicationInfo if submitted application
[KYUUBI #4777] Deregister event handlers when stopping server with event handler made auto-closeable
[KYUUBI #4780] Get engine application info with interval to prevent frequent call to resource manager
[KYUUBI #4786] Support yarn-client and yarn-cluster for YarnApplicationOperation
[KYUUBI #4790] Initial implement Batch V2
[KYUUBI #4791] Add helper method to simplify REST enabled judgment
[KYUUBI #4792] Enhance hardcode session keywords and remove unused code
[KYUUBI #4796] Expose JVM attributes to metrics
[KYUUBI #4798] Allows BatchJobSubmission to run in sync mode
[KYUUBI #4801] Using different engine submit timeout config for kubernetes and yarn
[KYUUBI #4807] Fix flaky test “basic batch rest client”
[KYUUBI #4807] Retry more times to fix flaky test “spnego batch rest client”
[KYUUBI #4811] Do not update app info after batch or application terminated
[KYUUBI #4813] Add the fetch orientation parameter to the /v1/operations/:operationId/log interface
[KYUUBI #4814] Introduce Apache Atlas hook support in lineage plugin
[KYUUBI #4816] Correct the implementation of cleanup K8s terminated appInfo
[KYUUBI #4829] Support to expose operation metrics
[KYUUBI #4830] Fix flaky test “support to interrupt the thrift request if remote engine is broken”
[KYUUBI #4835] Using hive conf to check whether to apply HIVE_DELEGATION_TOKEN
[KYUUBI #4836] Set UncaughtExceptionHandler for thread to log exception
[KYUUBI #4840] Return cached appInfo iif both op and app are terminated
[KYUUBI #4841] Log more info for closing session
[KYUUBI #4842] Fix typo in etcd client config
[KYUUBI #4843] Support multiple kubernetes contexts and namespaces
[KYUUBI #4847] Close the session immediately when engine corrupt
[KYUUBI #4847] Close the session immediately when engine connection closed
[KYUUBI #4850] Change default EC spec to secp521r1 for internal session variable signature
[KYUUBI #4855] Change socket “connection refused” message match pattern
[KYUUBI #4857] Fix flaky test TFrontendServiceSuite
[KYUUBI #4858] Relax test message assertion in “sync query causes engine crash”
[KYUUBI #4859] HttpException message should not be null
[KYUUBI #4865] Fix flaky test BackendServiceMetricSuite
[KYUUBI #4876] fix Flaky test BackendServiceMetricSuite (#4924)
[KYUUBI #4889] Admin command line supports list server command
[KYUUBI #4890] Return operation metrics in OperationData and fix typo
[KYUUBI #4893] Prevent null collection for rest dto
[KYUUBI #4898] Fix logOperation multiple read with missing line
[KYUUBI #4919] Fix the flaky tests in the “open batch session” functionality within the “BatchesResourceSuite”
[KYUUBI #4920] Add OperationRestApi
[KYUUBI #4946] Alter the order of initLoggerEventHandler
[KYUUBI #4949] For operation getNextRowSet method, with operation lock required
[KYUUBI #4950] Migrate Kyuubi embedded database from Derby to SQLite
[KYUUBI #4950] Misc improvement for usage of SQLite
[KYUUBI #4950] SQLite schema files should start from 1.8.0
[KYUUBI #4951] Add reload4j support
[KYUUBI #4977] Log error message when REST API invocation error occurs
[KYUUBI #4978] Fix flaky test: close expired operations
[KYUUBI #4978] Fix flaky test: close expired operations
[KYUUBI #4979] Fix flaky test: support to interrupt the thrift request if remote engine is broken
[KYUUBI #4981] Refactor code of closeBatchSession
[KYUUBI #4985] Fix log level restore behavior of KyuubiFunSuite#withLogAppender
[KYUUBI #4986] Always use Files#deleteIfExists
[KYUUBI #4994] Respect the engine type and share level for listing all engines
[KYUUBI #4994] Support listing all engines
[KYUUBI #4996] Support to refresh kubernetes configs dynamically
[KYUUBI #5004] Fix typo for kubernetes allowed context and namespace check
[KYUUBI #5010] Make Kyuubi server’s connection URL configurable
[KYUUBI #5029] Close the alive probe session after engine session closed
[KYUUBI #5036] Fix Operation.close not update complete timestamp
[KYUUBI #5043] Destroy the build process when waitCompletion is false and the engine is running in cluster mode
[KYUUBI #5050] Add admin checking in getProxyUser
[KYUUBI #5056] Return more helpful error message to http client
[KYUUBI #5063] Support to filter batch with batch name
[KYUUBI #5065] Call destroy first on killing Spark startup process to allows it release temp files
[KYUUBI #5065] Graceful close the process when launch engine timeout
[KYUUBI #5078] Make kyuubi_instance nullable in metadata table schema
[KYUUBI #5081] Minor refactor JDBCMetadataStore
[KYUUBI #5089] Limit maximum engine startup concurrency of kyuubi server
[KYUUBI #5130] Allow setting Zookeeper SPN using Kyuubi configuration
[KYUUBI #5131] Create index on metastore.create_time
[KYUUBI #5138] Fix flaky test BatchesResourceSuite - get batch session list
[KYUUBI #5145] Change embedded Zookeeper server to method-local variable
[KYUUBI #5158] Allow embedded Zookeeper binding IP address
[KYUUBI #5165] Build Spark Driver/Executor Pod Name(Prefix) in process
[KYUUBI #5185] Use Set collection for order-insensitive configs
[KYUUBI #5187] Fix info message in UserFilter
[KYUUBI #5196] Extract spark core scala version lazily and respect engine env
[KYUUBI #5201] Allow disabling batch impl v2 on globally
[KYUUBI #5204] Thrift HTTP FE checks req conf nullable before evaluating proxy user
[KYUUBI #5206] Try to kill pod with label if no ApplicationInfo found to prevent pod leak
[KYUUBI #5210] Cancel operation will cause the log file node to leak
[KYUUBI #5213] Check config value by enum values
[KYUUBI #5216] Workaround for negative counter in SessionLimiter
[KYUUBI #5218] Improve logs of KubernetesApplicationOperation
[KYUUBI #5220] Batch submitter should only block submitting stage
[KYUUBI #5220] Batch submitted considers application state
[KYUUBI #5243] Distinguish metadata between batch impl v2 and recovery
[KYUUBI #5289] RESTful API should always print audit log
[KYUUBI #5290] Batch impl v2 should pick batch jobs in FIFO
[KYUUBI #5300] Batch v2 should not update kyuubi_instance when open session
[KYUUBI #5305] Return dummy log when batch job before being added to session manager
[KYUUBI #5306] YarnApplicationOperation supports proxy user
[KYUUBI #5310] Batch session recovery should start after HTTP server getting started
[KYUUBI #5327] Add priority field in batch session metadata table definition
[KYUUBI #5328] Batch supports priority scheduling
[KYUUBI #5340] AdminResourceSuite make engine idle time 3min
[KYUUBI #5380] Create PySpark batch jobs tests for RESTful API
[KYUUBI #5381] Change the default metrics reporter to Prometheus
[KYUUBI #5391] Close batch session after batch operation completion for idle timeout
[KYUUBI #5392] Add query timeout monitor on server-side in ExecuteStatement
[KYUUBI #5408] MetadataManager tries MySQL 8 driver class first
[KYUUBI #5412] Resolve the relative zk configuration dir based on KYUUBI_HOME
[KYUUBI #5441] Make the configuration kyuubi.zookeeper.embedded.data.log.dir effective
[KYUUBI #5451] Ignore NoSuchFileException during OperationLog.close()
[KYUUBI #5499] [KYUUBI #2503] Catch any exception when closing idle session
[KYUUBI #5513] Always redirect delete batch request to Kyuubi instance that owns batch session
[KYUUBI #5522] Ignore main class for PySpark batch job submission
[KYUUBI #5528] Remove duplicate properties tag and respect KYUUBI_LOG_DIR
[KYUUBI #5566] InternalRestClient respects kyuubi.engine.security.enabled to add HTTP auth header
[KYUUBI #5566] Check InternalSecurityAccessor is initialized only when kyuubi.engine.security.enabled is true
[KYUUBI #5568] Check administrator only when security is enabled
[KYUUBI #5568] Fix security enabled judgement
[KYUUBI #5606] Handle engine listing request properly for users who have not created engine

Client

[KYUUBI #3887] Fix kyuubiServerPrincipal logic in KyuubiCommands
[KYUUBI #4305] Backport HIVE-15820: comment at the head of beeline -e
[KYUUBI #4438] Render the batch info map in kyuubi-ctl
[KYUUBI #4439] Add dto class for operation data
[KYUUBI #4452] Strip the redundant leading and tailing slash of getZooKeeperNamespace.
[KYUUBI #4479] Restore JDBC Kerberos authentication behavior for UGI.doAs
[KYUUBI #4489] Fix missing org.apache.commons.logging by including jcl-over-slf4j dependency in REST client
[KYUUBI #4492] Correct engine subdomain calculation in kyuubi-ctl
[KYUUBI #4508] Beeline should reset stdin after consuming init SQL file
[KYUUBI #4581] Add KyuubiCommands parse python snippets unit test
[KYUUBI #4609] get engineRefId in KyuubiConnection
[KYUUBI #4619] Fix beeline with -e when there are other SQL statements before the source statement, the source statement cannot be executed normally
[KYUUBI #4657] Building rest client to kyuubi instance including original host urls
[KYUUBI #4962] Backport HIVE-19048: init script errors are ignored
[KYUUBI #4965] Beeline supports --python-mode option and remove comments for non-python mode
[KYUUBI #4969] Run JUnit tests on beeline module
[KYUUBI #5054] Refine the kyuubi client version properties file to prevent conflicts
[KYUUBI #5100] Fix unpaired quote in message pattern
[KYUUBI #5101] Remove unused StringBuilders in ResultSets
[KYUUBI #5129] KyuubiBeeline should redirect JDK logging
[KYUUBI #5179] Use Iterable instead of Seq in kyuubi-ctl commands
[KYUUBI #5297] New RetryableClient get http://null server uri from metadata.kyuubiInstance
[KYUUBI #5480] Support setting kyuubi hive jdbc client protocol version
[KYUUBI #5564] Support to config ticket cache location in JDBC URL

Web UI

[KYUUBI #3420] Kyuubi Server Proxy Engine UI
[KYUUBI #3646] Front-end style should bracket same line
[KYUUBI #3646] Init Session Statistic Page
[KYUUBI #3648] Add Session Detail Page
[KYUUBI #3650] Add Operation Statistics Page
[KYUUBI #3652] Add Kyuubi Server Management page
[KYUUBI #3654] Add Engine Manager Page
[KYUUBI #4537] Enable Vite proxy server for web ui development
[KYUUBI #4565] Fix out-date README.md about Development Project
[KYUUBI #4622] Session api use admin api for security
[KYUUBI #4626] Fix Web UI CI check fail with pnpm the lockfile needs updates
[KYUUBI #4757] Move Statistics to Management Menu
[KYUUBI #4982] Add query string to forward requests to Engine UI
[KYUUBI #5259] Web UI supports Swagger pages
[KYUUBI #5463] Adjust the file directory structure
[KYUUBI #5484] Remove legacy Web UI
[KYUUBI #5573] Delete parts of the Kyuubi Web UI that are not useful
[KYUUBI #5586] Kyuubi Web UI show current version
[KYUUBI #5608] Kyuubi version should always use the light color
[KYUUBI #5620] Remove dummy information card from Overview page

Documentation

[KYUUBI #4308] Make README more welcoming
[KYUUBI #4312] Fix over-matched README pattern for markdown styling
[KYUUBI #4367] Kyuubi requires Flink 1.16/1.17
[KYUUBI #4387] Update Flink version requirements
[KYUUBI #4568] Make kyuubi-ctl doc enable variable ‘release’ automatic substitution
[KYUUBI #4583] Rename Flink Table Store to Apache Paimon (Incubating)
[KYUUBI #4593] Keep promote the release step update-to-date
[KYUUBI #4599] Simplify project version evaluation
[KYUUBI #4607] Rename Flink Table Store to Apache Paimon (Incubating) in docs Connectors for Hive SQL Query Engine
[KYUUBI #4608] Rename Flink Table Store to Apache Paimon (Incubating) in docs Connectors for Trino SQL Query Engine
[KYUUBI #4614] Rename Flink Table Store to Apache Paimon (Incubating) in docs Connectors for Flink SQL Query Engine
[KYUUBI #4624] Fix table headers in kyuubi_vs_hive.md
[KYUUBI #4627] Fix a typo in rest_api.md.
[KYUUBI #4655] Enrich docs for Kyuubi Hive JDBC driver
[KYUUBI #4655] Recover the missing JDBC URL
[KYUUBI #4722] Kyuubi requires Spark 3.1 and above now
[KYUUBI #4727] kyuubi-spark-lineage has no transitive deps
[KYUUBI #4734] Fix typo in docs of custom event handler
[KYUUBI #4742] Add docs for Flink application mode
[KYUUBI #4763] Fix the Kyuubi JDBC kerberos parameters
[KYUUBI #4800] Update readthedocs.yaml
[KYUUBI #4864] Fix grammar in HA section
[KYUUBI #4933] Mark spark.sql.optimizer.insertRepartitionNum config for Spark 3.1 only
[KYUUBI #4954] Fix variable reference in quick_start_with_jdbc
[KYUUBI #4960] Fix doc for pyhive client.
[KYUUBI #5049] PyHive Kerberos usage doc
[KYUUBI #5083] Add LDAP document
[KYUUBI #5085] Update config section of deploy on kubernetes
[KYUUBI #5094] Add Python JayDeBeApi doc
[KYUUBI #5099] Document link failure
[KYUUBI #5118] Add Flink KDF doc
[KYUUBI #5122] Hive KDF usage
[KYUUBI #5127] Improvements for markdown builder
[KYUUBI #5146] Fix link of IntelliJ IDEA Setup Guide
[KYUUBI #5153] Minor change w/ to with
[KYUUBI #5154] Move configuration docs to the top level
[KYUUBI #5162] Add statements about contributions co-authored with AIGC in developer guide page
[KYUUBI #5178] Make MarkdownBuilder not extending Growable
[KYUUBI #5227] Added deploy mode info
[KYUUBI #5247] Correct building command for Spark authz/lineage plugins
[KYUUBI #5273] Update release docs
[KYUUBI #5275] Improve and fix comparison and regeneration for golden files
[KYUUBI #5294] Update supported dialects for JDBC engine
[KYUUBI #5335] Set markdown file EOL
[KYUUBI #5345] Fix non-kerberized code description in quick_start_with_jdbc
[KYUUBI #5347] Fix java code errors in quick_start_with_jdbc
[KYUUBI #5409] Update config docs regeneration hints in developer guide
[KYUUBI #5432] Fix typo in README.md
[KYUUBI #5523] Update the Kyuubi supported components version

Building, Infrastructure and Dependency

[KYUUBI #4078] Fix shellcheck violations in scripts of /bin folder
[KYUUBI #4187] Windows build-info script supports revision time
[KYUUBI #4278] Use new Apache ‘closer.lua’ syntax to obtain Maven
[KYUUBI #4295] Introduce super-linter action for linting JSON, XML, ENV files and bash_exec
[KYUUBI #4299] Bump JUnit to 4.13.2
[KYUUBI #4299] Bump MySQL JDBC driver to 8.0.32
[KYUUBI #4315] Check modules available before build step in dependency workflow
[KYUUBI #4347] Bump maven download plugin from 1.6.6 to 1.6.8
[KYUUBI #4348] Cache engine archives in CI jobs for maven download plugin
[KYUUBI #4397] build/dist supports --web-ui
[KYUUBI #4413] Build docker image should include web UI
[KYUUBI #4447] Bump log4j from 2.19.0 to 2.20.0
[KYUUBI #4448] Bump Netty from 4.1.87.Final to 4.1.89.Final
[KYUUBI #4451] Skip etcd test if no docker env
[KYUUBI #4453] Bump Kubernetes Client Version to 6.4.1
[KYUUBI #4461] Add Kyuubi Kubernetes IT Module to CI style check
[KYUUBI #4466] Use bitnami/minio docker image
[KYUUBI #4471] Prompted command for docs generation should skip Java tests
[KYUUBI #4484] [K8S][HELM] Set specific Kyuubi version to the chart appVersion
[KYUUBI #4494] bin/kyuubi should use exec to run Kyuubi server
[KYUUBI #4513] Bump Kyuubi 1.7.0 in Playground
[KYUUBI #4546] Fully exclude metrics dir from git
[KYUUBI #4557] [K8S][HELM] Kyuubi server should bind Pod IP by default
[KYUUBI #4572] Bump Iceberg from 1.1.0 to 1.2.0
[KYUUBI #4576] Multi arch build for ARM support in Docker Hub
[KYUUBI #4589] [K8S][HELM] Update template on port-forward usage guide NOTES.txt
[KYUUBI #4590] Bump delta from 2.2.0 to 2.3.0
[KYUUBI #4598] Bump Flink-1.15 from 1.15.3 to 1.15.4
[KYUUBI #4600] Bump Iceberg 1.2.0 for playground
[KYUUBI #4601] Bump Hadoop 3.3.5 for playground
[KYUUBI #4605] [K8S][HELM] Set IfNotPresent pullPolicy by default
[KYUUBI #4606] [K8S][HELM] Add command and args configuration support
[KYUUBI #4615] Bump Ranger from 2.3.0 to 2.4.0
[KYUUBI #4628] Add Dependency management ISSUE_TEMPLATE
[KYUUBI #4632] Add rule for labeler to tag module:ui
[KYUUBI #4638] Update dependency.yml as label in not permitted in markdown block
[KYUUBI #4652] Upgrade Hadoop from 3.3.4 to 3.3.5
[KYUUBI #4695] Bump super-linter action from v4 to v5
[KYUUBI #4696] Upgrade scalafmt from 3.7.1 to 3.7.3
[KYUUBI #4697] [K8S][HELM] Add template rendering info to README
[KYUUBI #4698] [K8S][HELM] Centralize Kyuubi labels definition
[KYUUBI #4706] [K8S][HELM] Use template comments for the chart license header
[KYUUBI #4707] [K8S][HELM] Fix README typo and minor reformatting
[KYUUBI #4712] Bump Spark from 3.2.3 to 3.2.4
[KYUUBI #4737] Restore Project & Community Status in README.md
[KYUUBI #4744] Remove Hudi integration tests
[KYUUBI #4747] Install and cache build/mvn in CI builds
[KYUUBI #4768] Guidelines for Document and Code contributions
[KYUUBI #4772] Bump Jersey from 2.39 to 2.39.1
[KYUUBI #4783] Use pnpm v8 and update pnpm lockfile to v6 schema
[KYUUBI #4784] Use nodejs v18 in CI builds and maven web-ui builds
[KYUUBI #4788] [K8S][HELM] Use StatefulSet instead of Deployment
[KYUUBI #4805] Bump openai-java from 0.11.1 to 0.12.0
[KYUUBI #4808] Bump Jetty from 9.4.50 to 9.4.51
[KYUUBI #4818] [K8S][HELM] Update default Kyuubi version to 1.7.1
[KYUUBI #4819] [K8S][HELM] Refactor liveliness and readiness probes
[KYUUBI #4824] Bump Jackson from 2.14.2 to 2.15.0
[KYUUBI #4827] Bump Apache Arrow from 11.0.0 to 12.0.0
[KYUUBI #4828] Exclude macOS tar extended metadata in build/dist
[KYUUBI #4849] Open modules to enable JDK 17 support
[KYUUBI #4851] Bump Maven 3.8.8
[KYUUBI #4852] Switch to Kyuubi Shaded Zookeeper
[KYUUBI #4856] Add restore-keys for action/cache
[KYUUBI #4870] Add kyuubi-util and kyuubi-util-scala modules
[KYUUBI #4877] Bump Flink 1.17 to 1.17.1
[KYUUBI #4879] Refactor and promote reflection utils and cleanup similar reflection methods
[KYUUBI #4883] Bump Iceberg 1.3.0
[KYUUBI #4887] Refactor and add ut for ClassUtils
[KYUUBI #4891] Bump Flink 1.16.2
[KYUUBI #4894] Bump Mockito from 4.9.0 to 4.11.0 and ScalaTest from 3.2.15 to 3.2.16
[KYUUBI #4904] Move AssertionUtils to kyuubi-util-scala module
[KYUUBI #4905] Generalize util method for loading class from service loader
[KYUUBI #4907] Add a maven profile ‘remote-debug’
[KYUUBI #4908] Bump scalafmt from 3.7.3 to 3.7.4
[KYUUBI #4911] Remove duplicate test annotations org.apache.kyuubi.tags.* in kyuubi-common
[KYUUBI #4912] Replace Scala’s assert in tests with scalatest’s for prettified error message
[KYUUBI #4931] Bump vite from 3.0.0 to 4.2.3 and vitest from 0.22.0 to 0.32.0
[KYUUBI #4937] Remove redundant quoteIfNeeded method
[KYUUBI #4943] Bump Guava from 31.1 to 32.0.1
[KYUUBI #4944] Code improvement for Java
[KYUUBI #4947] Bump pnpm version in maven’s frontend plugin to 8.6.1 for lockfileVersion 6.1
[KYUUBI #4948] Reuse versions of NodeJS and pnpm from maven properties
[KYUUBI #4959] Code improvements for Scala
[KYUUBI #4966] Bump Apache parent pom from 29 to 30
[KYUUBI #4968] Simplify Option conversion
[KYUUBI #4970] Unified reflection methods invokeAs and getField
[KYUUBI #4974] Eliminate raw-types compilation warning in KyuubiBeeLine
[KYUUBI #4975] Fixed run test link in PR template
[KYUUBI #4990] Bump Spark 3.4.1
[KYUUBI #4992] Bump Netty 4.1.93.Final
[KYUUBI #5001] Bump Scala from 2.12.17 to 2.12.18
[KYUUBI #5003] [K8S][HELM] Add doc strings to the chart values.yaml
[KYUUBI #5007] Bump scalafmt from 3.7.4 to 3.7.5
[KYUUBI #5019] Shows details of compilation info in SparkUI’s engine tab
[KYUUBI #5024] Bump grpc-protobuf from 1.48.0 to 1.53.0
[KYUUBI #5034] Bump tough-cookie from 4.0.0 to 4.1.3 in /kyuubi-server/web-ui
[KYUUBI #5039] Use semantic versions and remove redundant version comparison methods
[KYUUBI #5058] [K8S][HELM] Allow the user to set the priorityClass used by Kyuubi server Pod
[KYUUBI #5059] Upgrade kubernetes client version to 6.7.2
[KYUUBI #5064] Audit kubernetes application state change in separate log files
[KYUUBI #5068] Bump Spark in spark-master profile from 3.5.0-SNAPSHOT to 4.0.0-SNAPSHOT
[KYUUBI #5070] Bump word-wrap from 1.2.3 to 1.2.4 in web UI
[KYUUBI #5072] Fix KyuubiOperationWithEngineSecuritySuite and related issues
[KYUUBI #5090] Fix AllKyuubiConfiguration to generate redundant blank lines in Windows
[KYUUBI #5113] Remove dummy Spark Kudu connector module
[KYUUBI #5116] Bump Hadoop 3.3.6
[KYUUBI #5117] Code improvements for Java and Scala code
[KYUUBI #5121] Extract dev scripts for regenerating the golden files
[KYUUBI #5132] [K8S][HELM] Kyuubi add server metrics monitor
[KYUUBI #5135] Exclude web-ui coverage reports in rat plugins
[KYUUBI #5149] Correct error message of ReflectUtils’s invokeAs when method not found
[KYUUBI #5150] Bump Spark 3.3.3
[KYUUBI #5152] Check milestone and assignees when merging pull request
[KYUUBI #5177] Use Scala binary version placeholder in Maven module’s artifactId suffix
[KYUUBI #5183] Introducing profile scala-2.13 for development use
[KYUUBI #5196] Enable CI Test on Scala 2.13 and support custom or spark-core extracted Scala version for Spark’s engine
[KYUUBI #5202] Update upload_nexus_staging
[KYUUBI #5205] Upgrade kubernetes-client to 6.8.1
[KYUUBI #5209] Reformat Scala source code crossing versions
[KYUUBI #5212] Fix configuration errors causing by helm charts of prometheus services
[KYUUBI #5214] Correct file path for scala tests in util-scala module
[KYUUBI #5234] Upgrade Jetty version from 9.4.51.v20230217 to 9.4.52.v20230823
[KYUUBI #5235] Daily deploy snapshot version on master and branch-1.7
[KYUUBI #5250] Bump Iceberg from 1.3.0 to 1.3.1
[KYUUBI #5252] Remove incubator link
[KYUUBI #5253] Upgrade testcontainers-scala from 0.40.12 to 0.41.0
[KYUUBI #5253] Supply testcontainers-scala-scalatest deps
[KYUUBI #5293] Upgrade snakeyaml from 1.33 to 2.2
[KYUUBI #5305] Fix get local log UT
[KYUUBI #5308] Ensure release using Java 8
[KYUUBI #5318] Pin maven-source-plugin 3.2.1
[KYUUBI #5358] Bump iceberg 1.4.0
[KYUUBI #5365] Don’t use Log4j2’s extended throwable conversion pattern in default logging configurations
[KYUUBI #5425] Add thread name and ID to log4j2-defaults.xml
[KYUUBI #5435] Improve Kyuubi On Kubernetes IT
[KYUUBI #5443] Add kyuubi-kubernetes-it module to reformat script
[KYUUBI #5444] Check style with profiles of Spark 3.4 and 3.5
[KYUUBI #5483] Fix deploy command for Scala 2.13 TPC-DS/H connectors
[KYUUBI #5483] Release Spark TPC-H/DS Connectors with Scala 2.13
[KYUUBI #5486] Bump Kafka client version from 3.4.0 to 3.5.1
[KYUUBI #5508] Upgrade download-maven-plugin from 1.6.8 to 1.7.1
[KYUUBI #5544] Release contains Spark 3.5 extension
[KYUUBI #5561] Binary distribution artifact ships database schema scripts
[KYUUBI #5561] Rename folder to db-scripts
[KYUUBI #5569] Fix pnpm-lock file version

Credits

Last but not least, this release would not have been possible without the following contributors:

Deng An
GitHub Bot
CavemanIV
Fu Chen
Shaoyun Chen
Chao Chen
Derick Cypho
Yang Du
FourSpaces
Xu Guo
Karsonnel
Denis Krivenko
Kyofin
Pengqi Li
Xieming Li
Guanhua Li
Bowen Liang
Tianlin Liao
Hanna Liashchuk
Paul Lin
Senmiao Liu
Peiyue Liu
Naijie Liu
Chenliang Lu
Ruilei Ma
Omkar Makhare
Drake Youngkun Min
Sidhant Nagpal
Cheng Pan
QianyongY
Anurag Rajawat
Rojin Raju
Fei Wang
Zhen Wang
Kang Wang
Miao Wang
Bruce Wong
XorSum
Junjie Xu
Binjie Yang
Remzi Yang
Kent Yao
Xianxun Ye
Kaifei Yi
Xiduo You
Ruguo Yu
David Yuan
Qiang Zhang
Yaodong Zhang
Yangming Zhao
He Zhao
Min Zhao
Yifan Zhou
Yi Zhu
Yaogai Zhu
Alex Zou
Junjie Zu
biangjuang
camper42
fantasticKe
huangzhir
huapan123456
liaoyt
mans2singh
mrtisttt
phyyou
rhh777
runzhliu
touchida
uicosp
wangyepeng2
weixi62961
yehere

Release Notes - 1.7.3

2023 Sep 25

Highlight

Fix file handle leak caused by cancel operation

Server

[KYUUBI #5210] Cancel operation will cause the log file node to leak

Spark Authz Plugin

[KYUUBI #5269] [Bug] Dupllicated AnalyzesTablesCommand in kyuubi authz

Credits

Last but not least, this release would not have been possible without the following contributors:

AngersZhuuuu
ASiegeLion

Release Notes - 1.7.2

2023 Sep 18

Highlight

Enhance JDBC Client and Beeline Client, like: Support running Python scripts and codes in Beeline, fix Http authentication exception message is empty, backport HIVE-19048: Initscript errors are ignored, backport HIVE-15820: comment at the head of beeline -e.
Enhance submission engine related code, like: support yarn-client and yarn-cluster for YarnApplicationOperation, different engine submit timeout config, destroy build process when waitCompletion is false and engine is running in cluster mode
Fix many flaky tests and incorrect documentations

Server

[KYUUBI #4786] Support yarn-client and yarn-cluster for YarnApplicationOperation
[KYUUBI #4801] Using different engine submit timeout config for kubernetes and yarn
[KYUUBI #4811] Do not update app info after batch or application terminated
[KYUUBI #4816] [K8S] Correct the implementation of cleanup terminated appInfo
[KYUUBI #4830] [TEST] Fix flaky test “support to interrupt the thrift request if remote engine is broken”
[KYUUBI #4840] Return cached appInfo iif both op and app are terminated
[KYUUBI #4841] [MINOR] Log more info for closing session
[KYUUBI #4807] [TEST] Retry more times to fix flaky test “spnego batch rest client”
[KYUUBI #4807] [FOLLOWUP] Fix flaky test “basic batch rest client”
[KYUUBI #4855] Change socket “connection refused” message match pattern
[KYUUBI #4858] Relax test message assertion in “sync query causes engine crash”
[KYUUBI #4919] Fix the flaky tests in the “open batch session” functionality within the “BatchesResourceSuite”
[KYUUBI #4876] fix Flaky test BackendServiceMetricSuite
[KYUUBI #4979] Fix flaky test: support to interrupt the thrift request if remote engine is broken
[KYUUBI #4977] Log error message when REST API invocation error occurs
[KYUUBI #4946] Alter the order of initLoggerEventHandler
[KYUUBI #5072] [TEST] Fix KyuubiOperationWithEngineSecuritySuite and related issues
[KYUUBI #5043] [1.7] Destroy the build process when waitCompletion is false and the engine is running in cluster mode
[KYUUBI #3920] [TEST] Fix flaky test - distribute lock
[KYUUBI #5204] Thrift HTTP FE checks req conf nullable before evaluating proxy user

Spark Engine

[KYUUBI #4838] Fix spark operation exception leak in withLocalProperties method
[KYUUBI #4835] [K8S] Using hive conf to check whether to apply HIVE_DELEGATION_TOKEN
[KYUUBI #4915] Adapt database not exist error message change for Spark 3.4
[KYUUBI #4952] Enhance AWS Glue default database does not exist error message on opening session
[KYUUBI #5005] Remove default settings spark.sql.execution.topKSortFallbackThreshold
[KYUUBI #5025] Use formatDuration instead of formatDurationVerbose in SparkUI’s engine tab
[KYUUBI #5019] Shows details of compilation info in SparkUI’s engine tab
[KYUUBI #5045] Initialize EXECUTOR_POD_NAME_PREFIX_MAX_LENGTH before using
[KYUUBI #5105] [Bug] getColumns gets duplicate column names when Spark integrates with Iceberg
[KYUUBI #5148] Improve spark.driver.host assignment in Spark on K8s client mode
[KYUUBI #5232] In SparkOperation#cleanup always calls cancelJobGroup even though it’s in the completed state

Spark Extensions

[KYUUBI #4995] Use hadoop conf and hive conf from catalog options
[KYUUBI #5028] Update session hadoop conf to catalog hadoop conf

Client

[KYUUBI #4859] HttpException message should not be null
[KYUUBI #4969] [TEST] Run JUnit tests on beeline module
[KYUUBI #4962] [1.7] Backport HIVE-19048: Initscript errors are ignored
[KYUUBI #4965] [BEELINE] Support –python-mode option and remove comments for non-python mode
[KYUUBI #4305] Backport HIVE-15820: comment at the head of beeline -e
[KYUUBI #5100] [Minor] Fix unpaired quote in message pattern
[KYUUBI #5101] [MINOR] Remove unused StringBuilders in ResultSets
[KYUUBI #5129] KyuubiBeeline should redirect JDK logging

Common

[KYUUBI #4808] Bump Jetty from 9.4.50 to 9.4.51
[KYUUBI #4836] Set UncaughtExceptionHandler for thread to log exception
[KYUUBI #4857] Fix flaky test TFrontendServiceSuite
[KYUUBI #4898] fix logOperation multiple read with missing line
[KYUUBI #4949] For operation getNextRowSet method, with operation lock required
[KYUUBI #4951] Add reload4j support
[KYUUBI #4978] Fix flaky test: close expired operations
[KYUUBI #4986] Always use Files#deleteIfExists
[KYUUBI #4978] [FOLLOWUP] Fix flaky test: close expired operations
[KYUUBI #4985] [TEST] Fix loglevel restore behavior of KyuubiFunSuite#withLogAppender
[KYUUBI #5036] Fix Operation.close not update complete timestamp
[KYUUBI #5090] Fix AllKyuubiConfiguration to generate redundant blank lines in Windows
[KYUUBI #5158] Allow embedded Zookeeper binding IP address
[KYUUBI #5187] [Minor] Fix info message in UserFilter

Deployment

[KYUUBI #4818] [K8S] [HELM] Update default Kyuubi version to 1.7.1
[KYUUBI #4828] [BUILD] Exclude macOS tar extended metadata in build/dist
[KYUUBI #4851] [BUILD] Bump Maven 3.8.8

Documentation

[KYUUBI #4864] [DOCS] Fix grammar in HA section
[KYUUBI #4954] [DOCS] Fix variable reference in quick_start_with_jdbc
[KYUUBI #4960] [DOC] Fix doc for pyhive client.
[KYUUBI #5049] [DOCS] PyHive Kerberos usage doc
[KYUUBI #5085] Update config section of deploy on kubernetes
[KYUUBI #5094] Add Python JayDeBeApi doc
[KYUUBI #5099] Document link failure
[KYUUBI #5083] [DOC] Add LDAP document
[KYUUBI #5146] [DOC] Fix link of IntelliJ IDEA Setup Guide

Credits

Last but not least, this release would not have been possible without the following contributors:

BruceWong96
CavemanIV
QianyongY
bowenliang123
cxzl25
dev-lpq
dnskr
huangzhir
lightning-L
liunaijie
lsm1
mans2singh
pan3793
risyomei
touchida
turboFei
uicosp
ulysses-you
wForget
yaooqinn
zhaohehuhu
zhaomin1423

Release Notes - 1.7.1

2023 May 5

Highlight

Kyuubi requires Spark 3.1 and above
Upgrade Hadoop from 3.3.4 to 3.3.5
A series of improvements for Arrow based serialization
A series of improvements for batch job of Spark on K8s/YARN
Bug fix for ‘beeline -e’ with SQLs before ‘source’ statement
Spark Authz plugin fixes privilege checks on source tables of the permanent view in Spark 3.1

Server

[KYUUBI #4652][FOLLOWUP] Fix JaasConfiguration ClassNotFoundException for Hadoop 3.3.4 and previous
[KYUUBI #3887][FOLLOWUP] Fix kyuubiServerPrincipal logic in KyuubiCommands
[KYUUBI #4763][DOCS] Fix the Kyuubi JDBC kerberos parameters
[KYUUBI #4738] AdminResource.getEngineSpace should use primary group name on GROUP share level
[KYUUBI #4655][DOCS][FOLLOWUP] Recover the missing JDBC URL
[KYUUBI #4753][KYUUBI 4752][Improvement] KyuubiConf.unset should not log deprecation warning
[KYUUBI #4407][FOLLOWUP] Adapt SLF4J2
[KYUUBI #4746] Do not recreate async request executor if has been shutdown
[KYUUBI #4739] Add operation lock instead of locking state Enumeration
[KYUUBI #4719] Support submission timeout for yarn application manager and get the applicationInfo in-memory
[KYUUBI #4731] Support batch session conf advisor
[KYUUBI #4734][Docs] Fix typo in docs of custom event handler
[KYUUBI #4711] JDBC client should catch task failed exception instead of NPE in the incremental mode
[KYUUBI #4688] Fix the failure to read the operation log after executing catalog and database operation
[KYUUBI #4652] Upgrade Hadoop from 3.3.4 to 3.3.5
[KYUUBI #4691][REST] Configure FAIL_ON_UNKNOWN_PROPERTIES to false for KyuubiScalaObjectMapper
[KYUUBI #4658][FOLLOWUP] Improve unit tests
[KYUUBI #4669] LDAP authentication allows auth user contains domain when bind.dn/pw enabled
[KYUUBI #4653][KYUUBI 4650][Improvement] LogDivertAppender supports reading RollingFileAppender pattern
[KYUUBI #4623][FOLLOWUP] Fix malformed docs
[KYUUBI #4522] use:catalog should execute before than use:database
[KYUUBI #4655][DOCS] Enrich docs for Kyuubi Hive JDBC driver
[KYUUBI #4644] Manually terminate the Py4JServer during engine shutdown
[KYUUBI #4623][FOLLOWUP] Remove unused import
[KYUUBI #4623][K8S] KubernetesApplicationOperation uses Informer instead of list
[KYUUBI #3872][BATCH] Skip setting proxy user when keytab is provided
[KYUUBI #4639] Support to specify confOverlay when executing statement with RESTful API
[KYUUBI #4627][Docs] Fix a typo in rest_api.md.
[KYUUBI #4626][UI][CI] Fix Web UI Ci check fail with pnpm the lockfile needs updates
[KYUUBI #4624][Docs] Fix table headers in kyuubi_vs_hive.md
[KYUUBI #4620][KSHC] Cut off transitive dependencies
[KYUUBI #4599][DOCS] Simplify project version evaluation
[KYUUBI #4480] Engine alive probe should close thrift connection on engine lost
[KYUUBI #4593][DOCS] Keep promote the release step update-to-date
[KYUUBI #4579] Fix the unstable situation of test cases for Trino-Fronted-Service
[KYUUBI #4584] Post the session exception to EventBus in time
[KYUUBI #4563][Improvement] Format error log output in case of metrics json file not found
[KYUUBI #3739][FOLLOWUP][REST] Remove unused parameters in SessionsResource rest docs (#4542)
[KYUUBI #4581][TEST] Add KyuubiCommands parse python snippets unit test
[KYUUBI #4575] Fix the empty last line may causes the session to exit directly
[KYUUBI #4568] Make kyuubi-ctl doc enable variable ‘release’ automatic substitution
[KYUUBI #4526][FOLLOWUP] Defer evaluation for GroupProvider
[KYUUBI #4519] Update metadata when batch application state changed
[KYUUBI #4467][K8S][BATCH] Tolerate Driver Pod ephemerally invisible after submitting
[KYUUBI #4496] Call super.clusterManager() when batchConf does not have spark.master
[KYUUBI #4494] bin/kyuubi should use exec to run Kyuubi server
[KYUUBI #4491] Fix Trino typo
[KYUUBI #4489][REST] Fix missing org.apache.commons.logging by including jcl-over-slf4j dependency in REST client
[KYUUBI #4488][KSHC] Keep object original name defined in HiveBridgeHelper
[KYUUBI #4471] Prompted command for docs generation should skip Java tests
[KYUUBI #4482] Support to disable batch resource upload function
[KYUUBI #4452] Strip the redundant leading and tailing slash of getZooKeeperNamespace.
[KYUUBI #4451] Skip etcd test if no docker env
[KYUUBI #4433] Fix the case with create a view from view
[KYUUBI #4464] Simplify and improve log for JDBCMetadataStore
[KYUUBI #4462] Fix variable usage issue in SessionManager#stop
[KYUUBI #4443] Do not set engine session init sql for alive probe session

Spark Engine

[KYUUBI #4754][ARROW] Use KyuubiArrowConveters#toBatchIterator instead of ArrowConveters#toBatchIterator
[KYUUBI #4749] Fix flaky test issues in SchedulerPoolSuite
[KYUUBI #4722][DOCS] Kyuubi requires Spark 3.1 and above now
[KYUUBI #4720][ARROW] Fix java.lang.NoSuchFieldError: IpcOption.DEFAULT for Spark-3.1/3.2
[KYUUBI #4713][TEST] Fix false positive result in SchedulerPoolSuite (#4714)
[KYUUBI #4713][TEST] Fix false positive result in SchedulerPoolSuite
[KYUUBI #4710][ARROW] LocalTableScanExec should not trigger job
[KYUUBI #4702][ARROW] CommandResultExec should not trigger job
[KYUUBI #4662][ARROW] Arrow serialization should not introduce extra shuffle for outermost limit
[KYUUBI #4587][SPARK][EXT] Take care of table cache query stage

Spark Authz Plugin

[KYUUBI #4585] Authz policy file generation
[KYUUBI #4658][Authz][Bug] Fix InsertIntoHiveDirCommand classname so that we can extract the query in it when authorization.
[KYUUBI #4617][AUTHZ] Collect results for filtered show objects ahead to prevent holding unserializable spark plan
[KYUUBI #4504][Authz][Bug] Fix source table privilege requirement when querying permanent view in Spark 3.1 and below
[KYUUBI #4532][AUTHZ] Displays the columns involved in extracting the aggregation operator
[KYUUBI #4450] Ignore unknown fields policyPriority when reading policy json file

Other Spark Extensions

[KYUUBI #4727][DOC] kyuubi-spark-lineage has no transitive deps
[KYUUBI #4586][TEST][1.7] Fix to clean up views in lineage unit tests
[KYUUBI #4525][KSHC] Partitioning predicates should take effect to filter data
[KYUUBI #4331][KYUUBI #4431] Lineage supports Union and MergeInto

Flink Engine

[KYUUBI #4666] Support Flink varbinary type in query operation
[KYUUBI #4635] Support Flink time type in query operation
[KYUUBI #4614][DOCS] Rename Flink Table Store to Apache Paimon (Incubating) in docs Connectors for Flink SQL Query Engine
[KYUUBI #4607][DOCS] Rename Flink Table Store to Apache Paimon (Incubating) in docs Connectors for Hive SQL Query Engine
[KYUUBI #4608][DOCS] Rename Flink Table Store to Apache Paimon (Incubating) in docs Connectors for Trino SQL Query Engine
[KYUUBI #4598] Bump Flink-1.15 from 1.15.3 to 1.15.4
[KYUUBI #4583][DOCS] Rename Flink Table Store to Apache Paimon (Incubating)
[KYUUBI #4517][FLINK] Fix multiple executions lead to abnormal results on Flink 1.1
[KYUUBI #4446] Fix connections blocked by Flink insert statements

Client

[KYUUBI #4619] Fix beeline with -e When there are other SQL statements before the source statement, the source statement cannot be executed normally
[KYUUBI #4508][BEELINE] Beeline should reset stdin after consuming init SQL file
[KYUUBI #4492] Correct engine subdomain calculation in kyuubi-ctl
[KYUUBI #4479] Restore JDBC Kerberos authentication behavior for UGI.doAs
[KYUUBI #4438] Render the batch info map in kyuubi-ctl

Deployment

[KYUUBI #4706][K8S][HELM] Use template comments for the chart license header
[KYUUBI #4707][K8S][HELM] Fix README typo and minor reformatting
[KYUUBI #4606][K8S][HELM] Add command and args configuration support
[KYUUBI #4605][K8S][HELM] Set IfNotPresent pullPolicy by default
[KYUUBI #4589][K8S][HELM] Update template on port-foward usage guide NOTES.txt
[KYUUBI #4557][K8S][HELM] Kyuubi server should bind Pod IP by default
[KYUUBI #4484][K8S][HELM] Set specific Kyuubi version to the chart appVersion

Credits

Last but not least, this release would not have been possible without the following contributors:

Alex
Bowen Liang
Chao Chen
Cheng Pan
Derick Cypho
Fei Wang
Fu Chen
Karsonnel
Paul Lin
Ruguo Yu
Tianlin Liao
Xieming LI
Yikf
dnskr
fwang12
guanhua.lgh
huangzhir
maruilei
miaowang
minyk
odone
pengqli
phyyou
remzi
senmiaoliu
sychen
ulysses-you
wangyepeng2
xu.guo
zwangsheng

Release Notes - 1.7.0

2023 Mar 8

Highlight

Fully tested w/ Spark 3.1, 3.2 and 3.3, initial support oncoming 3.4
Fully tested w/ Flink 1.14, 1.15 and 1.16
Introduce Trino frontend (experimental)
Introduce Spark Lineage plugin (experimental)
Introduce Spark Hive connector (experimental)
Spark engine supports arrow-based results serialization
Spark engine supports executing Python/PySpark scripts(experimental)
Several Kyuubi Helm Chart & Spark on K8s improvements
REST and Batch API functions, clients and stability enhancements
Major Spark Authz plugin improvements of including Apache Iceberg support
Security enhancements including LDAP authentication, internal encrypted transports, etc.

Server

Client

Spark Engine

Spark Authz Plugin

Other Spark Extensions

Flink Engine

Other Notable Changes

Credits

Last but not least, this release would not have been possible without the following contributors:

AdithyaPadmashali
Alex Wiss-Wolferding
Bowen Liang
Brandon Grams
CHzxp
Cheng Pan
Deng An
Fei Wang
Fu Chen
HYBG
Hanna Liashchuk
Hyukjin Kwon
John Caveman
Kent Yao
Luning Wang
Min Zhao
Nietzsche_w
Paul Lin
Ruguo Yu
SteNicholas
Thomas Prelle
Tianlin Liao
Wang Zhen
William Tong
Xuedong Luan
Yikf
asagjj
df_liu
dnskr
dongkelun
edddddy
firefox
hongdongdong
hongkunyoo
jiaoqingbo
jinsilei
lcy999
liangbowen
liaopeng
maming
odone
pengqli
runzhliu
senmiaoliu
sundy xiong
sychen
ulysses-you
wForget
wangjunbo
waywtdcc
wxmimperio
xuefeimiaoao
yehere
yexianxun
yikf
yongqian
yuqi
zhouyifan279
zwangsheng
毕博
佘志铭
胡安忠
袁福元

Release Notes - 1.6.1-incubating

2022 Nov 16

Highlight

Server

Fast fail the kyuubi connection if the engine application has been terminated
Transfer the TGetInfoReq to kyuubi engine side to check the connection valid and keep connection alive
GetInfo supports returning server/engine information
Handle zookeeper watch events

Client

Implement isSigned method of ResultSetMetaData for Kyuubi JDBC driver
Extend JDBC URL to support catalog

Engine

[SPARK] Fix scala NPE issue when adding non-local jar URI to class loader
[SPARK] Kyuubi should set env SPARK_USER_NAME for K8s deployment
[SPARK] Spark engines running on K8s give preference to IP for service discovery registration
[FLINK] Flink SQL engine fixes GetTables operation
[FLINK] Flink SQL engine supports getColumns operation
[FLINK] Flink SQL engine supports run DDL across versions

ChangeList

[KYUUBI #3744][Bug] InvalidACL appears in engine after upgrade to Kyuubi 1.6.0
[KYUUBI #3774] Output OpenSessionCount when Engine is waiting for exit because it reaches the Max lifetime
[KYUUBI #3746] Support to specify multiple config items with --conf in kyuubi command line
[KYUUBI #3748] Adding metrics kyuubi.metadata.request.opened and fix flaky test - MetadataManagerSuite
[KYUUBI #3745] Closing existing seekable reader when adding extra log
Revert “[KYUUBI #3385] Set spark.kubernetes.executor.podNamePrefix if missing”
[KYUUBI #3730] Change the parameter annotation of the runRemoveJarOperation method to Remove-jar operation instead of Add-jar operation
[KYUUBI #3726] [DOCS] Place comments above line to avoid mislead users
[KYUUBI #3720][FOLLOWUP] Using correct config version for CTL_BATCH_LOG_ON_FAILURE_TIMEOUT
[KYUUBI #3720] Support to show remaining logs with timeout if the batch failed
[KYUUBI #3728] [CI] Store kyuubi-spark-batch-submit.log for IT spark on kubernetes batch test
[KYUUBI #3714] Use ServicesResourceTransformer to concatenating service entries
[KYUUBI #3711] PlanOnly should output engine logs to log files
[KYUUBI #3710] Delete the swagger UI related resource files from the source folder
[KYUUBI #3703][DOCS] Fix typo in localLog REST API
[KYUUBI #3715] Remove unexpected mysql connector jar
[KYUUBI #3701] Allow JDBC authencation to connect auth db with no user name
[KYUUBI #3705] Add docs for JDBC authentication usage with in-memory database
[KYUUBI #3685] Fix flaky test BatchRestApiSuite - basic batch rest client with invalid user
[KYUUBI #3385] Set spark.kubernetes.executor.podNamePrefix if missing
[KYUUBI #3664] [SPARK][K8S][BATCH] Kyuubi should set env SPARK_USER_NAME on submitting Spark batch job to K8s
[KYUUBI #3670] [SPARK][K8S] Fix Spark master match rule on tagging Spark application
[KYUUBI #3624][REST] Fix description typo of session REST open api
[KYUUBI #3627] Support vanilla Jetty for Spark packaged by sbt
[KYUUBI #3543][FOLLOWUP][1.6] Handle zookeeper watch events
[KYUUBI #3543] Handle zookeeper watch events
[KYUUBI #3619] Enhance JAVA_HOME detection to support RPM
[KYUUBI #3600] [SPARK][K8S] Respect default static port for web UI on Spark K8s cluster mode
[KYUUBI #3597] Engine should prefer ip for registering on K8s cluster mode
[KYUUBI #3555] [FEATURE] Helm install help set kyuubi pod global env
[KYUUBI #3598] Beeline doesn’t work with -e option when started in background
[KYUUBI #3504] Extend JDBC URL to support catalog
[KYUUBI #3527][SPARK][FOLLOWUP] spark.kubernetes.executorEnv should be spark.executorEnv
[KYUUBI #3568] [Bug] [Doris Engine] Fix return decimal, date, timestamp using string
[KYUUBI #3579] Use strict path in dist script to avoid unexpected cp behavior
[KYUUBI #3372] [BUILD] Enable -x on build/dist
[KYUUBI #3571] Fix NPE for batch command line
[KYUUBI #3566] Shade Zookeeper into spark engine jar
[KYUUBI #3519] Flink SQL Engine - GetColumns Operation
[KYUUBI #3539] [FEATURE][TPCDS] Add white list help run the specified queries
[KYUUBI #3562] Fix the outdate docs index
[KYUUBI #3560] Flink SQL engine supports run DDL across versions
[KYUUBI #3521] [TEST] Fix Flink flaky test - select count
[KYUUBI #3549][FOLLOWUP] Simplify test
[KYUUBI #3549] Support query id in Flink engine
[KYUUBI #3544] Fix bin/kyuubi stop check
[KYUUBI #3406] [FOLLOWUP] Add create datasource table DDL usage to Pyspark docs
[KYUUBI #3547] Fix Flink statements results validation
[KYUUBI #3406] [Subtask] [Doc] Add PySpark client docs
[KYUUBI #3527] [SPARK] Kyuubi should set env SPARK_USER_NAME for K8s deployment
[KYUUBI #3451] Implement GetInfo for JDBC engine
[KYUUBI #3510] Reading an uninitialized log should return empty rowSet
[KYUUBI #3522][FOLLOWUP] Fix benchmark results-dir
[KYUUBI #3522] [BUG][TPCDS] Fix benchmark –results-dir
[KYUUBI #3498] Fix GetTables operation in Flink
[KYUUBI #3502] [DOCS] Fix typo and use beeline provided by Kyuubi
[KYUUBI #3484] Implement isSigned method of ResultSetMetaData for Kyuubi JDBC driver
[KYUUBI #3452] Implement GetInfo for Trino engine
[KYUUBI #3459] [BUILD] Share jars to reduce binary release tarball size
[KYUUBI #3051][FOLLOWUP] Remove redundant serverIpAddress method in SparkSessionImpl
[KYUUBI #3453] [MINOR] Remove unused spark.version property from the pom file of kyuubi-extension-spark-common module
[KYUUBI #3122] GetInfo supports return server/engine info
[KYUUBI #3318] Transfer the TGetInfoReq to kyuubi engine side to check the connection valid and keep connection alive
[KYUUBI #3064] Fix scala NPE issue when adding non-local jar URI to class loader
[KYUUBI #3408] [TEST] Remove the unstable flink process builder test case
[KYUUBI #3208] Fix Flaky Test - MetadataManagerSuite: metadata request metrics
[KYUUBI #3315] Fast fail the kyuubi connection if the engine application has been terminated

Credits

Last but not least, this release would not have been possible without the following contributors:

Bowen Liang
Cheng Pan
Fei Wang
Fu Chen
HYBG
Hanna Liashchuk
Luning Wang
Min Zhao
Paul Lin
SteNicholas
Wang Zhen
Yikf
asagjj
df_liu
hongdongdong
jiaoqingbo
senmiaoliu
sychen
ulysses-you
zhouyifan279
zwangsheng

Release Notes - 1.6.0-incubating

2022 Sep 6

Highlight

Server

Support Batch(Jar) Job submission
Support Thrift HTTP mode
Support Etcd as Discovery Service
Support JDBC authentication mechanism
Unify the authentication for Thrift and REST v1 API
Enhance metrics system

Client

Introduce REST client for Batch(Jar) Job submission
Decouple Hive and Hadoop dependencies from builtin JDBC driver
Builtin JDBC driver supports using keytab for Kerberos authentication
Enhance beeline to support Spark console progress bar

Engine

Fully verified with the latest Spark 3.1/3.2/3.3
Fully verified with the latest Flink 1.14/1.15
Fully verified with the Trino 363
Support Hive backend Engine (Beta)
Support JDBC backend Engine (Beta)

Plugin

Introduce Spark TPC-DS connector
Introduce Spark TPC-H connector
Introduce Spark Authz (Apache Ranger) plugin

Docs

User Documentation refactoring and improvement

ChangeList

Credits

Last but not least, this release would not have been possible without the following contributors:

Ada Wong
BoYang
Brandon Grams
Cheng Pan
Deng An
Fei Wang
Fu Chen
Kent Yao
Luning Wang
Mahmoud Bahaa
Martin Tzvetanov Grigorov
Min Zhao
Nick Song
ParisaTork
Paul Lin
Shengkai Fang
Nicholas Jiang
Tianlin Liao
Wang Zhen
Xiao Zhao
benjobs
gabrywu
guanhua.lgh
hongdongdong
jason
jiadongdong
jiaoqingbo
liangbowen
liaoyt
liuzy04
lxorc
maming
odone
packyan
pengqli
senmiaoliu
sychen
taokelu
Xiduo You
wangjunbo
Zhen Wang
winfys
xsun4
Vino Yang
yangrong688
yikf
zhangliang
zhangrenhua
zhanqian
zhouyifan279
zwangsheng

Release Notes - 1.5.2-incubating

2022 Jun 18

ChangeList

Credits

Last but not least, this release would not have been possible without the following contributors:

Cheng Pan
Fei Wang
jiaoqingbo
winfys
Xiduo You
yangrong688
Zhen Wang

Release Notes - 1.5.1-incubating

2022 Apr 21

ChangeList

Credits

Last but not least, this release would not have been possible without the following contributors:

Cheng Pan
Fei Wang
HYBG
Nick-0723
cxzl25
jiaoqingbo
ulysses

Release Notes - 1.5.0-incubating

2022 Mar 25

Highlight

Support Flink SQL Engine (Beta)
Support Trino/ Presto engine (Beta)
Support Running Scala for Spark SQL Engine
Upgrade Hive Service RPC Protocol to 3.1.2
Engine Launch Process Switch to Async by default
Add Session Conf Provider as Developer API
Correctness - Fix float precision loss for Spark SQL
Migrate from log4j1 to log4j2

ChangeList

Credits

Last but not least, this release would not have been possible without the following contributors:

Ada Wong
Brian Yue
Cheng Pan
Fei Wang
Fu Chen
HYBG-1126
I521853
Jeff Zhang
Kent Yao
Martin Tzvetanov Grigorov
Muru Muthusamy
Paul Lam
Paul Lin
Rishi Kumar Ray
Simon
SteNicholas
Tianlin Liao
Wang Zhen
fwang12
hongdongdong
hwk42
jiaoqingbo
jing10.gao
mask
minyk
odone
rayliu
senmiaoliu
shenbing
simon
sundapeng
sunfangbin
sychen
ulysses-you
wForget
xifeng yang
yanghua
zhenjiaguo
zhichaoleo
zhouyifan279
zwangsheng