Release Notes - 1.8.1

Server

• [KYUUBI #6058] Make Jetty server stop timeout configurable
• [KYUUBI #6048] Assign serviceNode and add volatile for variables
• [KYUUBI #6045] Sync the AdminRestApi with the AdminResource Apis
• [KYUUBI #6028] Exited spark-submit process should not block batch submit queue
• [KYUUBI #6001] Fix RESTful protocol security enabled evaluation
• [KYUUBI #5999] loads the interval of sessionConfCache from configurable file in FileSessionConfAdvisor
• [KYUUBI #5990] Always take the first declared SASL/PLAIN auth type
• [KYUUBI #5922] Enhance YARN app state judgment
• [KYUUBI #5900] Get the engine service node to delete instead of delete engine space
• [KYUUBI #5900] Support to deregister the engine on open failure
• [KYUUBI #5892] Metadata store JDBC url supports <KYUUBI_HOME> variable
• [KYUUBI #5869] Add volatile to variables in KyuubiSessionImpl that may be operated by multiple threads
• [KYUUBI #5244] Make engineAliveMaxFailCount configurable
• [KYUUBI #5795] Using correct time unit for pod cleanup executor
• [KYUUBI #5832] Always perform closing action in OperationLog to avoid fd leak
• [KYUUBI #5853] Ensure closing the operation log for batch submission in fast-failed case
• [KYUUBI #5833] Rename service registered endpoint key from serviceUri to serverUri
• [KYUUBI #5795] Support to cleanup the spark driver pod periodically
• [KYUUBI #5680] Optimize Spark engine pod name generation
• [KYUUBI #5807] Simple norm the use of kyuubi.operation.result.arrow.timestampAsString and kyuubi.operation.result.format
• [KYUUBI #5767] Extract common utils for assembling key value pairs with config option prefix in processbuilder
• [KYUUBI #5765] Avoid array copy in generating process builder commands
• [KYUUBI #5771] Cleanup the terminated app info trigger before clear kubernetes clients
• [KYUUBI #5711] Fix typo for container states audit
• [KYUUBI #5711] Fix typo and audit pod and container states
• [KYUUBI #4847] Check engine alive for KyuubiSessionImpl only
• [KYUUBI #5761] Pretty ProcBuilder generated command
• [KYUUBI #5754] Fix engine id of the engine probe failures logging messages
• [KYUUBI #5730] Tolerate execeptions for periodical scheduled tasks
• [KYUUBI #5210] Close batch operation log even batch is in cancel state
• [KYUUBI #5731] Support to cleanup the spark driver pod with specified cleanup strategy
• [KYUUBI #5725] Handle the session exception for all kyuubi operations during running
• [KYUUBI #2503] Catch all exceptions for sessions timeout check task
• [KYUUBI #5717] Infer the proxy user automatically for delete batch operation
• [KYUUBI #5711] Support to build application state and error from pod container
• [KYUUBI #5714] Fix NPE when deleting terminated application pod
• [KYUUBI #5714] Support to cleanup the spark driver pod after application terminates for retain period
• [KYUUBI #5696] Return remote session id and remote operation id
• [KYUUBI #5654] Introduce new config to forcibly rewrite pod name
• [KYUUBI #5438] Add common method to get session level config

Spark Engine, Connectors and Plugins

• [KYUUBI #5952] Disconnect connections without running operations after engine maxlife time graceful period
• [KYUUBI #6018] Speed up GetTables operation for Spark session catalog
• [KYUUBI #5966] Error occurs on clicking on the Spark UI Kyuubi Query Engine tab
• [KYUUBI #5831] Pre-allocate array list capacity for TColumns and TColumnValues in TRowSet generation
• [KYUUBI #5811] Reuse time formatters instance in value serialization of TRowSet generation
• [KYUUBI #5766] Default spark.yarn.maxAppAttempts to 1 for spark engine
• [KYUUBI #5750] Fix the inaccurate issue of obtaining COLUMN_SIZE in the decimal field jdbc of spark engine
• [KYUUBI #5331] Spark engine should throw an exception when it fails to start
• [KYUUBI #5991] Error on reading Atlas properties composed of multi values

Other Engines

• [KYUUBI #5977] Fix argument position on constructing Trino ClientSession
• [KYUUBI #5969] Trino engine add default schema
• [KYUUBI #5878] Hive engine should recognize hiveserver2-site.xml
• [KYUUBI #5756] Introduce specified initialized SQL to every engine
• [KYUUBI #5814] Skip the max-rows limitation if the statement is not DQL in Flink engine
• [KYUUBI #5799] Fix fetch timeout in session conf doesn’t support ISO-8601 in Flink engine
• [KYUUBI #5789] Flink engine kyuubi.session.engine.flink.fetch.timeout parameter on the server is not effective
• [KYUUBI #5784] Implement HiveTBinaryFrontendService#RenewDelegationToken
• [KYUUBI #5782] Flink Engine GetInfo support CLI_ODBC_KEYWORDS
• [KYUUBI #5778] Hive engine shall respect hive.server2.enable.doAs
• [KYUUBI #5777] Include hive-service-rpc in Hive engine jar
• [KYUUBI #5507] Support Initialize SQL in Flink Engine
• [KYUUBI #5280] Hive engine compatible with Hive 2.3

Client

• [KYUUBI #6036] JDBC driver conditional sets fetchSize on opening session
• [KYUUBI #6016] Conditional call GetInfo CLI_ODBC_KEYWORDS to restore compatible with lower version Kyuubi and HS2
• [KYUUBI #5961] Prevent NPE when checking ticket cache exists
• [KYUUBI #5961] Support to specify client kerberosAuthType as fromTicketCache
• [KYUUBI #5849] Incorrectly parse JDBC URL while variable includes colon
• [KYUUBI #5822] Fix the usage msg of KyuubiBeeLine
• [KYUUBI #5752] JDBC driver implements getDate, getTimestamp and getTime with Calendar in ResultSet
• [KYUUBI #5713] Backport HIVE-27271: Client connection to HS2 fails when transportMode=http, ssl=true, sslTrustStore specified without trustStorePassword in the JDBC URL

Build, Test and Misc

• [KYUUBI #5981] Deploy Spark Hive connector with Scala 2.13 to Maven Central
• [KYUUBI #6047] Free up disk space
• [KYUUBI #5987] Always export KYUUBI_HOME in load-kyuubi-env.sh
• [KYUUBI #5987] Enhance KYUUBI_HOME detection in shell script to handle softlink cases
• [KYUUBI #6000] Modify the incorrect configuration file in the trino-cli documentation
• [KYUUBI #5978] Canonicalize Trino IT in GitHub Action workflow
• [KYUUBI #5962] Bump Flink 1.17.1 to 1.17.2
• [KYUUBI #5953] Update NOTICE
• [KYUUBI #5951] Improve doc for Session Conf Advisor
• [KYUUBI #5933] Happy New Year 2024
• [KYUUBI #5244] Corrected ENGINE_ALIVE_MAX_FAILURES conf version
• [KYUUBI #5856] Bump Spark 3.4.2
• [KYUUBI #5857] Dependencies CI check when dev/dependencyList changes
• [KYUUBI #5837] Only enable Jacoco with explicit profile
• [KYUUBI #5827] Fix wrong test code about directory lineage
• [KYUUBI #5819] Fix rst syntax of missing blank line after code-block
• [KYUUBI #5802] Add scala-2.13 as supported Scala profile to the docs of building from source
• [KYUUBI #5801] Add spark-3.4 and spark-3.5 as supported Spark profiles to the docs of building from source
• [KYUUBI #5785] Fix flaky test - JpsApplicationOperation with spark local mode
• [KYUUBI #5737] Correct JDBC URL demo in Flink docs
• [KYUUBI #5617] Make flaky test spnego batch rest client reliable
• [KYUUBI #5720] Fix flaky test in TPCHQuerySuite
• [KYUUBI #5697] Fix incorrect url reference in code and docs
• [KYUUBI #5599] Correct the parameter name of fetching op log in RESTful API
• [KYUUBI #5641] Bump Kyuubi 1.8.0 in Helm charts
• [KYUUBI #5640] Upgrade playground to Kyuubi 1.8.0 and Spark 3.4.1
• [KYUUBI #5637] Add known contributor translation
• [KYUUBI #5624] Remove npm run build warning about NODE_ENV=production
• [KYUUBI #5634] Smoother the icon animation on collapsing sidebar
• [KYUUBI #5626] Fix sytle linting violation in web ui

Credits

Last but not least, this release would not have been possible without the following contributors:

• Shaoyun Chen
• Chao Chen
• Flyangz
• Pengqi Li
• Bowen Liang
• Paul Lin
• Senmiao Liu
• Xiao Liu
• Peiyue Liu
• Junjie Ma
• Ocean22
• Cheng Pan
• Gianluca Principini
• Zhiming She
• SwordyZhao
• William Tong
• Fei Wang
• Zhen Wang
• Tao Wang
• Zeyu Wang
• Kang Wang
• Binjie Yang
• Kaifei Yi
• David Yuan
• Yuwei Zhan
• He Zhao
• Yifan Zhou
• Mingliang Zhu
• Yi Zhu
• liaoyt
• mrtisttt