Release 2.18.0

[pallavisontakke]

This release introduces the ability to add secondary indexes to the columnstore, improves group by and filtering performance through columnstore vectorization, and contains the highly upvoted community request of transition table support. We recommend that you upgrade at the next available opportunity.

Highlighted features in TimescaleDB v2.17.0

• The ability to add secondary indexes to the columnstore through the new hypercore table access method.
• Significant performance improvements through vectorization (SIMD) for aggregations using a group by with one column and/or using a filter clause when querying the columnstore.
• Hypertables support triggers for transition tables, which is one of the most upvoted community feature requests.
• Updated methods to manage Timescale's hybrid row-columnar store (hypercore) that highlight the usage of the columnstore which includes both an optimized columnar format as well as compression.

Dropping support for Bitnami images

After the recent change in Bitnami’s LTS support policy, we are no longer building Bitnami images for TimescaleDB. We recommend using the official TimescaleDB Docker image

Deprecation Notice

We are deprecating the following parameters, functions, procedures and views. They will be removed with the next major release of TimescaleDB. Please find the replacements in the table below:

Deprecated	Replacement	Type
decompress_chunk	convert_to_rowstore	Procedure
compress_chunk	convert_to_columnstore	Procedure
add_compression_policy	add_columnstore_policy	Function
remove_compression_policy	remove_columnstore_policy	Function
hypertable_compression_stats	hypertable_columnstore_stats	Function
chunk_compression_stats	chunk_columnstore_stats	Function
hypertable_compression_settings	hypertable_columnstore_settings	View
chunk_compression_settings	chunk_columnstore_settings	View
compression_settings	columnstore_settings	View
timescaledb.compress	timescaledb.enable_columnstore	Parameter
timescaledb.compress_segmentby	timescaledb.segmentby	Parameter
timescaledb.compress_orderby	timescaledb.orderby	Parameter

Features

• Vectorized aggregation with grouping by one fixed-size column #7341: Vectorized aggregation with grouping by one fixed-size by-value compressed column (such as arithmetic types).
• Table access method for compressed hypertables #7104: Hypercore table access method.
• Add support for transition tables #6901: Add hypertable support for transition tables.
• Optimize segmentwise recompression #7482: Optimize recompression of partially compressed chunks.
• Vectorize aggregate FILTER clause #7458: Support vectorized aggregation with aggregate filter clauses that are also vectorizable.
• Add support for merging chunks #7433: Add support for merging chunks.
• Push down ORDER BY for realtime caggs #7271: Push down order by in real-time continuous aggregate queries.
• Allow DROP NOT NULL on column of compressed table #7455: Support drop not null on compressed hypertables.
• Add ALTER TABLE SET ACCESS METHOD support #7295: Support alter table set access method on hypertable.
• Change parameter name to enable Hypercore TAM #7411: Change parameter name to enable hypercore table access method.
• Add index creation on orderby columns #7436: Add index creation on order by columns.
• Add aliases for compression functions and views #7443: Add hypercore function and view aliases.
• Add optional force argument to refresh_continuous_aggregate #7521: Add optional force argument to refresh_continuous_aggregate.
• Apply sort transform optimizations to compressed chunks #7528: Transform sorting on time_bucket to sorting on time for compressed chunks in some cases.
• Add hint when hypertable creation fails #7565: Add hint when hypertable creation fails.
• Remove custom hashagg code #7390: Disable custom hashagg planner code.
• Add a cagg config parameter to override the instance-level timescaledb.enable_tiered_reads GUC #7587: Add include_tiered_data parameter to add_continuous_aggregate_policy API.
• Prevent building against pg versions with broken ABI #7486: Prevent building against PostgreSQL versions with broken ABI.
• Add GUC for hypercore_use_access_method default #7412: Add GUC for the hypercore_use_access_method default.
• Add GUC for segmentwise recompression #7413: Add GUC for segmentwise recompression.

Bugfixes

• Remove obsolete job #7378: Remove obsolete job referencing policy_job_error_retention.
• Update bgw job table when altering procedure #7409: Update bgw_job table when altering procedure.
• [Bug]: "Aggregated Compressed Column Not Found" Error on Aggregation Query in TimescaleDB #7410: Fix the aggregated compressed column not found error on aggregation query.
• Fix datetime parse error in chunk constraint creation #7426: Fix datetime parsing error in chunk constraint creation.
• Verify that heap tuple is valid before using #7432: Verify that the heap tuple is valid before using.
• Fix segfault when setting chunk replica identity #7434: Fix the segfault when internally setting the replica identity for a given chunk.
• Emit error on transition table trigger for chunks #7488: Emit error for transition table trigger on chunks.
• Fix error: invalid child of chunk append #7514: Fix the error: invalid child of chunk append.
• Fix CAgg migration performance regression #7517: Fix the performance regression on the cagg_migrate procedure.
• Restart scheduler on error #7527: Restart scheduler on error.
• Fix NULL handling for in-memory tuple filtering #7557: Fix null handling for in-memory tuple filtering.
• Improve transaction check in refresh_cagg #7566: Improve transaction check in CAGG refresh.
• Fix NaN-handling for vectorized aggregation #7584: Fix NaN-handling for vectorized aggregation.

Thanks

• @bharrisau for reporting the segfault when creating chunks.
• @k-rus for suggesting that we add a hint when hypertable creation fails.
• @pgloader for reporting the issue in an internal background job.
• @staticlibs for sending the pull request that improves the transaction check in CAGG refresh.
• @uasiddiqi for reporting the aggregated compressed column not found error.

Disable-check: force-changelog-file

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.32%. Comparing base (59f50f2) to head (996eb05).
Report is 710 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7596      +/-   ##
==========================================
+ Coverage   80.06%   82.32%   +2.25%     
==========================================
  Files         190      239      +49     
  Lines       37181    44171    +6990     
  Branches     9450    11104    +1654     
==========================================
+ Hits        29770    36363    +6593     
- Misses       2997     3426     +429     
+ Partials     4414     4382      -32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[billy-the-fish]

Couple of small suggestions.

[svenklemm]

We usually add a section to the top with some additional context for the main new feature of a minor release

[svenklemm]

I think we are missing a note about bitnami deprecation.

[atovpeko]

A few nitpicks

[philkra]

LGTM

[philkra]

@svenklemm addressed #7596 (comment) and #7596 (comment)

[atovpeko]

Suggested change

	* #7598: Match the Postgres NaN comparison behavior in WHERE clause over compressed tables.
	* #7598: Match the PostgreSQL NaN comparison behavior in WHERE clause over compressed tables.