[Bug]: <failure to create index SQL Error [XX000]: ERROR: assertion failed: meta_page.get_num_dimensions_to_index() > 0 && <= 2000>

[nestorSainsburys]

What happened?

I use a docker image timescale/timescaledb-ha:pg17

I've created a table for embeddings:

CREATE TABLE IF NOT EXISTS products.products_embedding ( embedding_id int8 NOT NULL PRIMARY KEY, "text" varchar(255), embedding vector(3072), metadata json );

I've installed a vectorscale extension:

CREATE EXTENSION IF NOT EXISTS vectorscale CASCADE;

That all works well and I can see the extension created:

SELECT *
FROM pg_extension; - that lists my vectorscale extension

Now, as per documentation, when I attempt to creatre an index, as in:

CREATE INDEX argos_products_embedding_idx ON products.products_embedding
USING diskann (embedding) WITH(num_neighbors=20);

, that blows in my face with the following error:

SQL Error [XX000]: ERROR: assertion failed: meta_page.get_num_dimensions_to_index() > 0 && meta_page.get_num_dimensions_to_index() <= 2000

I tried all sort of values for num_neighbours between 10 and 100, or omitting this all together - still getting the same error.
This has also been reported here: https://stackoverflow.com/questions/79317581/pgvectorscale-failure-to-create-index-sql-error-xx000-error-assertion-faile?noredirect=1#comment139869940_79317581

pgvectorscale extension affected

0.5.1

PostgreSQL version used

17

What operating system did you use?

MacOs 14.6

What installation method did you use?

Docker

What platform did you run on?

On prem/Self-hosted

Relevant log output and stack trace

SQL Error [XX000]: ERROR: assertion failed: meta_page.get_num_dimensions_to_index() > 0 &&
    meta_page.get_num_dimensions_to_index() <= 2000

How can we reproduce the bug?

The steps are in the description

Are you going to work on the bugfix?

🆘 No, could someone else please work on the bugfix?

[Mickael-van-der-Beek]

@nestorSainsburys I'm not part of the project but I suspect, based on the error message, that your problem is coming from the fact that you are creating a table with an embedding column with a VECTOR type containing more than 2000 dimensions which is the currently maximum supported by PGVectorScale.

[cevian]

should be solved by #181, and be available in next release