memory-mapped faiss indices

[alexanderguzhva]

Summary

A prototype of memory-mapped files that contain Faiss Indices (IndexFlatCodes and HNSW at this moment).

zilliztech/knowhere#996

@mdouze Could you please give your comments on the approach, just in case I've missed some important corner cases? Despite the link is from the Faiss fork, it should be pretty straightforward to understand the changes. Basically, I replace std::vector<> with a custom container that supports mmap-ping.

The PR for Faiss will be implemented as soon as we agree on the details.
Thanks.

[mdouze]

Yes that seems very good.
I started a similar effort but never got a chance to finalize it, see . So please go ahead if you want to commit it.
For ref, this is shipit-facebookresearch-faiss-D56685773.patch what I was working on.