GPTCache doesn't seem to work with LlamaIndex implementation #361
-
I have a built a question answering app over my organization's data using LlamaIndex and OpenAI. I'm trying to use GPTCache with that, but it doesn't look like it's being used. Here' a snippet of code for an exact match test
The response times are as below (in seconds)
Do I need to do something different? Just for reference I am using the text-davinci-003 model. |
Beta Was this translation helpful? Give feedback.
Replies: 12 comments 4 replies
-
Can you show me the import codes |
Beta Was this translation helpful? Give feedback.
-
One thing I realized is that index.query might not be using the gptcache's openai implementation. |
Beta Was this translation helpful? Give feedback.
-
If you use the openai by the langchain, you can set the GPTCache to the langchain.llm_cache, like:
|
Beta Was this translation helpful? Give feedback.
-
sorry, it's a typo, it should be:
|
Beta Was this translation helpful? Give feedback.
-
Can you give a demo code and gptcache version? From the error stack, i guess the param usage problem. |
Beta Was this translation helpful? Give feedback.
-
or you can reference the langchain doc, link: https://python.langchain.com/en/latest/modules/models/llms/examples/llm_caching.html |
Beta Was this translation helpful? Give feedback.
-
My suspicion is that GPTCache is not completely compatible with LlamaIndex. I'm digging into the LlamaIndex code and then the adapt function in This is the code snippet where the error is coming from in adapter.py
|
Beta Was this translation helpful? Give feedback.
-
yes, maybe you need to change the pre_embendding_func, because the |
Beta Was this translation helpful? Give feedback.
-
If it's possible, please give a full demo code, and i will try to integrated the |
Beta Was this translation helpful? Give feedback.
-
I’ll figure it out and post it here
…On Thu, May 18, 2023 at 8:07 PM SimFG ***@***.***> wrote:
If it's possible, please give a full demo code, and i will try to
integrated the LlamaIndex
—
Reply to this email directly, view it on GitHub
<#361 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AL4L2Q5UXCAAOBDVD7HL6B3XGYXZ7ANCNFSM6AAAAAAYF6GHP4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
@ashwinr1980 Hello, I write a webpage qa example by using the GPTCache and llama_index, reference: https://gptcache.readthedocs.io/en/latest/bootcamp/llama_index/webpage_qa.html |
Beta Was this translation helpful? Give feedback.
-
This is very helpful. I'll try it today. One thing I do in my app is to save the index to disk, so that I can load the index json from my local folder to save time (don't have to recreate index everytime) So the first time I do this
And in subsequent calls I just do this
I'm going to try and if I can insert the cache in here somewhere. |
Beta Was this translation helpful? Give feedback.
@ashwinr1980 Hello, I write a webpage qa example by using the GPTCache and llama_index, reference: https://gptcache.readthedocs.io/en/latest/bootcamp/llama_index/webpage_qa.html