Asynchronous file upload (google.generativeai.upload_file) and other blocking methods?

[zavocc]

Description of the feature request:

Currently google.generativeai.upload_file is a blocking function, it would be nice if there are async versions of these methods along with other I/O related methods like get_file and delete_file though upload_file can be made asynchronous if uploading lets say 100mb of file.

What problem are you trying to solve with this feature?

Adding asynchronous versions of upload file methods is good in applications where it is async-driven (e.g. Discord bots, web) which currently using this to utilize multimodal features can potentially block the entire script which is not good for such applications

Any other information you'd like to share?

Right now I'm using something like asyncio.to_thread to make it asynchronous, but offering a native async methods would be nice

Right now there are only two async methods with this SDK are generate_content and send_message
Although it would also be a good idea if adding async variant to SDK as well

[MarkDaoust]

asyncio.to_thread, interesting. That's new to me.

The implementation of that upload function is kind of messy (see #327). When I was writing it there just didn't seem to be a clear way to simply make an async implementation.

Am I reading this right? (Once 327 is fixed) we can make the implementation for upload_file_async just:

async def upload_file_async(*args, **kwargs)
  return await asyncio.to_thread(blocking_function, *args, **kwargs)

[zavocc]

Either https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor or https://docs.python.org/3/library/asyncio-task.html#asyncio.to_thread can be used... Both makes methods blocking, but ngl the python SDK is quite messy to deal with and I'm not entirely sure if this is the correct way/workaround for making it async but it works Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Mark Daoust ***@***.***> Sent: Thursday, October 3, 2024 8:00:09 AM To: google-gemini/generative-ai-python ***@***.***> Cc: WMCB Tech (marcusz) ***@***.***>; Author ***@***.***> Subject: Re: [google-gemini/generative-ai-python] Asynchronous file upload (`google.generativeai.upload_file`) and other blocking methods? (Issue #564) asyncio.to_thread, interesting. That's new to me. The implementation of that upload function is kind of messy (see #327<#327>). When I was writing it there just didn't seem to be a clear way to simply make an async implementation. Am I reading this right? (Once 327 is fixed) we can make the implementation for upload_file_async just: async def upload_file_async(*args, **kwargs) return await asyncio.to_thread(blocking_function, *args, **kwargs) — Reply to this email directly, view it on GitHub<#564 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A4QQBQM3OSFCXFIKOZXWRFLZZSCITAVCNFSM6AAAAABOTSP2SKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJQGA2DONJUGM>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[MarkDaoust]

#586

[gokturkDev]

I occasionally get the following error when using asyncio.to_thread
I tried hard to diagnose what exactly is triggering this but in vain, it sometimes works and sometimes breaks for the same image.
Eventually mitigated it with a retry mechanism.
Posting this because it might be related @MarkDaoust

 Encountered error: 503 IOCP/Socket: Connection reset (An existing connection was forcibly closed by the remote host.

 -- 10054