Refactor multiple flows to enhance performance, boost scalability, and ensure stability. - #131
Refactor multiple flows to enhance performance, boost scalability, and ensure stability.#131luuquangvu wants to merge 295 commits into
Conversation
…s found in either the raw or cleaned history.
… for better Gemini compatibility.
…eeds METADATA_TTL_MINUTES.
…tion from being saved
…ystem instruction when reusing a session to save tokens.
… text file attachment - When multiple chunks are sent simultaneously, Google will immediately invalidate the access token and reject the request - When a prompt contains a structured format like JSON, splitting it can break the format and may cause the model to misunderstand the context - Another minor tweak as Copilot suggested
…e sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR.
…ith reusable sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR. - Introducing a new feature for real-time streaming responses. - Fully resolve the problem with reusable sessions. - Break down similar flow logic into helper functions. - All endpoints now support inline Markdown images. - Switch large prompts to use BytesIO to avoid reading and writing to disk.
…ith reusable sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR. - Introducing a new feature for real-time streaming responses. - Fully resolve the problem with reusable sessions. - Break down similar flow logic into helper functions. - All endpoints now support inline Markdown images. - Switch large prompts to use BytesIO to avoid reading and writing to disk. - Remove duplicate images when saving and responding.
…ith reusable sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR. - Introducing a new feature for real-time streaming responses. - Fully resolve the problem with reusable sessions. - Break down similar flow logic into helper functions. - All endpoints now support inline Markdown images. - Switch large prompts to use BytesIO to avoid reading and writing to disk. - Remove duplicate images when saving and responding.
…ith reusable sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR. - Introducing a new feature for real-time streaming responses. - Fully resolve the problem with reusable sessions. - Break down similar flow logic into helper functions. - All endpoints now support inline Markdown images. - Switch large prompts to use BytesIO to avoid reading and writing to disk. - Remove duplicate images when saving and responding.
…ith reusable sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR. - Introducing a new feature for real-time streaming responses. - Fully resolve the problem with reusable sessions. - Break down similar flow logic into helper functions. - All endpoints now support inline Markdown images. - Switch large prompts to use BytesIO to avoid reading and writing to disk. - Remove duplicate images when saving and responding.
…r to prevent CPU spikes when handling large output frames
|
The gemini-fastapi docker container starts successfully, initializes the Gemini client, and passes periodic health checks. However, when sending a POST request to the /v1/responses endpoint, the application crashes with a 500 Internal Server Error and throws an ASGI exception. |
|
@Vigno04 Because the log is truncated, I can't see the exact error. I can only guess from error 500 that the cause might be due to incorrect parameters you provided. Please provide more detailed information so I can understand why the error occurred. |
|
config are copied from the repo with just api key and stuff like that added, this is the logs: |
|
@Vigno04 I’ve made a small update to address the issue. Please try updating now to confirm if it resolves the problem. |
|
unfortunatelly not: |
|
@Vigno04 The two log files show different errors. Which client are you using to connect? |
|
I'm using openwebui sending "ciao" with no extra tool activa other than the base ones, i did docker compose down and docker compose up -d then sent the message and registered the logs |
|
@Vigno04 I’ve made a quick update to fix the issue. Go ahead and update now to see if it solves the problem. |
|
it was still broken but i may have found a fix, the problem was just with the responses format and i was able to fix it by changing the line 1562 in chat .py from: |
|
@Vigno04 I’ve made an update to fix another issue with the strict schema validator rejecting null fields. Go ahead and update now to see if it resolves the problem. |
|
fixed thx |
2 participants
This PR is still a work in progress and uses features that aren't yet officially available in the Gemini-API library, so we'll need to wait for the library's official update before merging. Feel free to try it out and share any feedback or report any issues you encounter. Thanks!
Here are some highlights of the changes: