Added offline token cost estimation with hard-isolated execution path#72
Conversation
|
Thanks for the suggestion! litellm is great but it's a pretty heavy dependency just for pricing lookups - it pulls in a lot of LLM client code we don't need. Found a lighter option: https://github.com/pydantic/genai-prices Or we could just vendor their slim JSON directly. Either way, we should add a disclaimer to the CLI output warning users that cost estimates are approximate and may not reflect current pricing. What do you think? |
|
I see, that makes sense. genai-prices looks like a good fit and avoids pulling unnecessary client code I kept pricing static initially to stay strictly within the original issue scope. |
Sounds good, I’m aligned. Thanks for the thoughtful work on this. |
|
Updated the PR to use genai-prices for dynamic pricing and added a clear disclaimer that estimates are approximate |
A bit late but will do! |
|
|
ff774ae to
87fa619
Compare
There are still some imports still in the code instead of in the top of the file. @saksham-jain177 |
Hey @brammittendorff, moved the remaining imports to the top of |
Will check |
|
LGTM; |
Implements
--estimate-costto provide an offline, pre-flight estimate of token usage and cost.Closes #57