utils.llm_cache¶
- class celi_framework.utils.llm_cache.LLMCache(simulate_live=False)
Bases:
object- async cache_llm_response(response, **kwargs)
- async check_llm_cache(**kwargs)
- async close()
- async connection()
- static randomize_delay(starting_delay)
- async celi_framework.utils.llm_cache.disable_llm_caching()¶
- celi_framework.utils.llm_cache.enable_llm_caching(simulate_live=False)¶
Enables LLM caching for all calls using the celi_framework.utils.llms functions.
If simulate_live is set to true, imposes a delay on each call to simulate live calls. Useful when testing timing or for demos.
- celi_framework.utils.llm_cache.generate_hash_id(obj)¶
Generates a unique template ID based on the hash of stringable object.
- Parameters:
config (stringable type) – An object that can be turned into a string to hash.
- Returns:
A unique template ID.
- Return type:
str
- celi_framework.utils.llm_cache.get_cache_dir()¶
Returns the cache directory. Uses the standard XDG conventions of ~/.cache/celi unless XDG_CACHE_HOME is set.
Ensures that the cache directory is created.
- celi_framework.utils.llm_cache.get_celi_llm_cache()¶