feat: make LLM backoff configurable end-to-end
- extend LLMConfig with backoff delay/attempt/factor fields and thread them through LLMExtractionStrategy, LLMContentFilter, table extraction, and Docker API handlers - expose the backoff parameter knobs on perform_completion_with_backoff/aperform_completion_with_backoff and document them in the md_v2 guides
This commit is contained in:
@@ -108,7 +108,10 @@ async def handle_llm_qa(
|
||||
prompt_with_variables=prompt,
|
||||
api_token=get_llm_api_key(config), # Returns None to let litellm handle it
|
||||
temperature=get_llm_temperature(config),
|
||||
base_url=get_llm_base_url(config)
|
||||
base_url=get_llm_base_url(config),
|
||||
base_delay=config["llm"].get("backoff_base_delay", 2),
|
||||
max_attempts=config["llm"].get("backoff_max_attempts", 3),
|
||||
exponential_factor=config["llm"].get("backoff_exponential_factor", 2)
|
||||
)
|
||||
|
||||
return response.choices[0].message.content
|
||||
|
||||
Reference in New Issue
Block a user