fix(models): support float timestamps in CrawlStats

Modify CrawlStats class to handle both datetime and float timestamp formats for start_time and end_time fields. This change improves compatibility with different time formats while maintaining existing functionality.

Other minor changes:
- Add datetime import in async_dispatcher
- Update JsonElementExtractionStrategy kwargs handling

No breaking changes.
This commit is contained in:
UncleCode
2025-03-06 20:30:57 +08:00
parent 2327db6fdc
commit 29f7915b79
3 changed files with 38 additions and 7 deletions

View File

@@ -1064,7 +1064,7 @@ class JsonElementExtractionStrategy(ExtractionStrategy):
api_token (str): Legacy Parameter. API token for LLM provider
llm_config (LLMConfig): LLM configuration object
prompt (str, optional): Custom prompt template to use
**kwargs: Additional args passed to perform_completion_with_backoff
**kwargs: Additional args passed to LLM processor
Returns:
dict: Generated schema following the JsonElementExtractionStrategy format
@@ -1130,7 +1130,8 @@ In this scenario, use your best judgment to generate the schema. Try to maximize
prompt_with_variables="\n\n".join([system_message["content"], user_message["content"]]),
json_response = True,
api_token=llm_config.api_token,
**kwargs
base_url=llm_config.base_url,
extra_args=kwargs
)
# Extract and return schema