Unclecode 53d1176d53 chore: Update extraction strategy to support GPU, MPS, and CPU, add batch processing for CPU devices

2024-05-19 16:18:58 +00:00

..

models/onnx

- Add ONNX embedding model for CPU devices, Update the similarithy threshold, improve the embedding speed.

2024-05-19 22:30:10 +08:00

__init__.py

Change the project folder name from crawler to crawl4ai

2024-05-09 22:16:28 +08:00

chunking_strategy.py

chore: Update extraction strategy to support GPU, MPS, and CPU, add batch processing for CPU devices

2024-05-19 16:18:58 +00:00

config.py

Update:

2024-05-16 19:50:20 +08:00

crawler_strategy.py

chore: Update extraction strategy to support GPU, MPS, and CPU, add batch processing for CPU devices

2024-05-19 16:18:58 +00:00

database.py

Update:

2024-05-16 17:31:44 +08:00

extraction_strategy.py

chore: Update extraction strategy to support GPU, MPS, and CPU, add batch processing for CPU devices

2024-05-19 16:18:58 +00:00

model_loader.py

- Add ONNX embedding model for CPU devices, Update the similarithy threshold, improve the embedding speed.

2024-05-19 22:30:10 +08:00

models.py

Update:

2024-05-16 17:31:44 +08:00

onnx_embedding.py

- Add ONNX embedding model for CPU devices, Update the similarithy threshold, improve the embedding speed.

2024-05-19 22:30:10 +08:00

prompts.py

Update:

2024-05-16 17:31:44 +08:00

train.py

Remove dependency on Spacy model.

2024-05-17 15:08:03 +08:00

utils.py

Update:

2024-05-16 19:50:20 +08:00

web_crawler.py

chore: Add support for GPU, MPS, and CPU

2024-05-17 21:56:13 +08:00