ayrisdev/crawl4ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: Update import statements to use crawl4ai package

Browse Source
This commit is contained in:
unclecode
2024-05-09 22:17:15 +08:00
parent 3ff1d15702
commit c71dd9189b
1 changed files with 6 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@@ -35,8 +35,8 @@ pip install git+https://github.com/unclecode/crawl4ai.git
2. Import the necessary modules in your Python script: 2. Import the necessary modules in your Python script:
```python ```python
from crawler.web_crawler import WebCrawler from crawl4ai.web_crawler import WebCrawler
from crawler.models import UrlModel from crawl4ai.models import UrlModel
``` ```
3. Use the Crawl4AI library in your project as needed. Refer to the [Usage with Python](#usage-with-python-) section for more details. 3. Use the Crawl4AI library in your project as needed. Refer to the [Usage with Python](#usage-with-python-) section for more details.
@@ -135,8 +135,8 @@ pip install -r requirements.txt
2. Import the necessary modules and initialize the `WebCrawler`: 2. Import the necessary modules and initialize the `WebCrawler`:
```python ```python
from crawler.web_crawler import WebCrawler from crawl4ai.web_crawler import WebCrawler
from crawler.models import UrlModel from crawl4ai.models import UrlModel
import os import os
crawler = WebCrawler(db_path='crawler_data.db') crawler = WebCrawler(db_path='crawler_data.db')
@@ -145,7 +145,7 @@ crawler = WebCrawler(db_path='crawler_data.db')
3. Fetch a single page: 3. Fetch a single page:
```python ```python
single_url = UrlModel(url='https://kidocode.com', forced=True) single_url = UrlModel(url='https://kidocode.com', forced=True)
result = crawler.fetch_page( result = crawl4ai.fetch_page(
single_url, single_url,
provider= "openai/gpt-3.5-turbo", provider= "openai/gpt-3.5-turbo",
api_token = os.getenv('OPENAI_API_KEY'), api_token = os.getenv('OPENAI_API_KEY'),
@@ -161,7 +161,7 @@ urls = [
UrlModel(url='http://example.com', forced=False), UrlModel(url='http://example.com', forced=False),
UrlModel(url='http://example.org', forced=False) UrlModel(url='http://example.org', forced=False)
] ]
results = crawler.fetch_pages( results = crawl4ai.fetch_pages(
urls, urls,
provider= "openai/gpt-3.5-turbo", provider= "openai/gpt-3.5-turbo",
api_token = os.getenv('OPENAI_API_KEY'), api_token = os.getenv('OPENAI_API_KEY'),