Commit Graph

Select branches

Hide Pull Requests

0.3.5

0.3.6

0.3.7

0.3.72

0.3.73

0.3.74

0.3.742

0.3.743

0.3.744

0.3.745

0.3.75

0.4.0

0.4.1

0.4.2

2025-JUN-1

add-claude-github-actions-1759553116682

bug/proxy_config

bugfix/arun-many-cdp-managed-browser

claude/fix-update-pyopenssl-security-011CUPexU25DkNvoxfu5ZrnB

claude/implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp

coderabbitai/docstrings/14vTVzYa3bH06l5wYNY9jTghrrj9FxxWL

codex/add-httpx-and-https-http2]-packages

codex/add-memory_wait_timeout-parameter-to-memoryadaptivedispatche

codex/add-use_stemming-parameter-to-bm25contentfiler

codex/add-vnc-streaming-endpoint-to-docker-server

codex/find-and-fix-a-bug

codex/fix-indexerror-in-browser-manager-py-with-use-managed-browse

copilot/modify-page-creation-and-logging

deploy

develop

devin/1748137705-fix-bm25contentfilter-docs

docker-test

docker/add_features

docker/base_config_overrides

docker/fix_sig

docs

docs-llm-strategies-update

docs-proxy-security

extract-media

feat/ahmed_dev

feat/undetected-browser

feature/agent-oai

feature/async-llm-extaction

feature/c4a-script

feature/configHealthMonitor

feature/content-filter

feature/content-filter-nasrin-1

feature/docker-cluster

feature/docker-hooks

feature/docker-llm-parameters

feature/marketplace-sponsor-logo

feature/nasrin-cli-deep-crawl

feature/scraper

feature/scraping-strategy

feature/telemetry

fix-async-url-seeder-redirect-verification

fix-cors-disable-web-security

fix/adaptive-crawler-llm-config

fix/async-llm-extraction-arunMany

fix/case_senstive_params

fix/cdp

fix/configurable-backoff

fix/deep-crawl-scoring

fix/deep-crawl-scoring-priority

fix/deprecated_pydantic

fix/dfs_deep_crawling

fix/docker

fix/docker-filter

fix/docker-jwt

fix/docker-llmEnvFile

fix/exit_with_q

fix/https-reditrect

fix/json-infinity-serialization

fix/linkPreviewScoring

fix/marketplace

fix/n-playwright-stealth

fix/playwright-stealth

fix/proxy_deprecation

fix/relative_url

fix/release-notes-demo-code

fix/request-crawl-stream

fix/serialize-proxy-config

fix/sitemap_seeder

fix/viewport_in_managed_browser

format-inline-tags

hooks

image-description

image-filterizer

implement-webhook-crawl-feature-011CULZY1Jy8N5MUkZqXkRVp

main

main-0.3.7

main-1

main-75

main-img-captionify

main-v0.2.72

merge-pr971

new-release-0.0.2

new-release-0.0.2-no-spacy

next

next-2-batch-crawl

next-JUN

next-MAY

next-alpine-docker

next-browser-farm

patch/generate_schema

pdf_processing

proxy-support

pull-84

release/v0.7.0

release/v0.7.1

release/v0.7.2

release/v0.7.3

release/v0.7.4

release/v0.7.5

release/v0.7.6

release/v0.7.7

release/v0.7.8

release/v0.8.0

run-many-deep-crawling

scraper-uc

scrapper

sponsors/thor_data

ssh-server

staging

unclecode-patch-1

unclecode-patch-2

unclecode-patch-3

unclecode-patch-4

unclecode-patch-5

unclecode-patch-6

unclecode-patch-7

unclecode-patch-8

unclecode/issue157

unclecode/issue167

v0.2.74

v0.2.76

v0.4.24

v0.4.241

v0.4.242

v0.4.243

v0.5.5

vr0.4.244

vr0.4.245

vr0.4.246

vr0.4.267

vr0.4.3b1

vr0.4.3b2

vr0.4.3b3

vr0.5.0.post1

vr0.5.0.post5

#1004

#1030

#1054

#1058

#1058

#1059

#1059

#1060

#1062

#1065

#1065

#1068

#1073

#1074

#1077

#1078

#108

#1081

#1081

#1083

#1085

#1085

#109

#1090

#1093

#1093

#1094

#1098

#1098

#1100

#1102

#1104

#1106

#1106

#1107

#1108

#1110

#1113

#1122

#1123

#1124

#1124

#1133

#1137

#1140

#1145

#1152

#1155

#1155

#1156

#1157

#1159

#1159

#1161

#1170

#1175

#1179

#1179

#1180

#1180

#1184

#1186

#119

#1192

#1193

#1195

#1200

#1200

#1207

#1207

#1208

#1209

#1210

#1211

#1211

#1212

#1214

#1220

#1220

#1223

#1223

#1225

#1225

#1232

#1234

#1234

#1238

#1238

#1239

#1245

#1249

#125

#1255

#1255

#1263

#1263

#1265

#1266

#1267

#1272

#1272

#1274

#128

#1281

#1282

#1285

#1289

#1289

#129

#1290

#1290

#1296

#13

#1303

#1304

#1305

#1307

#1308

#1308

#1313

#1319

#1334

#1334

#1336

#1337

#1339

#134

#135

#1351

#1356

#1358

#1361

#1364

#1366

#1368

#1369

#1371

#1372

#1373

#1376

#1378

#1381

#1383

#1384

#1386

#1387

#1388

#1389

#139

#1390

#1393

#1395

#1398

#1399

#14

#1402

#1408

#1413

#1416

#1416

#1417

#1417

#1420

#1422

#1425

#1425

#1426

#1432

#1433

#1435

#1436

#1440

#1441

#1444

#1447

#1448

#1450

#1451

#1454

#1463

#1464

#1465

#1467

#1469

#1470

#1471

#1478

#1482

#1483

#1483

#1486

#1488

#1488

#149

#1494

#1494

#1495

#1496

#1497

#1501

#1508

#1513

#1514

#1518

#1519

#1525

#1527

#1528

#1529

#1530

#1531

#1532

#1533

#1533

#1535

#1536

#1537

#1539

#1546

#1547

#1548

#1550

#1554

#1555

#1556

#1557

#1558

#1560

#1565

#1568

#1569

#1570

#1572

#1572

#1576

#158

#1580

#1580

#1588

#1589

#1590

#1592

#1592

#1595

#1596

#1597

#1598

#1599

#1600

#1605

#1607

#1609

#1612

#1613

#1617

#1617

#1619

#1620

#1622

#1622

#1623

#1624

#1628

#1630

#1633

#1637

#1640

#1641

#1643

#1645

#1648

#1650

#1650

#1653

#1655

#1661

#1662

#1667

#1668

#1668

#1674

#1674

#1676

#1677

#1681

#1683

#1683

#1685

#1689

#1689

#169

#1694

#1696

#1697

#1698

#1700

#1702

#1702

#1703

#1706

#1706

#1707

#1707

#1710

#1712

#1713

#1714

#1715

#1715

#1716

#1716

#1717

#1718

#1719

#172

#1720

#1721

#1722

#1722

#1723

#1724

#1729

#1729

#1730

#1730

#1733

#1734

#1734

#1744

#1746

#1752

#1755

#1756

#1756

#1759

#1759

#176

#1760

#1760

#1761

#1761

#1763

#1763

#1764

#1764

#1765

#1765

#1766

#1766

#194

#200

#215

#218

#229

#232

#234

#24

#249

#255

#269

#271

#279

#286

#288

#293

#294

#298

#299

#3

#300

#304

#312

#312

#313

#314

#324

#33

#332

#332

#335

#335

#337

#34

#357

#358

#369

#37

#379

#387

#389

#390

#394

#403

#410

#411

#416

#416

#419

#419

#427

#440

#444

#445

#458

#462

#462

#465

#472

#475

#475

#496

#510

#562

#581

#60

#605

#605

#606

#609

#612

#617

#618

#622

#64

#640

#65

#657

#658

#66

#662

#671

#671

#679

#680

#681

#681

#685

#687

#706

#708

#723

#723

#724

#729

#734

#741

#741

#749

#75

#752

#754

#775

#776

#777

#788

#792

#799

#799

#80

#800

#800

#806

#808

#821

#84

#84

#846

#85

#864

#865

#868

#891

#899

#901

#901

#903

#914

#915

#916

#918

#929

#93

#931

#945

#948

#948

#95

#961

#967

#967

#969

#970

#971

#973

#977

#983

#983

#988

#988

#990

#994

#999

#999

0.3.4

docker-rebuild-v0.7.5

docker-rebuild-v0.7.6

docker-rebuild-v0.7.7

docker-rebuild-v0.7.8

docker-rebuild-v0.8.0

v.3.72

v0.0.75

v0.1.0

v0.2.0

v0.2.1

v0.2.2

v0.2.4

v0.2.6

v0.2.7

v0.2.71

v0.2.72

v0.2.73

v0.2.74

v0.2.77

v0.3.0

v0.3.3

v0.3.6

v0.3.745

v0.3.746

v0.4.24

v0.4.243

v0.5.0.post1

v0.6.3

v0.7.0

v0.7.1

v0.7.2

v0.7.3

v0.7.4

v0.7.5

v0.7.6

v0.7.7

v0.7.8

v0.8.0

vr0.6.0

vr0.6.0rc1

vr0.6.3

Mono Color

• a0dff192ae Update README for speed example unclecode 2024-06-24 23:06:12 +08:00
• 1fffeeedd2 Update Readme: Showcase the speed unclecode 2024-06-24 23:02:08 +08:00
• f51b078042 Update reame example. unclecode 2024-06-24 22:54:29 +08:00
• b6023a51fb Add star chart unclecode 2024-06-24 22:47:46 +08:00
• 7e95c38acb Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-24 14:40:48 +00:00
• 78cfad8b2f chore: Update version to 0.2.7 and improve extraction function speed v0.2.7 unclecode 2024-06-24 22:39:56 +08:00
• c697bf23e4 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-22 16:37:27 +00:00
• b951d34ed0 chore: Update fetch URL to use HTTPS Unclecode 2024-06-22 16:37:21 +00:00
• 68b3dff74a Update CSS unclecode 2024-06-23 00:36:03 +08:00
• bfc4abd6e8 Update documents unclecode 2024-06-22 20:57:03 +08:00
• c8a10dc455 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-22 12:54:41 +00:00
• 8c77a760fc Fixed: - Redirect "/" to mkdocs unclecode 2024-06-22 20:54:32 +08:00
• 9e0ded8da0 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-22 12:41:52 +00:00
• b9bf8ac9d7 Fix mounting the "/" to mkdocs site folder unclecode 2024-06-22 20:41:39 +08:00
• 48c27899b7 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-22 12:38:14 +00:00
• d6182bedd7 chore: - Add demo page to the new mkdocs - Set website home page to mkdocs unclecode 2024-06-22 20:36:01 +08:00
• 2217904876 Update .gitignore unclecode 2024-06-22 18:12:12 +08:00
• 2c2362b4d3 issue 19 is resolved - Update Dockerfile to install mkdocs and build documentation v0.2.6 unclecode 2024-06-22 17:18:00 +08:00
• 612ed3fef2 chore: Update print statement to use markdown format unclecode 2024-06-21 19:10:13 +08:00
• fb2a6d0d04 chore: Update documentation link in README.md unclecode 2024-06-21 18:05:18 +08:00
• 19d3d39115 Update Marge the DOCS branch unclecode 2024-06-21 18:04:13 +08:00
• 3c32b0abed Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-21 09:58:17 +00:00
• c1413e6916 chore: Update documentation link in README.md docs unclecode 2024-06-21 17:57:47 +08:00
• e7705e661a ADD MKDocs unclecode 2024-06-21 17:56:54 +08:00
• 21b110bfd7 Update LLMExtractionStrategy to disable chunking if specified, Add example of summarization for a web page. unclecode 2024-06-19 19:03:35 +08:00
• 1fcb573909 chore: Update table of contents in README.md unclecode 2024-06-19 18:53:22 +08:00
• a215ec08d6 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-19 10:51:31 +00:00
• 0f6c5f5453 chore: Update configuration values, create new example, and update Dockerfile and README unclecode 2024-06-19 18:50:58 +08:00
• 350ca1511b chore: Update configuration values, create new example, and update Dockerfile and README unclecode 2024-06-19 18:48:20 +08:00
• 539263a8ba chore: Update configuration values for chunk token threshold, overlap rate, and minimum word threshold. Create a new example for LLMExtraction Strategy, update Dockerfile, and README unclecode 2024-06-19 18:32:20 +08:00
• 3f0e265baf Merge branch 'format-inline-tags' unclecode 2024-06-19 00:48:38 +08:00
• 21e2538e57 Update quickstart.py unclecode 2024-06-19 00:37:53 +08:00
• 5d3fef45f7 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-18 12:02:29 +00:00
• 480902bd66 Update README unclecode 2024-06-18 20:02:21 +08:00
• 853b9d59d8 feat: Add hooks for enhanced control over Selenium drivers unclecode 2024-06-18 20:00:51 +08:00
• 6d04284c44 Merge branch 'hooks' unclecode 2024-06-18 19:53:50 +08:00
• 4d43880cde Playing with different Docker settings to find the best one docker-test unclecode 2024-06-18 19:08:46 +08:00
• 4a50781453 chore: Remove local and .files folders from .gitignore unclecode 2024-06-17 15:57:34 +08:00
• 18561c55ce Remove .files folder from repository unclecode 2024-06-17 15:56:56 +08:00
• 77da48050d chore: Add custom headers to LocalSeleniumCrawlerStrategy hooks unclecode 2024-06-17 15:50:03 +08:00
• 9a97aacd85 chore: Add hooks for customizing the LocalSeleniumCrawlerStrategy unclecode 2024-06-17 15:37:18 +08:00
• 52daf3936a Fix typo in README unclecode 2024-06-17 15:15:37 +08:00
• 2f246d19f4 Enhancement: Replaced inline HTML tags with textual format for better LLM context handling #45 format-inline-tags unclecode 2024-06-17 15:14:56 +08:00
• 413595542a Enhancement: Replaced inline HTML tags with textual format for better LLM context handling #24 unclecode 2024-06-17 15:14:34 +08:00
• 42a5da854d Update version and change log. v0.2.4 unclecode 2024-06-17 14:47:58 +08:00
• d1d83a6ef7 Fix issue #22: Use MD5 hash for caching HTML files to handle long URLs unclecode 2024-06-17 14:44:01 +08:00
• 194050705d chore: Add pillow library to requirements.txt unclecode 2024-06-10 23:03:32 +08:00
• 989f8c91c8 Update README unclecode 2024-06-08 18:50:35 +08:00
• edba5fb5e9 Update README unclecode 2024-06-08 18:48:21 +08:00
• faa1defa5c Update README unclecode 2024-06-08 18:47:23 +08:00
• 77df6db453 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-08 10:38:10 +00:00
• f7e0cee1b0 vital: Right now, only raw html is retrived from datbase, therefore, css selector and other filter will be executed every time. unclecode 2024-06-08 18:37:40 +08:00
• 2124652327 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-08 10:07:30 +00:00
• b3a0edaa6d - User agent - Extract Links - Extract Metadata - Update Readme - Update REST API document unclecode 2024-06-08 17:59:42 +08:00
• 255bde70c9 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-08 08:53:54 +00:00
• 9c34b30723 Extract internal and external links. unclecode 2024-06-08 16:53:06 +08:00
• 36a5847df5 Add css selector example unclecode 2024-06-07 20:47:20 +08:00
• 04808b5dc9 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-07 12:44:41 +00:00
• a19379aa58 Add recipe images, update README, and REST api example unclecode 2024-06-07 20:43:50 +08:00
• 768d048e1c Update rest call how to use unclecode 2024-06-07 18:10:45 +08:00
• 94c11a0262 Add image unclecode 2024-06-07 18:09:21 +08:00
• b3a150f3d1 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-07 08:26:43 +00:00
• 649b0bfd02 feat: Remove default checked state for bypass-cache-checkbox unclecode 2024-06-07 16:26:36 +08:00
• de80a2da09 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-07 08:25:49 +00:00
• 57a00ec677 Update Readme unclecode 2024-06-07 16:25:30 +08:00
• df4cda8322 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-07 08:24:46 +00:00
• aeb2114170 Add example of REST API call unclecode 2024-06-07 16:24:40 +08:00
• 7717a3b948 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-07 08:19:37 +00:00
• b8d405fddd Update version number in landing page header unclecode 2024-06-07 16:19:30 +08:00
• a4a6b2075f Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-07 08:18:19 +00:00
• b32013cb97 Fix README file hyperlink unclecode 2024-06-07 15:37:05 +08:00
• 226a62a3c0 feat: Add screenshot functionality to crawl_urls extract-media unclecode 2024-06-07 15:33:15 +08:00
• 8e73a482a2 feat: Add screenshot functionality to crawl_urls unclecode 2024-06-07 15:23:32 +08:00
• 0533aeb814 v0.2.3: - Extract all media tags - Take screenshot of the page unclecode 2024-06-07 15:23:13 +08:00
• aead6de888 Merge branch 'main' of https://github.com/unclecode/crawl4ai into extract-media unclecode 2024-06-07 13:41:48 +08:00
• 8d82fd4cfe Merge pull request #14 from gkhngyk/main UncleCode 2024-06-07 13:30:10 +08:00
• 8f44db6499 Update README.md Gökhan Geyik 2024-06-05 17:16:02 +03:00
• c7553b1280 Update research assistant example with package installation instructions unclecode 2024-06-04 23:18:19 +08:00
• 8b8683f22e Add research assistant example using Chainlit unclecode 2024-06-04 22:43:09 +08:00
• 774ace6e3b Update html page for tutorial. unclecode 2024-06-02 18:00:53 +08:00
• 4010558885 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-02 08:12:32 +00:00
• 4a8f91a0fc Set bypass_cached to True unclecode 2024-06-02 16:12:25 +08:00
• b0cf5076da Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-02 08:09:25 +00:00
• 18c9784b61 Update index.html (hide extract block check box) unclecode 2024-06-02 16:09:20 +08:00
• 0d6e9e37ca Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-02 08:06:56 +00:00
• e5d401c67c Update generated code sample unclecode 2024-06-02 16:06:43 +08:00
• 9b0f71ba88 Merge branch 'main' of https://github.com/unclecode/crawl4ai Unclecode 2024-06-02 07:56:00 +00:00
• ae77589a98 Update Readme unclecode 2024-06-02 15:42:13 +08:00
• ad373c0e19 Update Readme unclecode 2024-06-02 15:41:24 +08:00
• 51f26d12fe Update for v0.2.2 - Support multiple JS scripts - Fixed some of bugs - Resolved a few issue relevant to Colab installation unclecode 2024-06-02 15:40:18 +08:00
• f1b60b2016 chore: Update ONNX model loading process unclecode 2024-05-31 18:07:05 +08:00
• 8c2dc2b1e4 Create Dockerfile UncleCode 2024-05-29 17:56:57 +08:00
• dc9a44c12a Update and rename Dockerfile to Dockerfile-version-0 UncleCode 2024-05-29 17:56:34 +08:00
• d9753b6349 Update requirements.txt UncleCode 2024-05-24 14:49:48 +08:00
• a554c0b143 Update requirements.txt UncleCode 2024-05-23 12:52:31 +08:00
• 7381fa95e6 Merge pull request #3 from QIN2DIM/main UncleCode 2024-05-23 09:29:28 +08:00
• 6ddccc144c chore: Bump version to 0.2.2 in setup.py v0.2.2 Unclecode 2024-05-19 16:19:40 +00:00
• 53d1176d53 chore: Update extraction strategy to support GPU, MPS, and CPU, add batch processing for CPU devices Unclecode 2024-05-19 16:18:58 +00:00
• 52c4be0696 Update setup.py version to 0.2.1 v0.2.1 unclecode 2024-05-19 22:30:59 +08:00
• 13a3b21d19 - Add ONNX embedding model for CPU devices, Update the similarithy threshold, improve the embedding speed. unclecode 2024-05-19 22:30:10 +08:00