# How to Guide

🌟 Welcome to the Crawl4ai Quickstart Guide! Let's dive into some web crawling fun!
First Step: Create an instance of WebCrawler and call the `warmup()` function.

    crawler = WebCrawler()
    crawler.warmup()

🧠 Understanding 'bypass_cache' and 'include_raw_html' parameters:
First crawl (caches the result):

    result = crawler.run(url="https://www.nbcnews.com/business")

Second crawl (Force to crawl again):

    result = crawler.run(url="https://www.nbcnews.com/business", bypass_cache=True)

⚠️ Don't forget to set ``bypass_cache`` to True if you want to try different strategies for the same URL. Otherwise, the cached result will be returned. You can also set ``always_by_pass_cache`` in constructor to True to always bypass the cache.
Crawl result without raw HTML content:

    result = crawler.run(url="https://www.nbcnews.com/business", include_raw_html=False)

📄 The 'include_raw_html' parameter, when set to True, includes the raw HTML content in the response. By default, it is set to True.
Set `always_by_pass_cache` to True:

    crawler.always_by_pass_cache = True

📸 Let's take a screenshot of the page!

    result = crawler.run(
        url="https://www.nbcnews.com/business",
        screenshot=True
    )
    with open("screenshot.png", "wb") as f:
        f.write(base64.b64decode(result.screenshot))

🧩 Let's add a chunking strategy: RegexChunking!
Using RegexChunking:

    result = crawler.run(
        url="https://www.nbcnews.com/business",
        chunking_strategy=RegexChunking(patterns=["\n\n"])
    )

Using NlpSentenceChunking:

    result = crawler.run(
        url="https://www.nbcnews.com/business",
        chunking_strategy=NlpSentenceChunking()
    )

🧠 Let's get smarter with an extraction strategy: CosineStrategy!
Using CosineStrategy:

    result = crawler.run(
        url="https://www.nbcnews.com/business",
        extraction_strategy=CosineStrategy(word_count_threshold=10, max_dist=0.2, linkage_method="ward", top_k=3)
    )

🤖 Time to bring in the big guns: LLMExtractionStrategy without instructions!
Using LLMExtractionStrategy without instructions:

    result = crawler.run(
        url="https://www.nbcnews.com/business",
        extraction_strategy=LLMExtractionStrategy(provider="openai/gpt-4o", api_token=os.getenv('OPENAI_API_KEY'))
    )

📜 Let's make it even more interesting: LLMExtractionStrategy with instructions!
Using LLMExtractionStrategy with instructions:

    result = crawler.run(
        url="https://www.nbcnews.com/business",
        extraction_strategy=LLMExtractionStrategy(
        provider="openai/gpt-4o",
        api_token=os.getenv('OPENAI_API_KEY'),
        instruction="I am interested in only financial news"
    )
    )

🎯 Targeted extraction: Let's use a CSS selector to extract only H2 tags!
Using CSS selector to extract H2 tags:

    result = crawler.run(
        url="https://www.nbcnews.com/business",
        css_selector="h2"
    )

🖱️ Let's get interactive: Passing JavaScript code to click 'Load More' button!
Using JavaScript to click 'Load More' button:

    js_code = ["""
    const loadMoreButton = Array.from(document.querySelectorAll('button')).find(button => button.textContent.includes('Load More'));
    loadMoreButton && loadMoreButton.click();
    """]
    crawler = WebCrawler(verbos=crawler_strategy, always_by_pass_cache=True)
    result = crawler.run(url="https://www.nbcnews.com/business", js = js_code)

Remember that you can pass multiple JavaScript code snippets in the list. They all will be executed in the order they are passed.
🎉 Congratulations! You've made it through the Crawl4ai Quickstart Guide! Now go forth and crawl the web like a pro! 🕸️