ayrisdev/crawl4ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor Dockerfile and clean up main.py

Browse Source 
- Enhanced Dockerfile for platform-specific installations
    - Added ARG for TARGETPLATFORM and BUILDPLATFORM
    - Improved GPU support conditional on TARGETPLATFORM
  - Removed static pages mounting in main.py
  - Streamlined code structure to improve maintainability
This commit is contained in:
UncleCode
2024-11-29 20:08:09 +08:00
parent d202f3539b
commit 93bf3e8a1f
2 changed files with 16 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
Dockerfile
View File

@@ -1,6 +1,9 @@
# syntax=docker/dockerfile:1.4 # syntax=docker/dockerfile:1.4
# Build arguments ARG TARGETPLATFORM
ARG BUILDPLATFORM
# Other build arguments
ARG PYTHON_VERSION=3.10 ARG PYTHON_VERSION=3.10
# Base stage with system dependencies # Base stage with system dependencies
@@ -63,13 +66,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
# GPU support if enabled and architecture is supported # GPU support if enabled and architecture is supported
RUN if [ "$ENABLE_GPU" = "true" ] && [ "$(dpkg --print-architecture)" != "arm64" ] ; then \ RUN if [ "$ENABLE_GPU" = "true" ] && [ "$TARGETPLATFORM" = "linux/amd64" ] ; then \
apt-get update && apt-get install -y --no-install-recommends \ apt-get update && apt-get install -y --no-install-recommends \
nvidia-cuda-toolkit \ nvidia-cuda-toolkit \
&& rm -rf /var/lib/apt/lists/* ; \ && rm -rf /var/lib/apt/lists/* ; \
else \ else \
echo "Skipping NVIDIA CUDA Toolkit installation (unsupported architecture or GPU disabled)"; \ echo "Skipping NVIDIA CUDA Toolkit installation (unsupported platform or GPU disabled)"; \
fi fi
# Create and set working directory # Create and set working directory
WORKDIR /app WORKDIR /app
@@ -120,7 +123,11 @@ RUN pip install --no-cache-dir \
RUN mkdocs build RUN mkdocs build
# Install Playwright and browsers # Install Playwright and browsers
RUN playwright install RUN if [ "$TARGETPLATFORM" = "linux/amd64" ]; then \
playwright install chromium; \
elif [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
playwright install chromium; \
fi
# Expose port # Expose port
EXPOSE 8000 11235 9222 8080 EXPOSE 8000 11235 9222 8080

4
main.py
View File

@@ -340,9 +340,6 @@ app.add_middleware(
allow_headers=["*"], # Allows all headers allow_headers=["*"], # Allows all headers
) )
# Mount the pages directory as a static directory
app.mount("/pages", StaticFiles(directory=__location__ + "/pages"), name="pages")
# API token security # API token security
security = HTTPBearer() security = HTTPBearer()
CRAWL4AI_API_TOKEN = os.getenv("CRAWL4AI_API_TOKEN") or "test_api_code" CRAWL4AI_API_TOKEN = os.getenv("CRAWL4AI_API_TOKEN") or "test_api_code"
@@ -364,7 +361,6 @@ if os.path.exists(__location__ + "/site"):
app.mount("/mkdocs", StaticFiles(directory="site", html=True), name="mkdocs") app.mount("/mkdocs", StaticFiles(directory="site", html=True), name="mkdocs")
site_templates = Jinja2Templates(directory=__location__ + "/site") site_templates = Jinja2Templates(directory=__location__ + "/site")
templates = Jinja2Templates(directory=__location__ + "/pages")
crawler_service = CrawlerService() crawler_service = CrawlerService()