perf(server): unblock event loop on rate-limit waits and markitdown

Two complementary changes to mitigate intermittent TLS handshake
timeouts and "notifications/cancelled: Bad Request" seen against the
single-worker uvicorn deployment.

1. bedesten rate-limiter back-pressure
   - Add optional ``max_wait`` to ``_TokenBucket.acquire``: if the next
     wait would exceed it, raise ``BedestenRateLimited`` immediately
     instead of sleeping. After a server-side 429 the bucket pauses for
     up to 30s; previously a queued request sat in ``asyncio.sleep``
     for that whole window, holding the worker slot and pushing the
     MCP client past its cancellation timeout.
   - ``search_bedesten_unified`` / ``get_bedesten_document_markdown``
     catch ``BedestenRateLimited`` and reuse the existing structured
     429-style response, so callers get a fast, clean retry signal.
   - Tunable via ``BEDESTEN_RATE_MAX_WAIT_S`` (default 8.0s).

2. Offload sync markitdown conversions to a thread
   - Every ``markitdown.convert*`` call site is now wrapped in
     ``asyncio.to_thread(...)`` across 14 modules (bedesten, yargitay,
     danistay, anayasa norm + bireysel, uyusmazlik, emsal, rekabet,
     gib, kvkk, sayistay, bddk, sigorta_tahkim, kik_v2). PDF / large
     HTML parsing was stalling the event loop for seconds, which on a
     single-worker deployment delayed every other in-flight request
     and queued new TLS handshakes until they timed out.

Verified locally:
- ``ast.parse`` + ``importlib.import_module`` on all 15 modified files
- ``mcp_server_main.create_app()`` constructs successfully
- New ``_TokenBucket.acquire(max_wait=...)`` smoke-tested across 6
  paths: capacity-available, no-arg backward compat, max_wait raise,
  max_wait wait+succeed, ``penalize_until`` + max_wait fast-raise.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
saidsurucu
2026-05-11 14:31:23 +03:00
co-authored by Claude Opus 4.7
parent 26aa3dacc6
commit 96a5a538b2
15 changed files with 120 additions and 26 deletions
+10 -4
View File
@@ -1,5 +1,6 @@
# bddk_mcp_module/client.py
import asyncio
import httpx
from typing import List, Optional, Dict, Any
import logging
@@ -210,14 +211,19 @@ class BddkApiClient:
# Convert to Markdown based on content type
if "pdf" in content_type:
# Handle PDF documents
# Handle PDF documents. markitdown is sync; offload to thread
# so PDF parsing doesn't block the event-loop / other requests.
pdf_stream = io.BytesIO(response.content)
result = self.markitdown.convert_stream(pdf_stream, file_extension=".pdf")
result = await asyncio.to_thread(
self.markitdown.convert_stream, pdf_stream, file_extension=".pdf"
)
markdown_content = result.text_content
else:
# Handle HTML documents
# Handle HTML documents (sync conversion offloaded to thread)
html_stream = io.BytesIO(response.content)
result = self.markitdown.convert_stream(html_stream, file_extension=".html")
result = await asyncio.to_thread(
self.markitdown.convert_stream, html_stream, file_extension=".html"
)
markdown_content = result.text_content
# Clean up the markdown content