perf(server): unblock event loop on rate-limit waits and markitdown

Two complementary changes to mitigate intermittent TLS handshake
timeouts and "notifications/cancelled: Bad Request" seen against the
single-worker uvicorn deployment.

1. bedesten rate-limiter back-pressure
   - Add optional ``max_wait`` to ``_TokenBucket.acquire``: if the next
     wait would exceed it, raise ``BedestenRateLimited`` immediately
     instead of sleeping. After a server-side 429 the bucket pauses for
     up to 30s; previously a queued request sat in ``asyncio.sleep``
     for that whole window, holding the worker slot and pushing the
     MCP client past its cancellation timeout.
   - ``search_bedesten_unified`` / ``get_bedesten_document_markdown``
     catch ``BedestenRateLimited`` and reuse the existing structured
     429-style response, so callers get a fast, clean retry signal.
   - Tunable via ``BEDESTEN_RATE_MAX_WAIT_S`` (default 8.0s).

2. Offload sync markitdown conversions to a thread
   - Every ``markitdown.convert*`` call site is now wrapped in
     ``asyncio.to_thread(...)`` across 14 modules (bedesten, yargitay,
     danistay, anayasa norm + bireysel, uyusmazlik, emsal, rekabet,
     gib, kvkk, sayistay, bddk, sigorta_tahkim, kik_v2). PDF / large
     HTML parsing was stalling the event loop for seconds, which on a
     single-worker deployment delayed every other in-flight request
     and queued new TLS handshakes until they timed out.

Verified locally:
- ``ast.parse`` + ``importlib.import_module`` on all 15 modified files
- ``mcp_server_main.create_app()`` constructs successfully
- New ``_TokenBucket.acquire(max_wait=...)`` smoke-tested across 6
  paths: capacity-available, no-arg backward compat, max_wait raise,
  max_wait wait+succeed, ``penalize_until`` + max_wait fast-raise.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
saidsurucu
2026-05-11 14:31:23 +03:00
co-authored by Claude Opus 4.7
parent 26aa3dacc6
commit 96a5a538b2
15 changed files with 120 additions and 26 deletions
+2 -1
View File
@@ -1,5 +1,6 @@
# sayistay_mcp_module/client.py
import asyncio
import httpx
import re
from bs4 import BeautifulSoup
@@ -657,7 +658,7 @@ class SayistayApiClient:
)
# Convert HTML to Markdown using existing method
markdown_content = self._convert_html_to_markdown(html_content)
markdown_content = await asyncio.to_thread(self._convert_html_to_markdown, html_content)
if markdown_content and "Error converting HTML content" not in markdown_content:
logger.info(f"Successfully retrieved and converted document {decision_id} to Markdown")