Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
54e8d61f83 | ||
|
|
35382136c5 | ||
|
|
59d81d4b03 | ||
|
|
06a319c0ee | ||
|
|
a48b003121 | ||
|
|
1620d7a9b0 | ||
|
|
c10069bcc7 | ||
|
|
c0fe7e1305 | ||
|
|
f492109eb7 | ||
|
|
35f3b739d4 | ||
|
|
8f1ca6b854 | ||
|
|
71218996fd | ||
|
|
6bac51dc19 | ||
|
|
b898cad4f4 | ||
|
|
3d172c508a | ||
|
|
48efe74dc0 | ||
|
|
8ae5772c2c | ||
|
|
87cf4fd46d | ||
|
|
d590702272 | ||
|
|
1ebe8847fb | ||
|
|
cb318faeba | ||
|
|
83f54a86a8 | ||
|
|
f5f0f99678 | ||
|
|
b0d7151ba1 | ||
|
|
de9e337163 | ||
|
|
a7ebb8b27e | ||
|
|
01e58ab5ed | ||
|
|
7081bbbd91 | ||
|
|
ca89c9480d | ||
|
|
b751e94847 | ||
|
|
3ccb52e719 | ||
|
|
e8b92e347c | ||
|
|
e65b42abc8 | ||
|
|
71096ae67d | ||
|
|
9fb23dadae | ||
|
|
d77781709a | ||
|
|
c059ec30a7 | ||
|
|
6842ad207d | ||
|
|
bbc56d9409 | ||
|
|
66b5e2631e | ||
|
|
a17853b918 | ||
|
|
0a302bb13b | ||
|
|
28e9a47464 | ||
|
|
6c1efece31 | ||
|
|
eb9441a6f3 | ||
|
|
d006dc8a55 | ||
|
|
55cee6933d | ||
|
|
2f375d74e5 | ||
|
|
82856b25e9 | ||
|
|
8464aedceb | ||
|
|
64c3a2c138 | ||
|
|
318bedd4c5 | ||
|
|
91895f6c1a | ||
|
|
b86c18c842 | ||
|
|
5509380cef | ||
|
|
1c818756e4 | ||
|
|
34ac65dc02 |
@@ -56,6 +56,9 @@ LOG_LEVEL=info
|
||||
# Base URL for the application (used for OAuth callbacks and API URLs)
|
||||
BASE_URL=http://localhost:8000
|
||||
|
||||
# JWT Secret for MCP token generation
|
||||
JWT_SECRET_KEY=your_jwt_secret_key_here
|
||||
|
||||
# =============================================================================
|
||||
# MCP SERVER SETTINGS
|
||||
# =============================================================================
|
||||
|
||||
@@ -190,3 +190,5 @@ GEMINI.md
|
||||
fly.toml
|
||||
scripts/deploy-flyio.sh
|
||||
docs/DEPLOYMENT_FLYIO.md
|
||||
setup_jwt_template.py
|
||||
mcp_server_main.py.backup
|
||||
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
# -------- BASE IMAGE (includes Chromium & deps) ----------------------------
|
||||
FROM mcr.microsoft.com/playwright/python:v1.52.0-noble
|
||||
FROM mcr.microsoft.com/playwright/python:v1.53.0-noble
|
||||
|
||||
# -------- Runtime setup ----------------------------------------------------
|
||||
WORKDIR /app
|
||||
|
||||
@@ -26,6 +26,7 @@ Bu proje, çeşitli Türk hukuk kaynaklarına (Yargıtay, Danıştay, Emsal Kara
|
||||
* **KİK (Kamu İhale Kurulu):** Çeşitli kriterlerle Kurul kararlarını arama; uzun karar metinlerini (varsayılan 5.000 karakterlik) sayfalanmış Markdown formatında getirme.
|
||||
* **Rekabet Kurumu:** Çeşitli kriterlerle Kurul kararlarını arama; karar metinlerini Markdown formatında getirme.
|
||||
* **Sayıştay:** 3 karar türü ile kapsamlı denetim kararlarına erişim + **8 Daire Filtreleme** + **Tarih Aralığı & İçerik Arama** (Genel Kurul yorumlayıcı kararları, Temyiz Kurulu itiraz kararları, Daire ilk derece denetim kararları)
|
||||
* **KVKK (Kişisel Verilerin Korunması Kurulu):** Brave Search API ile veri koruma kararlarını arama; uzun karar metinlerini (5.000 karakterlik) sayfalanmış Markdown formatında getirme + **Türkçe Arama** + **Site Hedeflemeli Arama** (kvkk.gov.tr kararları)
|
||||
|
||||
* Karar metinlerinin daha kolay işlenebilmesi için Markdown formatına çevrilmesi.
|
||||
* Claude Desktop uygulaması ile `fastmcp install` komutu kullanılarak kolay entegrasyon.
|
||||
@@ -48,7 +49,7 @@ Bu bölüm, Yargı MCP aracını 5ire gibi Claude Desktop dışındaki MCP istem
|
||||
* **Name:** `Yargı MCP`
|
||||
* **Command:**
|
||||
```
|
||||
uvx --from git+https://github.com/saidsurucu/yargi-mcp yargi-mcp
|
||||
uvx yargi-mcp
|
||||
```
|
||||
* **Save** butonuna basarak kaydedin.
|
||||

|
||||
@@ -70,7 +71,6 @@ Bu bölüm, Yargı MCP aracını 5ire gibi Claude Desktop dışındaki MCP istem
|
||||
"Yargı MCP": {
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"--from", "git+https://github.com/saidsurucu/yargi-mcp",
|
||||
"yargi-mcp"
|
||||
]
|
||||
}
|
||||
@@ -101,8 +101,6 @@ Yargı MCP'yi Gemini CLI ile kullanmak için:
|
||||
"yargi_mcp": {
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"--from",
|
||||
"git+https://github.com/saidsurucu/yargi-mcp",
|
||||
"yargi-mcp"
|
||||
]
|
||||
}
|
||||
@@ -125,56 +123,41 @@ Yargı MCP'yi Gemini CLI ile kullanmak için:
|
||||
|
||||
🛠️ **Kullanılabilir Araçlar (MCP Tools)**
|
||||
|
||||
Bu FastMCP sunucusu aşağıdaki temel araçları sunar:
|
||||
Bu FastMCP sunucusu **30 MCP aracı** sunar:
|
||||
|
||||
### **Yargıtay Araçları (Dual API + 52 Daire Filtreleme)**
|
||||
* **Ana API:**
|
||||
* `search_yargitay_detailed(arananKelime, birimYrgKurulDaire, ...)`: Yargıtay kararlarını detaylı kriterlerle arar. **52 daire/kurul seçeneği** (Hukuk/Ceza Daireleri 1-23, Genel Kurullar, Başkanlar Kurulu)
|
||||
* `get_yargitay_document_markdown(id: str)`: Belirli bir Yargıtay kararının metnini Markdown formatında getirir.
|
||||
* **Bedesten API (Alternatif):**
|
||||
* `search_yargitay_bedesten(phrase, birimAdi, kararTarihiStart, kararTarihiEnd, ...)`: Bedesten API ile Yargıtay kararlarını arar. **Aynı 52 daire filtreleme** + **Tarih Filtreleme** + **Kesin Cümle Arama** (`"\"mülkiyet kararı\""`)
|
||||
* `get_yargitay_bedesten_document_markdown(documentId: str)`: Bedesten'den karar metni (HTML/PDF → Markdown)
|
||||
### **Yargıtay Araçları (Ana API + 52 Daire Filtreleme)**
|
||||
1. `search_yargitay_detailed(arananKelime, birimYrgKurulDaire, ...)`: Yargıtay kararlarını detaylı kriterlerle arar. **52 daire/kurul seçeneği** (Hukuk/Ceza Daireleri 1-23, Genel Kurullar, Başkanlar Kurulu)
|
||||
2. `get_yargitay_document_markdown(id: str)`: Belirli bir Yargıtay kararının metnini Markdown formatında getirir.
|
||||
|
||||
### **Danıştay Araçları (Triple API + 27 Daire Filtreleme)**
|
||||
* **Ana API'lar:**
|
||||
* `search_danistay_by_keyword(andKelimeler, orKelimeler, ...)`: Danıştay kararlarını anahtar kelimelerle arar.
|
||||
* `search_danistay_detailed(daire, esasYil, ...)`: Danıştay kararlarını detaylı kriterlerle arar.
|
||||
* `get_danistay_document_markdown(id: str)`: Belirli bir Danıştay kararının metnini Markdown formatında getirir.
|
||||
* **Bedesten API (Alternatif):**
|
||||
* `search_danistay_bedesten(phrase, birimAdi, kararTarihiStart, kararTarihiEnd, ...)`: Bedesten API ile Danıştay kararlarını arar. **27 daire/kurul seçeneği** + **Tarih Filtreleme** + **Kesin Cümle Arama** (`"\"idari işlem\""`) (1-17. Daireler, Vergi/İdare Kurulları, Askeri Mahkemeler)
|
||||
* `get_danistay_bedesten_document_markdown(documentId: str)`: Bedesten'den karar metni
|
||||
### **Danıştay Araçları (Dual API + 27 Daire Filtreleme)**
|
||||
3. `search_danistay_by_keyword(andKelimeler, orKelimeler, ...)`: Danıştay kararlarını anahtar kelimelerle arar.
|
||||
4. `search_danistay_detailed(daire, esasYil, ...)`: Danıştay kararlarını detaylı kriterlerle arar.
|
||||
5. `get_danistay_document_markdown(id: str)`: Belirli bir Danıştay kararının metnini Markdown formatında getirir.
|
||||
|
||||
### **Diğer Mahkemeler (Bedesten API + Gelişmiş Arama)**
|
||||
* **Yerel Hukuk Mahkemeleri:**
|
||||
* `search_yerel_hukuk_bedesten(phrase, kararTarihiStart, kararTarihiEnd, ...)`: Yerel hukuk mahkemesi kararlarını arar + **Tarih & Kesin Cümle Arama** (`"\"sözleşme ihlali\""`)
|
||||
* `get_yerel_hukuk_bedesten_document_markdown(documentId: str)`: Karar metni
|
||||
* **İstinaf Hukuk Mahkemeleri:**
|
||||
* `search_istinaf_hukuk_bedesten(phrase, kararTarihiStart, kararTarihiEnd, ...)`: İstinaf mahkemesi kararlarını arar + **Tarih & Kesin Cümle Arama** (`"\"temyiz incelemesi\""`)
|
||||
* `get_istinaf_hukuk_bedesten_document_markdown(documentId: str)`: Karar metni
|
||||
* **Kanun Yararına Bozma (KYB):**
|
||||
* `search_kyb_bedesten(phrase, kararTarihiStart, kararTarihiEnd, ...)`: Olağanüstü kanun yolu kararlarını arar + **Tarih & Kesin Cümle Arama** (`"\"kanun yararına bozma\""`)
|
||||
* `get_kyb_bedesten_document_markdown(documentId: str)`: Karar metni
|
||||
### **Birleşik Bedesten API Araçları (5 Mahkeme)**
|
||||
6. `search_bedesten_unified(phrase, court_types, birimAdi, kararTarihiStart, kararTarihiEnd, ...)`: **5 mahkeme türünü** birleşik arama (Yargıtay, Danıştay, Yerel Hukuk, İstinaf Hukuk, KYB) + **79 daire filtreleme** + **Tarih & Kesin Cümle Arama**
|
||||
7. `get_bedesten_document_markdown(documentId: str)`: Bedesten API'den herhangi bir belgeyi Markdown formatında getirir (HTML/PDF → Markdown)
|
||||
|
||||
* **Emsal Karar Araçları:**
|
||||
* `search_emsal_detailed_decisions(search_query: EmsalSearchRequest) -> CompactEmsalSearchResult`: Emsal (UYAP) kararlarını detaylı kriterlerle arar.
|
||||
* `get_emsal_document_markdown(id: str) -> EmsalDocumentMarkdown`: Belirli bir Emsal kararının metnini Markdown formatında getirir.
|
||||
### **Emsal Karar Araçları (UYAP)**
|
||||
8. `search_emsal_detailed_decisions(keyword, ...)`: Emsal (UYAP) kararlarını detaylı kriterlerle arar.
|
||||
9. `get_emsal_document_markdown(id: str)`: Belirli bir Emsal kararının metnini Markdown formatında getirir.
|
||||
|
||||
* **Uyuşmazlık Mahkemesi Araçları:**
|
||||
* `search_uyusmazlik_decisions(search_params: UyusmazlikSearchRequest) -> UyusmazlikSearchResponse`: Uyuşmazlık Mahkemesi kararlarını çeşitli form kriterleriyle arar.
|
||||
* `get_uyusmazlik_document_markdown_from_url(document_url: HttpUrl) -> UyusmazlikDocumentMarkdown`: Bir Uyuşmazlık kararını tam URL'sinden alıp Markdown formatında getirir.
|
||||
### **Uyuşmazlık Mahkemesi Araçları**
|
||||
10. `search_uyusmazlik_decisions(icerik, ...)`: Uyuşmazlık Mahkemesi kararlarını çeşitli form kriterleriyle arar.
|
||||
11. `get_uyusmazlik_document_markdown_from_url(document_url)`: Bir Uyuşmazlık kararını tam URL'sinden alıp Markdown formatında getirir.
|
||||
|
||||
* **Anayasa Mahkemesi (Norm Denetimi) Araçları:**
|
||||
* `search_anayasa_norm_denetimi_decisions(search_query: AnayasaNormDenetimiSearchRequest) -> AnayasaSearchResult`: AYM Norm Denetimi kararlarını kapsamlı kriterlerle arar.
|
||||
* `get_anayasa_norm_denetimi_document_markdown(document_url: str, page_number: Optional[int] = 1) -> AnayasaDocumentMarkdown`: Belirli bir AYM Norm Denetimi kararını URL'sinden alır ve 5.000 karakterlik sayfalanmış Markdown içeriğini getirir.
|
||||
### **Anayasa Mahkemesi Araçları (Norm Denetimi)**
|
||||
12. `search_anayasa_norm_denetimi_decisions(keywords_all, ...)`: AYM Norm Denetimi kararlarını kapsamlı kriterlerle arar.
|
||||
13. `get_anayasa_norm_denetimi_document_markdown(document_url, page_number)`: Belirli bir AYM Norm Denetimi kararını URL'sinden alır ve **sayfalanmış Markdown** içeriğini getirir.
|
||||
|
||||
* **Anayasa Mahkemesi (Bireysel Başvuru) Araçları:**
|
||||
* `search_anayasa_bireysel_basvuru_report(search_query: AnayasaBireyselReportSearchRequest) -> AnayasaBireyselReportSearchResult`: AYM Bireysel Başvuru "Karar Arama Raporu" oluşturur.
|
||||
* `get_anayasa_bireysel_basvuru_document_markdown(document_url_path: str, page_number: Optional[int] = 1) -> AnayasaBireyselBasvuruDocumentMarkdown`: Belirli bir AYM Bireysel Başvuru kararını URL path'inden alır ve 5.000 karakterlik sayfalanmış Markdown içeriğini getirir.
|
||||
### **Anayasa Mahkemesi Araçları (Bireysel Başvuru)**
|
||||
14. `search_anayasa_bireysel_basvuru_report(keywords, ...)`: AYM Bireysel Başvuru "Karar Arama Raporu" oluşturur.
|
||||
15. `get_anayasa_bireysel_basvuru_document_markdown(document_url_path, page_number)`: Belirli bir AYM Bireysel Başvuru kararını URL path'inden alır ve **sayfalanmış Markdown** içeriğini getirir.
|
||||
|
||||
* **KİK (Kamu İhale Kurulu) Araçları:**
|
||||
* `search_kik_decisions(search_query: KikSearchRequest) -> KikSearchResult`: KİK (Kamu İhale Kurulu) kararlarını arar.
|
||||
* `get_kik_document_markdown(karar_id: str, page_number: Optional[int] = 1) -> KikDocumentMarkdown`: Belirli bir KİK kararını, Base64 ile encode edilmiş `karar_id`'sini kullanarak alır ve 5.000 karakterlik sayfalanmış Markdown içeriğini getirir.
|
||||
* **Rekabet Kurumu Araçları:**
|
||||
### **KİK (Kamu İhale Kurulu) Araçları**
|
||||
16. `search_kik_decisions(karar_tipi, ...)`: KİK (Kamu İhale Kurulu) kararlarını arar.
|
||||
17. `get_kik_document_markdown(karar_id, page_number)`: Belirli bir KİK kararını, Base64 ile encode edilmiş `karar_id`'sini kullanarak alır ve **sayfalanmış Markdown** içeriğini getirir.
|
||||
### **Rekabet Kurumu Araçları**
|
||||
* `search_rekabet_kurumu_decisions(KararTuru: Literal[...], ...) -> RekabetSearchResult`: Rekabet Kurumu kararlarını arar. `KararTuru` için kullanıcı dostu isimler kullanılır (örn: "Birleşme ve Devralma").
|
||||
* `get_rekabet_kurumu_document(karar_id: str, page_number: Optional[int] = 1) -> RekabetDocument`: Belirli bir Rekabet Kurumu kararını `karar_id` ile alır. Kararın PDF formatındaki orijinalinden istenen sayfayı ayıklar ve Markdown formatında döndürür.
|
||||
|
||||
@@ -189,15 +172,20 @@ Bu FastMCP sunucusu aşağıdaki temel araçları sunar:
|
||||
* `get_sayistay_temyiz_kurulu_document_markdown(decision_id: str)`: Temyiz Kurulu kararının tam metnini Markdown formatında getirir
|
||||
* `get_sayistay_daire_document_markdown(decision_id: str)`: Daire kararının tam metnini Markdown formatında getirir
|
||||
|
||||
* **KVKK Araçları (Brave Search API + Türkçe Arama):**
|
||||
* `search_kvkk_decisions(keywords, page, pageSize, ...)`: KVKK (Kişisel Verilerin Korunması Kurulu) kararlarını Brave Search API ile arar. **Türkçe arama** + **Site hedeflemeli** (`site:kvkk.gov.tr "karar özeti"`) + **Sayfalama desteği**
|
||||
* `get_kvkk_document_markdown(decision_url: str, page_number: Optional[int] = 1)`: KVKK kararının tam metnini **sayfalanmış Markdown** formatında getirir (5.000 karakterlik sayfa)
|
||||
|
||||
|
||||
---
|
||||
|
||||
### **📊 Kapsamlı İstatistikler**
|
||||
- **Toplam Mahkeme/Kurum:** 12 farklı hukuki kurum
|
||||
- **Toplam MCP Tool:** 36+ arama ve belge getirme aracı
|
||||
- **Toplam Mahkeme/Kurum:** 13 farklı hukuki kurum (KVKK dahil)
|
||||
- **Toplam MCP Tool:** 30 arama ve belge getirme aracı
|
||||
- **Daire/Kurul Filtreleme:** 87 farklı seçenek (52 Yargıtay + 27 Danıştay + 8 Sayıştay)
|
||||
- **Tarih Filtreleme:** 5 Bedesten API aracında ISO 8601 formatında tam tarih aralığı desteği
|
||||
- **Kesin Cümle Arama:** 5 Bedesten API aracında çift tırnak ile tam cümle arama (`"\"mülkiyet kararı\""` formatı)
|
||||
- **Tarih Filtreleme:** Birleşik Bedesten API aracında ISO 8601 formatında tam tarih aralığı desteği
|
||||
- **Kesin Cümle Arama:** Birleşik Bedesten API aracında çift tırnak ile tam cümle arama (`"\"mülkiyet kararı\""` formatı)
|
||||
- **Birleşik API:** 10 ayrı Bedesten aracı → 2 birleşik araç (search_bedesten_unified + get_bedesten_document_markdown)
|
||||
- **API Kaynağı:** Dual/Triple API desteği ile maksimum kapsama
|
||||
- **Tam Türk Adalet Sistemi:** Yerel mahkemelerden en yüksek mahkemelere kadar
|
||||
|
||||
|
||||
@@ -7,8 +7,7 @@ from typing import Dict, Any, List, Optional, Tuple
|
||||
import logging
|
||||
import html
|
||||
import re
|
||||
import tempfile
|
||||
import os
|
||||
import io
|
||||
from urllib.parse import urlencode, urljoin, quote
|
||||
from markitdown import MarkItDown
|
||||
import math # For math.ceil for pagination
|
||||
@@ -230,23 +229,23 @@ class AnayasaBireyselBasvuruApiClient:
|
||||
html_input_for_markdown = processed_html
|
||||
|
||||
markdown_text = None
|
||||
temp_file_path = None
|
||||
try:
|
||||
md_converter = MarkItDown()
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file:
|
||||
# Ensure the content is wrapped in basic HTML structure if it's not already
|
||||
if not html_input_for_markdown.strip().lower().startswith(("<html", "<!doctype")):
|
||||
tmp_file.write(f"<html><head><meta charset=\"UTF-8\"></head><body>{html_input_for_markdown}</body></html>")
|
||||
html_content = f"<html><head><meta charset=\"UTF-8\"></head><body>{html_input_for_markdown}</body></html>"
|
||||
else:
|
||||
tmp_file.write(html_input_for_markdown)
|
||||
temp_file_path = tmp_file.name
|
||||
html_content = html_input_for_markdown
|
||||
|
||||
conversion_result = md_converter.convert(temp_file_path)
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = html_content.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown()
|
||||
conversion_result = md_converter.convert(html_stream)
|
||||
markdown_text = conversion_result.text_content
|
||||
except Exception as e:
|
||||
logger.error(f"AnayasaBireyselBasvuruApiClient: MarkItDown conversion error: {e}")
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path):
|
||||
os.remove(temp_file_path)
|
||||
return markdown_text
|
||||
|
||||
async def get_decision_document_as_markdown(
|
||||
|
||||
@@ -7,8 +7,7 @@ from typing import Dict, Any, List, Optional, Tuple
|
||||
import logging
|
||||
import html
|
||||
import re
|
||||
import tempfile
|
||||
import os
|
||||
import io
|
||||
from urllib.parse import urlencode, urljoin, quote
|
||||
from markitdown import MarkItDown
|
||||
import math # For math.ceil for pagination
|
||||
@@ -82,6 +81,13 @@ class AnayasaMahkemesiApiClient:
|
||||
if params.has_dissenting_opinion and params.has_dissenting_opinion.value and params.has_dissenting_opinion.value != "ALL": query_params.append(("KarsiOy", params.has_dissenting_opinion.value))
|
||||
if params.has_different_reasoning and params.has_different_reasoning.value and params.has_different_reasoning.value != "ALL": query_params.append(("FarkliGerekce", params.has_different_reasoning.value))
|
||||
|
||||
# Add pagination and sorting parameters as query params instead of URL path
|
||||
if params.results_per_page and params.results_per_page != 10:
|
||||
query_params.append(("SatirSayisi", str(params.results_per_page)))
|
||||
|
||||
if params.sort_by_criteria and params.sort_by_criteria != "KararTarihi":
|
||||
query_params.append(("Siralama", params.sort_by_criteria))
|
||||
|
||||
if params.page_to_fetch and params.page_to_fetch > 1:
|
||||
query_params.append(("page", str(params.page_to_fetch)))
|
||||
return query_params
|
||||
@@ -90,16 +96,8 @@ class AnayasaMahkemesiApiClient:
|
||||
self,
|
||||
params: AnayasaNormDenetimiSearchRequest
|
||||
) -> AnayasaSearchResult:
|
||||
path_segments = []
|
||||
if params.results_per_page and params.results_per_page != 10: # Default is 10
|
||||
path_segments.append(f"SatirSayisi/{params.results_per_page}")
|
||||
|
||||
if params.sort_by_criteria and params.sort_by_criteria != "KararTarihi": # Default is KararTarihi
|
||||
# Ensure correct quoting for criteria that might have Turkish chars or spaces
|
||||
path_segments.append(f"Siralama/{quote(params.sort_by_criteria)}")
|
||||
|
||||
path_segments.append(self.SEARCH_PATH_SEGMENT)
|
||||
request_path = "/" + "/".join(path_segments)
|
||||
# Use simple /Ara endpoint - the complex path structure seems to cause 404s
|
||||
request_path = f"/{self.SEARCH_PATH_SEGMENT}"
|
||||
|
||||
final_query_params = self._build_search_query_params_for_aym(params)
|
||||
logger.info(f"AnayasaMahkemesiApiClient: Performing Norm Denetimi search. Path: {request_path}, Params: {final_query_params}")
|
||||
@@ -222,24 +220,23 @@ class AnayasaMahkemesiApiClient:
|
||||
html_input_for_markdown = str(body_tag) if body_tag else processed_html
|
||||
|
||||
markdown_text = None
|
||||
temp_file_path = None
|
||||
try:
|
||||
md_converter = MarkItDown()
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file:
|
||||
# Ensure the content is wrapped in basic HTML structure if it's not already
|
||||
if not html_input_for_markdown.strip().lower().startswith(("<html", "<!doctype")):
|
||||
tmp_file.write(f"<html><head><meta charset=\"UTF-8\"></head><body>{html_input_for_markdown}</body></html>")
|
||||
html_content = f"<html><head><meta charset=\"UTF-8\"></head><body>{html_input_for_markdown}</body></html>"
|
||||
else:
|
||||
tmp_file.write(html_input_for_markdown)
|
||||
temp_file_path = tmp_file.name
|
||||
html_content = html_input_for_markdown
|
||||
|
||||
conversion_result = md_converter.convert(temp_file_path)
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = html_content.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown()
|
||||
conversion_result = md_converter.convert(html_stream)
|
||||
markdown_text = conversion_result.text_content
|
||||
except Exception as e:
|
||||
logger.error(f"AnayasaMahkemesiApiClient: MarkItDown conversion error: {e}")
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path):
|
||||
os.remove(temp_file_path)
|
||||
return markdown_text
|
||||
|
||||
async def get_decision_document_as_markdown(
|
||||
|
||||
@@ -114,7 +114,7 @@ class AnayasaNormDenetimiSearchRequest(BaseModel):
|
||||
review_outcomes: Optional[List[AnayasaIncelemeSonucuEnum]] = Field(default_factory=list, description="List of review types and outcomes (IncelemeTuruKararSonuclar_id[]).")
|
||||
reason_for_final_outcome: Optional[AnayasaSonucGerekcesiEnum] = Field(default=AnayasaSonucGerekcesiEnum.TUMU, description="Main reason for the decision outcome (KararSonucununGerekcesi).")
|
||||
basis_constitution_article_numbers: Optional[List[str]] = Field(default_factory=list, description="List of supporting Constitution article numbers (DayanakHukmu[]).")
|
||||
results_per_page: Optional[int] = Field(10, description="Number of results per page. Options: 10, 20, 30, 40, 50.")
|
||||
results_per_page: Optional[int] = Field(10, ge=1, le=10, description="Results per page.")
|
||||
page_to_fetch: Optional[int] = Field(1, ge=1, description="Page number to fetch for results list.")
|
||||
sort_by_criteria: Optional[str] = Field("KararTarihi", description="Sort criteria. Options: 'KararTarihi', 'YayinTarihi', 'Toplam' (keyword count).")
|
||||
|
||||
|
||||
+379
-34
@@ -10,8 +10,11 @@ Usage:
|
||||
"""
|
||||
|
||||
import os
|
||||
from fastapi import FastAPI, Request, HTTPException
|
||||
from fastapi.responses import JSONResponse
|
||||
import time
|
||||
import logging
|
||||
from datetime import datetime, timedelta
|
||||
from fastapi import FastAPI, Request, HTTPException, Query
|
||||
from fastapi.responses import JSONResponse, HTMLResponse
|
||||
from fastapi.exception_handlers import http_exception_handler
|
||||
from starlette.middleware import Middleware
|
||||
from starlette.middleware.cors import CORSMiddleware
|
||||
@@ -23,13 +26,16 @@ from mcp_server_main import app as mcp_server
|
||||
# Import Stripe webhook router
|
||||
from stripe_webhook import router as stripe_router
|
||||
|
||||
# Import MCP Auth HTTP adapter
|
||||
from mcp_auth_http_adapter import router as mcp_auth_router
|
||||
# Import simplified MCP Auth HTTP adapter
|
||||
from mcp_auth_http_simple import router as mcp_auth_router
|
||||
|
||||
# OAuth configuration from environment variables
|
||||
CLERK_ISSUER = os.getenv("CLERK_ISSUER", "https://accounts.yargimcp.com")
|
||||
BASE_URL = os.getenv("BASE_URL", "https://yargimcp.com")
|
||||
|
||||
# Setup logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Configure CORS middleware
|
||||
cors_origins = os.getenv("ALLOWED_ORIGINS", "*").split(",")
|
||||
custom_middleware = [
|
||||
@@ -42,25 +48,46 @@ custom_middleware = [
|
||||
),
|
||||
]
|
||||
|
||||
# Create MCP Starlette sub-application first
|
||||
# Create MCP Starlette sub-application (without auth wrapper)
|
||||
mcp_app = mcp_server.http_app(
|
||||
path="/",
|
||||
middleware=custom_middleware
|
||||
)
|
||||
|
||||
# Create FastAPI wrapper application with MCP app's lifespan
|
||||
# Configure JSON encoder for proper Turkish character support
|
||||
import json
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
class UTF8JSONResponse(JSONResponse):
|
||||
def __init__(self, content=None, status_code=200, headers=None, **kwargs):
|
||||
if headers is None:
|
||||
headers = {}
|
||||
headers["Content-Type"] = "application/json; charset=utf-8"
|
||||
super().__init__(content, status_code, headers, **kwargs)
|
||||
|
||||
def render(self, content) -> bytes:
|
||||
return json.dumps(
|
||||
content,
|
||||
ensure_ascii=False,
|
||||
allow_nan=False,
|
||||
indent=None,
|
||||
separators=(",", ":"),
|
||||
).encode("utf-8")
|
||||
|
||||
# Create FastAPI wrapper application with MCP lifespan
|
||||
app = FastAPI(
|
||||
title="Yargı MCP Server",
|
||||
description="MCP server for Turkish legal databases with OAuth authentication",
|
||||
version="0.1.0",
|
||||
middleware=custom_middleware,
|
||||
lifespan=mcp_app.lifespan # Critical: Get lifespan from mcp_app, not mcp_server
|
||||
lifespan=mcp_app.lifespan, # MCP app lifespan
|
||||
default_response_class=UTF8JSONResponse # Use UTF-8 JSON encoder
|
||||
)
|
||||
|
||||
# Add Stripe webhook router to FastAPI
|
||||
app.include_router(stripe_router, prefix="/api")
|
||||
|
||||
# Add MCP Auth HTTP adapter to FastAPI (replaces old OAuth router)
|
||||
# Add MCP Auth HTTP adapter to FastAPI (handles OAuth endpoints)
|
||||
app.include_router(mcp_auth_router)
|
||||
|
||||
# Custom 401 exception handler for MCP spec compliance
|
||||
@@ -80,23 +107,110 @@ async def custom_401_handler(request: Request, exc: HTTPException):
|
||||
|
||||
return response
|
||||
|
||||
# Mount MCP app as sub-application
|
||||
app.mount("/mcp", mcp_app)
|
||||
# Mount MCP app as sub-application at /mcp-server to avoid path conflicts
|
||||
app.mount("/mcp-server", mcp_app)
|
||||
|
||||
# Add POST handler for /mcp to forward to mounted app
|
||||
@app.post("/mcp")
|
||||
async def mcp_post_handler(request: Request):
|
||||
"""Forward POST /mcp requests to mounted MCP app"""
|
||||
# Forward to the mounted app by calling it directly
|
||||
# Add custom route to handle /mcp requests and forward to mounted app
|
||||
@app.api_route("/mcp", methods=["POST", "DELETE", "OPTIONS"])
|
||||
@app.api_route("/mcp/", methods=["POST", "DELETE", "OPTIONS"])
|
||||
async def mcp_protocol_handler(request: Request):
|
||||
"""Handle MCP protocol requests by forwarding to mounted app"""
|
||||
|
||||
# Handle DELETE requests for session termination
|
||||
if request.method == "DELETE":
|
||||
logger.info("DELETE request received for session termination")
|
||||
# For session termination, we just return 200 OK
|
||||
# The actual session cleanup is handled by the underlying MCP transport
|
||||
from starlette.responses import Response
|
||||
return Response(
|
||||
status_code=200,
|
||||
content="Session terminated successfully"
|
||||
)
|
||||
|
||||
# REQUIRED: Validate Bearer JWT tokens for all MCP requests
|
||||
auth_header = request.headers.get("Authorization")
|
||||
if not auth_header or not auth_header.startswith("Bearer "):
|
||||
logger.error("Missing or invalid Authorization header")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Missing or invalid Authorization header. Bearer token required."
|
||||
)
|
||||
|
||||
token = auth_header.split(" ")[1]
|
||||
try:
|
||||
# Check if this is a mock token for development/testing
|
||||
if token.startswith("mock_clerk_jwt_"):
|
||||
logger.info(f"Using mock JWT token for development: {token[:30]}...")
|
||||
# For mock tokens, we'll allow access with a mock user
|
||||
request.state.user_id = "mock_user_dev"
|
||||
request.state.session_id = "mock_session_dev"
|
||||
request.state.token_scopes = ["read", "search"]
|
||||
logger.info("Mock JWT token accepted for development")
|
||||
elif token.startswith("eyJ"):
|
||||
# This looks like a real JWT token (starts with eyJ which is base64 encoded '{"')
|
||||
logger.info(f"Processing real JWT token: {token[:30]}...")
|
||||
# Validate real Clerk JWT token
|
||||
from clerk_backend_api import Clerk, models
|
||||
import jwt
|
||||
|
||||
# Decode JWT token and extract user info
|
||||
try:
|
||||
decoded_token = jwt.decode(token, options={"verify_signature": False})
|
||||
user_id = decoded_token.get("user_id") or decoded_token.get("sub")
|
||||
user_email = decoded_token.get("email")
|
||||
token_scopes = decoded_token.get("scopes", ["read", "search"])
|
||||
session_id = decoded_token.get("sid", "jwt_session")
|
||||
|
||||
logger.info(f"JWT token claims - user_id: {user_id}, email: {user_email}, scopes: {token_scopes}")
|
||||
|
||||
if user_id and user_email:
|
||||
# JWT token is signed by Clerk and contains valid user info
|
||||
request.state.user_id = user_id
|
||||
request.state.user_email = user_email
|
||||
request.state.session_id = session_id
|
||||
request.state.token_scopes = token_scopes
|
||||
logger.info(f"Real JWT token accepted for user: {user_id}")
|
||||
else:
|
||||
logger.error(f"Missing required fields in JWT token - user_id: {bool(user_id)}, email: {bool(user_email)}")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Invalid token - missing user_id or email in claims"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"JWT token decoding failed: {e}")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Invalid JWT token format"
|
||||
)
|
||||
else:
|
||||
# Invalid token format - doesn't start with expected patterns
|
||||
logger.error(f"Invalid token format: {token[:30]}...")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Invalid token format - must be a valid JWT token"
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
# Re-raise HTTPException as-is
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Bearer token validation failed: {str(e)}")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail=f"Token validation failed: {str(e)}"
|
||||
)
|
||||
|
||||
# Forward the request to the mounted MCP app
|
||||
async def receive():
|
||||
return await request.receive()
|
||||
|
||||
# Create a new scope for the mounted app
|
||||
# Create new scope for the mounted app
|
||||
scope = request.scope.copy()
|
||||
scope["path"] = "/" # Root path for the mounted app
|
||||
scope["path"] = "/" # Root path for mounted app
|
||||
scope["path_info"] = "/"
|
||||
|
||||
# Capture response
|
||||
# Capture the response
|
||||
response_parts = {"status": 200, "headers": [], "body": b""}
|
||||
|
||||
async def send(message):
|
||||
@@ -124,6 +238,9 @@ async def mcp_post_handler(request: Request):
|
||||
)
|
||||
|
||||
|
||||
# SSE transport deprecated - removed
|
||||
|
||||
|
||||
# FastAPI health check endpoint
|
||||
@app.get("/health")
|
||||
async def health_check():
|
||||
@@ -153,6 +270,9 @@ async def root():
|
||||
"oauth_google": "/auth/google/login",
|
||||
"user_info": "/auth/user"
|
||||
},
|
||||
"transports": {
|
||||
"http": "/mcp"
|
||||
},
|
||||
"supported_databases": [
|
||||
"Yargıtay (Court of Cassation)",
|
||||
"Danıştay (Council of State)",
|
||||
@@ -174,13 +294,14 @@ async def root():
|
||||
})
|
||||
|
||||
# OAuth 2.0 Authorization Server Metadata proxy (for MCP clients that can't reach Clerk directly)
|
||||
@app.get("/.well-known/oauth-authorization-server")
|
||||
# MCP Auth Toolkit expects this to be under /mcp/.well-known/oauth-authorization-server
|
||||
@app.get("/mcp/.well-known/oauth-authorization-server")
|
||||
async def oauth_authorization_server():
|
||||
"""OAuth 2.0 Authorization Server Metadata proxy to Clerk"""
|
||||
"""OAuth 2.0 Authorization Server Metadata proxy to Clerk - MCP Auth Toolkit standard location"""
|
||||
return JSONResponse({
|
||||
"issuer": CLERK_ISSUER,
|
||||
"authorization_endpoint": f"{BASE_URL}/auth/login",
|
||||
"token_endpoint": f"{BASE_URL}/auth/callback",
|
||||
"issuer": BASE_URL,
|
||||
"authorization_endpoint": "https://yargimcp.com/mcp-callback",
|
||||
"token_endpoint": f"{BASE_URL}/token",
|
||||
"jwks_uri": f"{CLERK_ISSUER}/.well-known/jwks.json",
|
||||
"response_types_supported": ["code"],
|
||||
"grant_types_supported": ["authorization_code", "refresh_token"],
|
||||
@@ -191,7 +312,65 @@ async def oauth_authorization_server():
|
||||
"claims_supported": ["sub", "iss", "aud", "exp", "iat", "email", "name"],
|
||||
"code_challenge_methods_supported": ["S256"],
|
||||
"service_documentation": f"{BASE_URL}/mcp",
|
||||
"registration_endpoint": f"{BASE_URL}/auth/register",
|
||||
"registration_endpoint": f"{BASE_URL}/register",
|
||||
"resource_documentation": f"{BASE_URL}/mcp"
|
||||
})
|
||||
|
||||
# Claude AI MCP specific endpoint format
|
||||
@app.get("/.well-known/oauth-authorization-server/mcp")
|
||||
async def oauth_authorization_server_mcp_suffix():
|
||||
"""OAuth 2.0 Authorization Server Metadata - Claude AI MCP specific format"""
|
||||
return JSONResponse({
|
||||
"issuer": BASE_URL,
|
||||
"authorization_endpoint": "https://yargimcp.com/mcp-callback",
|
||||
"token_endpoint": f"{BASE_URL}/token",
|
||||
"jwks_uri": f"{CLERK_ISSUER}/.well-known/jwks.json",
|
||||
"response_types_supported": ["code"],
|
||||
"grant_types_supported": ["authorization_code", "refresh_token"],
|
||||
"token_endpoint_auth_methods_supported": ["client_secret_basic", "none"],
|
||||
"scopes_supported": ["read", "search", "openid", "profile", "email"],
|
||||
"subject_types_supported": ["public"],
|
||||
"id_token_signing_alg_values_supported": ["RS256"],
|
||||
"claims_supported": ["sub", "iss", "aud", "exp", "iat", "email", "name"],
|
||||
"code_challenge_methods_supported": ["S256"],
|
||||
"service_documentation": f"{BASE_URL}/mcp",
|
||||
"registration_endpoint": f"{BASE_URL}/register",
|
||||
"resource_documentation": f"{BASE_URL}/mcp"
|
||||
})
|
||||
|
||||
@app.get("/.well-known/oauth-protected-resource/mcp")
|
||||
async def oauth_protected_resource_mcp_suffix():
|
||||
"""OAuth 2.0 Protected Resource Metadata - Claude AI MCP specific format"""
|
||||
return JSONResponse({
|
||||
"resource": BASE_URL,
|
||||
"authorization_servers": [
|
||||
BASE_URL
|
||||
],
|
||||
"scopes_supported": ["read", "search"],
|
||||
"bearer_methods_supported": ["header"],
|
||||
"resource_documentation": f"{BASE_URL}/mcp",
|
||||
"resource_policy_uri": f"{BASE_URL}/privacy"
|
||||
})
|
||||
|
||||
# Keep root level for compatibility with some MCP clients
|
||||
@app.get("/.well-known/oauth-authorization-server")
|
||||
async def oauth_authorization_server_root():
|
||||
"""OAuth 2.0 Authorization Server Metadata proxy to Clerk - root level for compatibility"""
|
||||
return JSONResponse({
|
||||
"issuer": BASE_URL,
|
||||
"authorization_endpoint": "https://yargimcp.com/mcp-callback",
|
||||
"token_endpoint": f"{BASE_URL}/token",
|
||||
"jwks_uri": f"{CLERK_ISSUER}/.well-known/jwks.json",
|
||||
"response_types_supported": ["code"],
|
||||
"grant_types_supported": ["authorization_code", "refresh_token"],
|
||||
"token_endpoint_auth_methods_supported": ["client_secret_basic", "none"],
|
||||
"scopes_supported": ["read", "search", "openid", "profile", "email"],
|
||||
"subject_types_supported": ["public"],
|
||||
"id_token_signing_alg_values_supported": ["RS256"],
|
||||
"claims_supported": ["sub", "iss", "aud", "exp", "iat", "email", "name"],
|
||||
"code_challenge_methods_supported": ["S256"],
|
||||
"service_documentation": f"{BASE_URL}/mcp",
|
||||
"registration_endpoint": f"{BASE_URL}/register",
|
||||
"resource_documentation": f"{BASE_URL}/mcp"
|
||||
})
|
||||
|
||||
@@ -205,12 +384,12 @@ async def mcp_info():
|
||||
"version": "0.1.0",
|
||||
"description": "MCP server for Turkish legal databases",
|
||||
"protocol": "mcp/1.0",
|
||||
"transport": "http",
|
||||
"transport": ["http"],
|
||||
"authentication_required": True,
|
||||
"authentication": {
|
||||
"type": "oauth2",
|
||||
"authorization_url": f"{BASE_URL}/auth/login",
|
||||
"token_url": f"{BASE_URL}/auth/callback",
|
||||
"authorization_url": "https://yargimcp.com/sign-in?redirect_url=https://api.yargimcp.com/auth/mcp-callback",
|
||||
"token_url": f"{BASE_URL}/auth/mcp-token",
|
||||
"scopes": ["read", "search"],
|
||||
"provider": "clerk"
|
||||
},
|
||||
@@ -231,7 +410,7 @@ async def mcp_info():
|
||||
"note": "This is an MCP server. Use POST to /mcp/ with proper MCP protocol headers.",
|
||||
"headers_required": [
|
||||
"Content-Type: application/json",
|
||||
"Accept: application/json, text/event-stream",
|
||||
"Accept: application/json",
|
||||
"Authorization: Bearer <token>",
|
||||
"X-Session-ID: <session-id>"
|
||||
]
|
||||
@@ -322,12 +501,178 @@ async def status():
|
||||
"auth_status": "enabled" if os.getenv("ENABLE_AUTH", "false").lower() == "true" else "disabled"
|
||||
})
|
||||
|
||||
# Alternative: SSE transport (for compatibility)
|
||||
sse_app = mcp_server.http_app(
|
||||
path="/sse",
|
||||
transport="sse",
|
||||
middleware=custom_middleware
|
||||
# Note: JWT token validation is now handled entirely by Clerk
|
||||
# All authentication flows use Clerk JWT tokens directly
|
||||
|
||||
async def validate_clerk_session(request: Request, clerk_token: str = None) -> str:
|
||||
"""Validate Clerk session from cookies or JWT token and return user_id"""
|
||||
logger.info(f"Validating Clerk session - token provided: {bool(clerk_token)}")
|
||||
|
||||
try:
|
||||
# Try to import Clerk SDK
|
||||
from clerk_backend_api import Clerk
|
||||
clerk = Clerk(bearer_auth=os.getenv("CLERK_SECRET_KEY"))
|
||||
|
||||
# Try JWT token first (from URL parameter)
|
||||
if clerk_token:
|
||||
logger.info("Validating Clerk JWT token from URL parameter")
|
||||
try:
|
||||
# Extract session_id from JWT token and verify with Clerk
|
||||
import jwt
|
||||
decoded_token = jwt.decode(clerk_token, options={"verify_signature": False})
|
||||
session_id = decoded_token.get("sid") # Use standard JWT 'sid' claim
|
||||
|
||||
if session_id:
|
||||
# Verify with Clerk using session_id
|
||||
session = clerk.sessions.verify(session_id=session_id, token=clerk_token)
|
||||
user_id = session.user_id if session else None
|
||||
|
||||
if user_id:
|
||||
logger.info(f"JWT token validation successful - user_id: {user_id}")
|
||||
return user_id
|
||||
else:
|
||||
logger.error("JWT token validation failed - no user_id in session")
|
||||
else:
|
||||
logger.error("No session_id found in JWT token")
|
||||
except Exception as e:
|
||||
logger.error(f"JWT token validation failed: {str(e)}")
|
||||
# Fall through to cookie validation
|
||||
|
||||
# Fallback to cookie validation
|
||||
logger.info("Attempting cookie-based session validation")
|
||||
clerk_session = request.cookies.get("__session")
|
||||
if not clerk_session:
|
||||
logger.error("No Clerk session cookie found")
|
||||
raise HTTPException(status_code=401, detail="No Clerk session found")
|
||||
|
||||
# Validate session with Clerk
|
||||
session = clerk.sessions.verify_session(clerk_session)
|
||||
logger.info(f"Cookie session validation successful - user_id: {session.user_id}")
|
||||
return session.user_id
|
||||
|
||||
except ImportError:
|
||||
# Fallback for development without Clerk SDK
|
||||
logger.warning("Clerk SDK not available - using development fallback")
|
||||
return "dev_user_123"
|
||||
except Exception as e:
|
||||
logger.error(f"Session validation failed: {str(e)}")
|
||||
raise HTTPException(status_code=401, detail=f"Session validation failed: {str(e)}")
|
||||
|
||||
# MCP OAuth Callback Endpoint
|
||||
@app.get("/auth/mcp-callback")
|
||||
async def mcp_oauth_callback(request: Request, clerk_token: str = Query(None)):
|
||||
"""Handle OAuth callback for MCP token generation"""
|
||||
logger.info(f"MCP OAuth callback - clerk_token provided: {bool(clerk_token)}")
|
||||
|
||||
try:
|
||||
# Validate Clerk session with JWT token support
|
||||
user_id = await validate_clerk_session(request, clerk_token)
|
||||
logger.info(f"User authenticated successfully - user_id: {user_id}")
|
||||
|
||||
# Use the Clerk JWT token directly (no need to generate custom token)
|
||||
logger.info("User authenticated successfully via Clerk")
|
||||
|
||||
# Return success response
|
||||
return HTMLResponse(f"""
|
||||
<html>
|
||||
<head>
|
||||
<title>MCP Connection Successful</title>
|
||||
<style>
|
||||
body {{ font-family: Arial, sans-serif; text-align: center; padding: 50px; }}
|
||||
.success {{ color: #28a745; }}
|
||||
.token {{ background: #f8f9fa; padding: 15px; border-radius: 5px; margin: 20px 0; word-break: break-all; }}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1 class="success">✅ MCP Connection Successful!</h1>
|
||||
<p>Your Yargı MCP integration is now active.</p>
|
||||
<div class="token">
|
||||
<strong>Authentication:</strong><br>
|
||||
<code>Use your Clerk JWT token directly with Bearer authentication</code>
|
||||
</div>
|
||||
<p>You can now close this window and return to your MCP client.</p>
|
||||
<script>
|
||||
// Try to close the popup if opened as such
|
||||
if (window.opener) {{
|
||||
window.opener.postMessage({{
|
||||
type: 'MCP_AUTH_SUCCESS',
|
||||
token: 'use_clerk_jwt_token'
|
||||
}}, '*');
|
||||
setTimeout(() => window.close(), 3000);
|
||||
}}
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
""")
|
||||
|
||||
except HTTPException as e:
|
||||
logger.error(f"MCP OAuth callback failed: {e.detail}")
|
||||
return HTMLResponse(f"""
|
||||
<html>
|
||||
<head>
|
||||
<title>MCP Connection Failed</title>
|
||||
<style>
|
||||
body {{ font-family: Arial, sans-serif; text-align: center; padding: 50px; }}
|
||||
.error {{ color: #dc3545; }}
|
||||
.debug {{ background: #f8f9fa; padding: 10px; margin: 20px 0; border-radius: 5px; font-family: monospace; }}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1 class="error">❌ MCP Connection Failed</h1>
|
||||
<p>{e.detail}</p>
|
||||
<div class="debug">
|
||||
<strong>Debug Info:</strong><br>
|
||||
Clerk Token: {'✅ Provided' if clerk_token else '❌ Missing'}<br>
|
||||
Error: {e.detail}<br>
|
||||
Status: {e.status_code}
|
||||
</div>
|
||||
<p>Please try again or contact support.</p>
|
||||
<a href="https://yargimcp.com/sign-in">Return to Sign In</a>
|
||||
</body>
|
||||
</html>
|
||||
""", status_code=e.status_code)
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error in MCP OAuth callback: {str(e)}")
|
||||
return HTMLResponse(f"""
|
||||
<html>
|
||||
<head>
|
||||
<title>MCP Connection Error</title>
|
||||
<style>
|
||||
body {{ font-family: Arial, sans-serif; text-align: center; padding: 50px; }}
|
||||
.error {{ color: #dc3545; }}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1 class="error">❌ Unexpected Error</h1>
|
||||
<p>An unexpected error occurred during authentication.</p>
|
||||
<p>Error: {str(e)}</p>
|
||||
<a href="https://yargimcp.com/sign-in">Return to Sign In</a>
|
||||
</body>
|
||||
</html>
|
||||
""", status_code=500)
|
||||
|
||||
# OAuth2 Token Endpoint - Now uses Clerk JWT tokens directly
|
||||
@app.post("/auth/mcp-token")
|
||||
async def mcp_token_endpoint(request: Request):
|
||||
"""OAuth2 token endpoint for MCP clients - returns Clerk JWT token info"""
|
||||
try:
|
||||
# Validate Clerk session
|
||||
user_id = await validate_clerk_session(request)
|
||||
|
||||
return JSONResponse({
|
||||
"message": "Use your Clerk JWT token directly with Bearer authentication",
|
||||
"token_type": "Bearer",
|
||||
"scope": "yargi.read",
|
||||
"user_id": user_id,
|
||||
"instructions": "Include 'Authorization: Bearer YOUR_CLERK_JWT_TOKEN' in your requests"
|
||||
})
|
||||
except HTTPException as e:
|
||||
return JSONResponse(
|
||||
status_code=e.status_code,
|
||||
content={"error": "invalid_request", "error_description": e.detail}
|
||||
)
|
||||
|
||||
# Note: Only HTTP transport supported - SSE transport deprecated
|
||||
|
||||
# Export for uvicorn
|
||||
__all__ = ["app", "sse_app"]
|
||||
__all__ = ["app"]
|
||||
@@ -5,8 +5,7 @@ import base64
|
||||
from typing import Optional
|
||||
import logging
|
||||
from markitdown import MarkItDown
|
||||
import tempfile
|
||||
import os
|
||||
import io
|
||||
|
||||
from .models import (
|
||||
BedestenSearchRequest, BedestenSearchResponse,
|
||||
@@ -125,17 +124,14 @@ class BedestenApiClient:
|
||||
if not html_content:
|
||||
return None
|
||||
|
||||
temp_file_path = None
|
||||
try:
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = html_content.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown()
|
||||
|
||||
# Write HTML to temp file
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp:
|
||||
tmp.write(html_content)
|
||||
temp_file_path = tmp.name
|
||||
|
||||
# Convert
|
||||
result = md_converter.convert(temp_file_path)
|
||||
result = md_converter.convert(html_stream)
|
||||
markdown_content = result.text_content
|
||||
|
||||
logger.info("Successfully converted HTML to Markdown")
|
||||
@@ -144,27 +140,19 @@ class BedestenApiClient:
|
||||
except Exception as e:
|
||||
logger.error(f"Error converting HTML to Markdown: {e}")
|
||||
return f"Error converting HTML content: {str(e)}"
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path):
|
||||
os.remove(temp_file_path)
|
||||
|
||||
def _convert_pdf_to_markdown(self, pdf_bytes: bytes) -> Optional[str]:
|
||||
"""Convert PDF to Markdown using MarkItDown"""
|
||||
if not pdf_bytes:
|
||||
return None
|
||||
|
||||
temp_file_path = None
|
||||
try:
|
||||
# MarkItDown supports PDF with markitdown[pdf]
|
||||
# Create BytesIO stream from PDF bytes
|
||||
pdf_stream = io.BytesIO(pdf_bytes)
|
||||
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown()
|
||||
|
||||
# Write PDF to temp file
|
||||
with tempfile.NamedTemporaryFile(mode="wb", delete=False, suffix=".pdf") as tmp:
|
||||
tmp.write(pdf_bytes)
|
||||
temp_file_path = tmp.name
|
||||
|
||||
# Convert
|
||||
result = md_converter.convert(temp_file_path)
|
||||
result = md_converter.convert(pdf_stream)
|
||||
markdown_content = result.text_content
|
||||
|
||||
logger.info("Successfully converted PDF to Markdown")
|
||||
@@ -173,9 +161,6 @@ class BedestenApiClient:
|
||||
except Exception as e:
|
||||
logger.error(f"Error converting PDF to Markdown: {e}")
|
||||
return f"Error converting PDF content: {str(e)}. The document may be corrupted or in an unsupported format."
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path):
|
||||
os.remove(temp_file_path)
|
||||
|
||||
async def close_client_session(self):
|
||||
"""Close HTTP client session"""
|
||||
|
||||
@@ -7,6 +7,15 @@ from datetime import datetime
|
||||
# Import YargitayBirimEnum for chamber filtering
|
||||
from yargitay_mcp_module.models import YargitayBirimEnum
|
||||
|
||||
# Court Type Options for Unified Search
|
||||
BedestenCourtTypeEnum = Literal[
|
||||
"YARGITAYKARARI", # Yargıtay (Court of Cassation)
|
||||
"DANISTAYKARAR", # Danıştay (Council of State)
|
||||
"YERELHUKUK", # Local Civil Courts
|
||||
"ISTINAFHUKUK", # Civil Courts of Appeals
|
||||
"KYB" # Extraordinary Appeals (Kanun Yararına Bozma)
|
||||
]
|
||||
|
||||
# Danıştay Chamber/Board Options
|
||||
DanistayBirimEnum = Literal[
|
||||
"ALL", # "ALL" for all chambers
|
||||
@@ -33,65 +42,15 @@ DanistayBirimEnum = Literal[
|
||||
|
||||
# Search Request Models
|
||||
class BedestenSearchData(BaseModel):
|
||||
pageSize: int = Field(..., description="""Number of results per page.
|
||||
Range: 1-100 results per page
|
||||
Recommended: 10-50 for balanced performance
|
||||
Higher values for comprehensive analysis""")
|
||||
pageNumber: int = Field(..., description="""Page number to retrieve (1-indexed).
|
||||
Start with 1 for first page
|
||||
Calculate total pages from response.data.total / pageSize
|
||||
Navigate: pageNumber=2 gets next set of results""")
|
||||
itemTypeList: List[str] = Field(..., description="""Court type filter - determines which court decisions to search:
|
||||
• ["YARGITAYKARARI"]: Court of Cassation (Yargıtay) - supreme court civil/criminal decisions
|
||||
• ["DANISTAYKARAR"]: Council of State (Danıştay) - administrative court decisions
|
||||
• ["YERELHUKUK"]: Local Civil Courts (Yerel Hukuk Mahkemeleri) - first instance civil decisions
|
||||
• ["ISTINAFHUKUK"]: Civil Courts of Appeals (İstinaf Hukuk Mahkemeleri) - appellate court decisions
|
||||
• ["KYB"]: Extraordinary Appeal (Kanun Yararına Bozma) - extraordinary appeal decisions
|
||||
Note: Use single-item list for specific court type targeting""")
|
||||
phrase: str = Field(..., description="""Search phrase/keyword with advanced search support:
|
||||
• Regular search: "mülkiyet kararı" - searches words separately
|
||||
• Exact phrase: "\"mülkiyet kararı\"" - searches exact phrase (more precise)
|
||||
• Legal concepts: "\"idari işlem\"", "\"sözleşme ihlali\"", "\"tazminat davası\""
|
||||
• Empty string: searches all documents (use with filters)
|
||||
Exact phrases significantly reduce false positives for precise legal research""")
|
||||
birimAdi: Optional[Union[YargitayBirimEnum, DanistayBirimEnum]] = Field(None, description="""
|
||||
Chamber/Department (Daire) filter (optional). Available options depend on itemTypeList:
|
||||
|
||||
For YARGITAYKARARI - Court of Cassation (52 options):
|
||||
- None/null for ALL chambers
|
||||
- 'Civil General Assembly (Hukuk Genel Kurulu)', '1st Civil Chamber (1. Hukuk Dairesi)' through '23rd Civil Chamber (23. Hukuk Dairesi)'
|
||||
- 'Criminal General Assembly (Ceza Genel Kurulu)', '1st Criminal Chamber (1. Ceza Dairesi)' through '23rd Criminal Chamber (23. Ceza Dairesi)'
|
||||
- 'Civil Chambers Presidents Board (Hukuk Daireleri Başkanlar Kurulu)', 'Criminal Chambers Presidents Board (Ceza Daireleri Başkanlar Kurulu)'
|
||||
- 'Grand General Assembly (Büyük Genel Kurulu)'
|
||||
|
||||
For DANISTAYKARAR - Council of State (27 options):
|
||||
- None/null for ALL chambers
|
||||
- 'Grand General Assembly (Büyük Gen.Kur.)', 'Administrative Cases Chambers Council (İdare Dava Daireleri Kurulu)', 'Tax Cases Chambers Council (Vergi Dava Daireleri Kurulu)'
|
||||
- '1st Chamber (1. Daire)' through '17th Chamber (17. Daire)'
|
||||
- 'Precedents Unification Council (İçtihatları Birleştirme Kurulu)', 'Administrative Affairs Council (İdari İşler Kurulu)', 'Presidents Council (Başkanlar Kurulu)'
|
||||
- Military courts: 'Military High Administrative Court (Askeri Yüksek İdare Mahkemesi)' variants
|
||||
""")
|
||||
kararTarihiStart: Optional[str] = Field(None, description="""Decision start date (Karar Tarihi Başlangıç) filter (optional).
|
||||
Format: YYYY-MM-DDTHH:MM:SS.000Z (ISO 8601 with Z timezone)
|
||||
Examples:
|
||||
• "2024-01-01T00:00:00.000Z" - from beginning of 2024
|
||||
• "2023-06-15T00:00:00.000Z" - from June 15, 2023
|
||||
• "2024-03-01T00:00:00.000Z" - from March 1, 2024
|
||||
Use with kararTarihiEnd for date range, or alone for "from date" filtering""")
|
||||
kararTarihiEnd: Optional[str] = Field(None, description="""Decision end date (Karar Tarihi Bitiş) filter (optional).
|
||||
Format: YYYY-MM-DDTHH:MM:SS.000Z (ISO 8601 with Z timezone)
|
||||
Examples:
|
||||
• "2024-12-31T23:59:59.999Z" - until end of 2024
|
||||
• "2023-12-31T23:59:59.999Z" - until end of 2023
|
||||
• "2024-06-30T23:59:59.999Z" - until end of June 2024
|
||||
Use with kararTarihiStart for date range, or alone for "until date" filtering""")
|
||||
sortFields: List[str] = Field(default=["KARAR_TARIHI"], description="""Sorting field (Sıralama Alanı) specification.
|
||||
["KARAR_TARIHI"]: Sort by decision date (Karar Tarihi) [DEFAULT]
|
||||
Most common use case for chronological ordering""")
|
||||
sortDirection: str = Field(default="desc", description="""Sort direction (Sıralama Yönü) for results.
|
||||
"desc": Descending order - newest decisions first [DEFAULT]
|
||||
"asc": Ascending order - oldest decisions first
|
||||
Recommended: "desc" for latest legal developments""")
|
||||
pageSize: int = Field(..., description="Results per page (1-10)")
|
||||
pageNumber: int = Field(..., description="Page number (1-indexed)")
|
||||
itemTypeList: List[str] = Field(..., description="Court type filter (YARGITAYKARARI/DANISTAYKARAR/YERELHUKUK/ISTINAFHUKUK/KYB)")
|
||||
phrase: str = Field(..., description="Search phrase (use \"exact phrase\" for precise matching)")
|
||||
birimAdi: Optional[Union[YargitayBirimEnum, DanistayBirimEnum]] = Field(None, description="Chamber filter (optional)")
|
||||
kararTarihiStart: Optional[str] = Field(None, description="Start date (ISO 8601 format)")
|
||||
kararTarihiEnd: Optional[str] = Field(None, description="End date (ISO 8601 format)")
|
||||
sortFields: List[str] = Field(default=["KARAR_TARIHI"], description="Sort fields")
|
||||
sortDirection: str = Field(default="desc", description="Sort direction (asc/desc)")
|
||||
|
||||
class BedestenSearchRequest(BaseModel):
|
||||
data: BedestenSearchData
|
||||
@@ -125,7 +84,7 @@ class BedestenSearchDataResponse(BaseModel):
|
||||
start: int
|
||||
|
||||
class BedestenSearchResponse(BaseModel):
|
||||
data: BedestenSearchDataResponse
|
||||
data: Optional[BedestenSearchDataResponse]
|
||||
metadata: Dict[str, Any]
|
||||
|
||||
# Document Request/Response Models
|
||||
|
||||
@@ -6,8 +6,7 @@ from typing import Dict, Any, List, Optional
|
||||
import logging
|
||||
import html
|
||||
import re
|
||||
import tempfile
|
||||
import os
|
||||
import io
|
||||
from markitdown import MarkItDown
|
||||
|
||||
from .models import (
|
||||
@@ -124,31 +123,28 @@ class DanistayApiClient:
|
||||
html_input_for_markdown = processed_html
|
||||
|
||||
markdown_text = None
|
||||
temp_file_path = None
|
||||
try:
|
||||
md_converter = MarkItDown() # Basic conversion
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = html_input_for_markdown.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file:
|
||||
tmp_file.write(html_input_for_markdown) # Write the full HTML string
|
||||
temp_file_path = tmp_file.name
|
||||
|
||||
conversion_result = md_converter.convert(temp_file_path)
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown()
|
||||
conversion_result = md_converter.convert(html_stream)
|
||||
markdown_text = conversion_result.text_content
|
||||
logger.info("DanistayApiClient: HTML to Markdown conversion successful.")
|
||||
except Exception as e:
|
||||
logger.error(f"DanistayApiClient: Error during MarkItDown HTML to Markdown conversion: {e}")
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path):
|
||||
os.remove(temp_file_path)
|
||||
|
||||
return markdown_text
|
||||
|
||||
async def get_decision_document_as_markdown(self, id: str) -> DanistayDocumentMarkdown:
|
||||
"""
|
||||
Retrieves a specific Danıştay decision by ID and returns its content as Markdown.
|
||||
The /getDokuman endpoint for Danıştay returns direct HTML.
|
||||
The /getDokuman endpoint for Danıştay requires arananKelime parameter.
|
||||
"""
|
||||
document_api_url = f"{self.DOCUMENT_ENDPOINT}?id={id}"
|
||||
# Add required arananKelime parameter - using empty string as minimum requirement
|
||||
document_api_url = f"{self.DOCUMENT_ENDPOINT}?id={id}&arananKelime="
|
||||
source_url = f"{self.BASE_URL}{document_api_url}"
|
||||
logger.info(f"DanistayApiClient: Fetching Danistay document for Markdown (ID: {id}) from {source_url}")
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ from typing import List, Optional, Dict, Any
|
||||
|
||||
class DanistayBaseSearchRequest(BaseModel):
|
||||
"""Base model for common search parameters for Danistay."""
|
||||
pageSize: int = Field(default=10, ge=1, le=100)
|
||||
pageSize: int = Field(default=10, ge=1, le=10)
|
||||
pageNumber: int = Field(default=1, ge=1)
|
||||
# siralama and siralamaDirection are part of detailed search, not necessarily keyword search
|
||||
# as per user's provided payloads.
|
||||
@@ -21,11 +21,11 @@ class DanistayKeywordSearchRequestData(BaseModel):
|
||||
|
||||
class DanistayKeywordSearchRequest(BaseModel): # This is the model the MCP tool will accept
|
||||
"""Model for keyword-based search request for Danistay."""
|
||||
andKelimeler: List[str] = Field(default_factory=list, description="Keywords for AND logic (VE Mantığı), e.g., ['word1', 'word2']")
|
||||
orKelimeler: List[str] = Field(default_factory=list, description="Keywords for OR logic (VEYA Mantığı).")
|
||||
notAndKelimeler: List[str] = Field(default_factory=list, description="Keywords for NOT AND logic (VE DEĞİL Mantığı).")
|
||||
notOrKelimeler: List[str] = Field(default_factory=list, description="Keywords for NOT OR logic (VEYA DEĞİL Mantığı).")
|
||||
pageSize: int = Field(default=10, ge=1, le=100)
|
||||
andKelimeler: List[str] = Field(default_factory=list, description="AND keywords")
|
||||
orKelimeler: List[str] = Field(default_factory=list, description="OR keywords")
|
||||
notAndKelimeler: List[str] = Field(default_factory=list, description="NOT AND keywords")
|
||||
notOrKelimeler: List[str] = Field(default_factory=list, description="NOT OR keywords")
|
||||
pageSize: int = Field(default=10, ge=1, le=10)
|
||||
pageNumber: int = Field(default=1, ge=1)
|
||||
|
||||
class DanistayDetailedSearchRequestData(BaseModel): # Internal data model for detailed search payload
|
||||
@@ -51,20 +51,20 @@ class DanistayDetailedSearchRequestData(BaseModel): # Internal data model for de
|
||||
|
||||
class DanistayDetailedSearchRequest(DanistayBaseSearchRequest): # MCP tool will accept this
|
||||
"""Model for detailed search request for Danistay."""
|
||||
daire: Optional[str] = Field(None, description="Chamber/Department name (e.g., '1. Daire').")
|
||||
esasYil: Optional[str] = Field(None, description="Case year for 'Esas No'.")
|
||||
esasIlkSiraNo: Optional[str] = Field(None, description="Starting sequence for 'Esas No'.")
|
||||
esasSonSiraNo: Optional[str] = Field(None, description="Ending sequence for 'Esas No'.")
|
||||
kararYil: Optional[str] = Field(None, description="Decision year for 'Karar No'.")
|
||||
kararIlkSiraNo: Optional[str] = Field(None, description="Starting sequence for 'Karar No'.")
|
||||
kararSonSiraNo: Optional[str] = Field(None, description="Ending sequence for 'Karar No'.")
|
||||
baslangicTarihi: Optional[str] = Field(None, description="Start date for decision (DD.MM.YYYY).")
|
||||
bitisTarihi: Optional[str] = Field(None, description="End date for decision (DD.MM.YYYY).")
|
||||
mevzuatNumarasi: Optional[str] = Field(None, description="Legislation number.")
|
||||
mevzuatAdi: Optional[str] = Field(None, description="Legislation name.")
|
||||
madde: Optional[str] = Field(None, description="Article number.")
|
||||
siralama: str = Field("1", description="Sorting criteria (e.g., 1: Esas No, 3: Karar Tarihi).")
|
||||
siralamaDirection: str = Field("desc", description="Sorting direction ('asc' or 'desc').")
|
||||
daire: Optional[str] = Field(None, description="Chamber")
|
||||
esasYil: Optional[str] = Field(None, description="Case year")
|
||||
esasIlkSiraNo: Optional[str] = Field(None, description="Start case no")
|
||||
esasSonSiraNo: Optional[str] = Field(None, description="End case no")
|
||||
kararYil: Optional[str] = Field(None, description="Decision year")
|
||||
kararIlkSiraNo: Optional[str] = Field(None, description="Start decision no")
|
||||
kararSonSiraNo: Optional[str] = Field(None, description="End decision no")
|
||||
baslangicTarihi: Optional[str] = Field(None, description="Start date")
|
||||
bitisTarihi: Optional[str] = Field(None, description="End date")
|
||||
mevzuatNumarasi: Optional[str] = Field(None, description="Law number")
|
||||
mevzuatAdi: Optional[str] = Field(None, description="Law name")
|
||||
madde: Optional[str] = Field(None, description="Article")
|
||||
siralama: str = Field("1", description="Sort by")
|
||||
siralamaDirection: str = Field("desc", description="Direction")
|
||||
# Add a general keyword field if detailed search also supports it
|
||||
# arananKelime: Optional[str] = Field(None, description="General keyword for detailed search.")
|
||||
|
||||
@@ -76,15 +76,15 @@ class DanistayApiDecisionEntry(BaseModel):
|
||||
id: str
|
||||
# The API response for keyword search uses "daireKurul", detailed search example uses "daire".
|
||||
# We use an alias to handle both and map to a consistent field name "chamber".
|
||||
chamber: Optional[str] = Field(None, alias="daire", description="The chamber or board.")
|
||||
chamber: Optional[str] = Field(None, alias="daire", description="Chamber")
|
||||
esasNo: Optional[str] = Field(None)
|
||||
kararNo: Optional[str] = Field(None)
|
||||
kararTarihi: Optional[str] = Field(None)
|
||||
arananKelime: Optional[str] = Field(None, description="Matched keyword (Aranan Kelime) if provided in response.")
|
||||
arananKelime: Optional[str] = Field(None, description="Keyword")
|
||||
# index: Optional[int] = None # Present in response, can be added if needed by MCP tool
|
||||
# siraNo: Optional[int] = None # Present in detailed response, can be added
|
||||
|
||||
document_url: Optional[HttpUrl] = Field(None, description="URL (Belge URL) to the full document, constructed by the client.")
|
||||
document_url: Optional[HttpUrl] = Field(None, description="Document URL")
|
||||
|
||||
model_config = ConfigDict(populate_by_name=True, extra='ignore') # Important for alias to work and ignore extra fields
|
||||
|
||||
@@ -93,11 +93,11 @@ class DanistayApiResponseInnerData(BaseModel):
|
||||
data: List[DanistayApiDecisionEntry]
|
||||
recordsTotal: int
|
||||
recordsFiltered: int
|
||||
draw: Optional[int] = Field(None, description="Draw counter (Çizim Sayıcısı) from API, usually for DataTables.")
|
||||
draw: Optional[int] = Field(None, description="Draw counter")
|
||||
|
||||
class DanistayApiResponse(BaseModel):
|
||||
"""Model for the complete search response from the Danistay API."""
|
||||
data: DanistayApiResponseInnerData
|
||||
data: Optional[DanistayApiResponseInnerData] = Field(None, description="Response data, can be null when no results found")
|
||||
metadata: Optional[Dict[str, Any]] = Field(None, description="Optional metadata (Meta Veri) from API.")
|
||||
|
||||
class DanistayDocumentMarkdown(BaseModel):
|
||||
|
||||
@@ -6,8 +6,7 @@ from typing import Dict, Any, List, Optional
|
||||
import logging
|
||||
import html
|
||||
import re
|
||||
import tempfile
|
||||
import os
|
||||
import io
|
||||
from markitdown import MarkItDown
|
||||
|
||||
from .models import (
|
||||
@@ -114,22 +113,18 @@ class EmsalApiClient:
|
||||
html_input_for_markdown = content
|
||||
|
||||
markdown_text = None
|
||||
temp_file_path = None
|
||||
try:
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = html_input_for_markdown.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown()
|
||||
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file:
|
||||
tmp_file.write(html_input_for_markdown)
|
||||
temp_file_path = tmp_file.name
|
||||
|
||||
conversion_result = md_converter.convert(temp_file_path)
|
||||
conversion_result = md_converter.convert(html_stream)
|
||||
markdown_text = conversion_result.text_content
|
||||
logger.info("EmsalApiClient: HTML to Markdown conversion successful.")
|
||||
except Exception as e:
|
||||
logger.error(f"EmsalApiClient: Error during MarkItDown HTML to Markdown conversion for Emsal: {e}")
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path):
|
||||
os.remove(temp_file_path)
|
||||
|
||||
return markdown_text
|
||||
|
||||
|
||||
+20
-20
@@ -17,7 +17,7 @@ class EmsalDetailedSearchRequestData(BaseModel):
|
||||
# Add other specific court type fields from the form if they are separate keys in payload
|
||||
# E.g., "Ceza Mahkemeleri", "İdari Mahkemeler" etc.
|
||||
|
||||
birimHukukMah: Optional[str] = Field("", description="List of selected Regional Civil Chambers (Bölge Hukuk Mahkemeleri), '+' separated.")
|
||||
birimHukukMah: Optional[str] = Field("", description="Regional chambers (+ separated)")
|
||||
|
||||
esasYil: Optional[str] = ""
|
||||
esasIlkSiraNo: Optional[str] = ""
|
||||
@@ -36,42 +36,42 @@ class EmsalDetailedSearchRequestData(BaseModel):
|
||||
|
||||
class EmsalSearchRequest(BaseModel): # This is the model the MCP tool will accept
|
||||
"""Model for Emsal detailed search request, with user-friendly field names."""
|
||||
keyword: Optional[str] = Field(None, description="Keyword (Anahtar Kelime) to search.")
|
||||
keyword: Optional[str] = Field(None, description="Keyword")
|
||||
|
||||
selected_bam_civil_court: Optional[str] = Field(None, description="Selected BAM Civil Court (Seçilen BAM Hukuk Mahkemesi) (maps to 'Bam Hukuk Mahkemeleri' payload key).")
|
||||
selected_civil_court: Optional[str] = Field(None, description="Selected Civil Court (Seçilen Hukuk Mahkemesi) (maps to 'Hukuk Mahkemeleri' payload key).")
|
||||
selected_regional_civil_chambers: Optional[List[str]] = Field(default_factory=list, description="Selected Regional Civil Chambers (Seçilen Bölge Hukuk Daireleri) (for 'birimHukukMah', joined by '+').")
|
||||
selected_bam_civil_court: Optional[str] = Field(None, description="BAM Civil Court")
|
||||
selected_civil_court: Optional[str] = Field(None, description="Civil Court")
|
||||
selected_regional_civil_chambers: Optional[List[str]] = Field(default_factory=list, description="Regional chambers")
|
||||
|
||||
case_year_esas: Optional[str] = Field(None, description="Case year (Dava Yılı) for 'Esas No'.")
|
||||
case_start_seq_esas: Optional[str] = Field(None, description="Starting sequence (Başlangıç Sırası) for 'Esas No'.")
|
||||
case_end_seq_esas: Optional[str] = Field(None, description="Ending sequence (Bitiş Sırası) for 'Esas No'.")
|
||||
case_year_esas: Optional[str] = Field(None, description="Case year")
|
||||
case_start_seq_esas: Optional[str] = Field(None, description="Start case no")
|
||||
case_end_seq_esas: Optional[str] = Field(None, description="End case no")
|
||||
|
||||
decision_year_karar: Optional[str] = Field(None, description="Decision year (Karar Yılı) for 'Karar No'.")
|
||||
decision_start_seq_karar: Optional[str] = Field(None, description="Starting sequence (Başlangıç Sırası) for 'Karar No'.")
|
||||
decision_end_seq_karar: Optional[str] = Field(None, description="Ending sequence (Bitiş Sırası) for 'Karar No'.")
|
||||
decision_year_karar: Optional[str] = Field(None, description="Decision year")
|
||||
decision_start_seq_karar: Optional[str] = Field(None, description="Start decision no")
|
||||
decision_end_seq_karar: Optional[str] = Field(None, description="End decision no")
|
||||
|
||||
start_date: Optional[str] = Field(None, description="Start date (Başlangıç Tarihi) for decision (DD.MM.YYYY).")
|
||||
end_date: Optional[str] = Field(None, description="End date (Bitiş Tarihi) for decision (DD.MM.YYYY).")
|
||||
start_date: Optional[str] = Field(None, description="Start date (DD.MM.YYYY)")
|
||||
end_date: Optional[str] = Field(None, description="End date (DD.MM.YYYY)")
|
||||
|
||||
sort_criteria: str = Field("1", description="Sorting criteria (Sıralama Kriteri) (e.g., 1: Esas No).")
|
||||
sort_direction: str = Field("desc", description="Sorting direction (Sıralama Yönü) ('asc' or 'desc').")
|
||||
sort_criteria: str = Field("1", description="Sort by")
|
||||
sort_direction: str = Field("desc", description="Direction")
|
||||
|
||||
page_number: int = Field(default=1, ge=1)
|
||||
page_size: int = Field(default=10, ge=1, le=100)
|
||||
page_size: int = Field(default=10, ge=1, le=10)
|
||||
|
||||
|
||||
class EmsalApiDecisionEntry(BaseModel):
|
||||
"""Model for an individual decision entry from the Emsal API search response."""
|
||||
id: str
|
||||
daire: Optional[str] = Field(None, description="The chamber/court (Daire/Mahkeme) that made the decision.")
|
||||
daire: Optional[str] = Field(None, description="Chamber")
|
||||
esasNo: Optional[str] = Field(None)
|
||||
kararNo: Optional[str] = Field(None)
|
||||
kararTarihi: Optional[str] = Field(None)
|
||||
arananKelime: Optional[str] = Field(None, description="Matched keyword (Aranan Kelime) from the search.")
|
||||
durum: Optional[str] = Field(None, description="Status (Durum) of the decision (e.g., 'KESİNLEŞMEDİ').")
|
||||
arananKelime: Optional[str] = Field(None, description="Keyword")
|
||||
durum: Optional[str] = Field(None, description="Status")
|
||||
# index: Optional[int] = None # Present in Emsal response, can be added if tool needs it
|
||||
|
||||
document_url: Optional[HttpUrl] = Field(None, description="URL (Belge URL) to the full document, constructed by the client.")
|
||||
document_url: Optional[HttpUrl] = Field(None, description="Document URL")
|
||||
|
||||
model_config = ConfigDict(extra='ignore')
|
||||
|
||||
|
||||
+792
-47
@@ -18,7 +18,8 @@ import html as html_parser
|
||||
from markitdown import MarkItDown
|
||||
import os
|
||||
import math
|
||||
import tempfile
|
||||
import io
|
||||
import random
|
||||
|
||||
from .models import (
|
||||
KikSearchRequest,
|
||||
@@ -69,14 +70,79 @@ class KikApiClient:
|
||||
self.playwright_instance = await async_playwright().start()
|
||||
if not self.browser or not self.browser.is_connected():
|
||||
if self.browser: await self.browser.close()
|
||||
self.browser = await self.playwright_instance.chromium.launch(headless=True)
|
||||
# Ultra stealth browser configuration
|
||||
self.browser = await self.playwright_instance.chromium.launch(
|
||||
headless=True,
|
||||
args=[
|
||||
# Disable automation indicators
|
||||
'--no-first-run',
|
||||
'--no-default-browser-check',
|
||||
'--disable-dev-shm-usage',
|
||||
'--disable-extensions',
|
||||
'--disable-gpu',
|
||||
'--disable-default-apps',
|
||||
'--disable-translate',
|
||||
'--disable-blink-features=AutomationControlled',
|
||||
'--disable-ipc-flooding-protection',
|
||||
'--disable-renderer-backgrounding',
|
||||
'--disable-backgrounding-occluded-windows',
|
||||
'--disable-client-side-phishing-detection',
|
||||
'--disable-sync',
|
||||
'--disable-features=TranslateUI,BlinkGenPropertyTrees',
|
||||
'--disable-component-extensions-with-background-pages',
|
||||
'--no-sandbox', # Sometimes needed for headless
|
||||
'--disable-web-security',
|
||||
'--disable-features=VizDisplayCompositor',
|
||||
# Language and locale
|
||||
'--lang=tr-TR',
|
||||
'--accept-lang=tr-TR,tr;q=0.9,en;q=0.8',
|
||||
# Performance optimizations
|
||||
'--memory-pressure-off',
|
||||
'--max_old_space_size=4096',
|
||||
]
|
||||
)
|
||||
browser_recreated = True
|
||||
if not self.context or browser_recreated:
|
||||
if self.context: await self.context.close()
|
||||
if not self.browser: raise PlaywrightError("Browser not initialized.")
|
||||
# Ultra realistic context configuration
|
||||
self.context = await self.browser.new_context(
|
||||
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.0.0 Safari/537.36",
|
||||
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
|
||||
viewport={'width': 1920, 'height': 1080},
|
||||
screen={'width': 1920, 'height': 1080},
|
||||
device_scale_factor=1.0,
|
||||
is_mobile=False,
|
||||
has_touch=False,
|
||||
# Localization
|
||||
locale='tr-TR',
|
||||
timezone_id='Europe/Istanbul',
|
||||
# Realistic browser features
|
||||
java_script_enabled=True,
|
||||
accept_downloads=True,
|
||||
ignore_https_errors=True,
|
||||
# Color scheme and media
|
||||
color_scheme='light',
|
||||
reduced_motion='no-preference',
|
||||
forced_colors='none',
|
||||
# Additional headers for realism
|
||||
extra_http_headers={
|
||||
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
|
||||
'Accept-Encoding': 'gzip, deflate, br',
|
||||
'Accept-Language': 'tr-TR,tr;q=0.9,en;q=0.8',
|
||||
'Cache-Control': 'max-age=0',
|
||||
'DNT': '1',
|
||||
'Upgrade-Insecure-Requests': '1',
|
||||
'Sec-Ch-Ua': '"Google Chrome";v="131", "Chromium";v="131", "Not_A Brand";v="24"',
|
||||
'Sec-Ch-Ua-Mobile': '?0',
|
||||
'Sec-Ch-Ua-Platform': '"Windows"',
|
||||
'Sec-Fetch-Dest': 'document',
|
||||
'Sec-Fetch-Mode': 'navigate',
|
||||
'Sec-Fetch-Site': 'none',
|
||||
'Sec-Fetch-User': '?1',
|
||||
},
|
||||
# Permissions to appear realistic
|
||||
permissions=['geolocation'],
|
||||
geolocation={'latitude': 41.0082, 'longitude': 28.9784}, # Istanbul
|
||||
)
|
||||
context_recreated = True
|
||||
if not self.page or self.page.is_closed() or force_new_page or context_recreated or browser_recreated:
|
||||
@@ -86,6 +152,9 @@ class KikApiClient:
|
||||
if not self.page: raise PlaywrightError("Failed to create new page.")
|
||||
self.page.set_default_navigation_timeout(self.request_timeout)
|
||||
self.page.set_default_timeout(self.request_timeout)
|
||||
|
||||
# CRITICAL: Anti-detection JavaScript injection
|
||||
await self._inject_stealth_scripts()
|
||||
if not self.page or self.page.is_closed():
|
||||
raise PlaywrightError("Playwright page initialization failed.")
|
||||
logger.debug("_ensure_playwright_ready completed.")
|
||||
@@ -99,29 +168,623 @@ class KikApiClient:
|
||||
if self.playwright_instance: await self.playwright_instance.stop(); self.playwright_instance = None
|
||||
logger.info("KikApiClient (Playwright): Resources closed.")
|
||||
|
||||
async def _inject_stealth_scripts(self):
|
||||
"""
|
||||
Inject comprehensive stealth JavaScript to evade bot detection.
|
||||
Overrides navigator properties and other fingerprinting vectors.
|
||||
"""
|
||||
if not self.page:
|
||||
logger.warning("Cannot inject stealth scripts: page is None")
|
||||
return
|
||||
|
||||
logger.debug("Injecting comprehensive stealth scripts...")
|
||||
|
||||
stealth_script = '''
|
||||
// Override navigator.webdriver
|
||||
Object.defineProperty(navigator, 'webdriver', {
|
||||
get: () => undefined,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
// Override navigator properties to appear more human
|
||||
Object.defineProperty(navigator, 'languages', {
|
||||
get: () => ['tr-TR', 'tr', 'en-US', 'en'],
|
||||
configurable: true
|
||||
});
|
||||
|
||||
Object.defineProperty(navigator, 'platform', {
|
||||
get: () => 'Win32',
|
||||
configurable: true
|
||||
});
|
||||
|
||||
Object.defineProperty(navigator, 'vendor', {
|
||||
get: () => 'Google Inc.',
|
||||
configurable: true
|
||||
});
|
||||
|
||||
Object.defineProperty(navigator, 'deviceMemory', {
|
||||
get: () => 8,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
Object.defineProperty(navigator, 'hardwareConcurrency', {
|
||||
get: () => 8,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
Object.defineProperty(navigator, 'maxTouchPoints', {
|
||||
get: () => 0,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
// Override plugins to appear realistic
|
||||
Object.defineProperty(navigator, 'plugins', {
|
||||
get: () => {
|
||||
return [
|
||||
{
|
||||
0: {type: "application/x-google-chrome-pdf", suffixes: "pdf", description: "Portable Document Format", enabledPlugin: Plugin},
|
||||
description: "Portable Document Format",
|
||||
filename: "internal-pdf-viewer",
|
||||
length: 1,
|
||||
name: "Chrome PDF Plugin"
|
||||
},
|
||||
{
|
||||
0: {type: "application/pdf", suffixes: "pdf", description: "", enabledPlugin: Plugin},
|
||||
description: "",
|
||||
filename: "mhjfbmdgcfjbbpaeojofohoefgiehjai",
|
||||
length: 1,
|
||||
name: "Chrome PDF Viewer"
|
||||
}
|
||||
];
|
||||
},
|
||||
configurable: true
|
||||
});
|
||||
|
||||
// Override permissions
|
||||
const originalQuery = window.navigator.permissions.query;
|
||||
window.navigator.permissions.query = (parameters) => (
|
||||
parameters.name === 'notifications' ?
|
||||
Promise.resolve({ state: Notification.permission }) :
|
||||
originalQuery(parameters)
|
||||
);
|
||||
|
||||
// Override WebGL rendering context
|
||||
const getParameter = WebGLRenderingContext.prototype.getParameter;
|
||||
WebGLRenderingContext.prototype.getParameter = function(parameter) {
|
||||
if (parameter === 37445) { // UNMASKED_VENDOR_WEBGL
|
||||
return 'Intel Inc.';
|
||||
}
|
||||
if (parameter === 37446) { // UNMASKED_RENDERER_WEBGL
|
||||
return 'Intel(R) Iris(R) Plus Graphics 640';
|
||||
}
|
||||
return getParameter(parameter);
|
||||
};
|
||||
|
||||
// Override canvas fingerprinting
|
||||
const toBlob = HTMLCanvasElement.prototype.toBlob;
|
||||
const toDataURL = HTMLCanvasElement.prototype.toDataURL;
|
||||
const getImageData = CanvasRenderingContext2D.prototype.getImageData;
|
||||
|
||||
const noisify = (canvas, context) => {
|
||||
const imageData = context.getImageData(0, 0, canvas.width, canvas.height);
|
||||
for (let i = 0; i < imageData.data.length; i += 4) {
|
||||
imageData.data[i] += Math.floor(Math.random() * 10) - 5;
|
||||
imageData.data[i + 1] += Math.floor(Math.random() * 10) - 5;
|
||||
imageData.data[i + 2] += Math.floor(Math.random() * 10) - 5;
|
||||
}
|
||||
context.putImageData(imageData, 0, 0);
|
||||
};
|
||||
|
||||
Object.defineProperty(HTMLCanvasElement.prototype, 'toBlob', {
|
||||
value: function(callback, type, encoderOptions) {
|
||||
noisify(this, this.getContext('2d'));
|
||||
return toBlob.apply(this, arguments);
|
||||
}
|
||||
});
|
||||
|
||||
Object.defineProperty(HTMLCanvasElement.prototype, 'toDataURL', {
|
||||
value: function(type, encoderOptions) {
|
||||
noisify(this, this.getContext('2d'));
|
||||
return toDataURL.apply(this, arguments);
|
||||
}
|
||||
});
|
||||
|
||||
// Override AudioContext for audio fingerprinting
|
||||
const audioCtx = new (window.AudioContext || window.webkitAudioContext)();
|
||||
const originalAnalyser = audioCtx.createAnalyser;
|
||||
audioCtx.createAnalyser = function() {
|
||||
const analyser = originalAnalyser.apply(this, arguments);
|
||||
const getFloatFrequencyData = analyser.getFloatFrequencyData;
|
||||
analyser.getFloatFrequencyData = function(array) {
|
||||
getFloatFrequencyData.apply(this, arguments);
|
||||
for (let i = 0; i < array.length; i++) {
|
||||
array[i] += Math.random() * 0.0001;
|
||||
}
|
||||
};
|
||||
return analyser;
|
||||
};
|
||||
|
||||
// Override screen properties
|
||||
Object.defineProperty(window.screen, 'colorDepth', {
|
||||
get: () => 24,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
Object.defineProperty(window.screen, 'pixelDepth', {
|
||||
get: () => 24,
|
||||
configurable: true
|
||||
});
|
||||
|
||||
// Override timezone
|
||||
Date.prototype.getTimezoneOffset = function() {
|
||||
return -180; // UTC+3 (Istanbul)
|
||||
};
|
||||
|
||||
// Override document.cookie to prevent tracking
|
||||
const originalCookieDescriptor = Object.getOwnPropertyDescriptor(Document.prototype, 'cookie') ||
|
||||
Object.getOwnPropertyDescriptor(HTMLDocument.prototype, 'cookie');
|
||||
if (originalCookieDescriptor && originalCookieDescriptor.configurable) {
|
||||
Object.defineProperty(document, 'cookie', {
|
||||
get: function() {
|
||||
return originalCookieDescriptor.get.call(this);
|
||||
},
|
||||
set: function(val) {
|
||||
console.log('Cookie set blocked:', val);
|
||||
return originalCookieDescriptor.set.call(this, val);
|
||||
},
|
||||
configurable: true
|
||||
});
|
||||
}
|
||||
|
||||
// Remove automation traces
|
||||
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Array;
|
||||
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Promise;
|
||||
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Symbol;
|
||||
delete window.cdc_adoQpoasnfa76pfcZLmcfl_JSON;
|
||||
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Object;
|
||||
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Proxy;
|
||||
|
||||
// Add realistic performance timing
|
||||
if (window.performance && window.performance.timing) {
|
||||
const timing = window.performance.timing;
|
||||
const now = Date.now();
|
||||
Object.defineProperty(timing, 'navigationStart', { value: now - Math.floor(Math.random() * 1000) + 1000, configurable: false });
|
||||
Object.defineProperty(timing, 'loadEventEnd', { value: now - Math.floor(Math.random() * 100) + 100, configurable: false });
|
||||
}
|
||||
|
||||
console.log('✓ Stealth scripts injected successfully');
|
||||
'''
|
||||
|
||||
try:
|
||||
await self.page.add_init_script(stealth_script)
|
||||
logger.debug("✅ Stealth scripts injected successfully")
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ Failed to inject stealth scripts: {e}")
|
||||
|
||||
async def _simulate_human_behavior(self, fast_mode: bool = True):
|
||||
"""
|
||||
Simulate realistic human behavior patterns to avoid detection.
|
||||
Includes mouse movements, typing patterns, and natural delays.
|
||||
|
||||
Args:
|
||||
fast_mode: If True, use minimal timing for speed optimization
|
||||
"""
|
||||
if not self.page:
|
||||
logger.warning("Cannot simulate human behavior: page is None")
|
||||
return
|
||||
|
||||
logger.debug("🤖 Simulating human behavior patterns...")
|
||||
|
||||
try:
|
||||
if fast_mode:
|
||||
# ULTRA-FAST MODE: Minimal human behavior
|
||||
viewport_size = self.page.viewport_size
|
||||
if viewport_size and random.random() < 0.7: # 70% chance to do movement
|
||||
width, height = viewport_size['width'], viewport_size['height']
|
||||
|
||||
# Single quick mouse movement
|
||||
x = random.randint(200, width - 200)
|
||||
y = random.randint(200, height - 200)
|
||||
await self.page.mouse.move(x, y)
|
||||
|
||||
# Brief scroll (50% chance)
|
||||
if random.random() < 0.5:
|
||||
await self.page.mouse.wheel(0, random.randint(50, 100))
|
||||
|
||||
# Ultra-minimal delay
|
||||
await asyncio.sleep(random.uniform(0.05, 0.15)) # Reduced from 0.1-0.3
|
||||
|
||||
else:
|
||||
# FULL MODE: Original comprehensive behavior
|
||||
viewport_size = self.page.viewport_size
|
||||
if viewport_size:
|
||||
width, height = viewport_size['width'], viewport_size['height']
|
||||
|
||||
# Generate 3-5 random mouse movements
|
||||
movements = random.randint(3, 5)
|
||||
logger.debug(f" 🖱️ Performing {movements} random mouse movements")
|
||||
|
||||
for i in range(movements):
|
||||
x = random.randint(100, width - 100)
|
||||
y = random.randint(100, height - 100)
|
||||
|
||||
# Move mouse with realistic speed (not instant)
|
||||
await self.page.mouse.move(x, y)
|
||||
await asyncio.sleep(random.uniform(0.1, 0.3))
|
||||
|
||||
# 2. Scroll simulation
|
||||
logger.debug(" 📜 Simulating scroll behavior")
|
||||
scroll_amount = random.randint(100, 300)
|
||||
await self.page.mouse.wheel(0, scroll_amount)
|
||||
await asyncio.sleep(random.uniform(0.2, 0.5))
|
||||
|
||||
# Scroll back up
|
||||
await self.page.mouse.wheel(0, -scroll_amount)
|
||||
await asyncio.sleep(random.uniform(0.2, 0.4))
|
||||
|
||||
# 3. Random page interaction delays
|
||||
await asyncio.sleep(random.uniform(0.5, 1.5))
|
||||
|
||||
logger.debug("✅ Human behavior simulation completed")
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ Human behavior simulation failed: {e}")
|
||||
|
||||
async def _human_type(self, selector: str, text: str, clear_first: bool = True, fast_mode: bool = True):
|
||||
"""
|
||||
Type text with human-like patterns and delays.
|
||||
|
||||
Args:
|
||||
selector: CSS selector for the input element
|
||||
text: Text to type
|
||||
clear_first: Whether to clear the field first
|
||||
fast_mode: If True, use minimal delays for speed optimization
|
||||
"""
|
||||
if not self.page:
|
||||
logger.warning("Cannot perform human typing: page is None")
|
||||
return
|
||||
|
||||
try:
|
||||
if fast_mode:
|
||||
# FAST MODE: Direct fill for speed
|
||||
await self.page.fill(selector, text)
|
||||
await asyncio.sleep(random.uniform(0.02, 0.05)) # Reduced from 0.05-0.1
|
||||
else:
|
||||
# FULL MODE: Character-by-character human typing
|
||||
# Focus on the element first
|
||||
await self.page.focus(selector)
|
||||
await asyncio.sleep(random.uniform(0.1, 0.3))
|
||||
|
||||
# Clear field if requested
|
||||
if clear_first:
|
||||
await self.page.keyboard.press('Control+a')
|
||||
await asyncio.sleep(random.uniform(0.05, 0.15))
|
||||
await self.page.keyboard.press('Delete')
|
||||
await asyncio.sleep(random.uniform(0.05, 0.15))
|
||||
|
||||
# Type each character with human-like delays
|
||||
for char in text:
|
||||
await self.page.keyboard.type(char)
|
||||
# Human typing speed: 50-150ms between characters
|
||||
delay = random.uniform(0.05, 0.15)
|
||||
|
||||
# Occasional longer pauses (thinking)
|
||||
if random.random() < 0.1: # 10% chance
|
||||
delay += random.uniform(0.2, 0.8)
|
||||
|
||||
await asyncio.sleep(delay)
|
||||
|
||||
# Brief pause after typing
|
||||
await asyncio.sleep(random.uniform(0.2, 0.6))
|
||||
|
||||
logger.debug(f"✅ Human-typed '{text}' into {selector}")
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ Human typing failed: {e}")
|
||||
|
||||
async def _human_click(self, selector: str, wait_before: bool = True, wait_after: bool = True, fast_mode: bool = True):
|
||||
"""
|
||||
Perform a human-like click with realistic delays and mouse movement.
|
||||
|
||||
Args:
|
||||
selector: CSS selector or element to click
|
||||
wait_before: Whether to wait before clicking
|
||||
wait_after: Whether to wait after clicking
|
||||
fast_mode: If True, use minimal delays for speed optimization
|
||||
"""
|
||||
if not self.page:
|
||||
logger.warning("Cannot perform human click: page is None")
|
||||
return
|
||||
|
||||
try:
|
||||
if fast_mode:
|
||||
# FAST MODE: Direct click with minimal delay
|
||||
if wait_before:
|
||||
await asyncio.sleep(random.uniform(0.02, 0.08)) # Reduced from 0.05-0.15
|
||||
|
||||
await self.page.click(selector)
|
||||
|
||||
if wait_after:
|
||||
await asyncio.sleep(random.uniform(0.02, 0.08)) # Reduced from 0.05-0.15
|
||||
|
||||
else:
|
||||
# FULL MODE: Realistic mouse movement and timing
|
||||
# Wait before clicking (thinking time)
|
||||
if wait_before:
|
||||
await asyncio.sleep(random.uniform(0.3, 0.8))
|
||||
|
||||
# Get element bounds for realistic mouse movement
|
||||
element = await self.page.query_selector(selector)
|
||||
if element:
|
||||
box = await element.bounding_box()
|
||||
if box:
|
||||
# Move to element with slight randomness
|
||||
center_x = box['x'] + box['width'] / 2
|
||||
center_y = box['y'] + box['height'] / 2
|
||||
|
||||
# Add small random offset
|
||||
offset_x = random.uniform(-10, 10)
|
||||
offset_y = random.uniform(-5, 5)
|
||||
|
||||
await self.page.mouse.move(center_x + offset_x, center_y + offset_y)
|
||||
await asyncio.sleep(random.uniform(0.1, 0.3))
|
||||
|
||||
# Perform click
|
||||
await self.page.mouse.click(center_x + offset_x, center_y + offset_y)
|
||||
|
||||
logger.debug(f"✅ Human-clicked {selector}")
|
||||
else:
|
||||
# Fallback to regular click
|
||||
await self.page.click(selector)
|
||||
logger.debug(f"✅ Fallback-clicked {selector}")
|
||||
else:
|
||||
logger.warning(f"⚠️ Element not found for human click: {selector}")
|
||||
return
|
||||
|
||||
# Wait after clicking (processing time)
|
||||
if wait_after:
|
||||
await asyncio.sleep(random.uniform(0.2, 0.6))
|
||||
|
||||
logger.debug(f"✅ Human-clicked {selector}")
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"⚠️ Human click failed: {e}")
|
||||
|
||||
async def _simulate_page_exploration(self, fast_mode: bool = True):
|
||||
"""
|
||||
Simulate natural page exploration before performing the main task.
|
||||
This helps establish a more human-like session.
|
||||
|
||||
Args:
|
||||
fast_mode: If True, use minimal exploration for speed optimization
|
||||
"""
|
||||
if not self.page:
|
||||
return
|
||||
|
||||
logger.debug("🕵️ Simulating page exploration...")
|
||||
|
||||
try:
|
||||
if fast_mode:
|
||||
# ULTRA-FAST MODE: Minimal exploration
|
||||
await asyncio.sleep(random.uniform(0.05, 0.1)) # Reduced from 0.1-0.3
|
||||
|
||||
# Single mouse movement (optional)
|
||||
try:
|
||||
elements = await self.page.query_selector_all("input, button")
|
||||
if elements and random.random() < 0.5: # 50% chance to skip
|
||||
element = random.choice(elements)
|
||||
box = await element.bounding_box()
|
||||
if box:
|
||||
center_x = box['x'] + box['width'] / 2
|
||||
center_y = box['y'] + box['height'] / 2
|
||||
await self.page.mouse.move(center_x, center_y)
|
||||
except:
|
||||
pass
|
||||
|
||||
await asyncio.sleep(random.uniform(0.02, 0.05)) # Reduced from 0.05-0.15
|
||||
|
||||
else:
|
||||
# FULL MODE: Comprehensive exploration
|
||||
# 1. Brief pause to "read" the page
|
||||
await asyncio.sleep(random.uniform(1.0, 2.5))
|
||||
|
||||
# 2. Move mouse to various UI elements (like a human would explore)
|
||||
explore_selectors = [
|
||||
"h1", "h2", ".navbar", "#header", ".logo",
|
||||
"input", "button", "a", ".form-group"
|
||||
]
|
||||
|
||||
explored = 0
|
||||
for selector in explore_selectors:
|
||||
elements = await self.page.query_selector_all(selector)
|
||||
if elements and explored < 3: # Explore max 3 elements
|
||||
element = random.choice(elements)
|
||||
box = await element.bounding_box()
|
||||
if box:
|
||||
center_x = box['x'] + box['width'] / 2
|
||||
center_y = box['y'] + box['height'] / 2
|
||||
|
||||
await self.page.mouse.move(center_x, center_y)
|
||||
await asyncio.sleep(random.uniform(0.3, 0.8))
|
||||
explored += 1
|
||||
|
||||
# 3. Small scroll to simulate reading
|
||||
await self.page.mouse.wheel(0, random.randint(50, 150))
|
||||
await asyncio.sleep(random.uniform(0.5, 1.2))
|
||||
|
||||
logger.debug("✅ Page exploration completed")
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"⚠️ Page exploration failed: {e}")
|
||||
|
||||
def _parse_decision_entries_from_soup(self, soup: BeautifulSoup, search_karar_tipi: KikKararTipi) -> List[KikDecisionEntry]:
|
||||
entries: List[KikDecisionEntry] = []
|
||||
table = soup.find("table", {"id": self.RESULTS_TABLE_ID})
|
||||
if not table: return entries
|
||||
|
||||
logger.debug(f"Looking for table with ID: {self.RESULTS_TABLE_ID}")
|
||||
if not table:
|
||||
logger.warning(f"Table with ID '{self.RESULTS_TABLE_ID}' not found in HTML")
|
||||
# Log available tables for debugging
|
||||
all_tables = soup.find_all("table")
|
||||
logger.debug(f"Found {len(all_tables)} tables in HTML")
|
||||
for idx, tbl in enumerate(all_tables):
|
||||
table_id = tbl.get('id', 'no-id')
|
||||
table_class = tbl.get('class', 'no-class')
|
||||
rows = tbl.find_all('tr')
|
||||
logger.debug(f"Table {idx}: id='{table_id}', class='{table_class}', rows={len(rows)}")
|
||||
|
||||
# If this looks like a results table, try to use it
|
||||
if (table_id and ('grd' in table_id.lower() or 'kurul' in table_id.lower() or 'sonuc' in table_id.lower())) or \
|
||||
(isinstance(table_class, list) and any('grid' in cls.lower() or 'result' in cls.lower() for cls in table_class)) or \
|
||||
len(rows) > 3: # Table with multiple rows might be results
|
||||
logger.info(f"Trying to parse table {idx} as potential results table: id='{table_id}'")
|
||||
table = tbl
|
||||
break
|
||||
|
||||
if not table:
|
||||
logger.error("No suitable results table found")
|
||||
return entries
|
||||
|
||||
rows = table.find_all("tr")
|
||||
logger.info(f"Found {len(rows)} rows in results table")
|
||||
|
||||
for row_idx, row in enumerate(rows):
|
||||
if row_idx < 2: continue
|
||||
# Skip first row (search bar with colspan=7) and second row (header with 6 cells)
|
||||
if row_idx < 2:
|
||||
logger.debug(f"Skipping header row {row_idx}")
|
||||
continue
|
||||
|
||||
cells = row.find_all("td")
|
||||
if len(cells) == 6:
|
||||
logger.debug(f"Row {row_idx}: Found {len(cells)} cells")
|
||||
|
||||
# Log cell contents for debugging
|
||||
if cells and row_idx < 5: # Log first few data rows
|
||||
for cell_idx, cell in enumerate(cells):
|
||||
cell_text = cell.get_text(strip=True)[:50] # First 50 chars
|
||||
logger.debug(f" Cell {cell_idx}: '{cell_text}...'")
|
||||
|
||||
# Be more flexible with cell count - try 6 cells first, then adapt
|
||||
if len(cells) >= 5: # At least 5 cells for minimum required data
|
||||
try:
|
||||
preview_button_tag = cells[0].find("a", id=re.compile(r"btnOnizle$"))
|
||||
# Try to find preview button in first cell or any cell with a link
|
||||
preview_button_tag = None
|
||||
event_target = ""
|
||||
if preview_button_tag and preview_button_tag.has_attr('href'):
|
||||
match = re.search(r"__doPostBack\('([^']*)','([^']*)'\)", preview_button_tag['href'])
|
||||
if match: event_target = match.group(1)
|
||||
karar_no_span = cells[1].find("span", id=re.compile(r"lblKno$"))
|
||||
karar_tarihi_span = cells[2].find("span", id=re.compile(r"lblKtar$"))
|
||||
idare_span = cells[3].find("span", id=re.compile(r"lblIdare$"))
|
||||
basvuru_sahibi_span = cells[4].find("span", id=re.compile(r"lblSikayetci$"))
|
||||
ihale_span = cells[5].find("span", id=re.compile(r"lblIhale$"))
|
||||
if not (event_target and karar_no_span and karar_tarihi_span): continue
|
||||
|
||||
# Look for preview button in first few cells
|
||||
for cell_idx in range(min(3, len(cells))):
|
||||
cell = cells[cell_idx]
|
||||
# Try multiple patterns for preview button (based on actual HTML structure)
|
||||
preview_candidates = [
|
||||
cell.find("a", id="btnOnizle"), # Exact match
|
||||
cell.find("a", id=re.compile(r"btnOnizle$")),
|
||||
cell.find("a", id=re.compile(r"btn.*Onizle")),
|
||||
cell.find("a", id=re.compile(r".*Onizle.*")),
|
||||
cell.find("a", href=re.compile(r"__doPostBack"))
|
||||
]
|
||||
|
||||
for candidate in preview_candidates:
|
||||
if candidate and candidate.has_attr('href'):
|
||||
match = re.search(r"__doPostBack\('([^']*)','([^']*)'\)", candidate['href'])
|
||||
if match:
|
||||
event_target = match.group(1)
|
||||
preview_button_tag = candidate
|
||||
logger.debug(f"Row {row_idx}: Found event_target '{event_target}' in cell {cell_idx}")
|
||||
break
|
||||
|
||||
if preview_button_tag:
|
||||
break
|
||||
|
||||
if not preview_button_tag:
|
||||
logger.debug(f"Row {row_idx}: No preview button found in any cell")
|
||||
# Log what links we found
|
||||
for cell_idx, cell in enumerate(cells[:3]):
|
||||
links_in_cell = cell.find_all("a")
|
||||
logger.debug(f" Cell {cell_idx}: {len(links_in_cell)} links")
|
||||
for link in links_in_cell[:2]:
|
||||
logger.debug(f" Link id='{link.get('id')}', href='{link.get('href', '')[:50]}...'")
|
||||
|
||||
# Try to find decision data spans with more flexible patterns
|
||||
karar_no_span = None
|
||||
karar_tarihi_span = None
|
||||
idare_span = None
|
||||
basvuru_sahibi_span = None
|
||||
ihale_span = None
|
||||
|
||||
# Try different span patterns for karar no (usually in cell 1)
|
||||
for cell_idx in range(min(4, len(cells))):
|
||||
if not karar_no_span:
|
||||
cell = cells[cell_idx]
|
||||
candidates = [
|
||||
cell.find("span", id="lblKno"), # Exact match based on actual HTML
|
||||
cell.find("span", id=re.compile(r"lblKno$")),
|
||||
cell.find("span", id=re.compile(r".*Kno.*")),
|
||||
cell.find("span", id=re.compile(r".*KararNo.*")),
|
||||
cell.find("span", id=re.compile(r".*No.*"))
|
||||
]
|
||||
for candidate in candidates:
|
||||
if candidate and candidate.get_text(strip=True):
|
||||
karar_no_span = candidate
|
||||
logger.debug(f"Row {row_idx}: Found karar_no in cell {cell_idx}")
|
||||
break
|
||||
|
||||
# Try different patterns for karar tarihi (usually in cell 2)
|
||||
for cell_idx in range(min(4, len(cells))):
|
||||
if not karar_tarihi_span:
|
||||
cell = cells[cell_idx]
|
||||
candidates = [
|
||||
cell.find("span", id="lblKtar"), # Exact match based on actual HTML
|
||||
cell.find("span", id=re.compile(r"lblKtar$")),
|
||||
cell.find("span", id=re.compile(r".*Ktar.*")),
|
||||
cell.find("span", id=re.compile(r".*Tarih.*")),
|
||||
cell.find("span", id=re.compile(r".*Date.*"))
|
||||
]
|
||||
for candidate in candidates:
|
||||
if candidate and candidate.get_text(strip=True):
|
||||
# Check if it looks like a date
|
||||
text = candidate.get_text(strip=True)
|
||||
if re.match(r'\d{1,2}[./]\d{1,2}[./]\d{4}', text):
|
||||
karar_tarihi_span = candidate
|
||||
logger.debug(f"Row {row_idx}: Found karar_tarihi in cell {cell_idx}")
|
||||
break
|
||||
|
||||
# Find other spans in remaining cells (if we have 6 cells) - using exact IDs
|
||||
if len(cells) >= 6:
|
||||
idare_span = cells[3].find("span", id="lblIdare") or cells[3].find("span")
|
||||
basvuru_sahibi_span = cells[4].find("span", id="lblSikayetci") or cells[4].find("span")
|
||||
ihale_span = cells[5].find("span", id="lblIhale") or cells[5].find("span")
|
||||
elif len(cells) == 5:
|
||||
# Adjust for 5-cell layout
|
||||
idare_span = cells[2].find("span") if cells[2] != cells[1] else None
|
||||
basvuru_sahibi_span = cells[3].find("span") if len(cells) > 3 else None
|
||||
ihale_span = cells[4].find("span") if len(cells) > 4 else None
|
||||
|
||||
# Log what we found
|
||||
logger.debug(f"Row {row_idx}: karar_no_span={karar_no_span is not None}, "
|
||||
f"karar_tarihi_span={karar_tarihi_span is not None}, "
|
||||
f"event_target={bool(event_target)}")
|
||||
|
||||
# For KIK, we need at least karar_no and karar_tarihi, event_target is helpful but not critical
|
||||
if not (karar_no_span and karar_tarihi_span):
|
||||
logger.debug(f"Row {row_idx}: Missing required fields (karar_no or karar_tarihi), skipping")
|
||||
# Log what spans we found in cells
|
||||
for i, cell in enumerate(cells):
|
||||
spans = cell.find_all("span")
|
||||
if spans:
|
||||
span_info = []
|
||||
for s in spans:
|
||||
span_id = s.get('id', 'no-id')
|
||||
span_text = s.get_text(strip=True)[:20]
|
||||
span_info.append(f"{span_id}:'{span_text}...'")
|
||||
logger.debug(f" Cell {i} spans: {span_info}")
|
||||
continue
|
||||
|
||||
# If we don't have event_target, we can still create an entry but mark it specially
|
||||
if not event_target:
|
||||
logger.warning(f"Row {row_idx}: No event_target found, document retrieval won't work")
|
||||
event_target = f"missing_target_row_{row_idx}" # Placeholder
|
||||
|
||||
# Karar tipini arama parametresinden alıyoruz, çünkü HTML'de direkt olarak bulunmuyor.
|
||||
try:
|
||||
entry = KikDecisionEntry(
|
||||
preview_event_target=event_target,
|
||||
kararNo=karar_no_span.get_text(strip=True),
|
||||
@@ -132,8 +795,17 @@ class KikApiClient:
|
||||
ihaleKonusu=ihale_span.get_text(strip=True) if ihale_span else None,
|
||||
)
|
||||
entries.append(entry)
|
||||
logger.info(f"Row {row_idx}: Successfully parsed decision: {entry.karar_no_str}")
|
||||
except Exception as e:
|
||||
logger.error(f"Error parsing a KIK decision entry row: {e}", exc_info=True)
|
||||
logger.error(f"Row {row_idx}: Error creating KikDecisionEntry: {e}")
|
||||
continue
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error parsing row {row_idx}: {e}", exc_info=True)
|
||||
else:
|
||||
logger.debug(f"Row {row_idx}: Expected at least 5 cells but found {len(cells)}, skipping")
|
||||
|
||||
logger.info(f"Parsed {len(entries)} decision entries from {len(rows)} rows")
|
||||
return entries
|
||||
|
||||
def _parse_total_records_from_soup(self, soup: BeautifulSoup) -> int:
|
||||
@@ -163,6 +835,10 @@ class KikApiClient:
|
||||
try:
|
||||
if page.url != search_url:
|
||||
await page.goto(search_url, wait_until="networkidle", timeout=self.request_timeout)
|
||||
|
||||
# Simulate natural page exploration after navigation (FAST MODE)
|
||||
await self._simulate_page_exploration(fast_mode=True)
|
||||
|
||||
search_button_selector = f"a[id='{self.FIELD_LOCATORS['search_button_id']}']"
|
||||
await page.wait_for_selector(search_button_selector, state="visible", timeout=self.request_timeout)
|
||||
|
||||
@@ -170,12 +846,18 @@ class KikApiClient:
|
||||
radio_locator_selector = f"{self.FIELD_LOCATORS['karar_tipi_radio_group']}[value='{current_karar_tipi_value}']"
|
||||
if not await page.locator(radio_locator_selector).is_checked():
|
||||
js_target_radio = f"ctl00$ContentPlaceHolder1${current_karar_tipi_value}"
|
||||
logger.info(f"Selecting radio button: {js_target_radio}")
|
||||
async with page.expect_navigation(wait_until="networkidle", timeout=self.request_timeout):
|
||||
await page.evaluate(f"javascript:__doPostBack('{js_target_radio}','')")
|
||||
await page.wait_for_timeout(1000)
|
||||
# Ultra-fast wait for page to stabilize after radio button change
|
||||
await page.wait_for_timeout(300) # Reduced from 1000ms
|
||||
logger.info("Radio button selection completed")
|
||||
|
||||
async def fill_if_value(selector_key: str, value: Optional[str]):
|
||||
if value is not None: await page.fill(self.FIELD_LOCATORS[selector_key], value)
|
||||
# Helper function for human-like form filling (FAST MODE)
|
||||
async def human_fill_if_value(selector_key: str, value: Optional[str]):
|
||||
if value is not None:
|
||||
selector = self.FIELD_LOCATORS[selector_key]
|
||||
await self._human_type(selector, value, fast_mode=True)
|
||||
|
||||
# Karar No'yu KİK sitesine göndermeden önce '_' -> '/' dönüşümü yap
|
||||
karar_no_for_kik_form = None
|
||||
@@ -183,43 +865,105 @@ class KikApiClient:
|
||||
karar_no_for_kik_form = search_params.karar_no.replace('_', '/')
|
||||
logger.info(f"Using karar_no '{karar_no_for_kik_form}' (transformed from '{search_params.karar_no}') for KIK form.")
|
||||
|
||||
await fill_if_value('karar_metni', search_params.karar_metni)
|
||||
await fill_if_value('karar_no', karar_no_for_kik_form) # Dönüştürülmüş halini kullan
|
||||
# ... (diğer fill_if_value çağrıları aynı) ...
|
||||
await fill_if_value('karar_tarihi_baslangic', search_params.karar_tarihi_baslangic)
|
||||
await fill_if_value('karar_tarihi_bitis', search_params.karar_tarihi_bitis)
|
||||
await fill_if_value('resmi_gazete_sayisi', search_params.resmi_gazete_sayisi)
|
||||
await fill_if_value('resmi_gazete_tarihi', search_params.resmi_gazete_tarihi)
|
||||
await fill_if_value('basvuru_konusu_ihale', search_params.basvuru_konusu_ihale)
|
||||
await fill_if_value('basvuru_sahibi', search_params.basvuru_sahibi)
|
||||
await fill_if_value('ihaleyi_yapan_idare', search_params.ihaleyi_yapan_idare)
|
||||
# Fill form fields with FAST human-like behavior
|
||||
logger.info("Filling form fields with fast mode...")
|
||||
|
||||
# Start with FAST mouse behavior simulation
|
||||
await self._simulate_human_behavior(fast_mode=True)
|
||||
|
||||
await human_fill_if_value('karar_metni', search_params.karar_metni)
|
||||
await human_fill_if_value('karar_no', karar_no_for_kik_form) # Dönüştürülmüş halini kullan
|
||||
await human_fill_if_value('karar_tarihi_baslangic', search_params.karar_tarihi_baslangic)
|
||||
await human_fill_if_value('karar_tarihi_bitis', search_params.karar_tarihi_bitis)
|
||||
await human_fill_if_value('resmi_gazete_sayisi', search_params.resmi_gazete_sayisi)
|
||||
await human_fill_if_value('resmi_gazete_tarihi', search_params.resmi_gazete_tarihi)
|
||||
await human_fill_if_value('basvuru_konusu_ihale', search_params.basvuru_konusu_ihale)
|
||||
await human_fill_if_value('basvuru_sahibi', search_params.basvuru_sahibi)
|
||||
await human_fill_if_value('ihaleyi_yapan_idare', search_params.ihaleyi_yapan_idare)
|
||||
|
||||
if search_params.yil:
|
||||
await page.select_option(self.FIELD_LOCATORS['yil'], value=search_params.yil)
|
||||
await page.wait_for_timeout(50) # Reduced from 100ms
|
||||
|
||||
logger.info("Form filling completed, preparing for search...")
|
||||
|
||||
# Additional FAST human behavior before search
|
||||
await self._simulate_human_behavior(fast_mode=True)
|
||||
|
||||
action_is_search_button_click = (search_params.page == 1)
|
||||
event_target_for_submit: str
|
||||
if action_is_search_button_click:
|
||||
event_target_for_submit = self.FIELD_LOCATORS['search_button_id']
|
||||
else: # Pagination
|
||||
page_link_ctl_number = search_params.page + 2
|
||||
event_target_for_submit = f"ctl00$ContentPlaceHolder1$grdKurulKararSorguSonuc$ctl14$ctl{page_link_ctl_number:02d}"
|
||||
|
||||
try:
|
||||
async with page.expect_navigation(wait_until="networkidle", timeout=self.request_timeout):
|
||||
if action_is_search_button_click:
|
||||
await page.locator(search_button_selector).click()
|
||||
else:
|
||||
event_target_for_submit = self.FIELD_LOCATORS['search_button_id']
|
||||
# Use human-like clicking for search button
|
||||
search_button_selector = f"a[id='{event_target_for_submit}']"
|
||||
logger.info(f"Performing human-like search button click...")
|
||||
|
||||
try:
|
||||
# FAST Human-like click on search button
|
||||
await self._human_click(search_button_selector, wait_before=True, wait_after=False, fast_mode=True)
|
||||
|
||||
# Wait for navigation
|
||||
await page.wait_for_load_state("networkidle", timeout=self.request_timeout)
|
||||
logger.info("Search navigation completed successfully")
|
||||
except Exception as e:
|
||||
logger.warning(f"Human click failed, falling back to JavaScript: {e}")
|
||||
# Fallback to original method
|
||||
async with page.expect_navigation(wait_until="networkidle", timeout=self.request_timeout):
|
||||
await page.evaluate(f"javascript:__doPostBack('{event_target_for_submit}','')")
|
||||
logger.info("Search navigation completed via fallback")
|
||||
else:
|
||||
# Pagination - use original method for consistency
|
||||
page_link_ctl_number = search_params.page + 2
|
||||
event_target_for_submit = f"ctl00$ContentPlaceHolder1$grdKurulKararSorguSonuc$ctl14$ctl{page_link_ctl_number:02d}"
|
||||
logger.info(f"Executing pagination with event target: {event_target_for_submit}")
|
||||
|
||||
async with page.expect_navigation(wait_until="networkidle", timeout=self.request_timeout):
|
||||
await page.evaluate(f"javascript:__doPostBack('{event_target_for_submit}','')")
|
||||
logger.info("Pagination navigation completed successfully")
|
||||
except PlaywrightTimeoutError:
|
||||
await page.wait_for_timeout(2000)
|
||||
logger.warning("Search navigation timed out, but continuing...")
|
||||
await page.wait_for_timeout(5000) # Longer wait if navigation fails
|
||||
|
||||
# Ultra-fast wait time for results to load
|
||||
logger.info("Waiting for search results to load...")
|
||||
await page.wait_for_timeout(500) # Reduced from 1000ms
|
||||
|
||||
results_table_dom_selector = f"table#{self.RESULTS_TABLE_ID}"
|
||||
try:
|
||||
await page.wait_for_selector(results_table_dom_selector, timeout=30000, state="attached")
|
||||
await page.wait_for_timeout(2000)
|
||||
# First wait for any tables to appear (more general check)
|
||||
logger.info("Waiting for any tables to appear...")
|
||||
await page.wait_for_function("""
|
||||
() => document.querySelectorAll('table').length > 0
|
||||
""", timeout=4000) # Reduced from 8000ms
|
||||
logger.info("At least one table appeared")
|
||||
|
||||
# Then wait for our specific table
|
||||
await page.wait_for_selector(results_table_dom_selector, timeout=4000, state="attached") # Reduced from 8000ms
|
||||
logger.debug("Results table attached to DOM")
|
||||
|
||||
# Wait for table to have some content (more than just headers)
|
||||
await page.wait_for_function(f"""
|
||||
() => {{
|
||||
const table = document.querySelector('{results_table_dom_selector}');
|
||||
return table && table.querySelectorAll('tr').length > 2;
|
||||
}}
|
||||
""", timeout=4000) # Reduced from 20000ms
|
||||
logger.debug("Results table populated with data")
|
||||
|
||||
# Ultra-fast additional wait for any remaining JavaScript
|
||||
await page.wait_for_timeout(500) # Reduced from 3000ms
|
||||
|
||||
except PlaywrightTimeoutError:
|
||||
logger.warning(f"Timeout waiting for results table '{results_table_dom_selector}'.")
|
||||
# Try one more wait for content placeholder
|
||||
try:
|
||||
await page.wait_for_selector("#ctl00_ContentPlaceHolder1", timeout=10000)
|
||||
logger.info("ContentPlaceHolder1 found, checking for tables...")
|
||||
await page.wait_for_timeout(5000)
|
||||
except PlaywrightTimeoutError:
|
||||
logger.warning("ContentPlaceHolder1 also not found - content may not have loaded")
|
||||
|
||||
html_content = await page.content()
|
||||
soup = BeautifulSoup(html_content, "html.parser")
|
||||
@@ -251,16 +995,17 @@ class KikApiClient:
|
||||
# ... (öncekiyle aynı) ...
|
||||
if not html_fragment: return None
|
||||
cleaned_html = self._clean_html_for_markdown(html_fragment)
|
||||
markdown_output = None; temp_file_path = None
|
||||
markdown_output = None
|
||||
try:
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = cleaned_html.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown(enable_plugins=True, remove_alt_whitespace=True, keep_underline=True)
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_html_file:
|
||||
tmp_html_file.write(cleaned_html); temp_file_path = tmp_html_file.name
|
||||
markdown_output = md_converter.convert(temp_file_path).text_content
|
||||
markdown_output = md_converter.convert(html_stream).text_content
|
||||
if markdown_output: markdown_output = re.sub(r'\n{3,}', '\n\n', markdown_output).strip()
|
||||
except Exception as e: logger.error(f"MarkItDown conversion error: {e}", exc_info=True)
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path): os.remove(temp_file_path)
|
||||
return markdown_output
|
||||
|
||||
|
||||
|
||||
+30
-31
@@ -12,29 +12,29 @@ class KikKararTipi(str, Enum):
|
||||
|
||||
class KikSearchRequest(BaseModel):
|
||||
"""Model for KIK Decision search criteria."""
|
||||
karar_tipi: KikKararTipi = Field(KikKararTipi.UYUSMAZLIK, description="Type of KIK Decision.")
|
||||
karar_no: Optional[str] = Field(None, description="Decision Number (e.g., '2024/UH.II-1766').")
|
||||
karar_tarihi_baslangic: Optional[str] = Field(None, description="Decision Date Start (DD.MM.YYYY).", pattern=r"^\d{2}\.\d{2}\.\d{4}$")
|
||||
karar_tarihi_bitis: Optional[str] = Field(None, description="Decision Date End (DD.MM.YYYY).", pattern=r"^\d{2}\.\d{2}\.\d{4}$")
|
||||
resmi_gazete_sayisi: Optional[str] = Field(None, description="Official Gazette Number.")
|
||||
resmi_gazete_tarihi: Optional[str] = Field(None, description="Official Gazette Date (DD.MM.YYYY).", pattern=r"^\d{2}\.\d{2}\.\d{4}$")
|
||||
basvuru_konusu_ihale: Optional[str] = Field(None, description="Tender subject of the application.")
|
||||
basvuru_sahibi: Optional[str] = Field(None, description="Applicant.")
|
||||
ihaleyi_yapan_idare: Optional[str] = Field(None, description="Procuring Entity.")
|
||||
yil: Optional[str] = Field(None, description="Year of the decision.")
|
||||
karar_metni: Optional[str] = Field(None, description="Keyword/phrase in decision text.")
|
||||
page: int = Field(1, ge=1, description="Results page number.")
|
||||
karar_tipi: KikKararTipi = Field(KikKararTipi.UYUSMAZLIK, description="Type")
|
||||
karar_no: Optional[str] = Field(None, description="No")
|
||||
karar_tarihi_baslangic: Optional[str] = Field(None, description="Start", pattern=r"^\d{2}\.\d{2}\.\d{4}$")
|
||||
karar_tarihi_bitis: Optional[str] = Field(None, description="End", pattern=r"^\d{2}\.\d{2}\.\d{4}$")
|
||||
resmi_gazete_sayisi: Optional[str] = Field(None, description="Gazette")
|
||||
resmi_gazete_tarihi: Optional[str] = Field(None, description="Date", pattern=r"^\d{2}\.\d{2}\.\d{4}$")
|
||||
basvuru_konusu_ihale: Optional[str] = Field(None, description="Subject")
|
||||
basvuru_sahibi: Optional[str] = Field(None, description="Applicant")
|
||||
ihaleyi_yapan_idare: Optional[str] = Field(None, description="Entity")
|
||||
yil: Optional[str] = Field(None, description="Year")
|
||||
karar_metni: Optional[str] = Field(None, description="Text")
|
||||
page: int = Field(1, ge=1, description="Page")
|
||||
|
||||
class KikDecisionEntry(BaseModel):
|
||||
"""Represents a single decision entry from KIK search results."""
|
||||
preview_event_target: str = Field(..., description="Internal event target for fetching details.")
|
||||
karar_no_str: str = Field(..., alias="kararNo", description="Raw decision number as extracted from KIK (e.g., '2024/UH.II-1766').")
|
||||
karar_tipi: KikKararTipi = Field(..., description="The type of decision this entry belongs to.")
|
||||
preview_event_target: str = Field(..., description="Event target")
|
||||
karar_no_str: str = Field(..., alias="kararNo", description="Decision number")
|
||||
karar_tipi: KikKararTipi = Field(..., description="Decision type")
|
||||
|
||||
karar_tarihi_str: str = Field(..., alias="kararTarihi", description="Decision date.")
|
||||
idare_str: Optional[str] = Field(None, alias="idare", description="Procuring entity.")
|
||||
basvuru_sahibi_str: Optional[str] = Field(None, alias="basvuruSahibi", description="Applicant.")
|
||||
ihale_konusu_str: Optional[str] = Field(None, alias="ihaleKonusu", description="Tender subject.")
|
||||
karar_tarihi_str: str = Field(..., alias="kararTarihi", description="Date")
|
||||
idare_str: Optional[str] = Field(None, alias="idare", description="Entity")
|
||||
basvuru_sahibi_str: Optional[str] = Field(None, alias="basvuruSahibi", description="Applicant")
|
||||
ihale_konusu_str: Optional[str] = Field(None, alias="ihaleKonusu", description="Subject")
|
||||
|
||||
@computed_field
|
||||
@property
|
||||
@@ -58,18 +58,17 @@ class KikDocumentMarkdown(BaseModel):
|
||||
"""
|
||||
KIK decision document, with Markdown content potentially paginated.
|
||||
"""
|
||||
retrieved_with_karar_id: Optional[str] = Field(None, description="The Base64 encoded karar_id that was used to request this document.")
|
||||
# Decode edilmiş karar no ve tipini de yanıt olarak ekleyelim, Claude için faydalı olabilir.
|
||||
retrieved_karar_no: Optional[str] = Field(None, description="The raw KIK Decision Number (e.g., '2024/UH.II-1766') this document pertains to.")
|
||||
retrieved_karar_tipi: Optional[KikKararTipi] = Field(None, description="The KIK Decision Type this document pertains to.")
|
||||
retrieved_with_karar_id: Optional[str] = Field(None, description="Request ID")
|
||||
retrieved_karar_no: Optional[str] = Field(None, description="Decision number")
|
||||
retrieved_karar_tipi: Optional[KikKararTipi] = Field(None, description="Decision type")
|
||||
|
||||
karar_id_param_from_url: Optional[str] = Field(None, alias="kararIdParam", description="The KIK system's internal KararId parameter from the document's display URL (KurulKararGoster.aspx).")
|
||||
markdown_chunk: Optional[str] = Field(None, description="The requested chunk of the decision content converted to Markdown.")
|
||||
source_url: Optional[str] = Field(None, description="The source URL of the original document (KurulKararGoster.aspx).")
|
||||
error_message: Optional[str] = Field(None, description="Error message if document retrieval or processing failed.")
|
||||
current_page: int = Field(1, description="The current page number of the markdown chunk being returned.")
|
||||
total_pages: int = Field(1, description="The total number of pages the full markdown content is divided into.")
|
||||
is_paginated: bool = Field(False, description="True if the full markdown content is split into multiple pages.")
|
||||
full_content_char_count: Optional[int] = Field(None, description="Total character count of the full markdown content before chunking.")
|
||||
karar_id_param_from_url: Optional[str] = Field(None, alias="kararIdParam", description="Internal ID")
|
||||
markdown_chunk: Optional[str] = Field(None, description="Content")
|
||||
source_url: Optional[str] = Field(None, description="Source URL")
|
||||
error_message: Optional[str] = Field(None, description="Error")
|
||||
current_page: int = Field(1, description="Page")
|
||||
total_pages: int = Field(1, description="Total pages")
|
||||
is_paginated: bool = Field(False, description="Paginated")
|
||||
full_content_char_count: Optional[int] = Field(None, description="Char count")
|
||||
|
||||
model_config = ConfigDict(populate_by_name=True)
|
||||
@@ -0,0 +1 @@
|
||||
# kvkk_mcp_module/__init__.py
|
||||
@@ -0,0 +1,372 @@
|
||||
# kvkk_mcp_module/client.py
|
||||
|
||||
import httpx
|
||||
from bs4 import BeautifulSoup
|
||||
from typing import List, Optional, Dict, Any
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import io
|
||||
import math
|
||||
from urllib.parse import urljoin, urlparse, parse_qs
|
||||
from markitdown import MarkItDown
|
||||
from pydantic import HttpUrl
|
||||
|
||||
from .models import (
|
||||
KvkkSearchRequest,
|
||||
KvkkDecisionSummary,
|
||||
KvkkSearchResult,
|
||||
KvkkDocumentMarkdown
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
if not logger.hasHandlers():
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
|
||||
)
|
||||
|
||||
class KvkkApiClient:
|
||||
"""
|
||||
API client for searching and retrieving KVKK (Personal Data Protection Authority) decisions
|
||||
using Brave Search API for discovery and direct HTTP requests for content retrieval.
|
||||
"""
|
||||
|
||||
BRAVE_API_URL = "https://api.search.brave.com/res/v1/web/search"
|
||||
KVKK_BASE_URL = "https://www.kvkk.gov.tr"
|
||||
DOCUMENT_MARKDOWN_CHUNK_SIZE = 5000 # Character limit per page
|
||||
|
||||
def __init__(self, request_timeout: float = 60.0):
|
||||
"""Initialize the KVKK API client."""
|
||||
self.brave_api_token = os.getenv("BRAVE_API_TOKEN")
|
||||
if not self.brave_api_token:
|
||||
# Fallback to provided free token
|
||||
self.brave_api_token = "BSAuaRKB-dvSDSQxIN0ft1p2k6N82Kq"
|
||||
logger.info("Using fallback Brave API token (limited free token)")
|
||||
else:
|
||||
logger.info("Using Brave API token from environment variable")
|
||||
|
||||
self.http_client = httpx.AsyncClient(
|
||||
headers={
|
||||
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
|
||||
"Accept-Language": "tr-TR,tr;q=0.9,en-US;q=0.8,en;q=0.7",
|
||||
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
|
||||
},
|
||||
timeout=request_timeout,
|
||||
verify=True,
|
||||
follow_redirects=True
|
||||
)
|
||||
|
||||
def _construct_search_query(self, keywords: str) -> str:
|
||||
"""Construct the search query for Brave API."""
|
||||
base_query = 'site:kvkk.gov.tr "karar özeti"'
|
||||
if keywords.strip():
|
||||
return f"{base_query} {keywords.strip()}"
|
||||
return base_query
|
||||
|
||||
def _extract_decision_id_from_url(self, url: str) -> Optional[str]:
|
||||
"""Extract decision ID from KVKK decision URL."""
|
||||
try:
|
||||
# Example URL: https://www.kvkk.gov.tr/Icerik/7288/2021-1303
|
||||
parsed_url = urlparse(url)
|
||||
path_parts = parsed_url.path.strip('/').split('/')
|
||||
|
||||
if len(path_parts) >= 3 and path_parts[0] == 'Icerik':
|
||||
# Extract the decision ID from the path
|
||||
decision_id = '/'.join(path_parts[1:]) # e.g., "7288/2021-1303"
|
||||
return decision_id
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not extract decision ID from URL {url}: {e}")
|
||||
|
||||
return None
|
||||
|
||||
def _extract_decision_metadata_from_title(self, title: str) -> Dict[str, Optional[str]]:
|
||||
"""Extract decision metadata from title string."""
|
||||
metadata = {
|
||||
"decision_date": None,
|
||||
"decision_number": None
|
||||
}
|
||||
|
||||
if not title:
|
||||
return metadata
|
||||
|
||||
# Extract decision date (DD/MM/YYYY format)
|
||||
date_match = re.search(r'(\d{1,2}/\d{1,2}/\d{4})', title)
|
||||
if date_match:
|
||||
metadata["decision_date"] = date_match.group(1)
|
||||
|
||||
# Extract decision number (YYYY/XXXX format)
|
||||
number_match = re.search(r'(\d{4}/\d+)', title)
|
||||
if number_match:
|
||||
metadata["decision_number"] = number_match.group(1)
|
||||
|
||||
return metadata
|
||||
|
||||
async def search_decisions(self, params: KvkkSearchRequest) -> KvkkSearchResult:
|
||||
"""Search for KVKK decisions using Brave API."""
|
||||
|
||||
search_query = self._construct_search_query(params.keywords)
|
||||
logger.info(f"KvkkApiClient: Searching with query: {search_query}")
|
||||
|
||||
try:
|
||||
# Calculate offset for pagination
|
||||
offset = (params.page - 1) * params.pageSize
|
||||
|
||||
response = await self.http_client.get(
|
||||
self.BRAVE_API_URL,
|
||||
headers={
|
||||
"Accept": "application/json",
|
||||
"Accept-Encoding": "gzip",
|
||||
"x-subscription-token": self.brave_api_token
|
||||
},
|
||||
params={
|
||||
"q": search_query,
|
||||
"country": "TR",
|
||||
"search_lang": "tr",
|
||||
"ui_lang": "tr-TR",
|
||||
"offset": offset,
|
||||
"count": params.pageSize
|
||||
}
|
||||
)
|
||||
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
# Extract search results
|
||||
decisions = []
|
||||
web_results = data.get("web", {}).get("results", [])
|
||||
|
||||
for result in web_results:
|
||||
title = result.get("title", "")
|
||||
url = result.get("url", "")
|
||||
description = result.get("description", "")
|
||||
|
||||
# Extract metadata from title
|
||||
metadata = self._extract_decision_metadata_from_title(title)
|
||||
|
||||
# Extract decision ID from URL
|
||||
decision_id = self._extract_decision_id_from_url(url)
|
||||
|
||||
decision = KvkkDecisionSummary(
|
||||
title=title,
|
||||
url=HttpUrl(url) if url else None,
|
||||
description=description,
|
||||
decision_id=decision_id,
|
||||
publication_date=metadata.get("decision_date"),
|
||||
decision_number=metadata.get("decision_number")
|
||||
)
|
||||
decisions.append(decision)
|
||||
|
||||
# Get total results if available
|
||||
total_results = None
|
||||
query_info = data.get("query", {})
|
||||
if "total_results" in query_info:
|
||||
total_results = query_info["total_results"]
|
||||
|
||||
return KvkkSearchResult(
|
||||
decisions=decisions,
|
||||
total_results=total_results,
|
||||
page=params.page,
|
||||
pageSize=params.pageSize,
|
||||
query=search_query
|
||||
)
|
||||
|
||||
except httpx.RequestError as e:
|
||||
logger.error(f"KvkkApiClient: HTTP request error during search: {e}")
|
||||
return KvkkSearchResult(
|
||||
decisions=[],
|
||||
total_results=0,
|
||||
page=params.page,
|
||||
pageSize=params.pageSize,
|
||||
query=search_query
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"KvkkApiClient: Unexpected error during search: {e}")
|
||||
return KvkkSearchResult(
|
||||
decisions=[],
|
||||
total_results=0,
|
||||
page=params.page,
|
||||
pageSize=params.pageSize,
|
||||
query=search_query
|
||||
)
|
||||
|
||||
def _extract_decision_content_from_html(self, html: str, url: str) -> Dict[str, Any]:
|
||||
"""Extract decision content from KVKK decision page HTML."""
|
||||
try:
|
||||
soup = BeautifulSoup(html, 'html.parser')
|
||||
|
||||
# Extract title
|
||||
title = None
|
||||
title_element = soup.find('h3', class_='blog-post-title')
|
||||
if title_element:
|
||||
title = title_element.get_text(strip=True)
|
||||
elif soup.title:
|
||||
title = soup.title.get_text(strip=True)
|
||||
|
||||
# Extract decision content from the main content div
|
||||
content_div = soup.find('div', class_='blog-post-inner')
|
||||
if not content_div:
|
||||
# Fallback to other possible content containers
|
||||
content_div = soup.find('div', style='text-align:justify;')
|
||||
if not content_div:
|
||||
logger.warning(f"Could not find decision content div in {url}")
|
||||
return {
|
||||
"title": title,
|
||||
"decision_date": None,
|
||||
"decision_number": None,
|
||||
"subject_summary": None,
|
||||
"html_content": None
|
||||
}
|
||||
|
||||
# Extract decision metadata from table
|
||||
decision_date = None
|
||||
decision_number = None
|
||||
subject_summary = None
|
||||
|
||||
table = content_div.find('table')
|
||||
if table:
|
||||
rows = table.find_all('tr')
|
||||
for row in rows:
|
||||
cells = row.find_all('td')
|
||||
if len(cells) >= 3:
|
||||
field_name = cells[0].get_text(strip=True)
|
||||
field_value = cells[2].get_text(strip=True)
|
||||
|
||||
if 'Karar Tarihi' in field_name:
|
||||
decision_date = field_value
|
||||
elif 'Karar No' in field_name:
|
||||
decision_number = field_value
|
||||
elif 'Konu Özeti' in field_name:
|
||||
subject_summary = field_value
|
||||
|
||||
return {
|
||||
"title": title,
|
||||
"decision_date": decision_date,
|
||||
"decision_number": decision_number,
|
||||
"subject_summary": subject_summary,
|
||||
"html_content": str(content_div)
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error extracting content from HTML for {url}: {e}")
|
||||
return {
|
||||
"title": None,
|
||||
"decision_date": None,
|
||||
"decision_number": None,
|
||||
"subject_summary": None,
|
||||
"html_content": None
|
||||
}
|
||||
|
||||
def _convert_html_to_markdown(self, html_content: str) -> Optional[str]:
|
||||
"""Convert HTML content to Markdown using MarkItDown with BytesIO to avoid filename length issues."""
|
||||
if not html_content:
|
||||
return None
|
||||
|
||||
try:
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = html_content.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown(enable_plugins=False)
|
||||
result = md_converter.convert(html_stream)
|
||||
return result.text_content
|
||||
except Exception as e:
|
||||
logger.error(f"Error converting HTML to Markdown: {e}")
|
||||
return None
|
||||
|
||||
async def get_decision_document(self, decision_url: str, page_number: int = 1) -> KvkkDocumentMarkdown:
|
||||
"""Retrieve and convert a KVKK decision document to paginated Markdown."""
|
||||
logger.info(f"KvkkApiClient: Getting decision document from: {decision_url}, page: {page_number}")
|
||||
|
||||
try:
|
||||
# Fetch the decision page
|
||||
response = await self.http_client.get(decision_url)
|
||||
response.raise_for_status()
|
||||
|
||||
# Extract content from HTML
|
||||
extracted_data = self._extract_decision_content_from_html(response.text, decision_url)
|
||||
|
||||
# Convert HTML content to Markdown
|
||||
full_markdown_content = None
|
||||
if extracted_data["html_content"]:
|
||||
full_markdown_content = self._convert_html_to_markdown(extracted_data["html_content"])
|
||||
|
||||
if not full_markdown_content:
|
||||
return KvkkDocumentMarkdown(
|
||||
source_url=HttpUrl(decision_url),
|
||||
title=extracted_data["title"],
|
||||
decision_date=extracted_data["decision_date"],
|
||||
decision_number=extracted_data["decision_number"],
|
||||
subject_summary=extracted_data["subject_summary"],
|
||||
markdown_chunk=None,
|
||||
current_page=page_number,
|
||||
total_pages=0,
|
||||
is_paginated=False,
|
||||
error_message="Could not convert document content to Markdown"
|
||||
)
|
||||
|
||||
# Calculate pagination
|
||||
content_length = len(full_markdown_content)
|
||||
total_pages = math.ceil(content_length / self.DOCUMENT_MARKDOWN_CHUNK_SIZE)
|
||||
if total_pages == 0:
|
||||
total_pages = 1
|
||||
|
||||
# Clamp page number to valid range
|
||||
current_page_clamped = max(1, min(page_number, total_pages))
|
||||
|
||||
# Extract the requested chunk
|
||||
start_index = (current_page_clamped - 1) * self.DOCUMENT_MARKDOWN_CHUNK_SIZE
|
||||
end_index = start_index + self.DOCUMENT_MARKDOWN_CHUNK_SIZE
|
||||
markdown_chunk = full_markdown_content[start_index:end_index]
|
||||
|
||||
return KvkkDocumentMarkdown(
|
||||
source_url=HttpUrl(decision_url),
|
||||
title=extracted_data["title"],
|
||||
decision_date=extracted_data["decision_date"],
|
||||
decision_number=extracted_data["decision_number"],
|
||||
subject_summary=extracted_data["subject_summary"],
|
||||
markdown_chunk=markdown_chunk,
|
||||
current_page=current_page_clamped,
|
||||
total_pages=total_pages,
|
||||
is_paginated=(total_pages > 1),
|
||||
error_message=None
|
||||
)
|
||||
|
||||
except httpx.HTTPStatusError as e:
|
||||
error_msg = f"HTTP error {e.response.status_code} when fetching decision document"
|
||||
logger.error(f"KvkkApiClient: {error_msg}")
|
||||
return KvkkDocumentMarkdown(
|
||||
source_url=HttpUrl(decision_url),
|
||||
title=None,
|
||||
decision_date=None,
|
||||
decision_number=None,
|
||||
subject_summary=None,
|
||||
markdown_chunk=None,
|
||||
current_page=page_number,
|
||||
total_pages=0,
|
||||
is_paginated=False,
|
||||
error_message=error_msg
|
||||
)
|
||||
except Exception as e:
|
||||
error_msg = f"Unexpected error when fetching decision document: {str(e)}"
|
||||
logger.error(f"KvkkApiClient: {error_msg}")
|
||||
return KvkkDocumentMarkdown(
|
||||
source_url=HttpUrl(decision_url),
|
||||
title=None,
|
||||
decision_date=None,
|
||||
decision_number=None,
|
||||
subject_summary=None,
|
||||
markdown_chunk=None,
|
||||
current_page=page_number,
|
||||
total_pages=0,
|
||||
is_paginated=False,
|
||||
error_message=error_msg
|
||||
)
|
||||
|
||||
async def close_client_session(self):
|
||||
"""Close the HTTP client session."""
|
||||
if hasattr(self, 'http_client') and self.http_client and not self.http_client.is_closed:
|
||||
await self.http_client.aclose()
|
||||
logger.info("KvkkApiClient: HTTP client session closed.")
|
||||
@@ -0,0 +1,49 @@
|
||||
# kvkk_mcp_module/models.py
|
||||
|
||||
from pydantic import BaseModel, Field, HttpUrl
|
||||
from typing import List, Optional, Any
|
||||
|
||||
class KvkkSearchRequest(BaseModel):
|
||||
"""Model for KVKK (Personal Data Protection Authority) search request via Brave API."""
|
||||
keywords: str = Field(..., description="""
|
||||
Keywords to search for in KVKK decisions.
|
||||
The search will automatically include 'site:kvkk.gov.tr "karar özeti"' to target KVKK decision summaries.
|
||||
Examples: "açık rıza", "veri güvenliği", "kişisel veri işleme"
|
||||
""")
|
||||
page: int = Field(1, ge=1, le=50, description="Page number for search results (1-50).")
|
||||
pageSize: int = Field(10, ge=1, le=10, description="Number of results per page (1-10).")
|
||||
|
||||
class KvkkDecisionSummary(BaseModel):
|
||||
"""Model for a single KVKK decision summary from Brave search results."""
|
||||
title: Optional[str] = Field(None, description="Decision title from search results.")
|
||||
url: Optional[HttpUrl] = Field(None, description="URL to the KVKK decision page.")
|
||||
description: Optional[str] = Field(None, description="Brief description or snippet from search results.")
|
||||
decision_id: Optional[str] = Field(None, description="Value")
|
||||
publication_date: Optional[str] = Field(None, description="Value")
|
||||
decision_number: Optional[str] = Field(None, description="Value")
|
||||
|
||||
class KvkkSearchResult(BaseModel):
|
||||
"""Model for the overall search result for KVKK decisions."""
|
||||
decisions: List[KvkkDecisionSummary] = Field(default_factory=list, description="List of KVKK decisions found.")
|
||||
total_results: Optional[int] = Field(None, description="Value")
|
||||
page: int = Field(1, description="Current page number of results.")
|
||||
pageSize: int = Field(10, description="Number of results per page.")
|
||||
query: Optional[str] = Field(None, description="The actual search query sent to Brave API.")
|
||||
|
||||
class KvkkDocumentMarkdown(BaseModel):
|
||||
"""Model for KVKK decision document content converted to paginated Markdown."""
|
||||
source_url: HttpUrl = Field(description="URL of the original KVKK decision page.")
|
||||
title: Optional[str] = Field(None, description="Title of the KVKK decision.")
|
||||
decision_date: Optional[str] = Field(None, description="Decision date (Karar Tarihi).")
|
||||
decision_number: Optional[str] = Field(None, description="Decision number (Karar No).")
|
||||
subject_summary: Optional[str] = Field(None, description="Subject summary (Konu Özeti).")
|
||||
markdown_chunk: Optional[str] = Field(None, description="A 5,000 character chunk of the Markdown content.")
|
||||
current_page: int = Field(description="The current page number of the markdown chunk (1-indexed).")
|
||||
total_pages: int = Field(description="Total number of pages for the full markdown content.")
|
||||
is_paginated: bool = Field(description="True if the full markdown content is split into multiple pages.")
|
||||
error_message: Optional[str] = Field(None, description="Value")
|
||||
|
||||
class Config:
|
||||
json_encoders = {
|
||||
HttpUrl: str
|
||||
}
|
||||
+107
-62
@@ -135,19 +135,17 @@ async def authorize_endpoint(
|
||||
@router.get("/auth/callback")
|
||||
async def oauth_callback(
|
||||
request: Request,
|
||||
state: Optional[str] = Query(None)
|
||||
state: Optional[str] = Query(None),
|
||||
clerk_token: Optional[str] = Query(None)
|
||||
):
|
||||
"""Handle OAuth callback from Clerk - simplified for custom domains"""
|
||||
"""Handle OAuth callback from Clerk - supports both JWT token and cookie auth"""
|
||||
|
||||
logger.info(f"OAuth callback received - state: {state}")
|
||||
logger.info(f"Query params: {dict(request.query_params)}")
|
||||
logger.info(f"Cookies: {dict(request.cookies)}")
|
||||
logger.info(f"Clerk JWT token provided: {bool(clerk_token)}")
|
||||
|
||||
# For Clerk custom domains, we'll assume authentication succeeded
|
||||
# if Clerk redirected the user to our callback URL
|
||||
|
||||
# For custom domains, we'll skip complex session verification
|
||||
# and rely on the fact that Clerk only redirects here after successful auth
|
||||
# Support both JWT token (for cross-domain) and cookie auth (for subdomain)
|
||||
|
||||
try:
|
||||
if not state:
|
||||
@@ -189,17 +187,79 @@ async def oauth_callback(
|
||||
content={"error": "invalid_request", "error_description": "OAuth session expired or not found"}
|
||||
)
|
||||
|
||||
# Check if we have a JWT token (for cross-domain auth)
|
||||
user_authenticated = False
|
||||
auth_method = "none"
|
||||
|
||||
if clerk_token:
|
||||
logger.info("Attempting JWT token validation")
|
||||
try:
|
||||
# Validate JWT token with Clerk
|
||||
from clerk_backend_api import Clerk
|
||||
clerk = Clerk(bearer_auth=os.getenv("CLERK_SECRET_KEY"))
|
||||
|
||||
# Extract session_id from JWT token and verify with Clerk
|
||||
import jwt
|
||||
decoded_token = jwt.decode(clerk_token, options={"verify_signature": False})
|
||||
session_id = decoded_token.get("sid") or decoded_token.get("session_id")
|
||||
|
||||
if session_id:
|
||||
# Verify with Clerk using session_id
|
||||
session = clerk.sessions.verify(session_id=session_id, token=clerk_token)
|
||||
user_id = session.user_id if session else None
|
||||
else:
|
||||
user_id = None
|
||||
|
||||
if user_id:
|
||||
logger.info(f"JWT token validation successful - user_id: {user_id}")
|
||||
user_authenticated = True
|
||||
auth_method = "jwt_token"
|
||||
# Store user info in session for token exchange
|
||||
oauth_session["user_id"] = user_id
|
||||
oauth_session["auth_method"] = "jwt_token"
|
||||
else:
|
||||
logger.error("JWT token validation failed - no user_id in claims")
|
||||
except Exception as e:
|
||||
logger.error(f"JWT token validation failed: {str(e)}")
|
||||
# Fall through to cookie validation
|
||||
|
||||
# If no JWT token or validation failed, check cookies
|
||||
if not user_authenticated:
|
||||
logger.info("Checking for Clerk session cookies")
|
||||
# Check for Clerk session cookies (for subdomain auth)
|
||||
clerk_session_cookie = request.cookies.get("__session")
|
||||
if clerk_session_cookie:
|
||||
logger.info("Found Clerk session cookie, assuming authenticated")
|
||||
user_authenticated = True
|
||||
auth_method = "cookie"
|
||||
oauth_session["auth_method"] = "cookie"
|
||||
else:
|
||||
logger.info("No Clerk session cookie found")
|
||||
|
||||
# For custom domains, we'll also trust that Clerk redirected here
|
||||
if not user_authenticated:
|
||||
logger.info("Trusting Clerk redirect for custom domain flow")
|
||||
user_authenticated = True
|
||||
auth_method = "trusted_redirect"
|
||||
oauth_session["auth_method"] = "trusted_redirect"
|
||||
|
||||
logger.info(f"User authenticated: {user_authenticated}, method: {auth_method}")
|
||||
|
||||
# Generate simple authorization code for custom domain flow
|
||||
auth_code = f"clerk_custom_{session_id}_{int(time.time())}"
|
||||
|
||||
# Store the code mapping for token exchange
|
||||
code_data = {
|
||||
"session_id": session_id,
|
||||
"clerk_authenticated": True,
|
||||
"clerk_authenticated": user_authenticated,
|
||||
"auth_method": auth_method,
|
||||
"custom_domain_flow": True,
|
||||
"created_at": time.time(),
|
||||
"expires_at": (datetime.utcnow() + timedelta(minutes=5)).timestamp(),
|
||||
}
|
||||
if "user_id" in oauth_session:
|
||||
code_data["user_id"] = oauth_session["user_id"]
|
||||
|
||||
oauth_provider.storage.set_session(f"code_{auth_code}", code_data)
|
||||
|
||||
# Build redirect URL back to Claude
|
||||
@@ -264,70 +324,55 @@ async def token_endpoint(request: Request):
|
||||
)
|
||||
|
||||
try:
|
||||
# Import here to avoid circular imports
|
||||
from mcp_server_main import app as mcp_app
|
||||
from mcp_auth_factory import get_oauth_provider
|
||||
# OAuth token exchange - validate code and return Clerk JWT
|
||||
# This supports proper OAuth flow while using Clerk JWT tokens
|
||||
|
||||
# Get OAuth provider
|
||||
oauth_provider = get_oauth_provider(mcp_app)
|
||||
if not oauth_provider:
|
||||
raise HTTPException(status_code=500, detail="OAuth provider not configured")
|
||||
|
||||
# Extract session info from code
|
||||
code_session = None
|
||||
if code.startswith("clerk_"):
|
||||
# Get the code mapping
|
||||
code_session = oauth_provider.storage.get_session(f"code_{code}")
|
||||
if code_session:
|
||||
session_id = code_session.get("session_id")
|
||||
else:
|
||||
logger.error(f"Code mapping not found for: {code}")
|
||||
if not code or not redirect_uri:
|
||||
logger.error("Missing required parameters: code or redirect_uri")
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Invalid authorization code"}
|
||||
)
|
||||
else:
|
||||
session_id = code
|
||||
|
||||
session = oauth_provider.storage.get_session(session_id)
|
||||
|
||||
if not session:
|
||||
logger.error(f"Session {session_id} not found for token exchange")
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Invalid authorization code"}
|
||||
content={"error": "invalid_request", "error_description": "Missing code or redirect_uri"}
|
||||
)
|
||||
|
||||
# Validate PKCE if present
|
||||
if "pkce_challenge" in session and code_verifier:
|
||||
# Validate PKCE challenge
|
||||
if not oauth_provider.validate_pkce(code_verifier, session["pkce_challenge"]):
|
||||
logger.error("PKCE challenge validation failed")
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Invalid code verifier"}
|
||||
)
|
||||
logger.info("PKCE validation successful")
|
||||
else:
|
||||
logger.info("No PKCE validation required")
|
||||
# Validate OAuth code with Clerk
|
||||
if CLERK_AVAILABLE:
|
||||
try:
|
||||
clerk = Clerk(bearer_auth=os.getenv("CLERK_SECRET_KEY"))
|
||||
|
||||
# Create JWT token
|
||||
access_token = oauth_provider._create_mcp_token(
|
||||
session["scopes"],
|
||||
session.get("clerk_token", ""),
|
||||
session_id
|
||||
)
|
||||
|
||||
# Clean up sessions
|
||||
oauth_provider.storage.delete_session(session_id)
|
||||
if code_session:
|
||||
oauth_provider.storage.delete_session(f"code_{code}")
|
||||
# In a real implementation, you'd validate the code with Clerk
|
||||
# For now, we'll assume the code is valid if it looks like a Clerk code
|
||||
if len(code) > 10: # Basic validation
|
||||
# Create a mock session with the code
|
||||
# In practice, this would be validated with Clerk's OAuth flow
|
||||
|
||||
# Return Clerk JWT token format
|
||||
# This should be the actual Clerk JWT token from the OAuth flow
|
||||
return JSONResponse({
|
||||
"access_token": access_token,
|
||||
"access_token": f"mock_clerk_jwt_{code}",
|
||||
"token_type": "Bearer",
|
||||
"expires_in": 3600,
|
||||
"scope": " ".join(session["scopes"])
|
||||
"scope": "yargi.read yargi.search"
|
||||
})
|
||||
else:
|
||||
logger.error(f"Invalid code format: {code}")
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Invalid authorization code"}
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Clerk validation failed: {e}")
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Authorization code validation failed"}
|
||||
)
|
||||
else:
|
||||
logger.warning("Clerk SDK not available, using mock response")
|
||||
return JSONResponse({
|
||||
"access_token": "mock_jwt_token_for_development",
|
||||
"token_type": "Bearer",
|
||||
"expires_in": 3600,
|
||||
"scope": "yargi.read yargi.search"
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
|
||||
@@ -0,0 +1,522 @@
|
||||
"""
|
||||
Simplified MCP OAuth HTTP adapter - only Clerk JWT based authentication
|
||||
Uses Redis for authorization code storage to support multi-machine deployment
|
||||
"""
|
||||
|
||||
import os
|
||||
import logging
|
||||
from typing import Optional
|
||||
from urllib.parse import urlencode, quote
|
||||
|
||||
from fastapi import APIRouter, Request, Query, HTTPException
|
||||
from fastapi.responses import RedirectResponse, JSONResponse
|
||||
|
||||
# Import Redis session store
|
||||
from redis_session_store import get_redis_store
|
||||
|
||||
# Try to import Clerk SDK
|
||||
try:
|
||||
from clerk_backend_api import Clerk
|
||||
CLERK_AVAILABLE = True
|
||||
except ImportError:
|
||||
CLERK_AVAILABLE = False
|
||||
Clerk = None
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# OAuth configuration
|
||||
BASE_URL = os.getenv("BASE_URL", "https://api.yargimcp.com")
|
||||
CLERK_DOMAIN = os.getenv("CLERK_DOMAIN", "accounts.yargimcp.com")
|
||||
|
||||
# Initialize Redis store
|
||||
redis_store = None
|
||||
|
||||
def get_redis_session_store():
|
||||
"""Get Redis store instance with lazy initialization."""
|
||||
global redis_store
|
||||
if redis_store is None:
|
||||
try:
|
||||
import concurrent.futures
|
||||
import functools
|
||||
|
||||
# Use thread pool with timeout to prevent hanging
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
|
||||
future = executor.submit(get_redis_store)
|
||||
try:
|
||||
# 5 second timeout for Redis initialization
|
||||
redis_store = future.result(timeout=5.0)
|
||||
if redis_store:
|
||||
logger.info("Redis session store initialized for OAuth handler")
|
||||
else:
|
||||
logger.warning("Redis store initialization returned None")
|
||||
except concurrent.futures.TimeoutError:
|
||||
logger.error("Redis initialization timed out after 5 seconds")
|
||||
redis_store = None
|
||||
future.cancel() # Try to cancel the hanging operation
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to initialize Redis store: {e}")
|
||||
redis_store = None
|
||||
|
||||
if redis_store is None:
|
||||
# Fall back to in-memory storage with warning
|
||||
logger.warning("Falling back to in-memory storage - multi-machine deployment will not work")
|
||||
|
||||
return redis_store
|
||||
|
||||
@router.get("/.well-known/oauth-authorization-server")
|
||||
async def get_oauth_metadata():
|
||||
"""OAuth 2.0 Authorization Server Metadata (RFC 8414)"""
|
||||
return JSONResponse({
|
||||
"issuer": BASE_URL,
|
||||
"authorization_endpoint": "https://yargimcp.com/mcp-callback",
|
||||
"token_endpoint": f"{BASE_URL}/token",
|
||||
"registration_endpoint": f"{BASE_URL}/register",
|
||||
"response_types_supported": ["code"],
|
||||
"grant_types_supported": ["authorization_code"],
|
||||
"code_challenge_methods_supported": ["S256"],
|
||||
"token_endpoint_auth_methods_supported": ["none"],
|
||||
"scopes_supported": ["read", "search", "openid", "profile", "email"],
|
||||
"service_documentation": f"{BASE_URL}/mcp/"
|
||||
})
|
||||
|
||||
@router.get("/auth/login")
|
||||
async def oauth_authorize(
|
||||
request: Request,
|
||||
client_id: str = Query(...),
|
||||
redirect_uri: str = Query(...),
|
||||
response_type: str = Query("code"),
|
||||
scope: Optional[str] = Query("read search"),
|
||||
state: Optional[str] = Query(None),
|
||||
code_challenge: Optional[str] = Query(None),
|
||||
code_challenge_method: Optional[str] = Query(None)
|
||||
):
|
||||
"""OAuth 2.1 Authorization Endpoint - redirects to Clerk"""
|
||||
|
||||
logger.info(f"OAuth authorize request - client_id: {client_id}")
|
||||
logger.info(f"Redirect URI: {redirect_uri}")
|
||||
logger.info(f"State: {state}")
|
||||
logger.info(f"PKCE Challenge: {bool(code_challenge)}")
|
||||
|
||||
try:
|
||||
# Build callback URL with all necessary parameters
|
||||
callback_url = f"{BASE_URL}/auth/callback"
|
||||
callback_params = {
|
||||
"client_id": client_id,
|
||||
"redirect_uri": redirect_uri,
|
||||
"state": state or "",
|
||||
"scope": scope or "read search"
|
||||
}
|
||||
|
||||
# Add PKCE parameters if present
|
||||
if code_challenge:
|
||||
callback_params["code_challenge"] = code_challenge
|
||||
callback_params["code_challenge_method"] = code_challenge_method or "S256"
|
||||
|
||||
# Encode callback URL as redirect_url for Clerk
|
||||
callback_with_params = f"{callback_url}?{urlencode(callback_params)}"
|
||||
|
||||
# Build Clerk sign-in URL - use yargimcp.com frontend for JWT token generation
|
||||
clerk_params = {
|
||||
"redirect_url": callback_with_params
|
||||
}
|
||||
|
||||
# Use frontend sign-in page that handles JWT token generation
|
||||
clerk_signin_url = f"https://yargimcp.com/sign-in?{urlencode(clerk_params)}"
|
||||
|
||||
logger.info(f"Redirecting to Clerk: {clerk_signin_url}")
|
||||
|
||||
return RedirectResponse(url=clerk_signin_url)
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Authorization failed: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
@router.get("/auth/callback")
|
||||
async def oauth_callback(
|
||||
request: Request,
|
||||
client_id: str = Query(...),
|
||||
redirect_uri: str = Query(...),
|
||||
state: Optional[str] = Query(None),
|
||||
scope: Optional[str] = Query("read search"),
|
||||
code_challenge: Optional[str] = Query(None),
|
||||
code_challenge_method: Optional[str] = Query(None),
|
||||
clerk_token: Optional[str] = Query(None)
|
||||
):
|
||||
"""OAuth callback from Clerk - generates authorization code"""
|
||||
|
||||
logger.info(f"OAuth callback - client_id: {client_id}")
|
||||
logger.info(f"Clerk token provided: {bool(clerk_token)}")
|
||||
|
||||
try:
|
||||
# Validate user with Clerk and generate real JWT token
|
||||
user_authenticated = False
|
||||
user_id = None
|
||||
session_id = None
|
||||
real_jwt_token = None
|
||||
|
||||
if clerk_token and CLERK_AVAILABLE:
|
||||
try:
|
||||
# Extract user info from JWT token (no Clerk session verification needed)
|
||||
import jwt
|
||||
decoded_token = jwt.decode(clerk_token, options={"verify_signature": False})
|
||||
user_id = decoded_token.get("user_id") or decoded_token.get("sub")
|
||||
user_email = decoded_token.get("email")
|
||||
token_scopes = decoded_token.get("scopes", ["read", "search"])
|
||||
|
||||
logger.info(f"JWT token claims - user_id: {user_id}, email: {user_email}, scopes: {token_scopes}")
|
||||
|
||||
if user_id and user_email:
|
||||
# JWT token is already signed by Clerk and contains valid user info
|
||||
user_authenticated = True
|
||||
logger.info(f"User authenticated via JWT token - user_id: {user_id}")
|
||||
|
||||
# Use the JWT token directly as the real token (it's already from Clerk template)
|
||||
real_jwt_token = clerk_token
|
||||
logger.info("Using Clerk JWT token directly (already real token)")
|
||||
|
||||
else:
|
||||
logger.error(f"Missing required fields in JWT token - user_id: {bool(user_id)}, email: {bool(user_email)}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"JWT validation failed: {e}")
|
||||
|
||||
# Fallback to cookie validation
|
||||
if not user_authenticated:
|
||||
clerk_session = request.cookies.get("__session")
|
||||
if clerk_session:
|
||||
user_authenticated = True
|
||||
logger.info("User authenticated via cookie")
|
||||
|
||||
# Try to get session from cookie and generate JWT
|
||||
if CLERK_AVAILABLE:
|
||||
try:
|
||||
clerk = Clerk(bearer_auth=os.getenv("CLERK_SECRET_KEY"))
|
||||
# Note: sessions.verify_session is deprecated, but we'll try
|
||||
# In practice, you'd need to extract session_id from cookie
|
||||
logger.info("Cookie authentication - JWT generation not implemented yet")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to generate JWT from cookie: {e}")
|
||||
|
||||
# Only generate authorization code if we have a real JWT token
|
||||
if user_authenticated and real_jwt_token:
|
||||
# Generate authorization code
|
||||
auth_code = f"clerk_auth_{os.urandom(16).hex()}"
|
||||
|
||||
# Prepare code data
|
||||
import time
|
||||
code_data = {
|
||||
"user_id": user_id,
|
||||
"session_id": session_id,
|
||||
"real_jwt_token": real_jwt_token,
|
||||
"user_authenticated": user_authenticated,
|
||||
"client_id": client_id,
|
||||
"redirect_uri": redirect_uri,
|
||||
"scope": scope or "read search"
|
||||
}
|
||||
|
||||
# Try to store in Redis, fall back to in-memory if Redis unavailable
|
||||
store = get_redis_session_store()
|
||||
if store:
|
||||
# Store in Redis with automatic expiration
|
||||
success = store.set_oauth_code(auth_code, code_data)
|
||||
if success:
|
||||
logger.info(f"Stored authorization code {auth_code[:10]}... in Redis with real JWT token")
|
||||
else:
|
||||
logger.error(f"Failed to store authorization code in Redis, falling back to in-memory")
|
||||
# Fall back to in-memory storage
|
||||
if not hasattr(oauth_callback, '_code_storage'):
|
||||
oauth_callback._code_storage = {}
|
||||
oauth_callback._code_storage[auth_code] = code_data
|
||||
else:
|
||||
# Fall back to in-memory storage
|
||||
logger.warning("Redis not available, using in-memory storage")
|
||||
if not hasattr(oauth_callback, '_code_storage'):
|
||||
oauth_callback._code_storage = {}
|
||||
oauth_callback._code_storage[auth_code] = code_data
|
||||
logger.info(f"Stored authorization code in memory (fallback)")
|
||||
|
||||
# Redirect back to client with authorization code
|
||||
redirect_params = {
|
||||
"code": auth_code,
|
||||
"state": state or ""
|
||||
}
|
||||
|
||||
final_redirect_url = f"{redirect_uri}?{urlencode(redirect_params)}"
|
||||
logger.info(f"Redirecting back to client: {final_redirect_url}")
|
||||
|
||||
return RedirectResponse(url=final_redirect_url)
|
||||
else:
|
||||
# No JWT token yet - redirect back to sign-in page to wait for authentication
|
||||
logger.info("No JWT token provided - redirecting back to sign-in to complete authentication")
|
||||
|
||||
# Keep the same redirect URL so the flow continues
|
||||
sign_in_params = {
|
||||
"redirect_url": f"{request.url._url}" # Current callback URL with all params
|
||||
}
|
||||
|
||||
sign_in_url = f"https://yargimcp.com/sign-in?{urlencode(sign_in_params)}"
|
||||
logger.info(f"Redirecting back to sign-in: {sign_in_url}")
|
||||
|
||||
return RedirectResponse(url=sign_in_url)
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Callback processing failed: {e}")
|
||||
return JSONResponse(
|
||||
status_code=500,
|
||||
content={"error": "server_error", "error_description": str(e)}
|
||||
)
|
||||
|
||||
@router.post("/auth/register")
|
||||
async def register_client(request: Request):
|
||||
"""Dynamic Client Registration (RFC 7591)"""
|
||||
|
||||
data = await request.json()
|
||||
logger.info(f"Client registration request: {data}")
|
||||
|
||||
# Simple dynamic registration - accept any client
|
||||
client_id = f"mcp-client-{os.urandom(8).hex()}"
|
||||
|
||||
return JSONResponse({
|
||||
"client_id": client_id,
|
||||
"client_secret": None, # Public client
|
||||
"redirect_uris": data.get("redirect_uris", []),
|
||||
"grant_types": ["authorization_code"],
|
||||
"response_types": ["code"],
|
||||
"client_name": data.get("client_name", "MCP Client"),
|
||||
"token_endpoint_auth_method": "none"
|
||||
})
|
||||
|
||||
@router.post("/auth/callback")
|
||||
async def oauth_callback_post(request: Request):
|
||||
"""OAuth callback POST endpoint for token exchange"""
|
||||
|
||||
# Parse form data (standard OAuth token exchange format)
|
||||
form_data = await request.form()
|
||||
grant_type = form_data.get("grant_type")
|
||||
code = form_data.get("code")
|
||||
redirect_uri = form_data.get("redirect_uri")
|
||||
client_id = form_data.get("client_id")
|
||||
code_verifier = form_data.get("code_verifier")
|
||||
|
||||
logger.info(f"OAuth callback POST - grant_type: {grant_type}")
|
||||
logger.info(f"Code: {code[:20] if code else 'None'}...")
|
||||
logger.info(f"Client ID: {client_id}")
|
||||
logger.info(f"PKCE verifier: {bool(code_verifier)}")
|
||||
|
||||
if grant_type != "authorization_code":
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "unsupported_grant_type"}
|
||||
)
|
||||
|
||||
if not code or not redirect_uri:
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_request", "error_description": "Missing code or redirect_uri"}
|
||||
)
|
||||
|
||||
try:
|
||||
# Validate authorization code
|
||||
if not code.startswith("clerk_auth_"):
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Invalid authorization code"}
|
||||
)
|
||||
|
||||
# Retrieve stored JWT token using authorization code from Redis or in-memory fallback
|
||||
stored_code_data = None
|
||||
|
||||
# Try to get from Redis first, then fall back to in-memory
|
||||
store = get_redis_session_store()
|
||||
if store:
|
||||
stored_code_data = store.get_oauth_code(code, delete_after_use=True)
|
||||
if stored_code_data:
|
||||
logger.info(f"Retrieved authorization code {code[:10]}... from Redis")
|
||||
else:
|
||||
logger.warning(f"Authorization code {code[:10]}... not found in Redis")
|
||||
|
||||
# Fall back to in-memory storage if Redis unavailable or code not found
|
||||
if not stored_code_data and hasattr(oauth_callback, '_code_storage'):
|
||||
stored_code_data = oauth_callback._code_storage.get(code)
|
||||
if stored_code_data:
|
||||
# Clean up in-memory storage
|
||||
oauth_callback._code_storage.pop(code, None)
|
||||
logger.info(f"Retrieved authorization code {code[:10]}... from in-memory storage")
|
||||
|
||||
if not stored_code_data:
|
||||
logger.error(f"No stored data found for authorization code: {code}")
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Authorization code not found or expired"}
|
||||
)
|
||||
|
||||
# Note: Redis TTL handles expiration automatically, but check for manual expiration for in-memory fallback
|
||||
import time
|
||||
expires_at = stored_code_data.get("expires_at", 0)
|
||||
if expires_at and time.time() > expires_at:
|
||||
logger.error(f"Authorization code expired: {code}")
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Authorization code expired"}
|
||||
)
|
||||
|
||||
# Get the real JWT token
|
||||
real_jwt_token = stored_code_data.get("real_jwt_token")
|
||||
|
||||
if real_jwt_token:
|
||||
logger.info("Returning real Clerk JWT token")
|
||||
# Note: Code already deleted from Redis, clean up in-memory fallback if used
|
||||
if hasattr(oauth_callback, '_code_storage'):
|
||||
oauth_callback._code_storage.pop(code, None)
|
||||
|
||||
return JSONResponse({
|
||||
"access_token": real_jwt_token,
|
||||
"token_type": "Bearer",
|
||||
"expires_in": 3600,
|
||||
"scope": "read search"
|
||||
})
|
||||
else:
|
||||
logger.warning("No real JWT token found, generating mock token")
|
||||
# Fallback to mock token for testing
|
||||
mock_token = f"mock_clerk_jwt_{code}"
|
||||
return JSONResponse({
|
||||
"access_token": mock_token,
|
||||
"token_type": "Bearer",
|
||||
"expires_in": 3600,
|
||||
"scope": "read search"
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"OAuth callback POST failed: {e}")
|
||||
return JSONResponse(
|
||||
status_code=500,
|
||||
content={"error": "server_error", "error_description": str(e)}
|
||||
)
|
||||
|
||||
@router.post("/register")
|
||||
async def register_client(request: Request):
|
||||
"""Dynamic Client Registration (RFC 7591)"""
|
||||
|
||||
data = await request.json()
|
||||
logger.info(f"Client registration request: {data}")
|
||||
|
||||
# Simple dynamic registration - accept any client
|
||||
client_id = f"mcp-client-{os.urandom(8).hex()}"
|
||||
|
||||
return JSONResponse({
|
||||
"client_id": client_id,
|
||||
"client_secret": None, # Public client
|
||||
"redirect_uris": data.get("redirect_uris", []),
|
||||
"grant_types": ["authorization_code"],
|
||||
"response_types": ["code"],
|
||||
"client_name": data.get("client_name", "MCP Client"),
|
||||
"token_endpoint_auth_method": "none"
|
||||
})
|
||||
|
||||
@router.post("/token")
|
||||
async def token_endpoint(request: Request):
|
||||
"""OAuth 2.1 Token Endpoint - exchanges code for Clerk JWT"""
|
||||
|
||||
# Parse form data
|
||||
form_data = await request.form()
|
||||
grant_type = form_data.get("grant_type")
|
||||
code = form_data.get("code")
|
||||
redirect_uri = form_data.get("redirect_uri")
|
||||
client_id = form_data.get("client_id")
|
||||
code_verifier = form_data.get("code_verifier")
|
||||
|
||||
logger.info(f"Token exchange - grant_type: {grant_type}")
|
||||
logger.info(f"Code: {code[:20] if code else 'None'}...")
|
||||
|
||||
if grant_type != "authorization_code":
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "unsupported_grant_type"}
|
||||
)
|
||||
|
||||
if not code or not redirect_uri:
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_request", "error_description": "Missing code or redirect_uri"}
|
||||
)
|
||||
|
||||
try:
|
||||
# Validate authorization code
|
||||
if not code.startswith("clerk_auth_"):
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Invalid authorization code"}
|
||||
)
|
||||
|
||||
# Retrieve stored JWT token using authorization code from Redis or in-memory fallback
|
||||
stored_code_data = None
|
||||
|
||||
# Try to get from Redis first, then fall back to in-memory
|
||||
store = get_redis_session_store()
|
||||
if store:
|
||||
stored_code_data = store.get_oauth_code(code, delete_after_use=True)
|
||||
if stored_code_data:
|
||||
logger.info(f"Retrieved authorization code {code[:10]}... from Redis (/token endpoint)")
|
||||
else:
|
||||
logger.warning(f"Authorization code {code[:10]}... not found in Redis (/token endpoint)")
|
||||
|
||||
# Fall back to in-memory storage if Redis unavailable or code not found
|
||||
if not stored_code_data and hasattr(oauth_callback, '_code_storage'):
|
||||
stored_code_data = oauth_callback._code_storage.get(code)
|
||||
if stored_code_data:
|
||||
# Clean up in-memory storage
|
||||
oauth_callback._code_storage.pop(code, None)
|
||||
logger.info(f"Retrieved authorization code {code[:10]}... from in-memory storage (/token endpoint)")
|
||||
|
||||
if not stored_code_data:
|
||||
logger.error(f"No stored data found for authorization code: {code}")
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Authorization code not found or expired"}
|
||||
)
|
||||
|
||||
# Note: Redis TTL handles expiration automatically, but check for manual expiration for in-memory fallback
|
||||
import time
|
||||
expires_at = stored_code_data.get("expires_at", 0)
|
||||
if expires_at and time.time() > expires_at:
|
||||
logger.error(f"Authorization code expired: {code}")
|
||||
return JSONResponse(
|
||||
status_code=400,
|
||||
content={"error": "invalid_grant", "error_description": "Authorization code expired"}
|
||||
)
|
||||
|
||||
# Get the real JWT token
|
||||
real_jwt_token = stored_code_data.get("real_jwt_token")
|
||||
|
||||
if real_jwt_token:
|
||||
logger.info("Returning real Clerk JWT token from /token endpoint")
|
||||
# Note: Code already deleted from Redis, clean up in-memory fallback if used
|
||||
if hasattr(oauth_callback, '_code_storage'):
|
||||
oauth_callback._code_storage.pop(code, None)
|
||||
|
||||
return JSONResponse({
|
||||
"access_token": real_jwt_token,
|
||||
"token_type": "Bearer",
|
||||
"expires_in": 3600,
|
||||
"scope": "read search"
|
||||
})
|
||||
else:
|
||||
logger.warning("No real JWT token found in /token endpoint, generating mock token")
|
||||
# Fallback to mock token for testing
|
||||
mock_token = f"mock_clerk_jwt_{code}"
|
||||
return JSONResponse({
|
||||
"access_token": mock_token,
|
||||
"token_type": "Bearer",
|
||||
"expires_in": 3600,
|
||||
"scope": "read search"
|
||||
})
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Token exchange failed: {e}")
|
||||
return JSONResponse(
|
||||
status_code=500,
|
||||
content={"error": "server_error", "error_description": str(e)}
|
||||
)
|
||||
+1345
-1526
File diff suppressed because it is too large
Load Diff
+3
-2
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "yargi-mcp"
|
||||
version = "0.1.2"
|
||||
version = "0.1.4"
|
||||
description = "MCP Server For Turkish Legal Databases"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
@@ -26,7 +26,7 @@ dependencies = [
|
||||
"pydantic>=2.11.4",
|
||||
"aiohttp>=3.11.18",
|
||||
"playwright>=1.52.0",
|
||||
"fastmcp>=2.9.2",
|
||||
"fastmcp>=2.10.5",
|
||||
"pypdf>=5.5.0",
|
||||
"fastapi>=0.115.14",
|
||||
"PyJWT>=2.8.0",
|
||||
@@ -48,6 +48,7 @@ production = [
|
||||
saas = [
|
||||
"clerk-backend-api>=3.0.0",
|
||||
"stripe>=9.1.0",
|
||||
"upstash-redis>=1.1.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
|
||||
@@ -0,0 +1,464 @@
|
||||
"""
|
||||
Redis Session Store for OAuth Authorization Codes and User Sessions
|
||||
|
||||
This module provides Redis-based storage for OAuth authorization codes and user sessions,
|
||||
enabling multi-machine deployment support by replacing in-memory storage.
|
||||
|
||||
Uses Upstash Redis via REST API for serverless-friendly operation.
|
||||
"""
|
||||
|
||||
import os
|
||||
import json
|
||||
import time
|
||||
import logging
|
||||
from typing import Optional, Dict, Any, Union
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
try:
|
||||
from upstash_redis import Redis
|
||||
UPSTASH_AVAILABLE = True
|
||||
except ImportError:
|
||||
UPSTASH_AVAILABLE = False
|
||||
Redis = None
|
||||
|
||||
# Use standard Python exceptions for Redis connection errors
|
||||
import socket
|
||||
from requests.exceptions import ConnectionError as RequestsConnectionError, Timeout as RequestsTimeout
|
||||
|
||||
class RedisSessionStore:
|
||||
"""
|
||||
Redis-based session store for OAuth flows and user sessions.
|
||||
|
||||
Uses Upstash Redis REST API for connection-free operation suitable for
|
||||
multi-instance deployments on platforms like Fly.io.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize Redis connection using environment variables."""
|
||||
if not UPSTASH_AVAILABLE:
|
||||
raise ImportError("upstash-redis package is required. Install with: pip install upstash-redis")
|
||||
|
||||
# Initialize Upstash Redis client from environment with optimized connection settings
|
||||
try:
|
||||
# Get Upstash Redis configuration
|
||||
redis_url = os.getenv("UPSTASH_REDIS_REST_URL")
|
||||
redis_token = os.getenv("UPSTASH_REDIS_REST_TOKEN")
|
||||
|
||||
if not redis_url or not redis_token:
|
||||
raise ValueError("UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN must be set")
|
||||
|
||||
logger.info(f"Connecting to Upstash Redis at {redis_url[:30]}...")
|
||||
|
||||
# Initialize with explicit configuration for better SSL handling
|
||||
self.redis = Redis(
|
||||
url=redis_url,
|
||||
token=redis_token
|
||||
)
|
||||
|
||||
logger.info("Upstash Redis client created")
|
||||
|
||||
# Skip connection test during initialization to prevent server hang
|
||||
# Connection will be tested during first actual operation
|
||||
logger.info("Redis client initialized - connection will be tested on first use")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to initialize Upstash Redis: {e}")
|
||||
raise
|
||||
|
||||
# TTL values (in seconds)
|
||||
self.oauth_code_ttl = int(os.getenv("OAUTH_CODE_TTL", "600")) # 10 minutes
|
||||
self.session_ttl = int(os.getenv("SESSION_TTL", "3600")) # 1 hour
|
||||
|
||||
def _serialize_data(self, data: Dict[str, Any]) -> Dict[str, str]:
|
||||
"""Convert data to Redis-compatible string format."""
|
||||
serialized = {}
|
||||
for key, value in data.items():
|
||||
if isinstance(value, (dict, list)):
|
||||
serialized[key] = json.dumps(value)
|
||||
elif isinstance(value, (int, float)):
|
||||
serialized[key] = str(value)
|
||||
elif isinstance(value, bool):
|
||||
serialized[key] = "true" if value else "false"
|
||||
else:
|
||||
serialized[key] = str(value)
|
||||
return serialized
|
||||
|
||||
def _deserialize_data(self, data: Dict[str, str]) -> Dict[str, Any]:
|
||||
"""Convert Redis string data back to original types."""
|
||||
if not data:
|
||||
return {}
|
||||
|
||||
deserialized = {}
|
||||
for key, value in data.items():
|
||||
if not isinstance(value, str):
|
||||
deserialized[key] = value
|
||||
continue
|
||||
|
||||
# Try to deserialize JSON
|
||||
if value.startswith(('[', '{')):
|
||||
try:
|
||||
deserialized[key] = json.loads(value)
|
||||
continue
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
# Try to convert numbers
|
||||
if value.isdigit():
|
||||
deserialized[key] = int(value)
|
||||
continue
|
||||
|
||||
if value.replace('.', '').isdigit():
|
||||
try:
|
||||
deserialized[key] = float(value)
|
||||
continue
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
# Handle booleans
|
||||
if value in ("true", "false"):
|
||||
deserialized[key] = value == "true"
|
||||
continue
|
||||
|
||||
# Keep as string
|
||||
deserialized[key] = value
|
||||
|
||||
return deserialized
|
||||
|
||||
# OAuth Authorization Code Methods
|
||||
|
||||
def set_oauth_code(self, code: str, data: Dict[str, Any]) -> bool:
|
||||
"""
|
||||
Store OAuth authorization code with automatic expiration.
|
||||
|
||||
Args:
|
||||
code: Authorization code string
|
||||
data: Code data including user_id, client_id, etc.
|
||||
|
||||
Returns:
|
||||
True if stored successfully, False otherwise
|
||||
"""
|
||||
try:
|
||||
key = f"oauth:code:{code}"
|
||||
|
||||
# Add timestamp for debugging
|
||||
data_with_timestamp = data.copy()
|
||||
data_with_timestamp.update({
|
||||
"created_at": time.time(),
|
||||
"expires_at": time.time() + self.oauth_code_ttl
|
||||
})
|
||||
|
||||
# Serialize and store - Upstash Redis doesn't support mapping parameter
|
||||
serialized_data = self._serialize_data(data_with_timestamp)
|
||||
|
||||
# Use individual hset calls for each field with retry logic
|
||||
max_retries = 3
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
# Clear any existing data first
|
||||
self.redis.delete(key)
|
||||
|
||||
# Set all fields in a pipeline-like manner
|
||||
for field, value in serialized_data.items():
|
||||
self.redis.hset(key, field, value)
|
||||
|
||||
# Set expiration
|
||||
self.redis.expire(key, self.oauth_code_ttl)
|
||||
|
||||
logger.info(f"Stored OAuth code {code[:10]}... with TTL {self.oauth_code_ttl}s (attempt {attempt + 1})")
|
||||
return True
|
||||
|
||||
except (RequestsConnectionError, RequestsTimeout, OSError, socket.error) as e:
|
||||
logger.warning(f"Redis connection error on attempt {attempt + 1}: {e}")
|
||||
if attempt == max_retries - 1:
|
||||
raise # Re-raise on final attempt
|
||||
time.sleep(0.5 * (attempt + 1)) # Exponential backoff
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to store OAuth code {code[:10]}... after {max_retries} attempts: {e}")
|
||||
return False
|
||||
|
||||
def get_oauth_code(self, code: str, delete_after_use: bool = True) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Retrieve OAuth authorization code data.
|
||||
|
||||
Args:
|
||||
code: Authorization code string
|
||||
delete_after_use: If True, delete the code after retrieval (one-time use)
|
||||
|
||||
Returns:
|
||||
Code data dictionary or None if not found/expired
|
||||
"""
|
||||
max_retries = 3
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
key = f"oauth:code:{code}"
|
||||
|
||||
# Get all hash fields with retry
|
||||
data = self.redis.hgetall(key)
|
||||
|
||||
if not data:
|
||||
logger.warning(f"OAuth code {code[:10]}... not found or expired (attempt {attempt + 1})")
|
||||
return None
|
||||
|
||||
# Deserialize data
|
||||
deserialized_data = self._deserialize_data(data)
|
||||
|
||||
# Check manual expiration (in case Redis TTL failed)
|
||||
expires_at = deserialized_data.get("expires_at", 0)
|
||||
if expires_at and time.time() > expires_at:
|
||||
logger.warning(f"OAuth code {code[:10]}... manually expired")
|
||||
try:
|
||||
self.redis.delete(key)
|
||||
except Exception as del_error:
|
||||
logger.warning(f"Failed to delete expired code: {del_error}")
|
||||
return None
|
||||
|
||||
# Delete after use for security (one-time use)
|
||||
if delete_after_use:
|
||||
try:
|
||||
self.redis.delete(key)
|
||||
logger.info(f"Retrieved and deleted OAuth code {code[:10]}... (attempt {attempt + 1})")
|
||||
except Exception as del_error:
|
||||
logger.warning(f"Failed to delete code after use: {del_error}")
|
||||
# Continue anyway since we got the data
|
||||
else:
|
||||
logger.info(f"Retrieved OAuth code {code[:10]}... (not deleted, attempt {attempt + 1})")
|
||||
|
||||
return deserialized_data
|
||||
|
||||
except (RequestsConnectionError, RequestsTimeout, OSError, socket.error) as e:
|
||||
logger.warning(f"Redis connection error on retrieval attempt {attempt + 1}: {e}")
|
||||
if attempt == max_retries - 1:
|
||||
logger.error(f"Failed to retrieve OAuth code {code[:10]}... after {max_retries} attempts: {e}")
|
||||
return None
|
||||
time.sleep(0.5 * (attempt + 1)) # Exponential backoff
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to retrieve OAuth code {code[:10]}... on attempt {attempt + 1}: {e}")
|
||||
if attempt == max_retries - 1:
|
||||
return None
|
||||
time.sleep(0.5 * (attempt + 1))
|
||||
|
||||
return None
|
||||
|
||||
# User Session Methods
|
||||
|
||||
def set_session(self, session_id: str, user_data: Dict[str, Any]) -> bool:
|
||||
"""
|
||||
Store user session data with sliding expiration.
|
||||
|
||||
Args:
|
||||
session_id: Unique session identifier
|
||||
user_data: User session data (user_id, email, scopes, etc.)
|
||||
|
||||
Returns:
|
||||
True if stored successfully, False otherwise
|
||||
"""
|
||||
try:
|
||||
key = f"session:{session_id}"
|
||||
|
||||
# Add session metadata
|
||||
session_data = user_data.copy()
|
||||
session_data.update({
|
||||
"session_id": session_id,
|
||||
"created_at": time.time(),
|
||||
"last_accessed": time.time()
|
||||
})
|
||||
|
||||
# Serialize and store - Upstash Redis doesn't support mapping parameter
|
||||
serialized_data = self._serialize_data(session_data)
|
||||
|
||||
# Use individual hset calls for each field (Upstash compatibility)
|
||||
for field, value in serialized_data.items():
|
||||
self.redis.hset(key, field, value)
|
||||
self.redis.expire(key, self.session_ttl)
|
||||
|
||||
logger.info(f"Stored session {session_id[:10]}... with TTL {self.session_ttl}s")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to store session {session_id[:10]}...: {e}")
|
||||
return False
|
||||
|
||||
def get_session(self, session_id: str, refresh_ttl: bool = True) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Retrieve user session data.
|
||||
|
||||
Args:
|
||||
session_id: Session identifier
|
||||
refresh_ttl: If True, extend session TTL on access
|
||||
|
||||
Returns:
|
||||
Session data dictionary or None if not found/expired
|
||||
"""
|
||||
try:
|
||||
key = f"session:{session_id}"
|
||||
|
||||
# Get session data
|
||||
data = self.redis.hgetall(key)
|
||||
|
||||
if not data:
|
||||
logger.warning(f"Session {session_id[:10]}... not found or expired")
|
||||
return None
|
||||
|
||||
# Deserialize data
|
||||
session_data = self._deserialize_data(data)
|
||||
|
||||
# Update last accessed time and refresh TTL
|
||||
if refresh_ttl:
|
||||
session_data["last_accessed"] = time.time()
|
||||
self.redis.hset(key, "last_accessed", str(time.time()))
|
||||
self.redis.expire(key, self.session_ttl)
|
||||
logger.debug(f"Refreshed session {session_id[:10]}... TTL")
|
||||
|
||||
return session_data
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to retrieve session {session_id[:10]}...: {e}")
|
||||
return None
|
||||
|
||||
def delete_session(self, session_id: str) -> bool:
|
||||
"""
|
||||
Delete user session (logout).
|
||||
|
||||
Args:
|
||||
session_id: Session identifier
|
||||
|
||||
Returns:
|
||||
True if deleted successfully, False otherwise
|
||||
"""
|
||||
try:
|
||||
key = f"session:{session_id}"
|
||||
result = self.redis.delete(key)
|
||||
|
||||
if result:
|
||||
logger.info(f"Deleted session {session_id[:10]}...")
|
||||
return True
|
||||
else:
|
||||
logger.warning(f"Session {session_id[:10]}... not found for deletion")
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to delete session {session_id[:10]}...: {e}")
|
||||
return False
|
||||
|
||||
# Health Check Methods
|
||||
|
||||
def health_check(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Perform Redis health check.
|
||||
|
||||
Returns:
|
||||
Health status dictionary
|
||||
"""
|
||||
try:
|
||||
# Test basic operations
|
||||
test_key = f"health:check:{int(time.time())}"
|
||||
test_value = {"timestamp": time.time(), "test": True}
|
||||
|
||||
# Test set - Use individual hset calls for Upstash compatibility
|
||||
serialized_test = self._serialize_data(test_value)
|
||||
for field, value in serialized_test.items():
|
||||
self.redis.hset(test_key, field, value)
|
||||
|
||||
# Test get
|
||||
retrieved = self.redis.hgetall(test_key)
|
||||
|
||||
# Test delete
|
||||
self.redis.delete(test_key)
|
||||
|
||||
return {
|
||||
"status": "healthy",
|
||||
"redis_connected": True,
|
||||
"operations_working": bool(retrieved),
|
||||
"timestamp": datetime.utcnow().isoformat()
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Redis health check failed: {e}")
|
||||
return {
|
||||
"status": "unhealthy",
|
||||
"redis_connected": False,
|
||||
"error": str(e),
|
||||
"timestamp": datetime.utcnow().isoformat()
|
||||
}
|
||||
|
||||
def get_stats(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get Redis usage statistics.
|
||||
|
||||
Returns:
|
||||
Statistics dictionary
|
||||
"""
|
||||
try:
|
||||
# Get basic info (not all Upstash plans support INFO command)
|
||||
stats = {
|
||||
"oauth_codes_pattern": "oauth:code:*",
|
||||
"sessions_pattern": "session:*",
|
||||
"timestamp": datetime.utcnow().isoformat()
|
||||
}
|
||||
|
||||
try:
|
||||
# Try to get counts (may fail on some Upstash plans)
|
||||
oauth_keys = self.redis.keys("oauth:code:*")
|
||||
session_keys = self.redis.keys("session:*")
|
||||
|
||||
stats.update({
|
||||
"active_oauth_codes": len(oauth_keys) if oauth_keys else 0,
|
||||
"active_sessions": len(session_keys) if session_keys else 0
|
||||
})
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not get detailed stats: {e}")
|
||||
stats["warning"] = "Detailed stats not available on this Redis plan"
|
||||
|
||||
return stats
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get Redis stats: {e}")
|
||||
return {"error": str(e), "timestamp": datetime.utcnow().isoformat()}
|
||||
|
||||
# Global instance for easy importing
|
||||
redis_store = None
|
||||
|
||||
def get_redis_store() -> Optional[RedisSessionStore]:
|
||||
"""
|
||||
Get global Redis store instance (singleton pattern).
|
||||
|
||||
Returns:
|
||||
RedisSessionStore instance or None if initialization fails
|
||||
"""
|
||||
global redis_store
|
||||
|
||||
if redis_store is None:
|
||||
try:
|
||||
logger.info("Initializing Redis store...")
|
||||
redis_store = RedisSessionStore()
|
||||
logger.info("Redis store initialized successfully")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to initialize Redis store: {e}")
|
||||
redis_store = None
|
||||
|
||||
return redis_store
|
||||
|
||||
def init_redis_store() -> RedisSessionStore:
|
||||
"""
|
||||
Initialize Redis store and perform health check.
|
||||
|
||||
Returns:
|
||||
RedisSessionStore instance
|
||||
|
||||
Raises:
|
||||
Exception if Redis is not available or unhealthy
|
||||
"""
|
||||
store = get_redis_store()
|
||||
|
||||
# Perform health check
|
||||
health = store.health_check()
|
||||
|
||||
if health["status"] != "healthy":
|
||||
raise Exception(f"Redis health check failed: {health}")
|
||||
|
||||
logger.info("Redis session store initialized and healthy")
|
||||
return store
|
||||
@@ -25,35 +25,31 @@ class RekabetKararTuruAdiEnum(str, Enum):
|
||||
|
||||
class RekabetKurumuSearchRequest(BaseModel):
|
||||
"""Model for Rekabet Kurumu (Turkish Competition Authority) search request."""
|
||||
sayfaAdi: Optional[str] = Field(None, description="Search in decision title (Başlık).")
|
||||
YayinlanmaTarihi: Optional[str] = Field(None, description="Publication date (Yayım Tarihi), e.g., DD.MM.YYYY.")
|
||||
PdfText: Optional[str] = Field(
|
||||
None,
|
||||
description='Search in decision text (Metin). For an exact phrase match, enclose the phrase in double quotes (e.g., "\\"vertical agreement\\" competition). The website indicates that using "" provides more precise results for phrases.'
|
||||
)
|
||||
# This field uses the GUID enum as it's used by the client to make the actual web request.
|
||||
KararTuruID: Optional[RekabetKararTuruGuidEnum] = Field(RekabetKararTuruGuidEnum.TUMU, description="Decision type (Karar Türü) GUID for internal client use, corresponding to the website's values.")
|
||||
KararSayisi: Optional[str] = Field(None, description="Decision number (Karar Sayısı).")
|
||||
KararTarihi: Optional[str] = Field(None, description="Decision date (Karar Tarihi), e.g., DD.MM.YYYY.")
|
||||
page: int = Field(1, ge=1, description="Page number to fetch for results list.")
|
||||
sayfaAdi: Optional[str] = Field(None, description="Title")
|
||||
YayinlanmaTarihi: Optional[str] = Field(None, description="Date")
|
||||
PdfText: Optional[str] = Field(None, description="Text")
|
||||
KararTuruID: Optional[RekabetKararTuruGuidEnum] = Field(RekabetKararTuruGuidEnum.TUMU, description="Type")
|
||||
KararSayisi: Optional[str] = Field(None, description="No")
|
||||
KararTarihi: Optional[str] = Field(None, description="Date")
|
||||
page: int = Field(1, ge=1, description="Page")
|
||||
|
||||
class RekabetDecisionSummary(BaseModel):
|
||||
"""Model for a single Rekabet Kurumu decision summary from search results."""
|
||||
publication_date: Optional[str] = Field(None, description="Publication Date (Yayımlanma Tarihi).")
|
||||
decision_number: Optional[str] = Field(None, description="Decision Number (Karar Sayısı).")
|
||||
decision_date: Optional[str] = Field(None, description="Decision Date (Karar Tarihi).")
|
||||
decision_type_text: Optional[str] = Field(None, description="Decision Type as text (Karar Türü - metin olarak).")
|
||||
title: Optional[str] = Field(None, description="Decision title or summary text.")
|
||||
decision_url: Optional[HttpUrl] = Field(None, description="URL to the decision's landing page (e.g., /Karar?kararId=...).")
|
||||
karar_id: Optional[str] = Field(None, description="GUID of the decision, extracted from its URL.")
|
||||
related_cases_url: Optional[HttpUrl] = Field(None, description="URL to related court cases page, if available.")
|
||||
publication_date: Optional[str] = Field(None, description="Pub date")
|
||||
decision_number: Optional[str] = Field(None, description="Number")
|
||||
decision_date: Optional[str] = Field(None, description="Date")
|
||||
decision_type_text: Optional[str] = Field(None, description="Type")
|
||||
title: Optional[str] = Field(None, description="Title")
|
||||
decision_url: Optional[HttpUrl] = Field(None, description="URL")
|
||||
karar_id: Optional[str] = Field(None, description="ID")
|
||||
related_cases_url: Optional[HttpUrl] = Field(None, description="Cases URL")
|
||||
|
||||
class RekabetSearchResult(BaseModel):
|
||||
"""Model for the overall search result for Rekabet Kurumu decisions."""
|
||||
decisions: List[RekabetDecisionSummary]
|
||||
total_records_found: Optional[int] = Field(None, description="Total number of records found matching the query.")
|
||||
retrieved_page_number: int = Field(description="The page number of the results that were retrieved.")
|
||||
total_pages: Optional[int] = Field(None, description="Total number of pages available for the query.")
|
||||
total_records_found: Optional[int] = Field(None, description="Total")
|
||||
retrieved_page_number: int = Field(description="Page")
|
||||
total_pages: Optional[int] = Field(None, description="Pages")
|
||||
|
||||
class RekabetDocument(BaseModel):
|
||||
"""
|
||||
@@ -61,16 +57,15 @@ class RekabetDocument(BaseModel):
|
||||
Contains metadata from the landing page, a link to the PDF,
|
||||
and the PDF's content converted to paginated Markdown.
|
||||
"""
|
||||
source_landing_page_url: HttpUrl = Field(description="The URL of the decision's landing page from which the PDF was identified.")
|
||||
karar_id: str = Field(description="GUID of the decision.")
|
||||
source_landing_page_url: HttpUrl = Field(description="Source URL")
|
||||
karar_id: str = Field(description="ID")
|
||||
|
||||
title_on_landing_page: Optional[str] = Field(None, description="Title as found on the landing page (e.g., from <title> tag or a main heading). Could be a generic title if direct PDF.")
|
||||
pdf_url: Optional[HttpUrl] = Field(None, description="Direct URL to the decision PDF document, if successfully found and resolved.")
|
||||
title_on_landing_page: Optional[str] = Field(None, description="Title")
|
||||
pdf_url: Optional[HttpUrl] = Field(None, description="PDF URL")
|
||||
|
||||
# Fields for Markdown content derived from the PDF
|
||||
markdown_chunk: Optional[str] = Field(None, description="A 5,000 character chunk of the Markdown content derived from the decision PDF.")
|
||||
current_page: int = Field(1, description="The current page number of the PDF-derived markdown chunk (1-indexed).")
|
||||
total_pages: int = Field(1, description="Total number of pages for the full PDF-derived markdown content. Will be 0 if content could not be processed.")
|
||||
is_paginated: bool = Field(False, description="True if the full PDF-derived markdown content is split into multiple pages.")
|
||||
markdown_chunk: Optional[str] = Field(None, description="Content")
|
||||
current_page: int = Field(1, description="Page")
|
||||
total_pages: int = Field(1, description="Total pages")
|
||||
is_paginated: bool = Field(False, description="Paginated")
|
||||
|
||||
error_message: Optional[str] = Field(None, description="Contains an error message if the document retrieval or processing failed at any stage.")
|
||||
error_message: Optional[str] = Field(None, description="Error")
|
||||
@@ -6,8 +6,7 @@ from bs4 import BeautifulSoup
|
||||
from typing import Dict, Any, List, Optional, Tuple
|
||||
import logging
|
||||
import html
|
||||
import tempfile
|
||||
import os
|
||||
import io
|
||||
from urllib.parse import urlencode, urljoin
|
||||
from markitdown import MarkItDown
|
||||
|
||||
@@ -532,21 +531,18 @@ class SayistayApiClient:
|
||||
raise
|
||||
|
||||
def _convert_html_to_markdown(self, html_content: str) -> Optional[str]:
|
||||
"""Convert HTML content to Markdown using MarkItDown."""
|
||||
"""Convert HTML content to Markdown using MarkItDown with BytesIO to avoid filename length issues."""
|
||||
if not html_content:
|
||||
return None
|
||||
|
||||
temp_file_path = None
|
||||
try:
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = html_content.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown()
|
||||
|
||||
# Write HTML to temp file
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp:
|
||||
tmp.write(html_content)
|
||||
temp_file_path = tmp.name
|
||||
|
||||
# Convert
|
||||
result = md_converter.convert(temp_file_path)
|
||||
result = md_converter.convert(html_stream)
|
||||
markdown_content = result.text_content
|
||||
|
||||
logger.info("Successfully converted HTML to Markdown")
|
||||
@@ -555,9 +551,6 @@ class SayistayApiClient:
|
||||
except Exception as e:
|
||||
logger.error(f"Error converting HTML to Markdown: {e}")
|
||||
return f"Error converting HTML content: {str(e)}"
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path):
|
||||
os.remove(temp_file_path)
|
||||
|
||||
async def get_document_as_markdown(self, decision_id: str, decision_type: str) -> SayistayDocumentMarkdown:
|
||||
"""
|
||||
|
||||
+23
-105
@@ -16,30 +16,18 @@ class GenelKurulSearchRequest(BaseModel):
|
||||
of the Turkish Court of Accounts, typically addressing interpretation of
|
||||
audit and accountability regulations.
|
||||
"""
|
||||
karar_no: Optional[str] = Field(None, description="Decision number (e.g., '5415')")
|
||||
karar_ek: Optional[str] = Field(None, description="Decision appendix number (max 99)")
|
||||
karar_no: Optional[str] = Field(None, description="Decision no")
|
||||
karar_ek: Optional[str] = Field(None, description="Appendix no")
|
||||
|
||||
karar_tarih_baslangic: Optional[str] = Field(None, description="""
|
||||
Decision start year for date range filtering.
|
||||
Available years: 2006-2024. Format: 'YYYY' (e.g., '2020')
|
||||
Use with karar_tarih_bitis for date range filtering.
|
||||
""")
|
||||
karar_tarih_baslangic: Optional[str] = Field(None, description="Start year (YYYY)")
|
||||
|
||||
karar_tarih_bitis: Optional[str] = Field(None, description="""
|
||||
Decision end year for date range filtering.
|
||||
Available years: 2006-2024. Format: 'YYYY' (e.g., '2024')
|
||||
Use with karar_tarih_baslangic for date range filtering.
|
||||
""")
|
||||
karar_tarih_bitis: Optional[str] = Field(None, description="End year")
|
||||
|
||||
karar_tamami: Optional[str] = Field(None, description="""
|
||||
Content/text search within decision summaries (max 400 characters).
|
||||
Searches in decision abstracts and main content.
|
||||
Example: 'belediye taşınmaz tahsis'
|
||||
""")
|
||||
karar_tamami: Optional[str] = Field(None, description="Value")
|
||||
|
||||
# DataTables pagination
|
||||
start: int = Field(0, description="Starting record for pagination (0-based)")
|
||||
length: int = Field(10, description="Number of records per page (1-100)")
|
||||
length: int = Field(10, description="Number of records per page (1-10)")
|
||||
|
||||
class GenelKurulDecision(BaseModel):
|
||||
"""Single Genel Kurul decision entry from search results."""
|
||||
@@ -66,62 +54,27 @@ class TemyizKuruluSearchRequest(BaseModel):
|
||||
Temyiz Kurulu reviews appeals against audit chamber decisions,
|
||||
providing higher-level review of audit findings and sanctions.
|
||||
"""
|
||||
ilam_dairesi: DaireEnum = Field("ALL", description="""
|
||||
Chamber/Department filter for appeals board decisions.
|
||||
• ALL: All chambers (default)
|
||||
• 1-8: Specific chamber number (1. Daire through 8. Daire)
|
||||
Each chamber specializes in different types of public institutions.
|
||||
""")
|
||||
ilam_dairesi: DaireEnum = Field("ALL", description="Value")
|
||||
|
||||
yili: Optional[str] = Field(None, description="""
|
||||
Account year filter (Hesap Yılı).
|
||||
Available years: 1993-2022. Format: 'YYYY' (e.g., '2020')
|
||||
Refers to the fiscal year being audited, not decision date.
|
||||
""")
|
||||
yili: Optional[str] = Field(None, description="Value")
|
||||
|
||||
karar_tarih_baslangic: Optional[str] = Field(None, description="""
|
||||
Decision start year for date range filtering.
|
||||
Available years: 2000, 2006-2024. Format: 'YYYY' (e.g., '2020')
|
||||
Use with karar_tarih_bitis for date range filtering.
|
||||
""")
|
||||
karar_tarih_baslangic: Optional[str] = Field(None, description="Value")
|
||||
|
||||
karar_tarih_bitis: Optional[str] = Field(None, description="""
|
||||
Decision end year for date range filtering.
|
||||
Available years: 2000, 2006-2024. Format: 'YYYY' (e.g., '2024')
|
||||
Use with karar_tarih_baslangic for date range filtering.
|
||||
""")
|
||||
karar_tarih_bitis: Optional[str] = Field(None, description="End year")
|
||||
|
||||
kamu_idaresi_turu: KamuIdaresiTuruEnum = Field("ALL", description="""
|
||||
Public administration type filter:
|
||||
• ALL: All institutions (default)
|
||||
• Genel Bütçe Kapsamındaki İdareler: General budget administrations
|
||||
• Yüksek Öğretim Kurumları: Higher education institutions
|
||||
• Belediyeler ve Bağlı İdareler: Municipalities and affiliates
|
||||
• Other specific institution types
|
||||
""")
|
||||
kamu_idaresi_turu: KamuIdaresiTuruEnum = Field("ALL", description="Value")
|
||||
|
||||
ilam_no: Optional[str] = Field(None, description="Audit report number (İlam No, max 50 chars)")
|
||||
dosya_no: Optional[str] = Field(None, description="File number for the case")
|
||||
temyiz_tutanak_no: Optional[str] = Field(None, description="Appeals board meeting minutes number")
|
||||
|
||||
temyiz_karar: Optional[str] = Field(None, description="""
|
||||
Content search within appeals decisions.
|
||||
Searches decision text and reasoning.
|
||||
Example: 'araç kiralama kasko'
|
||||
""")
|
||||
temyiz_karar: Optional[str] = Field(None, description="Value")
|
||||
|
||||
web_karar_konusu: WebKararKonusuEnum = Field("ALL", description="""
|
||||
Decision subject category filter:
|
||||
• ALL: All subjects (default)
|
||||
• İhale Mevzuatı ile İlgili Kararlar: Procurement legislation
|
||||
• Personel Mevzuatı ile İlgili Kararlar: Personnel legislation
|
||||
• Harcırah Mevzuatı ile İlgili Kararlar: Travel allowance legislation
|
||||
• Other specialized legal areas
|
||||
""")
|
||||
web_karar_konusu: WebKararKonusuEnum = Field("ALL", description="Value")
|
||||
|
||||
# DataTables pagination
|
||||
start: int = Field(0, description="Starting record for pagination (0-based)")
|
||||
length: int = Field(10, description="Number of records per page (1-100)")
|
||||
length: int = Field(10, description="Number of records per page (1-10)")
|
||||
|
||||
class TemyizKuruluDecision(BaseModel):
|
||||
"""Single Temyiz Kurulu decision entry from search results."""
|
||||
@@ -148,60 +101,25 @@ class DaireSearchRequest(BaseModel):
|
||||
Daire decisions are first-instance audit findings and sanctions
|
||||
issued by individual audit chambers before potential appeals.
|
||||
"""
|
||||
yargilama_dairesi: DaireEnum = Field("ALL", description="""
|
||||
Audit chamber filter:
|
||||
• ALL: All chambers (default)
|
||||
• 1-8: Specific chamber number (1. Daire through 8. Daire)
|
||||
Each chamber audits different types of public institutions.
|
||||
""")
|
||||
yargilama_dairesi: DaireEnum = Field("ALL", description="Value")
|
||||
|
||||
karar_tarih_baslangic: Optional[str] = Field(None, description="""
|
||||
Decision start year for date range filtering.
|
||||
Available years: 2012-2025. Format: 'YYYY' (e.g., '2020')
|
||||
Use with karar_tarih_bitis for date range filtering.
|
||||
""")
|
||||
karar_tarih_baslangic: Optional[str] = Field(None, description="Value")
|
||||
|
||||
karar_tarih_bitis: Optional[str] = Field(None, description="""
|
||||
Decision end year for date range filtering.
|
||||
Available years: 2012-2025. Format: 'YYYY' (e.g., '2024')
|
||||
Use with karar_tarih_baslangic for date range filtering.
|
||||
""")
|
||||
karar_tarih_bitis: Optional[str] = Field(None, description="End year")
|
||||
|
||||
ilam_no: Optional[str] = Field(None, description="Audit report number (İlam No, max 50 chars)")
|
||||
|
||||
kamu_idaresi_turu: KamuIdaresiTuruEnum = Field("ALL", description="""
|
||||
Public administration type filter:
|
||||
• ALL: All institutions (default)
|
||||
• Genel Bütçe Kapsamındaki İdareler: General budget administrations
|
||||
• Yüksek Öğretim Kurumları: Higher education institutions
|
||||
• Belediyeler ve Bağlı İdareler: Municipalities and affiliates
|
||||
• Other specific institution types
|
||||
""")
|
||||
kamu_idaresi_turu: KamuIdaresiTuruEnum = Field("ALL", description="Value")
|
||||
|
||||
hesap_yili: Optional[str] = Field(None, description="""
|
||||
Account year filter (Hesap Yılı).
|
||||
Available years: 2005, 2008-2023. Format: 'YYYY' (e.g., '2020')
|
||||
Refers to the fiscal year being audited, not decision date.
|
||||
""")
|
||||
hesap_yili: Optional[str] = Field(None, description="Value")
|
||||
|
||||
web_karar_konusu: WebKararKonusuEnum = Field("ALL", description="""
|
||||
Decision subject category filter:
|
||||
• ALL: All subjects (default)
|
||||
• İhale Mevzuatı ile İlgili Kararlar: Procurement legislation
|
||||
• Personel Mevzuatı ile İlgili Kararlar: Personnel legislation
|
||||
• Vergi Resmi Harç ve Diğer Gelirlerle İlgili Kararlar: Tax and fee legislation
|
||||
• Other specialized legal areas
|
||||
""")
|
||||
web_karar_konusu: WebKararKonusuEnum = Field("ALL", description="Value")
|
||||
|
||||
web_karar_metni: Optional[str] = Field(None, description="""
|
||||
Content search within chamber decisions.
|
||||
Searches decision text and audit findings.
|
||||
Example: 'birim fiyat revize edilmemesi'
|
||||
""")
|
||||
web_karar_metni: Optional[str] = Field(None, description="Value")
|
||||
|
||||
# DataTables pagination
|
||||
start: int = Field(0, description="Starting record for pagination (0-based)")
|
||||
length: int = Field(10, description="Number of records per page (1-100)")
|
||||
length: int = Field(10, description="Number of records per page (1-10)")
|
||||
|
||||
class DaireDecision(BaseModel):
|
||||
"""Single Daire decision entry from search results."""
|
||||
@@ -235,7 +153,7 @@ class SayistayDocumentMarkdown(BaseModel):
|
||||
decision types (Genel Kurul, Temyiz Kurulu, Daire).
|
||||
"""
|
||||
decision_id: str = Field(..., description="Unique decision identifier")
|
||||
decision_type: str = Field(..., description="Type of decision: 'genel_kurul', 'temyiz_kurulu', or 'daire'")
|
||||
decision_type: str = Field(..., description="Value")
|
||||
source_url: str = Field(..., description="Original URL where the document was retrieved")
|
||||
markdown_content: Optional[str] = Field(None, description="Full decision text converted to Markdown format")
|
||||
retrieval_date: Optional[str] = Field(None, description="Date when document was retrieved (ISO format)")
|
||||
|
||||
@@ -7,8 +7,7 @@ from typing import Dict, Any, List, Optional, Union, Tuple
|
||||
import logging
|
||||
import html
|
||||
import re
|
||||
import tempfile
|
||||
import os
|
||||
import io
|
||||
from markitdown import MarkItDown
|
||||
from urllib.parse import urljoin, urlencode # urlencode for aiohttp form data
|
||||
|
||||
@@ -196,21 +195,18 @@ class UyusmazlikApiClient:
|
||||
html_input_for_markdown = processed_html
|
||||
|
||||
markdown_text = None
|
||||
temp_file_path = None
|
||||
try:
|
||||
md_converter = MarkItDown()
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file:
|
||||
tmp_file.write(html_input_for_markdown)
|
||||
temp_file_path = tmp_file.name
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = html_input_for_markdown.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
conversion_result = md_converter.convert(temp_file_path)
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown()
|
||||
conversion_result = md_converter.convert(html_stream)
|
||||
markdown_text = conversion_result.text_content
|
||||
logger.info("UyusmazlikApiClient: HTML to Markdown conversion successful.")
|
||||
except Exception as e:
|
||||
logger.error(f"UyusmazlikApiClient: Error during MarkItDown HTML to Markdown conversion: {e}")
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path):
|
||||
os.remove(temp_file_path)
|
||||
return markdown_text
|
||||
|
||||
async def get_decision_document_as_markdown(self, document_url: str) -> UyusmazlikDocumentMarkdown:
|
||||
|
||||
@@ -28,41 +28,41 @@ class UyusmazlikKararSonucuEnum(str, Enum): # Based on checkbox text in the form
|
||||
|
||||
class UyusmazlikSearchRequest(BaseModel): # This is the model the MCP tool will accept
|
||||
"""Model for Uyuşmazlık Mahkemesi search request using user-friendly terms."""
|
||||
icerik: Optional[str] = Field("", description="Keyword or content for main text search (Icerik).")
|
||||
icerik: Optional[str] = Field("", description="Search text")
|
||||
|
||||
bolum: Optional[UyusmazlikBolumEnum] = Field(
|
||||
UyusmazlikBolumEnum.TUMU,
|
||||
description="Select the department (Bölüm)."
|
||||
description="Department"
|
||||
)
|
||||
uyusmazlik_turu: Optional[UyusmazlikTuruEnum] = Field(
|
||||
UyusmazlikTuruEnum.TUMU,
|
||||
description="Select the type of dispute (Uyuşmazlık)."
|
||||
description="Dispute type"
|
||||
)
|
||||
|
||||
# User provides a list of user-friendly names for Karar Sonucu
|
||||
karar_sonuclari: Optional[List[UyusmazlikKararSonucuEnum]] = Field( # Changed to list of Enums
|
||||
default_factory=list,
|
||||
description="List of desired 'Karar Sonucu' types."
|
||||
description="Decision types"
|
||||
)
|
||||
|
||||
esas_yil: Optional[str] = Field("", description="Case year ('Esas Yılı').")
|
||||
esas_sayisi: Optional[str] = Field("", description="Case number ('Esas Sayısı').")
|
||||
karar_yil: Optional[str] = Field("", description="Decision year ('Karar Yılı').")
|
||||
karar_sayisi: Optional[str] = Field("", description="Decision number ('Karar Sayısı').")
|
||||
kanun_no: Optional[str] = Field("", description="Relevant Law Number ('KanunNo').")
|
||||
esas_yil: Optional[str] = Field("", description="Case year")
|
||||
esas_sayisi: Optional[str] = Field("", description="Case no")
|
||||
karar_yil: Optional[str] = Field("", description="Decision year")
|
||||
karar_sayisi: Optional[str] = Field("", description="Decision no")
|
||||
kanun_no: Optional[str] = Field("", description="Law no")
|
||||
|
||||
karar_date_begin: Optional[str] = Field("", description="Decision start date (DD.MM.YYYY) ('KararDateBegin').")
|
||||
karar_date_end: Optional[str] = Field("", description="Decision end date (DD.MM.YYYY) ('KararDateEnd').")
|
||||
karar_date_begin: Optional[str] = Field("", description="Start date (DD.MM.YYYY)")
|
||||
karar_date_end: Optional[str] = Field("", description="End date (DD.MM.YYYY)")
|
||||
|
||||
resmi_gazete_sayi: Optional[str] = Field("", description="Official Gazette number ('ResmiGazeteSayi').")
|
||||
resmi_gazete_date: Optional[str] = Field("", description="Official Gazette date (DD.MM.YYYY) ('ResmiGazeteDate').")
|
||||
resmi_gazete_sayi: Optional[str] = Field("", description="Gazette no")
|
||||
resmi_gazete_date: Optional[str] = Field("", description="Gazette date (DD.MM.YYYY)")
|
||||
|
||||
# Detailed text search fields from the "icerikDetail" section of the form
|
||||
tumce: Optional[str] = Field("", description="Exact phrase search ('Tumce').")
|
||||
wild_card: Optional[str] = Field("", description="Search for phrase and its inflections ('WildCard').") # Changed from WildCard for Pythonic name
|
||||
hepsi: Optional[str] = Field("", description="Search for texts containing all specified words ('Hepsi').")
|
||||
herhangi_birisi: Optional[str] = Field("", description="Search for texts containing any of the specified words ('Herhangibirisi').")
|
||||
not_hepsi: Optional[str] = Field("", description="Exclude texts containing these specified words ('NotHepsi').")
|
||||
tumce: Optional[str] = Field("", description="Exact phrase")
|
||||
wild_card: Optional[str] = Field("", description="Wildcard search")
|
||||
hepsi: Optional[str] = Field("", description="All words")
|
||||
herhangi_birisi: Optional[str] = Field("", description="Any word")
|
||||
not_hepsi: Optional[str] = Field("", description="Exclude words")
|
||||
|
||||
class UyusmazlikApiDecisionEntry(BaseModel):
|
||||
"""Model for an individual decision entry parsed from Uyuşmazlık API's HTML search response."""
|
||||
@@ -71,9 +71,9 @@ class UyusmazlikApiDecisionEntry(BaseModel):
|
||||
bolum: Optional[str] = Field(None)
|
||||
uyusmazlik_konusu: Optional[str] = Field(None)
|
||||
karar_sonucu: Optional[str] = Field(None)
|
||||
popover_content: Optional[str] = Field(None, description="Summary/description from popover.")
|
||||
popover_content: Optional[str] = Field(None, description="Summary")
|
||||
document_url: HttpUrl # Full URL to the decision document HTML page
|
||||
pdf_url: Optional[HttpUrl] = Field(None, description="Direct URL to PDF if available.")
|
||||
pdf_url: Optional[HttpUrl] = Field(None, description="PDF URL")
|
||||
|
||||
class UyusmazlikSearchResponse(BaseModel): # This is what the MCP tool will return
|
||||
"""Response model for Uyuşmazlık Mahkemesi search results for the MCP tool."""
|
||||
|
||||
@@ -6,8 +6,7 @@ from typing import Dict, Any, List, Optional
|
||||
import logging
|
||||
import html
|
||||
import re
|
||||
import tempfile
|
||||
import os
|
||||
import io
|
||||
from markitdown import MarkItDown
|
||||
|
||||
from .models import (
|
||||
@@ -66,6 +65,19 @@ class YargitayOfficialApiClient:
|
||||
response.raise_for_status() # Raise an exception for HTTP 4xx or 5xx status codes
|
||||
response_json_data = response.json()
|
||||
|
||||
logger.debug(f"YargitayOfficialApiClient: Raw API response: {response_json_data}")
|
||||
|
||||
# Handle None or empty data response from API
|
||||
if response_json_data is None:
|
||||
logger.warning("YargitayOfficialApiClient: API returned None response")
|
||||
response_json_data = {"data": {"data": [], "recordsTotal": 0, "recordsFiltered": 0}}
|
||||
elif not isinstance(response_json_data, dict):
|
||||
logger.warning(f"YargitayOfficialApiClient: API returned unexpected response type: {type(response_json_data)}")
|
||||
response_json_data = {"data": {"data": [], "recordsTotal": 0, "recordsFiltered": 0}}
|
||||
elif response_json_data.get("data") is None:
|
||||
logger.warning("YargitayOfficialApiClient: API response data field is None")
|
||||
response_json_data["data"] = {"data": [], "recordsTotal": 0, "recordsFiltered": 0}
|
||||
|
||||
# Validate and parse the response using Pydantic models
|
||||
api_response = YargitayApiSearchResponse(**response_json_data)
|
||||
|
||||
@@ -108,25 +120,20 @@ class YargitayOfficialApiClient:
|
||||
html_to_convert = processed_html
|
||||
|
||||
markdown_output = None
|
||||
temp_file_path = None
|
||||
try:
|
||||
md_converter = MarkItDown() # Plugins disabled as per basic usage
|
||||
# Convert HTML string to bytes and create BytesIO stream
|
||||
html_bytes = html_to_convert.encode('utf-8')
|
||||
html_stream = io.BytesIO(html_bytes)
|
||||
|
||||
# Write the HTML to a temporary file for MarkItDown to process
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_html_file:
|
||||
tmp_html_file.write(html_to_convert)
|
||||
temp_file_path = tmp_html_file.name
|
||||
|
||||
conversion_result = md_converter.convert(temp_file_path)
|
||||
# Pass BytesIO stream to MarkItDown to avoid temp file creation
|
||||
md_converter = MarkItDown()
|
||||
conversion_result = md_converter.convert(html_stream)
|
||||
markdown_output = conversion_result.text_content
|
||||
|
||||
logger.info("Successfully converted HTML to Markdown.")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error during MarkItDown HTML to Markdown conversion: {e}")
|
||||
finally:
|
||||
if temp_file_path and os.path.exists(temp_file_path):
|
||||
os.remove(temp_file_path) # Clean up the temporary file
|
||||
|
||||
return markdown_output
|
||||
|
||||
|
||||
@@ -34,111 +34,74 @@ class YargitayDetailedSearchRequest(BaseModel):
|
||||
to Yargitay's detailed search endpoint (e.g., /aramadetaylist).
|
||||
Based on the payload provided by the user.
|
||||
"""
|
||||
arananKelime: Optional[str] = Field("", description="""Keyword to search for with advanced operators support:
|
||||
• Simple words: 'arsa payı' (OR logic - finds documents with ANY word)
|
||||
• Exact phrases: '"arsa payı"' (finds exact phrase)
|
||||
• AND logic: 'arsa+payı' (both words required)
|
||||
• Wildcards: 'bozma*' (matches bozma, bozması, bozmanın, etc.)
|
||||
• Multiple required: '+"arsa payı" +"bozma sebebi"'
|
||||
• Exclusion: '+"arsa payı" -"inşaat sözleşmesi"'
|
||||
Examples: arsa payı | "arsa payı" | +"mülkiyet hakkı" +"bozma sebebi" | hukuk*""")
|
||||
arananKelime: Optional[str] = Field("", description="Turkish keywords (supports +word -word \"phrase\" operators)")
|
||||
# Department/Board selection - Complete Court of Cassation chamber hierarchy
|
||||
birimYrgKurulDaire: Optional[str] = Field("ALL", description="""
|
||||
Court of Cassation (Yargıtay) chamber/board selection. Options include:
|
||||
- 'ALL' for all chambers
|
||||
- Civil: 'Civil General Assembly (Hukuk Genel Kurulu)', '1st Civil Chamber (1. Hukuk Dairesi)' through '23rd Civil Chamber (23. Hukuk Dairesi)', 'Civil Chambers Presidents Board (Hukuk Daireleri Başkanlar Kurulu)'
|
||||
- Criminal: 'Criminal General Assembly (Ceza Genel Kurulu)', '1st Criminal Chamber (1. Ceza Dairesi)' through '23rd Criminal Chamber (23. Ceza Dairesi)', 'Criminal Chambers Presidents Board (Ceza Daireleri Başkanlar Kurulu)'
|
||||
- General: 'Grand General Assembly (Büyük Genel Kurulu)'
|
||||
Total: 52 possible values (including 'ALL' for all chambers)
|
||||
""")
|
||||
birimYrgHukukDaire: Optional[str] = Field("", description="Legacy field - use birimYrgKurulDaire instead for chamber selection")
|
||||
birimYrgCezaDaire: Optional[str] = Field("", description="Legacy field - use birimYrgKurulDaire instead for chamber selection")
|
||||
birimYrgKurulDaire: Optional[str] = Field("ALL", description="Chamber (ALL or specific chamber name)")
|
||||
birimYrgHukukDaire: Optional[str] = Field("", description="Legacy field")
|
||||
birimYrgCezaDaire: Optional[str] = Field("", description="Legacy field")
|
||||
|
||||
esasYil: Optional[str] = Field("", description="""Case year for 'Esas No' filtering.
|
||||
Format: YYYY (e.g., '2024')
|
||||
Use with sequence numbers for precise case targeting""")
|
||||
esasIlkSiraNo: Optional[str] = Field("", description="""Starting sequence number for 'Esas No' range filtering.
|
||||
Format: numeric string (e.g., '1', '100')
|
||||
Use with esasSonSiraNo for range: cases 100-200 in specified year""")
|
||||
esasSonSiraNo: Optional[str] = Field("", description="""Ending sequence number for 'Esas No' range filtering.
|
||||
Format: numeric string (e.g., '500', '1000')
|
||||
Creates range from esasIlkSiraNo to this number""")
|
||||
esasYil: Optional[str] = Field("", description="Case year (YYYY)")
|
||||
esasIlkSiraNo: Optional[str] = Field("", description="Start case no")
|
||||
esasSonSiraNo: Optional[str] = Field("", description="End case no")
|
||||
|
||||
kararYil: Optional[str] = Field("", description="""Decision year for 'Karar No' filtering.
|
||||
Format: YYYY (e.g., '2024')
|
||||
Filters decisions by the year they were issued""")
|
||||
kararIlkSiraNo: Optional[str] = Field("", description="""Starting sequence number for 'Karar No' range filtering.
|
||||
Format: numeric string (e.g., '1', '50')
|
||||
Use with kararSonSiraNo for decision number ranges""")
|
||||
kararSonSiraNo: Optional[str] = Field("", description="""Ending sequence number for 'Karar No' range filtering.
|
||||
Format: numeric string (e.g., '100', '500')
|
||||
Creates range from kararIlkSiraNo to this number""")
|
||||
kararYil: Optional[str] = Field("", description="Decision year (YYYY)")
|
||||
kararIlkSiraNo: Optional[str] = Field("", description="Start decision no")
|
||||
kararSonSiraNo: Optional[str] = Field("", description="End decision no")
|
||||
|
||||
baslangicTarihi: Optional[str] = Field("", description="""Start date for decision search.
|
||||
Format: DD.MM.YYYY (e.g., '01.01.2024')
|
||||
Use with bitisTarihi for date range filtering
|
||||
Examples: '01.01.2024', '15.06.2023'""")
|
||||
bitisTarihi: Optional[str] = Field("", description="""End date for decision search.
|
||||
Format: DD.MM.YYYY (e.g., '31.12.2024')
|
||||
Creates date range from baslangicTarihi to this date
|
||||
Examples: '31.12.2024', '30.06.2023'""")
|
||||
baslangicTarihi: Optional[str] = Field("", description="Start date (DD.MM.YYYY)")
|
||||
bitisTarihi: Optional[str] = Field("", description="End date (DD.MM.YYYY)")
|
||||
|
||||
siralama: Optional[str] = Field("3", description="""Sorting criteria for search results:
|
||||
• '1': Esas No (Case Number) - sorts by case registration order
|
||||
• '2': Karar No (Decision Number) - sorts by decision issuance order
|
||||
• '3': Karar Tarihi (Decision Date) - sorts by chronological order [DEFAULT]
|
||||
Recommended: Use '3' for most recent decisions first""")
|
||||
siralamaDirection: Optional[str] = Field("desc", description="""Sorting direction for results:
|
||||
• 'desc': Descending order (newest/highest first) [DEFAULT]
|
||||
• 'asc': Ascending order (oldest/lowest first)
|
||||
Most common: 'desc' for latest decisions first""")
|
||||
siralama: Optional[str] = Field("3", description="Sort by (1=case, 2=decision, 3=date)")
|
||||
siralamaDirection: Optional[str] = Field("desc", description="Direction (asc/desc)")
|
||||
|
||||
pageSize: int = Field(10, ge=1, le=100, description="""Number of results per page.
|
||||
Range: 1-100 results per page
|
||||
Recommended: 10-50 for balanced performance and coverage
|
||||
Large values (50-100) for comprehensive analysis""")
|
||||
pageNumber: int = Field(1, ge=1, description="""Page number to retrieve (1-indexed).
|
||||
Start with 1 for first page
|
||||
Use with pageSize to navigate through large result sets
|
||||
Example: pageSize=50, pageNumber=3 gets results 101-150""")
|
||||
pageSize: int = Field(10, ge=1, le=10, description="Results per page (1-100)")
|
||||
pageNumber: int = Field(1, ge=1, description="Page number (1-indexed)")
|
||||
|
||||
class YargitayApiDecisionEntry(BaseModel):
|
||||
"""Model for an individual decision entry from the Yargitay API search response."""
|
||||
id: str # Unique system ID of the decision
|
||||
daire: Optional[str] = Field(None, description="The chamber (Daire) that made the decision.")
|
||||
esasNo: Optional[str] = Field(None, alias="esasNo", description="Case registry number (Esas No).")
|
||||
kararNo: Optional[str] = Field(None, alias="kararNo", description="Decision number (Karar No).")
|
||||
kararTarihi: Optional[str] = Field(None, alias="kararTarihi", description="Date of the decision (Karar Tarihi).")
|
||||
arananKelime: Optional[str] = Field(None, alias="arananKelime", description="Matched keyword (Aranan Kelime) in the search result item.")
|
||||
daire: Optional[str] = Field(None, description="Chamber")
|
||||
esasNo: Optional[str] = Field(None, alias="esasNo", description="Case no")
|
||||
kararNo: Optional[str] = Field(None, alias="kararNo", description="Decision no")
|
||||
kararTarihi: Optional[str] = Field(None, alias="kararTarihi", description="Date")
|
||||
# 'index' and 'siraNo' from API response are not critical for MCP tool, so omitted for brevity
|
||||
|
||||
# This field will be populated by the client after fetching the search list
|
||||
document_url: Optional[HttpUrl] = Field(None, description="Direct URL (Belge URL) to the decision document.")
|
||||
document_url: Optional[HttpUrl] = Field(None, description="Document URL")
|
||||
|
||||
model_config = ConfigDict(populate_by_name=True) # To allow populating by alias from API response
|
||||
|
||||
|
||||
class YargitayApiResponseInnerData(BaseModel):
|
||||
"""Model for the inner 'data' object in the Yargitay API search response."""
|
||||
data: List[YargitayApiDecisionEntry]
|
||||
data: List[YargitayApiDecisionEntry] = Field(default_factory=list)
|
||||
# draw: Optional[int] = None # Typically used by DataTables, not essential for MCP
|
||||
recordsTotal: int # Total number of records matching the query
|
||||
recordsFiltered: int # Total number of records after filtering (usually same as recordsTotal)
|
||||
recordsTotal: int = Field(default=0) # Total number of records matching the query
|
||||
recordsFiltered: int = Field(default=0) # Total number of records after filtering (usually same as recordsTotal)
|
||||
|
||||
class YargitayApiSearchResponse(BaseModel):
|
||||
"""Model for the complete search response from the Yargitay API."""
|
||||
data: YargitayApiResponseInnerData
|
||||
data: Optional[YargitayApiResponseInnerData] = Field(default_factory=lambda: YargitayApiResponseInnerData())
|
||||
# metadata: Optional[Dict[str, Any]] = None # Optional metadata from API
|
||||
|
||||
class YargitayDocumentMarkdown(BaseModel):
|
||||
"""Model for a Yargitay decision document, containing only Markdown content."""
|
||||
id: str = Field(..., description="The unique ID (Belge Kimliği) of the document.")
|
||||
markdown_content: Optional[str] = Field(None, description="The decision content (Karar İçeriği) converted to Markdown.")
|
||||
source_url: HttpUrl = Field(..., description="The source URL (Kaynak URL) of the original document.")
|
||||
id: str = Field(..., description="Document ID")
|
||||
markdown_content: Optional[str] = Field(None, description="Content")
|
||||
source_url: HttpUrl = Field(..., description="Source URL")
|
||||
|
||||
class CleanYargitayDecisionEntry(BaseModel):
|
||||
"""Clean decision entry without arananKelime field to reduce token usage."""
|
||||
id: str
|
||||
daire: Optional[str] = Field(None, description="Chamber")
|
||||
esasNo: Optional[str] = Field(None, description="Case no")
|
||||
kararNo: Optional[str] = Field(None, description="Decision no")
|
||||
kararTarihi: Optional[str] = Field(None, description="Date")
|
||||
document_url: Optional[HttpUrl] = Field(None, description="Document URL")
|
||||
|
||||
class CompactYargitaySearchResult(BaseModel):
|
||||
"""A more compact search result model for the MCP tool to return."""
|
||||
decisions: List[YargitayApiDecisionEntry]
|
||||
decisions: List[CleanYargitayDecisionEntry]
|
||||
total_records: int
|
||||
requested_page: int
|
||||
page_size: int
|
||||
Reference in New Issue
Block a user