81 Commits
Author SHA1 Message Date
saidsurucu ec40b9d6a2 Add BDDK module and prepare v0.1.5 release 2025-07-19 17:20:38 +03:00
saidsurucu daa16cae99 Update README.md 2025-07-19 17:15:57 +03:00
saidsurucu c3bc9e17eb add bddk module 2025-07-19 17:11:33 +03:00
saidsurucu c9344fc538 Update README.md 2025-07-18 12:39:22 +03:00
saidsurucu d12ad7d900 Update mcp_server_main.py 2025-07-18 10:46:13 +03:00
saidsurucu 891751043c Update mcp_server_main.py 2025-07-18 10:39:59 +03:00
saidsurucu ba447502fe fix empty string 2025-07-18 10:22:58 +03:00
saidsurucu c092a7af45 fix bedesten 2025-07-18 09:56:22 +03:00
saidsurucu 34216a9557 Update mcp_server_main.py 2025-07-17 23:52:43 +03:00
saidsurucu 753283f0e8 shorten enum schema 2025-07-17 23:23:32 +03:00
saidsurucu 611456fd49 compress bedesten enum 2025-07-17 22:43:55 +03:00
saidsurucu 7a1ff0b9ed shorten sayıştay enums 2025-07-17 22:14:54 +03:00
saidsurucu 95620285d9 Update mcp_server_main.py 2025-07-17 21:27:12 +03:00
saidsurucu 8a148899e3 convert enum sayıştay 2025-07-17 21:20:23 +03:00
saidsurucu fa6c448afa convert enums to literal 2025-07-17 21:14:03 +03:00
saidsurucu cc363e7a6d unify sayıştay tools 2025-07-17 20:44:09 +03:00
saidsurucu f96c1a2e44 unify anayasa tools 2025-07-17 20:09:50 +03:00
saidsurucu 856ecdf13d Update mcp_server_main.py 2025-07-17 00:49:39 +03:00
saidsurucu f7dac9363a fix model issue 2025-07-16 23:34:41 +03:00
saidsurucu b32aabf541 Update mcp_server_main.py 2025-07-16 12:39:44 +03:00
saidsurucu 945ffe6267 Update mcp_server_main.py 2025-07-16 12:16:56 +03:00
saidsurucu 17f9b109a2 Update mcp_server_main.py 2025-07-15 14:11:24 +03:00
saidsurucu 2a24ce03ad Update client.py 2025-07-14 16:13:56 +03:00
saidsurucu e34d81be26 optimize token usage 2025-07-14 15:45:19 +03:00
saidsurucu 54e8d61f83 Sürüm v0.1.4: FastMCP 2.10.5 güncellemesi ve optimizasyonlar
- FastMCP bağımlılığı 2.10.5'e güncellendi
- Bedesten araçları birleştirildi (10 araç → 2 birleşik araç)
- KVKK dokümantasyonu optimize edildi (147 → 27 satır, %82 azalma)
- README 30 MCP aracının güncel listesi ile güncellendi
- Eski tekil Bedesten API araçları kaldırıldı
- Araç açıklamaları ve Türkçe örnekler iyileştirildi
- Resource dokümantasyon verimliliği artırıldı
2025-07-14 00:39:46 +03:00
saidsurucu 35382136c5 Update mcp_server_main.py 2025-07-14 00:19:27 +03:00
saidsurucu 59d81d4b03 Update mcp_server_main.py 2025-07-14 00:10:19 +03:00
saidsurucu 06a319c0ee Update README.md 2025-07-13 23:58:51 +03:00
saidsurucu a48b003121 unify bedesten tools 2025-07-13 23:34:34 +03:00
saidsurucu 1620d7a9b0 Update mcp_server_main.py 2025-07-13 22:51:35 +03:00
saidsurucu c10069bcc7 Update mcp_server_main.py 2025-07-13 22:46:30 +03:00
saidsurucu c0fe7e1305 Update mcp_server_main.py 2025-07-13 22:44:21 +03:00
saidsurucu f492109eb7 add health check tool 2025-07-13 22:31:34 +03:00
saidsurucu 35f3b739d4 Update mcp_server_main.py 2025-07-13 22:17:36 +03:00
saidsurucu 8f1ca6b854 Update mcp_server_main.py 2025-07-13 22:16:26 +03:00
saidsurucu 71218996fd optimize token usage 2025-07-13 22:11:35 +03:00
saidsurucu 6bac51dc19 token optimization 2025-07-13 22:00:35 +03:00
saidsurucu b898cad4f4 optimize descriptions 2025-07-13 18:26:16 +03:00
saidsurucu 3d172c508a Update mcp_server_main.py 2025-07-13 18:12:23 +03:00
saidsurucu 48efe74dc0 Update mcp_server_main.py 2025-07-13 17:59:28 +03:00
saidsurucu 8ae5772c2c Update mcp_server_main.py 2025-07-13 17:26:05 +03:00
saidsurucu 87cf4fd46d update docker, bugfix 2025-07-12 15:25:18 +03:00
saidsurucu d590702272 Bump version to 0.1.3
- Add auto-install Playwright browsers functionality
- Fix KIK tool page_number parameter descriptions
- Add 'accepts int' notes to all page_number parameters
- Improve PyPI deployment experience
2025-07-12 11:41:35 +03:00
saidsurucu 1ebe8847fb fix kik 2025-07-12 11:40:11 +03:00
saidsurucu cb318faeba add kvkk module, several bug fix 2025-07-11 23:50:56 +03:00
saidsurucu 83f54a86a8 Update mcp_server_main.py 2025-07-11 21:22:13 +03:00
saidsurucu f5f0f99678 add redis 2025-07-11 21:19:16 +03:00
saidsurucu b0d7151ba1 OAuth callback JWT token fix 2025-07-09 22:59:30 +03:00
saidsurucu de9e337163 fix auth 2025-07-09 22:37:26 +03:00
saidsurucu a7ebb8b27e fix auth 2025-07-09 21:36:05 +03:00
saidsurucu 01e58ab5ed Update mcp_auth_http_simple.py 2025-07-09 20:47:26 +03:00
saidsurucu 7081bbbd91 fix auth 2025-07-09 20:34:36 +03:00
saidsurucu ca89c9480d fix auth 2025-07-09 20:16:10 +03:00
saidsurucu b751e94847 Update mcp_server_main.py 2025-07-09 19:58:07 +03:00
saidsurucu 3ccb52e719 fix auth 2025-07-09 19:55:09 +03:00
saidsurucu e8b92e347c Update asgi_app.py 2025-07-09 19:44:31 +03:00
saidsurucu e65b42abc8 Update asgi_app.py 2025-07-09 19:26:24 +03:00
saidsurucu 71096ae67d Update asgi_app.py 2025-07-09 19:25:46 +03:00
saidsurucu 9fb23dadae Update asgi_app.py 2025-07-09 19:25:21 +03:00
saidsurucu d77781709a Update asgi_app.py 2025-07-09 19:15:19 +03:00
saidsurucu c059ec30a7 Update asgi_app.py 2025-07-09 19:09:37 +03:00
saidsurucu 6842ad207d Update asgi_app.py 2025-07-09 18:52:52 +03:00
saidsurucu bbc56d9409 Update asgi_app.py 2025-07-09 18:48:44 +03:00
saidsurucu 66b5e2631e Update asgi_app.py 2025-07-09 17:55:45 +03:00
saidsurucu a17853b918 Update asgi_app.py 2025-07-09 17:44:34 +03:00
saidsurucu 0a302bb13b Update asgi_app.py 2025-07-09 17:35:12 +03:00
saidsurucu 28e9a47464 Update asgi_app.py 2025-07-09 17:27:30 +03:00
saidsurucu 6c1efece31 Update mcp_server_main.py 2025-07-09 17:15:06 +03:00
saidsurucu eb9441a6f3 Update mcp_server_main.py 2025-07-09 16:27:14 +03:00
saidsurucu d006dc8a55 Update asgi_app.py 2025-07-09 15:44:09 +03:00
saidsurucu 55cee6933d upgrade fastmcp 2025-07-09 15:38:04 +03:00
saidsurucu 2f375d74e5 Update asgi_app.py 2025-07-09 14:54:39 +03:00
saidsurucu 82856b25e9 Update asgi_app.py 2025-07-09 14:24:31 +03:00
saidsurucu 8464aedceb Update asgi_app.py 2025-07-09 13:46:04 +03:00
saidsurucu 64c3a2c138 Update asgi_app.py 2025-07-08 23:52:08 +03:00
saidsurucu 318bedd4c5 add jwt support 2025-07-08 23:18:44 +03:00
saidsurucu 91895f6c1a Update asgi_app.py 2025-07-08 22:46:58 +03:00
saidsurucu b86c18c842 update auth logic 2025-07-08 22:34:02 +03:00
saidsurucu 5509380cef Rename fastapi_app.py to example_fastapi_app.py 2025-07-03 17:54:54 +03:00
saidsurucu 1c818756e4 Update README.md 2025-07-03 12:53:38 +03:00
saidsurucu 34ac65dc02 Update README.md 2025-07-03 12:52:40 +03:00
40 changed files with 5983 additions and 2673 deletions
+3
View File
@@ -56,6 +56,9 @@ LOG_LEVEL=info
# Base URL for the application (used for OAuth callbacks and API URLs) # Base URL for the application (used for OAuth callbacks and API URLs)
BASE_URL=http://localhost:8000 BASE_URL=http://localhost:8000
# JWT Secret for MCP token generation
JWT_SECRET_KEY=your_jwt_secret_key_here
# ============================================================================= # =============================================================================
# MCP SERVER SETTINGS # MCP SERVER SETTINGS
# ============================================================================= # =============================================================================
+23
View File
@@ -190,3 +190,26 @@ GEMINI.md
fly.toml fly.toml
scripts/deploy-flyio.sh scripts/deploy-flyio.sh
docs/DEPLOYMENT_FLYIO.md docs/DEPLOYMENT_FLYIO.md
setup_jwt_template.py
mcp_server_main.py.backup
mcp_overhead_content.json
ANTHROPIC_TEST_README.md
extract_mcp_overhead.py
mcp_overhead_content.txt
mcp_overhead_summary.txt
run_http_server.py
run_local_test.py
# MCP overhead analysis files
mcp_overhead_*.json
mcp_overhead_*.txt
mcp_test_results_*.json
mcp_quick_test_*.json
# General text files (temporary notes, etc)
*.txt
analyze_playwright_mcp.py
measure_mcp_directly.py
playwright_mcp_overhead.json
simple_test.py
analyze_anayasa_html.py
+1 -1
View File
@@ -1,5 +1,5 @@
# -------- BASE IMAGE (includes Chromium & deps) ---------------------------- # -------- BASE IMAGE (includes Chromium & deps) ----------------------------
FROM mcr.microsoft.com/playwright/python:v1.52.0-noble FROM mcr.microsoft.com/playwright/python:v1.53.0-noble
# -------- Runtime setup ---------------------------------------------------- # -------- Runtime setup ----------------------------------------------------
WORKDIR /app WORKDIR /app
+77 -58
View File
@@ -2,12 +2,14 @@
[![Star History Chart](https://api.star-history.com/svg?repos=saidsurucu/yargi-mcp&type=Date)](https://www.star-history.com/#saidsurucu/yargi-mcp&Date) [![Star History Chart](https://api.star-history.com/svg?repos=saidsurucu/yargi-mcp&type=Date)](https://www.star-history.com/#saidsurucu/yargi-mcp&Date)
Bu proje, çeşitli Türk hukuk kaynaklarına (Yargıtay, Danıştay, Emsal Kararlar, Uyuşmazlık Mahkemesi, Anayasa Mahkemesi - Norm Denetimi ile Bireysel Başvuru Kararları, Kamu İhale Kurulu Kararları, Rekabet Kurumu Kararları ve Sayıştay Kararları) erişimi kolaylaştıran bir [FastMCP](https://gofastmcp.com/) sunucusu oluşturur. Bu sayede, bu kaynaklardan veri arama ve belge getirme işlemleri, Model Context Protocol (MCP) destekleyen LLM (Büyük Dil Modeli) uygulamaları (örneğin Claude Desktop veya [5ire](https://5ire.app)) ve diğer istemciler tarafından araç (tool) olarak kullanılabilir hale gelir. Bu proje, çeşitli Türk hukuk kaynaklarına (Yargıtay, Danıştay, Emsal Kararlar, Uyuşmazlık Mahkemesi, Anayasa Mahkemesi - Norm Denetimi ile Bireysel Başvuru Kararları, Kamu İhale Kurulu Kararları, Rekabet Kurumu Kararları, Sayıştay Kararları, KVKK Kararları ve BDDK Kararları) erişimi kolaylaştıran bir [FastMCP](https://gofastmcp.com/) sunucusu oluşturur. Bu sayede, bu kaynaklardan veri arama ve belge getirme işlemleri, Model Context Protocol (MCP) destekleyen LLM (Büyük Dil Modeli) uygulamaları (örneğin Claude Desktop veya [5ire](https://5ire.app)) ve diğer istemciler tarafından araç (tool) olarak kullanılabilir hale gelir.
![örnek](./ornek.png) ![örnek](./ornek.png)
🎯 **Temel Özellikler** 🎯 **Temel Özellikler**
🚀 **YÜKSEK PERFORMANS OPTİMİZASYONU:** Bu MCP sunucusu **%61.8 token azaltma** ile optimize edilmiştir (8,692 token tasarrufu). Claude AI ile daha hızlı yanıt süreleri ve daha verimli etkileşim sağlar.
* Çeşitli Türk hukuk veritabanlarına programatik erişim için standart bir MCP arayüzü. * Çeşitli Türk hukuk veritabanlarına programatik erişim için standart bir MCP arayüzü.
* **Kapsamlı Mahkeme Daire/Kurul Filtreleme:** 79 farklı daire/kurul filtreleme seçeneği * **Kapsamlı Mahkeme Daire/Kurul Filtreleme:** 79 farklı daire/kurul filtreleme seçeneği
* **Dual/Triple API Desteği:** Her mahkeme için birden fazla API kaynağı ile maksimum kapsama * **Dual/Triple API Desteği:** Her mahkeme için birden fazla API kaynağı ile maksimum kapsama
@@ -26,12 +28,15 @@ Bu proje, çeşitli Türk hukuk kaynaklarına (Yargıtay, Danıştay, Emsal Kara
* **KİK (Kamu İhale Kurulu):** Çeşitli kriterlerle Kurul kararlarını arama; uzun karar metinlerini (varsayılan 5.000 karakterlik) sayfalanmış Markdown formatında getirme. * **KİK (Kamu İhale Kurulu):** Çeşitli kriterlerle Kurul kararlarını arama; uzun karar metinlerini (varsayılan 5.000 karakterlik) sayfalanmış Markdown formatında getirme.
* **Rekabet Kurumu:** Çeşitli kriterlerle Kurul kararlarını arama; karar metinlerini Markdown formatında getirme. * **Rekabet Kurumu:** Çeşitli kriterlerle Kurul kararlarını arama; karar metinlerini Markdown formatında getirme.
* **Sayıştay:** 3 karar türü ile kapsamlı denetim kararlarına erişim + **8 Daire Filtreleme** + **Tarih Aralığı & İçerik Arama** (Genel Kurul yorumlayıcı kararları, Temyiz Kurulu itiraz kararları, Daire ilk derece denetim kararları) * **Sayıştay:** 3 karar türü ile kapsamlı denetim kararlarına erişim + **8 Daire Filtreleme** + **Tarih Aralığı & İçerik Arama** (Genel Kurul yorumlayıcı kararları, Temyiz Kurulu itiraz kararları, Daire ilk derece denetim kararları)
* **KVKK (Kişisel Verilerin Korunması Kurulu):** Brave Search API ile veri koruma kararlarını arama; uzun karar metinlerini (5.000 karakterlik) sayfalanmış Markdown formatında getirme + **Türkçe Arama** + **Site Hedeflemeli Arama** (kvkk.gov.tr kararları)
* **BDDK (Bankacılık Düzenleme ve Denetleme Kurumu):** Bankacılık düzenleme kararlarını arama; karar metinlerini Markdown formatında getirme + **Optimized Search** + **"Karar Sayısı" Targeting** + **Spesifik URL Filtreleme** (bddk.org.tr/Mevzuat/DokumanGetir)
* Karar metinlerinin daha kolay işlenebilmesi için Markdown formatına çevrilmesi. * Karar metinlerinin daha kolay işlenebilmesi için Markdown formatına çevrilmesi.
* Claude Desktop uygulaması ile `fastmcp install` komutu kullanılarak kolay entegrasyon. * Claude Desktop uygulaması ile `fastmcp install` komutu kullanılarak kolay entegrasyon.
* Yargı MCP artık [5ire](https://5ire.app) gibi Claude Desktop haricindeki MCP istemcilerini de destekliyor! * Yargı MCP artık [5ire](https://5ire.app) gibi Claude Desktop haricindeki MCP istemcilerini de destekliyor!
--- ---
🚀 **Claude Haricindeki Modellerle Kullanmak İçin Çok Kolay Kurulum (Örnek: 5ire için)** <details>
<summary>🚀 <strong>Claude Haricindeki Modellerle Kullanmak İçin Çok Kolay Kurulum (Örnek: 5ire için)</strong></summary>
Bu bölüm, Yargı MCP aracını 5ire gibi Claude Desktop dışındaki MCP istemcileriyle kullanmak isteyenler içindir. Bu bölüm, Yargı MCP aracını 5ire gibi Claude Desktop dışındaki MCP istemcileriyle kullanmak isteyenler içindir.
@@ -48,16 +53,18 @@ Bu bölüm, Yargı MCP aracını 5ire gibi Claude Desktop dışındaki MCP istem
* **Name:** `Yargı MCP` * **Name:** `Yargı MCP`
* **Command:** * **Command:**
``` ```
uvx --from git+https://github.com/saidsurucu/yargi-mcp yargi-mcp uvx yargi-mcp
``` ```
* **Save** butonuna basarak kaydedin. * **Save** butonuna basarak kaydedin.
![5ire ayarları](./5ire-settings.png) ![5ire ayarları](./5ire-settings.png)
* Şimdi **Tools** altında **Yargı MCP**'yi görüyor olmalısınız. Üstüne geldiğinizde sağda çıkan butona tıklayıp etkinleştirin (yeşil ışık yanmalı). * Şimdi **Tools** altında **Yargı MCP**'yi görüyor olmalısınız. Üstüne geldiğinizde sağda çıkan butona tıklayıp etkinleştirin (yeşil ışık yanmalı).
* Artık Yargı MCP ile konuşabilirsiniz. * Artık Yargı MCP ile konuşabilirsiniz.
--- </details>
⚙️ **Claude Desktop Manuel Kurulumu**
---
<details>
<summary>⚙️ <strong>Claude Desktop Manuel Kurulumu</strong></summary>
1. **Ön Gereksinimler:** Python, `uv`, (Windows için) Microsoft Visual C++ Redistributable'ın sisteminizde kurulu olduğundan emin olun. Detaylı bilgi için yukarıdaki "5ire için Kurulum" bölümündeki ilgili adımlara bakabilirsiniz. 1. **Ön Gereksinimler:** Python, `uv`, (Windows için) Microsoft Visual C++ Redistributable'ın sisteminizde kurulu olduğundan emin olun. Detaylı bilgi için yukarıdaki "5ire için Kurulum" bölümündeki ilgili adımlara bakabilirsiniz.
2. Claude Desktop **Settings -> Developer -> Edit Config**. 2. Claude Desktop **Settings -> Developer -> Edit Config**.
@@ -70,7 +77,6 @@ Bu bölüm, Yargı MCP aracını 5ire gibi Claude Desktop dışındaki MCP istem
"Yargı MCP": { "Yargı MCP": {
"command": "uvx", "command": "uvx",
"args": [ "args": [
"--from", "git+https://github.com/saidsurucu/yargi-mcp",
"yargi-mcp" "yargi-mcp"
] ]
} }
@@ -79,8 +85,11 @@ Bu bölüm, Yargı MCP aracını 5ire gibi Claude Desktop dışındaki MCP istem
``` ```
4. Claude Desktop'ı kapatıp yeniden başlatın. 4. Claude Desktop'ı kapatıp yeniden başlatın.
</details>
--- ---
🌟 **Gemini CLI ile Kullanım** <details>
<summary>🌟 <strong>Gemini CLI ile Kullanım</strong></summary>
Yargı MCP'yi Gemini CLI ile kullanmak için: Yargı MCP'yi Gemini CLI ile kullanmak için:
@@ -101,8 +110,6 @@ Yargı MCP'yi Gemini CLI ile kullanmak için:
"yargi_mcp": { "yargi_mcp": {
"command": "uvx", "command": "uvx",
"args": [ "args": [
"--from",
"git+https://github.com/saidsurucu/yargi-mcp",
"yargi-mcp" "yargi-mcp"
] ]
} }
@@ -123,58 +130,39 @@ Yargı MCP'yi Gemini CLI ile kullanmak için:
- "Danıştay'ın imar planı iptaline ilişkin kararlarını bul" - "Danıştay'ın imar planı iptaline ilişkin kararlarını bul"
- "Anayasa Mahkemesi'nin ifade özgürlüğü kararlarını getir" - "Anayasa Mahkemesi'nin ifade özgürlüğü kararlarını getir"
🛠️ **Kullanılabilir Araçlar (MCP Tools)** </details>
Bu FastMCP sunucusu aşağıdaki temel araçları sunar: <details>
<summary>🛠️ <strong>Kullanılabilir Araçlar (MCP Tools)</strong></summary>
### **Yargıtay Araçları (Dual API + 52 Daire Filtreleme)** Bu FastMCP sunucusu **19 optimize edilmiş MCP aracı** sunar (token verimliliği için optimize edilmiş):
* **Ana API:**
* `search_yargitay_detailed(arananKelime, birimYrgKurulDaire, ...)`: Yargıtay kararlarını detaylı kriterlerle arar. **52 daire/kurul seçeneği** (Hukuk/Ceza Daireleri 1-23, Genel Kurullar, Başkanlar Kurulu)
* `get_yargitay_document_markdown(id: str)`: Belirli bir Yargıtay kararının metnini Markdown formatında getirir.
* **Bedesten API (Alternatif):**
* `search_yargitay_bedesten(phrase, birimAdi, kararTarihiStart, kararTarihiEnd, ...)`: Bedesten API ile Yargıtay kararlarını arar. **Aynı 52 daire filtreleme** + **Tarih Filtreleme** + **Kesin Cümle Arama** (`"\"mülkiyet kararı\""`)
* `get_yargitay_bedesten_document_markdown(documentId: str)`: Bedesten'den karar metni (HTML/PDF → Markdown)
### **Danıştay Araçları (Triple API + 27 Daire Filtreleme)** ### **Yargıtay Araçları (Birleşik Bedesten API - Token Optimized)**
* **Ana API'lar:** *Not: Yargıtay araçları token verimliliği için birleşik Bedesten API'ye entegre edilmiştir*
* `search_danistay_by_keyword(andKelimeler, orKelimeler, ...)`: Danıştay kararlarını anahtar kelimelerle arar.
* `search_danistay_detailed(daire, esasYil, ...)`: Danıştay kararlarını detaylı kriterlerle arar.
* `get_danistay_document_markdown(id: str)`: Belirli bir Danıştay kararının metnini Markdown formatında getirir.
* **Bedesten API (Alternatif):**
* `search_danistay_bedesten(phrase, birimAdi, kararTarihiStart, kararTarihiEnd, ...)`: Bedesten API ile Danıştay kararlarını arar. **27 daire/kurul seçeneği** + **Tarih Filtreleme** + **Kesin Cümle Arama** (`"\"idari işlem\""`) (1-17. Daireler, Vergi/İdare Kurulları, Askeri Mahkemeler)
* `get_danistay_bedesten_document_markdown(documentId: str)`: Bedesten'den karar metni
### **Diğer Mahkemeler (Bedesten API + Gelişmiş Arama)** ### **Danıştay Araçları (Birleşik Bedesten API - Token Optimized)**
* **Yerel Hukuk Mahkemeleri:** *Not: Danıştay araçları token verimliliği için birleşik Bedesten API'ye entegre edilmiştir*
* `search_yerel_hukuk_bedesten(phrase, kararTarihiStart, kararTarihiEnd, ...)`: Yerel hukuk mahkemesi kararlarını arar + **Tarih & Kesin Cümle Arama** (`"\"sözleşme ihlali\""`)
* `get_yerel_hukuk_bedesten_document_markdown(documentId: str)`: Karar metni
* **İstinaf Hukuk Mahkemeleri:**
* `search_istinaf_hukuk_bedesten(phrase, kararTarihiStart, kararTarihiEnd, ...)`: İstinaf mahkemesi kararlarını arar + **Tarih & Kesin Cümle Arama** (`"\"temyiz incelemesi\""`)
* `get_istinaf_hukuk_bedesten_document_markdown(documentId: str)`: Karar metni
* **Kanun Yararına Bozma (KYB):**
* `search_kyb_bedesten(phrase, kararTarihiStart, kararTarihiEnd, ...)`: Olağanüstü kanun yolu kararlarını arar + **Tarih & Kesin Cümle Arama** (`"\"kanun yararına bozma\""`)
* `get_kyb_bedesten_document_markdown(documentId: str)`: Karar metni
* **Emsal Karar Araçları:** ### **Birleşik Bedesten API Araçları (5 Mahkeme) - 🚀 TOKEN OPTİMİZE**
* `search_emsal_detailed_decisions(search_query: EmsalSearchRequest) -> CompactEmsalSearchResult`: Emsal (UYAP) kararlarını detaylı kriterlerle arar. 1. `search_bedesten_unified(phrase, court_types, birimAdi, kararTarihiStart, kararTarihiEnd, ...)`: **5 mahkeme türünü** birleşik arama (Yargıtay, Danıştay, Yerel Hukuk, İstinaf Hukuk, KYB) + **79 daire filtreleme** + **Tarih & Kesin Cümle Arama**
* `get_emsal_document_markdown(id: str) -> EmsalDocumentMarkdown`: Belirli bir Emsal kararının metnini Markdown formatında getirir. 2. `get_bedesten_document_markdown(documentId: str)`: Bedesten API'den herhangi bir belgeyi Markdown formatında getirir (HTML/PDF → Markdown)
* **Uyuşmazlık Mahkemesi Araçları:** ### **Emsal Karar Araçları (UYAP)**
* `search_uyusmazlik_decisions(search_params: UyusmazlikSearchRequest) -> UyusmazlikSearchResponse`: Uyuşmazlık Mahkemesi kararlarını çeşitli form kriterleriyle arar. 3. `search_emsal_detailed_decisions(keyword, ...)`: Emsal (UYAP) kararlarını detaylı kriterlerle arar.
* `get_uyusmazlik_document_markdown_from_url(document_url: HttpUrl) -> UyusmazlikDocumentMarkdown`: Bir Uyuşmazlık kararını tam URL'sinden alıp Markdown formatında getirir. 4. `get_emsal_document_markdown(id: str)`: Belirli bir Emsal kararının metnini Markdown formatında getirir.
* **Anayasa Mahkemesi (Norm Denetimi) Araçları:** ### **Uyuşmazlık Mahkemesi Araçları**
* `search_anayasa_norm_denetimi_decisions(search_query: AnayasaNormDenetimiSearchRequest) -> AnayasaSearchResult`: AYM Norm Denetimi kararlarını kapsamlı kriterlerle arar. 5. `search_uyusmazlik_decisions(icerik, ...)`: Uyuşmazlık Mahkemesi kararlarını çeşitli form kriterleriyle arar.
* `get_anayasa_norm_denetimi_document_markdown(document_url: str, page_number: Optional[int] = 1) -> AnayasaDocumentMarkdown`: Belirli bir AYM Norm Denetimi kararını URL'sinden alır ve 5.000 karakterlik sayfalanmış Markdown içeriğini getirir. 6. `get_uyusmazlik_document_markdown_from_url(document_url)`: Bir Uyuşmazlık kararını tam URL'sinden alıp Markdown formatında getirir.
* **Anayasa Mahkemesi (Bireysel Başvuru) Araçları:** ### **Anayasa Mahkemesi Araçları (Birleşik API) - 🚀 TOKEN OPTİMİZE**
* `search_anayasa_bireysel_basvuru_report(search_query: AnayasaBireyselReportSearchRequest) -> AnayasaBireyselReportSearchResult`: AYM Bireysel Başvuru "Karar Arama Raporu" oluşturur. 7. `search_anayasa_unified(decision_type, keywords_all, ...)`: AYM kararlarını birleşik arama (Norm Denetimi + Bireysel Başvuru) - **4 araç → 2 araç optimizasyonu**
* `get_anayasa_bireysel_basvuru_document_markdown(document_url_path: str, page_number: Optional[int] = 1) -> AnayasaBireyselBasvuruDocumentMarkdown`: Belirli bir AYM Bireysel Başvuru kararını URL path'inden alır ve 5.000 karakterlik sayfalanmış Markdown içeriğini getirir. 8. `get_anayasa_document_unified(document_url, page_number)`: AYM kararlarını birleşik belge getirme - **sayfalanmış Markdown** içeriği
* **KİK (Kamu İhale Kurulu) Araçları:** ### **KİK (Kamu İhale Kurulu) Araçları**
* `search_kik_decisions(search_query: KikSearchRequest) -> KikSearchResult`: KİK (Kamu İhale Kurulu) kararlarını arar. 9. `search_kik_decisions(karar_tipi, ...)`: KİK (Kamu İhale Kurulu) kararlarını arar.
* `get_kik_document_markdown(karar_id: str, page_number: Optional[int] = 1) -> KikDocumentMarkdown`: Belirli bir KİK kararını, Base64 ile encode edilmiş `karar_id`'sini kullanarak alır ve 5.000 karakterlik sayfalanmış Markdown içeriğini getirir. 10. `get_kik_document_markdown(karar_id, page_number)`: Belirli bir KİK kararını, Base64 ile encode edilmiş `karar_id`'sini kullanarak alır ve **sayfalanmış Markdown** içeriğini getirir.
* **Rekabet Kurumu Araçları:** ### **Rekabet Kurumu Araçları**
    * `search_rekabet_kurumu_decisions(KararTuru: Literal[...], ...) -> RekabetSearchResult`: Rekabet Kurumu kararlarını arar. `KararTuru` için kullanıcı dostu isimler kullanılır (örn: "Birleşme ve Devralma").     * `search_rekabet_kurumu_decisions(KararTuru: Literal[...], ...) -> RekabetSearchResult`: Rekabet Kurumu kararlarını arar. `KararTuru` için kullanıcı dostu isimler kullanılır (örn: "Birleşme ve Devralma").
    * `get_rekabet_kurumu_document(karar_id: str, page_number: Optional[int] = 1) -> RekabetDocument`: Belirli bir Rekabet Kurumu kararını `karar_id` ile alır. Kararın PDF formatındaki orijinalinden istenen sayfayı ayıklar ve Markdown formatında döndürür.     * `get_rekabet_kurumu_document(karar_id: str, page_number: Optional[int] = 1) -> RekabetDocument`: Belirli bir Rekabet Kurumu kararını `karar_id` ile alır. Kararın PDF formatındaki orijinalinden istenen sayfayı ayıklar ve Markdown formatında döndürür.
@@ -189,15 +177,34 @@ Bu FastMCP sunucusu aşağıdaki temel araçları sunar:
* `get_sayistay_temyiz_kurulu_document_markdown(decision_id: str)`: Temyiz Kurulu kararının tam metnini Markdown formatında getirir * `get_sayistay_temyiz_kurulu_document_markdown(decision_id: str)`: Temyiz Kurulu kararının tam metnini Markdown formatında getirir
* `get_sayistay_daire_document_markdown(decision_id: str)`: Daire kararının tam metnini Markdown formatında getirir * `get_sayistay_daire_document_markdown(decision_id: str)`: Daire kararının tam metnini Markdown formatında getirir
* **KVKK Araçları (Brave Search API + Türkçe Arama):**
* `search_kvkk_decisions(keywords, page, pageSize, ...)`: KVKK (Kişisel Verilerin Korunması Kurulu) kararlarını Brave Search API ile arar. **Türkçe arama** + **Site hedeflemeli** (`site:kvkk.gov.tr "karar özeti"`) + **Sayfalama desteği**
* `get_kvkk_document_markdown(decision_url: str, page_number: Optional[int] = 1)`: KVKK kararının tam metnini **sayfalanmış Markdown** formatında getirir (5.000 karakterlik sayfa)
### BDDK Araçları
* `search_bddk_decisions(keywords, page)`: BDDK (Bankacılık Düzenleme ve Denetleme Kurumu) kararlarını arar. **"Karar Sayısı" targeting** + **Spesifik URL filtreleme** (`bddk.org.tr/Mevzuat/DokumanGetir`) + **Optimized search**
* `get_bddk_document_markdown(document_id: str, page_number: Optional[int] = 1)`: BDDK kararının tam metnini **sayfalanmış Markdown** formatında getirir (5.000 karakterlik sayfa)
</details>
--- ---
### **📊 Kapsamlı İstatistikler** <details>
- **Toplam Mahkeme/Kurum:** 12 farklı hukuki kurum <summary>📊 <strong>Kapsamlı İstatistikler & Optimizasyon Başarıları</strong></summary>
- **Toplam MCP Tool:** 36+ arama ve belge getirme aracı
🚀 **TOKEN OPTİMİZASYON BAŞARISI:**
- **%61.8 Token Azaltma:** 14,061 → 5,369 tokens (8,692 token tasarrufu)
- **Hedef Aşım:** 10,000 token hedefini 4,631 token aştık
- **Daha Hızlı Yanıt:** Claude AI ile optimize edilmiş etkileşim
- **Korunan İşlevsellik:** %100 özellik desteği devam ediyor
**GENEL İSTATİSTİKLER:**
- **Toplam Mahkeme/Kurum:** 13 farklı hukuki kurum (KVKK dahil)
- **Toplam MCP Tool:** 19 optimize edilmiş arama ve belge getirme aracı
- **Daire/Kurul Filtreleme:** 87 farklı seçenek (52 Yargıtay + 27 Danıştay + 8 Sayıştay) - **Daire/Kurul Filtreleme:** 87 farklı seçenek (52 Yargıtay + 27 Danıştay + 8 Sayıştay)
- **Tarih Filtreleme:** 5 Bedesten API aracında ISO 8601 formatında tam tarih aralığı desteği - **Tarih Filtreleme:** Birleşik Bedesten API aracında ISO 8601 formatında tam tarih aralığı desteği
- **Kesin Cümle Arama:** 5 Bedesten API aracında çift tırnak ile tam cümle arama (`"\"mülkiyet kararı\""` formatı) - **Kesin Cümle Arama:** Birleşik Bedesten API aracında çift tırnak ile tam cümle arama (`"\"mülkiyet kararı\""` formatı)
- **Birleşik API:** 10 ayrı Bedesten aracı → 2 birleşik araç (search_bedesten_unified + get_bedesten_document_markdown)
- **API Kaynağı:** Dual/Triple API desteği ile maksimum kapsama - **API Kaynağı:** Dual/Triple API desteği ile maksimum kapsama
- **Tam Türk Adalet Sistemi:** Yerel mahkemelerden en yüksek mahkemelere kadar - **Tam Türk Adalet Sistemi:** Yerel mahkemelerden en yüksek mahkemelere kadar
@@ -222,9 +229,19 @@ Bedesten API Bedesten API Dual/Triple API Norm+Bireysel API
- Kesin arama: `"\"mülkiyet kararı\""` (tam cümle olarak) - Kesin arama: `"\"mülkiyet kararı\""` (tam cümle olarak)
- Daha kesin sonuçlar için hukuki terimler ve kavramlar - Daha kesin sonuçlar için hukuki terimler ve kavramlar
**🔧 OPTİMİZASYON DETAYLARI:**
- **Anayasa Mahkemesi:** 4 araç → 2 birleşik araç (search_anayasa_unified + get_anayasa_document_unified)
- **Yargıtay & Danıştay:** Ana API araçları birleşik Bedesten API'ye entegre edildi
- **Sayıştay:** 6 araç → 2 birleşik araç (search_sayistay_unified + get_sayistay_document_unified)
- **Parameter Optimizasyonu:** pageSize parametreleri optimize edildi
- **Açıklama Optimizasyonu:** Uzun açıklamalar kısaltıldı (örn: KIK karar_metni)
</details>
--- ---
🌐 **Web Service / ASGI Deployment** <details>
<summary>🌐 <strong>Web Service / ASGI Deployment</strong></summary>
Yargı MCP artık web servisi olarak da çalıştırılabilir! ASGI desteği sayesinde: Yargı MCP artık web servisi olarak da çalıştırılabilir! ASGI desteği sayesinde:
@@ -246,6 +263,8 @@ uvicorn asgi_app:app --host 0.0.0.0 --port 8000
Detaylı deployment rehberi için: [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) Detaylı deployment rehberi için: [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md)
</details>
--- ---
📜 **Lisans** 📜 **Lisans**
+25 -26
View File
@@ -7,8 +7,7 @@ from typing import Dict, Any, List, Optional, Tuple
import logging import logging
import html import html
import re import re
import tempfile import io
import os
from urllib.parse import urlencode, urljoin, quote from urllib.parse import urlencode, urljoin, quote
from markitdown import MarkItDown from markitdown import MarkItDown
import math # For math.ceil for pagination import math # For math.ceil for pagination
@@ -100,11 +99,11 @@ class AnayasaBireyselBasvuruApiClient:
for decision_div in decision_divs: for decision_div in decision_divs:
title_tag = decision_div.find("h4") title_tag = decision_div.find("h4")
title_text = title_tag.get_text(strip=True) if title_tag and title_tag.strong else (title_tag.get_text(strip=True) if title_tag else None) title_text = title_tag.get_text(strip=True) if title_tag and title_tag.strong else (title_tag.get_text(strip=True) if title_tag else "")
alti_cizili_div = decision_div.find("div", class_="AltiCizili") alti_cizili_div = decision_div.find("div", class_="AltiCizili")
ref_no, dec_type, body, app_date, dec_date, url_path = None, None, None, None, None, None ref_no, dec_type, body, app_date, dec_date, url_path = "", "", "", "", "", ""
if alti_cizili_div: if alti_cizili_div:
link_tag = alti_cizili_div.find("a", href=True) link_tag = alti_cizili_div.find("a", href=True)
if link_tag: if link_tag:
@@ -125,14 +124,14 @@ class AnayasaBireyselBasvuruApiClient:
ref_no = parts[current_idx] ref_no = parts[current_idx]
current_idx += 1 current_idx += 1
dec_type = parts[current_idx] if len(parts) > current_idx else None dec_type = parts[current_idx] if len(parts) > current_idx else ""
current_idx += 1 current_idx += 1
body = parts[current_idx] if len(parts) > current_idx else None body = parts[current_idx] if len(parts) > current_idx else ""
current_idx += 1 current_idx += 1
app_date_raw = parts[current_idx] if len(parts) > current_idx else None app_date_raw = parts[current_idx] if len(parts) > current_idx else ""
current_idx += 1 current_idx += 1
dec_date_raw = parts[current_idx] if len(parts) > current_idx else None dec_date_raw = parts[current_idx] if len(parts) > current_idx else ""
if app_date_raw and "Başvuru Tarihi :" in app_date_raw: if app_date_raw and "Başvuru Tarihi :" in app_date_raw:
app_date = app_date_raw.replace("Başvuru Tarihi :", "").strip() app_date = app_date_raw.replace("Başvuru Tarihi :", "").strip()
@@ -149,7 +148,7 @@ class AnayasaBireyselBasvuruApiClient:
subject_div = decision_div.find(lambda tag: tag.name == 'div' and not tag.has_attr('class') and tag.get_text(strip=True).startswith("BAŞVURU KONUSU :")) subject_div = decision_div.find(lambda tag: tag.name == 'div' and not tag.has_attr('class') and tag.get_text(strip=True).startswith("BAŞVURU KONUSU :"))
subject_text = subject_div.get_text(strip=True).replace("BAŞVURU KONUSU :", "").strip() if subject_div else None subject_text = subject_div.get_text(strip=True).replace("BAŞVURU KONUSU :", "").strip() if subject_div else ""
details_list: List[AnayasaBireyselReportDecisionDetail] = [] details_list: List[AnayasaBireyselReportDecisionDetail] = []
karar_detaylari_div = decision_div.find_next_sibling("div", id="KararDetaylari") # Corrected: was KararDetaylari karar_detaylari_div = decision_div.find_next_sibling("div", id="KararDetaylari") # Corrected: was KararDetaylari
@@ -160,13 +159,13 @@ class AnayasaBireyselBasvuruApiClient:
cells = row.find_all("td") cells = row.find_all("td")
if len(cells) == 4: # Hak, Müdahale İddiası, Sonuç, Giderim if len(cells) == 4: # Hak, Müdahale İddiası, Sonuç, Giderim
details_list.append(AnayasaBireyselReportDecisionDetail( details_list.append(AnayasaBireyselReportDecisionDetail(
hak=cells[0].get_text(strip=True) or None, hak=cells[0].get_text(strip=True) or "",
mudahale_iddiasi=cells[1].get_text(strip=True) or None, mudahale_iddiasi=cells[1].get_text(strip=True) or "",
sonuc=cells[2].get_text(strip=True) or None, sonuc=cells[2].get_text(strip=True) or "",
giderim=cells[3].get_text(strip=True) or None, giderim=cells[3].get_text(strip=True) or "",
)) ))
full_decision_page_url = urljoin(self.BASE_URL, url_path) if url_path else None full_decision_page_url = urljoin(self.BASE_URL, url_path) if url_path else ""
processed_decisions.append(AnayasaBireyselReportDecisionSummary( processed_decisions.append(AnayasaBireyselReportDecisionSummary(
title=title_text, title=title_text,
@@ -230,23 +229,23 @@ class AnayasaBireyselBasvuruApiClient:
html_input_for_markdown = processed_html html_input_for_markdown = processed_html
markdown_text = None markdown_text = None
temp_file_path = None
try: try:
md_converter = MarkItDown() # Ensure the content is wrapped in basic HTML structure if it's not already
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file: if not html_input_for_markdown.strip().lower().startswith(("<html", "<!doctype")):
if not html_input_for_markdown.strip().lower().startswith(("<html", "<!doctype")): html_content = f"<html><head><meta charset=\"UTF-8\"></head><body>{html_input_for_markdown}</body></html>"
tmp_file.write(f"<html><head><meta charset=\"UTF-8\"></head><body>{html_input_for_markdown}</body></html>") else:
else: html_content = html_input_for_markdown
tmp_file.write(html_input_for_markdown)
temp_file_path = tmp_file.name
conversion_result = md_converter.convert(temp_file_path) # Convert HTML string to bytes and create BytesIO stream
html_bytes = html_content.encode('utf-8')
html_stream = io.BytesIO(html_bytes)
# Pass BytesIO stream to MarkItDown to avoid temp file creation
md_converter = MarkItDown()
conversion_result = md_converter.convert(html_stream)
markdown_text = conversion_result.text_content markdown_text = conversion_result.text_content
except Exception as e: except Exception as e:
logger.error(f"AnayasaBireyselBasvuruApiClient: MarkItDown conversion error: {e}") logger.error(f"AnayasaBireyselBasvuruApiClient: MarkItDown conversion error: {e}")
finally:
if temp_file_path and os.path.exists(temp_file_path):
os.remove(temp_file_path)
return markdown_text return markdown_text
async def get_decision_document_as_markdown( async def get_decision_document_as_markdown(
+37 -35
View File
@@ -7,8 +7,7 @@ from typing import Dict, Any, List, Optional, Tuple
import logging import logging
import html import html
import re import re
import tempfile import io
import os
from urllib.parse import urlencode, urljoin, quote from urllib.parse import urlencode, urljoin, quote
from markitdown import MarkItDown from markitdown import MarkItDown
import math # For math.ceil for pagination import math # For math.ceil for pagination
@@ -51,37 +50,44 @@ class AnayasaMahkemesiApiClient:
for kw in params.keywords_any: query_params.append(("HerhangiBirKelimeAra[]", kw)) for kw in params.keywords_any: query_params.append(("HerhangiBirKelimeAra[]", kw))
if params.keywords_exclude: if params.keywords_exclude:
for kw in params.keywords_exclude: query_params.append(("BulunmayanKelimeAra[]", kw)) for kw in params.keywords_exclude: query_params.append(("BulunmayanKelimeAra[]", kw))
if params.period and params.period.value and params.period.value != "ALL": query_params.append(("Donemler_id", params.period.value)) if params.period and params.period and params.period != "ALL": query_params.append(("Donemler_id", params.period))
if params.case_number_esas: query_params.append(("EsasNo", params.case_number_esas)) if params.case_number_esas: query_params.append(("EsasNo", params.case_number_esas))
if params.decision_number_karar: query_params.append(("KararNo", params.decision_number_karar)) if params.decision_number_karar: query_params.append(("KararNo", params.decision_number_karar))
if params.first_review_date_start: query_params.append(("IlkIncelemeTarihiIlk", params.first_review_date_start)) if params.first_review_date_start: query_params.append(("IlkIncelemeTarihiIlk", params.first_review_date_start))
if params.first_review_date_end: query_params.append(("IlkIncelemeTarihiSon", params.first_review_date_end)) if params.first_review_date_end: query_params.append(("IlkIncelemeTarihiSon", params.first_review_date_end))
if params.decision_date_start: query_params.append(("KararTarihiIlk", params.decision_date_start)) if params.decision_date_start: query_params.append(("KararTarihiIlk", params.decision_date_start))
if params.decision_date_end: query_params.append(("KararTarihiSon", params.decision_date_end)) if params.decision_date_end: query_params.append(("KararTarihiSon", params.decision_date_end))
if params.application_type and params.application_type.value and params.application_type.value != "ALL": query_params.append(("BasvuruTurler_id", params.application_type.value)) if params.application_type and params.application_type and params.application_type != "ALL": query_params.append(("BasvuruTurler_id", params.application_type))
if params.applicant_general_name: query_params.append(("BasvuranGeneller_id", params.applicant_general_name)) if params.applicant_general_name: query_params.append(("BasvuranGeneller_id", params.applicant_general_name))
if params.applicant_specific_name: query_params.append(("BasvuranOzeller_id", params.applicant_specific_name)) if params.applicant_specific_name: query_params.append(("BasvuranOzeller_id", params.applicant_specific_name))
if params.attending_members_names: if params.attending_members_names:
for name in params.attending_members_names: query_params.append(("Uyeler_id[]", name)) for name in params.attending_members_names: query_params.append(("Uyeler_id[]", name))
if params.rapporteur_name: query_params.append(("Raportorler_id", params.rapporteur_name)) if params.rapporteur_name: query_params.append(("Raportorler_id", params.rapporteur_name))
if params.norm_type and params.norm_type.value and params.norm_type.value != "ALL": query_params.append(("NormunTurler_id", params.norm_type.value)) if params.norm_type and params.norm_type and params.norm_type != "ALL": query_params.append(("NormunTurler_id", params.norm_type))
if params.norm_id_or_name: query_params.append(("NormunNumarasiAdlar_id", params.norm_id_or_name)) if params.norm_id_or_name: query_params.append(("NormunNumarasiAdlar_id", params.norm_id_or_name))
if params.norm_article: query_params.append(("NormunMaddeNumarasi", params.norm_article)) if params.norm_article: query_params.append(("NormunMaddeNumarasi", params.norm_article))
if params.review_outcomes: if params.review_outcomes:
for outcome_enum_val in params.review_outcomes: for outcome_val in params.review_outcomes:
if outcome_enum_val.value and outcome_enum_val.value != "ALL": query_params.append(("IncelemeTuruKararSonuclar_id[]", outcome_enum_val.value)) if outcome_val and outcome_val != "ALL": query_params.append(("IncelemeTuruKararSonuclar_id[]", outcome_val))
if params.reason_for_final_outcome and params.reason_for_final_outcome.value and params.reason_for_final_outcome.value != "ALL": if params.reason_for_final_outcome and params.reason_for_final_outcome and params.reason_for_final_outcome != "ALL":
query_params.append(("KararSonucununGerekcesi", params.reason_for_final_outcome.value)) query_params.append(("KararSonucununGerekcesi", params.reason_for_final_outcome))
if params.basis_constitution_article_numbers: if params.basis_constitution_article_numbers:
for article_no in params.basis_constitution_article_numbers: query_params.append(("DayanakHukmu[]", article_no)) for article_no in params.basis_constitution_article_numbers: query_params.append(("DayanakHukmu[]", article_no))
if params.official_gazette_date_start: query_params.append(("ResmiGazeteTarihiIlk", params.official_gazette_date_start)) if params.official_gazette_date_start: query_params.append(("ResmiGazeteTarihiIlk", params.official_gazette_date_start))
if params.official_gazette_date_end: query_params.append(("ResmiGazeteTarihiSon", params.official_gazette_date_end)) if params.official_gazette_date_end: query_params.append(("ResmiGazeteTarihiSon", params.official_gazette_date_end))
if params.official_gazette_number_start: query_params.append(("ResmiGazeteSayisiIlk", params.official_gazette_number_start)) if params.official_gazette_number_start: query_params.append(("ResmiGazeteSayisiIlk", params.official_gazette_number_start))
if params.official_gazette_number_end: query_params.append(("ResmiGazeteSayisiSon", params.official_gazette_number_end)) if params.official_gazette_number_end: query_params.append(("ResmiGazeteSayisiSon", params.official_gazette_number_end))
if params.has_press_release and params.has_press_release.value and params.has_press_release.value != "ALL": query_params.append(("BasinDuyurusu", params.has_press_release.value)) if params.has_press_release and params.has_press_release and params.has_press_release != "ALL": query_params.append(("BasinDuyurusu", params.has_press_release))
if params.has_dissenting_opinion and params.has_dissenting_opinion.value and params.has_dissenting_opinion.value != "ALL": query_params.append(("KarsiOy", params.has_dissenting_opinion.value)) if params.has_dissenting_opinion and params.has_dissenting_opinion and params.has_dissenting_opinion != "ALL": query_params.append(("KarsiOy", params.has_dissenting_opinion))
if params.has_different_reasoning and params.has_different_reasoning.value and params.has_different_reasoning.value != "ALL": query_params.append(("FarkliGerekce", params.has_different_reasoning.value)) if params.has_different_reasoning and params.has_different_reasoning and params.has_different_reasoning != "ALL": query_params.append(("FarkliGerekce", params.has_different_reasoning))
# Add pagination and sorting parameters as query params instead of URL path
if params.results_per_page and params.results_per_page != 10:
query_params.append(("SatirSayisi", str(params.results_per_page)))
if params.sort_by_criteria and params.sort_by_criteria != "KararTarihi":
query_params.append(("Siralama", params.sort_by_criteria))
if params.page_to_fetch and params.page_to_fetch > 1: if params.page_to_fetch and params.page_to_fetch > 1:
query_params.append(("page", str(params.page_to_fetch))) query_params.append(("page", str(params.page_to_fetch)))
return query_params return query_params
@@ -90,16 +96,8 @@ class AnayasaMahkemesiApiClient:
self, self,
params: AnayasaNormDenetimiSearchRequest params: AnayasaNormDenetimiSearchRequest
) -> AnayasaSearchResult: ) -> AnayasaSearchResult:
path_segments = [] # Use simple /Ara endpoint - the complex path structure seems to cause 404s
if params.results_per_page and params.results_per_page != 10: # Default is 10 request_path = f"/{self.SEARCH_PATH_SEGMENT}"
path_segments.append(f"SatirSayisi/{params.results_per_page}")
if params.sort_by_criteria and params.sort_by_criteria != "KararTarihi": # Default is KararTarihi
# Ensure correct quoting for criteria that might have Turkish chars or spaces
path_segments.append(f"Siralama/{quote(params.sort_by_criteria)}")
path_segments.append(self.SEARCH_PATH_SEGMENT)
request_path = "/" + "/".join(path_segments)
final_query_params = self._build_search_query_params_for_aym(params) final_query_params = self._build_search_query_params_for_aym(params)
logger.info(f"AnayasaMahkemesiApiClient: Performing Norm Denetimi search. Path: {request_path}, Params: {final_query_params}") logger.info(f"AnayasaMahkemesiApiClient: Performing Norm Denetimi search. Path: {request_path}, Params: {final_query_params}")
@@ -222,24 +220,23 @@ class AnayasaMahkemesiApiClient:
html_input_for_markdown = str(body_tag) if body_tag else processed_html html_input_for_markdown = str(body_tag) if body_tag else processed_html
markdown_text = None markdown_text = None
temp_file_path = None
try: try:
md_converter = MarkItDown() # Ensure the content is wrapped in basic HTML structure if it's not already
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file: if not html_input_for_markdown.strip().lower().startswith(("<html", "<!doctype")):
# Ensure the content is wrapped in basic HTML structure if it's not already html_content = f"<html><head><meta charset=\"UTF-8\"></head><body>{html_input_for_markdown}</body></html>"
if not html_input_for_markdown.strip().lower().startswith(("<html", "<!doctype")): else:
tmp_file.write(f"<html><head><meta charset=\"UTF-8\"></head><body>{html_input_for_markdown}</body></html>") html_content = html_input_for_markdown
else:
tmp_file.write(html_input_for_markdown)
temp_file_path = tmp_file.name
conversion_result = md_converter.convert(temp_file_path) # Convert HTML string to bytes and create BytesIO stream
html_bytes = html_content.encode('utf-8')
html_stream = io.BytesIO(html_bytes)
# Pass BytesIO stream to MarkItDown to avoid temp file creation
md_converter = MarkItDown()
conversion_result = md_converter.convert(html_stream)
markdown_text = conversion_result.text_content markdown_text = conversion_result.text_content
except Exception as e: except Exception as e:
logger.error(f"AnayasaMahkemesiApiClient: MarkItDown conversion error: {e}") logger.error(f"AnayasaMahkemesiApiClient: MarkItDown conversion error: {e}")
finally:
if temp_file_path and os.path.exists(temp_file_path):
os.remove(temp_file_path)
return markdown_text return markdown_text
async def get_decision_document_as_markdown( async def get_decision_document_as_markdown(
@@ -277,6 +274,11 @@ class AnayasaMahkemesiApiClient:
if not karar_metni_div: # Fallback if not in KararMetni if not karar_metni_div: # Fallback if not in KararMetni
karar_metni_div = soup.find("div", class_="WordSection1") karar_metni_div = soup.find("div", class_="WordSection1")
# Initialize with empty string defaults
decision_ek_no_from_page = ""
decision_date_from_page = ""
official_gazette_from_page = ""
if karar_metni_div: if karar_metni_div:
# Attempt to find E.K. No (Esas No, Karar No) # Attempt to find E.K. No (Esas No, Karar No)
# Norm Denetimi pages often have this in bold <p> tags directly or in the WordSection1 # Norm Denetimi pages often have this in bold <p> tags directly or in the WordSection1
+100 -82
View File
@@ -1,43 +1,21 @@
# anayasa_mcp_module/models.py # anayasa_mcp_module/models.py
from pydantic import BaseModel, Field, HttpUrl from pydantic import BaseModel, Field, HttpUrl
from typing import List, Optional, Dict, Any from typing import List, Optional, Dict, Any, Literal
from enum import Enum from enum import Enum
# --- Enums (AnayasaDonemEnum, AnayasaBasvuruTuruEnum, etc. - same as before) --- # --- Enums (AnayasaDonemEnum, etc. - same as before) ---
class AnayasaDonemEnum(str, Enum): class AnayasaDonemEnum(str, Enum):
TUMU = "ALL" TUMU = "ALL"
DONEM_1961 = "1" DONEM_1961 = "1"
DONEM_1982 = "2" DONEM_1982 = "2"
class AnayasaBasvuruTuruEnum(str, Enum):
TUMU = "ALL"
IPTAL = "1"
ITIRAZ = "2"
DIGER = "3"
class AnayasaVarYokEnum(str, Enum): class AnayasaVarYokEnum(str, Enum):
TUMU = "ALL" TUMU = "ALL"
YOK = "0" YOK = "0"
VAR = "1" VAR = "1"
class AnayasaNormTuruEnum(str, Enum):
TUMU = "ALL"
ANAYASA = "1"
ANAYASA_DEGISTIREN_KANUN = "2"
CUMHURBASKANLIGI_KARARNAMESI = "14"
ICTUZUK = "3"
KANUN = "4"
KANUN_HUKMUNDE_KARARNAME = "5"
KARAR = "6"
NIZAMNAME = "7"
TALIMATNAME = "8"
TARIFE = "9"
TBMM_KARARI = "10"
TEZKERE = "11"
TUZUK = "12"
YOK_SECENEGI = "0"
YONETMELIK = "13"
class AnayasaIncelemeSonucuEnum(str, Enum): class AnayasaIncelemeSonucuEnum(str, Enum):
TUMU = "ALL" TUMU = "ALL"
@@ -89,60 +67,60 @@ class AnayasaNormDenetimiSearchRequest(BaseModel):
keywords_all: Optional[List[str]] = Field(default_factory=list, description="Keywords for AND logic (KelimeAra[]).") keywords_all: Optional[List[str]] = Field(default_factory=list, description="Keywords for AND logic (KelimeAra[]).")
keywords_any: Optional[List[str]] = Field(default_factory=list, description="Keywords for OR logic (HerhangiBirKelimeAra[]).") keywords_any: Optional[List[str]] = Field(default_factory=list, description="Keywords for OR logic (HerhangiBirKelimeAra[]).")
keywords_exclude: Optional[List[str]] = Field(default_factory=list, description="Keywords to exclude (BulunmayanKelimeAra[]).") keywords_exclude: Optional[List[str]] = Field(default_factory=list, description="Keywords to exclude (BulunmayanKelimeAra[]).")
period: Optional[AnayasaDonemEnum] = Field(default=AnayasaDonemEnum.TUMU, description="Constitutional period (Donemler_id).") period: Optional[Literal["ALL", "1", "2"]] = Field(default="ALL", description="Constitutional period (Donemler_id).")
case_number_esas: Optional[str] = Field(None, description="Case registry number (EsasNo), e.g., '2023/123'.") case_number_esas: str = Field("", description="Case registry number (EsasNo), e.g., '2023/123'.")
decision_number_karar: Optional[str] = Field(None, description="Decision number (KararNo), e.g., '2023/456'.") decision_number_karar: str = Field("", description="Decision number (KararNo), e.g., '2023/456'.")
first_review_date_start: Optional[str] = Field(None, description="First review start date (IlkIncelemeTarihiIlk), format DD/MM/YYYY.") first_review_date_start: str = Field("", description="First review start date (IlkIncelemeTarihiIlk), format DD/MM/YYYY.")
first_review_date_end: Optional[str] = Field(None, description="First review end date (IlkIncelemeTarihiSon), format DD/MM/YYYY.") first_review_date_end: str = Field("", description="First review end date (IlkIncelemeTarihiSon), format DD/MM/YYYY.")
decision_date_start: Optional[str] = Field(None, description="Decision start date (KararTarihiIlk), format DD/MM/YYYY.") decision_date_start: str = Field("", description="Decision start date (KararTarihiIlk), format DD/MM/YYYY.")
decision_date_end: Optional[str] = Field(None, description="Decision end date (KararTarihiSon), format DD/MM/YYYY.") decision_date_end: str = Field("", description="Decision end date (KararTarihiSon), format DD/MM/YYYY.")
application_type: Optional[AnayasaBasvuruTuruEnum] = Field(default=AnayasaBasvuruTuruEnum.TUMU, description="Type of application (BasvuruTurler_id).") application_type: Optional[Literal["ALL", "1", "2", "3"]] = Field(default="ALL", description="Type of application (BasvuruTurler_id).")
applicant_general_name: Optional[str] = Field(None, description="General applicant name (BasvuranGeneller_id).") applicant_general_name: str = Field("", description="General applicant name (BasvuranGeneller_id).")
applicant_specific_name: Optional[str] = Field(None, description="Specific applicant name (BasvuranOzeller_id).") applicant_specific_name: str = Field("", description="Specific applicant name (BasvuranOzeller_id).")
official_gazette_date_start: Optional[str] = Field(None, description="Official Gazette start date (ResmiGazeteTarihiIlk), format DD/MM/YYYY.") official_gazette_date_start: str = Field("", description="Official Gazette start date (ResmiGazeteTarihiIlk), format DD/MM/YYYY.")
official_gazette_date_end: Optional[str] = Field(None, description="Official Gazette end date (ResmiGazeteTarihiSon), format DD/MM/YYYY.") official_gazette_date_end: str = Field("", description="Official Gazette end date (ResmiGazeteTarihiSon), format DD/MM/YYYY.")
official_gazette_number_start: Optional[str] = Field(None, description="Official Gazette starting number (ResmiGazeteSayisiIlk).") official_gazette_number_start: str = Field("", description="Official Gazette starting number (ResmiGazeteSayisiIlk).")
official_gazette_number_end: Optional[str] = Field(None, description="Official Gazette ending number (ResmiGazeteSayisiSon).") official_gazette_number_end: str = Field("", description="Official Gazette ending number (ResmiGazeteSayisiSon).")
has_press_release: Optional[AnayasaVarYokEnum] = Field(default=AnayasaVarYokEnum.TUMU, description="Press release available (BasinDuyurusu).") has_press_release: Optional[Literal["ALL", "0", "1"]] = Field(default="ALL", description="Press release available (BasinDuyurusu).")
has_dissenting_opinion: Optional[AnayasaVarYokEnum] = Field(default=AnayasaVarYokEnum.TUMU, description="Dissenting opinion exists (KarsiOy).") has_dissenting_opinion: Optional[Literal["ALL", "0", "1"]] = Field(default="ALL", description="Dissenting opinion exists (KarsiOy).")
has_different_reasoning: Optional[AnayasaVarYokEnum] = Field(default=AnayasaVarYokEnum.TUMU, description="Different reasoning exists (FarkliGerekce).") has_different_reasoning: Optional[Literal["ALL", "0", "1"]] = Field(default="ALL", description="Different reasoning exists (FarkliGerekce).")
attending_members_names: Optional[List[str]] = Field(default_factory=list, description="List of attending members' exact names (Uyeler_id[]).") attending_members_names: Optional[List[str]] = Field(default_factory=list, description="List of attending members' exact names (Uyeler_id[]).")
rapporteur_name: Optional[str] = Field(None, description="Rapporteur's exact name (Raportorler_id).") rapporteur_name: str = Field("", description="Rapporteur's exact name (Raportorler_id).")
norm_type: Optional[AnayasaNormTuruEnum] = Field(default=AnayasaNormTuruEnum.TUMU, description="Type of the reviewed norm (NormunTurler_id).") norm_type: Optional[Literal["ALL", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "0"]] = Field(default="ALL", description="Type of the reviewed norm (NormunTurler_id).")
norm_id_or_name: Optional[str] = Field(None, description="Number or name of the norm (NormunNumarasiAdlar_id).") norm_id_or_name: str = Field("", description="Number or name of the norm (NormunNumarasiAdlar_id).")
norm_article: Optional[str] = Field(None, description="Article number of the norm (NormunMaddeNumarasi).") norm_article: str = Field("", description="Article number of the norm (NormunMaddeNumarasi).")
review_outcomes: Optional[List[AnayasaIncelemeSonucuEnum]] = Field(default_factory=list, description="List of review types and outcomes (IncelemeTuruKararSonuclar_id[]).") review_outcomes: Optional[List[Literal["1", "2", "3", "4", "5", "6", "7", "8", "12"]]] = Field(default_factory=list, description="List of review types and outcomes (IncelemeTuruKararSonuclar_id[]).")
reason_for_final_outcome: Optional[AnayasaSonucGerekcesiEnum] = Field(default=AnayasaSonucGerekcesiEnum.TUMU, description="Main reason for the decision outcome (KararSonucununGerekcesi).") reason_for_final_outcome: Optional[Literal["ALL", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24", "25", "26", "27", "29", "30"]] = Field(default="ALL", description="Main reason for the decision outcome (KararSonucununGerekcesi).")
basis_constitution_article_numbers: Optional[List[str]] = Field(default_factory=list, description="List of supporting Constitution article numbers (DayanakHukmu[]).") basis_constitution_article_numbers: Optional[List[str]] = Field(default_factory=list, description="List of supporting Constitution article numbers (DayanakHukmu[]).")
results_per_page: Optional[int] = Field(10, description="Number of results per page. Options: 10, 20, 30, 40, 50.") results_per_page: int = Field(10, ge=1, le=10, description="Results per page.")
page_to_fetch: Optional[int] = Field(1, ge=1, description="Page number to fetch for results list.") page_to_fetch: int = Field(1, ge=1, description="Page number to fetch for results list.")
sort_by_criteria: Optional[str] = Field("KararTarihi", description="Sort criteria. Options: 'KararTarihi', 'YayinTarihi', 'Toplam' (keyword count).") sort_by_criteria: str = Field("KararTarihi", description="Sort criteria. Options: 'KararTarihi', 'YayinTarihi', 'Toplam' (keyword count).")
class AnayasaReviewedNormInfo(BaseModel): class AnayasaReviewedNormInfo(BaseModel):
"""Details of a norm reviewed within an AYM decision summary.""" """Details of a norm reviewed within an AYM decision summary."""
norm_name_or_number: Optional[str] = None norm_name_or_number: str = Field("", description="Norm name or number")
article_number: Optional[str] = None article_number: str = Field("", description="Article number")
review_type_and_outcome: Optional[str] = None review_type_and_outcome: str = Field("", description="Review type and outcome")
outcome_reason: Optional[str] = None outcome_reason: str = Field("", description="Outcome reason")
basis_constitution_articles_cited: List[str] = Field(default_factory=list) basis_constitution_articles_cited: List[str] = Field(default_factory=list)
postponement_period: Optional[str] = None postponement_period: str = Field("", description="Postponement period")
class AnayasaDecisionSummary(BaseModel): class AnayasaDecisionSummary(BaseModel):
"""Model for a single Anayasa Mahkemesi (Norm Denetimi) decision summary from search results.""" """Model for a single Anayasa Mahkemesi (Norm Denetimi) decision summary from search results."""
decision_reference_no: Optional[str] = None decision_reference_no: str = Field("", description="Decision reference number")
decision_page_url: Optional[HttpUrl] = None decision_page_url: str = Field("", description="Decision page URL")
keywords_found_count: Optional[int] = None keywords_found_count: Optional[int] = Field(0, description="Keywords found count")
application_type_summary: Optional[str] = None application_type_summary: str = Field("", description="Application type summary")
applicant_summary: Optional[str] = None applicant_summary: str = Field("", description="Applicant summary")
decision_outcome_summary: Optional[str] = None decision_outcome_summary: str = Field("", description="Decision outcome summary")
decision_date_summary: Optional[str] = None decision_date_summary: str = Field("", description="Decision date summary")
reviewed_norms: List[AnayasaReviewedNormInfo] = Field(default_factory=list) reviewed_norms: List[AnayasaReviewedNormInfo] = Field(default_factory=list)
class AnayasaSearchResult(BaseModel): class AnayasaSearchResult(BaseModel):
"""Model for the overall search result for Anayasa Mahkemesi Norm Denetimi decisions.""" """Model for the overall search result for Anayasa Mahkemesi Norm Denetimi decisions."""
decisions: List[AnayasaDecisionSummary] decisions: List[AnayasaDecisionSummary]
total_records_found: Optional[int] = None total_records_found: int = Field(0, description="Total records found")
retrieved_page_number: Optional[int] = None retrieved_page_number: int = Field(1, description="Retrieved page number")
class AnayasaDocumentMarkdown(BaseModel): class AnayasaDocumentMarkdown(BaseModel):
""" """
@@ -150,10 +128,10 @@ class AnayasaDocumentMarkdown(BaseModel):
and pagination information. and pagination information.
""" """
source_url: HttpUrl source_url: HttpUrl
decision_reference_no_from_page: Optional[str] = Field(None, description="E.K. No parsed from the document page.") decision_reference_no_from_page: str = Field("", description="E.K. No parsed from the document page.")
decision_date_from_page: Optional[str] = Field(None, description="Decision date parsed from the document page.") decision_date_from_page: str = Field("", description="Decision date parsed from the document page.")
official_gazette_info_from_page: Optional[str] = Field(None, description="Official Gazette info parsed from the document page.") official_gazette_info_from_page: str = Field("", description="Official Gazette info parsed from the document page.")
markdown_chunk: Optional[str] = Field(None, description="A 5,000 character chunk of the Markdown content.") # Corrected chunk size markdown_chunk: str = Field("", description="A 5,000 character chunk of the Markdown content.") # Corrected chunk size
current_page: int = Field(description="The current page number of the markdown chunk (1-indexed).") current_page: int = Field(description="The current page number of the markdown chunk (1-indexed).")
total_pages: int = Field(description="Total number of pages for the full markdown content.") total_pages: int = Field(description="Total number of pages for the full markdown content.")
is_paginated: bool = Field(description="True if the full markdown content is split into multiple pages.") is_paginated: bool = Field(description="True if the full markdown content is split into multiple pages.")
@@ -168,27 +146,27 @@ class AnayasaBireyselReportSearchRequest(BaseModel):
class AnayasaBireyselReportDecisionDetail(BaseModel): class AnayasaBireyselReportDecisionDetail(BaseModel):
"""Details of a specific right/claim within a Bireysel Başvuru decision summary in a report.""" """Details of a specific right/claim within a Bireysel Başvuru decision summary in a report."""
hak: Optional[str] = Field(None, description="İhlal edildiği iddia edilen hak (örneğin, Mülkiyet hakkı).") hak: str = Field("", description="İhlal edildiği iddia edilen hak (örneğin, Mülkiyet hakkı).")
mudahale_iddiasi: Optional[str] = Field(None, description="İhlale neden olan müdahale iddiası.") mudahale_iddiasi: str = Field("", description="İhlale neden olan müdahale iddiası.")
sonuc: Optional[str] = Field(None, description="İnceleme sonucu (örneğin, İhlal, Düşme).") sonuc: str = Field("", description="İnceleme sonucu (örneğin, İhlal, Düşme).")
giderim: Optional[str] = Field(None, description="Kararlaştırılan giderim (örneğin, Yeniden yargılama).") giderim: str = Field("", description="Kararlaştırılan giderim (örneğin, Yeniden yargılama).")
class AnayasaBireyselReportDecisionSummary(BaseModel): class AnayasaBireyselReportDecisionSummary(BaseModel):
"""Model for a single Anayasa Mahkemesi (Bireysel Başvuru) decision summary from a 'Karar Arama Raporu'.""" """Model for a single Anayasa Mahkemesi (Bireysel Başvuru) decision summary from a 'Karar Arama Raporu'."""
title: Optional[str] = Field(None, description="Başvurunun başlığı (e.g., 'HASAN DURMUŞ Başvurusuna İlişkin Karar').") title: str = Field("", description="Başvurunun başlığı (e.g., 'HASAN DURMUŞ Başvurusuna İlişkin Karar').")
decision_reference_no: Optional[str] = Field(None, description="Başvuru Numarası (e.g., '2019/19126').") decision_reference_no: str = Field("", description="Başvuru Numarası (e.g., '2019/19126').")
decision_page_url: Optional[HttpUrl] = Field(None, description="URL to the full decision page.") decision_page_url: str = Field("", description="URL to the full decision page.")
decision_type_summary: Optional[str] = Field(None, description="Karar Türü (Başvuru Sonucu) (e.g., 'Esas (İhlal)').") decision_type_summary: str = Field("", description="Karar Türü (Başvuru Sonucu) (e.g., 'Esas (İhlal)').")
decision_making_body: Optional[str] = Field(None, description="Kararı Veren Birim (e.g., 'Genel Kurul', 'Birinci Bölüm').") decision_making_body: str = Field("", description="Kararı Veren Birim (e.g., 'Genel Kurul', 'Birinci Bölüm').")
application_date_summary: Optional[str] = Field(None, description="Başvuru Tarihi (DD/MM/YYYY).") application_date_summary: str = Field("", description="Başvuru Tarihi (DD/MM/YYYY).")
decision_date_summary: Optional[str] = Field(None, description="Karar Tarihi (DD/MM/YYYY).") decision_date_summary: str = Field("", description="Karar Tarihi (DD/MM/YYYY).")
application_subject_summary: Optional[str] = Field(None, description="Başvuru konusunun özeti.") application_subject_summary: str = Field("", description="Başvuru konusunun özeti.")
details: List[AnayasaBireyselReportDecisionDetail] = Field(default_factory=list, description="İncelenen haklar ve sonuçlarına ilişkin detaylar.") details: List[AnayasaBireyselReportDecisionDetail] = Field(default_factory=list, description="İncelenen haklar ve sonuçlarına ilişkin detaylar.")
class AnayasaBireyselReportSearchResult(BaseModel): class AnayasaBireyselReportSearchResult(BaseModel):
"""Model for the overall search result for Anayasa Mahkemesi 'Karar Arama Raporu'.""" """Model for the overall search result for Anayasa Mahkemesi 'Karar Arama Raporu'."""
decisions: List[AnayasaBireyselReportDecisionSummary] decisions: List[AnayasaBireyselReportDecisionSummary]
total_records_found: Optional[int] = Field(None, description="Raporda bulunan toplam karar sayısı.") total_records_found: int = Field(0, description="Raporda bulunan toplam karar sayısı.")
retrieved_page_number: int = Field(description="Alınan rapor sayfa numarası.") retrieved_page_number: int = Field(description="Alınan rapor sayfa numarası.")
@@ -209,4 +187,44 @@ class AnayasaBireyselBasvuruDocumentMarkdown(BaseModel):
total_pages: int = Field(description="Total number of pages for the full markdown content.") total_pages: int = Field(description="Total number of pages for the full markdown content.")
is_paginated: bool = Field(description="True if the full markdown content is split into multiple pages.") is_paginated: bool = Field(description="True if the full markdown content is split into multiple pages.")
# --- End Models for Bireysel Başvuru --- # --- End Models for Bireysel Başvuru ---
# --- Unified Models ---
class AnayasaUnifiedSearchRequest(BaseModel):
"""Unified search request for both Norm Denetimi and Bireysel Başvuru."""
decision_type: Literal["norm_denetimi", "bireysel_basvuru"] = Field(..., description="Decision type: norm_denetimi or bireysel_basvuru")
# Common parameters
keywords: List[str] = Field(default_factory=list, description="Keywords to search for")
page_to_fetch: int = Field(1, ge=1, le=100, description="Page number to fetch (1-100)")
results_per_page: int = Field(10, ge=1, le=100, description="Results per page (1-100)")
# Norm Denetimi specific parameters (ignored for bireysel_basvuru)
keywords_all: List[str] = Field(default_factory=list, description="All keywords must be present (norm_denetimi only)")
keywords_any: List[str] = Field(default_factory=list, description="Any of these keywords (norm_denetimi only)")
decision_type_norm: Literal["ALL", "1", "2", "3"] = Field("ALL", description="Decision type for norm denetimi")
application_date_start: str = Field("", description="Application start date (norm_denetimi only)")
application_date_end: str = Field("", description="Application end date (norm_denetimi only)")
# Bireysel Başvuru specific parameters (ignored for norm_denetimi)
decision_start_date: str = Field("", description="Decision start date (bireysel_basvuru only)")
decision_end_date: str = Field("", description="Decision end date (bireysel_basvuru only)")
norm_type: Literal["ALL", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "0"] = Field("ALL", description="Norm type (bireysel_basvuru only)")
subject_category: str = Field("", description="Subject category (bireysel_basvuru only)")
class AnayasaUnifiedSearchResult(BaseModel):
"""Unified search result containing decisions from either system."""
decision_type: Literal["norm_denetimi", "bireysel_basvuru"] = Field(..., description="Type of decisions returned")
decisions: List[Dict[str, Any]] = Field(default_factory=list, description="Decision list (structure varies by type)")
total_records_found: int = Field(0, description="Total number of records found")
retrieved_page_number: int = Field(1, description="Page number that was retrieved")
class AnayasaUnifiedDocumentMarkdown(BaseModel):
"""Unified document model for both Norm Denetimi and Bireysel Başvuru."""
decision_type: Literal["norm_denetimi", "bireysel_basvuru"] = Field(..., description="Type of document")
source_url: HttpUrl = Field(..., description="Source URL of the document")
document_data: Dict[str, Any] = Field(default_factory=dict, description="Document content and metadata")
markdown_chunk: Optional[str] = Field(None, description="Markdown content chunk")
current_page: int = Field(1, description="Current page number")
total_pages: int = Field(1, description="Total number of pages")
is_paginated: bool = Field(False, description="Whether document is paginated")
+122
View File
@@ -0,0 +1,122 @@
# anayasa_mcp_module/unified_client.py
# Unified client for both Norm Denetimi and Bireysel Başvuru
import logging
from typing import Optional
from urllib.parse import urlparse
from .models import (
AnayasaUnifiedSearchRequest,
AnayasaUnifiedSearchResult,
AnayasaUnifiedDocumentMarkdown,
# Removed AnayasaDecisionTypeEnum - now using string literals
AnayasaNormDenetimiSearchRequest,
AnayasaBireyselReportSearchRequest
)
from .client import AnayasaMahkemesiApiClient
from .bireysel_client import AnayasaBireyselBasvuruApiClient
logger = logging.getLogger(__name__)
class AnayasaUnifiedClient:
"""Unified client that handles both Norm Denetimi and Bireysel Başvuru searches."""
def __init__(self, request_timeout: float = 60.0):
self.norm_client = AnayasaMahkemesiApiClient(request_timeout)
self.bireysel_client = AnayasaBireyselBasvuruApiClient(request_timeout)
async def search_unified(self, params: AnayasaUnifiedSearchRequest) -> AnayasaUnifiedSearchResult:
"""Unified search that routes to appropriate client based on decision_type."""
if params.decision_type == "norm_denetimi":
# Convert to norm denetimi request
norm_params = AnayasaNormDenetimiSearchRequest(
keywords_all=params.keywords_all or params.keywords,
keywords_any=params.keywords_any,
application_type=params.decision_type_norm,
page_to_fetch=params.page_to_fetch,
results_per_page=params.results_per_page
)
result = await self.norm_client.search_norm_denetimi_decisions(norm_params)
# Convert to unified format
decisions_list = [decision.model_dump() for decision in result.decisions]
return AnayasaUnifiedSearchResult(
decision_type="norm_denetimi",
decisions=decisions_list,
total_records_found=result.total_records_found,
retrieved_page_number=result.retrieved_page_number
)
elif params.decision_type == "bireysel_basvuru":
# Convert to bireysel başvuru request
bireysel_params = AnayasaBireyselReportSearchRequest(
keywords=params.keywords,
decision_start_date=params.decision_start_date,
decision_end_date=params.decision_end_date,
norm_type=params.norm_type,
subject_category=params.subject_category,
page_to_fetch=params.page_to_fetch,
results_per_page=params.results_per_page
)
result = await self.bireysel_client.search_bireysel_basvuru_report(bireysel_params)
# Convert to unified format
decisions_list = [decision.model_dump() for decision in result.decisions]
return AnayasaUnifiedSearchResult(
decision_type="bireysel_basvuru",
decisions=decisions_list,
total_records_found=result.total_records_found,
retrieved_page_number=result.retrieved_page_number
)
else:
raise ValueError(f"Unsupported decision type: {params.decision_type}")
async def get_document_unified(self, document_url: str, page_number: int = 1) -> AnayasaUnifiedDocumentMarkdown:
"""Unified document retrieval that auto-detects the appropriate client."""
# Auto-detect decision type based on URL
parsed_url = urlparse(document_url)
if "normkararlarbilgibankasi" in parsed_url.netloc or "/ND/" in document_url:
# Norm Denetimi document
result = await self.norm_client.get_decision_document_as_markdown(document_url, page_number)
return AnayasaUnifiedDocumentMarkdown(
decision_type="norm_denetimi",
source_url=result.source_url,
document_data=result.model_dump(),
markdown_chunk=result.markdown_chunk,
current_page=result.current_page,
total_pages=result.total_pages,
is_paginated=result.is_paginated
)
elif "kararlarbilgibankasi" in parsed_url.netloc or "/BB/" in document_url:
# Bireysel Başvuru document
result = await self.bireysel_client.get_decision_document_as_markdown(document_url, page_number)
return AnayasaUnifiedDocumentMarkdown(
decision_type="bireysel_basvuru",
source_url=result.source_url,
document_data=result.model_dump(),
markdown_chunk=result.markdown_chunk,
current_page=result.current_page,
total_pages=result.total_pages,
is_paginated=result.is_paginated
)
else:
raise ValueError(f"Cannot determine document type from URL: {document_url}")
async def close_client_session(self):
"""Close both client sessions."""
if hasattr(self.norm_client, 'close_client_session'):
await self.norm_client.close_client_session()
if hasattr(self.bireysel_client, 'close_client_session'):
await self.bireysel_client.close_client_session()
+380 -35
View File
@@ -10,8 +10,11 @@ Usage:
""" """
import os import os
from fastapi import FastAPI, Request, HTTPException import time
from fastapi.responses import JSONResponse import logging
from datetime import datetime, timedelta
from fastapi import FastAPI, Request, HTTPException, Query
from fastapi.responses import JSONResponse, HTMLResponse
from fastapi.exception_handlers import http_exception_handler from fastapi.exception_handlers import http_exception_handler
from starlette.middleware import Middleware from starlette.middleware import Middleware
from starlette.middleware.cors import CORSMiddleware from starlette.middleware.cors import CORSMiddleware
@@ -23,13 +26,16 @@ from mcp_server_main import app as mcp_server
# Import Stripe webhook router # Import Stripe webhook router
from stripe_webhook import router as stripe_router from stripe_webhook import router as stripe_router
# Import MCP Auth HTTP adapter # Import simplified MCP Auth HTTP adapter
from mcp_auth_http_adapter import router as mcp_auth_router from mcp_auth_http_simple import router as mcp_auth_router
# OAuth configuration from environment variables # OAuth configuration from environment variables
CLERK_ISSUER = os.getenv("CLERK_ISSUER", "https://accounts.yargimcp.com") CLERK_ISSUER = os.getenv("CLERK_ISSUER", "https://accounts.yargimcp.com")
BASE_URL = os.getenv("BASE_URL", "https://yargimcp.com") BASE_URL = os.getenv("BASE_URL", "https://yargimcp.com")
# Setup logging
logger = logging.getLogger(__name__)
# Configure CORS middleware # Configure CORS middleware
cors_origins = os.getenv("ALLOWED_ORIGINS", "*").split(",") cors_origins = os.getenv("ALLOWED_ORIGINS", "*").split(",")
custom_middleware = [ custom_middleware = [
@@ -42,25 +48,46 @@ custom_middleware = [
), ),
] ]
# Create MCP Starlette sub-application first # Create MCP Starlette sub-application (without auth wrapper)
mcp_app = mcp_server.http_app( mcp_app = mcp_server.http_app(
path="/", path="/",
middleware=custom_middleware middleware=custom_middleware
) )
# Create FastAPI wrapper application with MCP app's lifespan # Configure JSON encoder for proper Turkish character support
import json
from fastapi.responses import JSONResponse
class UTF8JSONResponse(JSONResponse):
def __init__(self, content=None, status_code=200, headers=None, **kwargs):
if headers is None:
headers = {}
headers["Content-Type"] = "application/json; charset=utf-8"
super().__init__(content, status_code, headers, **kwargs)
def render(self, content) -> bytes:
return json.dumps(
content,
ensure_ascii=False,
allow_nan=False,
indent=None,
separators=(",", ":"),
).encode("utf-8")
# Create FastAPI wrapper application with MCP lifespan
app = FastAPI( app = FastAPI(
title="Yargı MCP Server", title="Yargı MCP Server",
description="MCP server for Turkish legal databases with OAuth authentication", description="MCP server for Turkish legal databases with OAuth authentication",
version="0.1.0", version="0.1.0",
middleware=custom_middleware, middleware=custom_middleware,
lifespan=mcp_app.lifespan # Critical: Get lifespan from mcp_app, not mcp_server lifespan=mcp_app.lifespan, # MCP app lifespan
default_response_class=UTF8JSONResponse # Use UTF-8 JSON encoder
) )
# Add Stripe webhook router to FastAPI # Add Stripe webhook router to FastAPI
app.include_router(stripe_router, prefix="/api") app.include_router(stripe_router, prefix="/api")
# Add MCP Auth HTTP adapter to FastAPI (replaces old OAuth router) # Add MCP Auth HTTP adapter to FastAPI (handles OAuth endpoints)
app.include_router(mcp_auth_router) app.include_router(mcp_auth_router)
# Custom 401 exception handler for MCP spec compliance # Custom 401 exception handler for MCP spec compliance
@@ -80,23 +107,110 @@ async def custom_401_handler(request: Request, exc: HTTPException):
return response return response
# Mount MCP app as sub-application # Mount MCP app as sub-application at /mcp-server to avoid path conflicts
app.mount("/mcp", mcp_app) app.mount("/mcp-server", mcp_app)
# Add POST handler for /mcp to forward to mounted app # Add custom route to handle /mcp requests and forward to mounted app
@app.post("/mcp") @app.api_route("/mcp", methods=["POST", "DELETE", "OPTIONS"])
async def mcp_post_handler(request: Request): @app.api_route("/mcp/", methods=["POST", "DELETE", "OPTIONS"])
"""Forward POST /mcp requests to mounted MCP app""" async def mcp_protocol_handler(request: Request):
# Forward to the mounted app by calling it directly """Handle MCP protocol requests by forwarding to mounted app"""
# Handle DELETE requests for session termination
if request.method == "DELETE":
logger.info("DELETE request received for session termination")
# For session termination, we just return 200 OK
# The actual session cleanup is handled by the underlying MCP transport
from starlette.responses import Response
return Response(
status_code=200,
content="Session terminated successfully"
)
# REQUIRED: Validate Bearer JWT tokens for all MCP requests
auth_header = request.headers.get("Authorization")
if not auth_header or not auth_header.startswith("Bearer "):
logger.error("Missing or invalid Authorization header")
raise HTTPException(
status_code=401,
detail="Missing or invalid Authorization header. Bearer token required."
)
token = auth_header.split(" ")[1]
try:
# Check if this is a mock token for development/testing
if token.startswith("mock_clerk_jwt_"):
logger.info(f"Using mock JWT token for development: {token[:30]}...")
# For mock tokens, we'll allow access with a mock user
request.state.user_id = "mock_user_dev"
request.state.session_id = "mock_session_dev"
request.state.token_scopes = ["read", "search"]
logger.info("Mock JWT token accepted for development")
elif token.startswith("eyJ"):
# This looks like a real JWT token (starts with eyJ which is base64 encoded '{"')
logger.info(f"Processing real JWT token: {token[:30]}...")
# Validate real Clerk JWT token
from clerk_backend_api import Clerk, models
import jwt
# Decode JWT token and extract user info
try:
decoded_token = jwt.decode(token, options={"verify_signature": False})
user_id = decoded_token.get("user_id") or decoded_token.get("sub")
user_email = decoded_token.get("email")
token_scopes = decoded_token.get("scopes", ["read", "search"])
session_id = decoded_token.get("sid", "jwt_session")
logger.info(f"JWT token claims - user_id: {user_id}, email: {user_email}, scopes: {token_scopes}")
if user_id and user_email:
# JWT token is signed by Clerk and contains valid user info
request.state.user_id = user_id
request.state.user_email = user_email
request.state.session_id = session_id
request.state.token_scopes = token_scopes
logger.info(f"Real JWT token accepted for user: {user_id}")
else:
logger.error(f"Missing required fields in JWT token - user_id: {bool(user_id)}, email: {bool(user_email)}")
raise HTTPException(
status_code=401,
detail="Invalid token - missing user_id or email in claims"
)
except Exception as e:
logger.error(f"JWT token decoding failed: {e}")
raise HTTPException(
status_code=401,
detail="Invalid JWT token format"
)
else:
# Invalid token format - doesn't start with expected patterns
logger.error(f"Invalid token format: {token[:30]}...")
raise HTTPException(
status_code=401,
detail="Invalid token format - must be a valid JWT token"
)
except HTTPException:
# Re-raise HTTPException as-is
raise
except Exception as e:
logger.error(f"Bearer token validation failed: {str(e)}")
raise HTTPException(
status_code=401,
detail=f"Token validation failed: {str(e)}"
)
# Forward the request to the mounted MCP app
async def receive(): async def receive():
return await request.receive() return await request.receive()
# Create a new scope for the mounted app # Create new scope for the mounted app
scope = request.scope.copy() scope = request.scope.copy()
scope["path"] = "/" # Root path for the mounted app scope["path"] = "/" # Root path for mounted app
scope["path_info"] = "/" scope["path_info"] = "/"
# Capture response # Capture the response
response_parts = {"status": 200, "headers": [], "body": b""} response_parts = {"status": 200, "headers": [], "body": b""}
async def send(message): async def send(message):
@@ -124,6 +238,9 @@ async def mcp_post_handler(request: Request):
) )
# SSE transport deprecated - removed
# FastAPI health check endpoint # FastAPI health check endpoint
@app.get("/health") @app.get("/health")
async def health_check(): async def health_check():
@@ -153,6 +270,9 @@ async def root():
"oauth_google": "/auth/google/login", "oauth_google": "/auth/google/login",
"user_info": "/auth/user" "user_info": "/auth/user"
}, },
"transports": {
"http": "/mcp"
},
"supported_databases": [ "supported_databases": [
"Yargıtay (Court of Cassation)", "Yargıtay (Court of Cassation)",
"Danıştay (Council of State)", "Danıştay (Council of State)",
@@ -174,13 +294,14 @@ async def root():
}) })
# OAuth 2.0 Authorization Server Metadata proxy (for MCP clients that can't reach Clerk directly) # OAuth 2.0 Authorization Server Metadata proxy (for MCP clients that can't reach Clerk directly)
@app.get("/.well-known/oauth-authorization-server") # MCP Auth Toolkit expects this to be under /mcp/.well-known/oauth-authorization-server
@app.get("/mcp/.well-known/oauth-authorization-server")
async def oauth_authorization_server(): async def oauth_authorization_server():
"""OAuth 2.0 Authorization Server Metadata proxy to Clerk""" """OAuth 2.0 Authorization Server Metadata proxy to Clerk - MCP Auth Toolkit standard location"""
return JSONResponse({ return JSONResponse({
"issuer": CLERK_ISSUER, "issuer": BASE_URL,
"authorization_endpoint": f"{BASE_URL}/auth/login", "authorization_endpoint": "https://yargimcp.com/mcp-callback",
"token_endpoint": f"{BASE_URL}/auth/callback", "token_endpoint": f"{BASE_URL}/token",
"jwks_uri": f"{CLERK_ISSUER}/.well-known/jwks.json", "jwks_uri": f"{CLERK_ISSUER}/.well-known/jwks.json",
"response_types_supported": ["code"], "response_types_supported": ["code"],
"grant_types_supported": ["authorization_code", "refresh_token"], "grant_types_supported": ["authorization_code", "refresh_token"],
@@ -191,7 +312,65 @@ async def oauth_authorization_server():
"claims_supported": ["sub", "iss", "aud", "exp", "iat", "email", "name"], "claims_supported": ["sub", "iss", "aud", "exp", "iat", "email", "name"],
"code_challenge_methods_supported": ["S256"], "code_challenge_methods_supported": ["S256"],
"service_documentation": f"{BASE_URL}/mcp", "service_documentation": f"{BASE_URL}/mcp",
"registration_endpoint": f"{BASE_URL}/auth/register", "registration_endpoint": f"{BASE_URL}/register",
"resource_documentation": f"{BASE_URL}/mcp"
})
# Claude AI MCP specific endpoint format
@app.get("/.well-known/oauth-authorization-server/mcp")
async def oauth_authorization_server_mcp_suffix():
"""OAuth 2.0 Authorization Server Metadata - Claude AI MCP specific format"""
return JSONResponse({
"issuer": BASE_URL,
"authorization_endpoint": "https://yargimcp.com/mcp-callback",
"token_endpoint": f"{BASE_URL}/token",
"jwks_uri": f"{CLERK_ISSUER}/.well-known/jwks.json",
"response_types_supported": ["code"],
"grant_types_supported": ["authorization_code", "refresh_token"],
"token_endpoint_auth_methods_supported": ["client_secret_basic", "none"],
"scopes_supported": ["read", "search", "openid", "profile", "email"],
"subject_types_supported": ["public"],
"id_token_signing_alg_values_supported": ["RS256"],
"claims_supported": ["sub", "iss", "aud", "exp", "iat", "email", "name"],
"code_challenge_methods_supported": ["S256"],
"service_documentation": f"{BASE_URL}/mcp",
"registration_endpoint": f"{BASE_URL}/register",
"resource_documentation": f"{BASE_URL}/mcp"
})
@app.get("/.well-known/oauth-protected-resource/mcp")
async def oauth_protected_resource_mcp_suffix():
"""OAuth 2.0 Protected Resource Metadata - Claude AI MCP specific format"""
return JSONResponse({
"resource": BASE_URL,
"authorization_servers": [
BASE_URL
],
"scopes_supported": ["read", "search"],
"bearer_methods_supported": ["header"],
"resource_documentation": f"{BASE_URL}/mcp",
"resource_policy_uri": f"{BASE_URL}/privacy"
})
# Keep root level for compatibility with some MCP clients
@app.get("/.well-known/oauth-authorization-server")
async def oauth_authorization_server_root():
"""OAuth 2.0 Authorization Server Metadata proxy to Clerk - root level for compatibility"""
return JSONResponse({
"issuer": BASE_URL,
"authorization_endpoint": "https://yargimcp.com/mcp-callback",
"token_endpoint": f"{BASE_URL}/token",
"jwks_uri": f"{CLERK_ISSUER}/.well-known/jwks.json",
"response_types_supported": ["code"],
"grant_types_supported": ["authorization_code", "refresh_token"],
"token_endpoint_auth_methods_supported": ["client_secret_basic", "none"],
"scopes_supported": ["read", "search", "openid", "profile", "email"],
"subject_types_supported": ["public"],
"id_token_signing_alg_values_supported": ["RS256"],
"claims_supported": ["sub", "iss", "aud", "exp", "iat", "email", "name"],
"code_challenge_methods_supported": ["S256"],
"service_documentation": f"{BASE_URL}/mcp",
"registration_endpoint": f"{BASE_URL}/register",
"resource_documentation": f"{BASE_URL}/mcp" "resource_documentation": f"{BASE_URL}/mcp"
}) })
@@ -205,12 +384,12 @@ async def mcp_info():
"version": "0.1.0", "version": "0.1.0",
"description": "MCP server for Turkish legal databases", "description": "MCP server for Turkish legal databases",
"protocol": "mcp/1.0", "protocol": "mcp/1.0",
"transport": "http", "transport": ["http"],
"authentication_required": True, "authentication_required": True,
"authentication": { "authentication": {
"type": "oauth2", "type": "oauth2",
"authorization_url": f"{BASE_URL}/auth/login", "authorization_url": "https://yargimcp.com/sign-in?redirect_url=https://api.yargimcp.com/auth/mcp-callback",
"token_url": f"{BASE_URL}/auth/callback", "token_url": f"{BASE_URL}/auth/mcp-token",
"scopes": ["read", "search"], "scopes": ["read", "search"],
"provider": "clerk" "provider": "clerk"
}, },
@@ -231,7 +410,7 @@ async def mcp_info():
"note": "This is an MCP server. Use POST to /mcp/ with proper MCP protocol headers.", "note": "This is an MCP server. Use POST to /mcp/ with proper MCP protocol headers.",
"headers_required": [ "headers_required": [
"Content-Type: application/json", "Content-Type: application/json",
"Accept: application/json, text/event-stream", "Accept: application/json",
"Authorization: Bearer <token>", "Authorization: Bearer <token>",
"X-Session-ID: <session-id>" "X-Session-ID: <session-id>"
] ]
@@ -322,12 +501,178 @@ async def status():
"auth_status": "enabled" if os.getenv("ENABLE_AUTH", "false").lower() == "true" else "disabled" "auth_status": "enabled" if os.getenv("ENABLE_AUTH", "false").lower() == "true" else "disabled"
}) })
# Alternative: SSE transport (for compatibility) # Note: JWT token validation is now handled entirely by Clerk
sse_app = mcp_server.http_app( # All authentication flows use Clerk JWT tokens directly
path="/sse",
transport="sse", async def validate_clerk_session(request: Request, clerk_token: str = None) -> str:
middleware=custom_middleware """Validate Clerk session from cookies or JWT token and return user_id"""
) logger.info(f"Validating Clerk session - token provided: {bool(clerk_token)}")
try:
# Try to import Clerk SDK
from clerk_backend_api import Clerk
clerk = Clerk(bearer_auth=os.getenv("CLERK_SECRET_KEY"))
# Try JWT token first (from URL parameter)
if clerk_token:
logger.info("Validating Clerk JWT token from URL parameter")
try:
# Extract session_id from JWT token and verify with Clerk
import jwt
decoded_token = jwt.decode(clerk_token, options={"verify_signature": False})
session_id = decoded_token.get("sid") # Use standard JWT 'sid' claim
if session_id:
# Verify with Clerk using session_id
session = clerk.sessions.verify(session_id=session_id, token=clerk_token)
user_id = session.user_id if session else None
if user_id:
logger.info(f"JWT token validation successful - user_id: {user_id}")
return user_id
else:
logger.error("JWT token validation failed - no user_id in session")
else:
logger.error("No session_id found in JWT token")
except Exception as e:
logger.error(f"JWT token validation failed: {str(e)}")
# Fall through to cookie validation
# Fallback to cookie validation
logger.info("Attempting cookie-based session validation")
clerk_session = request.cookies.get("__session")
if not clerk_session:
logger.error("No Clerk session cookie found")
raise HTTPException(status_code=401, detail="No Clerk session found")
# Validate session with Clerk
session = clerk.sessions.verify_session(clerk_session)
logger.info(f"Cookie session validation successful - user_id: {session.user_id}")
return session.user_id
except ImportError:
# Fallback for development without Clerk SDK
logger.warning("Clerk SDK not available - using development fallback")
return "dev_user_123"
except Exception as e:
logger.error(f"Session validation failed: {str(e)}")
raise HTTPException(status_code=401, detail=f"Session validation failed: {str(e)}")
# MCP OAuth Callback Endpoint
@app.get("/auth/mcp-callback")
async def mcp_oauth_callback(request: Request, clerk_token: str = Query(None)):
"""Handle OAuth callback for MCP token generation"""
logger.info(f"MCP OAuth callback - clerk_token provided: {bool(clerk_token)}")
try:
# Validate Clerk session with JWT token support
user_id = await validate_clerk_session(request, clerk_token)
logger.info(f"User authenticated successfully - user_id: {user_id}")
# Use the Clerk JWT token directly (no need to generate custom token)
logger.info("User authenticated successfully via Clerk")
# Return success response
return HTMLResponse(f"""
<html>
<head>
<title>MCP Connection Successful</title>
<style>
body {{ font-family: Arial, sans-serif; text-align: center; padding: 50px; }}
.success {{ color: #28a745; }}
.token {{ background: #f8f9fa; padding: 15px; border-radius: 5px; margin: 20px 0; word-break: break-all; }}
</style>
</head>
<body>
<h1 class="success">✅ MCP Connection Successful!</h1>
<p>Your Yargı MCP integration is now active.</p>
<div class="token">
<strong>Authentication:</strong><br>
<code>Use your Clerk JWT token directly with Bearer authentication</code>
</div>
<p>You can now close this window and return to your MCP client.</p>
<script>
// Try to close the popup if opened as such
if (window.opener) {{
window.opener.postMessage({{
type: 'MCP_AUTH_SUCCESS',
token: 'use_clerk_jwt_token'
}}, '*');
setTimeout(() => window.close(), 3000);
}}
</script>
</body>
</html>
""")
except HTTPException as e:
logger.error(f"MCP OAuth callback failed: {e.detail}")
return HTMLResponse(f"""
<html>
<head>
<title>MCP Connection Failed</title>
<style>
body {{ font-family: Arial, sans-serif; text-align: center; padding: 50px; }}
.error {{ color: #dc3545; }}
.debug {{ background: #f8f9fa; padding: 10px; margin: 20px 0; border-radius: 5px; font-family: monospace; }}
</style>
</head>
<body>
<h1 class="error">❌ MCP Connection Failed</h1>
<p>{e.detail}</p>
<div class="debug">
<strong>Debug Info:</strong><br>
Clerk Token: {'✅ Provided' if clerk_token else '❌ Missing'}<br>
Error: {e.detail}<br>
Status: {e.status_code}
</div>
<p>Please try again or contact support.</p>
<a href="https://yargimcp.com/sign-in">Return to Sign In</a>
</body>
</html>
""", status_code=e.status_code)
except Exception as e:
logger.error(f"Unexpected error in MCP OAuth callback: {str(e)}")
return HTMLResponse(f"""
<html>
<head>
<title>MCP Connection Error</title>
<style>
body {{ font-family: Arial, sans-serif; text-align: center; padding: 50px; }}
.error {{ color: #dc3545; }}
</style>
</head>
<body>
<h1 class="error">❌ Unexpected Error</h1>
<p>An unexpected error occurred during authentication.</p>
<p>Error: {str(e)}</p>
<a href="https://yargimcp.com/sign-in">Return to Sign In</a>
</body>
</html>
""", status_code=500)
# OAuth2 Token Endpoint - Now uses Clerk JWT tokens directly
@app.post("/auth/mcp-token")
async def mcp_token_endpoint(request: Request):
"""OAuth2 token endpoint for MCP clients - returns Clerk JWT token info"""
try:
# Validate Clerk session
user_id = await validate_clerk_session(request)
return JSONResponse({
"message": "Use your Clerk JWT token directly with Bearer authentication",
"token_type": "Bearer",
"scope": "yargi.read",
"user_id": user_id,
"instructions": "Include 'Authorization: Bearer YOUR_CLERK_JWT_TOKEN' in your requests"
})
except HTTPException as e:
return JSONResponse(
status_code=e.status_code,
content={"error": "invalid_request", "error_description": e.detail}
)
# Note: Only HTTP transport supported - SSE transport deprecated
# Export for uvicorn # Export for uvicorn
__all__ = ["app", "sse_app"] __all__ = ["app"]
+17
View File
@@ -0,0 +1,17 @@
# bddk_mcp_module/__init__.py
from .client import BddkApiClient
from .models import (
BddkSearchRequest,
BddkDecisionSummary,
BddkSearchResult,
BddkDocumentMarkdown
)
__all__ = [
"BddkApiClient",
"BddkSearchRequest",
"BddkDecisionSummary",
"BddkSearchResult",
"BddkDocumentMarkdown"
]
+247
View File
@@ -0,0 +1,247 @@
# bddk_mcp_module/client.py
import httpx
from typing import List, Optional, Dict, Any
import logging
import os
import re
import io
import math
from urllib.parse import urlparse
from markitdown import MarkItDown
from .models import (
BddkSearchRequest,
BddkDecisionSummary,
BddkSearchResult,
BddkDocumentMarkdown
)
logger = logging.getLogger(__name__)
if not logger.hasHandlers():
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
class BddkApiClient:
"""
API client for searching and retrieving BDDK (Banking Regulation Authority) decisions
using Tavily Search API for discovery and direct HTTP requests for content retrieval.
"""
TAVILY_API_URL = "https://api.tavily.com/search"
BDDK_BASE_URL = "https://www.bddk.org.tr"
DOCUMENT_URL_TEMPLATE = "https://www.bddk.org.tr/Mevzuat/DokumanGetir/{document_id}"
DOCUMENT_MARKDOWN_CHUNK_SIZE = 5000 # Character limit per page
def __init__(self, request_timeout: float = 60.0):
"""Initialize the BDDK API client."""
self.tavily_api_key = os.getenv("TAVILY_API_KEY")
if not self.tavily_api_key:
# Fallback to development token
self.tavily_api_key = "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"
logger.info("Using fallback Tavily API token (development token)")
else:
logger.info("Using Tavily API key from environment variable")
self.http_client = httpx.AsyncClient(
headers={
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36"
},
timeout=httpx.Timeout(request_timeout)
)
self.markitdown = MarkItDown()
async def close_client_session(self):
"""Close the HTTP client session."""
await self.http_client.aclose()
logger.info("BddkApiClient: HTTP client session closed.")
def _extract_document_id(self, url: str) -> Optional[str]:
"""Extract document ID from BDDK URL."""
# Primary pattern: https://www.bddk.org.tr/Mevzuat/DokumanGetir/310
match = re.search(r'/DokumanGetir/(\d+)', url)
if match:
return match.group(1)
# Alternative patterns for different BDDK URL formats
# Pattern: /Liste/55 -> use as document ID
match = re.search(r'/Liste/(\d+)', url)
if match:
return match.group(1)
# Pattern: /EkGetir/13?ekId=381 -> use ekId as document ID
match = re.search(r'ekId=(\d+)', url)
if match:
return match.group(1)
return None
async def search_decisions(
self,
request: BddkSearchRequest
) -> BddkSearchResult:
"""
Search for BDDK decisions using Tavily API.
Args:
request: Search request parameters
Returns:
BddkSearchResult with matching decisions
"""
try:
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {self.tavily_api_key}"
}
# Tavily API request - enhanced for BDDK decision documents
query = f"{request.keywords} \"Karar Sayısı\""
payload = {
"query": query,
"country": "turkey",
"include_domains": ["https://www.bddk.org.tr/Mevzuat/DokumanGetir"],
"max_results": request.pageSize,
"search_depth": "advanced"
}
# Calculate offset for pagination
if request.page > 1:
# Tavily doesn't have direct pagination, so we'll need to handle this
# For now, we'll just return empty for pages > 1
logger.warning(f"Tavily API doesn't support pagination. Page {request.page} requested.")
response = await self.http_client.post(
self.TAVILY_API_URL,
json=payload,
headers=headers
)
response.raise_for_status()
data = response.json()
# Log raw Tavily response for debugging
logger.info(f"Tavily returned {len(data.get('results', []))} results")
# Convert Tavily results to our format
decisions = []
for result in data.get("results", []):
# Extract document ID from URL
url = result.get("url", "")
logger.debug(f"Processing URL: {url}")
doc_id = self._extract_document_id(url)
if doc_id:
decision = BddkDecisionSummary(
title=result.get("title", "").replace("[PDF] ", "").strip(),
document_id=doc_id,
content=result.get("content", "")[:500] # Limit content length
)
decisions.append(decision)
logger.debug(f"Added decision: {decision.title} (ID: {doc_id})")
else:
logger.warning(f"Could not extract document ID from URL: {url}")
return BddkSearchResult(
decisions=decisions,
total_results=len(data.get("results", [])),
page=request.page,
pageSize=request.pageSize
)
except httpx.HTTPStatusError as e:
logger.error(f"HTTP error searching BDDK decisions: {e}")
if e.response.status_code == 401:
raise Exception("Tavily API authentication failed. Check API key.")
raise Exception(f"Failed to search BDDK decisions: {str(e)}")
except Exception as e:
logger.error(f"Error searching BDDK decisions: {e}")
raise Exception(f"Failed to search BDDK decisions: {str(e)}")
async def get_document_markdown(
self,
document_id: str,
page_number: int = 1
) -> BddkDocumentMarkdown:
"""
Retrieve a BDDK document and convert it to Markdown format.
Args:
document_id: BDDK document ID (e.g., '310')
page_number: Page number for paginated content (1-indexed)
Returns:
BddkDocumentMarkdown with paginated content
"""
try:
# Try different URL patterns for BDDK documents
potential_urls = [
f"https://www.bddk.org.tr/Mevzuat/DokumanGetir/{document_id}",
f"https://www.bddk.org.tr/Mevzuat/Liste/{document_id}",
f"https://www.bddk.org.tr/KurumHakkinda/EkGetir/13?ekId={document_id}",
f"https://www.bddk.org.tr/KurumHakkinda/EkGetir/5?ekId={document_id}"
]
document_url = None
response = None
# Try each URL pattern until one works
for url in potential_urls:
try:
logger.info(f"Trying BDDK document URL: {url}")
response = await self.http_client.get(
url,
follow_redirects=True
)
response.raise_for_status()
document_url = url
break
except httpx.HTTPStatusError:
continue
if not response or not document_url:
raise Exception(f"Could not find document with ID {document_id}")
logger.info(f"Successfully fetched BDDK document from: {document_url}")
# Determine content type
content_type = response.headers.get("content-type", "").lower()
# Convert to Markdown based on content type
if "pdf" in content_type:
# Handle PDF documents
pdf_stream = io.BytesIO(response.content)
result = self.markitdown.convert_stream(pdf_stream, file_extension=".pdf")
markdown_content = result.text_content
else:
# Handle HTML documents
html_stream = io.BytesIO(response.content)
result = self.markitdown.convert_stream(html_stream, file_extension=".html")
markdown_content = result.text_content
# Clean up the markdown content
markdown_content = markdown_content.strip()
# Calculate pagination
total_length = len(markdown_content)
total_pages = math.ceil(total_length / self.DOCUMENT_MARKDOWN_CHUNK_SIZE)
# Extract the requested page
start_idx = (page_number - 1) * self.DOCUMENT_MARKDOWN_CHUNK_SIZE
end_idx = start_idx + self.DOCUMENT_MARKDOWN_CHUNK_SIZE
page_content = markdown_content[start_idx:end_idx]
return BddkDocumentMarkdown(
document_id=document_id,
markdown_content=page_content,
page_number=page_number,
total_pages=total_pages
)
except httpx.HTTPStatusError as e:
logger.error(f"HTTP error fetching BDDK document {document_id}: {e}")
raise Exception(f"Failed to fetch BDDK document: {str(e)}")
except Exception as e:
logger.error(f"Error processing BDDK document {document_id}: {e}")
raise Exception(f"Failed to process BDDK document: {str(e)}")
+43
View File
@@ -0,0 +1,43 @@
# bddk_mcp_module/models.py
from pydantic import BaseModel, Field
from typing import List, Optional
class BddkSearchRequest(BaseModel):
"""
Request model for searching BDDK decisions via Tavily API.
BDDK (Bankacılık Düzenleme ve Denetleme Kurumu) is Turkey's Banking
Regulation and Supervision Agency responsible for banking licenses,
electronic money institutions, and financial regulations.
"""
keywords: str = Field(..., description="Search keywords in Turkish")
page: int = Field(1, ge=1, description="Page number (1-indexed)")
pageSize: int = Field(10, ge=1, le=50, description="Results per page (1-50)")
class BddkDecisionSummary(BaseModel):
"""Summary of a BDDK decision from search results."""
title: str = Field(..., description="Decision title")
document_id: str = Field(..., description="BDDK document ID (e.g., '310')")
content: str = Field(..., description="Decision summary/excerpt")
class BddkSearchResult(BaseModel):
"""Response model for BDDK decision search results."""
decisions: List[BddkDecisionSummary] = Field(
default_factory=list,
description="List of matching BDDK decisions"
)
total_results: int = Field(0, description="Total number of results")
page: int = Field(1, description="Current page number")
pageSize: int = Field(10, description="Results per page")
class BddkDocumentMarkdown(BaseModel):
"""
BDDK decision document converted to Markdown format.
Supports paginated content for long documents (5000 chars per page).
"""
document_id: str = Field(..., description="BDDK document ID")
markdown_content: str = Field("", description="Document content in Markdown")
page_number: int = Field(1, description="Current page number")
total_pages: int = Field(1, description="Total number of pages")
+26 -28
View File
@@ -5,14 +5,14 @@ import base64
from typing import Optional from typing import Optional
import logging import logging
from markitdown import MarkItDown from markitdown import MarkItDown
import tempfile import io
import os
from .models import ( from .models import (
BedestenSearchRequest, BedestenSearchResponse, BedestenSearchRequest, BedestenSearchResponse,
BedestenDocumentRequest, BedestenDocumentResponse, BedestenDocumentRequest, BedestenDocumentResponse,
BedestenDocumentMarkdown, BedestenDocumentRequestData BedestenDocumentMarkdown, BedestenDocumentRequestData
) )
from .enums import get_full_birim_adi
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -50,10 +50,22 @@ class BedestenApiClient:
""" """
logger.info(f"BedestenApiClient: Searching documents with phrase: {search_request.data.phrase}") logger.info(f"BedestenApiClient: Searching documents with phrase: {search_request.data.phrase}")
# Map abbreviated birimAdi to full Turkish name before sending to API
original_birim_adi = search_request.data.birimAdi
mapped_birim_adi = get_full_birim_adi(original_birim_adi)
search_request.data.birimAdi = mapped_birim_adi
if original_birim_adi != "ALL":
logger.info(f"BedestenApiClient: Mapped birimAdi '{original_birim_adi}' to '{mapped_birim_adi}'")
try: try:
# Create request dict and remove birimAdi if empty
request_dict = search_request.model_dump()
if not request_dict["data"]["birimAdi"]: # Remove if empty string
del request_dict["data"]["birimAdi"]
response = await self.http_client.post( response = await self.http_client.post(
self.SEARCH_ENDPOINT, self.SEARCH_ENDPOINT,
json=search_request.model_dump() json=request_dict
) )
response.raise_for_status() response.raise_for_status()
response_json = response.json() response_json = response.json()
@@ -125,17 +137,14 @@ class BedestenApiClient:
if not html_content: if not html_content:
return None return None
temp_file_path = None
try: try:
# Convert HTML string to bytes and create BytesIO stream
html_bytes = html_content.encode('utf-8')
html_stream = io.BytesIO(html_bytes)
# Pass BytesIO stream to MarkItDown to avoid temp file creation
md_converter = MarkItDown() md_converter = MarkItDown()
result = md_converter.convert(html_stream)
# Write HTML to temp file
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp:
tmp.write(html_content)
temp_file_path = tmp.name
# Convert
result = md_converter.convert(temp_file_path)
markdown_content = result.text_content markdown_content = result.text_content
logger.info("Successfully converted HTML to Markdown") logger.info("Successfully converted HTML to Markdown")
@@ -144,27 +153,19 @@ class BedestenApiClient:
except Exception as e: except Exception as e:
logger.error(f"Error converting HTML to Markdown: {e}") logger.error(f"Error converting HTML to Markdown: {e}")
return f"Error converting HTML content: {str(e)}" return f"Error converting HTML content: {str(e)}"
finally:
if temp_file_path and os.path.exists(temp_file_path):
os.remove(temp_file_path)
def _convert_pdf_to_markdown(self, pdf_bytes: bytes) -> Optional[str]: def _convert_pdf_to_markdown(self, pdf_bytes: bytes) -> Optional[str]:
"""Convert PDF to Markdown using MarkItDown""" """Convert PDF to Markdown using MarkItDown"""
if not pdf_bytes: if not pdf_bytes:
return None return None
temp_file_path = None
try: try:
# MarkItDown supports PDF with markitdown[pdf] # Create BytesIO stream from PDF bytes
pdf_stream = io.BytesIO(pdf_bytes)
# Pass BytesIO stream to MarkItDown to avoid temp file creation
md_converter = MarkItDown() md_converter = MarkItDown()
result = md_converter.convert(pdf_stream)
# Write PDF to temp file
with tempfile.NamedTemporaryFile(mode="wb", delete=False, suffix=".pdf") as tmp:
tmp.write(pdf_bytes)
temp_file_path = tmp.name
# Convert
result = md_converter.convert(temp_file_path)
markdown_content = result.text_content markdown_content = result.text_content
logger.info("Successfully converted PDF to Markdown") logger.info("Successfully converted PDF to Markdown")
@@ -173,9 +174,6 @@ class BedestenApiClient:
except Exception as e: except Exception as e:
logger.error(f"Error converting PDF to Markdown: {e}") logger.error(f"Error converting PDF to Markdown: {e}")
return f"Error converting PDF content: {str(e)}. The document may be corrupted or in an unsupported format." return f"Error converting PDF content: {str(e)}. The document may be corrupted or in an unsupported format."
finally:
if temp_file_path and os.path.exists(temp_file_path):
os.remove(temp_file_path)
async def close_client_session(self): async def close_client_session(self):
"""Close HTTP client session""" """Close HTTP client session"""
+113
View File
@@ -0,0 +1,113 @@
# bedesten_mcp_module/enums.py
from typing import Literal
# Unified compressed enum for both Yargıtay and Danıştay chambers
BirimAdiEnum = Literal[
"ALL", # All chambers
# Yargıtay (Court of Cassation) - Civil Chambers
"H1", "H2", "H3", "H4", "H5", "H6", "H7", "H8", "H9", "H10",
"H11", "H12", "H13", "H14", "H15", "H16", "H17", "H18", "H19", "H20",
"H21", "H22", "H23",
# Yargıtay - Criminal Chambers
"C1", "C2", "C3", "C4", "C5", "C6", "C7", "C8", "C9", "C10",
"C11", "C12", "C13", "C14", "C15", "C16", "C17", "C18", "C19", "C20",
"C21", "C22", "C23",
# Yargıtay - Councils and Assemblies
"HGK", # Hukuk Genel Kurulu
"CGK", # Ceza Genel Kurulu
"BGK", # Büyük Genel Kurulu
"HBK", # Hukuk Daireleri Başkanlar Kurulu
"CBK", # Ceza Daireleri Başkanlar Kurulu
# Danıştay (Council of State) - Chambers
"D1", "D2", "D3", "D4", "D5", "D6", "D7", "D8", "D9", "D10",
"D11", "D12", "D13", "D14", "D15", "D16", "D17",
# Danıştay - Councils and Boards
"DBGK", # Büyük Gen.Kur. (Grand General Assembly)
"IDDK", # İdare Dava Daireleri Kurulu
"VDDK", # Vergi Dava Daireleri Kurulu
"IBK", # İçtihatları Birleştirme Kurulu
"IIK", # İdari İşler Kurulu
"DBK", # Başkanlar Kurulu
# Military High Administrative Court
"AYIM", # Askeri Yüksek İdare Mahkemesi
"AYIMDK", # Askeri Yüksek İdare Mahkemesi Daireler Kurulu
"AYIMB", # Askeri Yüksek İdare Mahkemesi Başsavcılığı
"AYIM1", # Askeri Yüksek İdare Mahkemesi 1. Daire
"AYIM2", # Askeri Yüksek İdare Mahkemesi 2. Daire
"AYIM3" # Askeri Yüksek İdare Mahkemesi 3. Daire
]
# Mapping from abbreviated values to full Turkish API values
BIRIM_ADI_MAPPING = {
"ALL": None, # Will be handled specially in client
# Yargıtay Civil Chambers (1-23)
"H1": "1. Hukuk Dairesi", "H2": "2. Hukuk Dairesi", "H3": "3. Hukuk Dairesi",
"H4": "4. Hukuk Dairesi", "H5": "5. Hukuk Dairesi", "H6": "6. Hukuk Dairesi",
"H7": "7. Hukuk Dairesi", "H8": "8. Hukuk Dairesi", "H9": "9. Hukuk Dairesi",
"H10": "10. Hukuk Dairesi", "H11": "11. Hukuk Dairesi", "H12": "12. Hukuk Dairesi",
"H13": "13. Hukuk Dairesi", "H14": "14. Hukuk Dairesi", "H15": "15. Hukuk Dairesi",
"H16": "16. Hukuk Dairesi", "H17": "17. Hukuk Dairesi", "H18": "18. Hukuk Dairesi",
"H19": "19. Hukuk Dairesi", "H20": "20. Hukuk Dairesi", "H21": "21. Hukuk Dairesi",
"H22": "22. Hukuk Dairesi", "H23": "23. Hukuk Dairesi",
# Yargıtay Criminal Chambers (1-23)
"C1": "1. Ceza Dairesi", "C2": "2. Ceza Dairesi", "C3": "3. Ceza Dairesi",
"C4": "4. Ceza Dairesi", "C5": "5. Ceza Dairesi", "C6": "6. Ceza Dairesi",
"C7": "7. Ceza Dairesi", "C8": "8. Ceza Dairesi", "C9": "9. Ceza Dairesi",
"C10": "10. Ceza Dairesi", "C11": "11. Ceza Dairesi", "C12": "12. Ceza Dairesi",
"C13": "13. Ceza Dairesi", "C14": "14. Ceza Dairesi", "C15": "15. Ceza Dairesi",
"C16": "16. Ceza Dairesi", "C17": "17. Ceza Dairesi", "C18": "18. Ceza Dairesi",
"C19": "19. Ceza Dairesi", "C20": "20. Ceza Dairesi", "C21": "21. Ceza Dairesi",
"C22": "22. Ceza Dairesi", "C23": "23. Ceza Dairesi",
# Yargıtay Councils and Assemblies
"HGK": "Hukuk Genel Kurulu",
"CGK": "Ceza Genel Kurulu",
"BGK": "Büyük Genel Kurulu",
"HBK": "Hukuk Daireleri Başkanlar Kurulu",
"CBK": "Ceza Daireleri Başkanlar Kurulu",
# Danıştay Chambers (1-17)
"D1": "1. Daire", "D2": "2. Daire", "D3": "3. Daire", "D4": "4. Daire",
"D5": "5. Daire", "D6": "6. Daire", "D7": "7. Daire", "D8": "8. Daire",
"D9": "9. Daire", "D10": "10. Daire", "D11": "11. Daire", "D12": "12. Daire",
"D13": "13. Daire", "D14": "14. Daire", "D15": "15. Daire", "D16": "16. Daire",
"D17": "17. Daire",
# Danıştay Councils and Boards
"DBGK": "Büyük Gen.Kur.",
"IDDK": "İdare Dava Daireleri Kurulu",
"VDDK": "Vergi Dava Daireleri Kurulu",
"IBK": "İçtihatları Birleştirme Kurulu",
"IIK": "İdari İşler Kurulu",
"DBK": "Başkanlar Kurulu",
# Military High Administrative Court
"AYIM": "Askeri Yüksek İdare Mahkemesi",
"AYIMDK": "Askeri Yüksek İdare Mahkemesi Daireler Kurulu",
"AYIMB": "Askeri Yüksek İdare Mahkemesi Başsavcılığı",
"AYIM1": "Askeri Yüksek İdare Mahkemesi 1. Daire",
"AYIM2": "Askeri Yüksek İdare Mahkemesi 2. Daire",
"AYIM3": "Askeri Yüksek İdare Mahkemesi 3. Daire"
}
# Helper function to get full Turkish name from abbreviated value
def get_full_birim_adi(abbreviated_value: str) -> str:
"""Convert abbreviated birimAdi value to full Turkish name for API calls."""
if abbreviated_value == "ALL" or not abbreviated_value:
return "" # Empty string for ALL or None
return BIRIM_ADI_MAPPING.get(abbreviated_value, abbreviated_value)
# Helper function to validate abbreviated value
def is_valid_birim_adi(abbreviated_value: str) -> bool:
"""Check if abbreviated birimAdi value is valid."""
return abbreviated_value in BIRIM_ADI_MAPPING
+23 -84
View File
@@ -4,94 +4,33 @@ from pydantic import BaseModel, Field
from typing import List, Optional, Dict, Any, Literal, Union from typing import List, Optional, Dict, Any, Literal, Union
from datetime import datetime from datetime import datetime
# Import YargitayBirimEnum for chamber filtering # Import compressed BirimAdiEnum for chamber filtering
from yargitay_mcp_module.models import YargitayBirimEnum from .enums import BirimAdiEnum
# Danıştay Chamber/Board Options # Court Type Options for Unified Search
DanistayBirimEnum = Literal[ BedestenCourtTypeEnum = Literal[
"ALL", # "ALL" for all chambers "YARGITAYKARARI", # Yargıtay (Court of Cassation)
# Main Councils "DANISTAYKARAR", # Danıştay (Council of State)
"Büyük Gen.Kur.", # Grand General Assembly "YERELHUKUK", # Local Civil Courts
"İdare Dava Daireleri Kurulu", # Administrative Cases Chambers Council "ISTINAFHUKUK", # Civil Courts of Appeals
"Vergi Dava Daireleri Kurulu", # Tax Cases Chambers Council "KYB" # Extraordinary Appeals (Kanun Yararına Bozma)
"İçtihatları Birleştirme Kurulu", # Precedents Unification Council
"İdari İşler Kurulu", # Administrative Affairs Council
"Başkanlar Kurulu", # Presidents Council
# Chambers
"1. Daire", "2. Daire", "3. Daire", "4. Daire", "5. Daire",
"6. Daire", "7. Daire", "8. Daire", "9. Daire", "10. Daire",
"11. Daire", "12. Daire", "13. Daire", "14. Daire", "15. Daire",
"16. Daire", "17. Daire",
# Military High Administrative Court
"Askeri Yüksek İdare Mahkemesi",
"Askeri Yüksek İdare Mahkemesi Daireler Kurulu",
"Askeri Yüksek İdare Mahkemesi Başsavcılığı",
"Askeri Yüksek İdare Mahkemesi 1. Daire",
"Askeri Yüksek İdare Mahkemesi 2. Daire",
"Askeri Yüksek İdare Mahkemesi 3. Daire"
] ]
# Search Request Models # Search Request Models
class BedestenSearchData(BaseModel): class BedestenSearchData(BaseModel):
pageSize: int = Field(..., description="""Number of results per page. pageSize: int = Field(..., description="Results per page (1-10)")
Range: 1-100 results per page pageNumber: int = Field(..., description="Page number (1-indexed)")
Recommended: 10-50 for balanced performance itemTypeList: List[str] = Field(..., description="Court type filter (YARGITAYKARARI/DANISTAYKARAR/YERELHUKUK/ISTINAFHUKUK/KYB)")
Higher values for comprehensive analysis""") phrase: str = Field(..., description="Search phrase (use \"exact phrase\" for precise matching)")
pageNumber: int = Field(..., description="""Page number to retrieve (1-indexed). birimAdi: BirimAdiEnum = Field("ALL", description="""
Start with 1 for first page Chamber filter (optional). Abbreviated values with Turkish names:
Calculate total pages from response.data.total / pageSize • Yargıtay: H1-H23 (1-23. Hukuk Dairesi), C1-C23 (1-23. Ceza Dairesi), HGK (Hukuk Genel Kurulu), CGK (Ceza Genel Kurulu), BGK (Büyük Genel Kurulu), HBK (Hukuk Daireleri Başkanlar Kurulu), CBK (Ceza Daireleri Başkanlar Kurulu)
Navigate: pageNumber=2 gets next set of results""") • Danıştay: D1-D17 (1-17. Daire), DBGK (Büyük Gen.Kur.), IDDK (İdare Dava Daireleri Kurulu), VDDK (Vergi Dava Daireleri Kurulu), IBK (İçtihatları Birleştirme Kurulu), IIK (İdari İşler Kurulu), DBK (Başkanlar Kurulu), AYIM (Askeri Yüksek İdare Mahkemesi), AYIM1-3 (Askeri Yüksek İdare Mahkemesi 1-3. Daire)
itemTypeList: List[str] = Field(..., description="""Court type filter - determines which court decisions to search: """)
• ["YARGITAYKARARI"]: Court of Cassation (Yargıtay) - supreme court civil/criminal decisions kararTarihiStart: Optional[str] = Field(None, description="Start date (ISO 8601 format)")
• ["DANISTAYKARAR"]: Council of State (Danıştay) - administrative court decisions kararTarihiEnd: Optional[str] = Field(None, description="End date (ISO 8601 format)")
• ["YERELHUKUK"]: Local Civil Courts (Yerel Hukuk Mahkemeleri) - first instance civil decisions sortFields: List[str] = Field(default=["KARAR_TARIHI"], description="Sort fields")
• ["ISTINAFHUKUK"]: Civil Courts of Appeals (İstinaf Hukuk Mahkemeleri) - appellate court decisions sortDirection: str = Field(default="desc", description="Sort direction (asc/desc)")
• ["KYB"]: Extraordinary Appeal (Kanun Yararına Bozma) - extraordinary appeal decisions
Note: Use single-item list for specific court type targeting""")
phrase: str = Field(..., description="""Search phrase/keyword with advanced search support:
• Regular search: "mülkiyet kararı" - searches words separately
• Exact phrase: "\"mülkiyet kararı\"" - searches exact phrase (more precise)
• Legal concepts: "\"idari işlem\"", "\"sözleşme ihlali\"", "\"tazminat davası\""
• Empty string: searches all documents (use with filters)
Exact phrases significantly reduce false positives for precise legal research""")
birimAdi: Optional[Union[YargitayBirimEnum, DanistayBirimEnum]] = Field(None, description="""
Chamber/Department (Daire) filter (optional). Available options depend on itemTypeList:
For YARGITAYKARARI - Court of Cassation (52 options):
- None/null for ALL chambers
- 'Civil General Assembly (Hukuk Genel Kurulu)', '1st Civil Chamber (1. Hukuk Dairesi)' through '23rd Civil Chamber (23. Hukuk Dairesi)'
- 'Criminal General Assembly (Ceza Genel Kurulu)', '1st Criminal Chamber (1. Ceza Dairesi)' through '23rd Criminal Chamber (23. Ceza Dairesi)'
- 'Civil Chambers Presidents Board (Hukuk Daireleri Başkanlar Kurulu)', 'Criminal Chambers Presidents Board (Ceza Daireleri Başkanlar Kurulu)'
- 'Grand General Assembly (Büyük Genel Kurulu)'
For DANISTAYKARAR - Council of State (27 options):
- None/null for ALL chambers
- 'Grand General Assembly (Büyük Gen.Kur.)', 'Administrative Cases Chambers Council (İdare Dava Daireleri Kurulu)', 'Tax Cases Chambers Council (Vergi Dava Daireleri Kurulu)'
- '1st Chamber (1. Daire)' through '17th Chamber (17. Daire)'
- 'Precedents Unification Council (İçtihatları Birleştirme Kurulu)', 'Administrative Affairs Council (İdari İşler Kurulu)', 'Presidents Council (Başkanlar Kurulu)'
- Military courts: 'Military High Administrative Court (Askeri Yüksek İdare Mahkemesi)' variants
""")
kararTarihiStart: Optional[str] = Field(None, description="""Decision start date (Karar Tarihi Başlangıç) filter (optional).
Format: YYYY-MM-DDTHH:MM:SS.000Z (ISO 8601 with Z timezone)
Examples:
"2024-01-01T00:00:00.000Z" - from beginning of 2024
"2023-06-15T00:00:00.000Z" - from June 15, 2023
"2024-03-01T00:00:00.000Z" - from March 1, 2024
Use with kararTarihiEnd for date range, or alone for "from date" filtering""")
kararTarihiEnd: Optional[str] = Field(None, description="""Decision end date (Karar Tarihi Bitiş) filter (optional).
Format: YYYY-MM-DDTHH:MM:SS.000Z (ISO 8601 with Z timezone)
Examples:
"2024-12-31T23:59:59.999Z" - until end of 2024
"2023-12-31T23:59:59.999Z" - until end of 2023
"2024-06-30T23:59:59.999Z" - until end of June 2024
Use with kararTarihiStart for date range, or alone for "until date" filtering""")
sortFields: List[str] = Field(default=["KARAR_TARIHI"], description="""Sorting field (Sıralama Alanı) specification.
["KARAR_TARIHI"]: Sort by decision date (Karar Tarihi) [DEFAULT]
Most common use case for chronological ordering""")
sortDirection: str = Field(default="desc", description="""Sort direction (Sıralama Yönü) for results.
"desc": Descending order - newest decisions first [DEFAULT]
"asc": Ascending order - oldest decisions first
Recommended: "desc" for latest legal developments""")
class BedestenSearchRequest(BaseModel): class BedestenSearchRequest(BaseModel):
data: BedestenSearchData data: BedestenSearchData
@@ -125,7 +64,7 @@ class BedestenSearchDataResponse(BaseModel):
start: int start: int
class BedestenSearchResponse(BaseModel): class BedestenSearchResponse(BaseModel):
data: BedestenSearchDataResponse data: Optional[BedestenSearchDataResponse]
metadata: Dict[str, Any] metadata: Dict[str, Any]
# Document Request/Response Models # Document Request/Response Models
+21
View File
@@ -0,0 +1,21 @@
#!/usr/bin/env python3
from fastmcp import Client
from mcp_server_main import app
import json
import asyncio
async def check_response_format():
client = Client(app)
async with client:
result = await client.call_tool('search_bedesten_unified', {
'phrase': 'mülkiyet',
'court_types': ['YARGITAYKARARI'],
'birimAdi': 'H1',
'pageSize': 3
})
if result and result.content:
data = json.loads(result.content[0].text)
print('Response keys:', list(data.keys()))
print('Sample response:', json.dumps(data, indent=2, ensure_ascii=False)[:500])
asyncio.run(check_response_format())
+17 -17
View File
@@ -6,8 +6,7 @@ from typing import Dict, Any, List, Optional
import logging import logging
import html import html
import re import re
import tempfile import io
import os
from markitdown import MarkItDown from markitdown import MarkItDown
from .models import ( from .models import (
@@ -77,12 +76,16 @@ class DanistayApiClient:
mevzuatNumarasi=params.mevzuatNumarasi or "", mevzuatNumarasi=params.mevzuatNumarasi or "",
mevzuatAdi=params.mevzuatAdi or "", mevzuatAdi=params.mevzuatAdi or "",
madde=params.madde or "", madde=params.madde or "",
siralama=params.siralama, siralama="1",
siralamaDirection=params.siralamaDirection, siralamaDirection="desc",
pageSize=params.pageSize, pageSize=params.pageSize,
pageNumber=params.pageNumber pageNumber=params.pageNumber
) )
final_payload = {"data": data_for_payload.model_dump(exclude_defaults=False, exclude_none=False)} # Create request dict and remove empty string fields to avoid API issues
payload_dict = data_for_payload.model_dump(exclude_defaults=False, exclude_none=False)
# Remove empty string fields that might cause API issues
cleaned_payload = {k: v for k, v in payload_dict.items() if v != ""}
final_payload = {"data": cleaned_payload}
logger.info(f"DanistayApiClient: Performing DETAILED search via {self.DETAILED_SEARCH_ENDPOINT} with payload: {final_payload}") logger.info(f"DanistayApiClient: Performing DETAILED search via {self.DETAILED_SEARCH_ENDPOINT} with payload: {final_payload}")
return await self._execute_api_search(self.DETAILED_SEARCH_ENDPOINT, final_payload) return await self._execute_api_search(self.DETAILED_SEARCH_ENDPOINT, final_payload)
@@ -124,31 +127,28 @@ class DanistayApiClient:
html_input_for_markdown = processed_html html_input_for_markdown = processed_html
markdown_text = None markdown_text = None
temp_file_path = None
try: try:
md_converter = MarkItDown() # Basic conversion # Convert HTML string to bytes and create BytesIO stream
html_bytes = html_input_for_markdown.encode('utf-8')
html_stream = io.BytesIO(html_bytes)
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file: # Pass BytesIO stream to MarkItDown to avoid temp file creation
tmp_file.write(html_input_for_markdown) # Write the full HTML string md_converter = MarkItDown()
temp_file_path = tmp_file.name conversion_result = md_converter.convert(html_stream)
conversion_result = md_converter.convert(temp_file_path)
markdown_text = conversion_result.text_content markdown_text = conversion_result.text_content
logger.info("DanistayApiClient: HTML to Markdown conversion successful.") logger.info("DanistayApiClient: HTML to Markdown conversion successful.")
except Exception as e: except Exception as e:
logger.error(f"DanistayApiClient: Error during MarkItDown HTML to Markdown conversion: {e}") logger.error(f"DanistayApiClient: Error during MarkItDown HTML to Markdown conversion: {e}")
finally:
if temp_file_path and os.path.exists(temp_file_path):
os.remove(temp_file_path)
return markdown_text return markdown_text
async def get_decision_document_as_markdown(self, id: str) -> DanistayDocumentMarkdown: async def get_decision_document_as_markdown(self, id: str) -> DanistayDocumentMarkdown:
""" """
Retrieves a specific Danıştay decision by ID and returns its content as Markdown. Retrieves a specific Danıştay decision by ID and returns its content as Markdown.
The /getDokuman endpoint for Danıştay returns direct HTML. The /getDokuman endpoint for Danıştay requires arananKelime parameter.
""" """
document_api_url = f"{self.DOCUMENT_ENDPOINT}?id={id}" # Add required arananKelime parameter - using empty string as minimum requirement
document_api_url = f"{self.DOCUMENT_ENDPOINT}?id={id}&arananKelime="
source_url = f"{self.BASE_URL}{document_api_url}" source_url = f"{self.BASE_URL}{document_api_url}"
logger.info(f"DanistayApiClient: Fetching Danistay document for Markdown (ID: {id}) from {source_url}") logger.info(f"DanistayApiClient: Fetching Danistay document for Markdown (ID: {id}) from {source_url}")
+27 -29
View File
@@ -5,7 +5,7 @@ from typing import List, Optional, Dict, Any
class DanistayBaseSearchRequest(BaseModel): class DanistayBaseSearchRequest(BaseModel):
"""Base model for common search parameters for Danistay.""" """Base model for common search parameters for Danistay."""
pageSize: int = Field(default=10, ge=1, le=100) pageSize: int = Field(default=10, ge=1, le=10)
pageNumber: int = Field(default=1, ge=1) pageNumber: int = Field(default=1, ge=1)
# siralama and siralamaDirection are part of detailed search, not necessarily keyword search # siralama and siralamaDirection are part of detailed search, not necessarily keyword search
# as per user's provided payloads. # as per user's provided payloads.
@@ -21,11 +21,11 @@ class DanistayKeywordSearchRequestData(BaseModel):
class DanistayKeywordSearchRequest(BaseModel): # This is the model the MCP tool will accept class DanistayKeywordSearchRequest(BaseModel): # This is the model the MCP tool will accept
"""Model for keyword-based search request for Danistay.""" """Model for keyword-based search request for Danistay."""
andKelimeler: List[str] = Field(default_factory=list, description="Keywords for AND logic (VE Mantığı), e.g., ['word1', 'word2']") andKelimeler: List[str] = Field(default_factory=list, description="AND keywords")
orKelimeler: List[str] = Field(default_factory=list, description="Keywords for OR logic (VEYA Mantığı).") orKelimeler: List[str] = Field(default_factory=list, description="OR keywords")
notAndKelimeler: List[str] = Field(default_factory=list, description="Keywords for NOT AND logic (VE DEĞİL Mantığı).") notAndKelimeler: List[str] = Field(default_factory=list, description="NOT AND keywords")
notOrKelimeler: List[str] = Field(default_factory=list, description="Keywords for NOT OR logic (VEYA DEĞİL Mantığı).") notOrKelimeler: List[str] = Field(default_factory=list, description="NOT OR keywords")
pageSize: int = Field(default=10, ge=1, le=100) pageSize: int = Field(default=10, ge=1, le=10)
pageNumber: int = Field(default=1, ge=1) pageNumber: int = Field(default=1, ge=1)
class DanistayDetailedSearchRequestData(BaseModel): # Internal data model for detailed search payload class DanistayDetailedSearchRequestData(BaseModel): # Internal data model for detailed search payload
@@ -51,20 +51,18 @@ class DanistayDetailedSearchRequestData(BaseModel): # Internal data model for de
class DanistayDetailedSearchRequest(DanistayBaseSearchRequest): # MCP tool will accept this class DanistayDetailedSearchRequest(DanistayBaseSearchRequest): # MCP tool will accept this
"""Model for detailed search request for Danistay.""" """Model for detailed search request for Danistay."""
daire: Optional[str] = Field(None, description="Chamber/Department name (e.g., '1. Daire').") daire: str = Field("", description="Chamber")
esasYil: Optional[str] = Field(None, description="Case year for 'Esas No'.") esasYil: str = Field("", description="Case year")
esasIlkSiraNo: Optional[str] = Field(None, description="Starting sequence for 'Esas No'.") esasIlkSiraNo: str = Field("", description="Start case no")
esasSonSiraNo: Optional[str] = Field(None, description="Ending sequence for 'Esas No'.") esasSonSiraNo: str = Field("", description="End case no")
kararYil: Optional[str] = Field(None, description="Decision year for 'Karar No'.") kararYil: str = Field("", description="Decision year")
kararIlkSiraNo: Optional[str] = Field(None, description="Starting sequence for 'Karar No'.") kararIlkSiraNo: str = Field("", description="Start decision no")
kararSonSiraNo: Optional[str] = Field(None, description="Ending sequence for 'Karar No'.") kararSonSiraNo: str = Field("", description="End decision no")
baslangicTarihi: Optional[str] = Field(None, description="Start date for decision (DD.MM.YYYY).") baslangicTarihi: str = Field("", description="Start date")
bitisTarihi: Optional[str] = Field(None, description="End date for decision (DD.MM.YYYY).") bitisTarihi: str = Field("", description="End date")
mevzuatNumarasi: Optional[str] = Field(None, description="Legislation number.") mevzuatNumarasi: str = Field("", description="Law number")
mevzuatAdi: Optional[str] = Field(None, description="Legislation name.") mevzuatAdi: str = Field("", description="Law name")
madde: Optional[str] = Field(None, description="Article number.") madde: str = Field("", description="Article")
siralama: str = Field("1", description="Sorting criteria (e.g., 1: Esas No, 3: Karar Tarihi).")
siralamaDirection: str = Field("desc", description="Sorting direction ('asc' or 'desc').")
# Add a general keyword field if detailed search also supports it # Add a general keyword field if detailed search also supports it
# arananKelime: Optional[str] = Field(None, description="General keyword for detailed search.") # arananKelime: Optional[str] = Field(None, description="General keyword for detailed search.")
@@ -76,15 +74,15 @@ class DanistayApiDecisionEntry(BaseModel):
id: str id: str
# The API response for keyword search uses "daireKurul", detailed search example uses "daire". # The API response for keyword search uses "daireKurul", detailed search example uses "daire".
# We use an alias to handle both and map to a consistent field name "chamber". # We use an alias to handle both and map to a consistent field name "chamber".
chamber: Optional[str] = Field(None, alias="daire", description="The chamber or board.") chamber: str = Field("", alias="daire", description="Chamber")
esasNo: Optional[str] = Field(None) esasNo: str = Field("", description="Case number")
kararNo: Optional[str] = Field(None) kararNo: str = Field("", description="Decision number")
kararTarihi: Optional[str] = Field(None) kararTarihi: str = Field("", description="Decision date")
arananKelime: Optional[str] = Field(None, description="Matched keyword (Aranan Kelime) if provided in response.") arananKelime: str = Field("", description="Keyword")
# index: Optional[int] = None # Present in response, can be added if needed by MCP tool # index: Optional[int] = None # Present in response, can be added if needed by MCP tool
# siraNo: Optional[int] = None # Present in detailed response, can be added # siraNo: Optional[int] = None # Present in detailed response, can be added
document_url: Optional[HttpUrl] = Field(None, description="URL (Belge URL) to the full document, constructed by the client.") document_url: Optional[HttpUrl] = Field(None, description="Document URL")
model_config = ConfigDict(populate_by_name=True, extra='ignore') # Important for alias to work and ignore extra fields model_config = ConfigDict(populate_by_name=True, extra='ignore') # Important for alias to work and ignore extra fields
@@ -93,17 +91,17 @@ class DanistayApiResponseInnerData(BaseModel):
data: List[DanistayApiDecisionEntry] data: List[DanistayApiDecisionEntry]
recordsTotal: int recordsTotal: int
recordsFiltered: int recordsFiltered: int
draw: Optional[int] = Field(None, description="Draw counter (Çizim Sayıcısı) from API, usually for DataTables.") draw: int = Field(0, description="Draw counter")
class DanistayApiResponse(BaseModel): class DanistayApiResponse(BaseModel):
"""Model for the complete search response from the Danistay API.""" """Model for the complete search response from the Danistay API."""
data: DanistayApiResponseInnerData data: Optional[DanistayApiResponseInnerData] = Field(None, description="Response data, can be null when no results found")
metadata: Optional[Dict[str, Any]] = Field(None, description="Optional metadata (Meta Veri) from API.") metadata: Optional[Dict[str, Any]] = Field(None, description="Optional metadata (Meta Veri) from API.")
class DanistayDocumentMarkdown(BaseModel): class DanistayDocumentMarkdown(BaseModel):
"""Model for a Danistay decision document, containing only Markdown content.""" """Model for a Danistay decision document, containing only Markdown content."""
id: str id: str
markdown_content: Optional[str] = Field(None, description="The decision content (Karar İçeriği) converted to Markdown.") markdown_content: str = Field("", description="The decision content (Karar İçeriği) converted to Markdown.")
source_url: HttpUrl source_url: HttpUrl
class CompactDanistaySearchResult(BaseModel): class CompactDanistaySearchResult(BaseModel):
+12 -13
View File
@@ -6,8 +6,7 @@ from typing import Dict, Any, List, Optional
import logging import logging
import html import html
import re import re
import tempfile import io
import os
from markitdown import MarkItDown from markitdown import MarkItDown
from .models import ( from .models import (
@@ -64,7 +63,11 @@ class EmsalApiClient:
pageNumber=params.page_number pageNumber=params.page_number
) )
final_payload = {"data": data_for_api_payload.model_dump(by_alias=True, exclude_none=True)} # Create request dict and remove empty string fields to avoid API issues
payload_dict = data_for_api_payload.model_dump(by_alias=True, exclude_none=True)
# Remove empty string fields that might cause API issues
cleaned_payload = {k: v for k, v in payload_dict.items() if v != ""}
final_payload = {"data": cleaned_payload}
logger.info(f"EmsalApiClient: Performing DETAILED search with payload: {final_payload}") logger.info(f"EmsalApiClient: Performing DETAILED search with payload: {final_payload}")
return await self._execute_api_search(self.DETAILED_SEARCH_ENDPOINT, final_payload) return await self._execute_api_search(self.DETAILED_SEARCH_ENDPOINT, final_payload)
@@ -114,22 +117,18 @@ class EmsalApiClient:
html_input_for_markdown = content html_input_for_markdown = content
markdown_text = None markdown_text = None
temp_file_path = None
try: try:
# Convert HTML string to bytes and create BytesIO stream
html_bytes = html_input_for_markdown.encode('utf-8')
html_stream = io.BytesIO(html_bytes)
# Pass BytesIO stream to MarkItDown to avoid temp file creation
md_converter = MarkItDown() md_converter = MarkItDown()
conversion_result = md_converter.convert(html_stream)
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file:
tmp_file.write(html_input_for_markdown)
temp_file_path = tmp_file.name
conversion_result = md_converter.convert(temp_file_path)
markdown_text = conversion_result.text_content markdown_text = conversion_result.text_content
logger.info("EmsalApiClient: HTML to Markdown conversion successful.") logger.info("EmsalApiClient: HTML to Markdown conversion successful.")
except Exception as e: except Exception as e:
logger.error(f"EmsalApiClient: Error during MarkItDown HTML to Markdown conversion for Emsal: {e}") logger.error(f"EmsalApiClient: Error during MarkItDown HTML to Markdown conversion for Emsal: {e}")
finally:
if temp_file_path and os.path.exists(temp_file_path):
os.remove(temp_file_path)
return markdown_text return markdown_text
+27 -27
View File
@@ -12,12 +12,12 @@ class EmsalDetailedSearchRequestData(BaseModel):
""" """
arananKelime: Optional[str] = "" arananKelime: Optional[str] = ""
Bam_Hukuk_Mahkemeleri: Optional[str] = Field(None, alias="Bam Hukuk Mahkemeleri") Bam_Hukuk_Mahkemeleri: str = Field("", alias="Bam Hukuk Mahkemeleri")
Hukuk_Mahkemeleri: Optional[str] = Field(None, alias="Hukuk Mahkemeleri") Hukuk_Mahkemeleri: str = Field("", alias="Hukuk Mahkemeleri")
# Add other specific court type fields from the form if they are separate keys in payload # Add other specific court type fields from the form if they are separate keys in payload
# E.g., "Ceza Mahkemeleri", "İdari Mahkemeler" etc. # E.g., "Ceza Mahkemeleri", "İdari Mahkemeler" etc.
birimHukukMah: Optional[str] = Field("", description="List of selected Regional Civil Chambers (Bölge Hukuk Mahkemeleri), '+' separated.") birimHukukMah: Optional[str] = Field("", description="Regional chambers (+ separated)")
esasYil: Optional[str] = "" esasYil: Optional[str] = ""
esasIlkSiraNo: Optional[str] = "" esasIlkSiraNo: Optional[str] = ""
@@ -36,42 +36,42 @@ class EmsalDetailedSearchRequestData(BaseModel):
class EmsalSearchRequest(BaseModel): # This is the model the MCP tool will accept class EmsalSearchRequest(BaseModel): # This is the model the MCP tool will accept
"""Model for Emsal detailed search request, with user-friendly field names.""" """Model for Emsal detailed search request, with user-friendly field names."""
keyword: Optional[str] = Field(None, description="Keyword (Anahtar Kelime) to search.") keyword: str = Field("", description="Keyword")
selected_bam_civil_court: Optional[str] = Field(None, description="Selected BAM Civil Court (Seçilen BAM Hukuk Mahkemesi) (maps to 'Bam Hukuk Mahkemeleri' payload key).") selected_bam_civil_court: str = Field("", description="BAM Civil Court")
selected_civil_court: Optional[str] = Field(None, description="Selected Civil Court (Seçilen Hukuk Mahkemesi) (maps to 'Hukuk Mahkemeleri' payload key).") selected_civil_court: str = Field("", description="Civil Court")
selected_regional_civil_chambers: Optional[List[str]] = Field(default_factory=list, description="Selected Regional Civil Chambers (Seçilen Bölge Hukuk Daireleri) (for 'birimHukukMah', joined by '+').") selected_regional_civil_chambers: List[str] = Field(default_factory=list, description="Regional chambers")
case_year_esas: Optional[str] = Field(None, description="Case year (Dava Yılı) for 'Esas No'.") case_year_esas: str = Field("", description="Case year")
case_start_seq_esas: Optional[str] = Field(None, description="Starting sequence (Başlangıç Sırası) for 'Esas No'.") case_start_seq_esas: str = Field("", description="Start case no")
case_end_seq_esas: Optional[str] = Field(None, description="Ending sequence (Bitiş Sırası) for 'Esas No'.") case_end_seq_esas: str = Field("", description="End case no")
decision_year_karar: Optional[str] = Field(None, description="Decision year (Karar Yılı) for 'Karar No'.") decision_year_karar: str = Field("", description="Decision year")
decision_start_seq_karar: Optional[str] = Field(None, description="Starting sequence (Başlangıç Sırası) for 'Karar No'.") decision_start_seq_karar: str = Field("", description="Start decision no")
decision_end_seq_karar: Optional[str] = Field(None, description="Ending sequence (Bitiş Sırası) for 'Karar No'.") decision_end_seq_karar: str = Field("", description="End decision no")
start_date: Optional[str] = Field(None, description="Start date (Başlangıç Tarihi) for decision (DD.MM.YYYY).") start_date: str = Field("", description="Start date (DD.MM.YYYY)")
end_date: Optional[str] = Field(None, description="End date (Bitiş Tarihi) for decision (DD.MM.YYYY).") end_date: str = Field("", description="End date (DD.MM.YYYY)")
sort_criteria: str = Field("1", description="Sorting criteria (Sıralama Kriteri) (e.g., 1: Esas No).") sort_criteria: str = Field("1", description="Sort by")
sort_direction: str = Field("desc", description="Sorting direction (Sıralama Yönü) ('asc' or 'desc').") sort_direction: str = Field("desc", description="Direction")
page_number: int = Field(default=1, ge=1) page_number: int = Field(default=1, ge=1)
page_size: int = Field(default=10, ge=1, le=100) page_size: int = Field(default=10, ge=1, le=10)
class EmsalApiDecisionEntry(BaseModel): class EmsalApiDecisionEntry(BaseModel):
"""Model for an individual decision entry from the Emsal API search response.""" """Model for an individual decision entry from the Emsal API search response."""
id: str id: str
daire: Optional[str] = Field(None, description="The chamber/court (Daire/Mahkeme) that made the decision.") daire: str = Field("", description="Chamber")
esasNo: Optional[str] = Field(None) esasNo: str = Field("", description="Case number")
kararNo: Optional[str] = Field(None) kararNo: str = Field("", description="Decision number")
kararTarihi: Optional[str] = Field(None) kararTarihi: str = Field("", description="Decision date")
arananKelime: Optional[str] = Field(None, description="Matched keyword (Aranan Kelime) from the search.") arananKelime: str = Field("", description="Keyword")
durum: Optional[str] = Field(None, description="Status (Durum) of the decision (e.g., 'KESİNLEŞMEDİ').") durum: str = Field("", description="Status")
# index: Optional[int] = None # Present in Emsal response, can be added if tool needs it # index: Optional[int] = None # Present in Emsal response, can be added if tool needs it
document_url: Optional[HttpUrl] = Field(None, description="URL (Belge URL) to the full document, constructed by the client.") document_url: Optional[HttpUrl] = Field(None, description="Document URL")
model_config = ConfigDict(extra='ignore') model_config = ConfigDict(extra='ignore')
@@ -80,7 +80,7 @@ class EmsalApiResponseInnerData(BaseModel):
data: List[EmsalApiDecisionEntry] data: List[EmsalApiDecisionEntry]
recordsTotal: int recordsTotal: int
recordsFiltered: int recordsFiltered: int
draw: Optional[int] = Field(None, description="Draw counter (Çizim Sayıcısı) from API, usually for DataTables.") draw: int = Field(0, description="Draw counter (Çizim Sayıcısı) from API, usually for DataTables.")
class EmsalApiResponse(BaseModel): class EmsalApiResponse(BaseModel):
"""Model for the complete search response from the Emsal API.""" """Model for the complete search response from the Emsal API."""
@@ -90,7 +90,7 @@ class EmsalApiResponse(BaseModel):
class EmsalDocumentMarkdown(BaseModel): class EmsalDocumentMarkdown(BaseModel):
"""Model for an Emsal decision document, containing only Markdown content.""" """Model for an Emsal decision document, containing only Markdown content."""
id: str id: str
markdown_content: Optional[str] = Field(None, description="The decision content (Karar İçeriği) converted to Markdown.") markdown_content: str = Field("", description="The decision content (Karar İçeriği) converted to Markdown.")
source_url: HttpUrl source_url: HttpUrl
class CompactEmsalSearchResult(BaseModel): class CompactEmsalSearchResult(BaseModel):
+1 -1
View File
@@ -1677,4 +1677,4 @@ async def get_statistics():
if __name__ == "__main__": if __name__ == "__main__":
import uvicorn import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000) uvicorn.run(app, host="0.0.0.0", port=8000)
+816 -58
View File
@@ -18,7 +18,8 @@ import html as html_parser
from markitdown import MarkItDown from markitdown import MarkItDown
import os import os
import math import math
import tempfile import io
import random
from .models import ( from .models import (
KikSearchRequest, KikSearchRequest,
@@ -69,14 +70,79 @@ class KikApiClient:
self.playwright_instance = await async_playwright().start() self.playwright_instance = await async_playwright().start()
if not self.browser or not self.browser.is_connected(): if not self.browser or not self.browser.is_connected():
if self.browser: await self.browser.close() if self.browser: await self.browser.close()
self.browser = await self.playwright_instance.chromium.launch(headless=True) # Ultra stealth browser configuration
self.browser = await self.playwright_instance.chromium.launch(
headless=True,
args=[
# Disable automation indicators
'--no-first-run',
'--no-default-browser-check',
'--disable-dev-shm-usage',
'--disable-extensions',
'--disable-gpu',
'--disable-default-apps',
'--disable-translate',
'--disable-blink-features=AutomationControlled',
'--disable-ipc-flooding-protection',
'--disable-renderer-backgrounding',
'--disable-backgrounding-occluded-windows',
'--disable-client-side-phishing-detection',
'--disable-sync',
'--disable-features=TranslateUI,BlinkGenPropertyTrees',
'--disable-component-extensions-with-background-pages',
'--no-sandbox', # Sometimes needed for headless
'--disable-web-security',
'--disable-features=VizDisplayCompositor',
# Language and locale
'--lang=tr-TR',
'--accept-lang=tr-TR,tr;q=0.9,en;q=0.8',
# Performance optimizations
'--memory-pressure-off',
'--max_old_space_size=4096',
]
)
browser_recreated = True browser_recreated = True
if not self.context or browser_recreated: if not self.context or browser_recreated:
if self.context: await self.context.close() if self.context: await self.context.close()
if not self.browser: raise PlaywrightError("Browser not initialized.") if not self.browser: raise PlaywrightError("Browser not initialized.")
# Ultra realistic context configuration
self.context = await self.browser.new_context( self.context = await self.browser.new_context(
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.0.0 Safari/537.36", user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
viewport={'width': 1920, 'height': 1080},
screen={'width': 1920, 'height': 1080},
device_scale_factor=1.0,
is_mobile=False,
has_touch=False,
# Localization
locale='tr-TR',
timezone_id='Europe/Istanbul',
# Realistic browser features
java_script_enabled=True, java_script_enabled=True,
accept_downloads=True,
ignore_https_errors=True,
# Color scheme and media
color_scheme='light',
reduced_motion='no-preference',
forced_colors='none',
# Additional headers for realism
extra_http_headers={
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
'Accept-Encoding': 'gzip, deflate, br',
'Accept-Language': 'tr-TR,tr;q=0.9,en;q=0.8',
'Cache-Control': 'max-age=0',
'DNT': '1',
'Upgrade-Insecure-Requests': '1',
'Sec-Ch-Ua': '"Google Chrome";v="131", "Chromium";v="131", "Not_A Brand";v="24"',
'Sec-Ch-Ua-Mobile': '?0',
'Sec-Ch-Ua-Platform': '"Windows"',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'none',
'Sec-Fetch-User': '?1',
},
# Permissions to appear realistic
permissions=['geolocation'],
geolocation={'latitude': 41.0082, 'longitude': 28.9784}, # Istanbul
) )
context_recreated = True context_recreated = True
if not self.page or self.page.is_closed() or force_new_page or context_recreated or browser_recreated: if not self.page or self.page.is_closed() or force_new_page or context_recreated or browser_recreated:
@@ -86,6 +152,9 @@ class KikApiClient:
if not self.page: raise PlaywrightError("Failed to create new page.") if not self.page: raise PlaywrightError("Failed to create new page.")
self.page.set_default_navigation_timeout(self.request_timeout) self.page.set_default_navigation_timeout(self.request_timeout)
self.page.set_default_timeout(self.request_timeout) self.page.set_default_timeout(self.request_timeout)
# CRITICAL: Anti-detection JavaScript injection
await self._inject_stealth_scripts()
if not self.page or self.page.is_closed(): if not self.page or self.page.is_closed():
raise PlaywrightError("Playwright page initialization failed.") raise PlaywrightError("Playwright page initialization failed.")
logger.debug("_ensure_playwright_ready completed.") logger.debug("_ensure_playwright_ready completed.")
@@ -99,41 +168,653 @@ class KikApiClient:
if self.playwright_instance: await self.playwright_instance.stop(); self.playwright_instance = None if self.playwright_instance: await self.playwright_instance.stop(); self.playwright_instance = None
logger.info("KikApiClient (Playwright): Resources closed.") logger.info("KikApiClient (Playwright): Resources closed.")
async def _inject_stealth_scripts(self):
"""
Inject comprehensive stealth JavaScript to evade bot detection.
Overrides navigator properties and other fingerprinting vectors.
"""
if not self.page:
logger.warning("Cannot inject stealth scripts: page is None")
return
logger.debug("Injecting comprehensive stealth scripts...")
stealth_script = '''
// Override navigator.webdriver
Object.defineProperty(navigator, 'webdriver', {
get: () => undefined,
configurable: true
});
// Override navigator properties to appear more human
Object.defineProperty(navigator, 'languages', {
get: () => ['tr-TR', 'tr', 'en-US', 'en'],
configurable: true
});
Object.defineProperty(navigator, 'platform', {
get: () => 'Win32',
configurable: true
});
Object.defineProperty(navigator, 'vendor', {
get: () => 'Google Inc.',
configurable: true
});
Object.defineProperty(navigator, 'deviceMemory', {
get: () => 8,
configurable: true
});
Object.defineProperty(navigator, 'hardwareConcurrency', {
get: () => 8,
configurable: true
});
Object.defineProperty(navigator, 'maxTouchPoints', {
get: () => 0,
configurable: true
});
// Override plugins to appear realistic
Object.defineProperty(navigator, 'plugins', {
get: () => {
return [
{
0: {type: "application/x-google-chrome-pdf", suffixes: "pdf", description: "Portable Document Format", enabledPlugin: Plugin},
description: "Portable Document Format",
filename: "internal-pdf-viewer",
length: 1,
name: "Chrome PDF Plugin"
},
{
0: {type: "application/pdf", suffixes: "pdf", description: "", enabledPlugin: Plugin},
description: "",
filename: "mhjfbmdgcfjbbpaeojofohoefgiehjai",
length: 1,
name: "Chrome PDF Viewer"
}
];
},
configurable: true
});
// Override permissions
const originalQuery = window.navigator.permissions.query;
window.navigator.permissions.query = (parameters) => (
parameters.name === 'notifications' ?
Promise.resolve({ state: Notification.permission }) :
originalQuery(parameters)
);
// Override WebGL rendering context
const getParameter = WebGLRenderingContext.prototype.getParameter;
WebGLRenderingContext.prototype.getParameter = function(parameter) {
if (parameter === 37445) { // UNMASKED_VENDOR_WEBGL
return 'Intel Inc.';
}
if (parameter === 37446) { // UNMASKED_RENDERER_WEBGL
return 'Intel(R) Iris(R) Plus Graphics 640';
}
return getParameter(parameter);
};
// Override canvas fingerprinting
const toBlob = HTMLCanvasElement.prototype.toBlob;
const toDataURL = HTMLCanvasElement.prototype.toDataURL;
const getImageData = CanvasRenderingContext2D.prototype.getImageData;
const noisify = (canvas, context) => {
const imageData = context.getImageData(0, 0, canvas.width, canvas.height);
for (let i = 0; i < imageData.data.length; i += 4) {
imageData.data[i] += Math.floor(Math.random() * 10) - 5;
imageData.data[i + 1] += Math.floor(Math.random() * 10) - 5;
imageData.data[i + 2] += Math.floor(Math.random() * 10) - 5;
}
context.putImageData(imageData, 0, 0);
};
Object.defineProperty(HTMLCanvasElement.prototype, 'toBlob', {
value: function(callback, type, encoderOptions) {
noisify(this, this.getContext('2d'));
return toBlob.apply(this, arguments);
}
});
Object.defineProperty(HTMLCanvasElement.prototype, 'toDataURL', {
value: function(type, encoderOptions) {
noisify(this, this.getContext('2d'));
return toDataURL.apply(this, arguments);
}
});
// Override AudioContext for audio fingerprinting
const audioCtx = new (window.AudioContext || window.webkitAudioContext)();
const originalAnalyser = audioCtx.createAnalyser;
audioCtx.createAnalyser = function() {
const analyser = originalAnalyser.apply(this, arguments);
const getFloatFrequencyData = analyser.getFloatFrequencyData;
analyser.getFloatFrequencyData = function(array) {
getFloatFrequencyData.apply(this, arguments);
for (let i = 0; i < array.length; i++) {
array[i] += Math.random() * 0.0001;
}
};
return analyser;
};
// Override screen properties
Object.defineProperty(window.screen, 'colorDepth', {
get: () => 24,
configurable: true
});
Object.defineProperty(window.screen, 'pixelDepth', {
get: () => 24,
configurable: true
});
// Override timezone
Date.prototype.getTimezoneOffset = function() {
return -180; // UTC+3 (Istanbul)
};
// Override document.cookie to prevent tracking
const originalCookieDescriptor = Object.getOwnPropertyDescriptor(Document.prototype, 'cookie') ||
Object.getOwnPropertyDescriptor(HTMLDocument.prototype, 'cookie');
if (originalCookieDescriptor && originalCookieDescriptor.configurable) {
Object.defineProperty(document, 'cookie', {
get: function() {
return originalCookieDescriptor.get.call(this);
},
set: function(val) {
console.log('Cookie set blocked:', val);
return originalCookieDescriptor.set.call(this, val);
},
configurable: true
});
}
// Remove automation traces
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Array;
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Promise;
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Symbol;
delete window.cdc_adoQpoasnfa76pfcZLmcfl_JSON;
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Object;
delete window.cdc_adoQpoasnfa76pfcZLmcfl_Proxy;
// Add realistic performance timing
if (window.performance && window.performance.timing) {
const timing = window.performance.timing;
const now = Date.now();
Object.defineProperty(timing, 'navigationStart', { value: now - Math.floor(Math.random() * 1000) + 1000, configurable: false });
Object.defineProperty(timing, 'loadEventEnd', { value: now - Math.floor(Math.random() * 100) + 100, configurable: false });
}
console.log('✓ Stealth scripts injected successfully');
'''
try:
await self.page.add_init_script(stealth_script)
logger.debug("✅ Stealth scripts injected successfully")
except Exception as e:
logger.warning(f"⚠️ Failed to inject stealth scripts: {e}")
async def _simulate_human_behavior(self, fast_mode: bool = True):
"""
Simulate realistic human behavior patterns to avoid detection.
Includes mouse movements, typing patterns, and natural delays.
Args:
fast_mode: If True, use minimal timing for speed optimization
"""
if not self.page:
logger.warning("Cannot simulate human behavior: page is None")
return
logger.debug("🤖 Simulating human behavior patterns...")
try:
if fast_mode:
# ULTRA-FAST MODE: Minimal human behavior
viewport_size = self.page.viewport_size
if viewport_size and random.random() < 0.7: # 70% chance to do movement
width, height = viewport_size['width'], viewport_size['height']
# Single quick mouse movement
x = random.randint(200, width - 200)
y = random.randint(200, height - 200)
await self.page.mouse.move(x, y)
# Brief scroll (50% chance)
if random.random() < 0.5:
await self.page.mouse.wheel(0, random.randint(50, 100))
# Ultra-minimal delay
await asyncio.sleep(random.uniform(0.05, 0.15)) # Reduced from 0.1-0.3
else:
# FULL MODE: Original comprehensive behavior
viewport_size = self.page.viewport_size
if viewport_size:
width, height = viewport_size['width'], viewport_size['height']
# Generate 3-5 random mouse movements
movements = random.randint(3, 5)
logger.debug(f" 🖱️ Performing {movements} random mouse movements")
for i in range(movements):
x = random.randint(100, width - 100)
y = random.randint(100, height - 100)
# Move mouse with realistic speed (not instant)
await self.page.mouse.move(x, y)
await asyncio.sleep(random.uniform(0.1, 0.3))
# 2. Scroll simulation
logger.debug(" 📜 Simulating scroll behavior")
scroll_amount = random.randint(100, 300)
await self.page.mouse.wheel(0, scroll_amount)
await asyncio.sleep(random.uniform(0.2, 0.5))
# Scroll back up
await self.page.mouse.wheel(0, -scroll_amount)
await asyncio.sleep(random.uniform(0.2, 0.4))
# 3. Random page interaction delays
await asyncio.sleep(random.uniform(0.5, 1.5))
logger.debug("✅ Human behavior simulation completed")
except Exception as e:
logger.warning(f"⚠️ Human behavior simulation failed: {e}")
async def _human_type(self, selector: str, text: str, clear_first: bool = True, fast_mode: bool = True):
"""
Type text with human-like patterns and delays.
Args:
selector: CSS selector for the input element
text: Text to type
clear_first: Whether to clear the field first
fast_mode: If True, use minimal delays for speed optimization
"""
if not self.page:
logger.warning("Cannot perform human typing: page is None")
return
try:
if fast_mode:
# FAST MODE: Direct fill for speed
await self.page.fill(selector, text)
await asyncio.sleep(random.uniform(0.02, 0.05)) # Reduced from 0.05-0.1
else:
# FULL MODE: Character-by-character human typing
# Focus on the element first
await self.page.focus(selector)
await asyncio.sleep(random.uniform(0.1, 0.3))
# Clear field if requested
if clear_first:
await self.page.keyboard.press('Control+a')
await asyncio.sleep(random.uniform(0.05, 0.15))
await self.page.keyboard.press('Delete')
await asyncio.sleep(random.uniform(0.05, 0.15))
# Type each character with human-like delays
for char in text:
await self.page.keyboard.type(char)
# Human typing speed: 50-150ms between characters
delay = random.uniform(0.05, 0.15)
# Occasional longer pauses (thinking)
if random.random() < 0.1: # 10% chance
delay += random.uniform(0.2, 0.8)
await asyncio.sleep(delay)
# Brief pause after typing
await asyncio.sleep(random.uniform(0.2, 0.6))
logger.debug(f"✅ Human-typed '{text}' into {selector}")
except Exception as e:
logger.warning(f"⚠️ Human typing failed: {e}")
async def _human_click(self, selector: str, wait_before: bool = True, wait_after: bool = True, fast_mode: bool = True):
"""
Perform a human-like click with realistic delays and mouse movement.
Args:
selector: CSS selector or element to click
wait_before: Whether to wait before clicking
wait_after: Whether to wait after clicking
fast_mode: If True, use minimal delays for speed optimization
"""
if not self.page:
logger.warning("Cannot perform human click: page is None")
return
try:
if fast_mode:
# FAST MODE: Direct click with minimal delay
if wait_before:
await asyncio.sleep(random.uniform(0.02, 0.08)) # Reduced from 0.05-0.15
await self.page.click(selector)
if wait_after:
await asyncio.sleep(random.uniform(0.02, 0.08)) # Reduced from 0.05-0.15
else:
# FULL MODE: Realistic mouse movement and timing
# Wait before clicking (thinking time)
if wait_before:
await asyncio.sleep(random.uniform(0.3, 0.8))
# Get element bounds for realistic mouse movement
element = await self.page.query_selector(selector)
if element:
box = await element.bounding_box()
if box:
# Move to element with slight randomness
center_x = box['x'] + box['width'] / 2
center_y = box['y'] + box['height'] / 2
# Add small random offset
offset_x = random.uniform(-10, 10)
offset_y = random.uniform(-5, 5)
await self.page.mouse.move(center_x + offset_x, center_y + offset_y)
await asyncio.sleep(random.uniform(0.1, 0.3))
# Perform click
await self.page.mouse.click(center_x + offset_x, center_y + offset_y)
logger.debug(f"✅ Human-clicked {selector}")
else:
# Fallback to regular click
await self.page.click(selector)
logger.debug(f"✅ Fallback-clicked {selector}")
else:
logger.warning(f"⚠️ Element not found for human click: {selector}")
return
# Wait after clicking (processing time)
if wait_after:
await asyncio.sleep(random.uniform(0.2, 0.6))
logger.debug(f"✅ Human-clicked {selector}")
except Exception as e:
logger.warning(f"⚠️ Human click failed: {e}")
async def _simulate_page_exploration(self, fast_mode: bool = True):
"""
Simulate natural page exploration before performing the main task.
This helps establish a more human-like session.
Args:
fast_mode: If True, use minimal exploration for speed optimization
"""
if not self.page:
return
logger.debug("🕵️ Simulating page exploration...")
try:
if fast_mode:
# ULTRA-FAST MODE: Minimal exploration
await asyncio.sleep(random.uniform(0.05, 0.1)) # Reduced from 0.1-0.3
# Single mouse movement (optional)
try:
elements = await self.page.query_selector_all("input, button")
if elements and random.random() < 0.5: # 50% chance to skip
element = random.choice(elements)
box = await element.bounding_box()
if box:
center_x = box['x'] + box['width'] / 2
center_y = box['y'] + box['height'] / 2
await self.page.mouse.move(center_x, center_y)
except:
pass
await asyncio.sleep(random.uniform(0.02, 0.05)) # Reduced from 0.05-0.15
else:
# FULL MODE: Comprehensive exploration
# 1. Brief pause to "read" the page
await asyncio.sleep(random.uniform(1.0, 2.5))
# 2. Move mouse to various UI elements (like a human would explore)
explore_selectors = [
"h1", "h2", ".navbar", "#header", ".logo",
"input", "button", "a", ".form-group"
]
explored = 0
for selector in explore_selectors:
elements = await self.page.query_selector_all(selector)
if elements and explored < 3: # Explore max 3 elements
element = random.choice(elements)
box = await element.bounding_box()
if box:
center_x = box['x'] + box['width'] / 2
center_y = box['y'] + box['height'] / 2
await self.page.mouse.move(center_x, center_y)
await asyncio.sleep(random.uniform(0.3, 0.8))
explored += 1
# 3. Small scroll to simulate reading
await self.page.mouse.wheel(0, random.randint(50, 150))
await asyncio.sleep(random.uniform(0.5, 1.2))
logger.debug("✅ Page exploration completed")
except Exception as e:
logger.debug(f"⚠️ Page exploration failed: {e}")
def _parse_decision_entries_from_soup(self, soup: BeautifulSoup, search_karar_tipi: KikKararTipi) -> List[KikDecisionEntry]: def _parse_decision_entries_from_soup(self, soup: BeautifulSoup, search_karar_tipi: KikKararTipi) -> List[KikDecisionEntry]:
entries: List[KikDecisionEntry] = [] entries: List[KikDecisionEntry] = []
table = soup.find("table", {"id": self.RESULTS_TABLE_ID}) table = soup.find("table", {"id": self.RESULTS_TABLE_ID})
if not table: return entries
logger.debug(f"Looking for table with ID: {self.RESULTS_TABLE_ID}")
if not table:
logger.warning(f"Table with ID '{self.RESULTS_TABLE_ID}' not found in HTML")
# Log available tables for debugging
all_tables = soup.find_all("table")
logger.debug(f"Found {len(all_tables)} tables in HTML")
for idx, tbl in enumerate(all_tables):
table_id = tbl.get('id', 'no-id')
table_class = tbl.get('class', 'no-class')
rows = tbl.find_all('tr')
logger.debug(f"Table {idx}: id='{table_id}', class='{table_class}', rows={len(rows)}")
# If this looks like a results table, try to use it
if (table_id and ('grd' in table_id.lower() or 'kurul' in table_id.lower() or 'sonuc' in table_id.lower())) or \
(isinstance(table_class, list) and any('grid' in cls.lower() or 'result' in cls.lower() for cls in table_class)) or \
len(rows) > 3: # Table with multiple rows might be results
logger.info(f"Trying to parse table {idx} as potential results table: id='{table_id}'")
table = tbl
break
if not table:
logger.error("No suitable results table found")
return entries
rows = table.find_all("tr") rows = table.find_all("tr")
logger.info(f"Found {len(rows)} rows in results table")
# Debug: Log first few rows structure
for i, row in enumerate(rows[:3]):
cells = row.find_all(["td", "th"])
cell_texts = [cell.get_text(strip=True)[:30] for cell in cells]
logger.info(f"Row {i} structure: {len(cells)} cells: {cell_texts}")
for row_idx, row in enumerate(rows): for row_idx, row in enumerate(rows):
if row_idx < 2: continue # Skip first row (search bar with colspan=7) and second row (header with 6 cells)
if row_idx < 2:
logger.debug(f"Skipping header row {row_idx}")
continue
cells = row.find_all("td") cells = row.find_all("td")
if len(cells) == 6: logger.debug(f"Row {row_idx}: Found {len(cells)} cells")
# Log cell contents for debugging
if cells and row_idx < 5: # Log first few data rows
for cell_idx, cell in enumerate(cells):
cell_text = cell.get_text(strip=True)[:50] # First 50 chars
logger.debug(f" Cell {cell_idx}: '{cell_text}...'")
# Be more flexible with cell count - try 6 cells first, then adapt
if len(cells) >= 5: # At least 5 cells for minimum required data
try: try:
preview_button_tag = cells[0].find("a", id=re.compile(r"btnOnizle$")) # Try to find preview button in first cell or any cell with a link
preview_button_tag = None
event_target = "" event_target = ""
if preview_button_tag and preview_button_tag.has_attr('href'):
match = re.search(r"__doPostBack\('([^']*)','([^']*)'\)", preview_button_tag['href']) # Look for preview button in first few cells
if match: event_target = match.group(1) for cell_idx in range(min(3, len(cells))):
karar_no_span = cells[1].find("span", id=re.compile(r"lblKno$")) cell = cells[cell_idx]
karar_tarihi_span = cells[2].find("span", id=re.compile(r"lblKtar$")) # Try multiple patterns for preview button (based on actual HTML structure)
idare_span = cells[3].find("span", id=re.compile(r"lblIdare$")) preview_candidates = [
basvuru_sahibi_span = cells[4].find("span", id=re.compile(r"lblSikayetci$")) cell.find("a", id="btnOnizle"), # Exact match
ihale_span = cells[5].find("span", id=re.compile(r"lblIhale$")) cell.find("a", id=re.compile(r"btnOnizle$")),
if not (event_target and karar_no_span and karar_tarihi_span): continue cell.find("a", id=re.compile(r"btn.*Onizle")),
cell.find("a", id=re.compile(r".*Onizle.*")),
cell.find("a", href=re.compile(r"__doPostBack"))
]
for candidate in preview_candidates:
if candidate and candidate.has_attr('href'):
match = re.search(r"__doPostBack\('([^']*)','([^']*)'\)", candidate['href'])
if match:
event_target = match.group(1)
preview_button_tag = candidate
logger.debug(f"Row {row_idx}: Found event_target '{event_target}' in cell {cell_idx}")
break
if preview_button_tag:
break
if not preview_button_tag:
logger.debug(f"Row {row_idx}: No preview button found in any cell")
# Log what links we found
for cell_idx, cell in enumerate(cells[:3]):
links_in_cell = cell.find_all("a")
logger.debug(f" Cell {cell_idx}: {len(links_in_cell)} links")
for link in links_in_cell[:2]:
logger.debug(f" Link id='{link.get('id')}', href='{link.get('href', '')[:50]}...'")
# Try to find decision data spans with more flexible patterns
karar_no_span = None
karar_tarihi_span = None
idare_span = None
basvuru_sahibi_span = None
ihale_span = None
# Try different span patterns for karar no (usually in cell 1)
for cell_idx in range(min(4, len(cells))):
if not karar_no_span:
cell = cells[cell_idx]
candidates = [
cell.find("span", id="lblKno"), # Exact match based on actual HTML
cell.find("span", id=re.compile(r"lblKno$")),
cell.find("span", id=re.compile(r".*Kno.*")),
cell.find("span", id=re.compile(r".*KararNo.*")),
cell.find("span", id=re.compile(r".*No.*"))
]
for candidate in candidates:
if candidate and candidate.get_text(strip=True):
karar_no_span = candidate
logger.debug(f"Row {row_idx}: Found karar_no in cell {cell_idx}")
break
# Try different patterns for karar tarihi (usually in cell 2)
for cell_idx in range(min(4, len(cells))):
if not karar_tarihi_span:
cell = cells[cell_idx]
candidates = [
cell.find("span", id="lblKtar"), # Exact match based on actual HTML
cell.find("span", id=re.compile(r"lblKtar$")),
cell.find("span", id=re.compile(r".*Ktar.*")),
cell.find("span", id=re.compile(r".*Tarih.*")),
cell.find("span", id=re.compile(r".*Date.*"))
]
for candidate in candidates:
if candidate and candidate.get_text(strip=True):
# Check if it looks like a date
text = candidate.get_text(strip=True)
if re.match(r'\d{1,2}[./]\d{1,2}[./]\d{4}', text):
karar_tarihi_span = candidate
logger.debug(f"Row {row_idx}: Found karar_tarihi in cell {cell_idx}")
break
# Find other spans in remaining cells (if we have 6 cells) - using exact IDs
if len(cells) >= 6:
idare_span = cells[3].find("span", id="lblIdare") or cells[3].find("span")
basvuru_sahibi_span = cells[4].find("span", id="lblSikayetci") or cells[4].find("span")
ihale_span = cells[5].find("span", id="lblIhale") or cells[5].find("span")
elif len(cells) == 5:
# Adjust for 5-cell layout
idare_span = cells[2].find("span") if cells[2] != cells[1] else None
basvuru_sahibi_span = cells[3].find("span") if len(cells) > 3 else None
ihale_span = cells[4].find("span") if len(cells) > 4 else None
# Log what we found
logger.debug(f"Row {row_idx}: karar_no_span={karar_no_span is not None}, "
f"karar_tarihi_span={karar_tarihi_span is not None}, "
f"event_target={bool(event_target)}")
# For KIK, we need at least karar_no and karar_tarihi, event_target is helpful but not critical
if not (karar_no_span and karar_tarihi_span):
logger.debug(f"Row {row_idx}: Missing required fields (karar_no or karar_tarihi), skipping")
# Log what spans we found in cells
for i, cell in enumerate(cells):
spans = cell.find_all("span")
if spans:
span_info = []
for s in spans:
span_id = s.get('id', 'no-id')
span_text = s.get_text(strip=True)[:20]
span_info.append(f"{span_id}:'{span_text}...'")
logger.debug(f" Cell {i} spans: {span_info}")
continue
# If we don't have event_target, we can still create an entry but mark it specially
if not event_target:
logger.warning(f"Row {row_idx}: No event_target found, document retrieval won't work")
event_target = f"missing_target_row_{row_idx}" # Placeholder
# Karar tipini arama parametresinden alıyoruz, çünkü HTML'de direkt olarak bulunmuyor. # Karar tipini arama parametresinden alıyoruz, çünkü HTML'de direkt olarak bulunmuyor.
entry = KikDecisionEntry( try:
preview_event_target=event_target, entry = KikDecisionEntry(
kararNo=karar_no_span.get_text(strip=True), preview_event_target=event_target,
karar_tipi=search_karar_tipi, # Arama yapılan karar tipini ekle kararNo=karar_no_span.get_text(strip=True),
kararTarihi=karar_tarihi_span.get_text(strip=True), karar_tipi=search_karar_tipi, # Arama yapılan karar tipini ekle
idare=idare_span.get_text(strip=True) if idare_span else None, kararTarihi=karar_tarihi_span.get_text(strip=True),
basvuruSahibi=basvuru_sahibi_span.get_text(strip=True) if basvuru_sahibi_span else None, idare=idare_span.get_text(strip=True) if idare_span else None,
ihaleKonusu=ihale_span.get_text(strip=True) if ihale_span else None, basvuruSahibi=basvuru_sahibi_span.get_text(strip=True) if basvuru_sahibi_span else None,
) ihaleKonusu=ihale_span.get_text(strip=True) if ihale_span else None,
entries.append(entry) )
entries.append(entry)
logger.info(f"Row {row_idx}: Successfully parsed decision: {entry.karar_no_str}")
except Exception as e:
logger.error(f"Row {row_idx}: Error creating KikDecisionEntry: {e}")
continue
except Exception as e: except Exception as e:
logger.error(f"Error parsing a KIK decision entry row: {e}", exc_info=True) logger.error(f"Error parsing row {row_idx}: {e}", exc_info=True)
else:
logger.warning(f"Row {row_idx}: Expected at least 5 cells but found {len(cells)}, skipping")
if len(cells) > 0:
cell_texts = [cell.get_text(strip=True)[:50] for cell in cells[:3]]
logger.debug(f"Row {row_idx} cells preview: {cell_texts}")
logger.info(f"Parsed {len(entries)} decision entries from {len(rows)} rows")
return entries return entries
def _parse_total_records_from_soup(self, soup: BeautifulSoup) -> int: def _parse_total_records_from_soup(self, soup: BeautifulSoup) -> int:
@@ -163,6 +844,10 @@ class KikApiClient:
try: try:
if page.url != search_url: if page.url != search_url:
await page.goto(search_url, wait_until="networkidle", timeout=self.request_timeout) await page.goto(search_url, wait_until="networkidle", timeout=self.request_timeout)
# Simulate natural page exploration after navigation (FAST MODE)
await self._simulate_page_exploration(fast_mode=True)
search_button_selector = f"a[id='{self.FIELD_LOCATORS['search_button_id']}']" search_button_selector = f"a[id='{self.FIELD_LOCATORS['search_button_id']}']"
await page.wait_for_selector(search_button_selector, state="visible", timeout=self.request_timeout) await page.wait_for_selector(search_button_selector, state="visible", timeout=self.request_timeout)
@@ -170,12 +855,18 @@ class KikApiClient:
radio_locator_selector = f"{self.FIELD_LOCATORS['karar_tipi_radio_group']}[value='{current_karar_tipi_value}']" radio_locator_selector = f"{self.FIELD_LOCATORS['karar_tipi_radio_group']}[value='{current_karar_tipi_value}']"
if not await page.locator(radio_locator_selector).is_checked(): if not await page.locator(radio_locator_selector).is_checked():
js_target_radio = f"ctl00$ContentPlaceHolder1${current_karar_tipi_value}" js_target_radio = f"ctl00$ContentPlaceHolder1${current_karar_tipi_value}"
logger.info(f"Selecting radio button: {js_target_radio}")
async with page.expect_navigation(wait_until="networkidle", timeout=self.request_timeout): async with page.expect_navigation(wait_until="networkidle", timeout=self.request_timeout):
await page.evaluate(f"javascript:__doPostBack('{js_target_radio}','')") await page.evaluate(f"javascript:__doPostBack('{js_target_radio}','')")
await page.wait_for_timeout(1000) # Ultra-fast wait for page to stabilize after radio button change
await page.wait_for_timeout(300) # Reduced from 1000ms
logger.info("Radio button selection completed")
async def fill_if_value(selector_key: str, value: Optional[str]): # Helper function for human-like form filling (FAST MODE)
if value is not None: await page.fill(self.FIELD_LOCATORS[selector_key], value) async def human_fill_if_value(selector_key: str, value: Optional[str]):
if value is not None:
selector = self.FIELD_LOCATORS[selector_key]
await self._human_type(selector, value, fast_mode=True)
# Karar No'yu KİK sitesine göndermeden önce '_' -> '/' dönüşümü yap # Karar No'yu KİK sitesine göndermeden önce '_' -> '/' dönüşümü yap
karar_no_for_kik_form = None karar_no_for_kik_form = None
@@ -183,43 +874,109 @@ class KikApiClient:
karar_no_for_kik_form = search_params.karar_no.replace('_', '/') karar_no_for_kik_form = search_params.karar_no.replace('_', '/')
logger.info(f"Using karar_no '{karar_no_for_kik_form}' (transformed from '{search_params.karar_no}') for KIK form.") logger.info(f"Using karar_no '{karar_no_for_kik_form}' (transformed from '{search_params.karar_no}') for KIK form.")
await fill_if_value('karar_metni', search_params.karar_metni) # Fill form fields with FAST human-like behavior
await fill_if_value('karar_no', karar_no_for_kik_form) # Dönüştürülmüş halini kullan logger.info("Filling form fields with fast mode...")
# ... (diğer fill_if_value çağrıları aynı) ...
await fill_if_value('karar_tarihi_baslangic', search_params.karar_tarihi_baslangic) # Start with FAST mouse behavior simulation
await fill_if_value('karar_tarihi_bitis', search_params.karar_tarihi_bitis) await self._simulate_human_behavior(fast_mode=True)
await fill_if_value('resmi_gazete_sayisi', search_params.resmi_gazete_sayisi)
await fill_if_value('resmi_gazete_tarihi', search_params.resmi_gazete_tarihi) await human_fill_if_value('karar_metni', search_params.karar_metni)
await fill_if_value('basvuru_konusu_ihale', search_params.basvuru_konusu_ihale) await human_fill_if_value('karar_no', karar_no_for_kik_form) # Dönüştürülmüş halini kullan
await fill_if_value('basvuru_sahibi', search_params.basvuru_sahibi) await human_fill_if_value('karar_tarihi_baslangic', search_params.karar_tarihi_baslangic)
await fill_if_value('ihaleyi_yapan_idare', search_params.ihaleyi_yapan_idare) await human_fill_if_value('karar_tarihi_bitis', search_params.karar_tarihi_bitis)
await human_fill_if_value('resmi_gazete_sayisi', search_params.resmi_gazete_sayisi)
await human_fill_if_value('resmi_gazete_tarihi', search_params.resmi_gazete_tarihi)
await human_fill_if_value('basvuru_konusu_ihale', search_params.basvuru_konusu_ihale)
await human_fill_if_value('basvuru_sahibi', search_params.basvuru_sahibi)
await human_fill_if_value('ihaleyi_yapan_idare', search_params.ihaleyi_yapan_idare)
if search_params.yil: if search_params.yil:
await page.select_option(self.FIELD_LOCATORS['yil'], value=search_params.yil) await page.select_option(self.FIELD_LOCATORS['yil'], value=search_params.yil)
await page.wait_for_timeout(50) # Reduced from 100ms
logger.info("Form filling completed, preparing for search...")
# Additional FAST human behavior before search
await self._simulate_human_behavior(fast_mode=True)
action_is_search_button_click = (search_params.page == 1) action_is_search_button_click = (search_params.page == 1)
event_target_for_submit: str event_target_for_submit: str
if action_is_search_button_click:
event_target_for_submit = self.FIELD_LOCATORS['search_button_id']
else: # Pagination
page_link_ctl_number = search_params.page + 2
event_target_for_submit = f"ctl00$ContentPlaceHolder1$grdKurulKararSorguSonuc$ctl14$ctl{page_link_ctl_number:02d}"
try: try:
async with page.expect_navigation(wait_until="networkidle", timeout=self.request_timeout): if action_is_search_button_click:
if action_is_search_button_click: event_target_for_submit = self.FIELD_LOCATORS['search_button_id']
await page.locator(search_button_selector).click() # Use human-like clicking for search button
else: search_button_selector = f"a[id='{event_target_for_submit}']"
logger.info(f"Performing human-like search button click...")
try:
# Hide datepicker first to prevent interference
await page.evaluate("$('#ui-datepicker-div').hide()")
# FAST Human-like click on search button
await self._human_click(search_button_selector, wait_before=True, wait_after=False, fast_mode=True)
# Wait for navigation
await page.wait_for_load_state("networkidle", timeout=self.request_timeout)
logger.info("Search navigation completed successfully")
except Exception as e:
logger.warning(f"Human click failed, falling back to JavaScript: {e}")
# Hide datepicker and use JavaScript fallback
await page.evaluate("$('#ui-datepicker-div').hide()")
async with page.expect_navigation(wait_until="networkidle", timeout=self.request_timeout):
await page.evaluate(f"javascript:__doPostBack('{event_target_for_submit}','')")
logger.info("Search navigation completed via fallback")
else:
# Pagination - use original method for consistency
page_link_ctl_number = search_params.page + 2
event_target_for_submit = f"ctl00$ContentPlaceHolder1$grdKurulKararSorguSonuc$ctl14$ctl{page_link_ctl_number:02d}"
logger.info(f"Executing pagination with event target: {event_target_for_submit}")
async with page.expect_navigation(wait_until="networkidle", timeout=self.request_timeout):
await page.evaluate(f"javascript:__doPostBack('{event_target_for_submit}','')") await page.evaluate(f"javascript:__doPostBack('{event_target_for_submit}','')")
logger.info("Pagination navigation completed successfully")
except PlaywrightTimeoutError: except PlaywrightTimeoutError:
await page.wait_for_timeout(2000) logger.warning("Search navigation timed out, but continuing...")
await page.wait_for_timeout(5000) # Longer wait if navigation fails
# Ultra-fast wait time for results to load
logger.info("Waiting for search results to load...")
await page.wait_for_timeout(500) # Reduced from 1000ms
results_table_dom_selector = f"table#{self.RESULTS_TABLE_ID}" results_table_dom_selector = f"table#{self.RESULTS_TABLE_ID}"
try: try:
await page.wait_for_selector(results_table_dom_selector, timeout=30000, state="attached") # First wait for any tables to appear (more general check)
await page.wait_for_timeout(2000) logger.info("Waiting for any tables to appear...")
await page.wait_for_function("""
() => document.querySelectorAll('table').length > 0
""", timeout=4000) # Reduced from 8000ms
logger.info("At least one table appeared")
# Then wait for our specific table
await page.wait_for_selector(results_table_dom_selector, timeout=4000, state="attached") # Reduced from 8000ms
logger.debug("Results table attached to DOM")
# Wait for table to have some content (more than just headers)
await page.wait_for_function(f"""
() => {{
const table = document.querySelector('{results_table_dom_selector}');
return table && table.querySelectorAll('tr').length > 2;
}}
""", timeout=4000) # Reduced from 20000ms
logger.debug("Results table populated with data")
# Ultra-fast additional wait for any remaining JavaScript
await page.wait_for_timeout(500) # Reduced from 3000ms
except PlaywrightTimeoutError: except PlaywrightTimeoutError:
logger.warning(f"Timeout waiting for results table '{results_table_dom_selector}'.") logger.warning(f"Timeout waiting for results table '{results_table_dom_selector}'.")
# Try one more wait for content placeholder
try:
await page.wait_for_selector("#ctl00_ContentPlaceHolder1", timeout=10000)
logger.info("ContentPlaceHolder1 found, checking for tables...")
await page.wait_for_timeout(5000)
except PlaywrightTimeoutError:
logger.warning("ContentPlaceHolder1 also not found - content may not have loaded")
html_content = await page.content() html_content = await page.content()
soup = BeautifulSoup(html_content, "html.parser") soup = BeautifulSoup(html_content, "html.parser")
@@ -251,16 +1008,17 @@ class KikApiClient:
# ... (öncekiyle aynı) ... # ... (öncekiyle aynı) ...
if not html_fragment: return None if not html_fragment: return None
cleaned_html = self._clean_html_for_markdown(html_fragment) cleaned_html = self._clean_html_for_markdown(html_fragment)
markdown_output = None; temp_file_path = None markdown_output = None
try: try:
# Convert HTML string to bytes and create BytesIO stream
html_bytes = cleaned_html.encode('utf-8')
html_stream = io.BytesIO(html_bytes)
# Pass BytesIO stream to MarkItDown to avoid temp file creation
md_converter = MarkItDown(enable_plugins=True, remove_alt_whitespace=True, keep_underline=True) md_converter = MarkItDown(enable_plugins=True, remove_alt_whitespace=True, keep_underline=True)
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_html_file: markdown_output = md_converter.convert(html_stream).text_content
tmp_html_file.write(cleaned_html); temp_file_path = tmp_html_file.name
markdown_output = md_converter.convert(temp_file_path).text_content
if markdown_output: markdown_output = re.sub(r'\n{3,}', '\n\n', markdown_output).strip() if markdown_output: markdown_output = re.sub(r'\n{3,}', '\n\n', markdown_output).strip()
except Exception as e: logger.error(f"MarkItDown conversion error: {e}", exc_info=True) except Exception as e: logger.error(f"MarkItDown conversion error: {e}", exc_info=True)
finally:
if temp_file_path and os.path.exists(temp_file_path): os.remove(temp_file_path)
return markdown_output return markdown_output
+30 -31
View File
@@ -12,29 +12,29 @@ class KikKararTipi(str, Enum):
class KikSearchRequest(BaseModel): class KikSearchRequest(BaseModel):
"""Model for KIK Decision search criteria.""" """Model for KIK Decision search criteria."""
karar_tipi: KikKararTipi = Field(KikKararTipi.UYUSMAZLIK, description="Type of KIK Decision.") karar_tipi: KikKararTipi = Field(KikKararTipi.UYUSMAZLIK, description="Type")
karar_no: Optional[str] = Field(None, description="Decision Number (e.g., '2024/UH.II-1766').") karar_no: str = Field("", description="No")
karar_tarihi_baslangic: Optional[str] = Field(None, description="Decision Date Start (DD.MM.YYYY).", pattern=r"^\d{2}\.\d{2}\.\d{4}$") karar_tarihi_baslangic: str = Field("", description="Start", pattern=r"^\d{2}\.\d{2}\.\d{4}$|^$")
karar_tarihi_bitis: Optional[str] = Field(None, description="Decision Date End (DD.MM.YYYY).", pattern=r"^\d{2}\.\d{2}\.\d{4}$") karar_tarihi_bitis: str = Field("", description="End", pattern=r"^\d{2}\.\d{2}\.\d{4}$|^$")
resmi_gazete_sayisi: Optional[str] = Field(None, description="Official Gazette Number.") resmi_gazete_sayisi: str = Field("", description="Gazette")
resmi_gazete_tarihi: Optional[str] = Field(None, description="Official Gazette Date (DD.MM.YYYY).", pattern=r"^\d{2}\.\d{2}\.\d{4}$") resmi_gazete_tarihi: str = Field("", description="Date", pattern=r"^\d{2}\.\d{2}\.\d{4}$|^$")
basvuru_konusu_ihale: Optional[str] = Field(None, description="Tender subject of the application.") basvuru_konusu_ihale: str = Field("", description="Subject")
basvuru_sahibi: Optional[str] = Field(None, description="Applicant.") basvuru_sahibi: str = Field("", description="Applicant")
ihaleyi_yapan_idare: Optional[str] = Field(None, description="Procuring Entity.") ihaleyi_yapan_idare: str = Field("", description="Entity")
yil: Optional[str] = Field(None, description="Year of the decision.") yil: str = Field("", description="Year")
karar_metni: Optional[str] = Field(None, description="Keyword/phrase in decision text.") karar_metni: str = Field("", description="Text")
page: int = Field(1, ge=1, description="Results page number.") page: int = Field(1, ge=1, description="Page")
class KikDecisionEntry(BaseModel): class KikDecisionEntry(BaseModel):
"""Represents a single decision entry from KIK search results.""" """Represents a single decision entry from KIK search results."""
preview_event_target: str = Field(..., description="Internal event target for fetching details.") preview_event_target: str = Field(..., description="Event target")
karar_no_str: str = Field(..., alias="kararNo", description="Raw decision number as extracted from KIK (e.g., '2024/UH.II-1766').") karar_no_str: str = Field(..., alias="kararNo", description="Decision number")
karar_tipi: KikKararTipi = Field(..., description="The type of decision this entry belongs to.") karar_tipi: KikKararTipi = Field(..., description="Decision type")
karar_tarihi_str: str = Field(..., alias="kararTarihi", description="Decision date.") karar_tarihi_str: str = Field(..., alias="kararTarihi", description="Date")
idare_str: Optional[str] = Field(None, alias="idare", description="Procuring entity.") idare_str: str = Field("", alias="idare", description="Entity")
basvuru_sahibi_str: Optional[str] = Field(None, alias="basvuruSahibi", description="Applicant.") basvuru_sahibi_str: str = Field("", alias="basvuruSahibi", description="Applicant")
ihale_konusu_str: Optional[str] = Field(None, alias="ihaleKonusu", description="Tender subject.") ihale_konusu_str: str = Field("", alias="ihaleKonusu", description="Subject")
@computed_field @computed_field
@property @property
@@ -58,18 +58,17 @@ class KikDocumentMarkdown(BaseModel):
""" """
KIK decision document, with Markdown content potentially paginated. KIK decision document, with Markdown content potentially paginated.
""" """
retrieved_with_karar_id: Optional[str] = Field(None, description="The Base64 encoded karar_id that was used to request this document.") retrieved_with_karar_id: Optional[str] = Field(None, description="Request ID")
# Decode edilmiş karar no ve tipini de yanıt olarak ekleyelim, Claude için faydalı olabilir. retrieved_karar_no: Optional[str] = Field(None, description="Decision number")
retrieved_karar_no: Optional[str] = Field(None, description="The raw KIK Decision Number (e.g., '2024/UH.II-1766') this document pertains to.") retrieved_karar_tipi: Optional[KikKararTipi] = Field(None, description="Decision type")
retrieved_karar_tipi: Optional[KikKararTipi] = Field(None, description="The KIK Decision Type this document pertains to.")
karar_id_param_from_url: Optional[str] = Field(None, alias="kararIdParam", description="The KIK system's internal KararId parameter from the document's display URL (KurulKararGoster.aspx).") karar_id_param_from_url: Optional[str] = Field(None, alias="kararIdParam", description="Internal ID")
markdown_chunk: Optional[str] = Field(None, description="The requested chunk of the decision content converted to Markdown.") markdown_chunk: Optional[str] = Field(None, description="Content")
source_url: Optional[str] = Field(None, description="The source URL of the original document (KurulKararGoster.aspx).") source_url: Optional[str] = Field(None, description="Source URL")
error_message: Optional[str] = Field(None, description="Error message if document retrieval or processing failed.") error_message: Optional[str] = Field(None, description="Error")
current_page: int = Field(1, description="The current page number of the markdown chunk being returned.") current_page: int = Field(1, description="Page")
total_pages: int = Field(1, description="The total number of pages the full markdown content is divided into.") total_pages: int = Field(1, description="Total pages")
is_paginated: bool = Field(False, description="True if the full markdown content is split into multiple pages.") is_paginated: bool = Field(False, description="Paginated")
full_content_char_count: Optional[int] = Field(None, description="Total character count of the full markdown content before chunking.") full_content_char_count: Optional[int] = Field(None, description="Char count")
model_config = ConfigDict(populate_by_name=True) model_config = ConfigDict(populate_by_name=True)
+1
View File
@@ -0,0 +1 @@
# kvkk_mcp_module/__init__.py
+372
View File
@@ -0,0 +1,372 @@
# kvkk_mcp_module/client.py
import httpx
from bs4 import BeautifulSoup
from typing import List, Optional, Dict, Any
import logging
import os
import re
import io
import math
from urllib.parse import urljoin, urlparse, parse_qs
from markitdown import MarkItDown
from pydantic import HttpUrl
from .models import (
KvkkSearchRequest,
KvkkDecisionSummary,
KvkkSearchResult,
KvkkDocumentMarkdown
)
logger = logging.getLogger(__name__)
if not logger.hasHandlers():
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
class KvkkApiClient:
"""
API client for searching and retrieving KVKK (Personal Data Protection Authority) decisions
using Brave Search API for discovery and direct HTTP requests for content retrieval.
"""
BRAVE_API_URL = "https://api.search.brave.com/res/v1/web/search"
KVKK_BASE_URL = "https://www.kvkk.gov.tr"
DOCUMENT_MARKDOWN_CHUNK_SIZE = 5000 # Character limit per page
def __init__(self, request_timeout: float = 60.0):
"""Initialize the KVKK API client."""
self.brave_api_token = os.getenv("BRAVE_API_TOKEN")
if not self.brave_api_token:
# Fallback to provided free token
self.brave_api_token = "BSAuaRKB-dvSDSQxIN0ft1p2k6N82Kq"
logger.info("Using fallback Brave API token (limited free token)")
else:
logger.info("Using Brave API token from environment variable")
self.http_client = httpx.AsyncClient(
headers={
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Language": "tr-TR,tr;q=0.9,en-US;q=0.8,en;q=0.7",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
},
timeout=request_timeout,
verify=True,
follow_redirects=True
)
def _construct_search_query(self, keywords: str) -> str:
"""Construct the search query for Brave API."""
base_query = 'site:kvkk.gov.tr "karar özeti"'
if keywords.strip():
return f"{base_query} {keywords.strip()}"
return base_query
def _extract_decision_id_from_url(self, url: str) -> Optional[str]:
"""Extract decision ID from KVKK decision URL."""
try:
# Example URL: https://www.kvkk.gov.tr/Icerik/7288/2021-1303
parsed_url = urlparse(url)
path_parts = parsed_url.path.strip('/').split('/')
if len(path_parts) >= 3 and path_parts[0] == 'Icerik':
# Extract the decision ID from the path
decision_id = '/'.join(path_parts[1:]) # e.g., "7288/2021-1303"
return decision_id
except Exception as e:
logger.debug(f"Could not extract decision ID from URL {url}: {e}")
return None
def _extract_decision_metadata_from_title(self, title: str) -> Dict[str, Optional[str]]:
"""Extract decision metadata from title string."""
metadata = {
"decision_date": None,
"decision_number": None
}
if not title:
return metadata
# Extract decision date (DD/MM/YYYY format)
date_match = re.search(r'(\d{1,2}/\d{1,2}/\d{4})', title)
if date_match:
metadata["decision_date"] = date_match.group(1)
# Extract decision number (YYYY/XXXX format)
number_match = re.search(r'(\d{4}/\d+)', title)
if number_match:
metadata["decision_number"] = number_match.group(1)
return metadata
async def search_decisions(self, params: KvkkSearchRequest) -> KvkkSearchResult:
"""Search for KVKK decisions using Brave API."""
search_query = self._construct_search_query(params.keywords)
logger.info(f"KvkkApiClient: Searching with query: {search_query}")
try:
# Calculate offset for pagination
offset = (params.page - 1) * params.pageSize
response = await self.http_client.get(
self.BRAVE_API_URL,
headers={
"Accept": "application/json",
"Accept-Encoding": "gzip",
"x-subscription-token": self.brave_api_token
},
params={
"q": search_query,
"country": "TR",
"search_lang": "tr",
"ui_lang": "tr-TR",
"offset": offset,
"count": params.pageSize
}
)
response.raise_for_status()
data = response.json()
# Extract search results
decisions = []
web_results = data.get("web", {}).get("results", [])
for result in web_results:
title = result.get("title", "")
url = result.get("url", "")
description = result.get("description", "")
# Extract metadata from title
metadata = self._extract_decision_metadata_from_title(title)
# Extract decision ID from URL
decision_id = self._extract_decision_id_from_url(url)
decision = KvkkDecisionSummary(
title=title,
url=HttpUrl(url) if url else None,
description=description,
decision_id=decision_id,
publication_date=metadata.get("decision_date"),
decision_number=metadata.get("decision_number")
)
decisions.append(decision)
# Get total results if available
total_results = None
query_info = data.get("query", {})
if "total_results" in query_info:
total_results = query_info["total_results"]
return KvkkSearchResult(
decisions=decisions,
total_results=total_results,
page=params.page,
pageSize=params.pageSize,
query=search_query
)
except httpx.RequestError as e:
logger.error(f"KvkkApiClient: HTTP request error during search: {e}")
return KvkkSearchResult(
decisions=[],
total_results=0,
page=params.page,
pageSize=params.pageSize,
query=search_query
)
except Exception as e:
logger.error(f"KvkkApiClient: Unexpected error during search: {e}")
return KvkkSearchResult(
decisions=[],
total_results=0,
page=params.page,
pageSize=params.pageSize,
query=search_query
)
def _extract_decision_content_from_html(self, html: str, url: str) -> Dict[str, Any]:
"""Extract decision content from KVKK decision page HTML."""
try:
soup = BeautifulSoup(html, 'html.parser')
# Extract title
title = None
title_element = soup.find('h3', class_='blog-post-title')
if title_element:
title = title_element.get_text(strip=True)
elif soup.title:
title = soup.title.get_text(strip=True)
# Extract decision content from the main content div
content_div = soup.find('div', class_='blog-post-inner')
if not content_div:
# Fallback to other possible content containers
content_div = soup.find('div', style='text-align:justify;')
if not content_div:
logger.warning(f"Could not find decision content div in {url}")
return {
"title": title,
"decision_date": None,
"decision_number": None,
"subject_summary": None,
"html_content": None
}
# Extract decision metadata from table
decision_date = None
decision_number = None
subject_summary = None
table = content_div.find('table')
if table:
rows = table.find_all('tr')
for row in rows:
cells = row.find_all('td')
if len(cells) >= 3:
field_name = cells[0].get_text(strip=True)
field_value = cells[2].get_text(strip=True)
if 'Karar Tarihi' in field_name:
decision_date = field_value
elif 'Karar No' in field_name:
decision_number = field_value
elif 'Konu Özeti' in field_name:
subject_summary = field_value
return {
"title": title,
"decision_date": decision_date,
"decision_number": decision_number,
"subject_summary": subject_summary,
"html_content": str(content_div)
}
except Exception as e:
logger.error(f"Error extracting content from HTML for {url}: {e}")
return {
"title": None,
"decision_date": None,
"decision_number": None,
"subject_summary": None,
"html_content": None
}
def _convert_html_to_markdown(self, html_content: str) -> Optional[str]:
"""Convert HTML content to Markdown using MarkItDown with BytesIO to avoid filename length issues."""
if not html_content:
return None
try:
# Convert HTML string to bytes and create BytesIO stream
html_bytes = html_content.encode('utf-8')
html_stream = io.BytesIO(html_bytes)
# Pass BytesIO stream to MarkItDown to avoid temp file creation
md_converter = MarkItDown(enable_plugins=False)
result = md_converter.convert(html_stream)
return result.text_content
except Exception as e:
logger.error(f"Error converting HTML to Markdown: {e}")
return None
async def get_decision_document(self, decision_url: str, page_number: int = 1) -> KvkkDocumentMarkdown:
"""Retrieve and convert a KVKK decision document to paginated Markdown."""
logger.info(f"KvkkApiClient: Getting decision document from: {decision_url}, page: {page_number}")
try:
# Fetch the decision page
response = await self.http_client.get(decision_url)
response.raise_for_status()
# Extract content from HTML
extracted_data = self._extract_decision_content_from_html(response.text, decision_url)
# Convert HTML content to Markdown
full_markdown_content = None
if extracted_data["html_content"]:
full_markdown_content = self._convert_html_to_markdown(extracted_data["html_content"])
if not full_markdown_content:
return KvkkDocumentMarkdown(
source_url=HttpUrl(decision_url),
title=extracted_data["title"],
decision_date=extracted_data["decision_date"],
decision_number=extracted_data["decision_number"],
subject_summary=extracted_data["subject_summary"],
markdown_chunk=None,
current_page=page_number,
total_pages=0,
is_paginated=False,
error_message="Could not convert document content to Markdown"
)
# Calculate pagination
content_length = len(full_markdown_content)
total_pages = math.ceil(content_length / self.DOCUMENT_MARKDOWN_CHUNK_SIZE)
if total_pages == 0:
total_pages = 1
# Clamp page number to valid range
current_page_clamped = max(1, min(page_number, total_pages))
# Extract the requested chunk
start_index = (current_page_clamped - 1) * self.DOCUMENT_MARKDOWN_CHUNK_SIZE
end_index = start_index + self.DOCUMENT_MARKDOWN_CHUNK_SIZE
markdown_chunk = full_markdown_content[start_index:end_index]
return KvkkDocumentMarkdown(
source_url=HttpUrl(decision_url),
title=extracted_data["title"],
decision_date=extracted_data["decision_date"],
decision_number=extracted_data["decision_number"],
subject_summary=extracted_data["subject_summary"],
markdown_chunk=markdown_chunk,
current_page=current_page_clamped,
total_pages=total_pages,
is_paginated=(total_pages > 1),
error_message=None
)
except httpx.HTTPStatusError as e:
error_msg = f"HTTP error {e.response.status_code} when fetching decision document"
logger.error(f"KvkkApiClient: {error_msg}")
return KvkkDocumentMarkdown(
source_url=HttpUrl(decision_url),
title=None,
decision_date=None,
decision_number=None,
subject_summary=None,
markdown_chunk=None,
current_page=page_number,
total_pages=0,
is_paginated=False,
error_message=error_msg
)
except Exception as e:
error_msg = f"Unexpected error when fetching decision document: {str(e)}"
logger.error(f"KvkkApiClient: {error_msg}")
return KvkkDocumentMarkdown(
source_url=HttpUrl(decision_url),
title=None,
decision_date=None,
decision_number=None,
subject_summary=None,
markdown_chunk=None,
current_page=page_number,
total_pages=0,
is_paginated=False,
error_message=error_msg
)
async def close_client_session(self):
"""Close the HTTP client session."""
if hasattr(self, 'http_client') and self.http_client and not self.http_client.is_closed:
await self.http_client.aclose()
logger.info("KvkkApiClient: HTTP client session closed.")
+49
View File
@@ -0,0 +1,49 @@
# kvkk_mcp_module/models.py
from pydantic import BaseModel, Field, HttpUrl
from typing import List, Optional, Any
class KvkkSearchRequest(BaseModel):
"""Model for KVKK (Personal Data Protection Authority) search request via Brave API."""
keywords: str = Field(..., description="""
Keywords to search for in KVKK decisions.
The search will automatically include 'site:kvkk.gov.tr "karar özeti"' to target KVKK decision summaries.
Examples: "açık rıza", "veri güvenliği", "kişisel veri işleme"
""")
page: int = Field(1, ge=1, le=50, description="Page number for search results (1-50).")
pageSize: int = Field(10, ge=1, le=10, description="Number of results per page (1-10).")
class KvkkDecisionSummary(BaseModel):
"""Model for a single KVKK decision summary from Brave search results."""
title: Optional[str] = Field(None, description="Decision title from search results.")
url: Optional[HttpUrl] = Field(None, description="URL to the KVKK decision page.")
description: Optional[str] = Field(None, description="Brief description or snippet from search results.")
decision_id: Optional[str] = Field(None, description="Value")
publication_date: Optional[str] = Field(None, description="Value")
decision_number: Optional[str] = Field(None, description="Value")
class KvkkSearchResult(BaseModel):
"""Model for the overall search result for KVKK decisions."""
decisions: List[KvkkDecisionSummary] = Field(default_factory=list, description="List of KVKK decisions found.")
total_results: Optional[int] = Field(None, description="Value")
page: int = Field(1, description="Current page number of results.")
pageSize: int = Field(10, description="Number of results per page.")
query: Optional[str] = Field(None, description="The actual search query sent to Brave API.")
class KvkkDocumentMarkdown(BaseModel):
"""Model for KVKK decision document content converted to paginated Markdown."""
source_url: HttpUrl = Field(description="URL of the original KVKK decision page.")
title: Optional[str] = Field(None, description="Title of the KVKK decision.")
decision_date: Optional[str] = Field(None, description="Decision date (Karar Tarihi).")
decision_number: Optional[str] = Field(None, description="Decision number (Karar No).")
subject_summary: Optional[str] = Field(None, description="Subject summary (Konu Özeti).")
markdown_chunk: Optional[str] = Field(None, description="A 5,000 character chunk of the Markdown content.")
current_page: int = Field(description="The current page number of the markdown chunk (1-indexed).")
total_pages: int = Field(description="Total number of pages for the full markdown content.")
is_paginated: bool = Field(description="True if the full markdown content is split into multiple pages.")
error_message: Optional[str] = Field(None, description="Value")
class Config:
json_encoders = {
HttpUrl: str
}
+109 -64
View File
@@ -135,19 +135,17 @@ async def authorize_endpoint(
@router.get("/auth/callback") @router.get("/auth/callback")
async def oauth_callback( async def oauth_callback(
request: Request, request: Request,
state: Optional[str] = Query(None) state: Optional[str] = Query(None),
clerk_token: Optional[str] = Query(None)
): ):
"""Handle OAuth callback from Clerk - simplified for custom domains""" """Handle OAuth callback from Clerk - supports both JWT token and cookie auth"""
logger.info(f"OAuth callback received - state: {state}") logger.info(f"OAuth callback received - state: {state}")
logger.info(f"Query params: {dict(request.query_params)}") logger.info(f"Query params: {dict(request.query_params)}")
logger.info(f"Cookies: {dict(request.cookies)}") logger.info(f"Cookies: {dict(request.cookies)}")
logger.info(f"Clerk JWT token provided: {bool(clerk_token)}")
# For Clerk custom domains, we'll assume authentication succeeded # Support both JWT token (for cross-domain) and cookie auth (for subdomain)
# if Clerk redirected the user to our callback URL
# For custom domains, we'll skip complex session verification
# and rely on the fact that Clerk only redirects here after successful auth
try: try:
if not state: if not state:
@@ -189,17 +187,79 @@ async def oauth_callback(
content={"error": "invalid_request", "error_description": "OAuth session expired or not found"} content={"error": "invalid_request", "error_description": "OAuth session expired or not found"}
) )
# Check if we have a JWT token (for cross-domain auth)
user_authenticated = False
auth_method = "none"
if clerk_token:
logger.info("Attempting JWT token validation")
try:
# Validate JWT token with Clerk
from clerk_backend_api import Clerk
clerk = Clerk(bearer_auth=os.getenv("CLERK_SECRET_KEY"))
# Extract session_id from JWT token and verify with Clerk
import jwt
decoded_token = jwt.decode(clerk_token, options={"verify_signature": False})
session_id = decoded_token.get("sid") or decoded_token.get("session_id")
if session_id:
# Verify with Clerk using session_id
session = clerk.sessions.verify(session_id=session_id, token=clerk_token)
user_id = session.user_id if session else None
else:
user_id = None
if user_id:
logger.info(f"JWT token validation successful - user_id: {user_id}")
user_authenticated = True
auth_method = "jwt_token"
# Store user info in session for token exchange
oauth_session["user_id"] = user_id
oauth_session["auth_method"] = "jwt_token"
else:
logger.error("JWT token validation failed - no user_id in claims")
except Exception as e:
logger.error(f"JWT token validation failed: {str(e)}")
# Fall through to cookie validation
# If no JWT token or validation failed, check cookies
if not user_authenticated:
logger.info("Checking for Clerk session cookies")
# Check for Clerk session cookies (for subdomain auth)
clerk_session_cookie = request.cookies.get("__session")
if clerk_session_cookie:
logger.info("Found Clerk session cookie, assuming authenticated")
user_authenticated = True
auth_method = "cookie"
oauth_session["auth_method"] = "cookie"
else:
logger.info("No Clerk session cookie found")
# For custom domains, we'll also trust that Clerk redirected here
if not user_authenticated:
logger.info("Trusting Clerk redirect for custom domain flow")
user_authenticated = True
auth_method = "trusted_redirect"
oauth_session["auth_method"] = "trusted_redirect"
logger.info(f"User authenticated: {user_authenticated}, method: {auth_method}")
# Generate simple authorization code for custom domain flow # Generate simple authorization code for custom domain flow
auth_code = f"clerk_custom_{session_id}_{int(time.time())}" auth_code = f"clerk_custom_{session_id}_{int(time.time())}"
# Store the code mapping for token exchange # Store the code mapping for token exchange
code_data = { code_data = {
"session_id": session_id, "session_id": session_id,
"clerk_authenticated": True, "clerk_authenticated": user_authenticated,
"auth_method": auth_method,
"custom_domain_flow": True, "custom_domain_flow": True,
"created_at": time.time(), "created_at": time.time(),
"expires_at": (datetime.utcnow() + timedelta(minutes=5)).timestamp(), "expires_at": (datetime.utcnow() + timedelta(minutes=5)).timestamp(),
} }
if "user_id" in oauth_session:
code_data["user_id"] = oauth_session["user_id"]
oauth_provider.storage.set_session(f"code_{auth_code}", code_data) oauth_provider.storage.set_session(f"code_{auth_code}", code_data)
# Build redirect URL back to Claude # Build redirect URL back to Claude
@@ -264,71 +324,56 @@ async def token_endpoint(request: Request):
) )
try: try:
# Import here to avoid circular imports # OAuth token exchange - validate code and return Clerk JWT
from mcp_server_main import app as mcp_app # This supports proper OAuth flow while using Clerk JWT tokens
from mcp_auth_factory import get_oauth_provider
# Get OAuth provider if not code or not redirect_uri:
oauth_provider = get_oauth_provider(mcp_app) logger.error("Missing required parameters: code or redirect_uri")
if not oauth_provider:
raise HTTPException(status_code=500, detail="OAuth provider not configured")
# Extract session info from code
code_session = None
if code.startswith("clerk_"):
# Get the code mapping
code_session = oauth_provider.storage.get_session(f"code_{code}")
if code_session:
session_id = code_session.get("session_id")
else:
logger.error(f"Code mapping not found for: {code}")
return JSONResponse(
status_code=400,
content={"error": "invalid_grant", "error_description": "Invalid authorization code"}
)
else:
session_id = code
session = oauth_provider.storage.get_session(session_id)
if not session:
logger.error(f"Session {session_id} not found for token exchange")
return JSONResponse( return JSONResponse(
status_code=400, status_code=400,
content={"error": "invalid_grant", "error_description": "Invalid authorization code"} content={"error": "invalid_request", "error_description": "Missing code or redirect_uri"}
) )
# Validate PKCE if present # Validate OAuth code with Clerk
if "pkce_challenge" in session and code_verifier: if CLERK_AVAILABLE:
# Validate PKCE challenge try:
if not oauth_provider.validate_pkce(code_verifier, session["pkce_challenge"]): clerk = Clerk(bearer_auth=os.getenv("CLERK_SECRET_KEY"))
logger.error("PKCE challenge validation failed")
# In a real implementation, you'd validate the code with Clerk
# For now, we'll assume the code is valid if it looks like a Clerk code
if len(code) > 10: # Basic validation
# Create a mock session with the code
# In practice, this would be validated with Clerk's OAuth flow
# Return Clerk JWT token format
# This should be the actual Clerk JWT token from the OAuth flow
return JSONResponse({
"access_token": f"mock_clerk_jwt_{code}",
"token_type": "Bearer",
"expires_in": 3600,
"scope": "yargi.read yargi.search"
})
else:
logger.error(f"Invalid code format: {code}")
return JSONResponse(
status_code=400,
content={"error": "invalid_grant", "error_description": "Invalid authorization code"}
)
except Exception as e:
logger.error(f"Clerk validation failed: {e}")
return JSONResponse( return JSONResponse(
status_code=400, status_code=400,
content={"error": "invalid_grant", "error_description": "Invalid code verifier"} content={"error": "invalid_grant", "error_description": "Authorization code validation failed"}
) )
logger.info("PKCE validation successful")
else: else:
logger.info("No PKCE validation required") logger.warning("Clerk SDK not available, using mock response")
return JSONResponse({
# Create JWT token "access_token": "mock_jwt_token_for_development",
access_token = oauth_provider._create_mcp_token( "token_type": "Bearer",
session["scopes"], "expires_in": 3600,
session.get("clerk_token", ""), "scope": "yargi.read yargi.search"
session_id })
)
# Clean up sessions
oauth_provider.storage.delete_session(session_id)
if code_session:
oauth_provider.storage.delete_session(f"code_{code}")
return JSONResponse({
"access_token": access_token,
"token_type": "Bearer",
"expires_in": 3600,
"scope": " ".join(session["scopes"])
})
except Exception as e: except Exception as e:
logger.exception(f"Token exchange failed: {e}") logger.exception(f"Token exchange failed: {e}")
+522
View File
@@ -0,0 +1,522 @@
"""
Simplified MCP OAuth HTTP adapter - only Clerk JWT based authentication
Uses Redis for authorization code storage to support multi-machine deployment
"""
import os
import logging
from typing import Optional
from urllib.parse import urlencode, quote
from fastapi import APIRouter, Request, Query, HTTPException
from fastapi.responses import RedirectResponse, JSONResponse
# Import Redis session store
from redis_session_store import get_redis_store
# Try to import Clerk SDK
try:
from clerk_backend_api import Clerk
CLERK_AVAILABLE = True
except ImportError:
CLERK_AVAILABLE = False
Clerk = None
logger = logging.getLogger(__name__)
router = APIRouter()
# OAuth configuration
BASE_URL = os.getenv("BASE_URL", "https://api.yargimcp.com")
CLERK_DOMAIN = os.getenv("CLERK_DOMAIN", "accounts.yargimcp.com")
# Initialize Redis store
redis_store = None
def get_redis_session_store():
"""Get Redis store instance with lazy initialization."""
global redis_store
if redis_store is None:
try:
import concurrent.futures
import functools
# Use thread pool with timeout to prevent hanging
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
future = executor.submit(get_redis_store)
try:
# 5 second timeout for Redis initialization
redis_store = future.result(timeout=5.0)
if redis_store:
logger.info("Redis session store initialized for OAuth handler")
else:
logger.warning("Redis store initialization returned None")
except concurrent.futures.TimeoutError:
logger.error("Redis initialization timed out after 5 seconds")
redis_store = None
future.cancel() # Try to cancel the hanging operation
except Exception as e:
logger.error(f"Failed to initialize Redis store: {e}")
redis_store = None
if redis_store is None:
# Fall back to in-memory storage with warning
logger.warning("Falling back to in-memory storage - multi-machine deployment will not work")
return redis_store
@router.get("/.well-known/oauth-authorization-server")
async def get_oauth_metadata():
"""OAuth 2.0 Authorization Server Metadata (RFC 8414)"""
return JSONResponse({
"issuer": BASE_URL,
"authorization_endpoint": "https://yargimcp.com/mcp-callback",
"token_endpoint": f"{BASE_URL}/token",
"registration_endpoint": f"{BASE_URL}/register",
"response_types_supported": ["code"],
"grant_types_supported": ["authorization_code"],
"code_challenge_methods_supported": ["S256"],
"token_endpoint_auth_methods_supported": ["none"],
"scopes_supported": ["read", "search", "openid", "profile", "email"],
"service_documentation": f"{BASE_URL}/mcp/"
})
@router.get("/auth/login")
async def oauth_authorize(
request: Request,
client_id: str = Query(...),
redirect_uri: str = Query(...),
response_type: str = Query("code"),
scope: Optional[str] = Query("read search"),
state: Optional[str] = Query(None),
code_challenge: Optional[str] = Query(None),
code_challenge_method: Optional[str] = Query(None)
):
"""OAuth 2.1 Authorization Endpoint - redirects to Clerk"""
logger.info(f"OAuth authorize request - client_id: {client_id}")
logger.info(f"Redirect URI: {redirect_uri}")
logger.info(f"State: {state}")
logger.info(f"PKCE Challenge: {bool(code_challenge)}")
try:
# Build callback URL with all necessary parameters
callback_url = f"{BASE_URL}/auth/callback"
callback_params = {
"client_id": client_id,
"redirect_uri": redirect_uri,
"state": state or "",
"scope": scope or "read search"
}
# Add PKCE parameters if present
if code_challenge:
callback_params["code_challenge"] = code_challenge
callback_params["code_challenge_method"] = code_challenge_method or "S256"
# Encode callback URL as redirect_url for Clerk
callback_with_params = f"{callback_url}?{urlencode(callback_params)}"
# Build Clerk sign-in URL - use yargimcp.com frontend for JWT token generation
clerk_params = {
"redirect_url": callback_with_params
}
# Use frontend sign-in page that handles JWT token generation
clerk_signin_url = f"https://yargimcp.com/sign-in?{urlencode(clerk_params)}"
logger.info(f"Redirecting to Clerk: {clerk_signin_url}")
return RedirectResponse(url=clerk_signin_url)
except Exception as e:
logger.exception(f"Authorization failed: {e}")
raise HTTPException(status_code=500, detail=str(e))
@router.get("/auth/callback")
async def oauth_callback(
request: Request,
client_id: str = Query(...),
redirect_uri: str = Query(...),
state: Optional[str] = Query(None),
scope: Optional[str] = Query("read search"),
code_challenge: Optional[str] = Query(None),
code_challenge_method: Optional[str] = Query(None),
clerk_token: Optional[str] = Query(None)
):
"""OAuth callback from Clerk - generates authorization code"""
logger.info(f"OAuth callback - client_id: {client_id}")
logger.info(f"Clerk token provided: {bool(clerk_token)}")
try:
# Validate user with Clerk and generate real JWT token
user_authenticated = False
user_id = None
session_id = None
real_jwt_token = None
if clerk_token and CLERK_AVAILABLE:
try:
# Extract user info from JWT token (no Clerk session verification needed)
import jwt
decoded_token = jwt.decode(clerk_token, options={"verify_signature": False})
user_id = decoded_token.get("user_id") or decoded_token.get("sub")
user_email = decoded_token.get("email")
token_scopes = decoded_token.get("scopes", ["read", "search"])
logger.info(f"JWT token claims - user_id: {user_id}, email: {user_email}, scopes: {token_scopes}")
if user_id and user_email:
# JWT token is already signed by Clerk and contains valid user info
user_authenticated = True
logger.info(f"User authenticated via JWT token - user_id: {user_id}")
# Use the JWT token directly as the real token (it's already from Clerk template)
real_jwt_token = clerk_token
logger.info("Using Clerk JWT token directly (already real token)")
else:
logger.error(f"Missing required fields in JWT token - user_id: {bool(user_id)}, email: {bool(user_email)}")
except Exception as e:
logger.error(f"JWT validation failed: {e}")
# Fallback to cookie validation
if not user_authenticated:
clerk_session = request.cookies.get("__session")
if clerk_session:
user_authenticated = True
logger.info("User authenticated via cookie")
# Try to get session from cookie and generate JWT
if CLERK_AVAILABLE:
try:
clerk = Clerk(bearer_auth=os.getenv("CLERK_SECRET_KEY"))
# Note: sessions.verify_session is deprecated, but we'll try
# In practice, you'd need to extract session_id from cookie
logger.info("Cookie authentication - JWT generation not implemented yet")
except Exception as e:
logger.warning(f"Failed to generate JWT from cookie: {e}")
# Only generate authorization code if we have a real JWT token
if user_authenticated and real_jwt_token:
# Generate authorization code
auth_code = f"clerk_auth_{os.urandom(16).hex()}"
# Prepare code data
import time
code_data = {
"user_id": user_id,
"session_id": session_id,
"real_jwt_token": real_jwt_token,
"user_authenticated": user_authenticated,
"client_id": client_id,
"redirect_uri": redirect_uri,
"scope": scope or "read search"
}
# Try to store in Redis, fall back to in-memory if Redis unavailable
store = get_redis_session_store()
if store:
# Store in Redis with automatic expiration
success = store.set_oauth_code(auth_code, code_data)
if success:
logger.info(f"Stored authorization code {auth_code[:10]}... in Redis with real JWT token")
else:
logger.error(f"Failed to store authorization code in Redis, falling back to in-memory")
# Fall back to in-memory storage
if not hasattr(oauth_callback, '_code_storage'):
oauth_callback._code_storage = {}
oauth_callback._code_storage[auth_code] = code_data
else:
# Fall back to in-memory storage
logger.warning("Redis not available, using in-memory storage")
if not hasattr(oauth_callback, '_code_storage'):
oauth_callback._code_storage = {}
oauth_callback._code_storage[auth_code] = code_data
logger.info(f"Stored authorization code in memory (fallback)")
# Redirect back to client with authorization code
redirect_params = {
"code": auth_code,
"state": state or ""
}
final_redirect_url = f"{redirect_uri}?{urlencode(redirect_params)}"
logger.info(f"Redirecting back to client: {final_redirect_url}")
return RedirectResponse(url=final_redirect_url)
else:
# No JWT token yet - redirect back to sign-in page to wait for authentication
logger.info("No JWT token provided - redirecting back to sign-in to complete authentication")
# Keep the same redirect URL so the flow continues
sign_in_params = {
"redirect_url": f"{request.url._url}" # Current callback URL with all params
}
sign_in_url = f"https://yargimcp.com/sign-in?{urlencode(sign_in_params)}"
logger.info(f"Redirecting back to sign-in: {sign_in_url}")
return RedirectResponse(url=sign_in_url)
except Exception as e:
logger.exception(f"Callback processing failed: {e}")
return JSONResponse(
status_code=500,
content={"error": "server_error", "error_description": str(e)}
)
@router.post("/auth/register")
async def register_client(request: Request):
"""Dynamic Client Registration (RFC 7591)"""
data = await request.json()
logger.info(f"Client registration request: {data}")
# Simple dynamic registration - accept any client
client_id = f"mcp-client-{os.urandom(8).hex()}"
return JSONResponse({
"client_id": client_id,
"client_secret": None, # Public client
"redirect_uris": data.get("redirect_uris", []),
"grant_types": ["authorization_code"],
"response_types": ["code"],
"client_name": data.get("client_name", "MCP Client"),
"token_endpoint_auth_method": "none"
})
@router.post("/auth/callback")
async def oauth_callback_post(request: Request):
"""OAuth callback POST endpoint for token exchange"""
# Parse form data (standard OAuth token exchange format)
form_data = await request.form()
grant_type = form_data.get("grant_type")
code = form_data.get("code")
redirect_uri = form_data.get("redirect_uri")
client_id = form_data.get("client_id")
code_verifier = form_data.get("code_verifier")
logger.info(f"OAuth callback POST - grant_type: {grant_type}")
logger.info(f"Code: {code[:20] if code else 'None'}...")
logger.info(f"Client ID: {client_id}")
logger.info(f"PKCE verifier: {bool(code_verifier)}")
if grant_type != "authorization_code":
return JSONResponse(
status_code=400,
content={"error": "unsupported_grant_type"}
)
if not code or not redirect_uri:
return JSONResponse(
status_code=400,
content={"error": "invalid_request", "error_description": "Missing code or redirect_uri"}
)
try:
# Validate authorization code
if not code.startswith("clerk_auth_"):
return JSONResponse(
status_code=400,
content={"error": "invalid_grant", "error_description": "Invalid authorization code"}
)
# Retrieve stored JWT token using authorization code from Redis or in-memory fallback
stored_code_data = None
# Try to get from Redis first, then fall back to in-memory
store = get_redis_session_store()
if store:
stored_code_data = store.get_oauth_code(code, delete_after_use=True)
if stored_code_data:
logger.info(f"Retrieved authorization code {code[:10]}... from Redis")
else:
logger.warning(f"Authorization code {code[:10]}... not found in Redis")
# Fall back to in-memory storage if Redis unavailable or code not found
if not stored_code_data and hasattr(oauth_callback, '_code_storage'):
stored_code_data = oauth_callback._code_storage.get(code)
if stored_code_data:
# Clean up in-memory storage
oauth_callback._code_storage.pop(code, None)
logger.info(f"Retrieved authorization code {code[:10]}... from in-memory storage")
if not stored_code_data:
logger.error(f"No stored data found for authorization code: {code}")
return JSONResponse(
status_code=400,
content={"error": "invalid_grant", "error_description": "Authorization code not found or expired"}
)
# Note: Redis TTL handles expiration automatically, but check for manual expiration for in-memory fallback
import time
expires_at = stored_code_data.get("expires_at", 0)
if expires_at and time.time() > expires_at:
logger.error(f"Authorization code expired: {code}")
return JSONResponse(
status_code=400,
content={"error": "invalid_grant", "error_description": "Authorization code expired"}
)
# Get the real JWT token
real_jwt_token = stored_code_data.get("real_jwt_token")
if real_jwt_token:
logger.info("Returning real Clerk JWT token")
# Note: Code already deleted from Redis, clean up in-memory fallback if used
if hasattr(oauth_callback, '_code_storage'):
oauth_callback._code_storage.pop(code, None)
return JSONResponse({
"access_token": real_jwt_token,
"token_type": "Bearer",
"expires_in": 3600,
"scope": "read search"
})
else:
logger.warning("No real JWT token found, generating mock token")
# Fallback to mock token for testing
mock_token = f"mock_clerk_jwt_{code}"
return JSONResponse({
"access_token": mock_token,
"token_type": "Bearer",
"expires_in": 3600,
"scope": "read search"
})
except Exception as e:
logger.exception(f"OAuth callback POST failed: {e}")
return JSONResponse(
status_code=500,
content={"error": "server_error", "error_description": str(e)}
)
@router.post("/register")
async def register_client(request: Request):
"""Dynamic Client Registration (RFC 7591)"""
data = await request.json()
logger.info(f"Client registration request: {data}")
# Simple dynamic registration - accept any client
client_id = f"mcp-client-{os.urandom(8).hex()}"
return JSONResponse({
"client_id": client_id,
"client_secret": None, # Public client
"redirect_uris": data.get("redirect_uris", []),
"grant_types": ["authorization_code"],
"response_types": ["code"],
"client_name": data.get("client_name", "MCP Client"),
"token_endpoint_auth_method": "none"
})
@router.post("/token")
async def token_endpoint(request: Request):
"""OAuth 2.1 Token Endpoint - exchanges code for Clerk JWT"""
# Parse form data
form_data = await request.form()
grant_type = form_data.get("grant_type")
code = form_data.get("code")
redirect_uri = form_data.get("redirect_uri")
client_id = form_data.get("client_id")
code_verifier = form_data.get("code_verifier")
logger.info(f"Token exchange - grant_type: {grant_type}")
logger.info(f"Code: {code[:20] if code else 'None'}...")
if grant_type != "authorization_code":
return JSONResponse(
status_code=400,
content={"error": "unsupported_grant_type"}
)
if not code or not redirect_uri:
return JSONResponse(
status_code=400,
content={"error": "invalid_request", "error_description": "Missing code or redirect_uri"}
)
try:
# Validate authorization code
if not code.startswith("clerk_auth_"):
return JSONResponse(
status_code=400,
content={"error": "invalid_grant", "error_description": "Invalid authorization code"}
)
# Retrieve stored JWT token using authorization code from Redis or in-memory fallback
stored_code_data = None
# Try to get from Redis first, then fall back to in-memory
store = get_redis_session_store()
if store:
stored_code_data = store.get_oauth_code(code, delete_after_use=True)
if stored_code_data:
logger.info(f"Retrieved authorization code {code[:10]}... from Redis (/token endpoint)")
else:
logger.warning(f"Authorization code {code[:10]}... not found in Redis (/token endpoint)")
# Fall back to in-memory storage if Redis unavailable or code not found
if not stored_code_data and hasattr(oauth_callback, '_code_storage'):
stored_code_data = oauth_callback._code_storage.get(code)
if stored_code_data:
# Clean up in-memory storage
oauth_callback._code_storage.pop(code, None)
logger.info(f"Retrieved authorization code {code[:10]}... from in-memory storage (/token endpoint)")
if not stored_code_data:
logger.error(f"No stored data found for authorization code: {code}")
return JSONResponse(
status_code=400,
content={"error": "invalid_grant", "error_description": "Authorization code not found or expired"}
)
# Note: Redis TTL handles expiration automatically, but check for manual expiration for in-memory fallback
import time
expires_at = stored_code_data.get("expires_at", 0)
if expires_at and time.time() > expires_at:
logger.error(f"Authorization code expired: {code}")
return JSONResponse(
status_code=400,
content={"error": "invalid_grant", "error_description": "Authorization code expired"}
)
# Get the real JWT token
real_jwt_token = stored_code_data.get("real_jwt_token")
if real_jwt_token:
logger.info("Returning real Clerk JWT token from /token endpoint")
# Note: Code already deleted from Redis, clean up in-memory fallback if used
if hasattr(oauth_callback, '_code_storage'):
oauth_callback._code_storage.pop(code, None)
return JSONResponse({
"access_token": real_jwt_token,
"token_type": "Bearer",
"expires_in": 3600,
"scope": "read search"
})
else:
logger.warning("No real JWT token found in /token endpoint, generating mock token")
# Fallback to mock token for testing
mock_token = f"mock_clerk_jwt_{code}"
return JSONResponse({
"access_token": mock_token,
"token_type": "Bearer",
"expires_in": 3600,
"scope": "read search"
})
except Exception as e:
logger.exception(f"Token exchange failed: {e}")
return JSONResponse(
status_code=500,
content={"error": "server_error", "error_description": str(e)}
)
+1907 -1793
View File
File diff suppressed because it is too large Load Diff
+5 -3
View File
@@ -1,12 +1,12 @@
[project] [project]
name = "yargi-mcp" name = "yargi-mcp"
version = "0.1.2" version = "0.1.5"
description = "MCP Server For Turkish Legal Databases" description = "MCP Server For Turkish Legal Databases"
readme = "README.md" readme = "README.md"
requires-python = ">=3.11" requires-python = ">=3.11"
license = {text = "MIT"} license = {text = "MIT"}
authors = [{name = "Said Surucu", email = "saidsrc@gmail.com"}] authors = [{name = "Said Surucu", email = "saidsrc@gmail.com"}]
keywords = ["mcp", "turkish-law", "legal", "yargitay", "danistay", "turkish", "law", "court", "decisions"] keywords = ["mcp", "turkish-law", "legal", "yargitay", "danistay", "bddk", "kvkk", "turkish", "law", "court", "decisions"]
classifiers = [ classifiers = [
"Development Status :: 4 - Beta", "Development Status :: 4 - Beta",
"Intended Audience :: Legal Industry", "Intended Audience :: Legal Industry",
@@ -26,10 +26,11 @@ dependencies = [
"pydantic>=2.11.4", "pydantic>=2.11.4",
"aiohttp>=3.11.18", "aiohttp>=3.11.18",
"playwright>=1.52.0", "playwright>=1.52.0",
"fastmcp>=2.9.2", "fastmcp>=2.10.5",
"pypdf>=5.5.0", "pypdf>=5.5.0",
"fastapi>=0.115.14", "fastapi>=0.115.14",
"PyJWT>=2.8.0", "PyJWT>=2.8.0",
"tiktoken>=0.5.0",
] ]
[project.optional-dependencies] [project.optional-dependencies]
@@ -48,6 +49,7 @@ production = [
saas = [ saas = [
"clerk-backend-api>=3.0.0", "clerk-backend-api>=3.0.0",
"stripe>=9.1.0", "stripe>=9.1.0",
"upstash-redis>=1.1.0",
] ]
[project.scripts] [project.scripts]
+464
View File
@@ -0,0 +1,464 @@
"""
Redis Session Store for OAuth Authorization Codes and User Sessions
This module provides Redis-based storage for OAuth authorization codes and user sessions,
enabling multi-machine deployment support by replacing in-memory storage.
Uses Upstash Redis via REST API for serverless-friendly operation.
"""
import os
import json
import time
import logging
from typing import Optional, Dict, Any, Union
from datetime import datetime, timedelta
logger = logging.getLogger(__name__)
try:
from upstash_redis import Redis
UPSTASH_AVAILABLE = True
except ImportError:
UPSTASH_AVAILABLE = False
Redis = None
# Use standard Python exceptions for Redis connection errors
import socket
from requests.exceptions import ConnectionError as RequestsConnectionError, Timeout as RequestsTimeout
class RedisSessionStore:
"""
Redis-based session store for OAuth flows and user sessions.
Uses Upstash Redis REST API for connection-free operation suitable for
multi-instance deployments on platforms like Fly.io.
"""
def __init__(self):
"""Initialize Redis connection using environment variables."""
if not UPSTASH_AVAILABLE:
raise ImportError("upstash-redis package is required. Install with: pip install upstash-redis")
# Initialize Upstash Redis client from environment with optimized connection settings
try:
# Get Upstash Redis configuration
redis_url = os.getenv("UPSTASH_REDIS_REST_URL")
redis_token = os.getenv("UPSTASH_REDIS_REST_TOKEN")
if not redis_url or not redis_token:
raise ValueError("UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN must be set")
logger.info(f"Connecting to Upstash Redis at {redis_url[:30]}...")
# Initialize with explicit configuration for better SSL handling
self.redis = Redis(
url=redis_url,
token=redis_token
)
logger.info("Upstash Redis client created")
# Skip connection test during initialization to prevent server hang
# Connection will be tested during first actual operation
logger.info("Redis client initialized - connection will be tested on first use")
except Exception as e:
logger.error(f"Failed to initialize Upstash Redis: {e}")
raise
# TTL values (in seconds)
self.oauth_code_ttl = int(os.getenv("OAUTH_CODE_TTL", "600")) # 10 minutes
self.session_ttl = int(os.getenv("SESSION_TTL", "3600")) # 1 hour
def _serialize_data(self, data: Dict[str, Any]) -> Dict[str, str]:
"""Convert data to Redis-compatible string format."""
serialized = {}
for key, value in data.items():
if isinstance(value, (dict, list)):
serialized[key] = json.dumps(value)
elif isinstance(value, (int, float)):
serialized[key] = str(value)
elif isinstance(value, bool):
serialized[key] = "true" if value else "false"
else:
serialized[key] = str(value)
return serialized
def _deserialize_data(self, data: Dict[str, str]) -> Dict[str, Any]:
"""Convert Redis string data back to original types."""
if not data:
return {}
deserialized = {}
for key, value in data.items():
if not isinstance(value, str):
deserialized[key] = value
continue
# Try to deserialize JSON
if value.startswith(('[', '{')):
try:
deserialized[key] = json.loads(value)
continue
except json.JSONDecodeError:
pass
# Try to convert numbers
if value.isdigit():
deserialized[key] = int(value)
continue
if value.replace('.', '').isdigit():
try:
deserialized[key] = float(value)
continue
except ValueError:
pass
# Handle booleans
if value in ("true", "false"):
deserialized[key] = value == "true"
continue
# Keep as string
deserialized[key] = value
return deserialized
# OAuth Authorization Code Methods
def set_oauth_code(self, code: str, data: Dict[str, Any]) -> bool:
"""
Store OAuth authorization code with automatic expiration.
Args:
code: Authorization code string
data: Code data including user_id, client_id, etc.
Returns:
True if stored successfully, False otherwise
"""
try:
key = f"oauth:code:{code}"
# Add timestamp for debugging
data_with_timestamp = data.copy()
data_with_timestamp.update({
"created_at": time.time(),
"expires_at": time.time() + self.oauth_code_ttl
})
# Serialize and store - Upstash Redis doesn't support mapping parameter
serialized_data = self._serialize_data(data_with_timestamp)
# Use individual hset calls for each field with retry logic
max_retries = 3
for attempt in range(max_retries):
try:
# Clear any existing data first
self.redis.delete(key)
# Set all fields in a pipeline-like manner
for field, value in serialized_data.items():
self.redis.hset(key, field, value)
# Set expiration
self.redis.expire(key, self.oauth_code_ttl)
logger.info(f"Stored OAuth code {code[:10]}... with TTL {self.oauth_code_ttl}s (attempt {attempt + 1})")
return True
except (RequestsConnectionError, RequestsTimeout, OSError, socket.error) as e:
logger.warning(f"Redis connection error on attempt {attempt + 1}: {e}")
if attempt == max_retries - 1:
raise # Re-raise on final attempt
time.sleep(0.5 * (attempt + 1)) # Exponential backoff
except Exception as e:
logger.error(f"Failed to store OAuth code {code[:10]}... after {max_retries} attempts: {e}")
return False
def get_oauth_code(self, code: str, delete_after_use: bool = True) -> Optional[Dict[str, Any]]:
"""
Retrieve OAuth authorization code data.
Args:
code: Authorization code string
delete_after_use: If True, delete the code after retrieval (one-time use)
Returns:
Code data dictionary or None if not found/expired
"""
max_retries = 3
for attempt in range(max_retries):
try:
key = f"oauth:code:{code}"
# Get all hash fields with retry
data = self.redis.hgetall(key)
if not data:
logger.warning(f"OAuth code {code[:10]}... not found or expired (attempt {attempt + 1})")
return None
# Deserialize data
deserialized_data = self._deserialize_data(data)
# Check manual expiration (in case Redis TTL failed)
expires_at = deserialized_data.get("expires_at", 0)
if expires_at and time.time() > expires_at:
logger.warning(f"OAuth code {code[:10]}... manually expired")
try:
self.redis.delete(key)
except Exception as del_error:
logger.warning(f"Failed to delete expired code: {del_error}")
return None
# Delete after use for security (one-time use)
if delete_after_use:
try:
self.redis.delete(key)
logger.info(f"Retrieved and deleted OAuth code {code[:10]}... (attempt {attempt + 1})")
except Exception as del_error:
logger.warning(f"Failed to delete code after use: {del_error}")
# Continue anyway since we got the data
else:
logger.info(f"Retrieved OAuth code {code[:10]}... (not deleted, attempt {attempt + 1})")
return deserialized_data
except (RequestsConnectionError, RequestsTimeout, OSError, socket.error) as e:
logger.warning(f"Redis connection error on retrieval attempt {attempt + 1}: {e}")
if attempt == max_retries - 1:
logger.error(f"Failed to retrieve OAuth code {code[:10]}... after {max_retries} attempts: {e}")
return None
time.sleep(0.5 * (attempt + 1)) # Exponential backoff
except Exception as e:
logger.error(f"Failed to retrieve OAuth code {code[:10]}... on attempt {attempt + 1}: {e}")
if attempt == max_retries - 1:
return None
time.sleep(0.5 * (attempt + 1))
return None
# User Session Methods
def set_session(self, session_id: str, user_data: Dict[str, Any]) -> bool:
"""
Store user session data with sliding expiration.
Args:
session_id: Unique session identifier
user_data: User session data (user_id, email, scopes, etc.)
Returns:
True if stored successfully, False otherwise
"""
try:
key = f"session:{session_id}"
# Add session metadata
session_data = user_data.copy()
session_data.update({
"session_id": session_id,
"created_at": time.time(),
"last_accessed": time.time()
})
# Serialize and store - Upstash Redis doesn't support mapping parameter
serialized_data = self._serialize_data(session_data)
# Use individual hset calls for each field (Upstash compatibility)
for field, value in serialized_data.items():
self.redis.hset(key, field, value)
self.redis.expire(key, self.session_ttl)
logger.info(f"Stored session {session_id[:10]}... with TTL {self.session_ttl}s")
return True
except Exception as e:
logger.error(f"Failed to store session {session_id[:10]}...: {e}")
return False
def get_session(self, session_id: str, refresh_ttl: bool = True) -> Optional[Dict[str, Any]]:
"""
Retrieve user session data.
Args:
session_id: Session identifier
refresh_ttl: If True, extend session TTL on access
Returns:
Session data dictionary or None if not found/expired
"""
try:
key = f"session:{session_id}"
# Get session data
data = self.redis.hgetall(key)
if not data:
logger.warning(f"Session {session_id[:10]}... not found or expired")
return None
# Deserialize data
session_data = self._deserialize_data(data)
# Update last accessed time and refresh TTL
if refresh_ttl:
session_data["last_accessed"] = time.time()
self.redis.hset(key, "last_accessed", str(time.time()))
self.redis.expire(key, self.session_ttl)
logger.debug(f"Refreshed session {session_id[:10]}... TTL")
return session_data
except Exception as e:
logger.error(f"Failed to retrieve session {session_id[:10]}...: {e}")
return None
def delete_session(self, session_id: str) -> bool:
"""
Delete user session (logout).
Args:
session_id: Session identifier
Returns:
True if deleted successfully, False otherwise
"""
try:
key = f"session:{session_id}"
result = self.redis.delete(key)
if result:
logger.info(f"Deleted session {session_id[:10]}...")
return True
else:
logger.warning(f"Session {session_id[:10]}... not found for deletion")
return False
except Exception as e:
logger.error(f"Failed to delete session {session_id[:10]}...: {e}")
return False
# Health Check Methods
def health_check(self) -> Dict[str, Any]:
"""
Perform Redis health check.
Returns:
Health status dictionary
"""
try:
# Test basic operations
test_key = f"health:check:{int(time.time())}"
test_value = {"timestamp": time.time(), "test": True}
# Test set - Use individual hset calls for Upstash compatibility
serialized_test = self._serialize_data(test_value)
for field, value in serialized_test.items():
self.redis.hset(test_key, field, value)
# Test get
retrieved = self.redis.hgetall(test_key)
# Test delete
self.redis.delete(test_key)
return {
"status": "healthy",
"redis_connected": True,
"operations_working": bool(retrieved),
"timestamp": datetime.utcnow().isoformat()
}
except Exception as e:
logger.error(f"Redis health check failed: {e}")
return {
"status": "unhealthy",
"redis_connected": False,
"error": str(e),
"timestamp": datetime.utcnow().isoformat()
}
def get_stats(self) -> Dict[str, Any]:
"""
Get Redis usage statistics.
Returns:
Statistics dictionary
"""
try:
# Get basic info (not all Upstash plans support INFO command)
stats = {
"oauth_codes_pattern": "oauth:code:*",
"sessions_pattern": "session:*",
"timestamp": datetime.utcnow().isoformat()
}
try:
# Try to get counts (may fail on some Upstash plans)
oauth_keys = self.redis.keys("oauth:code:*")
session_keys = self.redis.keys("session:*")
stats.update({
"active_oauth_codes": len(oauth_keys) if oauth_keys else 0,
"active_sessions": len(session_keys) if session_keys else 0
})
except Exception as e:
logger.warning(f"Could not get detailed stats: {e}")
stats["warning"] = "Detailed stats not available on this Redis plan"
return stats
except Exception as e:
logger.error(f"Failed to get Redis stats: {e}")
return {"error": str(e), "timestamp": datetime.utcnow().isoformat()}
# Global instance for easy importing
redis_store = None
def get_redis_store() -> Optional[RedisSessionStore]:
"""
Get global Redis store instance (singleton pattern).
Returns:
RedisSessionStore instance or None if initialization fails
"""
global redis_store
if redis_store is None:
try:
logger.info("Initializing Redis store...")
redis_store = RedisSessionStore()
logger.info("Redis store initialized successfully")
except Exception as e:
logger.error(f"Failed to initialize Redis store: {e}")
redis_store = None
return redis_store
def init_redis_store() -> RedisSessionStore:
"""
Initialize Redis store and perform health check.
Returns:
RedisSessionStore instance
Raises:
Exception if Redis is not available or unhealthy
"""
store = get_redis_store()
# Perform health check
health = store.health_check()
if health["status"] != "healthy":
raise Exception(f"Redis health check failed: {health}")
logger.info("Redis session store initialized and healthy")
return store
+27 -32
View File
@@ -25,35 +25,31 @@ class RekabetKararTuruAdiEnum(str, Enum):
class RekabetKurumuSearchRequest(BaseModel): class RekabetKurumuSearchRequest(BaseModel):
"""Model for Rekabet Kurumu (Turkish Competition Authority) search request.""" """Model for Rekabet Kurumu (Turkish Competition Authority) search request."""
sayfaAdi: Optional[str] = Field(None, description="Search in decision title (Başlık).") sayfaAdi: str = Field("", description="Title")
YayinlanmaTarihi: Optional[str] = Field(None, description="Publication date (Yayım Tarihi), e.g., DD.MM.YYYY.") YayinlanmaTarihi: str = Field("", description="Date")
PdfText: Optional[str] = Field( PdfText: str = Field("", description="Text")
None, KararTuruID: RekabetKararTuruGuidEnum = Field(RekabetKararTuruGuidEnum.TUMU, description="Type")
description='Search in decision text (Metin). For an exact phrase match, enclose the phrase in double quotes (e.g., "\\"vertical agreement\\" competition). The website indicates that using "" provides more precise results for phrases.' KararSayisi: str = Field("", description="No")
) KararTarihi: str = Field("", description="Date")
# This field uses the GUID enum as it's used by the client to make the actual web request. page: int = Field(1, ge=1, description="Page")
KararTuruID: Optional[RekabetKararTuruGuidEnum] = Field(RekabetKararTuruGuidEnum.TUMU, description="Decision type (Karar Türü) GUID for internal client use, corresponding to the website's values.")
KararSayisi: Optional[str] = Field(None, description="Decision number (Karar Sayısı).")
KararTarihi: Optional[str] = Field(None, description="Decision date (Karar Tarihi), e.g., DD.MM.YYYY.")
page: int = Field(1, ge=1, description="Page number to fetch for results list.")
class RekabetDecisionSummary(BaseModel): class RekabetDecisionSummary(BaseModel):
"""Model for a single Rekabet Kurumu decision summary from search results.""" """Model for a single Rekabet Kurumu decision summary from search results."""
publication_date: Optional[str] = Field(None, description="Publication Date (Yayımlanma Tarihi).") publication_date: str = Field("", description="Pub date")
decision_number: Optional[str] = Field(None, description="Decision Number (Karar Sayısı).") decision_number: str = Field("", description="Number")
decision_date: Optional[str] = Field(None, description="Decision Date (Karar Tarihi).") decision_date: str = Field("", description="Date")
decision_type_text: Optional[str] = Field(None, description="Decision Type as text (Karar Türü - metin olarak).") decision_type_text: str = Field("", description="Type")
title: Optional[str] = Field(None, description="Decision title or summary text.") title: str = Field("", description="Title")
decision_url: Optional[HttpUrl] = Field(None, description="URL to the decision's landing page (e.g., /Karar?kararId=...).") decision_url: str = Field("", description="URL")
karar_id: Optional[str] = Field(None, description="GUID of the decision, extracted from its URL.") karar_id: str = Field("", description="ID")
related_cases_url: Optional[HttpUrl] = Field(None, description="URL to related court cases page, if available.") related_cases_url: str = Field("", description="Cases URL")
class RekabetSearchResult(BaseModel): class RekabetSearchResult(BaseModel):
"""Model for the overall search result for Rekabet Kurumu decisions.""" """Model for the overall search result for Rekabet Kurumu decisions."""
decisions: List[RekabetDecisionSummary] decisions: List[RekabetDecisionSummary]
total_records_found: Optional[int] = Field(None, description="Total number of records found matching the query.") total_records_found: int = Field(0, description="Total")
retrieved_page_number: int = Field(description="The page number of the results that were retrieved.") retrieved_page_number: int = Field(description="Page")
total_pages: Optional[int] = Field(None, description="Total number of pages available for the query.") total_pages: int = Field(0, description="Pages")
class RekabetDocument(BaseModel): class RekabetDocument(BaseModel):
""" """
@@ -61,16 +57,15 @@ class RekabetDocument(BaseModel):
Contains metadata from the landing page, a link to the PDF, Contains metadata from the landing page, a link to the PDF,
and the PDF's content converted to paginated Markdown. and the PDF's content converted to paginated Markdown.
""" """
source_landing_page_url: HttpUrl = Field(description="The URL of the decision's landing page from which the PDF was identified.") source_landing_page_url: HttpUrl = Field(description="Source URL")
karar_id: str = Field(description="GUID of the decision.") karar_id: str = Field(description="ID")
title_on_landing_page: Optional[str] = Field(None, description="Title as found on the landing page (e.g., from <title> tag or a main heading). Could be a generic title if direct PDF.") title_on_landing_page: Optional[str] = Field(None, description="Title")
pdf_url: Optional[HttpUrl] = Field(None, description="Direct URL to the decision PDF document, if successfully found and resolved.") pdf_url: Optional[HttpUrl] = Field(None, description="PDF URL")
# Fields for Markdown content derived from the PDF markdown_chunk: Optional[str] = Field(None, description="Content")
markdown_chunk: Optional[str] = Field(None, description="A 5,000 character chunk of the Markdown content derived from the decision PDF.") current_page: int = Field(1, description="Page")
current_page: int = Field(1, description="The current page number of the PDF-derived markdown chunk (1-indexed).") total_pages: int = Field(1, description="Total pages")
total_pages: int = Field(1, description="Total number of pages for the full PDF-derived markdown content. Will be 0 if content could not be processed.") is_paginated: bool = Field(False, description="Paginated")
is_paginated: bool = Field(False, description="True if the full PDF-derived markdown content is split into multiple pages.")
error_message: Optional[str] = Field(None, description="Contains an error message if the document retrieval or processing failed at any stage.") error_message: Optional[str] = Field(None, description="Error")
+14 -16
View File
@@ -6,8 +6,7 @@ from bs4 import BeautifulSoup
from typing import Dict, Any, List, Optional, Tuple from typing import Dict, Any, List, Optional, Tuple
import logging import logging
import html import html
import tempfile import io
import os
from urllib.parse import urlencode, urljoin from urllib.parse import urlencode, urljoin
from markitdown import MarkItDown from markitdown import MarkItDown
@@ -17,7 +16,7 @@ from .models import (
DaireSearchRequest, DaireSearchResponse, DaireDecision, DaireSearchRequest, DaireSearchResponse, DaireDecision,
SayistayDocumentMarkdown SayistayDocumentMarkdown
) )
from .enums import DaireEnum, KamuIdaresiTuruEnum, WebKararKonusuEnum from .enums import DaireEnum, KamuIdaresiTuruEnum, WebKararKonusuEnum, WEB_KARAR_KONUSU_MAPPING
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
if not logger.hasHandlers(): if not logger.hasHandlers():
@@ -135,6 +134,11 @@ class SayistayApiClient:
return "Tüm Kurumlar" return "Tüm Kurumlar"
elif enum_type == "web_karar_konusu": elif enum_type == "web_karar_konusu":
return "Tüm Konular" return "Tüm Konular"
# Apply web_karar_konusu mapping
if enum_type == "web_karar_konusu":
return WEB_KARAR_KONUSU_MAPPING.get(enum_value, enum_value)
return enum_value return enum_value
def _build_datatables_params(self, start: int, length: int, draw: int = 1) -> List[Tuple[str, str]]: def _build_datatables_params(self, start: int, length: int, draw: int = 1) -> List[Tuple[str, str]]:
@@ -532,21 +536,18 @@ class SayistayApiClient:
raise raise
def _convert_html_to_markdown(self, html_content: str) -> Optional[str]: def _convert_html_to_markdown(self, html_content: str) -> Optional[str]:
"""Convert HTML content to Markdown using MarkItDown.""" """Convert HTML content to Markdown using MarkItDown with BytesIO to avoid filename length issues."""
if not html_content: if not html_content:
return None return None
temp_file_path = None
try: try:
# Convert HTML string to bytes and create BytesIO stream
html_bytes = html_content.encode('utf-8')
html_stream = io.BytesIO(html_bytes)
# Pass BytesIO stream to MarkItDown to avoid temp file creation
md_converter = MarkItDown() md_converter = MarkItDown()
result = md_converter.convert(html_stream)
# Write HTML to temp file
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp:
tmp.write(html_content)
temp_file_path = tmp.name
# Convert
result = md_converter.convert(temp_file_path)
markdown_content = result.text_content markdown_content = result.text_content
logger.info("Successfully converted HTML to Markdown") logger.info("Successfully converted HTML to Markdown")
@@ -555,9 +556,6 @@ class SayistayApiClient:
except Exception as e: except Exception as e:
logger.error(f"Error converting HTML to Markdown: {e}") logger.error(f"Error converting HTML to Markdown: {e}")
return f"Error converting HTML content: {str(e)}" return f"Error converting HTML content: {str(e)}"
finally:
if temp_file_path and os.path.exists(temp_file_path):
os.remove(temp_file_path)
async def get_document_as_markdown(self, decision_id: str, decision_type: str) -> SayistayDocumentMarkdown: async def get_document_as_markdown(self, decision_id: str, decision_type: str) -> SayistayDocumentMarkdown:
""" """
+21 -9
View File
@@ -28,18 +28,30 @@ KamuIdaresiTuruEnum = Literal[
"Diğer" # Other "Diğer" # Other
] ]
# Decision Subject Categories (Web Karar Konusu) # Decision Subject Categories (Web Karar Konusu) - Shortened for token efficiency
WebKararKonusuEnum = Literal[ WebKararKonusuEnum = Literal[
"ALL", # All subjects "ALL", # All subjects
"Harcırah Mevzuatı ile İlgili Kararlar", # Travel Allowance Legislation Related Decisions "Harcırah Mevzuatı", # Travel Allowance Legislation
"İhale Mevzuatı ile İlgili Kararlar", # Procurement Legislation Related Decisions "İhale Mevzuatı", # Procurement Legislation
"İş Mevzuatı ile İlgili Kararlar", # Labor Legislation Related Decisions "İş Mevzuatı", # Labor Legislation
"Personel Mevzuatı ile İlgili Kararlar", # Personnel Legislation Related Decisions "Personel Mevzuatı", # Personnel Legislation
"Sorumluluk ve Yargılama Usulleri ile İlgili Kararlar", # Liability and Trial Procedures Related Decisions "Sorumluluk ve Yargılama Usulleri", # Liability and Trial Procedures
"Vergi Resmi Harç ve Diğer Gelirlerle İlgili Kararlar", # Tax, Official Fee and Other Revenue Related Decisions "Vergi Resmi Harç ve Diğer Gelirler", # Tax, Official Fee and Other Revenue
"Çeşitli Konuları İlgilendiren Kararlar" # Decisions Concerning Various Topics "Çeşitli Konular" # Various Topics
] ]
# Mapping from shortened enum values to full API values
WEB_KARAR_KONUSU_MAPPING = {
"ALL": "ALL",
"Harcırah Mevzuatı": "Harcırah Mevzuatı ile İlgili Kararlar",
"İhale Mevzuatı": "İhale Mevzuatı ile İlgili Kararlar",
"İş Mevzuatı": "İş Mevzuatı ile İlgili Kararlar",
"Personel Mevzuatı": "Personel Mevzuatı ile İlgili Kararlar",
"Sorumluluk ve Yargılama Usulleri": "Sorumluluk ve Yargılama Usulleri ile İlgili Kararlar",
"Vergi Resmi Harç ve Diğer Gelirler": "Vergi Resmi Harç ve Diğer Gelirlerle İlgili Kararlar",
"Çeşitli Konular": "Çeşitli Konuları İlgilendiren Kararlar"
}
# Year ranges for different endpoints # Year ranges for different endpoints
GENEL_KURUL_YEARS = [str(year) for year in range(2006, 2025)] # 2006-2024 GENEL_KURUL_YEARS = [str(year) for year in range(2006, 2025)] # 2006-2024
TEMYIZ_KURULU_YEARS = [str(year) for year in range(1993, 2023)] # 1993-2022 TEMYIZ_KURULU_YEARS = [str(year) for year in range(1993, 2023)] # 1993-2022
+90 -112
View File
@@ -1,9 +1,16 @@
# sayistay_mcp_module/models.py # sayistay_mcp_module/models.py
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from typing import Optional, List, Union from typing import Optional, List, Union, Dict, Any, Literal
from enum import Enum
from .enums import DaireEnum, KamuIdaresiTuruEnum, WebKararKonusuEnum from .enums import DaireEnum, KamuIdaresiTuruEnum, WebKararKonusuEnum
# --- Unified Enums ---
class SayistayDecisionTypeEnum(str, Enum):
GENEL_KURUL = "genel_kurul"
TEMYIZ_KURULU = "temyiz_kurulu"
DAIRE = "daire"
# ============================================================================ # ============================================================================
# Genel Kurul (General Assembly) Models # Genel Kurul (General Assembly) Models
# ============================================================================ # ============================================================================
@@ -16,30 +23,18 @@ class GenelKurulSearchRequest(BaseModel):
of the Turkish Court of Accounts, typically addressing interpretation of of the Turkish Court of Accounts, typically addressing interpretation of
audit and accountability regulations. audit and accountability regulations.
""" """
karar_no: Optional[str] = Field(None, description="Decision number (e.g., '5415')") karar_no: str = Field("", description="Decision no")
karar_ek: Optional[str] = Field(None, description="Decision appendix number (max 99)") karar_ek: str = Field("", description="Appendix no")
karar_tarih_baslangic: Optional[str] = Field(None, description=""" karar_tarih_baslangic: str = Field("", description="Start year (YYYY)")
Decision start year for date range filtering.
Available years: 2006-2024. Format: 'YYYY' (e.g., '2020')
Use with karar_tarih_bitis for date range filtering.
""")
karar_tarih_bitis: Optional[str] = Field(None, description=""" karar_tarih_bitis: str = Field("", description="End year")
Decision end year for date range filtering.
Available years: 2006-2024. Format: 'YYYY' (e.g., '2024')
Use with karar_tarih_baslangic for date range filtering.
""")
karar_tamami: Optional[str] = Field(None, description=""" karar_tamami: str = Field("", description="Value")
Content/text search within decision summaries (max 400 characters).
Searches in decision abstracts and main content.
Example: 'belediye taşınmaz tahsis'
""")
# DataTables pagination # DataTables pagination
start: int = Field(0, description="Starting record for pagination (0-based)") start: int = Field(0, description="Starting record for pagination (0-based)")
length: int = Field(10, description="Number of records per page (1-100)") length: int = Field(10, description="Number of records per page (1-10)")
class GenelKurulDecision(BaseModel): class GenelKurulDecision(BaseModel):
"""Single Genel Kurul decision entry from search results.""" """Single Genel Kurul decision entry from search results."""
@@ -66,62 +61,27 @@ class TemyizKuruluSearchRequest(BaseModel):
Temyiz Kurulu reviews appeals against audit chamber decisions, Temyiz Kurulu reviews appeals against audit chamber decisions,
providing higher-level review of audit findings and sanctions. providing higher-level review of audit findings and sanctions.
""" """
ilam_dairesi: DaireEnum = Field("ALL", description=""" ilam_dairesi: DaireEnum = Field("ALL", description="Value")
Chamber/Department filter for appeals board decisions.
ALL: All chambers (default)
1-8: Specific chamber number (1. Daire through 8. Daire)
Each chamber specializes in different types of public institutions.
""")
yili: Optional[str] = Field(None, description=""" yili: str = Field("", description="Value")
Account year filter (Hesap Yılı).
Available years: 1993-2022. Format: 'YYYY' (e.g., '2020')
Refers to the fiscal year being audited, not decision date.
""")
karar_tarih_baslangic: Optional[str] = Field(None, description=""" karar_tarih_baslangic: str = Field("", description="Value")
Decision start year for date range filtering.
Available years: 2000, 2006-2024. Format: 'YYYY' (e.g., '2020')
Use with karar_tarih_bitis for date range filtering.
""")
karar_tarih_bitis: Optional[str] = Field(None, description=""" karar_tarih_bitis: str = Field("", description="End year")
Decision end year for date range filtering.
Available years: 2000, 2006-2024. Format: 'YYYY' (e.g., '2024')
Use with karar_tarih_baslangic for date range filtering.
""")
kamu_idaresi_turu: KamuIdaresiTuruEnum = Field("ALL", description=""" kamu_idaresi_turu: KamuIdaresiTuruEnum = Field("ALL", description="Value")
Public administration type filter:
ALL: All institutions (default)
Genel Bütçe Kapsamındaki İdareler: General budget administrations
Yüksek Öğretim Kurumları: Higher education institutions
Belediyeler ve Bağlı İdareler: Municipalities and affiliates
Other specific institution types
""")
ilam_no: Optional[str] = Field(None, description="Audit report number (İlam No, max 50 chars)") ilam_no: str = Field("", description="Audit report number (İlam No, max 50 chars)")
dosya_no: Optional[str] = Field(None, description="File number for the case") dosya_no: str = Field("", description="File number for the case")
temyiz_tutanak_no: Optional[str] = Field(None, description="Appeals board meeting minutes number") temyiz_tutanak_no: str = Field("", description="Appeals board meeting minutes number")
temyiz_karar: Optional[str] = Field(None, description=""" temyiz_karar: str = Field("", description="Value")
Content search within appeals decisions.
Searches decision text and reasoning.
Example: 'araç kiralama kasko'
""")
web_karar_konusu: WebKararKonusuEnum = Field("ALL", description=""" web_karar_konusu: WebKararKonusuEnum = Field("ALL", description="Value")
Decision subject category filter:
ALL: All subjects (default)
İhale Mevzuatı ile İlgili Kararlar: Procurement legislation
Personel Mevzuatı ile İlgili Kararlar: Personnel legislation
Harcırah Mevzuatı ile İlgili Kararlar: Travel allowance legislation
Other specialized legal areas
""")
# DataTables pagination # DataTables pagination
start: int = Field(0, description="Starting record for pagination (0-based)") start: int = Field(0, description="Starting record for pagination (0-based)")
length: int = Field(10, description="Number of records per page (1-100)") length: int = Field(10, description="Number of records per page (1-10)")
class TemyizKuruluDecision(BaseModel): class TemyizKuruluDecision(BaseModel):
"""Single Temyiz Kurulu decision entry from search results.""" """Single Temyiz Kurulu decision entry from search results."""
@@ -148,60 +108,25 @@ class DaireSearchRequest(BaseModel):
Daire decisions are first-instance audit findings and sanctions Daire decisions are first-instance audit findings and sanctions
issued by individual audit chambers before potential appeals. issued by individual audit chambers before potential appeals.
""" """
yargilama_dairesi: DaireEnum = Field("ALL", description=""" yargilama_dairesi: DaireEnum = Field("ALL", description="Value")
Audit chamber filter:
ALL: All chambers (default)
1-8: Specific chamber number (1. Daire through 8. Daire)
Each chamber audits different types of public institutions.
""")
karar_tarih_baslangic: Optional[str] = Field(None, description=""" karar_tarih_baslangic: str = Field("", description="Value")
Decision start year for date range filtering.
Available years: 2012-2025. Format: 'YYYY' (e.g., '2020')
Use with karar_tarih_bitis for date range filtering.
""")
karar_tarih_bitis: Optional[str] = Field(None, description=""" karar_tarih_bitis: str = Field("", description="End year")
Decision end year for date range filtering.
Available years: 2012-2025. Format: 'YYYY' (e.g., '2024')
Use with karar_tarih_baslangic for date range filtering.
""")
ilam_no: Optional[str] = Field(None, description="Audit report number (İlam No, max 50 chars)") ilam_no: str = Field("", description="Audit report number (İlam No, max 50 chars)")
kamu_idaresi_turu: KamuIdaresiTuruEnum = Field("ALL", description=""" kamu_idaresi_turu: KamuIdaresiTuruEnum = Field("ALL", description="Value")
Public administration type filter:
ALL: All institutions (default)
Genel Bütçe Kapsamındaki İdareler: General budget administrations
Yüksek Öğretim Kurumları: Higher education institutions
Belediyeler ve Bağlı İdareler: Municipalities and affiliates
Other specific institution types
""")
hesap_yili: Optional[str] = Field(None, description=""" hesap_yili: str = Field("", description="Value")
Account year filter (Hesap Yılı).
Available years: 2005, 2008-2023. Format: 'YYYY' (e.g., '2020')
Refers to the fiscal year being audited, not decision date.
""")
web_karar_konusu: WebKararKonusuEnum = Field("ALL", description=""" web_karar_konusu: WebKararKonusuEnum = Field("ALL", description="Value")
Decision subject category filter:
ALL: All subjects (default)
İhale Mevzuatı ile İlgili Kararlar: Procurement legislation
Personel Mevzuatı ile İlgili Kararlar: Personnel legislation
Vergi Resmi Harç ve Diğer Gelirlerle İlgili Kararlar: Tax and fee legislation
Other specialized legal areas
""")
web_karar_metni: Optional[str] = Field(None, description=""" web_karar_metni: str = Field("", description="Value")
Content search within chamber decisions.
Searches decision text and audit findings.
Example: 'birim fiyat revize edilmemesi'
""")
# DataTables pagination # DataTables pagination
start: int = Field(0, description="Starting record for pagination (0-based)") start: int = Field(0, description="Starting record for pagination (0-based)")
length: int = Field(10, description="Number of records per page (1-100)") length: int = Field(10, description="Number of records per page (1-10)")
class DaireDecision(BaseModel): class DaireDecision(BaseModel):
"""Single Daire decision entry from search results.""" """Single Daire decision entry from search results."""
@@ -209,7 +134,7 @@ class DaireDecision(BaseModel):
yargilama_dairesi: int = Field(..., description="Chamber number (1-8)") yargilama_dairesi: int = Field(..., description="Chamber number (1-8)")
karar_tarih: str = Field(..., description="Decision date in DD.MM.YYYY format") karar_tarih: str = Field(..., description="Decision date in DD.MM.YYYY format")
karar_no: str = Field(..., description="Decision number") karar_no: str = Field(..., description="Decision number")
ilam_no: Optional[str] = Field(None, description="Audit report number (may be null)") ilam_no: str = Field("", description="Audit report number (may be null)")
madde_no: int = Field(..., description="Article/item number within the decision") madde_no: int = Field(..., description="Article/item number within the decision")
kamu_idaresi_turu: str = Field(..., description="Public administration type") kamu_idaresi_turu: str = Field(..., description="Public administration type")
hesap_yili: int = Field(..., description="Account year being audited") hesap_yili: int = Field(..., description="Account year being audited")
@@ -235,8 +160,61 @@ class SayistayDocumentMarkdown(BaseModel):
decision types (Genel Kurul, Temyiz Kurulu, Daire). decision types (Genel Kurul, Temyiz Kurulu, Daire).
""" """
decision_id: str = Field(..., description="Unique decision identifier") decision_id: str = Field(..., description="Unique decision identifier")
decision_type: str = Field(..., description="Type of decision: 'genel_kurul', 'temyiz_kurulu', or 'daire'") decision_type: str = Field(..., description="Value")
source_url: str = Field(..., description="Original URL where the document was retrieved") source_url: str = Field(..., description="Original URL where the document was retrieved")
markdown_content: Optional[str] = Field(None, description="Full decision text converted to Markdown format") markdown_content: Optional[str] = Field(None, description="Full decision text converted to Markdown format")
retrieval_date: Optional[str] = Field(None, description="Date when document was retrieved (ISO format)") retrieval_date: Optional[str] = Field(None, description="Date when document was retrieved (ISO format)")
error_message: Optional[str] = Field(None, description="Error message if document retrieval failed") error_message: Optional[str] = Field(None, description="Error message if document retrieval failed")
# ============================================================================
# Unified Models
# ============================================================================
class SayistayUnifiedSearchRequest(BaseModel):
"""Unified search request for all Sayıştay decision types."""
decision_type: Literal["genel_kurul", "temyiz_kurulu", "daire"] = Field(..., description="Decision type: genel_kurul, temyiz_kurulu, or daire")
# Common pagination parameters
start: int = Field(0, ge=0, description="Starting record for pagination (0-based)")
length: int = Field(10, ge=1, le=100, description="Number of records per page (1-100)")
# Common search parameters
karar_tarih_baslangic: str = Field("", description="Start date (DD.MM.YYYY format)")
karar_tarih_bitis: str = Field("", description="End date (DD.MM.YYYY format)")
kamu_idaresi_turu: KamuIdaresiTuruEnum = Field("ALL", description="Public administration type filter")
ilam_no: str = Field("", description="Audit report number (İlam No, max 50 chars)")
web_karar_konusu: WebKararKonusuEnum = Field("ALL", description="Decision subject category filter")
# Genel Kurul specific parameters (ignored for other types)
karar_no: str = Field("", description="Decision number (genel_kurul only)")
karar_ek: str = Field("", description="Decision appendix number (genel_kurul only)")
karar_tamami: str = Field("", description="Full text search (genel_kurul only)")
# Temyiz Kurulu specific parameters (ignored for other types)
ilam_dairesi: DaireEnum = Field("ALL", description="Audit chamber selection (temyiz_kurulu only)")
yili: str = Field("", description="Year (YYYY format, temyiz_kurulu only)")
dosya_no: str = Field("", description="File number (temyiz_kurulu only)")
temyiz_tutanak_no: str = Field("", description="Appeals board meeting minutes number (temyiz_kurulu only)")
temyiz_karar: str = Field("", description="Appeals decision text search (temyiz_kurulu only)")
# Daire specific parameters (ignored for other types)
yargilama_dairesi: DaireEnum = Field("ALL", description="Chamber selection (daire only)")
hesap_yili: str = Field("", description="Account year (daire only)")
web_karar_metni: str = Field("", description="Decision text search (daire only)")
class SayistayUnifiedSearchResult(BaseModel):
"""Unified search result containing decisions from any Sayıştay decision type."""
decision_type: Literal["genel_kurul", "temyiz_kurulu", "daire"] = Field(..., description="Type of decisions returned")
decisions: List[Dict[str, Any]] = Field(default_factory=list, description="Decision list (structure varies by type)")
total_records: int = Field(0, description="Total number of records found")
total_filtered: int = Field(0, description="Number of records after filtering")
draw: int = Field(1, description="DataTables draw counter")
class SayistayUnifiedDocumentMarkdown(BaseModel):
"""Unified document model for all Sayıştay decision types."""
decision_type: Literal["genel_kurul", "temyiz_kurulu", "daire"] = Field(..., description="Type of document")
decision_id: str = Field(..., description="Decision ID")
source_url: str = Field(..., description="Source URL of the document")
document_data: Dict[str, Any] = Field(default_factory=dict, description="Document content and metadata")
markdown_content: Optional[str] = Field(None, description="Markdown content")
error_message: Optional[str] = Field(None, description="Error message if retrieval failed")
+133
View File
@@ -0,0 +1,133 @@
# sayistay_mcp_module/unified_client.py
# Unified client for all three Sayıştay decision types
import logging
from typing import Optional, Dict, Any
from urllib.parse import urlparse
from .models import (
SayistayUnifiedSearchRequest,
SayistayUnifiedSearchResult,
SayistayUnifiedDocumentMarkdown,
GenelKurulSearchRequest,
TemyizKuruluSearchRequest,
DaireSearchRequest
)
from .client import SayistayApiClient
logger = logging.getLogger(__name__)
class SayistayUnifiedClient:
"""Unified client that handles all three Sayıştay decision types."""
def __init__(self, request_timeout: float = 60.0):
self.client = SayistayApiClient(request_timeout)
async def search_unified(self, params: SayistayUnifiedSearchRequest) -> SayistayUnifiedSearchResult:
"""Unified search that routes to appropriate search method based on decision_type."""
if params.decision_type == "genel_kurul":
# Convert to genel kurul request
genel_kurul_params = GenelKurulSearchRequest(
karar_no=params.karar_no,
karar_ek=params.karar_ek,
karar_tarih_baslangic=params.karar_tarih_baslangic,
karar_tarih_bitis=params.karar_tarih_bitis,
karar_tamami=params.karar_tamami,
start=params.start,
length=params.length
)
result = await self.client.search_genel_kurul_decisions(genel_kurul_params)
# Convert to unified format
decisions_list = [decision.model_dump() for decision in result.decisions]
return SayistayUnifiedSearchResult(
decision_type="genel_kurul",
decisions=decisions_list,
total_records=result.total_records,
total_filtered=result.total_filtered,
draw=result.draw
)
elif params.decision_type == "temyiz_kurulu":
# Convert to temyiz kurulu request
temyiz_params = TemyizKuruluSearchRequest(
ilam_dairesi=params.ilam_dairesi,
yili=params.yili,
karar_tarih_baslangic=params.karar_tarih_baslangic,
karar_tarih_bitis=params.karar_tarih_bitis,
kamu_idaresi_turu=params.kamu_idaresi_turu,
ilam_no=params.ilam_no,
dosya_no=params.dosya_no,
temyiz_tutanak_no=params.temyiz_tutanak_no,
temyiz_karar=params.temyiz_karar,
web_karar_konusu=params.web_karar_konusu,
start=params.start,
length=params.length
)
result = await self.client.search_temyiz_kurulu_decisions(temyiz_params)
# Convert to unified format
decisions_list = [decision.model_dump() for decision in result.decisions]
return SayistayUnifiedSearchResult(
decision_type="temyiz_kurulu",
decisions=decisions_list,
total_records=result.total_records,
total_filtered=result.total_filtered,
draw=result.draw
)
elif params.decision_type == "daire":
# Convert to daire request
daire_params = DaireSearchRequest(
yargilama_dairesi=params.yargilama_dairesi,
karar_tarih_baslangic=params.karar_tarih_baslangic,
karar_tarih_bitis=params.karar_tarih_bitis,
ilam_no=params.ilam_no,
kamu_idaresi_turu=params.kamu_idaresi_turu,
hesap_yili=params.hesap_yili,
web_karar_konusu=params.web_karar_konusu,
web_karar_metni=params.web_karar_metni,
start=params.start,
length=params.length
)
result = await self.client.search_daire_decisions(daire_params)
# Convert to unified format
decisions_list = [decision.model_dump() for decision in result.decisions]
return SayistayUnifiedSearchResult(
decision_type="daire",
decisions=decisions_list,
total_records=result.total_records,
total_filtered=result.total_filtered,
draw=result.draw
)
else:
raise ValueError(f"Unsupported decision type: {params.decision_type}")
async def get_document_unified(self, decision_id: str, decision_type: str) -> SayistayUnifiedDocumentMarkdown:
"""Unified document retrieval for all Sayıştay decision types."""
# Use existing client method (decision_type is already a string)
result = await self.client.get_document_as_markdown(decision_id, decision_type)
return SayistayUnifiedDocumentMarkdown(
decision_type=decision_type,
decision_id=result.decision_id,
source_url=result.source_url,
document_data=result.model_dump(),
markdown_content=result.markdown_content,
error_message=result.error_message
)
async def close_client_session(self):
"""Close the underlying client session."""
if hasattr(self.client, 'close_client_session'):
await self.client.close_client_session()
+7 -11
View File
@@ -7,8 +7,7 @@ from typing import Dict, Any, List, Optional, Union, Tuple
import logging import logging
import html import html
import re import re
import tempfile import io
import os
from markitdown import MarkItDown from markitdown import MarkItDown
from urllib.parse import urljoin, urlencode # urlencode for aiohttp form data from urllib.parse import urljoin, urlencode # urlencode for aiohttp form data
@@ -196,21 +195,18 @@ class UyusmazlikApiClient:
html_input_for_markdown = processed_html html_input_for_markdown = processed_html
markdown_text = None markdown_text = None
temp_file_path = None
try: try:
md_converter = MarkItDown() # Convert HTML string to bytes and create BytesIO stream
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_file: html_bytes = html_input_for_markdown.encode('utf-8')
tmp_file.write(html_input_for_markdown) html_stream = io.BytesIO(html_bytes)
temp_file_path = tmp_file.name
conversion_result = md_converter.convert(temp_file_path) # Pass BytesIO stream to MarkItDown to avoid temp file creation
md_converter = MarkItDown()
conversion_result = md_converter.convert(html_stream)
markdown_text = conversion_result.text_content markdown_text = conversion_result.text_content
logger.info("UyusmazlikApiClient: HTML to Markdown conversion successful.") logger.info("UyusmazlikApiClient: HTML to Markdown conversion successful.")
except Exception as e: except Exception as e:
logger.error(f"UyusmazlikApiClient: Error during MarkItDown HTML to Markdown conversion: {e}") logger.error(f"UyusmazlikApiClient: Error during MarkItDown HTML to Markdown conversion: {e}")
finally:
if temp_file_path and os.path.exists(temp_file_path):
os.remove(temp_file_path)
return markdown_text return markdown_text
async def get_decision_document_as_markdown(self, document_url: str) -> UyusmazlikDocumentMarkdown: async def get_decision_document_as_markdown(self, document_url: str) -> UyusmazlikDocumentMarkdown:
+20 -20
View File
@@ -28,41 +28,41 @@ class UyusmazlikKararSonucuEnum(str, Enum): # Based on checkbox text in the form
class UyusmazlikSearchRequest(BaseModel): # This is the model the MCP tool will accept class UyusmazlikSearchRequest(BaseModel): # This is the model the MCP tool will accept
"""Model for Uyuşmazlık Mahkemesi search request using user-friendly terms.""" """Model for Uyuşmazlık Mahkemesi search request using user-friendly terms."""
icerik: Optional[str] = Field("", description="Keyword or content for main text search (Icerik).") icerik: Optional[str] = Field("", description="Search text")
bolum: Optional[UyusmazlikBolumEnum] = Field( bolum: Optional[UyusmazlikBolumEnum] = Field(
UyusmazlikBolumEnum.TUMU, UyusmazlikBolumEnum.TUMU,
description="Select the department (Bölüm)." description="Department"
) )
uyusmazlik_turu: Optional[UyusmazlikTuruEnum] = Field( uyusmazlik_turu: Optional[UyusmazlikTuruEnum] = Field(
UyusmazlikTuruEnum.TUMU, UyusmazlikTuruEnum.TUMU,
description="Select the type of dispute (Uyuşmazlık)." description="Dispute type"
) )
# User provides a list of user-friendly names for Karar Sonucu # User provides a list of user-friendly names for Karar Sonucu
karar_sonuclari: Optional[List[UyusmazlikKararSonucuEnum]] = Field( # Changed to list of Enums karar_sonuclari: Optional[List[UyusmazlikKararSonucuEnum]] = Field( # Changed to list of Enums
default_factory=list, default_factory=list,
description="List of desired 'Karar Sonucu' types." description="Decision types"
) )
esas_yil: Optional[str] = Field("", description="Case year ('Esas Yılı').") esas_yil: Optional[str] = Field("", description="Case year")
esas_sayisi: Optional[str] = Field("", description="Case number ('Esas Sayısı').") esas_sayisi: Optional[str] = Field("", description="Case no")
karar_yil: Optional[str] = Field("", description="Decision year ('Karar Yılı').") karar_yil: Optional[str] = Field("", description="Decision year")
karar_sayisi: Optional[str] = Field("", description="Decision number ('Karar Sayısı').") karar_sayisi: Optional[str] = Field("", description="Decision no")
kanun_no: Optional[str] = Field("", description="Relevant Law Number ('KanunNo').") kanun_no: Optional[str] = Field("", description="Law no")
karar_date_begin: Optional[str] = Field("", description="Decision start date (DD.MM.YYYY) ('KararDateBegin').") karar_date_begin: Optional[str] = Field("", description="Start date (DD.MM.YYYY)")
karar_date_end: Optional[str] = Field("", description="Decision end date (DD.MM.YYYY) ('KararDateEnd').") karar_date_end: Optional[str] = Field("", description="End date (DD.MM.YYYY)")
resmi_gazete_sayi: Optional[str] = Field("", description="Official Gazette number ('ResmiGazeteSayi').") resmi_gazete_sayi: Optional[str] = Field("", description="Gazette no")
resmi_gazete_date: Optional[str] = Field("", description="Official Gazette date (DD.MM.YYYY) ('ResmiGazeteDate').") resmi_gazete_date: Optional[str] = Field("", description="Gazette date (DD.MM.YYYY)")
# Detailed text search fields from the "icerikDetail" section of the form # Detailed text search fields from the "icerikDetail" section of the form
tumce: Optional[str] = Field("", description="Exact phrase search ('Tumce').") tumce: Optional[str] = Field("", description="Exact phrase")
wild_card: Optional[str] = Field("", description="Search for phrase and its inflections ('WildCard').") # Changed from WildCard for Pythonic name wild_card: Optional[str] = Field("", description="Wildcard search")
hepsi: Optional[str] = Field("", description="Search for texts containing all specified words ('Hepsi').") hepsi: Optional[str] = Field("", description="All words")
herhangi_birisi: Optional[str] = Field("", description="Search for texts containing any of the specified words ('Herhangibirisi').") herhangi_birisi: Optional[str] = Field("", description="Any word")
not_hepsi: Optional[str] = Field("", description="Exclude texts containing these specified words ('NotHepsi').") not_hepsi: Optional[str] = Field("", description="Exclude words")
class UyusmazlikApiDecisionEntry(BaseModel): class UyusmazlikApiDecisionEntry(BaseModel):
"""Model for an individual decision entry parsed from Uyuşmazlık API's HTML search response.""" """Model for an individual decision entry parsed from Uyuşmazlık API's HTML search response."""
@@ -71,9 +71,9 @@ class UyusmazlikApiDecisionEntry(BaseModel):
bolum: Optional[str] = Field(None) bolum: Optional[str] = Field(None)
uyusmazlik_konusu: Optional[str] = Field(None) uyusmazlik_konusu: Optional[str] = Field(None)
karar_sonucu: Optional[str] = Field(None) karar_sonucu: Optional[str] = Field(None)
popover_content: Optional[str] = Field(None, description="Summary/description from popover.") popover_content: Optional[str] = Field(None, description="Summary")
document_url: HttpUrl # Full URL to the decision document HTML page document_url: HttpUrl # Full URL to the decision document HTML page
pdf_url: Optional[HttpUrl] = Field(None, description="Direct URL to PDF if available.") pdf_url: Optional[HttpUrl] = Field(None, description="PDF URL")
class UyusmazlikSearchResponse(BaseModel): # This is what the MCP tool will return class UyusmazlikSearchResponse(BaseModel): # This is what the MCP tool will return
"""Response model for Uyuşmazlık Mahkemesi search results for the MCP tool.""" """Response model for Uyuşmazlık Mahkemesi search results for the MCP tool."""
+20 -13
View File
@@ -6,8 +6,7 @@ from typing import Dict, Any, List, Optional
import logging import logging
import html import html
import re import re
import tempfile import io
import os
from markitdown import MarkItDown from markitdown import MarkItDown
from .models import ( from .models import (
@@ -66,6 +65,19 @@ class YargitayOfficialApiClient:
response.raise_for_status() # Raise an exception for HTTP 4xx or 5xx status codes response.raise_for_status() # Raise an exception for HTTP 4xx or 5xx status codes
response_json_data = response.json() response_json_data = response.json()
logger.debug(f"YargitayOfficialApiClient: Raw API response: {response_json_data}")
# Handle None or empty data response from API
if response_json_data is None:
logger.warning("YargitayOfficialApiClient: API returned None response")
response_json_data = {"data": {"data": [], "recordsTotal": 0, "recordsFiltered": 0}}
elif not isinstance(response_json_data, dict):
logger.warning(f"YargitayOfficialApiClient: API returned unexpected response type: {type(response_json_data)}")
response_json_data = {"data": {"data": [], "recordsTotal": 0, "recordsFiltered": 0}}
elif response_json_data.get("data") is None:
logger.warning("YargitayOfficialApiClient: API response data field is None")
response_json_data["data"] = {"data": [], "recordsTotal": 0, "recordsFiltered": 0}
# Validate and parse the response using Pydantic models # Validate and parse the response using Pydantic models
api_response = YargitayApiSearchResponse(**response_json_data) api_response = YargitayApiSearchResponse(**response_json_data)
@@ -108,25 +120,20 @@ class YargitayOfficialApiClient:
html_to_convert = processed_html html_to_convert = processed_html
markdown_output = None markdown_output = None
temp_file_path = None
try: try:
md_converter = MarkItDown() # Plugins disabled as per basic usage # Convert HTML string to bytes and create BytesIO stream
html_bytes = html_to_convert.encode('utf-8')
html_stream = io.BytesIO(html_bytes)
# Write the HTML to a temporary file for MarkItDown to process # Pass BytesIO stream to MarkItDown to avoid temp file creation
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".html", encoding="utf-8") as tmp_html_file: md_converter = MarkItDown()
tmp_html_file.write(html_to_convert) conversion_result = md_converter.convert(html_stream)
temp_file_path = tmp_html_file.name
conversion_result = md_converter.convert(temp_file_path)
markdown_output = conversion_result.text_content markdown_output = conversion_result.text_content
logger.info("Successfully converted HTML to Markdown.") logger.info("Successfully converted HTML to Markdown.")
except Exception as e: except Exception as e:
logger.error(f"Error during MarkItDown HTML to Markdown conversion: {e}") logger.error(f"Error during MarkItDown HTML to Markdown conversion: {e}")
finally:
if temp_file_path and os.path.exists(temp_file_path):
os.remove(temp_file_path) # Clean up the temporary file
return markdown_output return markdown_output
+34 -75
View File
@@ -34,111 +34,70 @@ class YargitayDetailedSearchRequest(BaseModel):
to Yargitay's detailed search endpoint (e.g., /aramadetaylist). to Yargitay's detailed search endpoint (e.g., /aramadetaylist).
Based on the payload provided by the user. Based on the payload provided by the user.
""" """
arananKelime: Optional[str] = Field("", description="""Keyword to search for with advanced operators support: arananKelime: Optional[str] = Field("", description="Turkish keywords (supports +word -word \"phrase\" operators)")
Simple words: 'arsa payı' (OR logic - finds documents with ANY word)
Exact phrases: '"arsa payı"' (finds exact phrase)
AND logic: 'arsa+payı' (both words required)
Wildcards: 'bozma*' (matches bozma, bozması, bozmanın, etc.)
Multiple required: '+"arsa payı" +"bozma sebebi"'
Exclusion: '+"arsa payı" -"inşaat sözleşmesi"'
Examples: arsa payı | "arsa payı" | +"mülkiyet hakkı" +"bozma sebebi" | hukuk*""")
# Department/Board selection - Complete Court of Cassation chamber hierarchy # Department/Board selection - Complete Court of Cassation chamber hierarchy
birimYrgKurulDaire: Optional[str] = Field("ALL", description=""" birimYrgKurulDaire: Optional[str] = Field("ALL", description="Chamber (ALL or specific chamber name)")
Court of Cassation (Yargıtay) chamber/board selection. Options include:
- 'ALL' for all chambers
- Civil: 'Civil General Assembly (Hukuk Genel Kurulu)', '1st Civil Chamber (1. Hukuk Dairesi)' through '23rd Civil Chamber (23. Hukuk Dairesi)', 'Civil Chambers Presidents Board (Hukuk Daireleri Başkanlar Kurulu)'
- Criminal: 'Criminal General Assembly (Ceza Genel Kurulu)', '1st Criminal Chamber (1. Ceza Dairesi)' through '23rd Criminal Chamber (23. Ceza Dairesi)', 'Criminal Chambers Presidents Board (Ceza Daireleri Başkanlar Kurulu)'
- General: 'Grand General Assembly (Büyük Genel Kurulu)'
Total: 52 possible values (including 'ALL' for all chambers)
""")
birimYrgHukukDaire: Optional[str] = Field("", description="Legacy field - use birimYrgKurulDaire instead for chamber selection")
birimYrgCezaDaire: Optional[str] = Field("", description="Legacy field - use birimYrgKurulDaire instead for chamber selection")
esasYil: Optional[str] = Field("", description="""Case year for 'Esas No' filtering. esasYil: Optional[str] = Field("", description="Case year (YYYY)")
Format: YYYY (e.g., '2024') esasIlkSiraNo: Optional[str] = Field("", description="Start case no")
Use with sequence numbers for precise case targeting""") esasSonSiraNo: Optional[str] = Field("", description="End case no")
esasIlkSiraNo: Optional[str] = Field("", description="""Starting sequence number for 'Esas No' range filtering.
Format: numeric string (e.g., '1', '100')
Use with esasSonSiraNo for range: cases 100-200 in specified year""")
esasSonSiraNo: Optional[str] = Field("", description="""Ending sequence number for 'Esas No' range filtering.
Format: numeric string (e.g., '500', '1000')
Creates range from esasIlkSiraNo to this number""")
kararYil: Optional[str] = Field("", description="""Decision year for 'Karar No' filtering. kararYil: Optional[str] = Field("", description="Decision year (YYYY)")
Format: YYYY (e.g., '2024') kararIlkSiraNo: Optional[str] = Field("", description="Start decision no")
Filters decisions by the year they were issued""") kararSonSiraNo: Optional[str] = Field("", description="End decision no")
kararIlkSiraNo: Optional[str] = Field("", description="""Starting sequence number for 'Karar No' range filtering.
Format: numeric string (e.g., '1', '50')
Use with kararSonSiraNo for decision number ranges""")
kararSonSiraNo: Optional[str] = Field("", description="""Ending sequence number for 'Karar No' range filtering.
Format: numeric string (e.g., '100', '500')
Creates range from kararIlkSiraNo to this number""")
baslangicTarihi: Optional[str] = Field("", description="""Start date for decision search. baslangicTarihi: Optional[str] = Field("", description="Start date (DD.MM.YYYY)")
Format: DD.MM.YYYY (e.g., '01.01.2024') bitisTarihi: Optional[str] = Field("", description="End date (DD.MM.YYYY)")
Use with bitisTarihi for date range filtering
Examples: '01.01.2024', '15.06.2023'""")
bitisTarihi: Optional[str] = Field("", description="""End date for decision search.
Format: DD.MM.YYYY (e.g., '31.12.2024')
Creates date range from baslangicTarihi to this date
Examples: '31.12.2024', '30.06.2023'""")
siralama: Optional[str] = Field("3", description="""Sorting criteria for search results:
'1': Esas No (Case Number) - sorts by case registration order
'2': Karar No (Decision Number) - sorts by decision issuance order
'3': Karar Tarihi (Decision Date) - sorts by chronological order [DEFAULT]
Recommended: Use '3' for most recent decisions first""")
siralamaDirection: Optional[str] = Field("desc", description="""Sorting direction for results:
'desc': Descending order (newest/highest first) [DEFAULT]
'asc': Ascending order (oldest/lowest first)
Most common: 'desc' for latest decisions first""")
pageSize: int = Field(10, ge=1, le=100, description="""Number of results per page. pageSize: int = Field(10, ge=1, le=10, description="Results per page (1-100)")
Range: 1-100 results per page pageNumber: int = Field(1, ge=1, description="Page number (1-indexed)")
Recommended: 10-50 for balanced performance and coverage
Large values (50-100) for comprehensive analysis""")
pageNumber: int = Field(1, ge=1, description="""Page number to retrieve (1-indexed).
Start with 1 for first page
Use with pageSize to navigate through large result sets
Example: pageSize=50, pageNumber=3 gets results 101-150""")
class YargitayApiDecisionEntry(BaseModel): class YargitayApiDecisionEntry(BaseModel):
"""Model for an individual decision entry from the Yargitay API search response.""" """Model for an individual decision entry from the Yargitay API search response."""
id: str # Unique system ID of the decision id: str # Unique system ID of the decision
daire: Optional[str] = Field(None, description="The chamber (Daire) that made the decision.") daire: Optional[str] = Field(None, description="Chamber")
esasNo: Optional[str] = Field(None, alias="esasNo", description="Case registry number (Esas No).") esasNo: Optional[str] = Field(None, alias="esasNo", description="Case no")
kararNo: Optional[str] = Field(None, alias="kararNo", description="Decision number (Karar No).") kararNo: Optional[str] = Field(None, alias="kararNo", description="Decision no")
kararTarihi: Optional[str] = Field(None, alias="kararTarihi", description="Date of the decision (Karar Tarihi).") kararTarihi: Optional[str] = Field(None, alias="kararTarihi", description="Date")
arananKelime: Optional[str] = Field(None, alias="arananKelime", description="Matched keyword (Aranan Kelime) in the search result item.")
# 'index' and 'siraNo' from API response are not critical for MCP tool, so omitted for brevity # 'index' and 'siraNo' from API response are not critical for MCP tool, so omitted for brevity
# This field will be populated by the client after fetching the search list # This field will be populated by the client after fetching the search list
document_url: Optional[HttpUrl] = Field(None, description="Direct URL (Belge URL) to the decision document.") document_url: Optional[HttpUrl] = Field(None, description="Document URL")
model_config = ConfigDict(populate_by_name=True) # To allow populating by alias from API response model_config = ConfigDict(populate_by_name=True) # To allow populating by alias from API response
class YargitayApiResponseInnerData(BaseModel): class YargitayApiResponseInnerData(BaseModel):
"""Model for the inner 'data' object in the Yargitay API search response.""" """Model for the inner 'data' object in the Yargitay API search response."""
data: List[YargitayApiDecisionEntry] data: List[YargitayApiDecisionEntry] = Field(default_factory=list)
# draw: Optional[int] = None # Typically used by DataTables, not essential for MCP # draw: Optional[int] = None # Typically used by DataTables, not essential for MCP
recordsTotal: int # Total number of records matching the query recordsTotal: int = Field(default=0) # Total number of records matching the query
recordsFiltered: int # Total number of records after filtering (usually same as recordsTotal) recordsFiltered: int = Field(default=0) # Total number of records after filtering (usually same as recordsTotal)
class YargitayApiSearchResponse(BaseModel): class YargitayApiSearchResponse(BaseModel):
"""Model for the complete search response from the Yargitay API.""" """Model for the complete search response from the Yargitay API."""
data: YargitayApiResponseInnerData data: Optional[YargitayApiResponseInnerData] = Field(default_factory=lambda: YargitayApiResponseInnerData())
# metadata: Optional[Dict[str, Any]] = None # Optional metadata from API # metadata: Optional[Dict[str, Any]] = None # Optional metadata from API
class YargitayDocumentMarkdown(BaseModel): class YargitayDocumentMarkdown(BaseModel):
"""Model for a Yargitay decision document, containing only Markdown content.""" """Model for a Yargitay decision document, containing only Markdown content."""
id: str = Field(..., description="The unique ID (Belge Kimliği) of the document.") id: str = Field(..., description="Document ID")
markdown_content: Optional[str] = Field(None, description="The decision content (Karar İçeriği) converted to Markdown.") markdown_content: Optional[str] = Field(None, description="Content")
source_url: HttpUrl = Field(..., description="The source URL (Kaynak URL) of the original document.") source_url: HttpUrl = Field(..., description="Source URL")
class CleanYargitayDecisionEntry(BaseModel):
"""Clean decision entry without arananKelime field to reduce token usage."""
id: str
daire: Optional[str] = Field(None, description="Chamber")
esasNo: Optional[str] = Field(None, description="Case no")
kararNo: Optional[str] = Field(None, description="Decision no")
kararTarihi: Optional[str] = Field(None, description="Date")
document_url: Optional[HttpUrl] = Field(None, description="Document URL")
class CompactYargitaySearchResult(BaseModel): class CompactYargitaySearchResult(BaseModel):
"""A more compact search result model for the MCP tool to return.""" """A more compact search result model for the MCP tool to return."""
decisions: List[YargitayApiDecisionEntry] decisions: List[CleanYargitayDecisionEntry]
total_records: int total_records: int
requested_page: int requested_page: int
page_size: int page_size: int