tags containing "Esas No.:" and "Karar No.:" - - Karar Tarihi: Find bold text containing "Karar tarihi:" or regex "Karar Tarihi\s*:\s*([\d\.]+)" - - Resmi Gazete: Find text containing "Resmî Gazete tarih ve sayısı:" or "Resmi Gazete tarih/sayı:" - -**Pagination & Chunking:** -- Split markdown into 5,000 character chunks -- Calculate: total_pages = ceil(len(markdown) / 5000) -- Return current_page_clamped (max 1, min total_pages) -- Include pagination metadata: current_page, total_pages, is_paginated flag - -### 5. Data Models (models.py - 230 lines) - -**Norm Denetimi Models:** -- `AnayasaNormDenetimiSearchRequest`: 19 search parameters -- `AnayasaReviewedNormInfo`: norm_name_or_number, article_number, review_type_and_outcome, outcome_reason, basis_constitution_articles_cited[], postponement_period -- `AnayasaDecisionSummary`: decision_reference_no, decision_page_url, keywords_found_count, application_type_summary, applicant_summary, decision_outcome_summary, decision_date_summary, reviewed_norms[] -- `AnayasaSearchResult`: decisions[], total_records_found, retrieved_page_number -- `AnayasaDocumentMarkdown`: source_url, decision_reference_no_from_page, decision_date_from_page, official_gazette_info_from_page, markdown_chunk, current_page, total_pages, is_paginated - -**Bireysel Başvuru Models:** -- `AnayasaBireyselReportSearchRequest`: keywords[], page_to_fetch -- `AnayasaBireyselReportDecisionDetail`: hak, mudahale_iddiası, sonuç, giderim (4 fields per right examined) -- `AnayasaBireyselReportDecisionSummary`: title, decision_reference_no, decision_page_url, decision_type_summary, decision_making_body, application_date_summary, decision_date_summary, application_subject_summary, details[] -- `AnayasaBireyselReportSearchResult`: decisions[], total_records_found, retrieved_page_number -- `AnayasaBireyselBasvuruDocumentMarkdown`: source_url, basvuru_no_from_page, karar_tarihi_from_page, basvuru_tarihi_from_page, karari_veren_birim_from_page, karar_turu_from_page, resmi_gazete_info_from_page, markdown_chunk, current_page, total_pages, is_paginated - -**Unified Models:** -- `AnayasaUnifiedSearchRequest`: decision_type (norm_denetimi|bireysel_basvuru), keywords[], page_to_fetch, results_per_page, + type-specific parameters -- `AnayasaUnifiedSearchResult`: decision_type, decisions[] (Dict[str, Any]), total_records_found, retrieved_page_number -- `AnayasaUnifiedDocumentMarkdown`: decision_type, source_url, document_data (Dict), markdown_chunk, current_page, total_pages, is_paginated - -### 6. Unified Client Routing (unified_client.py - 122 lines) - -**AnayasaUnifiedClient class:** -- Maintains instances of both norm_client and bireysel_client -- `search_unified()`: Routes based on decision_type parameter - - norm_denetimi: Converts to AnayasaNormDenetimiSearchRequest, calls norm_client.search_norm_denetimi_decisions() - - bireysel_basvuru: Converts to AnayasaBireyselReportSearchRequest, calls bireysel_client.search_bireysel_basvuru_report() - - Returns unified AnayasaUnifiedSearchResult -- `get_document_unified()`: Auto-detects decision type from URL - - Checks for "normkararlarbilgibankasi" in netloc or "/ND/" in path → norm_denetimi - - Checks for "kararlarbilgibankasi" in netloc or "/BB/" in path → bireysel_basvuru - - Calls appropriate client, wraps result in unified model - -### 7. MCP Tool Integration (mcp_server_main.py) - -**Active Tools (2 tools - Phase 6 optimization):** - -```python -@app.tool( - description="Search Constitutional Court decisions from either Norm Control or Individual Applications", - annotations={"readOnlyHint": True, "openWorldHint": True, "idempotentHint": True} -) -async def search_anayasa_unified( - decision_type: Literal["norm_denetimi", "bireysel_basvuru"], - keywords: List[str], - page_to_fetch: int (1-100), - # Norm Denetimi specific (ignored for bireysel_basvuru) - keywords_all: List[str], - keywords_any: List[str], - decision_type_norm: Literal["ALL", "1", "2", "3"], - application_date_start: str, - application_date_end: str, - # Bireysel Başvuru specific (ignored for norm_denetimi) - decision_start_date: str, - decision_end_date: str, - norm_type: Literal["ALL", "1", "2", ...], - subject_category: str -) -> str (JSON) -``` - -```python -@app.tool( - description="Retrieve full text of Constitutional Court decision. Auto-detects decision type from URL", - annotations={"readOnlyHint": True, "openWorldHint": False, "idempotentHint": True} -) -async def get_anayasa_document_unified( - document_url: str, - page_number: int (1-indexed) -) -> str (JSON) -``` - -**Deactivated Tools (4 tools - Phase 6 optimization, marked with DEACTIVATED):** -- search_anayasa_norm_denetimi_decisions -- get_anayasa_norm_denetimi_document_markdown -- search_anayasa_bireysel_basvuru_report -- get_anayasa_bireysel_basvuru_document_markdown - -## Search Capabilities Analysis - -### Current Keyword-Based Search Strengths - -**Norm Denetimi - Rich Structural Filtering:** -1. Multi-keyword logic with AND/OR/NOT operators -2. Case/decision number search (exact matching) -3. Date range filtering (review, decision, gazette dates) -4. Norm categorization (14 norm types) -5. Application type filtering (3 categories) -6. Constitutional period selection (1961 vs 1982 constitutions) -7. Decision outcome filtering (8 outcome types) -8. Reasoning/grounds filtering (30 different grounds) -9. Member/rapporteur filtering -10. Constitutional articles cited filtering - -**Bireysel Başvuru - Report Format:** -1. Simple keyword search -2. Rights/claims detailed in structured table format -3. Remedy/solution tracking - -### Limitations of Current Keyword Search - -1. **No semantic understanding**: Different words for same concept ("mülkiyet hakkı" vs "property rights") -2. **No concept hierarchy**: Can't find related legal principles -3. **No cross-language**: Turkish-only, no English queries -4. **No abbreviation matching**: "HADD" vs "Hukuk Alanında Değerli Dosya Denetimi" -5. **No synonym support**: Formal vs informal terminology -6. **No semantic similarity**: Can't find similar cases with different terminology -7. **No legal concept graph**: Can't traverse related principles or doctrines -8. **No fuzzy matching**: Typos or spelling variations fail completely -9. **No legal reasoning search**: Can't query by legal arguments or doctrinal approaches -10. **No cross-system semantic linking**: Norm Denetimi and Bireysel Başvuru not semantically linked -11. **Order dependency**: Query order may affect results -12. **No ranking by relevance**: Just keyword presence/absence -13. **No query expansion**: No automatic synonym/related term expansion - -### HTML Document Structure - -**Norm Denetimi Search Results HTML:** -``` -div.birkarar (repeated for each decision) -├── div.bkararbaslik (header with E./K. numbers) -│ └── div.BulunanKelimeSayisi (keyword count) -└── div.kararbilgileri (metadata with | separators: application_type|applicant|outcome|date) - -Next sibling: -div.col-sm-12 -└── table.table > tbody > tr (one row per reviewed norm with 6 columns) - ├── td: norm name/number - ├── td: article number - ├── td: review type and outcome - ├── td: outcome reason - ├── td: constitutional articles cited (comma-separated) - └── td: postponement period -``` - -**Full Decision Content (both types):** -``` -div#Karar (decision tab) -└── div.KararMetni or div.WordSection1 - └── HTML content in MS Word format (many nested divs with styles) - -Metadata extracted from: -
Esas No.: [number]
-Karar No.: [number]
-Karar tarihi: [date]
-Resmî Gazete tarih ve sayısı: [info]
-``` - -### Document Content Characteristics - -- **Language**: Turkish legal language (specialized terminology) -- **Format**: Microsoft Word-generated HTML (nested divs, complex styles) -- **Content types**: - - Norm Denetimi: Constitutional principle analysis, legal reasoning, comparison with challenged norm - - Bireysel Başvuru: Right violated, remedy granted, procedural requirements -- **Typical length**: 5,000-50,000+ characters -- **Citations**: Internal cross-references to constitutional articles -- **Structure**: Formal legal document with sections, subsections, reasoning - -## Key Technical Insights for Semantic Search - -### Content Encoding -- Currently: HTML → BeautifulSoup parsing → MarkItDown → Markdown -- Extraction: Specific div/class/id selectors -- Metadata: Regex patterns and text parsing - -### Search Query Flow -1. User provides keywords/filters -2. Convert Pydantic model to URL query parameters -3. HTTP GET request to Constitutional Court API -4. HTML response parsed with BeautifulSoup -5. Decision summaries extracted and validated -6. Results returned as JSON - -### Document Retrieval Flow -1. Get document URL from search results -2. HTTP GET request to URL -3. Parse HTML for metadata extraction -4. MarkItDown converts HTML to Markdown -5. Chunk by 5,000 characters -6. Return paginated Markdown with metadata - -## Performance Baseline - -- **Search**: ~1-5 seconds (HTML parsing + regex extraction) -- **Document**: ~2-10 seconds (fetch + parse + MarkItDown + chunking) -- **Memory**: Minimal (5,000 char chunks, no full document in memory) -- **API Response Size**: Typically 50-500 KB HTML for search, 100-1000 KB for full decision - -## Next Steps for Semantic Search Integration - -1. **Vector Embeddings**: Embed decisions using Turkish legal model -2. **Concept Extraction**: Identify and tag legal concepts (rights, procedures, principles) -3. **Semantic Queries**: Convert natural language questions to embeddings -4. **Hybrid Search**: Combine keyword + semantic similarity -5. **Legal Ontology**: Map Turkish Constitutional Court concepts and relationships -6. **Cross-system Linking**: Semantically link Norm Denetimi and Bireysel Başvuru decisions -7. **Precedent Graph**: Extract citations and create legal precedent relationships -8. **Fine-tuned Embeddings**: Train embeddings specifically on Turkish Constitutional law -9. **Ranking**: Re-rank results by semantic relevance to user's legal intent -10. **Explanation**: Provide semantic reasoning for why result is relevant diff --git a/.serena/project.yml b/.serena/project.yml deleted file mode 100644 index f340017..0000000 --- a/.serena/project.yml +++ /dev/null @@ -1,139 +0,0 @@ -# list of languages for which language servers are started; choose from: -# al bash clojure cpp csharp csharp_omnisharp -# dart elixir elm erlang fortran go -# haskell java julia kotlin lua markdown -# nix perl php python python_jedi r -# rego ruby ruby_solargraph rust scala swift -# terraform typescript typescript_vts yaml zig -# Note: -# - For C, use cpp -# - For JavaScript, use typescript -# Special requirements: -# - csharp: Requires the presence of a .sln file in the project folder. -# When using multiple languages, the first language server that supports a given file will be used for that file. -# The first language is the default language and the respective language server will be used as a fallback. -# Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored. -languages: -- python - -# the encoding used by text files in the project -# For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings -encoding: "utf-8" - -# whether to use the project's gitignore file to ignore files -# Added on 2025-04-07 -ignore_all_files_in_gitignore: true - -# list of additional paths to ignore -# same syntax as gitignore, so you can use * and ** -# Was previously called `ignored_dirs`, please update your config if you are using that. -# Added (renamed) on 2025-04-07 -ignored_paths: [] - -# whether the project is in read-only mode -# If set to true, all editing tools will be disabled and attempts to use them will result in an error -# Added on 2025-04-18 -read_only: false - -# list of tool names to exclude. We recommend not excluding any tools, see the readme for more details. -# Below is the complete list of tools for convenience. -# To make sure you have the latest list of tools, and to view their descriptions, -# execute `uv run scripts/print_tool_overview.py`. -# -# * `activate_project`: Activates a project by name. -# * `check_onboarding_performed`: Checks whether project onboarding was already performed. -# * `create_text_file`: Creates/overwrites a file in the project directory. -# * `delete_lines`: Deletes a range of lines within a file. -# * `delete_memory`: Deletes a memory from Serena's project-specific memory store. -# * `execute_shell_command`: Executes a shell command. -# * `find_referencing_code_snippets`: Finds code snippets in which the symbol at the given location is referenced. -# * `find_referencing_symbols`: Finds symbols that reference the symbol at the given location (optionally filtered by type). -# * `find_symbol`: Performs a global (or local) search for symbols with/containing a given name/substring (optionally filtered by type). -# * `get_current_config`: Prints the current configuration of the agent, including the active and available projects, tools, contexts, and modes. -# * `get_symbols_overview`: Gets an overview of the top-level symbols defined in a given file. -# * `initial_instructions`: Gets the initial instructions for the current project. -# Should only be used in settings where the system prompt cannot be set, -# e.g. in clients you have no control over, like Claude Desktop. -# * `insert_after_symbol`: Inserts content after the end of the definition of a given symbol. -# * `insert_at_line`: Inserts content at a given line in a file. -# * `insert_before_symbol`: Inserts content before the beginning of the definition of a given symbol. -# * `list_dir`: Lists files and directories in the given directory (optionally with recursion). -# * `list_memories`: Lists memories in Serena's project-specific memory store. -# * `onboarding`: Performs onboarding (identifying the project structure and essential tasks, e.g. for testing or building). -# * `prepare_for_new_conversation`: Provides instructions for preparing for a new conversation (in order to continue with the necessary context). -# * `read_file`: Reads a file within the project directory. -# * `read_memory`: Reads the memory with the given name from Serena's project-specific memory store. -# * `remove_project`: Removes a project from the Serena configuration. -# * `replace_lines`: Replaces a range of lines within a file with new content. -# * `replace_symbol_body`: Replaces the full definition of a symbol. -# * `restart_language_server`: Restarts the language server, may be necessary when edits not through Serena happen. -# * `search_for_pattern`: Performs a search for a pattern in the project. -# * `summarize_changes`: Provides instructions for summarizing the changes made to the codebase. -# * `switch_modes`: Activates modes by providing a list of their names -# * `think_about_collected_information`: Thinking tool for pondering the completeness of collected information. -# * `think_about_task_adherence`: Thinking tool for determining whether the agent is still on track with the current task. -# * `think_about_whether_you_are_done`: Thinking tool for determining whether the task is truly completed. -# * `write_memory`: Writes a named memory (for future reference) to Serena's project-specific memory store. -excluded_tools: [] - -# initial prompt for the project. It will always be given to the LLM upon activating the project -# (contrary to the memories, which are loaded on demand). -initial_prompt: "" -# the name by which the project can be referenced within Serena -project_name: "yargi-mcp" - -# list of tools to include that would otherwise be disabled (particularly optional tools that are disabled by default) -included_optional_tools: [] - -# list of mode names to that are always to be included in the set of active modes -# The full set of modes to be activated is base_modes + default_modes. -# If the setting is undefined, the base_modes from the global configuration (serena_config.yml) apply. -# Otherwise, this setting overrides the global configuration. -# Set this to [] to disable base modes for this project. -# Set this to a list of mode names to always include the respective modes for this project. -base_modes: - -# list of mode names that are to be activated by default. -# The full set of modes to be activated is base_modes + default_modes. -# If the setting is undefined, the default_modes from the global configuration (serena_config.yml) apply. -# Otherwise, this overrides the setting from the global configuration (serena_config.yml). -# This setting can, in turn, be overridden by CLI parameters (--mode). -default_modes: - -# fixed set of tools to use as the base tool set (if non-empty), replacing Serena's default set of tools. -# This cannot be combined with non-empty excluded_tools or included_optional_tools. -fixed_tools: [] - -# override of the corresponding setting in serena_config.yml, see the documentation there. -# If null or missing, the value from the global config is used. -symbol_info_budget: - -# The language backend to use for this project. -# If not set, the global setting from serena_config.yml is used. -# Valid values: LSP, JetBrains -# Note: the backend is fixed at startup. If a project with a different backend -# is activated post-init, an error will be returned. -language_backend: - -# list of regex patterns which, when matched, mark a memory entry as read‑only. -# Extends the list from the global configuration, merging the two lists. -read_only_memory_patterns: [] - -# line ending convention to use when writing source files. -# Possible values: unset (use global setting), "lf", "crlf", or "native" (platform default) -# This does not affect Serena's own files (e.g. memories and configuration files), which always use native line endings. -line_ending: - -# list of regex patterns for memories to completely ignore. -# Matching memories will not appear in list_memories or activate_project output -# and cannot be accessed via read_memory or write_memory. -# To access ignored memory files, use the read_file tool on the raw file path. -# Extends the list from the global configuration, merging the two lists. -# Example: ["_archive/.*", "_episodes/.*"] -ignored_memory_patterns: [] - -# advanced configuration option allowing to configure language server-specific options. -# Maps the language key to the options. -# Have a look at the docstring of the constructors of the LS implementations within solidlsp (e.g., for C# or PHP) to see which options are available. -# No documentation on options means no options are available. -ls_specific_settings: {}