Commit Graph

3 Commits

Author SHA1 Message Date
Dotty Dotter
87874a7a14 Activate LSA: Wahlprogramme + ingest + frontend (#2)
Brings Sachsen-Anhalt online as the second supported Bundesland after
NRW. Closes the gap that issue #2 left open: with the PortalaAdapter
already in place from c7242f8, this commit adds the reference data and
flips the activation switch.

Wahlprogramme (LTW Sachsen-Anhalt 06.06.2021)
- Six PDFs added under app/static/referenzen/{cdu,spd,gruene,fdp,afd,
  linke}-lsa-2021.pdf, plus paged plain-text extractions under
  app/kontext/*.txt for the keyword fallback search.
- Sources verified by hand:
  - CDU "Unsere Heimat. Unsere Verantwortung." (cdulsa.de, 82 pages)
  - SPD "Zusammenhalt und neue Chancen" (FES library, 77 pages)
  - GRÜNE "Verlässlich für Sachsen-Anhalt" (gruene-lsa.de, 164 pages)
  - FDP "Wahlprogramm zur Landtagswahl 2021" (Naumann-Stiftung, 76 pages)
  - AfD "Alles für unsere Heimat!" (klimawahlen.de mirror, 64 pages)
  - LINKE "Wahlprogramm zur Landtagswahl 2021" (dielinke-sachsen-anhalt.de,
    88 pages)
- The CDU PDF was the trickiest: KAS blocks bot downloads via
  Cloudflare; the cdulsa.de copy was located by an autonomous web
  search and verified to be byte-identical with the official document.

Embeddings indexed (in production container, OpenAI-compatible
DashScope embeddings via the existing index_programm pipeline):
- CDU 134, SPD 145, GRÜNE 183, FDP 100, AfD 64, LINKE 143 chunks
- Total LSA: 769 new chunks alongside the existing 775 NRW chunks
  and 335 federal Grundsatzprogramm chunks.

wahlprogramme.py
- WAHLPROGRAMME["LSA"] populated with all six parties (canonical fraction
  codes, original titles, page counts).

embeddings.py
- PROGRAMME extended with the six new "<partei>-lsa-2021" entries that
  the indexer pipeline expects.

bundeslaender.py
- LSA flipped to aktiv=True. The frontend dropdown will now offer
  Sachsen-Anhalt as a selectable bundesland and analyzer.get_bundesland_
  context() will produce a real LSA prompt block (CDU/SPD/FDP as
  governing fractions, all six landtagsfraktionen).

End-to-end smoke test (live in production container before commit)
- Adapter: PortalaAdapter.search() returned current Anträge of März 2026
  (LINKE + GRÜNE) with correct titles and PDF URLs.
- Semantic search for an LSA "ÖPNV in der Altmark" sample antrag
  matched LINKE S.53, SPD S.68, FDP S.52 — all three with similarity
  > 0.6 and topical hits (Regionalisierungsmittel, ÖPNV-Förderprogramm,
  Wasserstoffnetz).

Resolves issue #2.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 22:12:32 +02:00
Dotty Dotter
ee0218b5af Refactor wahlprogramme/embeddings/analyzer for multi-state (#5)
Atomic refactor of the three modules that previously hardcoded NRW
behaviour. After this commit, every analysis path consults the central
BUNDESLAENDER registry for governing fractions, parliament name, and
state metadata.

wahlprogramme.py
- WAHLPROGRAMME is now nested {bundesland: {partei: meta}}; NRW data
  hoisted unchanged under the "NRW" key.
- New WAHLPROGRAMM_KONTEXT_FILES dict maps a state to its overview
  markdown file (currently only NRW).
- find_relevant_quotes(text, fraktionen, bundesland) — bundesland is
  now a required positional. Governing fractions for the requested
  state are merged with the submitting fractions before lookup.
- Helpers get_wahlprogramm() and parteien_mit_wahlprogramm() expose
  the new shape to other modules.
- ValueError on unknown bundesland (no silent fallback).

embeddings.py
- Schema migration in init_embeddings_db: adds a `bundesland` column
  to the chunks table when missing, plus an index, and backfills
  existing rows from the PROGRAMME registry. Grundsatzprogramme
  (federal level) keep bundesland NULL by design.
- find_relevant_chunks accepts a bundesland filter that matches state
  rows OR NULL — so federal Grundsatzprogramme remain visible to every
  analysis.
- get_relevant_quotes_for_antrag(text, fraktionen, bundesland, …) —
  bundesland required, governing fractions read from BUNDESLAENDER
  instead of hardcoded ["CDU","GRÜNE"]. Order-preserving dedup
  replaces the previous set-based merge.
- index_programm now writes the bundesland column on insert.
- Dropped the hardcoded "Wahlprogramm NRW 2022" label in
  format_quotes_for_prompt — bundesland context is implicit in the
  surrounding prompt block.

analyzer.py
- get_bundesland_context reads parlament_name, regierungsfraktionen,
  landtagsfraktionen and the optional WAHLPROGRAMM_KONTEXT_FILES entry
  from the central registry. Throws ValueError on unknown OR inactive
  bundesland — kills the silent NRW fallback that previously masked
  configuration gaps.
- The Antragsteller-detection heuristic now iterates
  BUNDESLAENDER[bundesland].landtagsfraktionen instead of
  WAHLPROGRAMME.keys(), so we recognise parties for which we don't
  yet have a Wahlprogramm PDF.
- Both quote lookups (semantic + keyword fallback) now receive the
  bundesland.

Resolves issue #5. Foundation for #2 (LSA), #3 (Berlin), #4 (MV).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 18:48:11 +02:00
Dotty Dotter
63de3ca20d Initial commit: GWÖ-Antragsprüfer v1.0
Features:
- GWÖ-Matrix 2.0 Analyse für NRW-Landtagsanträge
- Verbesserungsvorschläge im Redline-Format (Original/Vorschlag/Begründung)
- Wahlprogramm- und Parteiprogrammtreue-Bewertung
- Landtag-Suche via OPAL-API
- Tag-Wolke mit Multi-Select Filter
- Partei-Filter mit Durchschnittswerten
- PDF-Report-Generierung
- Security Headers (CSP, X-Frame-Options, etc.)
- Persistente SQLite-DB via Docker Volumes

Tech Stack:
- FastAPI + Jinja2
- Qwen LLM via DashScope API
- SQLite + aiosqlite
- WeasyPrint für PDF
- Docker Compose mit Traefik
2026-03-28 22:30:24 +01:00