gwoe-antragspruefer/tests/integration/conftest.py
Dotty Dotter 73a7f76472 Add E2E functional acceptance test suite (#50, #51, #52, #53, #54)
Vier Sub-Issues unter Umbrella #50 — opt-in via 'pytest -m integration',
Default-Suite (77 Unit-Tests) bleibt unberührt.

- Sub-Issue A (#51): test_adapters_live.py — pro aktivem BL Reachability,
  Drucksache-ID-Format, Type-Filter, Datum-/Fraktion-Plausibilität,
  PDF-Link-HEAD-Probe (slow). NI als xfail (Login-Wall).
- Sub-Issue B (#52): test_frontend_xref.py + ground_truth.py — pro BL
  ein manuell kuratiertes Frontend-Sample (Drucksache + Title-Substring +
  Fraktionen + Datum + PDF-URL), gegen das adapter.get_document() gespiegelt
  wird. Fängt Bug-Klasse 14 (Cross-Bundesland-Match).
- Sub-Issue C (#53): test_wahlprogramme_indexed.py — Indexing-Status pro
  aktivem BL aus embeddings.db, PDF-Inhalts-Plausibilität (14 Marker +
  Wahlperioden-Horizont), expliziter Anti-Marker für Bug-Klasse 8
  (CDU-BE 2021 vs 2026 PDF-Tausch durch abgeordnetenwatch).
- Sub-Issue D (#54): test_citations_substring.py — Property-Verification:
  jedes vom LLM zitierte Snippet muss als (whitespace-normalisierter)
  Substring auf der angegebenen PDF-Seite vorhanden sein. Strict-Match
  mit Truncation-Marker-Toleranz, kein Fuzzy. Liest reale Assessments
  aus gwoe-antraege.db. Fängt Bug-Klassen 7/10/17 (Halluzination).

Architektur: separates tests/integration/ Verzeichnis mit eigenem
conftest.py, das die Stubs der Unit-Suite (fitz/bs4/openai/pydantic_settings)
gezielt entfernt und auf echte Module umstellt — mit Fallback-Skip via
pytest.require_module wenn lokale Dev-Maschine die Prod-Deps nicht hat.

206 neue Integration-Tests, 13 Helper-Unit-Tests. 77 Unit-Tests bleiben grün.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 10:00:20 +02:00

102 lines
4.1 KiB
Python

"""Conftest for the integration test layer.
The Unit-Suite in ``tests/conftest.py`` aggressively stubs ``fitz``,
``bs4``, ``openai`` and ``pydantic_settings`` so the 77 fast tests can
run without the full prod-requirements installed. That's the right
trade-off for unit tests but blocks every E2E case in this directory:
- ``fitz`` (PyMuPDF) is needed to read Wahlprogramm-PDF pages for
citation verification (Sub-Issue D) and content plausibility (Sub-C)
- ``bs4`` (BeautifulSoup) is needed by the live NRWAdapter for HTML
parsing of OPAL responses (Sub-Issue A)
- ``openai`` is needed by ``embeddings.create_embedding`` if a test
ever wants to compute a query vector against the live DashScope API
- ``pydantic_settings`` provides the real ``Settings`` class with
paths to the prod-DB and the embeddings-DB
This conftest does NOT replace those modules. It only sets up:
- The ``app`` package import path so ``from app.parlamente import ...``
works when pytest is invoked from the webapp/ root
- A skip-on-import-error guard for tests that need a particular
optional dep but don't want to crash the whole collection if it
isn't installed locally
A test that runs in this directory must therefore have a real
``pip install -r requirements.txt -r requirements-dev.txt`` setup. The
``@pytest.mark.integration`` marker on every test in this directory
ensures the default ``pytest`` invocation skips them.
"""
import sys
from pathlib import Path
import pytest
# Make the `app` package importable when pytest is run from the webapp/ root.
ROOT = Path(__file__).resolve().parent.parent.parent
sys.path.insert(0, str(ROOT))
# The parent ``tests/conftest.py`` aggressively stubs ``fitz``, ``bs4``,
# ``openai`` and ``pydantic_settings`` in sys.modules so the unit suite
# can run without prod-requirements. Pytest loads parent conftests
# *first*, so by the time control reaches this file the stubs are
# already in place.
#
# For integration tests we want to use the *real* modules where they're
# installed. Strategy: per stubbed module, try to import the real one
# (after temporarily removing the stub from sys.modules). If the real
# module is available, keep it; if not, restore the stub so collection
# doesn't crash on import — individual tests that need the real module
# will skip via ``pytest.require_module(...)``.
_OPTIONAL_REAL_MODULES = ("fitz", "bs4", "openai", "pydantic_settings")
import importlib
_STUB_MODULES: dict[str, object] = {}
for _name in _OPTIONAL_REAL_MODULES:
_stub = sys.modules.pop(_name, None)
try:
importlib.import_module(_name)
# Real module found and now lives in sys.modules — drop the stub.
except ImportError:
# No real module available; restore the stub so unrelated
# imports of e.g. ``app.embeddings`` (which does ``from openai
# import OpenAI`` at module level) don't crash collection.
if _stub is not None:
sys.modules[_name] = _stub
_STUB_MODULES[_name] = _stub
del _name, _stub, importlib
def _require(module_name: str) -> None:
"""Skip the calling test if an optional dependency isn't installed
or is currently still represented by the parent-conftest stub.
Use as ``pytest.require_module("fitz")`` at the top of a test that
needs PyMuPDF.
"""
if module_name in _STUB_MODULES:
pytest.skip(
f"integration test skipped: real {module_name!r} not installed "
"in this environment (parent conftest stub still active)"
)
try:
__import__(module_name)
except ImportError as e:
pytest.skip(f"integration test skipped: {module_name} not installed ({e})")
# Make the helper available on the pytest module namespace
pytest.require_module = _require # type: ignore[attr-defined]
@pytest.fixture(scope="session")
def webapp_root() -> Path:
"""The webapp/ directory root, useful for resolving fixture paths."""
return ROOT
@pytest.fixture(scope="session")
def referenzen_dir(webapp_root: Path) -> Path:
"""The static/referenzen directory containing all Wahlprogramm-PDFs."""
return webapp_root / "app" / "static" / "referenzen"