LawBenchmark.pl›Benchmark 02›Questions›P04

P04 · Permit for a foreigner to acquire real estate: final (ostateczna) or legally binding (prawomocna)

administrative procedure · closed question (yes / no / it depends…) · with a trap · key v3 of 2026-10-18 · legal-state date 1 October 2026

W0GPT-6.1 Sol W0: bez halucynacji, nie ugruntowanaGPT-6 Astra W0: bez halucynacji, nie ugruntowanaGPT-5.6 Terra W0: halucynacja TClaude Opus 5.5 W0: halucynacja TClaude Sonnet 5.5 W0: halucynacja TClaude Haiku 5.5 W0: halucynacja TGemini 3.8 Flash W0: halucynacja TGemini 4 Argon W0: halucynacja TW1GPT-6.1 Sol W1: bez halucynacji, nie ugruntowanaGPT-6 Astra W1: bez halucynacji, nie ugruntowanaGPT-5.6 Terra W1: bez halucynacji, nie ugruntowanaClaude Opus 5.5 W1: bez halucynacji, nie ugruntowanaClaude Sonnet 5.5 W1: halucynacja TClaude Haiku 5.5 W1: halucynacja TGemini 3.8 Flash W1: bez halucynacji, nie ugruntowanaGemini 4 Argon W1: halucynacja TSolAstraTerraOpusSonnetHaikuFlashArgon

correctly groundedno hallucinationhallucinated · rows: W0 no tools, W1 with web search

The question as used in the run (Polish)#

Postanowienie (zgoda) na nabycie nieruchomości przez obywatela Ukrainy. Takiemu orzeczeniu nadaje się klauzulę prawomocności czy ostateczności?

The lawyer's original text (Polish): „Postanowienie (zgoda) na nabycie nieurchomości przez obywatela ukrainy, takiemu orzeczeniu nadaje się klauzule prawomocności czy ostatenczosic ?”. The benchmark version adds the facts needed for an unambiguous key; the original typos stay.

The trap

Double. (1) The question calls the permit a "postanowienie" (procedural order) — only the objection of the Minister of Defence or of the minister for rural development takes that form (art. 1(1a)). (2) The question assumes an endorsement certifying finality, as used for court rulings — in administrative procedure the status of a decision is confirmed by the authority's certificate. A typical error: "an administrative decision can only be final; legal force concerns court rulings" — contrary to art. 16 § 3 of the Code of Administrative Procedure.

The correct answer according to the key#

Core answer. Not a procedural order but a decision of the minister for internal affairs (art. 1(1) of the Act on the Acquisition of Real Estate by Foreigners), and not a clause but the authority's certificate (art. 217 § 2(2) in conjunction with art. 218 § 1 of the Code of Administrative Procedure). The decision is final when a request for reconsideration is no longer available (art. 16 § 1, art. 127 § 3; 14 days), and legally binding when it can no longer be challenged before the administrative court (art. 16 § 3) — once 30 days have passed since service without a complaint being lodged with the voivodeship administrative court (the first-instance administrative court) (art. 53 § 1 of the Law on Proceedings before Administrative Courts; the complaint may also be lodged without a request for reconsideration, art. 52 § 3).

Required bases. art. 1(1) of the Act of 24.03.1920 on the Acquisition of Real Estate by Foreigners; art. 16 § 1 and § 3 of the Code of Administrative Procedure; art. 217 § 2(2) of the Code of Administrative Procedure.

Expected calibration: categorical. The B2 point (form of the conclusion) goes to an answer that states the conclusion unequivocally; blurring a clear thesis without saying what it would depend on costs the point.

The full key (rubric, penalties, quotation patterns from the source, known rulings) is in the repository in keys/P04_cudzoziemiec_nieruchomosc_ostatecznosc.yaml. Every legal basis was checked in the official text as of the legal-state date. Rules of the key.

Results of the eight models#

Hallucinated answers (of 8)W0: 6 · W1: 3
Correctly grounded (of 8)W0: 0 · W1: 0
Error classes W0T 6
Error classes W1T 3
Mean block A score (16 answers)43.0 %
Full core (A-R = 2)0 of 16
ModelConditionBlock A %ResultCard
GPT-6.1 SolW044–card
W144–card
GPT-6 AstraW044–card
W144–card
GPT-5.6 TerraW011Tcard
W144–card
Claude Opus 5.5W056Tcard
W189–card
Claude Sonnet 5.5W022Tcard
W122Tcard
Claude Haiku 5.5W022Tcard
W139Tcard
Gemini 3.8 FlashW060Tcard
W144–card
Gemini 4 ArgonW033Tcard
W167Tcard

correctly groundedno hallucination; not correctly groundedhallucinatedprovider blockColumns: models in report order; rows: W0 no tools, W1 with web search.

✓ correctly grounded · – no hallucination, but the core is incomplete or no source was checked · T W F S error classes in a hallucinated answer (definitions) · block text blocked by the provider's filter, no label · block A = criteria checked against the key and the source (scoring criteria)

The numbers concern one answer of each model to this question in each condition. They say nothing about run-to-run variance or about other questions in this field.

← P01all questionsP05 →