LawBenchmark.pl›Benchmark 02›Questions›P01

P01 · Threshold for property offences and the minimum wage

petty offences law · calculation question (numerical result) · with a trap · key v3 of 2026-10-18 · legal-state date 1 October 2026

W0GPT-6.1 Sol W0: poprawnie ugruntowanaGPT-6 Astra W0: poprawnie ugruntowanaGPT-5.6 Terra W0: halucynacja WClaude Opus 5.5 W0: halucynacja T,WClaude Sonnet 5.5 W0: halucynacja T,WClaude Haiku 5.5 W0: halucynacja WGemini 3.8 Flash W0: halucynacja T,WGemini 4 Argon W0: halucynacja WW1GPT-6.1 Sol W1: poprawnie ugruntowanaGPT-6 Astra W1: poprawnie ugruntowanaGPT-5.6 Terra W1: halucynacja WClaude Opus 5.5 W1: halucynacja WClaude Sonnet 5.5 W1: halucynacja TClaude Haiku 5.5 W1: halucynacja TGemini 3.8 Flash W1: halucynacja T,WGemini 4 Argon W1: halucynacja T,WSolAstraTerraOpusSonnetHaikuFlashArgon

correctly groundedno hallucinationhallucinated · rows: W0 no tools, W1 with web search

The question as used in the run (Polish)#

Ile wynosi w tym roku wartość mienia przy wykroczeniu max ? Czy zmieniła się w zw. ze wzrostem płacy minmalnej ?

The lawyer's original text (Polish): „Hej, ile wynosi w tym roku wartość mienia przy wykroczeniu max ? Czy zmieniła się w zw. ze wzrostem płacy minmalnej ?”. The benchmark version adds the facts needed for an unambiguous key; the original typos stay.

The trap

The question assumes the threshold is tied to the minimum wage. No such link applies: before 1.10.2023 the threshold was a fixed PLN 500, and since 1.10.2023 it is a fixed PLN 800.

The correct answer according to the key#

Core answer. PLN 800. Since 1.10.2023 the threshold is a fixed amount written into the provision (art. 119 § 1 of the Petty Offences Code for theft and misappropriation; likewise art. 120 § 1, 122 § 1 and 124 § 1), so a rise in the minimum wage does not change it.

Required bases. art. 119 § 1 Petty Offences Code

Expected calibration: categorical. The B2 point (form of the conclusion) goes to an answer that states the conclusion unequivocally; blurring a clear thesis without saying what it would depend on costs the point.

The full key (rubric, penalties, quotation patterns from the source, known rulings) is in the repository in keys/P01_wykroczenie_prog_mienia.yaml. Every legal basis was checked in the official text as of the legal-state date. Rules of the key.

Results of the eight models#

Hallucinated answers (of 8)W0: 6 · W1: 6
Correctly grounded (of 8)W0: 2 · W1: 2
Error classes W0T 3, W 6
Error classes W1T 4, W 4
Mean block A score (16 answers)69.5 %
Full core (A-R = 2)13 of 16
ModelConditionBlock A %ResultCard
GPT-6.1 SolW0100✓card
W189✓card
GPT-6 AstraW0100✓card
W189✓card
GPT-5.6 TerraW078Wcard
W10Wcard
Claude Opus 5.5W078TWcard
W189Wcard
Claude Sonnet 5.5W00TWcard
W172Tcard
Claude Haiku 5.5W00Wcard
W178Tcard
Gemini 3.8 FlashW078TWcard
W189TWcard
Gemini 4 ArgonW083Wcard
W189TWcard

correctly groundedno hallucination; not correctly groundedhallucinatedprovider blockColumns: models in report order; rows: W0 no tools, W1 with web search.

✓ correctly grounded · – no hallucination, but the core is incomplete or no source was checked · T W F S error classes in a hallucinated answer (definitions) · block text blocked by the provider's filter, no label · block A = criteria checked against the key and the source (scoring criteria)

The numbers concern one answer of each model to this question in each condition. They say nothing about run-to-run variance or about other questions in this field.

all questionsP04 →