LawBenchmark.pl›Benchmark 01›Case study

Case study: the time limit for invoking the free-credit sanction

Benchmark 01, a pilot of September and October 2026. One fictional litigation case and 17 answers from general-purpose models in various configurations, repetitions included, scored with the pilot's own metric, i.e. its own criteria and scoring rules.

Date of the consumer's declaration in the case: May 2026 · status of pending matters as of 13.09.2026 · cards of 13.09 and 8–9.10.2026 · pilot metric A1–A8 and B1–B2

This is a pilot, not part of Benchmark 02. The case was scored with a different metric and through different interfaces, so its numbers should not be compared with the results of the ten questions. Five cards were reviewed by a lawyer in September 2026; the 12 cards of 8–9.10.2026 are preliminary scores filled in by a model, marked as such in the table and in each card. The pilot also scored five legal-research systems: three LexAlpha modules (LexAlpha co-organises the project) and two systems of other companies. Their results are not part of this publication.

The case#

The facts and the instruction exactly as given to the systems (in Polish, typos kept). The case is fictional.

SKD ##1 1.1 Stan faktyczny 1) Kredytobiorca zawarl umowę kredytu konsumenckiego w czerwcu 2022 r. na kwotę 50 000 zł, przy czym całkowita kwota kredytu wynosiła 40 000 zł 2) 10 000 zł to kredytowane koszty, od których bank również naliczał odsetki. 3) W czerwu 2024 r. kredytobiorca złozyl reklamacje do banku, wskazując, że postanowienia które pozwalają naliczać bankowi odsetki od kredytowanych kosztów są abuzywne, w z.w czym bank powinien rozliczyć nadpłacone środki z przeszłości i zmienić harmonogram na przyszłość. 4) Bank odmówił uwzględnienia reklamacji oraz zmiany rozliczeń umowy. 5) Kredytobiorca sam dokonał nowego przeliczenia haramogramu i od lipca 2024 r. spłacał do banku niższe raty tj. z odsetkami liczonymi od kwoty pomnniejszone o kredytowane koszty. 6) W grudniu 2024 r. ze względu na (zdaniem banku) niedopłatę bank wypowiedział umowę. 7) W maju 2026 r. kredytobiorca złożył do banku oświadczenie o skorzystaniu z sankcji kredytu darmowego, powołując się na najnowszy wyrok TSUE, zgodnie z którym bank może naliczać odsetki tylko od calkowitej kwoty kredytu. 8) Bank odmówił uwzględnienia stanowiska konsumenta wskazując, że TSUE nie przesądził od braku możliwości pobierania odsetek od kredytowancyh kosztów a ponadto nawet gdyby uznać stanowisko konsumenta za uzasadnione to i tak od momentu wypowiedzenia umowy - czyli jej wykonania zgodnie z ustawą minął rok i termin na złożenie oświadczenia wygasł 9)kredytobiorca nie zgodził się z tak przedstawionym stanowiskiem banku. 1.2 W tak przedstawionym stanie faktczynym. 1) Ustal na jaką datę trzeba ustalic treść przepisów dla poszczególnych zdarzeń prawnych. 2) Ustal przepisy prawnej mające zastosowanie w tej sprawie. 3) Podaj dokładną treść przepisu (zacytuj), który będzie mieć zastosowanie w konkretnej dacie przy analizie 4) Przenalizuj staowisko każdej ze stron, 5) Wskaż w świetle orzecznictwa oraz poglądów doktryny, które stanowisko jest bardziej uzasadnione i ma większe szanse na obronę w sądzie. 6) Dla każdego ze stanowisko przedstaw tok rozumowania oraz odpowiednią argumetnację prawną. 1.3 Oczekiwany format odpowiedzi: 1) Przepisy mające zastosowanie w sprawie 2) Daty na jaką należy ustalić treść przepisów 3) dosłowna treść przepisów (cytat) 4) analiza stanowiska każdej ze stron (max 10 zdań dla każdego stanowiska) 5)wskazanie, które stanowisko ma większą szansę na obronę w sądzie wraz z uzasadnieniem i przywołaniem odpowiedniego stanowiskka doktryny i orzecznictwa 1.4 Założenia 1) Skup się wyłącznie na zagadnieniu terminu do złożenia oświadczenia SKD. Kwestię kredytowania kosztów uwzględniaj tylko jeżeli ma to wpływ na zagadnienie terminu do złożenia oświadczenia SKD. 2) Przyjmij, że wypowiedzenie nie zawierało braków formalnych i dla jego skuteczności może mieć ew. wpływ zagadnienie kredytowanych kosztów. 3) Jeżeli któreś z zagadnień prawnych nie jest jednolicie przesądzone i wskaż na możliwe rozwiązania

What the case measures#

The instruction tells the system to focus solely on the time limit under art. 45(5) of the Consumer Credit Act and to consider the credit charges financed through the loan only where they affect that limit. Whether interest on financed credit charges was lawful may decide whether there were arrears, and so whether the termination was effective. Whether the termination starts the time limit depends in turn on how "performance of the agreement" in art. 45(5) is construed. The task requires taking this dependency into account. Add to this the dates relevant to each event, verbatim quotations of the provisions and the identification of disputed issues.

The pilot metric#

CriterionQuestionPoints
A1Was the date for determining the applicable wording of the provisions set correctly, separately for each event?0/1
A2Were the applicable provisions identified, including art. 75(1) of the Banking Law and art. 385¹ of the Civil Code?0/1
A3Are the quotations of provisions verbatim or marked as excerpts?0/1
A4Did the model notice that the financed credit charges matter for the time limit, and name the mechanism?0/1
A5Were the disputed issues identified candidly?0/1
A6Does the answer avoid citing non-existent sources or sources with content they do not contain? Since 11.10.2026 binary, with classes F, T, W, S as in Benchmark 02.0/1
A7Did the model address every point of the instruction, in the required format?0/1
A8Did the model stay within the scope of the task?0/1
B1Analysis of the consumer's and the bank's positions, separately: none, a summary, or an analysis of the arguments.0–2 per party
B2Indication of which position is more likely to prevail: without reasons, with superficial reasons, or with reasoning grounded in case law and legal scholarship.0–3

Block A has 8 points, so one point is 12.5 percentage points; block B has 7 points. The blocks are not summed. The difference Δ = A − B, computed from the rounded percentages, shows the gap between the quality of the argument and the block A criteria; the reason is visible in the card. The label "hallucinated" follows from A6 = 0. Error classes: F, the source does not exist; T, a real source with content it does not contain; W, wording from another date; S, a distorted identifier. "S (unconfirmed)" means the case number was not found, without deciding whether it is distorted or invented. Full metric: file METRYKI.md in the repository.

Results#

Cards reviewed by the lawyer

Answers from the providers' chat apps, cards of 13.09.2026, reviewed by the lawyer in September. 3 of 5 answers were labelled hallucinated. The numbers differ from the September article for two reasons: on 11.10.2026 the cards were recomputed under the binary A6 rule approved by the lawyer, and on 29.10.2026 the lawyer classified the case number "judgment of the Court of Appeal in Warsaw of 10 May 2022, V ACa 810/21" as non-existent (class F): it is absent from SAOS, from the internal case-law database and from the court's own portal. The change history is in each card.

ModelConditionBlock ABlock BΔ A−B ppHallucinated (per card)Files
Claude Opus 5.0 (High)
Anthropic · chat app
with web search7/8 · 88 %7/7 · 100 %-12nocard · answer
Claude Fable 5.1 (High)
Anthropic · chat app
with web search6/8 · 75 %7/7 · 100 %-25yes T, Scard · answer
Gemini 3.1 Pro
Google · chat app
without web search6/8 · 75 %5/7 · 71 %+4nocard · answer
Gemini 3.6 Flash
Google · chat app
with web search3/8 · 38 %5/7 · 71 %-33yes Tcard · answer
Gemini 3.6 Flash Thinking
Google · chat app
without web search2/8 · 25 %4/7 · 57 %-32yes Fcard · answer

Preliminary scores, not reviewed by the lawyer

Answers collected on 8–9.10.2026 by scripts: GPT-6.1 Sol through Codex CLI and open models through Ollama. The cards were filled in by a model (Claude); 7 of 12 were labelled hallucinated. The results may change after the lawyer's review.

ModelConditionBlock ABlock BΔ A−B ppHallucinated (per card)Files
Bielik 11B v3 (Q4_K_M)
SpeakLeash · Ollama, local
no tools1/8 · 13 %4/7 · 57 %-44yes Tcard · answer
Bielik 11B v3 (Q8_0)
SpeakLeash · Ollama, GCP cloud
no tools1/8 · 13 %3/7 · 43 %-30yes Tcard · answer
Bielik 11B v3 with web tools
SpeakLeash · Ollama, local
search available, not used1/8 · 13 %3/7 · 43 %-30yes T, S (unconfirmed)card · answer
Bielik 11B v3, forced search
SpeakLeash · Ollama, local
instructed to search; did not use the tool0/8 · 0 %3/7 · 43 %-43yes T, S (unconfirmed)card · answer
GPT-6.1 Sol, reasoning medium
OpenAI · Codex CLI
with web search8/8 · 100 %7/7 · 100 %+0nocard · answer
GPT-6.1 Sol, reasoning high
OpenAI · Codex CLI
with web search8/8 · 100 %7/7 · 100 %+0nocard · answer
GPT-6.1 Sol, reasoning medium (repeat)
OpenAI · Codex CLI
with web search7/8 · 88 %7/7 · 100 %-12nocard · answer
GPT-6.1 Sol, reasoning medium
OpenAI · Codex CLI
no tools6/8 · 75 %6/7 · 86 %-11nocard · answer
GPT-6.1 Sol, reasoning medium (repeat)
OpenAI · Codex CLI
no tools6/8 · 75 %7/7 · 100 %-25nocard · answer
Qwen 3.5 9B
Alibaba · Ollama, local
no tools1/8 · 13 %4/7 · 57 %-44yes F, T, Scard · answer
Qwen 3.5 9B with web tools
Alibaba · Ollama, local
with web search1/8 · 13 %4/7 · 57 %-44yes T, Scard · answer
Qwen 3.8 27B
Alibaba · Ollama, GCP cloud
no tools1/8 · 13 %6/7 · 86 %-73yes T, Scard · answer

One answer per system and condition. "Repeat" variants are a second attempt of the same model under the same conditions, shown to make the run-to-run variability visible. The no-tools answer of GPT-6.1 Sol with ChatGPT account apps enabled was left out, because the apps gave access to a search engine.

The key in short#

  • Relevant dates: unfairness of the terms as at the conclusion of the agreement (June 2022, art. 385² of the Civil Code); effectiveness of the termination as at December 2024; art. 45(5) of the Consumer Credit Act with reasons for the date chosen (the provision has not changed since 2011).
  • Required bases: art. 45(1) and (5) of the Consumer Credit Act, art. 75(1) of the Banking Law, art. 385¹ § 1 of the Civil Code.
  • Disputed as at the date of the declaration (May 2026): what starts the time limit under art. 45(5) (performance by the lender, by both parties, or termination; proceedings on the question of law III CZP 15/25 before the Supreme Court stayed); whether the consumer could lower the instalments on their own without a ruling on unfairness; whether termination is "performance" of the agreement.
  • Case law: the CJEU judgment C-744/24 of 23.04.2026 concerns interest on financed credit charges and does not decide the time limit; the preliminary references C-763/25 and C-828/25 on the time limit were pending as of 13.09.2026.

The key describes the state as of 13.09.2026 and has not been updated since. Matters pending then may have been decided; the case is not a description of the current line of case law and is not legal advice.

Data#

The case, the key, the pilot metric, the 17 answers and cards and the summary wyniki.json are in the repository under cases/skd-01/, licensed like the rest of the dataset (licences). Every file carries a canary, a unique string that makes some uses of these data for model training detectable. The case is now public, so it is no longer suitable for testing new models.