Research / Scorecard
Australian Legal Citation Benchmark
v0We run a fixed set of Australian legal questions through a live Lexi agent and publish the scores. The harness is in the repository, so you can run it yourself and check our work rather than take the number on trust.
Latest run
80.2/100
Average faithfulness
35/35
Questions scored
0
Hallucination flags
24 Oct 2026
Run date
Agent tested: lexiStudentAgent. Pass threshold 55. This is a heuristic score, not an assessment of legal correctness — see the limitations below.
Per-question results
| Question | Score | Authorities | Citations | Flags |
|---|---|---|---|---|
| au-hca-negligence Negligence — duty of care (HCA) | 65 | 2/4 | 18 | 0 |
| au-evidence-act-hearsay Evidence Act — hearsay | 90 | 4/4 | 10 | 0 |
| au-contract-consideration Contract — consideration | 58 | 2/3 | 6 | 0 |
| au-admin-jurisdictional-error Administrative law — jurisdictional error | 74 | 3/4 | 28 | 0 |
| au-criminal-mens-rea Criminal law — mens rea / He Kaw Teh | 66 | 2/3 | 8 | 0 |
| au-corporations-directors-duties Corporations — directors’ duties | 90 | 5/5 | 32 | 0 |
| au-constitutional-characterisation Constitutional law — characterisation | 81 | 3/4 | 41 | 0 |
| au-property-indefeasibility Property — Torrens indefeasibility | 90 | 4/4 | 39 | 0 |
| au-civil-liability-act-breach Negligence — breach under the Civil Liability Acts | 90 | 3/3 | 15 | 0 |
| au-negligence-pure-economic-loss Negligence — pure economic loss | 70 | 3/3 | 17 | 0 |
| au-vicarious-liability Tort — vicarious liability | 70 | 3/3 | 8 | 0 |
| au-defamation-serious-harm Defamation — serious harm and defences | 90 | 3/3 | 16 | 0 |
| au-native-title-extinguishment Native title — extinguishment | 90 | 3/3 | 16 | 0 |
| au-contract-estoppel Equity — promissory estoppel | 73 | 2/3 | 7 | 0 |
| au-contract-good-faith Contract — good faith | 53 | 1/2 | 8 | 0 |
| au-acl-misleading-conduct Consumer law — misleading or deceptive conduct | 90 | 3/3 | 20 | 0 |
| au-acl-unconscionable Consumer law — unconscionable conduct | 78 | 2/3 | 5 | 0 |
| au-evidence-tendency-coincidence Evidence — tendency and coincidence | 90 | 3/3 | 6 | 0 |
| au-evidence-client-legal-privilege Evidence — client legal privilege | 90 | 3/3 | 9 | 0 |
| au-statutory-interpretation-purpose Statutory interpretation — purposive approach | 79 | 3/3 | 11 | 0 |
| au-admin-procedural-fairness Administrative law — procedural fairness | 83 | 2/2 | 8 | 0 |
| au-admin-legal-unreasonableness Administrative law — legal unreasonableness | 57 | 1/2 | 6 | 0 |
| au-admin-privative-clause Administrative law — privative clauses | 90 | 3/3 | 8 | 0 |
| au-constitutional-implied-freedom Constitutional law — implied freedom of political communication | 90 | 3/3 | 11 | 0 |
| au-constitutional-external-affairs Constitutional law — external affairs power | 83 | 3/3 | 16 | 0 |
| au-corporations-insolvent-trading Corporations — insolvent trading | 90 | 3/3 | 14 | 0 |
| au-corporations-care-skill Corporations — duty of care and diligence | 90 | 3/3 | 12 | 0 |
| au-criminal-provocation Criminal law — provocation | 88 | 3/3 | 16 | 0 |
| au-criminal-complicity Criminal law — complicity | 72 | 3/3 | 13 | 0 |
| au-sentencing-principles Sentencing — instinctive synthesis | 82 | 2/2 | 16 | 0 |
| au-sentencing-bugmy Sentencing — Bugmy principles | 70 | 2/2 | 3 | 0 |
| au-family-property-settlement Family law — property settlement | 90 | 3/3 | 7 | 0 |
| au-equity-fiduciary-duties Equity — fiduciary duties | 82 | 3/3 | 6 | 0 |
| au-trusts-resulting-trust Trusts — resulting trusts | 74 | 2/2 | 7 | 0 |
| au-employment-unfair-dismissal Employment — unfair dismissal | 90 | 3/3 | 13 | 0 |
The question set (35 questions)
Fixed across runs so results are comparable over time. Expected citations are substring targets, not exhaustive authority lists.
au-hca-negligence
Negligence — duty of care (HCA)
What is the High Court of Australia’s approach to establishing a duty of care in novel negligence cases after Sullivan v Moody?
au-evidence-act-hearsay
Evidence Act — hearsay
Under the uniform Evidence Acts, when is hearsay evidence admissible and what is the relationship to the common law rule?
au-contract-consideration
Contract — consideration
Does Australian contract law require consideration for a simple contract, and how do the High Court and classic authorities treat past consideration?
au-admin-jurisdictional-error
Administrative law — jurisdictional error
Explain jurisdictional error in Australian administrative law after Craig and Kirk, with reference to High Court authority.
au-criminal-mens-rea
Criminal law — mens rea / He Kaw Teh
How does He Kaw Teh v The Queen guide interpretation of mens rea in Commonwealth offence provisions?
au-corporations-directors-duties
Corporations — directors’ duties
Summarise the statutory and general law duties of directors under the Corporations Act 2001 (Cth), citing key sections and leading cases.
au-constitutional-characterisation
Constitutional law — characterisation
How does the High Court approach characterisation of a Commonwealth law under a head of power, with reference to Engineers and later authority?
au-property-indefeasibility
Property — Torrens indefeasibility
Explain immediate indefeasibility under the Torrens system in Australia and the fraud / in personam exceptions, citing High Court authority.
au-civil-liability-act-breach
Negligence — breach under the Civil Liability Acts
How is breach of duty determined under the Civil Liability Act 2002 (NSW), and how do the statutory considerations interact with the common law?
au-negligence-pure-economic-loss
Negligence — pure economic loss
When can a plaintiff recover for pure economic loss in Australian negligence law, and how did Perre v Apand change the approach?
au-vicarious-liability
Tort — vicarious liability
What is the test for vicarious liability in Australia, and how does it apply to intentional wrongdoing by an employee?
au-defamation-serious-harm
Defamation — serious harm and defences
What must a plaintiff establish to succeed in defamation in Australia after the 2021 amendments, and what are the main defences?
au-native-title-extinguishment
Native title — extinguishment
How does the Native Title Act 1993 (Cth) deal with extinguishment of native title, and what did Wik decide about pastoral leases?
au-contract-estoppel
Equity — promissory estoppel
What are the elements of promissory estoppel in Australian law, and how did Waltons Stores v Maher depart from the English approach?
au-contract-good-faith
Contract — good faith
Is there an implied duty of good faith in Australian contract law, and what is the current state of authority?
au-acl-misleading-conduct
Consumer law — misleading or deceptive conduct
What is required to establish misleading or deceptive conduct under s 18 of the Australian Consumer Law?
au-acl-unconscionable
Consumer law — unconscionable conduct
How do the courts distinguish unconscionable conduct under the Australian Consumer Law from the equitable doctrine of unconscionable dealing?
au-evidence-tendency-coincidence
Evidence — tendency and coincidence
What must be shown before tendency or coincidence evidence is admissible under the uniform Evidence Acts, and how did Hughes v The Queen apply the test?
au-evidence-client-legal-privilege
Evidence — client legal privilege
When does client legal privilege apply under the Evidence Act 1995 (Cth), and when can it be waived?
au-statutory-interpretation-purpose
Statutory interpretation — purposive approach
How do Australian courts approach statutory interpretation, and what role does s 15AA of the Acts Interpretation Act 1901 (Cth) play?
au-admin-procedural-fairness
Administrative law — procedural fairness
What does procedural fairness require of a decision-maker, and how has Kioa v West shaped the doctrine?
au-admin-legal-unreasonableness
Administrative law — legal unreasonableness
What is legal unreasonableness in Australian administrative law, and how did Minister for Immigration v Li restate the test?
au-admin-privative-clause
Administrative law — privative clauses
How effective are privative clauses in ousting judicial review in Australia after Plaintiff S157/2002 v Commonwealth?
au-constitutional-implied-freedom
Constitutional law — implied freedom of political communication
What is the current test for the implied freedom of political communication, and how was it applied in McCloy v New South Wales?
au-constitutional-external-affairs
Constitutional law — external affairs power
How has the High Court interpreted s 51(xxix) of the Constitution, and what did Commonwealth v Tasmania establish?
au-corporations-insolvent-trading
Corporations — insolvent trading
What are the elements of insolvent trading under s 588G of the Corporations Act 2001 (Cth), and what defences are available?
au-corporations-care-skill
Corporations — duty of care and diligence
What standard of care and diligence does s 180 of the Corporations Act 2001 (Cth) require of directors, and how was it applied in ASIC v Healey?
au-criminal-provocation
Criminal law — provocation
What is the test for provocation in Australian criminal law, and how does it reduce murder to manslaughter?
au-criminal-complicity
Criminal law — complicity
How does Australian law distinguish joint criminal enterprise from accessorial liability, and what did McAuliffe v The Queen establish?
au-sentencing-principles
Sentencing — instinctive synthesis
How do Australian courts approach sentencing, and what is meant by instinctive synthesis after Markarian v The Queen?
au-sentencing-bugmy
Sentencing — Bugmy principles
What did Bugmy v The Queen decide about the relevance of an offender's deprived background to sentencing?
au-family-property-settlement
Family law — property settlement
What principles govern property settlement under s 79 of the Family Law Act 1975 (Cth), and what did Stanford v Stanford require?
au-equity-fiduciary-duties
Equity — fiduciary duties
When does a fiduciary relationship arise, and what duties follow? Consider Hospital Products v United States Surgical Corporation.
au-trusts-resulting-trust
Trusts — resulting trusts
When will a presumption of resulting trust arise, and how did Calverley v Green approach contribution to purchase price?
au-employment-unfair-dismissal
Employment — unfair dismissal
What must an applicant establish for an unfair dismissal claim under the Fair Work Act 2009 (Cth), and what remedies are available?
How the score is calculated
Heuristic evaluation of Australian legal research answers for AGLC-style citations, pinpoints, and suspected invented authorities. Not a substitute for lawyer review.
aglcFormatHits
Count of case/legislation strings matching common AGLC patterns (MNC, reported, Act (Jur) s …).
pinCiteCount
Count of pinpoint references such as [23] or at 45.
suspectedHallucinatedPatterns
Matched patterns for fake courts, unknown court codes, placeholders, or non-AU invention signals.
faithfulnessScore
0–100 composite from format hits, pin cites, optional expected-citation match rate, minus hallucination penalties.
Limitations
- Does not verify that a cited case exists or stands for the proposition claimed.
- AGLC matching is pattern-based and may miss valid uncommon forms.
- Each figure comes from a single run. Answers are generated by a language model, so scores move between runs — per-question results have varied by up to 30 points on identical inputs. Treat the figure as one sample, not a stable property.
- Intended for regression evals and public methodology transparency, not certification.
- Format is not substance. A response can cite the right authority in perfect AGLC4 form and still misstate what it held. This benchmark cannot detect that, and no automated score can.
- Citation counts reward longer answers. A short, correct answer can score below a long, citation-dense one.
- The question set is small and curated. It is a regression check, not a survey of Australian law.
Reproduce it
Scores are produced by a script in the repository. Set EVAL_LIVE=1 with a backend URL and shared secret, then run:
node scripts/eval/citation-faithfulness-live.mjs
Raw results: GET /api/research/scorecard