Research / Scorecard

Australian Legal Citation Benchmark

v0

We run a fixed set of Australian legal questions through a live Lexi agent and publish the scores. The harness is in the repository, so you can run it yourself and check our work rather than take the number on trust.

Latest run

80.2/100

Average faithfulness

35/35

Questions scored

0

Hallucination flags

24 Oct 2026

Run date

Agent tested: lexiStudentAgent. Pass threshold 55. This is a heuristic score, not an assessment of legal correctness — see the limitations below.

Per-question results

QuestionScoreAuthoritiesCitationsFlags
au-hca-negligence
Negligence — duty of care (HCA)
652/4180
au-evidence-act-hearsay
Evidence Act — hearsay
904/4100
au-contract-consideration
Contract — consideration
582/360
au-admin-jurisdictional-error
Administrative law — jurisdictional error
743/4280
au-criminal-mens-rea
Criminal law — mens rea / He Kaw Teh
662/380
au-corporations-directors-duties
Corporations — directors’ duties
905/5320
au-constitutional-characterisation
Constitutional law — characterisation
813/4410
au-property-indefeasibility
Property — Torrens indefeasibility
904/4390
au-civil-liability-act-breach
Negligence — breach under the Civil Liability Acts
903/3150
au-negligence-pure-economic-loss
Negligence — pure economic loss
703/3170
au-vicarious-liability
Tort — vicarious liability
703/380
au-defamation-serious-harm
Defamation — serious harm and defences
903/3160
au-native-title-extinguishment
Native title — extinguishment
903/3160
au-contract-estoppel
Equity — promissory estoppel
732/370
au-contract-good-faith
Contract — good faith
531/280
au-acl-misleading-conduct
Consumer law — misleading or deceptive conduct
903/3200
au-acl-unconscionable
Consumer law — unconscionable conduct
782/350
au-evidence-tendency-coincidence
Evidence — tendency and coincidence
903/360
au-evidence-client-legal-privilege
Evidence — client legal privilege
903/390
au-statutory-interpretation-purpose
Statutory interpretation — purposive approach
793/3110
au-admin-procedural-fairness
Administrative law — procedural fairness
832/280
au-admin-legal-unreasonableness
Administrative law — legal unreasonableness
571/260
au-admin-privative-clause
Administrative law — privative clauses
903/380
au-constitutional-implied-freedom
Constitutional law — implied freedom of political communication
903/3110
au-constitutional-external-affairs
Constitutional law — external affairs power
833/3160
au-corporations-insolvent-trading
Corporations — insolvent trading
903/3140
au-corporations-care-skill
Corporations — duty of care and diligence
903/3120
au-criminal-provocation
Criminal law — provocation
883/3160
au-criminal-complicity
Criminal law — complicity
723/3130
au-sentencing-principles
Sentencing — instinctive synthesis
822/2160
au-sentencing-bugmy
Sentencing — Bugmy principles
702/230
au-family-property-settlement
Family law — property settlement
903/370
au-equity-fiduciary-duties
Equity — fiduciary duties
823/360
au-trusts-resulting-trust
Trusts — resulting trusts
742/270
au-employment-unfair-dismissal
Employment — unfair dismissal
903/3130

The question set (35 questions)

Fixed across runs so results are comparable over time. Expected citations are substring targets, not exhaustive authority lists.

  • au-hca-negligence

    Negligence — duty of care (HCA)

    What is the High Court of Australia’s approach to establishing a duty of care in novel negligence cases after Sullivan v Moody?

  • au-evidence-act-hearsay

    Evidence Act — hearsay

    Under the uniform Evidence Acts, when is hearsay evidence admissible and what is the relationship to the common law rule?

  • au-contract-consideration

    Contract — consideration

    Does Australian contract law require consideration for a simple contract, and how do the High Court and classic authorities treat past consideration?

  • au-admin-jurisdictional-error

    Administrative law — jurisdictional error

    Explain jurisdictional error in Australian administrative law after Craig and Kirk, with reference to High Court authority.

  • au-criminal-mens-rea

    Criminal law — mens rea / He Kaw Teh

    How does He Kaw Teh v The Queen guide interpretation of mens rea in Commonwealth offence provisions?

  • au-corporations-directors-duties

    Corporations — directors’ duties

    Summarise the statutory and general law duties of directors under the Corporations Act 2001 (Cth), citing key sections and leading cases.

  • au-constitutional-characterisation

    Constitutional law — characterisation

    How does the High Court approach characterisation of a Commonwealth law under a head of power, with reference to Engineers and later authority?

  • au-property-indefeasibility

    Property — Torrens indefeasibility

    Explain immediate indefeasibility under the Torrens system in Australia and the fraud / in personam exceptions, citing High Court authority.

  • au-civil-liability-act-breach

    Negligence — breach under the Civil Liability Acts

    How is breach of duty determined under the Civil Liability Act 2002 (NSW), and how do the statutory considerations interact with the common law?

  • au-negligence-pure-economic-loss

    Negligence — pure economic loss

    When can a plaintiff recover for pure economic loss in Australian negligence law, and how did Perre v Apand change the approach?

  • au-vicarious-liability

    Tort — vicarious liability

    What is the test for vicarious liability in Australia, and how does it apply to intentional wrongdoing by an employee?

  • au-defamation-serious-harm

    Defamation — serious harm and defences

    What must a plaintiff establish to succeed in defamation in Australia after the 2021 amendments, and what are the main defences?

  • au-native-title-extinguishment

    Native title — extinguishment

    How does the Native Title Act 1993 (Cth) deal with extinguishment of native title, and what did Wik decide about pastoral leases?

  • au-contract-estoppel

    Equity — promissory estoppel

    What are the elements of promissory estoppel in Australian law, and how did Waltons Stores v Maher depart from the English approach?

  • au-contract-good-faith

    Contract — good faith

    Is there an implied duty of good faith in Australian contract law, and what is the current state of authority?

  • au-acl-misleading-conduct

    Consumer law — misleading or deceptive conduct

    What is required to establish misleading or deceptive conduct under s 18 of the Australian Consumer Law?

  • au-acl-unconscionable

    Consumer law — unconscionable conduct

    How do the courts distinguish unconscionable conduct under the Australian Consumer Law from the equitable doctrine of unconscionable dealing?

  • au-evidence-tendency-coincidence

    Evidence — tendency and coincidence

    What must be shown before tendency or coincidence evidence is admissible under the uniform Evidence Acts, and how did Hughes v The Queen apply the test?

  • au-evidence-client-legal-privilege

    Evidence — client legal privilege

    When does client legal privilege apply under the Evidence Act 1995 (Cth), and when can it be waived?

  • au-statutory-interpretation-purpose

    Statutory interpretation — purposive approach

    How do Australian courts approach statutory interpretation, and what role does s 15AA of the Acts Interpretation Act 1901 (Cth) play?

  • au-admin-procedural-fairness

    Administrative law — procedural fairness

    What does procedural fairness require of a decision-maker, and how has Kioa v West shaped the doctrine?

  • au-admin-legal-unreasonableness

    Administrative law — legal unreasonableness

    What is legal unreasonableness in Australian administrative law, and how did Minister for Immigration v Li restate the test?

  • au-admin-privative-clause

    Administrative law — privative clauses

    How effective are privative clauses in ousting judicial review in Australia after Plaintiff S157/2002 v Commonwealth?

  • au-constitutional-implied-freedom

    Constitutional law — implied freedom of political communication

    What is the current test for the implied freedom of political communication, and how was it applied in McCloy v New South Wales?

  • au-constitutional-external-affairs

    Constitutional law — external affairs power

    How has the High Court interpreted s 51(xxix) of the Constitution, and what did Commonwealth v Tasmania establish?

  • au-corporations-insolvent-trading

    Corporations — insolvent trading

    What are the elements of insolvent trading under s 588G of the Corporations Act 2001 (Cth), and what defences are available?

  • au-corporations-care-skill

    Corporations — duty of care and diligence

    What standard of care and diligence does s 180 of the Corporations Act 2001 (Cth) require of directors, and how was it applied in ASIC v Healey?

  • au-criminal-provocation

    Criminal law — provocation

    What is the test for provocation in Australian criminal law, and how does it reduce murder to manslaughter?

  • au-criminal-complicity

    Criminal law — complicity

    How does Australian law distinguish joint criminal enterprise from accessorial liability, and what did McAuliffe v The Queen establish?

  • au-sentencing-principles

    Sentencing — instinctive synthesis

    How do Australian courts approach sentencing, and what is meant by instinctive synthesis after Markarian v The Queen?

  • au-sentencing-bugmy

    Sentencing — Bugmy principles

    What did Bugmy v The Queen decide about the relevance of an offender's deprived background to sentencing?

  • au-family-property-settlement

    Family law — property settlement

    What principles govern property settlement under s 79 of the Family Law Act 1975 (Cth), and what did Stanford v Stanford require?

  • au-equity-fiduciary-duties

    Equity — fiduciary duties

    When does a fiduciary relationship arise, and what duties follow? Consider Hospital Products v United States Surgical Corporation.

  • au-trusts-resulting-trust

    Trusts — resulting trusts

    When will a presumption of resulting trust arise, and how did Calverley v Green approach contribution to purchase price?

  • au-employment-unfair-dismissal

    Employment — unfair dismissal

    What must an applicant establish for an unfair dismissal claim under the Fair Work Act 2009 (Cth), and what remedies are available?

How the score is calculated

Heuristic evaluation of Australian legal research answers for AGLC-style citations, pinpoints, and suspected invented authorities. Not a substitute for lawyer review.

aglcFormatHits

Count of case/legislation strings matching common AGLC patterns (MNC, reported, Act (Jur) s …).

pinCiteCount

Count of pinpoint references such as [23] or at 45.

suspectedHallucinatedPatterns

Matched patterns for fake courts, unknown court codes, placeholders, or non-AU invention signals.

faithfulnessScore

0–100 composite from format hits, pin cites, optional expected-citation match rate, minus hallucination penalties.

Limitations

  • Does not verify that a cited case exists or stands for the proposition claimed.
  • AGLC matching is pattern-based and may miss valid uncommon forms.
  • Each figure comes from a single run. Answers are generated by a language model, so scores move between runs — per-question results have varied by up to 30 points on identical inputs. Treat the figure as one sample, not a stable property.
  • Intended for regression evals and public methodology transparency, not certification.
  • Format is not substance. A response can cite the right authority in perfect AGLC4 form and still misstate what it held. This benchmark cannot detect that, and no automated score can.
  • Citation counts reward longer answers. A short, correct answer can score below a long, citation-dense one.
  • The question set is small and curated. It is a regression check, not a survey of Australian law.

Reproduce it

Scores are produced by a script in the repository. Set EVAL_LIVE=1 with a backend URL and shared secret, then run:

node scripts/eval/citation-faithfulness-live.mjs

Raw results: GET /api/research/scorecard