SOoL Research Workbench

loading… analysis set
Query Tool ↗

A guide for readers without a statistics background

This Workbench lets you put questions to about five thousand U.S. appellate decisions, each read and coded according to A Structural Ontology of the Law. You do not need to know statistics or databases. You can click one of the questions below, read the green "In plain words" box at the top of the answer, and stop there. Everything underneath is the evidence, for when you want it.

The idea being tested

SOoL holds that every legal effect (a right, a duty, a protection) depends on a chain of eight links. When law fails, it fails at a link, and the kind of failure has a structure of its own. The central claim is that this structure is the same across areas of law: a role contradiction in an employment case and one in an immigration case are the same kind of thing, whatever the doctrine. If that is true, there is a legal architecture underneath the doctrines. The Workbench is built to test that claim, and to try to break it.

The chain, in plain terms

Take a tenant's duty to pay April's rent.

  1. Source of authority. Something had the power to make the rule: here, the parties' power to contract.
  2. Norm. The rule itself: the lease's terms.
  3. Actor in role. Someone acting in a legal capacity: the tenant, as tenant.
  4. Triggering facts. What sets the rule off: April begins.
  5. Legal act or omission. The conduct the rule concerns: signing, paying, or not paying.
  6. Target. Whom or what the right or duty is directed at: the landlord.
  7. Legal effect. The right or duty itself comes into being. In this corpus it is coded from the claimant's side, so it closely tracks who won.
  8. Remedy. What happens if it is not honoured: eviction, damages.

The links fall into four clusters, the same four used by the Legal Ontologies Foundry: Authority (1–3), Effect (4–5), Recognition (6–7) and Repair (8). For each case, each link is coded as C closed (it held), P partial (it held with strain), F failed, or I indeterminate (the opinion does not say). The Workbench treats "partial or failed" as a break and treats "indeterminate" as missing information, never as a break.

The thirteen kinds of contradiction

Start here: questions you can ask with one click

Each button opens the right tool with sensible settings and runs it. Exploratory questions run on the discovery 70% of the corpus (see "Explore, then confirm" below).

How to read an answer

The green box
A sentence or two saying what the result means, written so you can quote it to a colleague. The numbers come from the analysis; the wording only translates them.
Ranges in brackets, like 0.42 [0.38, 0.46]
The first number is the best estimate. The bracket is the range of values the data still leave plausible (a "95% confidence interval"). A wide bracket means "we don't know very precisely". A bracket that excludes the value of "no difference" (0 for a difference, 1 for a ratio) means the difference is unlikely to be luck.
p and q
Roughly, how surprising the result would be if nothing were going on. Small (below 0.05) means surprising. When a tool runs many tests at once, some will look surprising by luck alone, so the Workbench reports q, a p-value adjusted for how many tests were run. Trust q over p.
Invariant, domain-specific, inconclusive
The verdicts that matter most for SOoL. Invariant means the data show, with confidence, that the structure is the same across areas of law, within a tolerance you can set. Domain-specific means it clearly differs. Inconclusive means the data cannot tell. Failing to find a difference is not treated as proof of sameness; that common error is the one the Workbench is designed to avoid.
Entailed, rule-forced, empirical
Some connections are true by definition. A Role Contradiction is defined on the actor-in-role link, so the two will always go together. The annotation instructions force others: the annotator was told to code Norm Indeterminacy whenever a court left a norm's scope open. These appear hatched or grey and are never counted as findings. Only empirical connections count: structure the data show beyond what the method built in.
Explore, then confirm
The corpus is split, once and permanently, into a 70% discovery set and a 30% confirmation set, balanced across areas of law. Look around freely in discovery. When you find something, press Confirm once: the same analysis runs on cases you have not seen. A pattern that holds there is much more believable. This is how researchers avoid fooling themselves.
The robustness bar
Small boxes showing whether a headline result survives some hard tests: only high-confidence annotations; the forced contradiction types removed; links 7 and 8 removed (they track who won); doubtful cases removed. A result that holds in all of them is sturdy.

Asking your own questions

A cohort is any group of cases you define, and every tool can be run on one. The quickest way to define a cohort is the chain pattern box on the Query tab. A few examples, each followed by what it means:

  • N2=PF !ct:NI: the norm broke, but nobody coded it as Norm Indeterminacy.
  • N3=F ct:RC3 domain:D2,D5: the actor-in-role link failed with a Role Contradiction, in employment or immigration cases.
  • N1..N6=C outcome:denied: every upstream link held, yet the claimant lost.
  • pattern:CCxC****: the first two links held, the third broke (x = partial or failed), and the fourth held.

Press Enter, click any case to read its chain and the annotator's reasons, and press Save as cohort. The cohort builder does the same with menus, if you prefer. The SQL console is for readers who know databases.

What this evidence can and cannot show

  • One AI reader. Every case was coded by a single AI annotator, with confidence ratings and a devil's-advocate check, but no second human coder yet. The Annotation Audit tool tests whether a result depends on the shakier codings.
  • Appellate opinions only. These are the cases that were appealed and written up, not all legal disputes.
  • Links 7 and 8 mostly reflect who won. Read structure from links 1–6.
  • Association is not causation, except where a tool is built to address it (the Event Study, cautiously).
  • Small numbers. Rare contradiction types rest on very few cases. Every table shows its n, the number of cases behind each figure.

Glossary

Odds ratio (OR)
How much more likely two things are to occur together than apart. 1 means no connection; 3 means strongly connected; 0.3 means they tend to exclude each other.
Risk ratio (RR)
How many times more often something happens in one group than another. 1.5 means 50% more often.
Cramér's V
A 0-to-1 measure of how strongly one classification (say, area of law) predicts another (say, type of failure). Below 0.1 is weak; above 0.3 is strong.
Between-domain spread (τ)
How much a connection's strength varies from one area of law to another. Zero means identical everywhere.
Signature
A case's full structural fingerprint: the state of its eight links plus its active contradiction types.
Isomorph
The same signature found in different areas of law.
Difference-in-differences
How much more an area of law changed after an event than other areas of law changed over the same period.
Placebo date
A date when nothing happened, used to check that a supposed effect is bigger than ordinary year-to-year change.
Discovery / confirmation
The 70% of cases you explore, and the 30% kept aside to check findings once.

To share any result, press Copy link at the top: the link rebuilds exactly what you see. To cite one, use Cite in the Log tab.

N3=F N4!=C N2=PF (any of) · N1..N3=C · pattern:CC?F**** (?/* any, x = partial or failed) · first_nonclosure:3 · ct:RF ct:RC3|CC !ct:JC · ct_count>=3 · domain:D6,D5 · outcome:denied · year:2005..2010 · circuit:9,DC. Tokens are ANDed; use the builder for OR/NOT groups.

Cohorts

A cohort is a saved query: its SQL, the case ids it returned, and the bundle it ran on. Every module takes cohorts. Built-in cohorts mirror the Query Tool's filters.

Pinned cases

Pin up to four cases from their cards to read cross-domain isomorphs side by side.

Session log

Every module run is logged with its recipe, headline estimate, robustness bar and discovery/confirmation status. Kept in this browser (localStorage) and exportable.

What the Workbench is for

The Query Tool answers the questions it was built to answer. The Workbench is for questions nobody has written a tab for yet. Its central question is SOoL's hypothesis itself: does the same kind of legal failure have the same structure across doctrinal domains, independent of doctrine content?

Rules it enforces

  • Definitional vs empirical, always marked. Every association is checked against the entailments table: links fixed by the ontology (a type's MLC link), by the v2 annotation rules (RULE 2/3/4/6 force RC3, NI, JC, CC) or by the disposition construction (RULE 0: N7 tracks the outcome; N8 follows N7). Only what remains can be a finding.
  • Absence of evidence is not invariance. Domain Invariance calls a structure invariant only when the confidence bound on its between-domain spread is below a stated tolerance.
  • Discovery and confirmation. The corpus is split 70/30 within each domain by a fixed seed. Explore on discovery; confirm once on the held-out set.
  • Robustness bar. Headline results are re-estimated on high-confidence annotations only, without rule-forced types, without N7/N8, without review-flagged or adversarially weak cases (adversarial strength ≥ 0.8), and without all-indeterminate chains.
  • Non-closure means partial or failed. Indeterminate means the opinion does not say; it is treated as missing, not as a break.
  • No outcome prediction, no accuracy or AUC.

Data