Research · Rule learning

Learning readable rules from examples

A company often knows how to tell good cases from bad ones without being able to state the rule that separates them: which orders to check, which customers are eligible, who is senior. Inductive logic programming, or ILP, recovers this rule from labelled examples. Unlike a statistical model, the result is a written rule that can be read, discussed and deployed.

How it works

Three things are provided: known facts about the cases, a bias stating which predicates the rule may use and how long it may be, and positive and negative examples. The engine, inspired by the learning-from-failures approach, enumerates candidate rules, tests each one against the examples, and turns every failure into a constraint that rules out whole families of candidates. Finally it assembles the smallest program that covers every positive example and no negative one.

Known facts
Bias
Labelled examples
Tested rule

No language-model call is required. An optional variant can ask the language model to propose a bias or starting rules from a description; the final program is always formally tested against the examples.

Business examples

Order control: which orders should be checked?

Customer service flagged four orders that deserved a check and five that did not. We look for the rule that separates them, from four signals.

[kb set "cmd_verif"]
[facts "cmd_verif"
  "montant_eleve(c1)." "nouveau_client(c1)."
  "montant_eleve(c2)." "nouveau_client(c2)." "carte_etrangere(c2)."
  "carte_etrangere(c3)." "adresse_differente(c3)."
  "carte_etrangere(c4)." "adresse_differente(c4)." "montant_eleve(c4)."
  "montant_eleve(c5)." "nouveau_client(c6)." "carte_etrangere(c7)."
  "adresse_differente(c8)." "nouveau_client(c8)."
  "montant_eleve(c9)." "adresse_differente(c9)."]
$biais = [ilp:bias [JSON '{
  "target": "a_verifier/1",
  "predicates": [
    {"name": "a_verifier",         "modes": ["+commande"], "role": "head", "is_bk": false},
    {"name": "montant_eleve",      "modes": ["+commande"]},
    {"name": "nouveau_client",     "modes": ["+commande"]},
    {"name": "carte_etrangere",    "modes": ["+commande"]},
    {"name": "adresse_differente", "modes": ["+commande"]}],
  "types": {"commande": "symbolic"}, "max_body": 2, "max_clauses": 3}']]
$positifs = [JSON '[["c1"], ["c2"], ["c3"], ["c4"]]']
$negatifs = [JSON '[["c5"], ["c6"], ["c7"], ["c8"], ["c9"]]']
$regle = [ilp:learn $biais "a_verifier" $positifs $negatifs [JSON '[]'] [JSON '{"beam_width": 128, "seed": 7}'] "cmd_verif"]
$regle.program

Result obtained on the server: two rules, “new customer and high amount” or “foreign card and different delivery address”, which cover the four cases to check and none of the other five.

Deploying the learned rule

The learned rule is a tested logic program. It is added as is to a production knowledge base and applies to new orders.

[kb set "cmd_prod"]
[facts "cmd_prod" "montant_eleve(c42)." "nouveau_client(c42)." "carte_etrangere(c43)."]
[for $clause $regle.program [[rules "cmd_prod" $clause]]]
[query VVL "cmd_prod" "a_verifier(c42)?"]      # oui : nouveau client, montant élevé
[query VVL "cmd_prod" "a_verifier(c43)?"]      # non : carte étrangère seule

Result obtained on the server: order c42 must be checked, order c43 need not be.

Human resources: who is senior?

Managers can point out senior profiles. Learning recovers the criterion: experienced and certified. Hugo, certified but without experience, and Carol, experienced but without certification, force the rule to combine both.

[kb set "rh_senior"]
[facts "rh_senior"
  "experimente(alice)." "experimente(carol)." "experimente(eve)." "experimente(grace)."
  "certifie(alice)." "certifie(eve)." "certifie(grace)." "certifie(hugo)."]
$biais = [ilp:bias [JSON '{
  "target": "senior/1",
  "predicates": [
    {"name": "senior",      "modes": ["+personne"], "role": "head", "is_bk": false},
    {"name": "experimente", "modes": ["+personne"]},
    {"name": "certifie",    "modes": ["+personne"]}],
  "types": {"personne": "symbolic"}, "max_body": 2, "max_clauses": 2}']]
$r = [ilp:learn $biais "senior" [JSON '[["alice"], ["eve"], ["grace"]]']
                [JSON '[["carol"], ["hugo"], ["bob"], ["dave"]]']
                [JSON '[]'] [JSON '{"beam_width": 128, "seed": 7}'] "rh_senior"]
[ilp:explain $r]

Result obtained on the server: senior(X) if certifie(X) and experimente(X), coverage of 3 out of 3, status tested.

Auditing an existing rule

A hand-written rule can be checked against the labelled examples before it goes live: how many good cases it covers, and whether it catches bad ones.

$clause = [JSON '["a_verifier(C) :- montant_eleve(C)."]']
[ilp:test $clause "a_verifier" $positifs $negatifs [JSON '[]'] "cmd_verif"]
# { n_pos, covered_pos, covered_neg, complete, consistent }

Result obtained on the server: the “high amount” rule covers 3 of the 4 orders to check but also flags 2 sound orders; it is neither complete nor consistent.

Guiding learning with a description

When the useful predicates are not obvious, a natural-language description can steer the search. The language model suggests leads, and the program retained is still tested against the examples.

$r = [ilp:induce
  "A grandparent is two generations of parent. Use parent/2, male/1, female/1."
  "grandparent" 2
  $pos $neg
  [JSON '{"n_seeds": 5, "max_body": 3, "beam_width": 128, "seed": 42}']
  "ilp_llm_bootstrap"]

What it guarantees, and its limits

The result is a readable rule, tested against every example, with its coverage: not a black box.
The rule retained is the simplest one consistent with the examples. If the examples do not distinguish two criteria, the rule will keep only one: the quality of the examples matters more than their number.
The bias bounds the search; widening it makes learning slower.
A learned rule can be subjected to a validation policy before it goes into production.

Apply this work to your processes?

The diagnosis starts from how you actually work and identifies the decisions that can be entrusted to AI.

Get my assessment