Embabel

DICE Release: 0.2.0

Built against embabel-agent: 1.5.2

© 2024-2026 Embabel Pty, Ltd

Rod Johnson, James Dunnam, Jasper Blues, Thomas Schilling, Alex Hein-Heifetz

1. Overview

DICE — Domain-Integrated Context Engineering — is a knowledge substrate for JVM applications. It turns text into confidence-weighted natural-language propositions, resolves the entity mentions inside them to canonical identifiers, and then projects those propositions into whatever representation a task needs: a Neo4j graph, a Prolog fact base, vector embeddings, agent working memory, or a human-readable report.

Propositions are the single system of record. Everything else is a derived view.

1.1. What is DICE?

DICE extends context engineering by insisting that a domain model is what gives context its shape, and by treating LLM outputs — not just inputs — as things worth structuring.

Despite their seductive ability to work with natural language, LLMs become safer to use the more we add structure to inputs and outputs. DICE helps LLMs converse in the established language of our business and applications.

Domain objects are not mere structs. They not only provide typing, but define focused behaviour. In an agentic system, behaviour can be exposed to manually authored code and selectively exposed to LLMs as tools.

— Rod Johnson
Context Engineering Needs Domain Understanding

1.1.1. Why domain integration matters

Benefit What it buys you

Structured context

Code fills the context window, rather than string concatenation. Less delicate, more repeatable.

System integration

Domain objects are the interface to systems you already run.

Reuse

A domain model captures business understanding once and is shared across agents.

Persistence

Structured query through SQL, Cypher and Prolog — not vector search alone.

Testability

Structure and encapsulation make behaviour assertable.

Observability

Debuggers and tracing tools understand typed objects.

1.1.2. When to reach for DICE

Use DICE when your application needs to accumulate knowledge from text over time and then query it in more than one shape:

  • A chatbot that should remember what a user told it three sessions ago, with a record of where each belief came from and how strongly it is held.

  • A document pipeline that builds a queryable graph of entities and relationships, incrementally, without re-processing what it has already seen.

  • An agent that needs working memory scoped to a conversation, with stale beliefs decaying rather than piling up.

  • Anything that needs to answer "why do you believe that?" with evidence.

Do not reach for DICE when a vector store and a retrieval prompt are enough. The proposition model earns its cost when knowledge is revised — merged, reinforced, contradicted, superseded — not when it is only ever appended.

1.1.3. A worked example in production

Impromptu is a classical-music exploration chatbot that builds a knowledge graph from conversation. It exercises most of the surface described in this guide: PropositionPipeline for extraction, incremental analysis for streaming conversation, an EscalatingEntityResolver with an LLM-backed candidate searcher, and Spring Boot integration with asynchronous processing.

1.2. Architecture

DICE uses a proposition-based architecture inspired by the General User Models (GUM) research from Stanford and Microsoft. Like GUM, it constructs confidence-weighted propositions through a cycle of propose, retrieve and revise.

Natural-language propositions are the system of record. They accumulate evidence over time and project into typed views, each serving a different kind of question.

        Text / Chunks
              |
              v
      LLM extraction  ......... concurrent, resolver-free
              |
              v
    Entity resolution  ........ always serial
              |
              v
  +-------------------------+
  |      PROPOSITIONS       |   the system of record
  | confidence + importance |
  |        + decay          |
  +-------------------------+
     |    |    |    |    |    \
     v    v    v    v    v     v
  Vector Neo4j Prolog Memory Oracle Report
  semantic graph  rules  agent  NL Q&A  rationale
  retrieval traversal    context        + structure

1.2.1. The flow, in one pass

  1. Chunk. Source material arrives as Chunk objects — from dice-ingestion, from a conversation, or built directly by your code.

  2. Extract. A PropositionExtractor (normally LlmPropositionExtractor) turns each chunk into suggested propositions and the entity mentions inside them. This stage touches no resolver and is safe to run concurrently.

  3. Resolve. Each mention is matched against known entities or minted as a new one. This stage is always serial, so entity identity holds across chunks regardless of how extraction was dispatched.

  4. Revise. If a PropositionReviser is configured, new propositions are compared against what is already stored and classified as new, merged, reinforced or contradicted.

  5. Gate. Admission gates run over the result before anything is written.

  6. Persist. The pipeline returns unsaved results. The caller decides the transaction boundary and calls persist(…​). Nothing is stored until it does.

  7. Project. Stored propositions are projected on demand into a graph, a Prolog fact base, agent memory, or a report.

The split at step 6 is deliberate: DICE never opens a transaction on your behalf.

1.2.2. Design notes

This guide describes how to use DICE. For why it behaves as it does — the decisions you cannot recover by reading a single class — the design notes live in docs/design/ in the repository:

  • extraction-pipeline.md — the two-stage extract/resolve flow and why the pipeline returns unsaved results.

  • proposition-lifecycle.md — trust scoring, source authority, supersession versus contradiction, decay instead of deletion.

  • knowledge-hygiene.md — admission gates, mark-and-sweep reclamation, and the consolidation dream loop.

  • consolidation-and-dream-loop.md — the pass abstraction and the four consolidation passes.

  • reclamation-and-collector.md — mark-and-sweep internals, sweep policy, dry-run versus live, and the audit trail.

  • graph-projection.md — edge lineage, projection outcomes, the stale cascade, and idempotent reconciliation.

  • retrieval-and-discovery.md — store-agnostic graph queries, query-time authority filtering, and link discovery.

  • durable-storage.md — backend selection, defence-in-depth dedup, two-phase save, and the decay tick.

  • events.md — the domain-event model the store and pipeline emit.

1.3. Modules

DICE publishes five consumer-facing artifacts under the com.embabel.dice group. There is no aggregator starter: pick the modules you need.

Artifact What it gives you Pull it in when

dice

The whole domain: the Proposition model, the PropositionStore and PropositionRepository SPIs, the extraction pipeline, revision and conflict detection, entity resolution, the graph, Prolog and memory projectors, graph and discovery queries, incremental analysis, the in-memory and JSON-file stores, tuProlog integration, and the optional REST endpoints.

Always. Everything else depends on it.

dice-storage

A Drivine/Neo4j implementation of PropositionRepository, ChunkHistoryStore and DecayManager.

You want propositions in Neo4j rather than in memory.

dice-storage-autoconfigure

Spring Boot auto-configuration: backend selection from embabel.dice.store.type, the scheduled decay tick, and the multi-signal duplicate collector.

You are on Spring Boot and want beans wired for you. This is the usual entry point.

dice-report

Output projectors over propositions: rationale (why a fact is believed, with its evidence), structured reports, and surprising-link discovery.

You need human-readable output.

dice-ingestion

An ingestion SPI turning artifacts into chunks, with a content-hash ledger so the same source is not extracted twice.

You are feeding files or documents in, rather than conversation turns.

dice-integration-tests is test-only: it holds the cross-feature end-to-end harness and is not published for consumption.

Dependency scope

embabel-agent-api and embabel-agent-rag-core are declared provided in dice. Your application brings the actual embabel-agent version at runtime; DICE only fixes the minimum it compiles against.

1.4. Glossary

The rest of this guide assumes these terms.

Proposition

A single natural-language claim, with a confidence, an importance, a decay factor, a status, a ContextId, the entity mentions it contains, and links back to the source material it came from. The system of record.

ContextId

The scope every proposition belongs to — a user, a tenant, a conversation, a document set. It is a Kotlin value class; Java callers use getContextIdValue(). Every query starts from a ContextId by design, so nothing accidentally loads the whole store.

Confidence and effective confidence

confidence is what the LLM assigned at extraction time and never changes on its own. effectiveConfidence() folds in decay and is what you rank and filter on.

Decay

A 0–1 factor that erodes a proposition’s effective confidence as it ages, anchored on contentRevised. Administrative changes — status, pinning, grounding — touch metadataRevised only and never reset the clock. Decay replaces deletion: a belief fades rather than vanishing.

Mention

A span of text in a proposition that names an entity, before or after it has been matched to a canonical identifier.

Entity resolution

Matching a mention to an existing entity or minting a new one. The shipped EscalatingEntityResolver climbs a ladder — exact match, heuristic, embedding, LLM verification, LLM bake-off — and stops at the cheapest level that succeeds.

Revision

Comparing a newly extracted proposition against what is already stored, and classifying it as new, merged, reinforced or contradicted.

Projection

A derived, typed view over propositions: a graph, a Prolog fact base, agent memory, a report. A projection is always rebuildable from the propositions.

Grounding

The link from a proposition back to the chunks it was extracted from, so "why do you believe this?" has an answer.

Provenance

Richer evidence links than grounding: a ProvenanceEntry records where a claim came from and, through SourceLocator, how to get back to it.

Admission gate

A check that runs on pipeline output before the caller persists it — a confidence floor, a trust threshold, a projection-eligibility rule.

Reclamation (the collector)

The mark-and-sweep pass that finds duplicate or stale propositions and decides each one’s fate, with an audit trail.

Consolidation (the dream loop)

A composed set of passes — session consolidation, abstraction, contradiction resolution, decay sweep — that runs periodically to keep a store coherent.

Authority

A tier assigned to a source, used by AuthorityWeightedTrustScorer to weight what a claim from that source is worth, and available as a query-time filter.

2. Quickstart

One path, from an empty Spring Boot application to a printed report. No Neo4j, no Docker, nothing external to create. Every decision that could be made takes the default and links out.

At the end you will have extracted propositions from a paragraph of text, persisted them, queried them back, and rendered a report.

2.1. Dependencies

DICE publishes no aggregator starter, so the quickstart uses two coordinates: the auto-configuration, which pulls dice and dice-storage transitively, and the report projectors.

<dependency>
    <!-- Spring Boot auto-configuration: store backend, decay tick, collector. -->
    <groupId>com.embabel.dice</groupId>
    <artifactId>dice-storage-autoconfigure</artifactId>
    <version>0.2.0</version>
</dependency>

<dependency>
    <!-- Rationale, structured reports, link discovery. -->
    <groupId>com.embabel.dice</groupId>
    <artifactId>dice-report</artifactId>
    <version>0.2.0</version>
</dependency>

You also need an embabel-agent runtime on the classpath. DICE declares embabel-agent-api and embabel-agent-rag-core as provided, so the version your application brings is the version that runs.

Snapshots and releases resolve from Embabel’s Artifactory:

<repositories>
    <repository>
        <id>embabel-snapshots</id>
        <url>https://repo.embabel.com/artifactory/libs-snapshot</url>
        <releases><enabled>false</enabled></releases>
        <snapshots><enabled>true</enabled></snapshots>
    </repository>
</repositories>

2.2. Configuration

The only thing you must configure is an LLM provider, and that is inherited from embabel-agent — set whatever API key your chosen provider expects. DICE’s own defaults need nothing:

embabel:
  dice:
    store:
      type: in-memory     # the default; stated here to be explicit

2.3. What auto-configuration gives you

With dice-storage-autoconfigure on the classpath and type left at in-memory, the following beans exist, each @ConditionalOnMissingBean so your own bean always wins:

Bean What it is for

PropositionRepository

InMemoryPropositionRepository, backed by the Ai bean for embeddings. Where propositions live.

ChunkHistoryStore

Remembers which chunks have already been processed, so incremental analysis does not redo work.

ProjectionRecordStore

Lineage for projected edges, so a projection can be reconciled rather than rebuilt.

CollectorRecordStore

The audit trail for reclamation runs.

DecayManager + the scheduled decay tick

Materialises effective confidence and applies lifecycle transitions, hourly by default.

Switching type to graph swaps the first four for their Drivine/Neo4j equivalents and changes nothing else in your code. See Graph-backed storage.

2.4. Extract

Build a pipeline from an extractor, then run a chunk through it. The pipeline is immutable — every with* method returns a new instance.

@Bean
PropositionPipeline propositionPipeline(PropositionExtractor extractor) {
    return PropositionPipeline.withExtractor(extractor);
}

@Bean
LlmPropositionExtractor llmPropositionExtractor(Ai ai, PropositionRepository repository) {
    return LlmPropositionExtractor
            .withLlm(llmOptions)
            .withAi(ai)
            .withPropositionRepository(repository)
            .withSchemaAdherence(SchemaAdherence.DEFAULT);
}

Then extract:

var context = new SourceAnalysisContext(
        dataDictionary,                                 // your domain schema
        EscalatingEntityResolver.create(entityRepo),    // how mentions become entities
        new ContextId("quickstart"));                   // the scope everything lands in

var chunk = new Chunk("chunk-1",
        "Ada Lovelace worked with Charles Babbage on the Analytical Engine.");

var results = pipeline.process(List.of(chunk), context);

results.getPropositions().forEach(p ->
        System.out.printf("%.2f  %s%n", p.getConfidence(), p.getText()));

You should see one or more claims printed with their confidence, something like:

0.92  Ada Lovelace worked with Charles Babbage.
0.88  Ada Lovelace worked on the Analytical Engine.

2.5. Persist

The pipeline stored nothing. It returned PersistablePropositions, and if you drop that on the floor the work is discarded. Persist inside your own transaction:

@Transactional
public void store(PersistablePropositions results) {
    results.persist(propositionRepository, namedEntityDataRepository);
}

persist saves only the entities actually referenced by the propositions being saved, saves the propositions themselves — the revised set if revision was enabled, the raw set otherwise — and creates the structural relationships linking chunks, propositions and entities.

2.6. Query

Queries always start from a ContextId. There is no PropositionQuery.create(), deliberately: it would make "load everything" a one-liner.

// Everything in this context.
var all = propositionRepository.query(
        PropositionQuery.againstContext(new ContextId("quickstart")));

// Only what is still strongly believed, best first.
var confident = propositionRepository.query(
        PropositionQuery.againstContext(new ContextId("quickstart"))
                .withMinEffectiveConfidence(0.7));

// Everything mentioning one entity.
var aboutAda = propositionRepository.query(
        PropositionQuery.againstContext(new ContextId("quickstart"))
                .withEntityId(adaId));

InMemoryPropositionRepository needs an EmbeddingService for vector search. The minimal path here queries by entity and ContextId only, which needs no embeddings at query time.

2.7. Report

Turn what you stored into something a person can read:

var report = reportProjector.report(confident, "What we know about Ada Lovelace");
System.out.println(report.summary());

2.8. Where to go next

  • Concepts — read the five concept pages in order. They define the vocabulary the rest of the guide uses.

  • How-to — task-shaped pages: extracting from documents, tuning resolution, querying, projecting to a graph, agent memory.

  • Features — the opt-in surfaces, each opening with the property or bean that switches it on.

  • Running in production — what changes when this stops being a quickstart.

3. Concepts

Five pages, meant to be read in order. Each uses terms the page before it defines.

3.1. Propositions

A Proposition is one claim, written in natural language, that DICE believes to some degree. Everything else in DICE is derived from propositions, which means a proposition is the only thing you must get right.

3.1.1. What a proposition carries

Field Meaning

text

The claim itself, in natural language. This is the payload.

contextId

The scope it belongs to. See Context and schema.

confidence

What the extractor assigned when the claim was made. Never changes on its own.

decay

A 0–1 erosion factor applied as the claim ages.

status

ACTIVE and its siblings — see below.

pinned

A pinned proposition is exempt from decay-driven reclamation.

level

0 for a directly extracted claim; higher for abstractions built from other propositions.

reinforceCount

How many times independent evidence has restated this claim.

mentions

The entity mentions in the text, resolved or not.

sourceIds / grounding

The chunks this claim came from.

provenanceEntries

Richer evidence links, each locatable through a SourceLocator.

temporal

Optional validity window, for claims that were true only for a period.

created, contentRevised, metadataRevised, lastAccessed

Four separate clocks. The distinction matters — see below.

3.1.2. Effective confidence is the number you use

confidence is a historical fact: it is what the LLM thought at extraction time. It is not what you should rank or filter on.

double now   = proposition.effectiveConfidence();          // decay applied, k = 2.0
double later = proposition.effectiveConfidenceAt(instant); // as of some other time

PropositionQuery exposes both: withMinConfidence(…​) filters the raw number, and withMinEffectiveConfidence(…​) filters the decayed one. Reach for the effective form unless you have a specific reason not to.

The decay rate multiplier k defaults to 2.0 and is configurable per query and, for the scheduled tick, through embabel.dice.store.decay.k.

3.1.3. The four clocks

contentRevised is the decay anchor. It moves only when the claim itself changes.

Administrative changes — flipping status, pinning, attaching grounding — touch metadataRevised and leave contentRevised alone. This is what stops housekeeping from silently making stale beliefs look fresh. lastAccessed is separate again, and touchAccessed(…​) on the store updates it in bulk without disturbing either revision clock.

3.1.4. Status

PropositionStatus tracks a claim’s standing. ACTIVE is the normal case; the other states record that a claim has been superseded, contradicted, or has decayed past usefulness. Transitions are governed by a StatusTransitionPolicy, which is an SPI you can replace.

DICE decays rather than deletes. A claim that stops being believed keeps its history, its evidence, and its links, so "we used to think X, then Y arrived" remains answerable. Hard deletion is a separate, opt-in sweep — see Knowledge hygiene.

3.1.5. Evidence accumulation

When the same claim arrives again from a different source, DICE does not store a duplicate. It folds the new evidence in:

Proposition merged = existing.absorbEvidence(incoming);

absorbEvidence combines confidence, grounding, provenance and reinforceCount. The inverse, withoutFoldedEvidence(…​), exists so a retraction can be applied without losing the rest.

3.1.6. Immutability

Proposition is a Kotlin data class and every mutator returns a copy: withText, withStatus, withConfidence, withPinned, withGrounding, withTemporal, withProvenance, withResolvedMentions, withMetadataValue. Nothing mutates in place, so a proposition can be shared across threads without ceremony.

Java callers get a Proposition.create(…​) factory and getContextIdValue() for the value-class ContextId.

Try it now
var p = propositionRepository.query(
        PropositionQuery.againstContext(new ContextId("quickstart"))).get(0);

System.out.printf("raw=%.2f effective=%.2f decay=%.2f reinforced=%d status=%s%n",
        p.getConfidence(), p.effectiveConfidence(), p.getDecay(),
        p.getReinforceCount(), p.getStatus());

Expect raw and effective confidence to be close on a freshly extracted claim, and to diverge as the decay tick runs against it.

3.2. Entity resolution

Extraction produces mentions — spans of text that name something. Resolution decides whether a mention refers to an entity you already know, or to one that should be minted.

Getting this wrong is the most expensive failure mode in a knowledge substrate: an unresolved duplicate splits everything you know about one thing into two half-graphs, and a wrong merge welds two things into one.

3.2.1. The escalating resolver

EscalatingEntityResolver climbs a ladder and stops at the cheapest rung that works. The rung it stopped at is reported back as a ResolutionLevel:

Level How it matches LLM call

EXACT_MATCH

Exact name match against the repository.

No

HEURISTIC_MATCH

Normalised name, then fuzzy and partial name matching.

No

EMBEDDING_MATCH

High-confidence embedding similarity.

No

LLM_VERIFICATION

One candidate, verified yes/no by an LLM.

Yes

LLM_BAKEOFF

Several candidates, an LLM picks the best.

Yes

NO_MATCH

Nothing matched at any level.

Each attempt returns a LevelResult carrying the level, the resolution, a confidence, and how many candidates were considered — which is what you look at when tuning.

Construct one with the shipped searcher chain:

// Full chain, including the vector searcher.
var resolver = EscalatingEntityResolver.create(entityRepository, candidateBakeoff);

// Same, minus the vector searcher — for stores with no vector index.
var resolver = EscalatingEntityResolver.withoutVector(entityRepository, candidateBakeoff);

Passing null for the bake-off means ambiguous cases mint a new entity rather than asking an LLM.

3.2.2. The searchers

Candidate recall is pluggable. CandidateSearcher implementations ship for exact name, normalised name, partial name, fuzzy name, id, vector similarity, and AgenticCandidateSearcher, which lets an LLM go looking. DefaultCandidateSearchers assembles the standard chains.

To add a strategy, implement CandidateSearcher and compose it into the resolver’s chain.

3.2.3. Composing resolvers

Resolvers compose, and the order is the policy:

Resolver Behaviour

KnownEntityResolver

Resolves against entities you handed in on the context — the current user, say. Put it first.

EscalatingEntityResolver

The ladder above, against the repository.

InMemoryEntityResolver

Remembers what this run has already resolved, so chunk 7 recognises what chunk 2 minted.

ChainedEntityResolver

Tries each resolver in order and takes the first hit.

AlwaysCreateEntityResolver

Mints unconditionally. Useful in tests and in throwaway contexts.

PropositionPipeline.process(…​) wraps your resolver with an InMemoryEntityResolver automatically — but only when mintNewEntities is on. With minting off the wrapper is skipped deliberately: remembering an entity that will never be created would produce mentions resolved to ids that do not exist.

3.2.4. Why resolution is serial

Extraction is stateless per chunk and can be dispatched concurrently. Resolution is not: it reads and writes shared identity state, and running it concurrently is how you get two entities for one thing. DICE keeps resolution serial regardless of the ExtractionExecutionStrategy in force, so cross-chunk identity holds however extraction was dispatched. See Concurrent extraction.

3.2.5. Mention filtering

Not every mention deserves resolution. A MentionFilter runs before resolution and drops low-quality spans — vague references, overly long spans, anything the schema says cannot be an entity. Filtering here is cheaper than resolving and then reclaiming.

Shipped filters include SchemaValidatedMentionFilter, the context-aware filters in ContextAwareMentionFilters, and ObservableMentionFilter, which wraps another filter to report what it rejected.

pipeline = pipeline.withMentionFilter(mentionFilter);
Try it now
var chunk = new Chunk("c1",
        "Ada Lovelace met Babbage in 1833. Lovelace later wrote the first algorithm.");
var results = pipeline.process(List.of(chunk), context);

results.getPropositions().stream()
        .flatMap(p -> p.getMentions().stream())
        .forEach(m -> System.out.println(m.getText() + " -> " + m.getResolvedId()));

"Ada Lovelace" and "Lovelace" should resolve to the same id. If they do not, that is the tuning problem Tune entity resolution exists for.

3.3. Storage and projections

Propositions are stored once and viewed many ways. This page covers the port they are stored through, and the views derived from them.

3.3.1. The store SPI

PropositionStore is the base port and is deliberately small: save, find by id, find by entity, find by status, find by grounding, find by context, delete, count — plus one composable query.

PropositionRepository extends it with optional capability interfaces. A backend implements only what it genuinely supports:

Capability What it adds

VectorSearchCapable

Similarity search over proposition embeddings.

GraphTraversalCapable

Traversal over the proposition/entity graph.

GraphQueryCapable

Native neighbourhood, path and lineage queries, plus the honorsAuthorityFilter opt-in that lets the portable graph facade push authority-filtered traversals down to the backend.

TemporalQueryCapable

Queries over validity windows.

Each capability interface carries default implementations expressed over the base primitives, so a backend that implements nothing extra still behaves correctly — just more slowly.

3.3.2. The shipped backends

Backend Where it lives Use it for

InMemoryPropositionRepository

dice

Development, tests, and single-process applications. Needs an EmbeddingService for vector search.

JsonFilePropositionRepository

dice

A store that survives a restart without any infrastructure.

Neo4jRagPropositionRepository

dice

Propositions inside an embabel-agent RAG-backed Neo4j.

DrivinePropositionRepository

dice-storage

Production Neo4j through Drivine, with the vector index, decay in the database, and lineage.

Selection is a property, not a code change — see Graph-backed storage.

3.3.3. Composable queries

PropositionQuery is a data class assembled through with* methods. The axes:

  • Scope: contextId, entityId, anyEntityIds, allEntityIds

  • Lifecycle: statuses, pinned, minLevel, maxLevel

  • Time: createdAfter/Before, revisedAfter/Before, accessedAfter/Before

  • Belief: minConfidence, minEffectiveConfidence (with effectiveConfidenceAsOf and decayK), belowEffectiveConfidence, minImportance, minReinforceCount, minTrustScore

  • Shape: orderBy, limit

var stale = PropositionQuery.againstContext(contextId)
        .withBelowEffectiveConfidence(0.2)
        .withStatus(PropositionStatus.ACTIVE)
        .withLimit(100);

Kotlin infix factories have Java-friendly withXxx and againstContext equivalents throughout.

3.3.4. Projections

A projection is a rebuildable, typed view. Nothing in a projection is authoritative — if it and the propositions disagree, the propositions win.

Projection What it produces

Graph (GraphProjector)

A typed entity-relationship graph. RelationBasedGraphProjector projects from declared Relations; LlmGraphProjector infers edges. Edges carry lineage, so a changed source cascades a stale marking rather than silently leaving a wrong edge in place, and reconciliation is idempotent.

Prolog (PrologProjector)

A tuProlog fact base you can run rules against. Experimental — see Prolog inference.

Memory (MemoryProjector)

Propositions sorted into semantic, episodic, procedural and working buckets, ready to drop into an agent’s context. See Give an agent memory.

Report (ReportProjector, RationaleProjector)

Human-readable output, including "why is this believed" with the evidence. In dice-report.

EventEmittingProjector and EventEmittingPropositionRepository are decorators that publish Spring application events on projection and on save. Wire them in when you want something to react.

3.3.5. Retrieval

RetrievalRouter is the single entry point for retrieval, choosing a RetrievalMode rather than making the caller pick between vector search, graph traversal and keyword overlap. Authority filtering is applied at query time, and pushed down to the backend where the backend says it can honour it.

Try it now
System.out.println(propositionRepository.getClass().getSimpleName());
System.out.println("vector:   " + (propositionRepository instanceof VectorSearchCapable));
System.out.println("graph:    " + (propositionRepository instanceof GraphQueryCapable));
System.out.println("temporal: " + (propositionRepository instanceof TemporalQueryCapable));

On the quickstart’s in-memory default this tells you exactly which capabilities you have before you write a query that needs one.

3.4. Context and schema

Two things bound what DICE extracts and what it will return: the ContextId that scopes a proposition, and the schema that tells extraction what the domain looks like.

3.4.1. ContextId

Every proposition belongs to exactly one ContextId. It is the primary scope, and the starting point of every query:

var query = PropositionQuery.againstContext(new ContextId("user:42"));

There is no PropositionQuery.create(). That is a design decision, not an omission: without a scope, "load everything" would be the shortest thing to write, and in a store that accumulates indefinitely that is a production incident waiting to happen.

ContextId is a Kotlin value class, so it costs nothing at runtime. Java callers reach the string through getContextIdValue(), on both Proposition and PropositionQuery.

What a context should represent is your decision. Common choices: one per user, one per tenant, one per conversation, one per document collection. Contexts do not nest; if you need a hierarchy, encode it in the value and query by prefix at your own layer.

3.4.2. SourceAnalysisContext

SourceAnalysisContext is the per-run configuration handed to the pipeline. It carries the schema, the resolver, the ContextId, and everything else extraction needs to know:

Field Purpose

schema

The DataDictionary describing your domain’s entity types.

entityResolver

How mentions become entities. See Entity resolution.

contextId

The scope everything extracted lands in.

knownEntities

Entities the run should already recognise — KnownEntity.asCurrentUser(user) is the common case.

relations

Declared predicates, used by extraction and by relation-based graph projection.

promptVariables

Extra values available to the extraction template.

sourceLocator

How to get back to the source material, for provenance.

perspective

Whose point of view the text is written from.

mintNewEntities

Whether unresolved mentions may create entities. Off by default.

mintedEntityProperties

Properties stamped onto anything minted.

It is a data class with with* builders, so a base context can be specialised per run:

var context = new SourceAnalysisContext(dataDictionary, resolver, contextId)
        .withKnownEntities(KnownEntity.asCurrentUser(user))
        .withRelations(relations)
        .withMintNewEntities(true)
        .withSourceLocator(sourceLocator);

3.4.3. The DataDictionary and SchemaAdherence

The DataDictionary is your domain’s schema: the entity types extraction should look for, and the relationship metadata that names the predicates it may use. SchemaRegistry holds them.

SchemaAdherence decides how strictly extraction obeys it. Two independent switches:

Constant entities predicates Behaviour

SchemaAdherence.STRICT

true

true

Only schema entity types, only schema predicates. Nothing outside the model gets in.

SchemaAdherence.DEFAULT

true

false

Entity types locked to the schema; any predicate allowed. The usual choice.

SchemaAdherence.RELAXED

false

false

Schema types preferred but not required, on both axes. Use when you are still discovering the domain.

Set it on the extractor:

LlmPropositionExtractor.withLlm(llmOptions)
        .withAi(ai)
        .withPropositionRepository(repository)
        .withSchemaAdherence(SchemaAdherence.STRICT);

Strictness is the main lever on extraction noise. Start at DEFAULT; move to STRICT when you trust the schema and want the extractor to stop inventing predicates; move to RELAXED only while exploring.

3.4.4. Relations

Relations declares the predicates that may hold between entities. Extraction uses them to phrase claims consistently, and RelationBasedGraphProjector uses them to decide which edges to create. Declaring relations is what turns free-text claims into a graph you can traverse rather than a bag of sentences.

Try it now
var strict = new SourceAnalysisContext(dataDictionary, resolver, new ContextId("strict-test"));
// ... run the same chunk through an extractor built with STRICT and with RELAXED,
// and compare how many propositions and mentions survive.

The gap between the two is the size of the noise your schema is currently suppressing.

3.5. Knowledge hygiene

A store that only ever accumulates gets worse over time. DICE has three interventions, at three different points in the life of a claim: gates keep bad claims out, reclamation removes claims that stopped earning their place, and consolidation rewrites what is left into something more useful.

3.5.1. 1. Admission gates

Gates run on pipeline output before the caller persists anything. They are the cheapest intervention, because nothing has been written yet.

Gate What it rejects

ConfidenceGate

Claims below a raw confidence floor.

EvidenceFloorGate

Claims with too little evidence behind them.

TrustGate

Claims whose trust score — source authority folded into confidence — is too low.

MergeCandidateGate

Routes claims that duplicate something already stored, rather than admitting a second copy.

ConflictClassificationGate

Routes claims that contradict something already stored.

ProjectionEligibilityGate

Admits the claim but marks it as not yet worth projecting.

Compose them with ExtractionGatePipeline, and wrap any of them in ObservableGate to see what is being rejected and why. A gate you cannot observe is a gate you will eventually mis-tune.

3.5.2. 2. Reclamation — the collector

Reclamation is mark-and-sweep over what is already stored. CollectorRunner walks the store, a CollectorStrategy marks candidates, and a SweepPolicy decides each marked proposition’s fate.

The vocabulary is small and worth knowing: a PropositionMark carries a MarkReason, and the policy produces a SweepAction.

Two shipped strategies:

  • DecayCollectorStrategy — marks claims whose effective confidence has fallen far enough.

  • DuplicateCollectorStrategy — marks claims that say the same thing as another.

The MultiSignalCollectorStrategy is the production duplicate matcher, blending several signals rather than trusting any one of them. See Multi-signal collector.

Two things make reclamation safe to run: it has a dry-run mode that reports what it would do without doing it, and every live run is recorded to a CollectorRecordStore, so a surprising result is explainable after the fact. Run it dry first. Always.

Pinned propositions are exempt.

3.5.3. 3. Consolidation — the dream loop

Consolidation rewrites what survives into something more useful. DreamLoopOrchestrator composes passes, and DefaultDreamLoopOrchestrator gates each pass on a threshold, so a quiet store does no work.

The four shipped passes:

Pass What it does

SessionConsolidationPass

Folds a session’s episodic detail into durable claims.

AbstractionPass

Builds higher-level propositions that generalise groups of level-0 claims.

ContradictionResolutionPass

Resolves claims that contradict one another, adjusting status rather than deleting.

DecaySweepPass

Applies decay and the resulting lifecycle transitions.

A run produces a DreamLoopReport describing what each pass did.

DefaultMemoryMaintenanceOrchestrator is the older four-step pipeline. Prefer the dream loop for new work.

3.5.4. Decay, not deletion

Underneath all three sits the same principle: DICE fades beliefs rather than erasing them. Decay lowers effective confidence, status records that a claim was superseded or contradicted, and the evidence stays attached. Hard deletion of STALE propositions is available but off by default — embabel.dice.store.decay.prune-stale is false — because most applications would rather answer "why did you stop believing that?" than reclaim the rows.

Try it now
// See what reclamation would do, without doing it.
var dryRun = collectorRunner.run(contextId, /* dryRun = */ true);
System.out.println(dryRun);

Read the marks and their reasons before you ever run this live.

4. How-to

Task-shaped pages, titled by what you want rather than by what the code is called. Each states its prerequisites first.

4.1. Extract knowledge from documents

Prerequisites: dice, dice-ingestion, a configured LLM, and a PropositionRepository.

4.1.1. Ingest, then extract

dice-ingestion turns artifacts into chunks and keeps a ledger so the same source is not extracted twice.

Type Role

IngestionHandler

Turns one artifact into chunks. TextIngestionHandler ships; implement the SPI for other formats.

IngestedArtifact

One thing that was ingested, with its content hash.

IngestionBatch

A set of artifacts ingested together.

IngestionLedger

The content-hash record that makes re-ingestion a no-op.

IngestionResult

What came out: the chunks, and what was skipped as already seen.

The ledger keys on content, not on filename or path. Renaming a file does not cause re-extraction; editing one does.

4.1.2. Run the chunks through the pipeline

var result = ingestionHandler.handle(artifact);

var context = new SourceAnalysisContext(dataDictionary, resolver, contextId)
        .withSourceLocator(sourceLocator)   // so provenance can point back at the file
        .withMintNewEntities(true);

var propositions = pipeline.process(result.getChunks(), context);

process handles cross-chunk entity identity for you: when minting is enabled it wraps your resolver so an entity minted in chunk 2 is recognised in chunk 7.

4.1.3. Persist within your own transaction

The pipeline stores nothing. That is deliberate — DICE never decides your transaction boundary.

@Transactional
public void ingest(Path file) {
    var result = ingestionHandler.handle(artifactFor(file));
    var propositions = pipeline.process(result.getChunks(), context);
    propositions.persist(propositionRepository, namedEntityDataRepository);
}

If you forget persist, nothing is saved and nothing complains.

4.1.4. Only process what changed

For a corpus that grows, use incremental analysis rather than reprocessing. ChunkHistoryStore records which chunks have been analysed; AbstractIncrementalAnalyzer and its subclasses skip those. dice-storage-autoconfigure provides a ChunkHistoryStore bean for whichever backend is selected.

4.1.5. Add grounding and provenance

Grounding links a claim to the chunk it came from and is created automatically. Provenance is richer: a ProvenanceEntry plus a SourceLocator records how to get back to the original material, which is what RationaleProjector uses to explain a belief.

var context = baseContext.withSourceLocator(sourceLocator);

4.1.6. Filter mentions before resolving

Document text carries far more noise than conversation does — headers, boilerplate, citations. A MentionFilter drops weak mentions before the expensive resolution step:

pipeline = pipeline.withMentionFilter(
        new ObservableMentionFilter(new SchemaValidatedMentionFilter(dataDictionary)));

Wrapping in ObservableMentionFilter while you tune tells you what is being dropped.

4.1.7. Speed it up

Extraction is the slow stage and is stateless per chunk, so it can be dispatched concurrently. Resolution stays serial regardless. See Concurrent extraction — including the thread-safety condition you must verify before turning it on.

4.2. Analyse a conversation incrementally

Prerequisites: dice, a configured LLM, a PropositionRepository and a ChunkHistoryStore.

A conversation is not a document. It arrives a turn at a time, the same claim gets restated and revised, and the user themselves is an entity the extractor should already know about.

4.2.1. The incremental analyser

IncrementalAnalyzer processes an IncrementalSource and remembers what it has already seen. ConversationSource wraps a conversation; ConversationSegmenter decides where one unit of analysis ends and the next begins, so a claim spanning three turns is not cut in half.

var source = new ConversationSource(conversation);
var result = analyzer.analyze(source, context);
result.persist(propositionRepository, entityRepository);

Re-analysing the same conversation is cheap: ChunkHistoryStore reports what has already been processed and the analyser skips it.

4.2.2. Tell the context who is talking

Without this, "I prefer the early recordings" produces a claim about nobody.

var context = SourceAnalysisContext
        .withContextId(user.currentContext())
        .withEntityResolver(entityResolverForUser(user))
        .withSchema(dataDictionary)
        .withRelations(relations)
        .withKnownEntities(KnownEntity.asCurrentUser(user));

KnownEntity.asCurrentUser(…​) is the important line: it gives the resolver a first-class identity for the speaker, so first-person claims attach to them.

4.2.3. Drive it from events

Conversation analysis costs an LLM call, so it should not sit in the request path. ConversationAnalysisRequestEvent exists for exactly this:

@Async
@Transactional
@EventListener
public void onConversationExchange(ConversationAnalysisRequestEvent event) {
    var context = SourceAnalysisContext
            .withContextId(event.user.currentContext())
            .withEntityResolver(entityResolverForUser(event.user))
            .withSchema(dataDictionary)
            .withRelations(relations)
            .withKnownEntities(KnownEntity.asCurrentUser(event.user));

    var source = new ConversationSource(event.conversation);
    var result = analyzer.analyze(source, context);

    result.persist(propositionRepository, entityRepository);
}

SourceAnalysisRequestEvent is the general form for non-conversation sources.

4.2.4. Turn on revision

In a conversation the same thing gets said repeatedly, and sometimes retracted. Without revision you accumulate near-duplicates; with it, restatements reinforce and reversals are classified as contradictions.

var pipeline = PropositionPipeline
        .withExtractor(extractor)
        .withRevision(reviser, propositionRepository);

Revision compares against what is already stored and classifies each new claim as new, merged, reinforced or contradicted. It still persists nothing — hasRevision tells you which set propositionsToPersist() will return.

4.2.5. Scope per user

Give each user their own ContextId. It is the boundary that keeps one user’s beliefs out of another’s retrieval, and it is what makes "forget everything about me" a bounded operation.

4.3. Tune entity resolution

Prerequisites: Entity resolution read, and a corpus you can re-run.

Resolution has two failure modes, and they pull in opposite directions. Under-merging splits one real thing into several entities, so nothing you know about it is ever retrieved together. Over-merging welds two real things into one, and that is much harder to notice and much harder to undo. Tune towards under-merging: a duplicate can be collected later, a bad merge cannot be cleanly unpicked.

4.3.1. Find out where resolution is happening

LevelResult reports the ResolutionLevel that produced each answer, and how many candidates were considered. That distribution is the diagnostic:

  • Mostly EXACT_MATCH — healthy, and free.

  • Heavy LLM_BAKEOFF — expensive. Your cheap searchers are not recalling the right candidates.

  • Heavy NO_MATCH with minting on — you are creating duplicates. Recall is too narrow.

  • Bake-offs with many candidates — recall is too wide, and you are paying an LLM to sort it out.

4.3.2. Fix recall before touching the LLM rungs

The cheap searchers decide what the expensive ones ever see.

Searcher When it earns its place

ByExactNameCandidateSearcher

Always. Free, and catches the majority case.

NormalizedNameCandidateSearcher

Casing, punctuation and whitespace variation. Almost always worth it.

PartialNameCandidateSearcher

"Lovelace" for "Ada Lovelace". Necessary for conversation, noisy for catalogues.

FuzzyNameCandidateSearcher

Typos and transliteration. Widens recall; watch the bake-off rate after enabling.

ByIdCandidateSearcher

When your source text carries real identifiers.

VectorCandidateSearcher

Aliases that share no characters. Needs a vector index.

AgenticCandidateSearcher

An LLM goes looking. The most capable and the most expensive — last resort.

DefaultCandidateSearchers.create(repository) assembles the standard chain; DefaultCandidateSearchers.withoutVector(repository) drops the vector searcher for stores with no index.

4.3.3. Decide what happens when nothing matches

// Ambiguity goes to an LLM.
var resolver = EscalatingEntityResolver.create(entityRepository, llmCandidateBakeoff);

// Ambiguity mints a new entity instead.
var resolver = EscalatingEntityResolver.create(entityRepository, null);

And separately, whether minting is allowed at all:

context = context.withMintNewEntities(false);   // unresolved mentions are vetoed, not minted

Minting off is the right default for a closed catalogue. Minting on is necessary for open-domain extraction, and is what makes mention filtering matter.

4.3.4. Put known entities first

ChainedEntityResolver tries resolvers in order and takes the first hit, so order is policy:

var resolver = new ChainedEntityResolver(List.of(
        new KnownEntityResolver(knownEntities),                     // the user, the tenant
        EscalatingEntityResolver.create(entityRepository, bakeoff)   // everything else
));

Do not add InMemoryEntityResolver by hand for a process(…​) run — the pipeline adds it when minting is on, and adding it when minting is off produces mentions resolved to ids that will never exist.

4.3.5. Reduce what reaches the resolver

Every mention that should never have been a mention is a chance to resolve wrongly. A MentionFilter is cheaper than the resolution it prevents:

pipeline = pipeline.withMentionFilter(
        new ObservableMentionFilter(new SchemaValidatedMentionFilter(dataDictionary)));

Tightening SchemaAdherence to STRICT has the same effect earlier still, at the extractor. See Context and schema.

4.3.6. Clean up what got through

Some duplication is unavoidable, and reclamation exists to absorb it. The MultiSignalCollectorStrategy blends vector, lexical, entity-overlap, grounding-overlap and provenance-overlap signals, with a polarity veto that stops a claim merging with its own negation. Tune embabel.dice.collector.match-threshold and the per-signal weights rather than loosening resolution — see Multi-signal collector.

Run it dry first, and read the trace.

4.4. Query propositions

Prerequisites: a populated PropositionRepository.

Every query starts from a ContextId. PropositionQuery has no create() factory, so there is no short way to write "load everything".

var base = PropositionQuery.againstContext(new ContextId("user:42"));
// or, equivalently
var base = PropositionQuery.forContextId(new ContextId("user:42"));

4.4.1. By belief

Filter on effective confidence, not raw confidence, unless you specifically want the extractor’s original opinion:

// What we currently believe.
repository.query(base.withMinEffectiveConfidence(0.7));

// What we believed at some past moment.
repository.query(base.withMinEffectiveConfidence(0.7)
                     .withEffectiveConfidenceAsOf(lastMonth));

// What has faded — reclamation candidates.
repository.query(base.withBelowEffectiveConfidence(0.2));

// Repeatedly corroborated claims only.
repository.query(base.withMinReinforceCount(3));

// Weighted by source authority.
repository.query(base.withMinTrustScore(0.6));

4.4.2. By entity

repository.query(base.withEntityId(adaId));                  // mentions this entity
repository.query(base.withAnyEntity(adaId, babbageId));      // mentions either
repository.query(base.withAllEntities(adaId, babbageId));    // mentions both

withAllEntities is how you ask about a relationship without traversing a graph: claims that mention both endpoints are the claims about the pair.

4.4.3. By lifecycle

repository.query(base.withStatus(PropositionStatus.ACTIVE));
repository.query(base.withPinned(true));
repository.query(base.withMinLevel(1));      // abstractions only
repository.query(base.withMaxLevel(0));      // directly extracted claims only

level is the abstraction ladder: 0 is extracted, higher levels were built by the consolidation AbstractionPass.

4.4.4. By time

Three independent clocks, three independent filters:

base.withCreatedAfter(t)    // when the claim first appeared
base.withRevisedAfter(t)    // when its content last changed — the decay anchor
base.withAccessedAfter(t)   // when it was last read

Filtering on revisedAfter is what you want for "what changed"; createdAfter misses claims that were revised, and accessedAfter tells you about readers, not writers.

4.4.5. Shape the result

var top = base.withMinEffectiveConfidence(0.5)
              .withOrderBy(OrderBy.EFFECTIVE_CONFIDENCE)
              .withLimit(20);

count(query) returns the size without materialising the rows.

4.4.6. Beyond the composable query

PropositionStore also offers direct lookups — findById, findByEntity, findByStatus, findByGrounding, findByMinLevel, findPinned — plus keywordOverlap(base, tokens, limit) for lexical scoring over a base query, and pin/unpin for exempting a claim from reclamation.

touchAccessed(ids) records that a set of propositions was read, without touching either revision clock.

4.4.7. Vector, graph and routed retrieval

Capabilities are optional per backend — check before you rely on one:

if (repository instanceof VectorSearchCapable v) { /* similarity search */ }
if (repository instanceof GraphQueryCapable g)   { /* neighbourhood, path, lineage */ }
if (repository instanceof TemporalQueryCapable t) { /* validity windows */ }

GraphQueryCapable gives you GraphNeighborhood, GraphPath and PropositionLineage. Backends that support it may also report honorsAuthorityFilter, letting authority-filtered traversals run natively rather than being filtered afterwards.

When you would rather not choose, use RetrievalRouter: it picks a RetrievalMode — vector, graph or lexical — from the shape of the query, and applies authority filtering at query time.

4.5. Project propositions into a graph

Prerequisites: propositions with resolved mentions, and a NamedEntityDataRepository. A graph backend is not required to project, but is what makes traversal worth doing.

Propositions are sentences. A graph is edges. Projection is the step that turns one into the other, and it is always rebuildable — if the graph and the propositions disagree, the propositions win.

4.5.1. Pick a projector

Projector How it decides edges

RelationBasedGraphProjector

From the Relations declared on your SourceAnalysisContext. Deterministic, cheap, predictable. Only produces edges for predicates you named.

LlmGraphProjector

An LLM infers edges from the claim text. Covers relationships you did not anticipate, at the cost of a call per batch and less predictability.

Start relation-based. Reach for the LLM projector when you find real relationships your Relations cannot express.

var projector = new RelationBasedGraphProjector(relations, persister, policy);
var outcome = projector.project(propositions);

GraphProjectionService wraps this for the common case; NamedEntityDataRepositoryGraphRelationshipPersister is the persister for entity repositories.

4.5.2. Control what gets projected

A ProjectionPolicy decides which propositions are eligible — typically a confidence or trust floor, so a weakly believed claim does not become a hard edge. ProjectionPolicySupport holds the shipped building blocks.

The ProjectionEligibilityGate does the same job earlier, at admission time, marking a claim as stored but not yet worth projecting.

Deciding this deliberately matters: an edge reads as a fact, even when the claim behind it was believed at 0.4.

4.5.3. Describe the edges

RelationshipDescriptionSynthesizer generates human-readable descriptions for projected relationships; LlmRelationshipDescriptionSynthesizer does it with an LLM. Useful when the graph is something people will look at rather than only traverse.

4.5.4. Lineage and reconciliation

Every projected edge records which propositions produced it, in a ProjectionRecordStore. Two things follow.

The stale cascade. When a source proposition changes, the edges derived from it are marked stale rather than left silently wrong. You can find them, and you can see why.

Idempotent reconciliation. Reconciler brings the graph back into agreement with the propositions without rebuilding it. Re-running projection over unchanged input is a no-op, so projection can be scheduled rather than orchestrated.

var health = graphQueries.projectionHealth(contextId);   // what is stale, what is current
reconciler.reconcile(contextId);                          // bring the graph back in line

The REST layer exposes the same information at GET /api/v1/contexts/{contextId}/discovery/projection-health.

4.5.5. Query the result

With a GraphQueryCapable repository:

GraphNeighborhood around = graphQueries.neighborhood(entityId, /* depth = */ 2);
List<GraphPath> paths   = graphQueries.paths(fromEntityId, toEntityId);
PropositionLineage why  = graphQueries.lineage(propositionId);

lineage is the "why does this edge exist" query: it walks back from a projected edge to the propositions and, through grounding and provenance, to the source text.

4.5.6. Emit events on projection

EventEmittingProjector decorates any projector and publishes Spring application events as projections happen — the hook for cache invalidation, search reindexing, or notifying a UI.

4.6. Give an agent memory

Prerequisites: a populated PropositionRepository scoped by ContextId.

An agent’s context window is small and its conversation is long. Memory projection decides what goes in.

4.6.1. The four kinds of memory

MemoryProjector.project(propositions) returns a MemoryProjection with propositions sorted into four buckets, following the standard cognitive split:

Bucket What belongs there

semantic

Durable facts. "The user works in oncology research."

episodic

Things that happened. "On Tuesday the user asked about drug interactions."

procedural

How to do things in this domain. "This user wants citations with every claim."

working

The current task’s scratch space. Short-lived by design.

KnowledgeType is the enum behind the split. MemoryProjection exposes size and all(), so you can budget the context window across buckets rather than truncating a flat list.

var projection = memoryProjector.project(
        repository.query(PropositionQuery.againstContext(contextId)
                                         .withMinEffectiveConfidence(0.5)));

System.out.printf("semantic=%d episodic=%d procedural=%d working=%d%n",
        projection.getSemantic().size(), projection.getEpisodic().size(),
        projection.getProcedural().size(), projection.getWorking().size());

4.6.2. Recall

MemoryRetriever is the read side, with four entry points shaped by how agents actually ask:

memoryRetriever.recall(contextId, query, limit);         // general, relevance-ranked
memoryRetriever.recallAbout(contextId, entityId, limit); // everything about one entity
memoryRetriever.recallByType(contextId, type, limit);    // one KnowledgeType only
memoryRetriever.recallRecent(contextId, limit);          // most recent first

DefaultMemoryRetriever implements all four. recallAbout is the one to reach for when the agent has already identified who or what it is talking about — it is more precise than a semantic query over the same text.

4.6.3. Keep memory from rotting

Memory is where accumulation hurts first: an agent whose recall is 80% stale beliefs is worse than one with no memory at all.

Run the dream loop periodically. DefaultDreamLoopOrchestrator gates each pass on a threshold, so a quiet context does nothing:

DreamLoopReport report = dreamLoopOrchestrator.run(contextId);

The passes fold sessions into durable claims, build abstractions, resolve contradictions and apply decay. See Knowledge hygiene.

MemoryConsolidator is the lower-level piece if you want to compose your own cycle; DefaultMemoryMaintenanceOrchestrator is the older four-step pipeline, kept for compatibility.

4.6.4. Pin what must not fade

Some things should never decay out of memory — a user’s name, a hard constraint, a standing instruction:

propositionRepository.pin(propositionId);

Pinned propositions are exempt from decay-driven reclamation. Use it sparingly; every pin is a permanent claim on the context window.

4.6.5. Serve it over HTTP

MemoryController exposes memory at /api/v1/contexts/{contextId}/memory — list, search, recall by entity, create, fetch and delete. It is opt-in. See Web API.

4.7. Ask questions in natural language

Prerequisites: a populated PropositionRepository and a configured LLM.

Oracle answers a natural-language question from what DICE knows.

Answer answer = oracle.ask("What does the user think about the early recordings?");

// or with the full question type
Answer answer = oracle.ask(new Question(text));

Two implementations ship:

Implementation How it answers

LlmOracle

Retrieves relevant propositions, then asks an LLM to answer from them. The general case.

ToolOracle

Exposes DICE query operations to the LLM as tools, so it can decide what to look up. Better for questions that need several lookups or a traversal.

PrologTools makes the Prolog projection available to the Oracle as a tool, so a question needing inference — rather than retrieval — can be answered by running rules. See Prolog inference.

4.7.1. Why the Oracle is not a chatbot

The Oracle answers from stored propositions, and nothing else. That is the point: every answer is traceable to claims that carry confidence, grounding and provenance. If it does not know, the honest answer is that it does not know — which is only useful because the propositions behind it are auditable.

To show your work, pair the Oracle with RationaleProjector from dice-report:

RationaleArtifact why = rationaleProjector.rationale(proposition);

LlmRationaleProjector produces the narrative form; rationale(PropositionGroup) explains a set of claims together rather than one at a time.

4.7.2. Scope the question

The Oracle answers within a ContextId like everything else. Asking a question against the wrong context is the most common way to get a confidently wrong answer — one user’s beliefs are not another’s.

4.8. Choose a storage backend

Prerequisites: dice-storage-autoconfigure on the classpath.

Backend selection is a property. The code you write against PropositionRepository does not change.

Backend embabel.dice.store.type Choose it when

In-memory

in-memory (default)

Development, tests, single-process applications, and anything where losing the store on restart is acceptable. Needs an Ai bean for embeddings.

JSON file

— (wire JsonFilePropositionRepository yourself)

You want survival across restarts with no infrastructure at all. Not for concurrent writers.

Graph (Drivine/Neo4j)

graph

Production. Durable, concurrent, with native graph traversal, a vector index, and decay applied in the database.

4.8.1. Switching to the graph backend

embabel:
  dice:
    store:
      type: graph

Plus Drivine’s own Neo4j connection configuration. Nothing else in your application changes: the auto-configuration swaps PropositionRepository, ChunkHistoryStore, DecayManager, ProjectionRecordStore and CollectorRecordStore for their Drivine equivalents, and registers the schema catalogues that create the constraints and the vector index.

See Graph-backed storage for the full activation detail.

4.8.2. What changes when you switch

In-memory Graph

Survives restart

No

Yes

Concurrent writers

Single process

Yes

Vector search

Through the Ai embedding service

Native Neo4j vector index

Graph traversal

Default implementations over the primitives

Native Cypher, via GraphQueryCapable

Decay tick

In-process, over the whole store

Executed in the database

Extra infrastructure

None

A Neo4j instance

4.8.3. Overriding the choice entirely

Every auto-configured bean is @ConditionalOnMissingBean. Define your own PropositionRepository and the auto-configuration steps aside — which is also how you implement PropositionStore against a backend DICE does not ship.

Implement only the capabilities you genuinely support. The default implementations on VectorSearchCapable, GraphTraversalCapable, GraphQueryCapable and TemporalQueryCapable express each operation over the base primitives, so an unimplemented capability degrades in performance rather than breaking.

4.8.4. Migrating between backends

There is no shipped migration tool. Propositions are the system of record and everything else is a projection, so a migration is: read every proposition and entity from the old store, write them to the new one, then re-project. Reconciliation is idempotent, so re-projecting is safe to repeat.

4.9. Observe and react to what DICE does

Prerequisites: a Spring application context.

DICE emits domain events, and every expensive or surprising decision has an observable wrapper.

4.9.1. Events

Two decorators publish Spring application events. Neither is wired by default — you opt in by wrapping:

@Bean
PropositionRepository propositionRepository(PropositionRepository delegate,
                                            ApplicationEventPublisher publisher) {
    return new EventEmittingPropositionRepository(delegate, publisher);
}

EventEmittingPropositionRepository publishes on save; EventEmittingProjector publishes on projection. PropositionPersisted is the canonical durable signal — the one to build on when you need to know something was actually stored.

DiceEvent is the event hierarchy; DiceEventListeners holds the listener support types.

4.9.2. Pipeline events

The pipeline emits its own, pre-persistence events:

pipeline = pipeline.withEventListener(listener);

process emits one aggregate ExtractionBatchCompleted per run. With a reviser configured, processChunk also emits a per-proposition candidate event for each RevisionResult.

Those candidate events are pre-persistence signals, and the distinction matters: a candidate event means the pipeline classified something, not that anything was written. If your listener has side effects, build them on PropositionPersisted instead.

A listener that throws cannot abort a run — the pipeline wraps whatever you register in SafeDiceEventListener. The default is DiceEventListener.DEV_NULL, so behaviour is unchanged when no listener is set.

4.9.3. Observable wrappers

Two decorators exist purely so you can see what is being rejected:

new ObservableMentionFilter(delegateFilter);   // which mentions were dropped, and why
new ObservableGate(delegateGate);              // which propositions a gate rejected

Both are for tuning. A filter or gate you cannot observe is one you will eventually set wrong and not notice — the symptom is silence, and silence looks like success.

4.9.4. Reclamation audit trail

Every live collector run is recorded to a CollectorRecordStore, and detailed decisions to a CollectorTraceStore. The trace records the edges considered, the connected components found, and the decision made for each — enough to explain any individual merge after the fact.

embabel:
  dice:
    collector:
      trace:
        enabled: true
        detail-retention-days: 30   # null keeps detail indefinitely

InMemoryCollectorTraceStore is the default; DrivineCollectorTraceStore persists to Neo4j when the graph backend is selected.

4.9.5. Projection health

var health = graphQueries.projectionHealth(contextId);

Tells you what is stale, which is the signal that a reconciliation pass is due. Also exposed at GET /api/v1/contexts/{contextId}/discovery/projection-health.

5. Features

One page per opt-in surface. Each opens with the exact bean or property that switches it on, so you never have to guess whether a feature is active.

Table 1. Activation conditions at a glance
Surface Activation condition

Graph-backed storage

embabel.dice.store.type=graph

Vector index on Neo4j

embabel.dice.store.vector-index.enabled=true (on by default, graph backend only)

Decay and stale pruning

embabel.dice.store.decay.enabled=true (on by default); prune-stale is false

Multi-signal collector

embabel.dice.collector.enabled — on unless set to false

Concurrent extraction

A parallel or batched ExtractionExecutionStrategy on the pipeline

Web API

@Import(DiceRestConfiguration.class) plus the beans the controllers need

API-key security

dice.security.api-key.enabled=true

Prolog inference

A PrologProjector bean. Experimental.

5.1. Graph-backed storage

Activation: embabel.dice.store.type=graph

Availability: dice-storage and dice-storage-autoconfigure, from DICE 0.2.0. Costs a Neo4j instance.

5.1.1. When to use it

Use it in production, and whenever you need any of: durability across restarts, more than one process writing, native graph traversal, or decay applied in the database rather than in your heap.

Leave it off for development, tests and single-process applications. The in-memory backend is the default precisely because most of what you do while building DICE into an application does not need Neo4j.

5.1.2. Impact

  • Infrastructure: a Neo4j instance, reached through Drivine (drivine4j-spring-boot-starter).

  • Latency: network round trips replace map lookups. Traversals get much faster; single-id lookups get slower.

  • Memory: propositions no longer live in your heap.

  • LLM calls: unchanged.

  • Startup: schema catalogues create constraints and the vector index on first run.

5.1.3. How to enable

embabel:
  dice:
    store:
      type: graph

Plus Drivine’s Neo4j connection configuration.

5.1.4. What you get

With type=graph, DiceStorageAutoConfiguration provides:

Bean Implementation

PropositionRepository

DrivinePropositionRepository — requires an Ai bean for embeddings

ChunkHistoryStore

DrivineChunkHistoryStore

DecayManager

GraphDecayManager — decay executed in the database

ProjectionRecordStore

DrivineProjectionRecordStore — edge lineage

CollectorRecordStore

DrivineCollectorRecordStore — reclamation audit trail

SchemaCatalog beans

Proposition constraints, lineage-record schema, and the vector index spec

Every one is @ConditionalOnMissingBean, so your own bean always wins.

5.1.5. The graph model

dice-storage maps propositions onto typed nodes: PropositionNode, SourceNode, ProcessedChunkNode, Mention and DerivedFrom, with PropositionView and PropositionWithProvenanceView as the read projections. PropositionGraphMapper handles the mapping; GraphProjectionCypher holds the traversal queries.

5.1.6. Example

Nothing in your code changes. This is the same call against either backend:

var confident = propositionRepository.query(
        PropositionQuery.againstContext(contextId).withMinEffectiveConfidence(0.7));

What changes is that on the graph backend the repository also implements GraphQueryCapable, so native traversal is available:

if (propositionRepository instanceof GraphQueryCapable g) {
    var neighbourhood = g.neighborhood(entityId, 2);
}
Kotlin version

dice-storage and dice-storage-autoconfigure build with Kotlin 2.2 — they need the 2.2 compiler for Drivine’s KSP-generated where { } DSL, which uses context parameters. The dice module builds with Kotlin 2.1.10. This matters only if you are building DICE, not if you are consuming it.

5.2. Vector index on Neo4j

Activation: embabel.dice.store.vector-index.enabled=true — on by default, and only takes effect with embabel.dice.store.type=graph and an Ai bean present.

Availability: dice-storage-autoconfigure. No extra infrastructure beyond the Neo4j you already need for the graph backend.

5.2.1. When to use it

Leave it on. Semantic retrieval over propositions is what makes "what do we know that is relevant to this?" answerable without an exact match, and it is also what lets VectorCandidateSearcher resolve aliases that share no characters with the canonical name.

Turn it off when your Neo4j deployment does not support vector indexes, when you are certain all retrieval is by entity and ContextId, or when the index build on a large existing store is something you want to schedule rather than have happen at startup.

5.2.2. Impact

  • Startup: the index is created if absent. On a large existing store this takes time.

  • Write: each proposition is embedded on save — an embedding call per proposition.

  • Storage: one embedding vector per proposition.

  • Read: similarity search becomes a native index lookup rather than a scan.

5.2.3. How to enable

embabel:
  dice:
    store:
      type: graph
      vector-index:
        enabled: true    # the default

To disable:

embabel:
  dice:
    store:
      vector-index:
        enabled: false

5.2.4. What is configurable

Only enabled. The index’s label, property, name and similarity function are fixed by the @VectorIndex annotation on PropositionNode.embedding and live as constants on DrivinePropositionRepository.

This is deliberate: those values must agree between the index definition and every query that uses it, and making them configurable would create a way to have them disagree.

The vector dimension comes from the configured EmbeddingService, so the index matches whatever embedding model your Ai bean is using.

5.2.5. Example

if (propositionRepository instanceof VectorSearchCapable v) {
    var similar = v.findSimilar(queryText, contextId, 10);
}

With the index disabled, VectorSearchCapable operations fall back to the default implementations expressed over the base primitives — correct, but a scan.

5.3. Decay and stale pruning

Activation: embabel.dice.store.decay.enabled=true — on by default. Hard deletion is separate: embabel.dice.store.decay.prune-stale, false by default.

Availability: dice-storage-autoconfigure, both backends. No extra infrastructure.

5.3.1. When to use it

Leave decay on. Without it, a claim made once two years ago competes on equal terms with one made this morning, and retrieval quality degrades as the store grows.

Turn prune-stale on only when you have decided that reclaiming rows matters more than being able to answer "why did you stop believing that?". For most applications it does not.

5.3.2. Impact

  • Scheduling: one tick per interval, hourly by default.

  • Write: the tick materialises cached effective confidence and applies lifecycle transitions.

  • Latency: on the graph backend the tick executes in the database; on in-memory it walks the store in process. Size the interval accordingly.

  • LLM calls: none. Decay is arithmetic.

5.3.3. How to enable

embabel:
  dice:
    store:
      decay:
        enabled: true          # default
        interval-ms: 3600000   # default: 1 hour
        k: 2.0                 # decay-rate multiplier
        prune-stale: false     # default: keep STALE propositions

5.3.4. What the tick does

Two phases:

  1. Materialise — recompute and cache effective confidence, so queries filtering on it do not have to compute decay per row.

  2. Lifecycle — apply the StatusTransitionPolicy, moving propositions whose effective confidence has fallen far enough into a stale status. With prune-stale: true, hard-delete them.

Pinned propositions are exempt.

5.3.5. Tuning k

k is the decay-rate multiplier. Higher k means faster erosion. It appears in three places, and they are independent:

  • embabel.dice.store.decay.k — used by the scheduled tick.

  • PropositionQuery.decayK — used when a query computes effective confidence, defaults to 2.0.

  • Proposition.effectiveConfidence(k) — the direct call, defaults to 2.0.

Setting the property without the query parameter means the tick and your queries disagree about how fast things fade. Set both, or leave both at the default.

5.3.6. Example

var p = repository.findById(id);

System.out.printf("raw=%.2f  now=%.2f  in 30 days=%.2f%n",
        p.getConfidence(),
        p.effectiveConfidence(),
        p.effectiveConfidenceAt(Instant.now().plus(30, ChronoUnit.DAYS)));

Run this against a claim you care about before choosing k. The right value is the one where a belief you would want to have forgotten has faded, and one you would want to keep has not.

5.3.7. The clock decay reads

Decay is anchored on contentRevised, not created and not metadataRevised. Changing a claim’s status, pinning it, or attaching grounding does not make it look fresh again. Only changing what it says does.

5.4. Multi-signal collector

Activation: embabel.dice.collector.enabled — on unless explicitly set to false.

Availability: dice-storage-autoconfigure. No extra infrastructure; the trace store follows your chosen backend.

5.4.1. When to use it

Use it when the same claim reaches you from more than one source and entity resolution alone has not collapsed the duplicates. That is most systems that ingest continuously.

Turn it off when your corpus is small enough to inspect, when duplicates are genuinely rare, or while you are still tuning extraction — merging is much harder to undo than to postpone.

5.4.2. Impact

  • LLM calls: none. Every signal is computed, not generated.

  • Latency: a run is proportional to the candidate pairs the pair sources produce, not to the store.

  • Storage: trace rows per run, retained per detail-retention-days.

  • Risk: this is the component that merges propositions. Dry-run first.

5.4.3. How it decides

MultiSignalCollectorStrategy scores each candidate pair on several signals and blends them. No single signal can carry a merge:

Signal What it measures

vector

Embedding similarity between the two claims.

lexical

Surface token overlap.

entity-overlap

How much the resolved entity sets agree.

grounding-overlap

Whether the claims came from the same chunks.

provenance-overlap

Whether they share evidence links.

polarity-veto

A veto, not a score: stops a claim merging with its own negation.

An edge scoring at or above match-threshold becomes eligible to merge its endpoints. Eligible edges form connected components, and CollectorSurvivorPolicy picks which proposition in each component survives, absorbing the others' evidence.

5.4.4. How to enable

embabel:
  dice:
    collector:
      enabled: true            # default
      match-threshold: 0.6     # aggregate score needed to merge; 0.0-1.0
      signals:
        vector:
          enabled: true
          weight: 0.4
          similarity-threshold: 0.85   # cosine floor for recall
          top-k: 20                    # max similar members per cluster seed
        lexical:
          enabled: true
          weight: 0.2
        entity-overlap:
          enabled: true
          weight: 0.2
        grounding-overlap:
          enabled: true
        provenance-overlap:
          enabled: false
        polarity-veto:
          enabled: true
      sweep:
        delta: false           # property binding only; no runner behind it yet
      trace:
        enabled: true
        detail-retention-days: 30

A null weight means "use the scorer’s own default", which is not the same as zero — the nullable type exists so "unset" and "set to the default number" stay distinguishable.

similarity-threshold and top-k apply to the vector signal only.

5.4.5. Tuning

Raise match-threshold if you see wrong merges. Lower it, or widen the vector signal’s similarity-threshold and top-k, if duplicates survive.

Prefer tuning here over loosening entity resolution: reclamation is auditable and reversible in review, and a bad resolution is neither.

5.4.6. Example

// Always start here.
var dryRun = collectorRunner.run(contextId, /* dryRun = */ true);
dryRun.getDecisions().forEach(System.out::println);

// Only once the decisions look right.
var live = collectorRunner.run(contextId, false);

Every live run is recorded to CollectorRecordStore; the detailed edges, components and decisions go to a CollectorTraceStoreInMemoryCollectorTraceStore by default, DrivineCollectorTraceStore on the graph backend.

5.5. Concurrent extraction

Activation: a parallel or batched ExtractionExecutionStrategy on the pipeline — pipeline.withExecutionStrategy(…​). The default is SerialExtractionStrategy.

Availability: dice. No extra infrastructure; it raises concurrent load on your LLM provider.

5.5.1. When to use it

Use it for bulk ingestion, where extraction dominates wall-clock time and each chunk is an independent LLM call.

Leave it off for conversation analysis — one chunk at a time gains nothing — and until you have verified the condition below.

5.5.2. The condition you must verify first

Only the extraction stage is dispatched concurrently. Enabling a parallel or batched strategy in production is gated on verifying that your PropositionExtractor is thread-safe. See ExtractionExecutionStrategy for the thread-safety verification requirement.

A stateful extractor under a parallel strategy produces corrupted output that looks plausible.

5.5.3. Impact

  • Latency: the extraction stage scales with the strategy’s parallelism. Resolution does not.

  • LLM calls: the same number, issued concurrently. Check your provider’s rate limits.

  • Cost: unchanged.

  • Memory: extraction results for in-flight chunks are held until the resolution stage consumes them.

5.5.4. What stays serial, and why

process(…​) has two stages:

  1. Extraction — stateless per chunk, touches no resolver. Dispatched by the strategy.

  2. Resolution — reads and writes shared entity identity. Always serial.

Keeping resolution serial is what preserves cross-chunk entity identity regardless of how extraction was dispatched. An entity minted while processing chunk 2 is recognised in chunk 7 under every strategy.

Input order is preserved by the strategy, and a chunk whose extraction failed yields a null slot that becomes a typed failed result in the resolution stage — one bad chunk does not fail the batch.

5.5.5. How to enable

// The default.
pipeline.withExecutionStrategy(SerialExtractionStrategy.INSTANCE);

// Parallel.
pipeline.withExecutionStrategy(new ParallelExtractionStrategy(concurrency));

// Batched — batchSize > 1 is concurrent, batchSize == 1 is effectively serial.
pipeline.withExecutionStrategy(new BatchedExtractionStrategy(batchSize));

withExecutionStrategy returns a new pipeline, like every other with* method.

5.5.6. Example

var bulk = PropositionPipeline
        .withExtractor(threadSafeExtractor)
        .withExecutionStrategy(new ParallelExtractionStrategy(8));

var results = bulk.process(manyChunks, context);
results.persist(propositionRepository, namedEntityDataRepository);

Measure before and after. If extraction was not your bottleneck, this buys nothing and adds a thread-safety obligation you now have to maintain.

5.6. Web API

Activation: @Import(DiceRestConfiguration.class), plus the beans the controllers need (PropositionPipeline, PropositionStore, and the rest) and spring-webmvc on the classpath.

Availability: dice. No extra module, no extra infrastructure.

5.6.1. When to use it

Use it when something outside the JVM needs to reach DICE — another service, a front end, a tool process.

Leave it off when DICE is a library inside one application. It is off by default because a knowledge store is not something to expose accidentally.

5.6.2. Impact

  • Surface: three controllers under /api/v1.

  • Security: none by default. Read API-key security before exposing this anywhere.

  • Latency and cost: the extraction endpoints make LLM calls, synchronously.

5.6.3. How to enable

@Configuration
@Import(DiceRestConfiguration.class)
public class MyAppConfiguration { }

DiceRestConfiguration imports the controllers. They are ordinary @RestController beans, and each needs its collaborators present — importing the configuration without the beans behind it will fail at startup rather than silently serve nothing.

5.6.4. The endpoints

Table 2. Extraction — PropositionPipelineController
Method Path What it does

POST

/extract

Runs text through the pipeline and returns an ExtractResponse.

POST

/extract/file

Same, from a multipart/form-data upload. Returns a FileExtractResponse with per-chunk detail.

Table 3. Memory — MemoryController, under /api/v1/contexts/{contextId}/memory
Method Path What it does

GET

/

Lists propositions for the context, filterable by status and confidence.

POST

/search

Semantic search within the context.

GET

/entity/{entityType}/{entityId}

Everything the context knows about one entity.

POST

/

Creates a proposition directly, bypassing extraction.

GET

/{propositionId}

Fetches one proposition.

DELETE

/{propositionId}

Deletes one proposition.

Table 4. Discovery — DiscoveryController, under /api/v1/contexts/{contextId}/discovery
Method Path What it does

POST

/query

Runs a discovery query.

GET

/path

Finds paths between entities.

GET

/why/{propositionId}

Explains why a proposition is believed — its evidence and lineage.

GET

/projection-health

Reports what is stale in the projection.

POST

/collector/dry-run

Reports what reclamation would do, without doing it.

5.6.5. DTOs, not domain objects

The controllers speak in DTOs — PropositionDto, EntityMentionDto, ProvenanceEntryDto, ExtractRequest, MemorySearchRequest and the rest — rather than serialising domain types. The wire format is a contract you can evolve separately from the model, and internal fields do not leak.

ExtractOptions carries per-request extraction settings, including a SchemaAdherenceDto mapping to Context and schema's three constants.

5.6.6. Example

curl -X POST http://localhost:8080/api/v1/extract \
  -H 'Content-Type: application/json' \
  -d '{
        "text": "Ada Lovelace worked with Charles Babbage.",
        "contextId": "demo",
        "options": { "schemaAdherence": "DEFAULT" }
      }'

As shown, that endpoint is unauthenticated and makes LLM calls on demand. Do not expose it without API-key security or your own security configuration.

5.7. API-key security

Activation: dice.security.api-key.enabled=true

Availability: dice. No extra module, no extra infrastructure.

5.7.1. When to use it

Use it whenever Web API is exposed beyond your own process. The REST endpoints are unauthenticated without it, and the extraction endpoints make LLM calls on demand — an open one is both a data-exposure problem and a billing one.

Leave it off when the endpoints are not exposed, or when you already have Spring Security in front of them.

5.7.2. Impact

  • Latency: one filter, one set lookup per request.

  • Cost: none.

  • Operational: keys in configuration are keys in your configuration. See the warning below.

5.7.3. How to enable

dice:
  security:
    api-key:
      enabled: true                 # default false
      keys:
        - ${DICE_API_KEY}
      header-name: X-API-Key        # default
      path-patterns:
        - /api/v1/**                # default

With enabled: true, ApiKeySecurityAutoConfiguration registers an InMemoryApiKeyAuthenticator over the configured keys and a servlet filter over the configured path patterns.

If you enable it and configure no keys, the authenticator logs a warning at startup — that message means every request is being rejected.

5.7.4. Use your own authenticator in production

The in-memory list is for development. Provide your own ApiKeyAuthenticator bean and the auto-configuration steps aside, because it is @ConditionalOnMissingBean:

@Bean
ApiKeyAuthenticator apiKeyAuthenticator(SecretStore secrets) {
    return request -> secrets.isValid(request.getHeader("X-API-Key"));
}

That is where key rotation, per-key scoping and revocation belong.

Keys listed under dice.security.api-key.keys are plain configuration values. Inject them from environment variables or a secret manager — never commit them, and never let them reach a log line.

5.7.5. Scope of protection

The filter protects path-patterns, which defaults to /api/v1/**. Anything outside that pattern — actuator endpoints, your own controllers, static resources — is untouched. If you mount DICE’s controllers under a different prefix, update the patterns to match, or you will have enabled security that guards nothing.

5.8. Prolog inference

Activation: a PrologProjector bean. Nothing is wired by default.

Availability: dice, via tuProlog (2p-kt). No extra infrastructure — the engine is a pure Kotlin library running in your JVM.

Experimental. The API may change without the usual deprecation cycle.

5.8.1. When to use it

Use it when the questions you need answered are derived rather than stored: transitive closure, mutual exclusion, "who else is affected if this is true", rules a domain expert can read and check.

Leave it off when retrieval answers your questions. A vector search and a graph traversal cover most of what applications ask, and neither requires you to maintain a rule base.

5.8.2. Impact

  • LLM calls: none. Inference is symbolic.

  • Memory: the fact base is materialised in the JVM.

  • Latency: query time depends on your rules, not on the store. An unbounded recursive rule will hang.

  • Maintenance: rules are code, and they need tests like any other code.

5.8.3. How to enable

Provide a projector, project, then query:

@Bean
PrologProjector prologProjector(/* collaborators */) {
    return new DefaultPrologProjector(/* ... */);
}
// Propositions -> projected relationships -> Prolog facts.
PrologProjectionResult result = prologProjector.projectAll(relationships);

// Load the facts and your rules into an engine.
PrologEngine engine = PrologEngine.of(result.getFacts(), rules);

5.8.4. Querying

boolean holds        = engine.query("collaborated(ada, babbage)");
List<QueryResult> rs = engine.queryAll("collaborated(ada, X)");
QueryResult first    = engine.queryFirst("collaborated(ada, X)");
List<String> names   = engine.findAll("collaborated(ada, X)", "X");

query is the yes/no form. queryAll returns every solution, queryFirst the first, and findAll extracts the bindings of one variable as strings — usually what you want when the answer is a list of entity ids.

5.8.5. Through the Oracle

PrologTools exposes the fact base to ToolOracle as a tool, so a natural-language question that needs inference rather than retrieval can be answered by running rules. See Ask questions in natural language.

5.8.6. What a projected fact looks like

PrologProjector extends the generic Projector and produces PrologFact values, one per projected relationship. The projection is derived — if the propositions change, re-project rather than editing facts.

PrologTypes holds the type mapping between DICE’s model and Prolog terms.

5.8.7. The tuProlog pin

DICE pins tuProlog to 1.0.4. The 1.1.x series is built with Kotlin 2.2.x and is binary-incompatible with the dice module’s Kotlin 2.1.10. This constrains DICE’s build, not yours — but if you bring your own tuProlog, match the pinned version.

6. Reference

6.1. Configuration properties

Every property DICE binds, with its default and the module that reads it.

6.1.1. embabel.dice.storeDiceStoreProperties

Read by dice-storage-autoconfigure.

Property Type Default Meaning

embabel.dice.store.type

String

in-memory

Backend kind: graph (Drivine/Neo4j) or in-memory.

embabel.dice.store.decay.enabled

boolean

true

Whether the scheduled decay tick runs.

embabel.dice.store.decay.interval-ms

long

3600000

Delay between ticks, in milliseconds. One hour.

embabel.dice.store.decay.k

double

2.0

Decay-rate multiplier used by the staleness policy.

embabel.dice.store.decay.prune-stale

boolean

false

Hard-delete STALE propositions during the lifecycle sweep.

embabel.dice.store.vector-index.enabled

boolean

true

Whether the proposition embedding vector index is created. Graph backend only.

The vector index’s label, property, name and similarity are not configurable — they are fixed by the @VectorIndex annotation on PropositionNode.embedding and live as constants on DrivinePropositionRepository.

6.1.2. embabel.dice.collectorCollectorProperties

Read by dice-storage-autoconfigure. Validated: match-threshold must be in [0.0, 1.0] and signal weights must be non-negative.

Property Type Default Meaning

embabel.dice.collector.enabled

boolean

true

Master switch for the whole collector.

embabel.dice.collector.match-threshold

double

0.6

Minimum aggregate score for an edge to be eligible to merge its endpoints.

embabel.dice.collector.signals.<name>.enabled

boolean

true

Whether this signal participates.

embabel.dice.collector.signals.<name>.weight

Double?

null

Contribution weight. null keeps the scorer’s own default — not the same as 0.

embabel.dice.collector.signals.<name>.similarity-threshold

Double?

null

Cosine floor for candidate recall. Vector signal only.

embabel.dice.collector.signals.<name>.top-k

Integer?

null

Max similar members considered per cluster seed. Vector signal only.

embabel.dice.collector.sweep.delta

boolean

false

Property binding only — there is no runner behind it yet.

embabel.dice.collector.trace.enabled

boolean

true

Whether runs are recorded to a CollectorTraceStore.

embabel.dice.collector.trace.detail-retention-days

Integer?

null

How long detailed trace rows are kept. null keeps them indefinitely.

Valid <name> values are the built-in signals: vector, lexical, entity-overlap, grounding-overlap, provenance-overlap, polarity-veto.

6.1.3. dice.security.api-keyDiceApiKeyProperties

Read by dice. Note the prefix: dice., not embabel.dice..

Property Type Default Meaning

dice.security.api-key.enabled

boolean

false

Enable API-key authentication over the REST endpoints.

dice.security.api-key.keys

List<String>

empty

Valid API keys. Development only — supply an ApiKeyAuthenticator bean in production.

dice.security.api-key.header-name

String

X-API-Key

Header carrying the key.

dice.security.api-key.path-patterns

List<String>

/api/v1/**

Path patterns the filter protects.

6.1.4. embabel.dice.source-analyzer

Read by dice, bound by LlmSourceAnalyzer. Configures the text-to-graph source analyser.

6.1.5. Properties DICE does not own

An LLM provider must be configured, and that configuration belongs to embabel-agent, not to DICE. Neo4j connection settings belong to Drivine. DICE reads neither.

6.1.6. A complete example

embabel:
  dice:
    store:
      type: graph
      decay:
        enabled: true
        interval-ms: 3600000
        k: 2.0
        prune-stale: false
      vector-index:
        enabled: true
    collector:
      enabled: true
      match-threshold: 0.65
      signals:
        vector:
          weight: 0.4
          similarity-threshold: 0.85
          top-k: 20
        lexical:
          weight: 0.2
      trace:
        enabled: true
        detail-retention-days: 30

dice:
  security:
    api-key:
      enabled: true
      keys:
        - ${DICE_API_KEY}

6.2. Package structure

The dice module is organised by responsibility. This is the map for finding the type you need.

Package What lives there

com.embabel.dice.proposition

Proposition, PropositionStatus, PropositionStore, PropositionRepository, PropositionQuery, DecayManager, DecaySweeper, Projection, EntityMention, Derivation, RelationshipTypes, and the capability fragments VectorSearchCapable, GraphTraversalCapable, GraphQueryCapable, TemporalQueryCapable.

com.embabel.dice.proposition.extraction

PropositionExtractor SPI and LlmPropositionExtractor.

com.embabel.dice.proposition.revision

PropositionReviser SPI and LlmPropositionReviser.

com.embabel.dice.proposition.store

InMemoryPropositionRepository, JsonFilePropositionRepository, Neo4jRagPropositionRepository, InMemoryDecayManager.

com.embabel.dice.proposition.gate

Admission gates: ExtractionGate, ExtractionGatePipeline, StandardGates, ObservableGate.

com.embabel.dice.pipeline

PropositionPipeline, PersistablePropositions, PropositionResults, ExtractionExecutionStrategy.

com.embabel.dice.spi

Policy extension points: TrustScorer, AuthorityWeightedTrustScorer, Authority, ConflictDetector, ConflictType, StatusTransitionPolicy, SweepPolicy, SweepAction, PropositionMark, MarkReason, CollectorSignals, CollectorTraceStore, and their defaults.

com.embabel.dice.common

Shared types: SourceAnalysisContext, EntityResolver, Relations, Relation, Resolution, KnownEntity, KnowledgeType, SchemaAdherence, SchemaRegistry, ContentHash, CanonicalNameSelector, EvidenceFloor, DiceEvent, DiceEventListeners, DiceMetadataKeys.

com.embabel.dice.common.filter

MentionFilter, SchemaValidatedMentionFilter, ContextAwareMentionFilters, ObservableMentionFilter.

com.embabel.dice.common.resolver

EscalatingEntityResolver, ChainedEntityResolver, KnownEntityResolver, InMemoryEntityResolver, AlwaysCreateEntityResolver, CandidateSearcher, CandidateBakeoff, LlmCandidateBakeoff, ContextCompressor.

com.embabel.dice.common.resolver.searcher

DefaultCandidateSearchers plus the individual searchers: by exact name, normalized name, partial name, fuzzy name, id, vector, and AgenticCandidateSearcher.

com.embabel.dice.entity

The entity-only pipeline: EntityPipeline, EntityExtractor, LlmEntityExtractor, EntityIncrementalAnalyzer, EntityResolutionService, EntityResolutionTools.

com.embabel.dice.incremental

IncrementalAnalyzer, AbstractIncrementalAnalyzer, IncrementalSource, ConversationSource, ConversationSegmenter, ChunkHistoryStore.

com.embabel.dice.projection.graph

GraphProjector, RelationBasedGraphProjector, LlmGraphProjector, GraphProjectionService, ProjectionPolicy, GraphRelationshipPersister, RelationshipDescriptionSynthesizer.

com.embabel.dice.projection.prolog

PrologProjector, DefaultPrologProjector, PrologEngine, PrologTypes.

com.embabel.dice.projection.memory

MemoryProjector, MemoryProjection, MemoryRetriever, MemoryConsolidator, DreamLoopOrchestrator, MemoryMaintenanceOrchestrator, CollectorRunner, CollectorStrategy, DecayCollectorStrategy, DuplicateCollectorStrategy.

com.embabel.dice.projection.memory.collector

MultiSignalCollectorStrategy, CollectorRunContext, the signal scorers and pair sources, CollectorSurvivorPolicy, and trace storage.

com.embabel.dice.projection.lineage

ProjectionRecordStore, Reconciler.

com.embabel.dice.projection.grounding

Grounding support — the link from claims back to chunks.

com.embabel.dice.operations

PropositionGroup, and under abstraction, consolidation and contrast: PropositionAbstractor, PropositionContraster, and the dream-loop passes (SessionConsolidationPass, AbstractionPass, ContradictionResolutionPass, DecaySweepPass).

com.embabel.dice.query.graph

GraphQuery, GraphNeighborhood, GraphPath, PropositionLineage.

com.embabel.dice.query.discovery

DiscoveryQuery, RetrievalRouter, RetrievalMode, and the discovery DTOs.

com.embabel.dice.query.oracle

Oracle, LlmOracle, ToolOracle, Question, PrologTools.

com.embabel.dice.text2graph

KnowledgeGraphBuilder, SourceAnalyzer, and the entity-merge and relationship-resolution policies, under builder, resolver and support.

com.embabel.dice.provenance

ProvenanceEntry, SourceLocator.

com.embabel.dice.temporal

TemporalMetadata and validity-window support.

com.embabel.dice.agent

Agent-facing integration with embabel-agent.

com.embabel.dice.web.rest

DiceRestConfiguration, PropositionPipelineController, MemoryController, DiscoveryController, and the DTOs. Under security: ApiKeySecurityAutoConfiguration, ApiKeyAuthenticator.

6.2.1. The other modules

Package What lives there

com.embabel.dice.storage (dice-storage)

DrivinePropositionRepository, DrivineChunkHistoryStore, GraphDecayManager, DrivineProjectionRecordStore, DrivineCollectorRecordStore, DrivineCollectorTraceStore, PropositionGraphMapper, GraphProjectionCypher, and under model the graph node and view types.

com.embabel.dice.storage.autoconfigure (dice-storage-autoconfigure)

DiceStorageAutoConfiguration, DiceStoreProperties, CollectorAutoConfiguration, CollectorProperties.

com.embabel.dice.report (dice-report)

ReportProjector, StructuredReportProjector, RationaleProjector, LlmRationaleProjector, SemanticLink, SemanticLinkDiscoverer, TwoHopSemanticLinkDiscoverer.

com.embabel.dice.ingestion (dice-ingestion)

IngestionHandler, TextIngestionHandler, IngestedArtifact, IngestionBatch, IngestionLedger, IngestionResult.

6.3. Extension points

The interfaces DICE expects you to implement. Everything here has a shipped default, so you extend only what your domain actually needs.

6.3.1. Belief and conflict — com.embabel.dice.spi

SPI What you decide

TrustScorer

How much a claim is worth, given its confidence and where it came from. AuthorityWeightedTrustScorer is the shipped default.

AuthorityResolver / Authority

Which authority tier a source belongs to. Feeds trust scoring and query-time filtering.

ConflictDetector

Whether two claims conflict, and how — the ConflictType vocabulary distinguishes supersession from contradiction.

StatusTransitionPolicy

When a proposition changes PropositionStatus.

SweepPolicy

What happens to a marked proposition. StatusTransitionSweepPolicy and MergingSweepPolicy ship; the vocabulary is SweepAction, PropositionMark and MarkReason.

CollectorTraceStore

Where reclamation decision traces are persisted. InMemoryCollectorTraceStore and DrivineCollectorTraceStore ship.

6.3.2. Extraction and resolution

SPI What you decide

PropositionExtractor

How text becomes claims. LlmPropositionExtractor ships. Must be thread-safe if you use a concurrent ExtractionExecutionStrategy.

PropositionReviser

How a new claim is reconciled against stored ones. LlmPropositionReviser ships.

EntityResolver

How a mention becomes an entity. Several ship — see Entity resolution.

CandidateSearcher

How candidate entities are recalled for a mention. This is the usual place to add a domain-specific strategy.

CandidateBakeoff

How one candidate is chosen from several. LlmCandidateBakeoff ships.

MentionFilter

Which mentions are worth resolving at all.

ExtractionGate

Which extracted propositions are admitted. StandardGates holds the shipped set.

ExtractionExecutionStrategy

How the extraction stage is dispatched. Serial, parallel and batched ship.

6.3.3. Storage

SPI What you decide

PropositionStore

The base port: CRUD plus a composable query. Implement this to add a backend.

PropositionRepository

PropositionStore plus the optional capabilities. Implement only the capabilities you genuinely support — defaults cover the rest.

DecayManager

How decay is applied. InMemoryDecayManager and GraphDecayManager ship.

ChunkHistoryStore

What has already been processed, for incremental analysis.

ProjectionRecordStore

Edge lineage, for the stale cascade and reconciliation.

CollectorRecordStore

The reclamation audit trail.

IngestionLedger

The content-hash record that makes re-ingestion a no-op.

IngestionHandler

How an artifact becomes chunks. TextIngestionHandler ships.

6.3.4. Projection and output

SPI What you decide

GraphProjector

Which edges propositions produce. Relation-based and LLM-based ship.

ProjectionPolicy

Which propositions are eligible to be projected at all.

RelationshipDescriptionSynthesizer

Human-readable descriptions for projected edges.

PrologProjector

Propositions as Prolog facts. Experimental.

MemoryProjector / MemoryRetriever

How propositions map to agent memory, and how memory is recalled.

CollectorStrategy

Which propositions get marked for reclamation. MultiSignalCollectorStrategy is the production duplicate matcher.

ReportProjector / RationaleProjector

Human-readable output. In dice-report.

SemanticLinkDiscoverer

Surprising connections between claims. TwoHopSemanticLinkDiscoverer ships.

6.3.5. Java interop

Kotlin infix factory methods on PropositionQuery have Java-friendly withXxx and againstContext equivalents. ContextId is a value class, so Java code uses getContextIdValue(). Proposition.create(…​) is the Java-friendly factory. @JvmOverloads is applied where defaults would otherwise be unreachable from Java.

7. Running in production

What changes when this stops being a quickstart.

7.1. Move off the in-memory store

The in-memory backend loses everything on restart and assumes one writer. Switch to the graph backend before you have data you care about — see Choose a storage backend and Graph-backed storage.

There is no migration tool, and none is needed for a small store: propositions are the system of record, so a migration is read-all, write-all, re-project. Reconciliation is idempotent, so re-projection is safe to repeat and safe to interrupt.

7.2. Choose decay settings deliberately

The defaults — hourly ticks, k=2.0, no pruning — are a reasonable start and a poor finish.

  • Interval: hourly is fine for a store measured in thousands. Lengthen it if the tick’s cost shows up in your metrics.

  • k: set it from how fast your domain’s facts actually go stale. Use effectiveConfidenceAt on real propositions to see what a value means before committing to it.

  • Query decayK: the property configures the tick, not your queries. If you change one, change both, or the two will disagree about how fast beliefs fade.

  • prune-stale: leave it false unless you have decided that reclaiming rows beats being able to explain why a belief was dropped.

7.3. Concurrency

Extraction is the parallelisable stage and resolution is not — that is a property of the design, not a limitation to work around. Before enabling a concurrent ExtractionExecutionStrategy, verify your PropositionExtractor is thread-safe. See Concurrent extraction.

Persistence is yours to bound. The pipeline returns unsaved results and never opens a transaction, so persist(…​) runs inside whatever scope you put it in. On the graph backend, that is the scope Drivine’s transaction management sees.

7.4. LLM cost

DICE makes LLM calls in five places. Four are optional:

Call site Optional How to reduce it

Extraction

No

Chunk larger; use SchemaAdherence.STRICT so less noise is extracted and later reclaimed.

Revision

Yes

Only enable it where restatement actually happens — conversation, not one-shot documents.

Entity resolution

Yes

The escalating resolver only reaches an LLM at the top two rungs. Fix candidate recall and most resolutions become free. See Tune entity resolution.

Consolidation passes

Yes

Threshold-gated by DefaultDreamLoopOrchestrator, so a quiet context costs nothing. Tune the thresholds, not the schedule.

Reports and rationale

Yes

Generate on demand rather than eagerly.

The multi-signal collector makes no LLM calls, which is why tuning duplicates there is cheaper than tuning them at resolution.

7.5. Observability

Instrument these four:

  1. Resolution levels. The ResolutionLevel distribution tells you whether resolution is cheap and correct. A rising LLM_BAKEOFF rate is a cost problem; a rising NO_MATCH rate with minting on is a duplication problem.

  2. Gate rejections. Wrap gates in ObservableGate and filters in ObservableMentionFilter. A mis-tuned gate fails silently, and silence looks like success.

  3. Projection health. projectionHealth(contextId) reports what is stale — the signal that reconciliation is due.

  4. Collector traces. Every live run is recorded. Set detail-retention-days to something you can actually afford to store, and read a trace before you trust a merge.

Wire EventEmittingPropositionRepository and EventEmittingProjector if you want to push these into your own telemetry. PropositionPersisted is the canonical durable signal — pipeline candidate events are pre-persistence and do not mean anything was written.

7.6. Schedule hygiene

Three passes, three cadences. None of them is wired to run automatically except the decay tick:

  • Decay tick — automatic, hourly by default.

  • Reclamation — run it dry, read the trace, then run it live. Weekly is a reasonable starting cadence for a busy store.

  • Consolidation — the dream loop is threshold-gated, so scheduling it often is cheap. Overnight, or per-session for conversational contexts.

Pin anything that must never fade, and pin sparingly: every pin is a permanent claim on retrieval.

7.7. Security

The REST endpoints are opt-in and unauthenticated by default. If you import DiceRestConfiguration, enable API-key security or put your own security in front of it — the extraction endpoints make LLM calls on demand, so an open one is a billing exposure as well as a data one.

Supply your own ApiKeyAuthenticator in production. The in-memory key list is for development, and keys in configuration files are keys in your configuration files.

Propositions carry what users told you. ContextId is the isolation boundary, and it is only as good as the discipline of always scoping queries by it — which is why PropositionQuery has no unscoped factory.

7.8. Backup

Back up the propositions and the entities. Everything else — the graph, the Prolog fact base, memory projections, reports — is derived and rebuildable, and reconciliation is idempotent.

Restore is therefore: restore the store, re-project, verify with projectionHealth.

8. Support

8.1. Compatibility

DICE is pre-1.0. These values track what the build currently uses and are not yet a support commitment. They get pinned at the 1.0 release.

8.1.1. Current build

Axis Current Notes

DICE

0.2.0

Pre-1.0. The API may change between minor versions.

embabel-agent

1.5.2

embabel-agent-api and embabel-agent-rag-core are provided in dice — your application brings the runtime version. DICE fixes only the minimum it compiles against.

JDK

21

The baseline inherited from embabel-build-parent, and what CI builds and tests on. Newer JDKs are untested.

Spring Boot

Inherited from embabel-agent’s dependency management

DICE does not pin Spring Boot itself.

Neo4j

Reached through Drivine (drivine4j-spring-boot-starter 0.0.79)

Only needed for the graph backend. The in-memory and JSON-file backends need no Neo4j.

Kotlin

2.1.10 (dice), 2.2.0 (dice-storage, dice-storage-autoconfigure)

A build concern, not a consumer one. The storage modules need the 2.2 compiler for Drivine’s KSP-generated where { } DSL, which uses context parameters.

tuProlog (2p-kt)

1.0.4, pinned

1.1.x is built with Kotlin 2.2.x and is binary-incompatible with `dice’s Kotlin 2.1.10. Upgrading requires Kotlin 2.2.21+.

8.1.2. Feature availability by module

Feature Needs Status

Proposition pipeline, revision, gates

dice

Stable

Entity resolution

dice

Stable

In-memory and JSON-file stores

dice

Stable

Graph projection

dice

Stable

Memory projection and recall

dice

Stable

Oracle

dice

Stable

REST endpoints

dice + spring-webmvc

Opt-in

API-key security

dice + spring-webmvc

Opt-in

Graph-backed storage

dice-storage (+ dice-storage-autoconfigure) + Neo4j

Stable

Vector index

dice-storage-autoconfigure + graph backend + Ai bean

Stable

Decay tick

dice-storage-autoconfigure

Stable

Multi-signal collector

dice-storage-autoconfigure

Stable

Reports, rationale, link discovery

dice-report

Stable

Ingestion SPI and ledger

dice-ingestion

Stable

Prolog inference

dice + a PrologProjector bean

Experimental

8.1.3. Building DICE itself

mvn verify                    # full build with tests
mvn install -DskipTests       # faster, no tests
mvn test -pl dice             # one module
mvn verify -pl dice-storage   # needs Docker

dice-storage integration tests start a Neo4j container through Testcontainers, so Docker must be running. The root pom pins api.version=1.41 as a Surefire JVM system property to meet the Docker engine’s minimum API version — without it, container-backed tests fail with HTTP 400.

Drivine’s query DSL is generated by a nested Gradle/KSP project at dice-storage/codegen-gradle, run automatically during generate-sources. Do not edit the generated files.

Snapshots and releases resolve from repo.embabel.com. An environment without internet access needs those repositories mirrored locally.

8.1.4. Building this guide

mvn -pl dice-user-guide generate-resources

Output lands in dice-user-guide/target/generated-docs. Add -P guide-pdf for a PDF.

8.2. FAQ

I called process(…​) and nothing was saved.

That is correct behaviour. The pipeline returns PersistablePropositions and stores nothing until you call persist(propositionRepository, namedEntityDataRepository) inside your own transaction. If you drop the result, the work is discarded silently.

Why is there no PropositionQuery.create()?

Deliberate. Without a required scope, "load everything" would be the shortest thing to write, and in a store that accumulates indefinitely that is a production incident. Start from againstContext(…​) or forContextId(…​).

Should I filter on confidence or effectiveConfidence?

effectiveConfidence, almost always. confidence is what the extractor thought at extraction time and never changes; effectiveConfidence folds in decay and is what "do we still believe this?" means.

The same entity appears twice.

Resolution did not match the second mention. Look at the ResolutionLevel distribution first: heavy NO_MATCH with minting on means candidate recall is too narrow. Fix the cheap searchers before touching the LLM rungs — see Tune entity resolution. Reclamation can collapse what got through.

Two different entities got merged into one.

More serious, and harder to undo. Check the collector’s match-threshold and the trace for that merge, and check whether a fuzzy or partial-name searcher is recalling too widely. Raise the threshold and re-run dry.

Why did decay not reset after I updated a proposition?

Decay is anchored on contentRevised, which moves only when the claim’s text changes. Status changes, pinning and grounding touch metadataRevised instead. This is what stops housekeeping from making stale beliefs look fresh.

My queries and the decay tick disagree about how fast things fade.

embabel.dice.store.decay.k configures the tick. PropositionQuery.decayK and effectiveConfidence(k) default to 2.0 independently. Set both, or leave both alone.

Can I run extraction in parallel?

Yes, with an ExtractionExecutionStrategy — but only after verifying your PropositionExtractor is thread-safe. Resolution stays serial regardless, which is what preserves cross-chunk entity identity. See Concurrent extraction.

Why is resolution serial when extraction is not?

Resolution reads and writes shared entity identity. Running it concurrently is how you get two entities for one thing.

The REST endpoints return 404.

They are opt-in. You need @Import(DiceRestConfiguration.class), spring-webmvc on the classpath, and the beans the controllers depend on. A missing bean fails at startup rather than 404-ing, so a 404 usually means the configuration was never imported.

I enabled API-key security and now everything is 401.

Check the startup log. If no keys were configured, ApiKeySecurityAutoConfiguration logs a warning and every request is rejected. Also check path-patterns matches where your controllers actually are.

Do I need Neo4j?

No. The in-memory backend is the default and needs nothing. Neo4j is for durability, concurrent writers and native traversal — see Choose a storage backend.

How do I migrate from in-memory to the graph backend?

Read every proposition and entity from the old store, write them to the new one, re-project. There is no tool because propositions are the system of record and everything else is derived.

What should a ContextId represent?

Your isolation boundary — usually one per user, tenant, conversation or document collection. Contexts do not nest. If you need a hierarchy, encode it in the value and query by prefix at your own layer.

Reclamation deleted something I needed.

It should not have deleted anything: DICE decays rather than deletes, and hard deletion requires embabel.dice.store.decay.prune-stale=true, which is off by default. Pin propositions that must never fade, and always dry-run reclamation first.

Where do I find out why the system behaves this way?

The design notes under docs/design/ in the repository. This guide covers use; those cover rationale.

9. Resources

9.1. Repositories

9.2. Design notes

The contributor-facing rationale lives in docs/design/ in the repository, with an index at docs/design/INDEX.md. Start with architecture.md for a system-level map, then follow to the extraction pipeline, the proposition lifecycle, knowledge hygiene, graph projection, retrieval and discovery, durable storage, and the event model.

AGENTS.md at the repository root is the navigation guide for working on DICE rather than with it.

9.3. Background reading

Johnson, R. (2025). Context Engineering Needs Domain Understanding. Medium. medium.com/@springrod/context-engineering-needs-domain-understanding-b4387e8e4bf8

General User Models (GUM), Stanford and Microsoft. arxiv.org/abs/2505.10831 — the propose/retrieve/revise architecture DICE’s proposition model is derived from.

The Blackboard pattern — the shared-memory architecture the agent framework’s execution model uses.

9.4. Technology

  • tuProlog (2p-kt) — pure Kotlin Prolog engine, used for inference.

  • Drivine — Neo4j access for the graph backend.

  • Spring Framework and Spring Boot — dependency injection, auto-configuration, optional web support.

  • Embabel Agent — LLM integration.

  • Kotlin — the implementation language, with Java-friendly APIs throughout.

9.5. License

Apache License, Version 2.0. © 2024-2026 Embabel Pty Ltd.