Skip to content

fix(core): let frontmatter keys satisfy schema fields - #2047

Merged
phernandez merged 3 commits into
mainfrom
fix/1973-schema-frontmatter-fields
Nov 4, 2026
Merged

phernandez merged 3 commits into
mainfrom
fix/1973-schema-frontmatter-fields

Conversation

@phernandez

Copy link
Copy Markdown
Member

Closes #1973. Implements the decision on the issue: a schema field is satisfied by a frontmatter key or an observation category, frontmatter checked first, with no per-field source syntax.

Changes

  • validate (picoschema/validator.py): a non-relation field takes its values from the note's frontmatter key first, then from [field] observations. Details:
    • A list gives one value per item.
    • A null value or an empty list counts as absent and falls back to observations.
    • Enum checks apply to either source.
    • The missing-field warning now reads Missing required field: attendees (expected attendees: in frontmatter or a [attendees] observation).
  • infer / diff (picoschema/inference.py, diff.py): analyze_fields replaces analyze_observations.
    • It counts frontmatter keys as fields, once per note, frontmatter first.
    • It skips the keys every note carries: title, type, permalink, tags, schema.
    • FieldFrequency.source can now be frontmatter: the place most of a field's notes keep it, frontmatter on a tie.
  • router: NoteData for infer/diff now carries the entity's frontmatter.
  • bm-schema(3) states the rule and shows the new warning.

Tests

  • tests/picoschema/test_frontmatter_fields.py covers:
    • validation from frontmatter, observations, and both (frontmatter wins)
    • enums, lists, and empty keys
    • the new message
    • infer counting and source
    • diff for kept and new frontmatter fields
  • tests/api/v2/test_schema_router.py::test_frontmatter_satisfies_schema_fields_in_validate_infer_and_diff runs the report's Meeting / attendees: 4 case through validate, infer and diff.
  • just fast-check; picoschema, schema router and tests/mcp suites pass (1,495 tests); the router tests also pass on Postgres.

๐Ÿค– Generated with Claude Code

https://claude.ai/code/session_01CMYVL4TcbWdXhdvAzxfqjo

A Picoschema field matched only observations, so a note with
`attendees: 4` in frontmatter failed `attendees: integer`, and infer
and diff ignored frontmatter. Scalar facts live in frontmatter, where
bm_write(metadata=...) and set_metadata put them.

- validate: a non-relation field takes its values from a frontmatter key
  first, then from [field] observations. Enum checks apply to either.
- infer/diff: analyze_fields replaces analyze_observations and counts
  frontmatter keys as fields, once per note, skipping title, type,
  permalink, tags and schema. A field's source is where most of its
  notes keep it.
- bm-schema(3) states the rule.

Closes #1973

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CMYVL4TcbWdXhdvAzxfqjo
Signed-off-by: phernandez <paul@basicmachines.co>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
๐Ÿ“ Code Review โœ… Completed 2026-11-04T09:19:31.377399Z 272066a New commits
โ„น๏ธ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with ๐Ÿ‘€ while any review is running, comments if it has suggestions, and reacts with ๐Ÿ‘ once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๐Ÿ’ก Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c749a37046

โ„น๏ธ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with ๐Ÿ‘.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/basic_memory/picoschema/diff.py Resolved
Comment thread src/basic_memory/picoschema/diff.py Resolved
โ€ฆhema diff

Codex review on #2047:
- An entity-reference field is satisfied only by relations, as in
  validation, so frontmatter or an observation of the same name no
  longer hides that no note has the relation.
- Keys declared under settings.frontmatter count as declared, so they
  are not reported as new fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CMYVL4TcbWdXhdvAzxfqjo
Signed-off-by: phernandez <paul@basicmachines.co>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

๐Ÿ’ก Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fe072e8724

โ„น๏ธ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with ๐Ÿ‘.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Inference skips keys every note carries (title, type, tags, ...). Drift
skipped them too, so a schema that declares tags reported it as dropped
while every note used it. Drift now skips only the built-in keys the
schema does not declare.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CMYVL4TcbWdXhdvAzxfqjo
Signed-off-by: phernandez <paul@basicmachines.co>
@phernandez
phernandez merged commit 44bc235 into main Nov 4, 2026
53 checks passed
@phernandez
phernandez deleted the fix/1973-schema-frontmatter-fields branch November 4, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Schema fields should be satisfiable by frontmatter, not only observations

1 participant