Open Knowledge Format Primer

Open Knowledge Format (OKF) is what it says on the tin – a format to represent knowledge for humans and AI agents. The spec is worth a read and I will refer to specific sections instead of reproducing already well-written content.

This post is based on OKF v0.2.

What problem is it solving?

There are three problems when it comes to storing knowledge:

1. Format
2. Structure
3. Semantics

OKF is attempting to solve (1) and bits of (2) with the aim of providing a format that both humans and AI agents can read, write, and action without special tools or libraries. It is also referred to as a navigable wiki for AI Agents.

It brings together metadata, references, access, lineage, security, data quality and other aspects in a flexible framework for mixed data. The trade-off against this flexibility is the hard work required to standardise between knowledge providers and consumers.

The Motivation section of the spec has the full details about the goals behind OKF and where OKF is not applicable.

Why an AI Agent?

Because AI Agents have tools that can navigate a directory structure based on index files and locate knowledge of interest. It also means you do not need to load all the information in one prompt.

See Part 1 of this post to understand more.

Format

Format is all about the notation used to record (e.g., Markdown or HTML). This provides a common standard to store, access, and exchange knowledge. Machines use formats like HTML, XML, and JSON to exchange knowledge.

Format also gives us a mechanism to layer the instructions for different consumers. For example, a web-page written in HTML can have a header processed by the browser and a body containing the content for human consumption.

For OKF UTF-8 Markdown is the chosen format which provides frontmatter in YAML for machines and humans to read (metadata) and the body which contains the actual content (again for humans or AI models to consume). OKF mandates only one metadata field called type.

Type

Type represents a label that classifies the concept being represented in an OKF Markdown file. There is no central repository of allowed Types and authors can use any label that best describes the concept. OKF provides some examples but the clear expectation is that builders can ‘bring their own’.

Structure

This build upon the format to define what constructs can be used to represent what aspects of the concept. For example, in Markdown one can choose to use a numbered list to represent a process, a code block to represent code, and a quote to represent contextual information. We may restrict the use of certain structures such as headings below the 2nd level.

This is where semantics start to appear. We associate structure with meaning as in the example above. When those mappings are agreed and understood then any reader reading a doc will understand that knowledge in a quote structure is mainly for context. This can change from org to org as one my choose to interpret quote as an important piece of knowledge instead of contextual information.

OKF does not mandate any structural mapping – there are some guidance items but broadly speaking you are free to do what you want.

Semantics

This is all about how knowledge is decomposed into concepts and those concepts broken down into structures using a given format.

OKF does not mandate any semantics around knowledge decomposition. Therefore, this is where the bulk of the organisational standardisation effort will lie for larger orgs. The consequence of not doing this will be same knowledge surfaced in different styles across the org which could lead to inconsistent machine / human interpretation.

The OKF Layout

OKF artefacts are called Knowledge Bundles where each bundle consists of multiple documents arranged in a directory structure representing a concept. The Terminology section of the spec is a good place to start.

Knowledge Bundle or bundle

A Knowledge bundle is a collection of documents containing knowledge. It is also a basic unit of distribution which means all your effort as an author (or builder of AI authors) will be focussed on creating these bundles and organising them for access. The Bundle Structure section is worth checking out.

A bundle can have nested directories to organise different concept documents (markdown files) but each directory needs to have an index.md file that describes the contents of that directory via links. The use of a markdown link to connect concepts is an example of semantics mapped to structure from above.

As an example let us assume we wanted to create an OKF bundle for writing code in python then we could create the following directory structure with a set of files. The directory names are surrounded by <> for clarity.

<Python_Programmer_Bundle> 
          |
          |
          |  - <Testing>
          |        |
          |          - unit_test.md
          |          - integration_test.md
          |          - system_test.md
          |          - index.md
          |
          |  - <Coding>
          |        |
          |          - python_syntax.md
          |          - python_patterns.md
          |          - python_optimisation.md
          |          - index.md
          |
          |  -  thinking_like_a_coder.md
          |  -  index.md
          |  -  log.md
        

Note an index.md file at each level aids in navigation. This hierarchy ensures localised interactions between concepts.

Reserved File Names

Concept files can have any name except the two reserved names:

• index.md – to define the index as a directory listing
• log.md – to record the changes in the bundle

Bundle Distribution

Bundles can be distributed as:

1. Git repository (or sub-repo) – which gives you all the goodness of source control.
2. Zip – very dangerous as content can change without any tracking.

Concept Files

Each concept within the topic that is being recorded needs to go into its own markdown file and be recorded in the index file.

What OKF does not describe is a standard way of breaking down a topic into concepts. Lack of org wide semantic standards in this space could mean multiple decompositions of the same topic into different concepts leading to inconsistent outcomes when used by AI. This becomes especially important when the ambition is for these artefacts to be continuously written and maintained by AI agents.

Further creating links between bundles and concepts can lead to problem of maintenance as changes happen. This is especially important for organisational knowledge that can change rapidly (e.g., when new products are released or old products retired).

Large organisations hoping to leverage OKF as a format for representing knowledge for AI Agents need to divide topic ownerships between business domains and establish cross-domain change management processes. They also need to establish semantic firewalls between these domains to ensure changes in one domain do not overwhelm other domains.

A concept file has two structural elements – the Frontmatter and the Body

The Frontmatter – which is where the metadata elements for the concept reside. It is in YAML for it to be machine readable without the use of AI.

The only required piece of metadata here is type described previously. There is a whole list of recommended metadata items including:

• Title – may be derived from filename if absent
• Description – single sentence used in index.md to describe the linked concept
• Resource – URI that connects the concept to the underlying asset described (e.g., table, document).
• Tags – YAML list of short strings for categorisation.
• Optional extensions that describe provenance, trust, lifecycle, and attested computation.

Producers may include other keys and Consumers must be able to ignore unknown keys instead of rejecting them. In other words there is no strict schema for metadata beyond the ‘type’ value.

The Body – which is where the core knowledge associated with the concept lies. The interesting point related to the format/structure/semantics layering is that as per the guidelines:

Producers SHOULD favor structural markdown (headings, lists, tables, fenced code blocks) over freeform prose, since structure aids both human reading and agent retrieval.

Therefore the body text is not natural language.

Claim attribution to external sources needs to be recorded using markdown footnotes keyed

There are no required body sections (remember format/structure/semantics and the blank canvas approach) but some conventions have been established as a SHOULD to indicate:

1. Schema – for schema of underlying asset under ‘# Schema’
2. Examples – usage examples under ‘# Examples’
3. Attested Computation – sanctioned computation under ‘# Computation’

The reason to have 1 and 2 above is to say describe a database table (schema), some example queries against that table, with a metadata item called Resource providing a URL to the table.

Relationships

This is another important semantic decisioning point. Both from linking of concepts as well as from a concept lifecycle perspective.

Within OKF links, paths, and references are loosely defined. Links are optional (a concept may be a standalone one or may be divided into multiple concepts). Links are interpreted as directed edges (A->B doesn’t imply B->A) without a specific relationship type. The lifecycle implications of typed relationships (such as parent-child, peer etc.) is not defined within OKF. Links can point to non-existent targets such as in cases where a particular item of knowledge doesn’t exist.

Paths can be found in several fields such as Markdown link or metadata items such as resource and sources. Paths can be:

• relative to the bundle-root (Python_Programmer_Bundle in the above example)
• relative to the current concept markdown (using . or .. operators)
• absolute URL (Database example above)

References can be in a references sub-directory which can contain artefacts such as code and external material (e.g., source PDFs) but this is not a hard requirement just a convention.

Actors

Given it is important to record knowledge attribution OKF provides a way to construct identities for Actors. There are three primary actors:

1. Human – identity format: human:<id> (<id> can be an email or employee id)
   • Example: human:321456
2. AI Agent – identity format: <producer agent>/<model version>
   • Example: customer_support_agent/chatgpt-5.5
3. Process – identity format: process:<id> (<id> is the process ID or name)
   • Example: process:table_cron_job

Index and Log Files

Index.md and log.md files are optional.

Index files implement progressive disclosure where the reader (human or agent) gets a gradual exposure to available knowledge under their own navigation control. This moves away from the ‘prompt and pray’ concept of crafting that perfect knowledge dump to trigger correct responses.

The index.md carries no frontmatter except the one in the root of the bundle which may carry the okf_version key.

Index.md can be generated automatically by the producer or on the fly by the consumer when the file is not detected.

For logging change in the OKF bundle log.md files are used. The location (which sub-directory) needs to be aligned with the scope of the changes being recorded. For example, in the Python_Programmer_Bundle example a log.md in the Testing folder cannot record change for the root folder.

Attested Computations

This particular concept deserves a post in itself but given it is an emerging part of an emerging format (newest of the new!) I will just cover the basics here.

The concept is simple: how to provide a value as well as a method to calculate it to verify. For example: you are calculating something based on a database query – here you can share the result and the query.

OKF merely records the result and the mechanism to validate it. No validation is carried out as OKF is not a compute engine.

Attested Computation works off a contract which is described in the frontmatter including runtime for execution, sources, attester computation, and verification information.