Monthly Archives: August 2026

Intellect, Senses, and the Language Model

Intellect: By convention there is sweetness, by convention bitter-ness, by convention color, in reality only atoms and the void.

Senses: Foolish intellect! Do you seek to overthrow us, while it is from us that you take your evidence?

– Democritus

Consider the humble language model with its attention heads, feed-forward layers and complete dependence on the prompt.

The prompt is the information coming in. To be ‘read’ by the senses (attention-heads) to be evaluated by the intellect (feed forward layers) distributed across stacked layers.

The quote above is highly relevant for how such models operate and how the combination of prompts and attention mechanisms impact the results. 

For the model the only place to sense is the context window and the only mechanism to action what changes there is via dependency on external tools. The intellect has limited functions to influence what it attempts to sense next both due to lack of sophisticated actuators (tools) and limited forms of sensing.

The so-what?

The implication is that prompts are becoming complex artefacts that evolve outside direct human control. The so called thought traces of models are a bit of smoke and mirrors in the process as supported by research into CoT faithfulness.

Cost estimates, token budgets, and tool integrations are art forms and not the hard science that a business case requires. The interdependence here creates the perfect fog of war uncertainty. Context enrichment via tool use will compound costs as each model run will require re-evaluation of all the context (assuming caching will be of limited use in this case).

When you have a human to validate outputs and continually tweak the sources of change the risk is less but not absent as humans can quickly lose situational awareness of what is changing.

Getting into the details…

You could have the sharpest intellect (generating layers) but they will generate rubbish if the senses are not aligned.

The senses are frozen in time – they don’t change with time as human senses do. Consider our senses that sharpen with experience (e.g., whilst driving or playing sports). Our sensory apparatus is constantly changing (and degrading as we age). No such advantage for AI models.

Therefore, same inputs broadly give similar outputs (perhaps with different language structures depending on the output randomisation). 

This makes prompting an optimisation problem where we are trying to find the right combination of words to ‘tickle’ the feed forward layers into providing the required response across each model layer.

There is no ‘taking a second look’ with these models by default unless we are operating in an ‘agentic’ manner. 

But then how do these models perform so well with such static senses? The secret lies in multiple attention heads. That is the model taking ‘n’ different looks at the same input and making the output align with the input. Furthermore, thanks to the layering the model is also taking ‘m’ different looks at different levels (e.g., text structures or syntax and semantics).

Now that said, these ‘n’ different views are also static and far less powerful than a single adaptive sensor. I am sure there is some level of sensor selection going on in state-of-the-art proprietary models (beyond plain MoE) where certain types of heads are preferred for specific input content. 

I also wonder if there is any kind of sensor augmentation going on when it comes to specific attention heads per user based on their content and vocabulary. That could enhance the responses. Where out of say 20 heads 5 might be user oriented and 15 generic. 

LoRA attempts to do the above but not at the per-head level. Where LoRA changes are clustered over specific heads in a layer similar outcomes may be achieved but at a macro level. The resolution of sensing will need to massively increase in the next generation of models for a material improvement in quality – specifically concepts like ‘targeted’ LoRA attaching itself to specific heads.

What about Agentic mode you ask?

Think of agentic mode like ‘continuous optimisation’ using external and internal sources of information.

The sensor heads are the same (and frozen) but we take an initial prompt (system + user) and the model is trained to keep tweaking and assessing the output. This tweaking is done using tools that:

  • Allow reflection (feed output back into the model with additional prompting for checking/validating/fixing).
  • Search the web for evidence/grounding.
  • Use other (private) data sources for customisation.
  • Navigate folder structures for grounding content such as skills and knowledge bundles.

This type of tweaking is quite ‘dynamic’ by nature and leads to complex interactions between static internal data (system and user prompt) and dynamic data (both internal and external). The complexity of the interaction grows with the length of the tweaking. 

In tools like Claude this is governed by the ‘Effort’ setting which is not a token budget but a behavioural signal, influencing how many tool calls the model makes and therefore how much external information enters the context.

All of this tweaking can be wasted effort if the senses and the intellect become misaligned due to the information flowing through. Imagine the model senses being hammered by all these bits of information. Each interaction nudging the internal layers and ultimately the next input (via the output) in a particular direction.

What is happening to the input in each loop?

Remember the senses are not changing – the input is. The intellect has the hard work of keeping the output pointing in the right direction as the input evolves. 

The lower layers (closer to the raw input) can influence both the senses and the intellect in the upper layers but not the other way around. Layer ‘m’ has no way to influence the input received from ‘m-1’ as there are no loops in an AI model. 

Loops make it harder for model training and inference to be parallelised like with Recurrent Neural Networks.

It is the richness of the senses working together with a sharp intellect that will help clear the fog of war. This augmentation of the senses and feedback (if it can be achieved at scale) from the intellect will enable longer unsupervised runs on complex tasks with minimum initial ‘prompt engineering’ allowing the agent to self-adapt the input to produce the required output.

Till then we will live with augmentation rather than true autonomy.

Open Knowledge Format Primer

Open Knowledge Format (OKF) is what it says on the tin – a format to represent knowledge for humans and AI agents. The spec is worth a read and I will refer to specific sections instead of reproducing already well-written content.

This post is based on OKF v0.2.

What problem is it solving?

There are three problems when it comes to storing knowledge:

  1. Format
  2. Structure
  3. Semantics

OKF is attempting to solve (1) and bits of (2) with the aim of providing a format that both humans and AI agents can read, write, and action without special tools or libraries. It is also referred to as a navigable wiki for AI Agents.

It brings together metadata, references, access, lineage, security, data quality and other aspects in a flexible framework for mixed data. The trade-off against this flexibility is the hard work required to standardise between knowledge providers and consumers.

The Motivation section of the spec has the full details about the goals behind OKF and where OKF is not applicable.

An Aside: Why an AI Agent?

Because AI Agents have tools that can navigate a directory structure based on index files and locate knowledge of interest. It also means you do not need to load all the information in one prompt.

See Part 1 of this post to understand more.

Format

Format is all about the notation used to record (e.g., Markdown or HTML). This provides a common standard to store, access, and exchange knowledge. Machines use formats like HTML, XML, and JSON to exchange knowledge.

Format also gives us a mechanism to layer the instructions for different consumers. For example, a web-page written in HTML can have a header processed by the browser and a body containing the content for human consumption.

For OKF UTF-8 Markdown is the chosen format which provides frontmatter in YAML for machines and humans to read (metadata) and the body which contains the actual content (again for humans or AI models to consume). OKF mandates only one metadata field called type.

Type

Type represents a label that classifies the concept being represented in an OKF Markdown file. There is no central repository of allowed Types and authors can use any label that best describes the concept. OKF provides some examples but the clear expectation is that builders can ‘bring their own’.

Structure

This builds upon the format to define what constructs can be used to represent what aspects of the concept. For example, in Markdown one can choose to use a numbered list to represent a process, a code block to represent code, and a quote to represent contextual information. We may restrict the use of certain structures such as headings below the 2nd level.

This is where semantics start to appear. We associate structure with meaning as in the example above. When those mappings are agreed and understood then any reader reading a doc will understand that knowledge in a quote structure is mainly for context. This can change from org to org as one may choose to interpret quote as an important piece of knowledge instead of contextual information.

OKF does not mandate any structural mapping – there are some guidance items but broadly speaking you are free to do what you want.

Semantics

This is all about how knowledge is decomposed into concepts and those concepts broken down into structures using a given format.

OKF does not mandate any semantics around knowledge decomposition. Therefore, this is where the bulk of the organisational standardisation effort will lie for larger orgs. The consequence of not doing this will be the same knowledge being surfaced in different styles across the org which could lead to inconsistent machine / human interpretation.

The OKF Layout

OKF artefacts are called Knowledge Bundles where each bundle consists of multiple documents arranged in a directory structure representing a concept. The Terminology section of the spec is a good place to start.

Knowledge Bundle or bundle

A Knowledge bundle is a collection of documents containing knowledge. It is also a basic unit of distribution which means all your effort as an author (or builder of AI authors) will be focussed on creating these bundles and organising them for access. The Bundle Structure section is worth checking out.

A bundle can have nested directories to organise different concept documents (markdown files) but each directory may have an index.md file that describes the contents of that directory via links (see here). The use of a markdown link to connect concepts is an example of semantics mapped to structure from above.

As an example let us assume we wanted to create an OKF bundle for writing code in python then we could create the following directory structure with a set of files. The directory names are surrounded by <> for clarity.

<Python_Programmer_Bundle> 
          |
          |
          |  - <Testing>
          |        |
          |          - unit_test.md
          |          - integration_test.md
          |          - system_test.md
          |          - index.md
          |
          |  - <Coding>
          |        |
          |          - python_syntax.md
          |          - python_patterns.md
          |          - python_optimisation.md
          |          - index.md
          |
          |  -  thinking_like_a_coder.md
          |  -  index.md
          |  -  log.md

Note an index.md file at each level aids in navigation. This hierarchy ensures localised interactions between concepts.

Reserved File Names

Concept files can have any name except the two reserved names:

  • index.md – to define the index as a directory listing
  • log.md – to record the changes in the bundle

Bundle Distribution

Bundles can be distributed as:

  1. Git repository (or sub-repo) – which gives you all the goodness of source control.
  2. Zip – risky as content can change and there is no tracking in place.

Concept Files

Each concept within the topic that is being recorded needs to go into its own markdown file and be recorded in the index file.

What OKF does not describe is a standard way of breaking down a topic into concepts. Lack of org wide semantic standards in this space could mean multiple decompositions of the same topic into different concepts leading to inconsistent outcomes when used by AI. This becomes especially important when the ambition is for these artefacts to be continuously written and maintained by AI agents.

Further creating links between bundles and concepts can lead to problems of maintenance as changes happen. This is especially important for organisational knowledge that can change rapidly (e.g., when new products are released or old products retired).

Large organisations hoping to leverage OKF as a format for representing knowledge for AI Agents need to divide topic ownerships between business domains and establish cross-domain change management processes. They also need to establish semantic firewalls between these domains to ensure changes in one domain do not overwhelm other domains.

A concept file has two structural elements – the Frontmatter and the Body

The Frontmatter – which is where the metadata elements for the concept reside. It is in YAML for it to be machine readable without the use of AI.

The only required piece of metadata here is type described previously. There is a whole list of recommended metadata items including:

  • Title – may be derived from filename if absent
  • Description – single sentence used in index.md to describe the linked concept
  • Resource – URI that connects the concept to the underlying asset described (e.g., table, document).
  • Tags – YAML list of short strings for categorisation.
  • Optional extensions that describe provenance, trust, lifecycle, and attested computation.

Producers may include other keys and Consumers must be able to ignore unknown keys instead of rejecting them. In other words there is no strict schema for metadata beyond the ‘type’ value.

The Body – which is where the core knowledge associated with the concept lies. The interesting point related to the format/structure/semantics layering is that as per the guidelines:

Producers SHOULD favour structural markdown (headings, lists, tables, fenced code blocks) over freeform prose, since structure aids both human reading and agent retrieval.

Therefore the body text is not free-flowing prose, it is structured using markdown.

Claim attribution to external sources needs to be recorded using markdown footnotes keyed by references.

There are no required body sections (remember format/structure/semantics and the blank canvas approach) but some conventions have been established as a SHOULD to indicate:

  1. Schema – for schema of underlying asset under ‘# Schema’
  2. Examples – usage examples under ‘# Examples’
  3. Attested Computation – sanctioned computation under ‘# Computation’

The reason to have 1 and 2 above is to (for example) describe a database table (schema), some example queries against that table, with a metadata item called Resource providing a URL to the table.

Relationships

This is another important semantic decisioning point. Both from linking of concepts as well as from a concept lifecycle perspective.

Within OKF links, paths, and references are loosely defined. Links are optional (a concept may be a standalone one or may be divided into multiple concepts). Links are interpreted as directed edges (A->B doesn’t imply B->A) without a specific relationship type. The lifecycle implications of typed relationships (such as parent-child, peer etc.) is not defined within OKF. Links can point to non-existent targets such as in cases where a particular item of knowledge doesn’t exist.

Paths can be found in several fields such as Markdown link or metadata items such as resource and sources. Paths can be:

  • relative to the bundle-root (Python_Programmer_Bundle in the above example)
  • relative to the current concept markdown (using . or .. operators)
  • absolute URL (Database example above)

References can be in a references sub-directory which can contain artefacts such as code and external material (e.g., source PDFs) but this is not a hard requirement just a convention.

Actors

Given it is important to record knowledge attribution OKF provides a way to construct identities for Actors. There are three primary actors:

  1. Human – identity format: human:<id> (<id> can be an email or employee id)
    • Example: human:321456
  2. AI Agent – identity format: <producer agent>/<model version>
    • Example: customer_support_agent/chatgpt-5.5
  3. Process – identity format: process:<id> (<id> is the process ID or name)
    • Example: process:table_cron_job

Index and Log Files

Index.md and log.md files are optional.

Index files implement progressive disclosure where the reader (human or agent) gets a gradual exposure to available knowledge under their own navigation control. This moves away from the ‘prompt and pray’ concept of crafting that perfect knowledge dump to trigger correct responses.

The index.md carries no frontmatter except the one in the root of the bundle which may carry the okf_version key.

Index.md can be generated automatically by the producer or on the fly by the consumer when the file is not detected.

For logging change in the OKF bundle log.md files are used. The location (which sub-directory) needs to be aligned with the scope of the changes being recorded. For example, in the Python_Programmer_Bundle example a log.md in the Testing folder cannot record changes for the root folder.

Attested Computations

This particular concept deserves a post in itself but given it is an emerging part of an emerging format (newest of the new!) I will just cover the basics here.

The concept is simple: how to provide a value as well as a method to calculate it to verify. For example: you are calculating something based on a database query – here you can share the result and the query.

OKF merely records the result and the mechanism to validate it. No validation is carried out as OKF is not a compute engine.

Attested Computation works off a contract which is described in the frontmatter including runtime for execution, sources, attester computation, and verification information.

Agents, Skills and OKF

This post explores how Agents can be powered by skills. It will also explore how we can provide knowledge to augment the skills and boost performance on complex tasks. The fast evolving Open Knowledge Format (see the spec here) provides one option to represent knowledge.

Part 1 of this post goes through the building blocks. Part 2 focuses on OKF concepts, Part 3 will focus on the code.

Building Blocks

There are three major building blocks for the system described above:

  1. The AI Model powering the agent.
  2. Knowledge Tools available to the agent.
  3. Skills and Knowledge artefacts available to the agent.

Let us tackle the two easy ones first.

AI Model

The AI model for this needs to be able to follow instructions, operate tools and have a fairly decent context window size (at least 16k for basic tasks). Reasoning mode also needs to be supported by the AI Model harness as well as conversational state management.

Context window size

This defines the amount of information the AI model can keep in view at a given time. This is super critical especially for complex tasks where lot of this precious space is taken up by summaries of linked content, planning outputs, reasoning hints, and guidance text.

The best way to visualise this is that the AI agent is building a bridge from the request to the required output one step at a time. The size of the step it can take is determined by the context window size.

Trade-off: when we use million token context models like Gemini Pro it gives some breathing room to the amount of information you can store. At the same time it can also create information overload, divergence, and mis-direction.

Overload happens when there is too much information about the task.

Divergence happens when two information items contradict.

Mis-direction happens when one particular information item dominates the landscape and prevents a smooth transition from explore (finding the best solution) to fulfil (executing the solution in an optimal manner).

Reasoning mode

Without reasoning mode context window size is of limited use. If the context window size is the step size as the AI agent builds the bridge from the request to the required output then the reasoning mode defines the effort spent in building each step.

Reasoning mode at its simplest is all about moving away from request-response of Gen AI to more of a request-reason-response expected from AI agents. It also provides a multi-layered safety net and improves output quality where the request and candidate response is further studied by the AI model, tools are used to fetch validation data, fetched data is compared with the request and candidate response, candidate response is re-written with added references and so on.

Trade-off: As with everything this kind of loop can take an AI agent down the wrong track leading to either wasted effort of generating and reviewing content that is irrelevant or (even worse) taking decisions that leave a longer term impact.

This is usually seen when weaker models are used with reasoning loops. I have had several instances of Gemma4 spinning its wheels attempting to decide whether I wanted it to solve a question or explain how to solve it without solving it.

Reasoning loops are also impacted by the amount of stuff in the context window. If it has previous turns from the current conversation and if those turns end up feeding overload, divergence or mis-direction then reasoning is like adding fuel to the fire.

The Gemma4 example above was because previously in the same conversation I was talking about solving a problem and the model was not able to detect conversational drift and therefore focus on the current ask.

Tool Use

This requires the model to be able to embed tool use within the reasoning loop. Tool use allows external information to be brought in as well as information to be persisted during the reasoning and generation process.

Trade-off: the big trade-off here is that tools can also add confusion to the process and be a mechanism that starts the overload, divergence, or mis-direction fire. When content is pulled from different sources we often are not in control of what is pulled.

Similarly when content is written we are not sure what is being persisted and what will be the context when read.

Knowledge Tools

Knowledge tools are tools that enable access to knowledge as well as ability to create and persist knowledge.

Read: Agents that Educate Themselves

Tools that enable the AI agent to read from a directory structure (usually sandboxed filesystem with strict permissions) on demand. This provides some flexibility around sourcing knowledge. This moves away from the older concept of prompt templates where all the information had to be in there or loaded piece by piece.

The concept of prompt templates only works for prior knowledge and not for runtime-required knowledge. With these kinds of tools prompt templates become prior knowledge injection points, more to bootstrap than actively run the agent. For example prompt templates can provide information about the task, guardrails, knowledge sources (e.g., directory structure and files), and tools to access those knowledge sources (read directory, read file).

An extension of this would be tools that allow some form of guarded web-search which allows AI agents to retrieve knowledge from curated sources. This is specifically important to enable the time-axis of knowledge where the AI agent can reacquire knowledge when a time threshold is crossed (e.g., latest interest or currency conversion rates).

Write: Agents that Leave Notes For Themselves

Tools that enable the AI agent to write files is the next step in knowledge fluidity. This allows the AI agent to write intermediate notes (within a request-reason-response cycle) to guide generation and to trace out its approach for explainability and human in the loop.

It can also be used to create long running guidance artefacts that persist across multiple interactions tracking past actions, outputs, user preference, and reasoning traces. This can then form the basis of a customer independent long term agent memory.

Skills and Knowledge Artefacts

This is perhaps the toughest section. Because we come from format to semantics.

Skills are based on one or more agreed definitions that explain the how of a task including tools to be used, process flows, sequencing, guardrails and supporting knowledge. Knowledge artefacts using formats such as OKF represent an ‘agent navigable wiki’ that stores a set of related concepts and metadata around it.

Given this concept is pretty fluid you could actually have a skills wiki that allows an agent to learn all the skills available and then those skills themselves could link to specific knowledge artefacts that educate the agent regarding the task.

As you start building out skills -> knowledge artefacts -> skills graphs you start defining what I call a skills network. Your AI application then simply navigates this network.

That’s all folks (for this part)…

Part 2 of this post is now live – around OKF concepts.

Part 3 of this post will focus on the technical implementation of some of these concepts including OKF (and lots of code!).