Skip to content

Topic: Docs-as-Code

Markdown simplicity, DITA rigor: bringing AI into documentation validation

Combine Markdown’s simplicity with DITA’s structural rigor using AI and knowledge graphs to detect information-architecture problems that conventional linters and validation rules cannot catch.

Olivier Carrère11 min read

On this page ▼

Markdown is much simpler than XML-based formats such as DITA and doesn’t require expensive tooling to present users with nicely laid-out documentation. So, this might look like the way to go for any company eager to minimize documentation production costs and avoid vendor lock-in.

However, as far as documentation is concerned, a fine layout isn’t enough. Users must be able to browse consistently structured content to quickly find the type of information they’re looking for. That’s where DITA and its well-defined DTDs help: they provide a predictable structure that lets users quickly find actionable conceptual information, procedures, or reference material. Writers, for their part, first select the type of topic they want to create and are then prevented from producing ill-structured content.

Linters are great at detecting and preventing syntax inconsistencies, but they won’t stop technical writers from mixing multiple information types in the same topic, which can frustrate users.

Deterministic linters such as markdownlint and Vale can ensure that basic formatting and stylistic rules are applied. AI-based checkers could assess whether topics comply with specific information types, although their interpretation would be probabilistic. Markdown would keep its authoring simplicity, while AI-assisted semantic analysis and deterministic validation would bring in some of the structural discipline traditionally associated with DITA, though not DITA’s authoring-time guarantees.

Example: mixing up tasks, concepts, and reference material

Let’s suppose you create this ill-typed topic, mixing several types of information.

# Configure a static IP address

Assign a fixed IPv4 address to a server so that other hosts can always reach
it at the same address.

## How Netplan works

Netplan doesn't configure network interfaces itself. It reads the YAML files in
`/etc/netplan/` and generates configuration for a back-end renderer, either
`systemd-networkd` or `NetworkManager`. This is why a mistake in a YAML file
can leave a server unreachable after the next reboot.

1. Open `/etc/netplan/01-netcfg.yaml` in your editor of choice.
2. Set `dhcp4` to `false`.
3. Add static settings:

   ~~~yaml
   network:
     version: 2
     ethernets:
       eth0:
         dhcp4: false
         addresses: [192.0.2.10/24]
         routes:
           - to: default
             via: 192.0.2.1
         nameservers:
           addresses: [192.0.2.53]
   ~~~

   | Key           | Accepted values       | Notes                    |
   |---------------|-----------------------|--------------------------|
   | `dhcp4`       | `true`, `false`       | Defaults to `false`.     |
   | `addresses`   | List of CIDR prefixes | Required for static IPs. |
   | `routes`      | List of `to`/`via`    | Replaces `gateway4`.     |
   | `nameservers` | `addresses`, `search` | Optional.                |

   If `dhcp4` stays `true`, the interface gets both a leased address and the
   static one, which makes routing harder to predict.

4. Run `sudo netplan try`.

   `netplan try` applies the configuration and reverts it after 120 seconds
   unless you confirm it.

5. Press Enter to keep the new configuration.

markdownlint’s default rules report no issues. Neither does Vale, depending on how you configured it. Yet a reader who wants to set up a static address must get past an architecture explanation and a parameter reference table before reaching the procedure they’re looking for. On the other hand, a reader only interested in detailed routes parameters must know they’re buried in a task, after some conceptual information.

AI semantic analysis

A Markdown parser sees a heading, a paragraph, and a list and doesn’t raise any alarm. A style linter checks the prose’s conformance to the corporate style guide and gives a green light.

Don’t leave it to users to catch this kind of error: a language model can help assess whether the information type is consistent before users are presented with unstructured content they will struggle to use effectively.

Asked to classify each chunk of content and cite its source lines, an LLM might return:

dominant_purpose: procedure   # task title, imperative numbered steps
passages:
  - { lines: 3-4,   type: goal,      confidence: high }
  - { lines: 6-11,  type: concept,   confidence: high, about: renderer model }
  - { lines: 13-46, type: steps,     confidence: high, steps: [13, 14, 15, 41, 46] }
  - { lines: 17-29, type: example,   confidence: high, parent: step 3 }
  - { lines: 31-36, type: reference, confidence: high,
      parameters: [dhcp4, addresses, routes, nameservers] }
  - { lines: 38-39, type: concept,   confidence: medium, about: dhcp4 }
  - { lines: 43-44, type: concept,   confidence: low, alternative: step result }

As the confidence levels imply, we don’t get clear-cut answers as we would with a DTD. But running these automated tests in the CI pipeline could be inexpensive. And nothing prevents us from leveraging LLMs to propose further evaluations beyond what a classical DTD does, provided we treat the model’s answers as candidates rather than verdicts.

The candidate structure

The model then proposes a graph of what each passage does for the reader. The heading hierarchy, by contrast, would make How Netplan works the parent of the steps:

precedesnotenote?embeddedexplains dhcp4Static IP addressGoal • lines 3–4Open YAML fileStep 1 • line 13Disable DHCPStep 2 • line 14Add static settingsStep 3 • line 15Run netplan tryStep 4 • line 41ConfirmStep 5 • line 46Renderer modelConcept • lines 6–11DHCP + staticConcept • lines 38–39Rollback timerConcept? • lines 43–44Interface keysReference • lines 31–36dhcp4 • addressesroutes • nameservers

A three-column graph. Center column: a task goal, Static IP address, followed by five ordered steps: Open YAML file, Disable DHCP, Add static settings, Run netplan try, and Confirm. Left column: three concepts. Renderer model precedes the task goal and is flagged. DHCP plus static is a note attached to step 3 and explains the dhcp4 key. Rollback timer is a note attached to step 4, with an uncertain classification. Right column: a reference table, Interface keys, with four reference parameters (dhcp4, addresses, routes, nameservers), embedded in step 3 and flagged.

Candidate structure extracted from the Netplan topic. The goal and five steps form a valid sequence. A topic-level concept and a reference table of four parameters embedded in step 3 are flagged by the task contract, and the passage under step 4 has an uncertain classification that needs human review.

Deterministic checks

Code then applies a validation contract that the team declared for task topics:

  • R1: Steps form one ordered path from the goal. ok
  • R2: Every reference parameter declares its accepted values. ok
  • R3: Every concept relationship resolves to a declared node. ok: DHCP + static resolves to dhcp4.
  • R4: Only the goal and prerequisites precede the steps. nok
  • R5: A step embeds at most two reference parameters. nok: step 3 embeds four.
  • R6: A step note contains no more than two sentences, like DITA’s <info>. ok for lines 38–39; dunno for lines 43–44, because their classification as a step note is uncertain.

R6 works on two levels. Counting sentences is deterministic. Deciding whether a passage is a step note in the first place is the model’s probabilistic call, and lines 43–44 might instead describe the step result. None of these rules says mixing is wrong. R5 and R6 let some reference and concept material in, and their thresholds are editorial choices, not truths the validator discovers. The code only establishes that the extracted model breaks two declared rules, not that the model reflects the prose (see What structural validation can and cannot prove).

The validation report

The report gives the author something to act on:

configure-static-ip.md  nok  (contract: task topic)

nok    lines 6-11   concept, high confidence
       "How Netplan works" sits between the goal and step 1.
       R4: only the goal and prerequisites may precede the steps.
nok    lines 31-36  reference, 4 parameters, high confidence
       Step 3 embeds a table of Netplan keys.
       R5: a step may embed at most two reference parameters.
dunno  lines 43-44  step note or step result, low confidence
       R6: step note status uncertain; sentence count not decisive.
       Review: rationale for netplan try, or what the step does?
ok     R1 sequence, R2 accepted values, R3 concept links

The fix it suggests is an old DITA habit: keep the task to its goal and steps, move the table into a reference topic and How Netplan works into a concept topic, and link the three. If nobody will reuse those keys, keeping the table and relaxing R5 for this topic is also a valid answer. The report asks for a decision, not a split: the model proposes, the code checks, and a human decides.

Why Markdown syntax isn’t sufficient

A validator has to keep three layers apart:

  • Syntax: the headings, lists, tables, and code fences that Markdown declares.
  • Semantic relationships: the transitions, prerequisites, conceptual dependencies, and links between resources that the prose describes.
  • Structural rules: what those relationships must satisfy for a given reader purpose.

Documentation contains relationships that Markdown syntax alone cannot express. An ordered list may hold dependent steps, prioritized tips, or independent options. Headings rarely form a strict conceptual tree. A structural validator must therefore infer these relationships rather than assume them from formatting, and then apply rules that fit what the document is for.

Grounding validation in reader purposes

This approach borrows DITA’s discipline, not its guarantees. In DITA, the structure is fixed before a word is written. Here, it is inferred afterward:

Table 1: DITA compared with the proposed Markdown approach
DITAProposed Markdown approach
Author selects a topic type. AI infers the reader purpose.
DTD defines the permitted structure. AI proposes a candidate structure.
XML validation checks conformance. Deterministic code checks a validation contract.
Invalid structures can be rejected. Uncertain interpretations can be escalated.

No single structural model fits all documentation. A no-loops rule that protects a tutorial from circular prerequisites would wrongly reject a troubleshooting guide’s retry loop. A reachability rule that suits a procedure means nothing for an API reference. The rules must depend on what the reader is trying to do.

To keep that choice predictable, the model picks from a small, provisional taxonomy rather than inventing categories. Three terms stay distinct:

Reader purpose
What the reader wants to accomplish.
Candidate structure
The entities and relationships that the model infers from the content.
Validation contract
The purpose-specific rules that deterministic code applies to that structure.

Reader purposes describe outcomes, not titles: a “Getting started” page might be a tutorial or a procedure, and each needs different checks. Real documents also mix purposes, so a validator would need to evaluate sections separately or report the ambiguity.

The taxonomy has five reader purposes, each with its own candidate structure:

Table 2: Reader purposes, candidate structures, and validation contracts
Reader purposeReader goalCandidate structureValidation contract focus
Procedure Complete a task.Linear sequenceSingle start; goal reachable; no dead-end steps.
Tutorial Learn a skill progressively.Prerequisite structurePrerequisites come first; no circular dependencies.
Explanation Understand concepts and design rationale.Concept mapRelationships cite textual evidence; feedback loops allowed.
Reference Look up precise specifications.Entities with structured attributesSchema conformity; required attributes present; identifiers resolve.
Troubleshooting Diagnose and recover from a fault.Diagnostic flow with retry loopsChecks branch on success and failure; retry loops have exits; paths end in recovery or escalation.

Navigation maps and release timelines, which serve readers looking for the right page or tracking what changed, are outside the scope of this proposal. They could be considered in future extensions.

Candidate documentation structures

The five structures fall into three families: procedures, concepts, and reference and troubleshooting. Each figure shows the shape; the bullets give what a validator checks and what it can’t establish.

Procedures: linear sequences

Linear sequence: sequential procedure flow Structure: Linear sequence of steps. Semantic purpose: sequential procedure. A sequence of four rectangular boxes connected from top to bottom by arrows: Prepare, Install CLI, Authenticate, and Service running. Prepare Step 1 Install CLI Step 2 Authenticate Step 3 Service running Terminal

Structure: Linear sequence of steps. Semantic purpose: sequential procedure. A sequence of four rectangular boxes connected from top to bottom by arrows: Prepare, Install CLI, Authenticate, and Service running.

Figure 1: Linear sequence of steps. Each procedural step except the last leads to exactly one next step, forming an unbranching sequence that leads directly to a terminal outcome. Example: A linear deployment process.
  • Steps form a single path from one entry point to one goal. A validator checks that the goal is reachable and that no step is a dead end.
  • A numbered list isn’t proof of dependency, and real procedures often include optional or recovery steps.

Concepts: prerequisite structures and concept maps

Dependency structure: tutorial curriculum prerequisites Structure: Dependency structure without circular dependencies. Semantic purpose: tutorial curriculum. A prerequisite structure starting at Core syntax, branching into Data modeling and API configuration tracks, both required by Pipeline assembly, leading to Certified build. enables enables requires requires produces Core syntax Prerequisite Data modeling Concept track API configuration Reference track Pipeline assembly Integration Certified build Outcome

Structure: Dependency structure without circular dependencies. Semantic purpose: tutorial curriculum. A prerequisite structure starting at Core syntax, branching into Data modeling and API configuration tracks, both required by Pipeline assembly, leading to Certified build.

Figure 2: Tutorial prerequisites branch into parallel learning tracks and merge without circular dependencies. Example: Git commit history, where each commit points back to its parent commits and no commit can be its own ancestor.
  • Topics depend on prerequisites. A validator checks that prerequisites come first and rejects circular dependencies, which would deadlock the learner.
  • It models formal prerequisites, not how readers actually skip ahead.
Concept map: interconnected explanatory domain model Structure: Concept map with labeled relationships. Semantic purpose: explanatory domain model. Four concepts: Docs-as-Code, Information architecture, Semantic validator, and Reader persona, connected by labeled relationships including a feedback loop. runs implements verifies serves evaluates Docs-as-Code Methodology Semantic validator Tooling Information architecture Domain Reader persona Audience

Structure: Concept map with labeled relationships. Semantic purpose: explanatory domain model. Four concepts: Docs-as-Code, Information architecture, Semantic validator, and Reader persona, connected by labeled relationships including a feedback loop.

Figure 3: Concept map with labeled relationships. Explanatory topics connect concepts through labeled relationships, allowing feedback loops without requiring a hierarchy or a loop-free structure. Example: A service architecture with dependencies and feedback relationships.
  • Concepts connect through labeled relationships. A validator checks that every relationship links concepts grounded in the text.
  • Feedback loops are legitimate here, and a connected map doesn’t prove a clear explanation.

Reference and troubleshooting: structured entities and diagnostic flows

Entities with attributes: reference documentation with entity metadata Structure: Entities with structured attributes. Semantic purpose: reference documentation. Three reference entities with embedded attributes: POST endpoint, ValidationSchema, and DecisionReport, connected by labeled arrows that carry their own attributes. validates against [required] returns [async] POST /v1/validate auth: Bearer token rate-limit: 100 req/min ValidationSchema spec: Draft 2020-12 strict: true DecisionReport payload: application/json status: 200 | 422

Structure: Entities with structured attributes. Semantic purpose: reference documentation. Three reference entities with embedded attributes: POST endpoint, ValidationSchema, and DecisionReport, connected by labeled arrows that carry their own attributes.

Figure 4: Entities with structured attributes. Entities and relationships carry typed key-value metadata directly, enabling contract verification without adding extra boxes for each attribute. Example: A network map whose devices and connections carry properties such as capacity, latency, or configuration.
  • Entities such as API endpoints or CLI flags carry typed attributes. A validator checks schema conformity, required attributes (such as authentication), and identifier resolution.
  • It can’t judge whether the descriptions are clear, accurate, or complete.
Diagnostic flow: troubleshooting guide with intentional retry loop Structure: Diagnostic flow with an intentional retry loop. Semantic purpose: troubleshooting procedure. A troubleshooting flow starting at Check link status, branching on failure to Restart interface which loops back to Check link status, and branching on success to Test DNS resolution which leads to Service restored or Escalate. Link down Retry test Link up DNS failed DNS ok Check link status Diagnostic test Restart interface Corrective action Test DNS resolution Secondary check Escalate to netops Terminal fallback Service restored Terminal resolution

Structure: Diagnostic flow with an intentional retry loop. Semantic purpose: troubleshooting procedure. A troubleshooting flow starting at Check link status, branching on failure to Restart interface which loops back to Check link status, and branching on success to Test DNS resolution which leads to Service restored or Escalate.

Figure 5: Diagnostic flow with an intentional retry loop. An intermediate corrective action loops back to re-evaluate the diagnostic check, demonstrating why a ban on loops is not a universal rule. Example: An incident-response runbook that retries health checks before escalating.
  • Checks, corrective actions, and retry loops lead to resolution or escalation. A validator checks that every check branches on both results and that every retry loop has an exit.
  • It can show that readers won’t loop forever, not that they’ll fix the fault.

The validation pipeline

  1. Parse Markdown
    Deterministic

    Extract syntax tree and source locations

    • Standard parser extracts headings, ordered and unordered lists, tables, code fences, and links
    • Records exact source line numbers and byte offsets as immutable evidence anchors
  2. Classify reader purpose
    AI inference

    Match against the 5-purpose taxonomy

    • Classifies content: Procedure, Tutorial, Explanation, Reference, Troubleshooting
    • Records reader outcome, dominant purpose, and supporting passages

    Boundary limit: Ambiguous or mixed-purpose content returns an inconclusive result rather than an invented category

  3. Extract relationships
    AI inference

    Derive actions, conditions, and transitions

    • Identifies elements: diagnostic checks, conditions, corrective actions, and terminal outcomes
    • Identifies relationships: transitions from one step to the next and sequential dependencies
    • Binds every inferred relationship to specific source passages as evidentiary links

    Boundary limit: Evidence links ground the extracted structure in source text, but do not eliminate fallible semantic interpretation errors

  4. Validate against the contract
    Deterministic

    Check rules required by the validation contract

    • Tests purpose-specific rules: every step reachable, required terminal states, and complete branches
    • Rejects circular dependencies only where the validation contract prohibits them (e.g. tutorials, but not troubleshooting guides with intentional retries)
    • Verifies transitions without modifying or reinterpreting the extracted structure

    Boundary limit: Verifies the extracted structure only; does not prove that structure accurately captures author intent

  5. Report the result
    Deterministic

    Apply configured policy to checks and evidence

    • ok: required checks pass with verifiable evidence (exit 0 / pass build)
    • nok: a defined contract rule is demonstrably violated (exit 1 / block build)
    • dunno: ambiguity in prose or purpose precludes a definitive pass or fail (warn / request review)

A five-stage architecture connecting Markdown parsing, AI document classification, relationship extraction, deterministic structural checks, and CI enforcement.

Figure 6: Five-stage semantic validation pipeline. Stages 1, 4, and 5 run deterministic checks; stages 2 and 3 use AI inference to classify purpose and extract relationships from prose.

Each of the five stages has one owner:

  1. Parse (conventional tools): A Markdown parser extracts the syntax tree and source line numbers.
  2. Classify (AI): A model proposes the reader purpose, cites supporting passages, and flags ambiguity.
  3. Extract (AI): A model proposes the candidate structure, linking every relationship to its source passage.
  4. Validate (deterministic code): Code evaluates the candidate structure against the validation contract for its reader purpose.
  5. Report (CI): The pipeline combines the checks with extraction confidence and reports one of three outcomes.

AI proposes the structure; it never certifies it. And because inference is involved, pass or fail isn’t enough: the pipeline must also be able to say it doesn’t know.

Table 3: Validation outcomes and recommended actions
ResultWhat the validator establishedSuggested CI action
ok
[PASS]
Structural checks pass on a structure extracted with sufficient evidence.exit 0: Pass the build, or start in advisory mode.
nok
[FAIL]
A structural rule violation is established on a confidently extracted structure, such as an unreachable step, an incomplete branch, or a circular dependency where the contract prohibits it.exit 1: Block or flag the pull request, citing the elements, source lines, and violated rule.
dunno
[REVIEW]
Interpretation or extraction is too uncertain to establish a reliable result.warn: Route to human review, separate from confirmed violations.

What structural validation can and cannot prove

Four properties are easy to conflate:

Structural correctness of the candidate model
The extracted structure satisfies its validation contract. Deterministic code can establish this with certainty, but that doesn’t prove the model accurately represents the source documentation.
Completeness of extraction
The structure captures every step, condition, and relationship in the prose. A model can miss a branch phrased idiomatically, and the checker never sees what wasn’t extracted.
Semantic accuracy
The structure means what the prose means. A model can invent a step or invert a condition. Evidence links make such errors inspectable, not impossible.
Interpretation uncertainty
How far the pipeline can trust the extraction. When uncertainty is high, the honest result is dunno.

A structurally valid graph proves a property of the graph. It doesn’t prove that the AI read the source correctly, and even an accurate model doesn’t prove that the documentation is factually correct, complete, clear, or useful.

Testing the hypothesis

Authors keep writing plain Markdown, so an experiment could run in advisory mode on pull requests. It would start with procedures and troubleshooting guides, whose semantics are the most tractable, and combine a Markdown parser, a schema-constrained extraction prompt with source citations, a deterministic rule checker, and a runner that records results without blocking builds.

To show value beyond linting, it would need to measure:

  • Consistency: Does the model assign the same reader purpose across runs and model versions?
  • Agreement: How often does the extracted structure match what experienced technical writers see?
  • Sensitivity: When known defects (circular prerequisites, unreachable steps, missing exit conditions) are seeded into test fixtures, does the pipeline flag them?
  • Error rates: How often does it invent transitions or miss real branches?
  • Dunno rate: Is the share of inconclusive results low enough to be practical?
  • Baseline: Run conventional linters such as markdownlint and Vale on the same fixtures. Which defects does only the pipeline catch, and at what cost in false alarms?

Until those numbers exist, AI-assisted structural validation remains a hypothesis: promising, but still a hypothesis.

External sources

Hero image: “Measuring instrument” via rawpixel, originally from PxHere, licensed under CC0.

Continue reading

All articles
  • DITA

    A decade of Word 97 conference files, rebuilt for the web for $30

    How we transformed unstructured Word 97/2000 files into a decade-long web publication of conferences-using Markdown, Python, and GPT-powered automation for just $30.

    6 min read

  • Information Architecture

    Graph theory for information architecture

    A sitemap tells readers where pages live. A graph tells you how they relate: which pages are orphans, which ones hold the network together, and whether a reader can actually get from a question to a finished task.

    22 min read

  • Docs-as-Code

    CI/CD for a print artifact: one principle, two projects

    Running make locally is one thing. Guaranteeing that every contributor’s push or YAML edit on GitHub produces the same press-ready artifact on a clean machine with a pinned toolchain is another. How two print projects implement that guarantee with GitHub Actions.

    12 min read