# A decade of Word 97 conference files, rebuilt for the web for $30

<blockquote class="border-l-4 border-indigo-500 pl-4 py-2 my-6 bg-indigo-50/50 dark:bg-indigo-950/20 text-indigo-950 dark:text-indigo-200">
**Core workflow:** Rebuilding a decades-old conference archive into a modern web publication succeeds when AI is embedded inside a structured, verifiable engineering pipeline—coupling Pandoc format normalization and Python chunking with GPT-4o metadata enrichment, Git hunk reviews, and static Astro compilation.
</blockquote>

<div class="not-prose grid grid-cols-2 sm:grid-cols-4 gap-3 my-6">
  <div class="rounded-xl border border-gray-200 dark:border-slate-800 bg-white dark:bg-slate-900/60 p-4 text-center">
    <div class="text-xl sm:text-2xl font-bold text-gray-900 dark:text-white">1,500</div>
    <div class="text-xs uppercase tracking-wider text-gray-500 dark:text-slate-400 font-semibold mt-1">Source Word Files</div>
  </div>
  <div class="rounded-xl border border-gray-200 dark:border-slate-800 bg-white dark:bg-slate-900/60 p-4 text-center">
    <div class="text-xl sm:text-2xl font-bold text-gray-900 dark:text-white">2,500</div>
    <div class="text-xs uppercase tracking-wider text-gray-500 dark:text-slate-400 font-semibold mt-1">Web Pages Generated</div>
  </div>
  <div class="rounded-xl border border-gray-200 dark:border-slate-800 bg-white dark:bg-slate-900/60 p-4 text-center">
    <div class="text-xl sm:text-2xl font-bold text-gray-900 dark:text-white">1.8M</div>
    <div class="text-xs uppercase tracking-wider text-gray-500 dark:text-slate-400 font-semibold mt-1">Words Processed</div>
  </div>
  <div class="rounded-xl border border-gray-200 dark:border-slate-800 bg-white dark:bg-slate-900/60 p-4 text-center">
    <div class="text-xl sm:text-2xl font-bold text-gray-900 dark:text-white">~$30</div>
    <div class="text-xs uppercase tracking-wider text-gray-500 dark:text-slate-400 font-semibold mt-1">Total GPT-4o Cost</div>
  </div>
</div>

<div class="my-6">

</div>

## Preserving a legacy of conferences

A non-profit organization held an irreplaceable cultural archive: decades of spoken conference transcripts stored in legacy **Microsoft Word 97/2000 files**.

The collection was **rich in content but poor in structure**: monolithic documents spanning hours of speaking time without consistent headings, standardized metadata, or clean web formatting. Making them publicly accessible, searchable, and SEO-friendly required a scalable modernization pipeline rather than manual editorial rewriting.

<div class="grid grid-cols-1 md:grid-cols-2 gap-4 my-6">
  <ConceptCard title="Archive Strengths" subtitle="The Content Goldmine">
    Decades of authentic oral transcripts, deep historical and philosophical lectures, and substantial cultural volume preserved by dedicated non-profit members.
  </ConceptCard>
  <ConceptCard title="Archive Bottlenecks" subtitle="The Structural Barrier">
    Proprietary binary Word formats, missing section breaks, arbitrary title styles, zero machine-readable metadata, and files too large for web browsing.
  </ConceptCard>
</div>

This post details how we transformed that legacy corpus into a structured, searchable digital publication using **Pandoc, Python, GPT-4o, and Astro**, completing the entire processing run for roughly **$30**.

## 2. Splitting monolithic transcripts into sections

Publishing a single 50-page transcript as one web page creates an unreadable wall of text and undermines search discoverability. We needed **one focused Markdown file per conference section**.

We developed a custom **Python script** that parsed Markdown heading delimiters and split each large file into autonomous section files:

<figure class="my-6">
  ![Multicolored folders illustrating content chunking and modular filing.](https://redaction-technique.org/images/blog/turn-word-files-seo-optimized-web-pages-ai-large.webp)
  <figcaption class="text-sm text-gray-500 dark:text-slate-400 mt-2 text-center">Modular chunking: splitting monolithic transcripts into individual conference files ready for per-page metadata.</figcaption>
</figure>

This sectioning step delivered immediate downstream benefits:

* **Granular URLs:** Each conference topic received a dedicated, linkable URL slug.
* **Focused reading experience:** Readers could explore specific questions and answers without scrolling through tens of thousands of words.
* **Targeted SEO indexing:** Search engines could index individual themes and concepts rather than burying them in massive documents.

---

## 3. Enriching metadata with GPT-4o

With the transcripts divided into focused sections, we used GPT-4o strictly for **metadata enrichment and structural signposting**, keeping the historical transcript text untouched.

The script passed each section to GPT-4o to extract and generate:

* **Thematic keywords:** Identifying core subjects to power faceted archive search and tag browsing.
* **Astro YAML frontmatter:** Automatically assembling valid frontmatter:
  * `title`: Formatted for search engine results pages (SERPs).
  * `description`: Concise 150-character summary for Google snippet previews.
* **Canonical dates:** Parsing human-readable dates embedded within legacy file headers into machine-readable ISO timestamps (`YYYY-MM-DD`).
* **Intermediary subheadings:** Suggesting relevant H2/H3 headings to break up extended oral speeches into scannable reading blocks.

```yaml
---
title: "The Architecture of Meaning: 1999 Spring Conference (Part 2)"
description: "An inquiry into symbolic language and technical understanding from the 1999 conference transcripts."
publishDate: "1999-04-12"
tags: ["Hermeneutics", "Oral History", "Philosophy of Language"]
---
```

---

## 4. Iterative workflow: Scale only after validation

Automating metadata across 1,500 files required strict quality controls. We never ran the script across the full archive blind:

<div class="my-6">

</div>

<blockquote class="border-l-4 border-amber-500 pl-4 py-2 my-6 bg-amber-50/50 dark:bg-amber-950/20 text-amber-950 dark:text-amber-200">
**Engineering rule: scale only after validation.** Never run an unreviewed prompt across an entire archive. Testing on small batches and inspecting diffs ensures you catch prompt drift, schema deviations, and formatting anomalies before processing millions of tokens.
</blockquote>

This discipline of using Git diffs as an editorial safety net is identical to the human-supervised translation loop detailed in [Translating legacy French docs with DeepL and GPT-4o](https://redaction-technique.org/ai-translation-legacy-technical-docs).

---

## 5. The scale and economics of the transformation

By establishing clean batch automation, we processed decades of cultural records on a modest non-profit budget:

<div class="grid grid-cols-1 sm:grid-cols-2 gap-4 my-6">
  <ConceptCard title="Corpus Volume" subtitle="1.8M words across 1,500 files">
    Transformed into **2,500 structured web pages**, with full-text search, date filtering, and thematic tag browsing.
  </ConceptCard>
  <ConceptCard title="Processing Budget" subtitle="~$30 total API cost">
    Consumed roughly **6 million GPT-4o tokens** by restricting prompt scope to metadata extraction rather than full document rewriting.
  </ConceptCard>
  <ConceptCard title="Environmental Footprint" subtitle="~300 km car journey equivalent">
    Processing 6 million tokens generated an estimated carbon footprint comparable to a 300 km passenger car trip, underscoring the need for efficient chunking.
  </ConceptCard>
  <ConceptCard title="Publication Horizon" subtitle="Continuous releases through 2032">
    Rather than publishing 2,500 pages in one overwhelming release, the site publishes conferences daily to maintain steady visibility and discovery.
  </ConceptCard>
</div>

---

## 6. A living digital archive

Thanks to this workflow, the non-profit established a **self-sustaining publication pipeline** built entirely on [plain Markdown files stored under Git](https://redaction-technique.org/manage-content-in-files-not-databases):

* **Daily automated publishing:** A continuous stream of daily conference releases extending through 2032.
* **Faceted search & exploration:** Readers and researchers can search by keyword, speaker, or year without loading slow database queries.
* **Long-term preservation:** Plain text in Git ensures that the archive remains readable decades after proprietary Word and CMS versions become obsolete.
* **Tiered access:** Immediate full-archive exploration for authorized non-profit members via password-protected sections, alongside public daily releases.

---

## Lessons learned from legacy archive automation

<div class="space-y-4 my-6">
  <ConceptCard title="1. Cost control requires staged iteration" subtitle="Validate small, scale once">
    Running scripts on 10-file subsets and tuning prompts prevented costly prompt failures, keeping the total API cost below $30 for 1.8 million words.
  </ConceptCard>
  <ConceptCard title="2. Human-in-the-loop oversight is non-negotiable" subtitle="Git diff as verification">
    Automated metadata extraction is powerful, but inspecting diffs caught date-parsing anomalies and ambiguous topic titles that no automated test could detect.
  </ConceptCard>
  <ConceptCard title="3. Domain expertise shapes script accuracy" subtitle="BeautifulSoup integration">
    Guiding AI requires technical direction. Explicitly prompting GPT to use Python's **BeautifulSoup** library ensured robust parsing of legacy Word HTML fragments embedded within Markdown blocks.
  </ConceptCard>
  <ConceptCard title="4. Environmental awareness belongs in architecture" subtitle="Token efficiency matters">
    Multi-million-token runs carry real compute footprints. Structuring prompts to extract only frontmatter minimized unnecessary token generation.
  </ConceptCard>
  <ConceptCard title="5. AI is a collaborator, not an autonomous engine" subtitle="Leverage within constraints">
    The project succeeded because AI operated inside a deterministic pipeline bounded by Pandoc, Python file-handling, and Git version control.
  </ConceptCard>
</div>

---

## Next steps for the digital archive pipeline

While the Python script and GPT-4o prompts were tuned to minimize hallucinations, automated metadata generation requires ongoing human validation:

<blockquote class="border-l-4 border-blue-500 pl-4 py-2 my-6 bg-blue-50/50 dark:bg-slate-800/60 text-slate-900 dark:text-slate-100">
**Progressive verification:** Nothing guarantees 100% accuracy in LLM-generated summaries. Non-profit members are systematically reviewing published pages against audio recordings. Future exploration includes running local open-source models via frameworks like LangChain to perform automated input/output consistency checks offline.
</blockquote>

Ongoing roadmap initiatives include:

* **Thematic cross-linking:** Automatically suggesting related conference sessions based on keyword co-occurrence.
* **Multilingual executive summaries:** Generating concise French and Spanish summaries for international researchers.
* **Historical image indexing:** Associating archived event photographs with specific conference sections.

---

## Related reading

- [Transforming a corpus of 7,000 pages into living knowledge](https://redaction-technique.org/transforming-corpus-ai-living-knowledge) - scaling archival discovery and daily quote generation.
- [Translating legacy French docs with DeepL and GPT-4o](https://redaction-technique.org/ai-translation-legacy-technical-docs) - iterative human-supervised translation with Git diffs.
- [Manage content in files, not databases](https://redaction-technique.org/manage-content-in-files-not-databases) - the architectural foundation for Git-based publishing.
- [Source format](https://docs.redaction-technique.org/en/tech-writing-process/source-format/) - why the source format determines a document's reach.

## External sources

- [Pandoc: legacy Word to Markdown conversion](https://pandoc.org/)
- [BeautifulSoup for embedded HTML handling](https://www.crummy.com/software/BeautifulSoup/)
- [GPT-4o API documentation](https://platform.openai.com/docs)

<small>*Hero image: ["microsoft-office-lens-scanner"](https://www.flickr.com/photos/downloadsourcefr/16831563400) by [downloadsource.fr](https://www.flickr.com/photos/downloadsourcefr/), licensed under [CC BY 2.0](https://creativecommons.org/licenses/by/2.0/).*</small>

---

Source: https://redaction-technique.org/turn-word-files-seo-optimized-web-pages-ai