AI & Automation / PROJECT NOTES
Forum Corpus & Local Retrieval
A traceable local corpus pipeline for paginated threads, incremental updates, topic browsing, and compact exports.
PythonJSONRAGData pipeline
01 / THE INTENT
Long threads, multiple reply pages, and changing versions are difficult to search and maintain.
02 / THE FLOW
How it works
- 01Read pages
- 02Clean and deduplicate
- 03Validate and merge
- 04Browse or export
03 / THE WORK
What exists
01
Source and post structure retained per thread
02
Incremental refresh with old-version protection
03
Lightweight browser and compact text JSON
04 / CURRENT POSITION
Where the project stands
The pipeline and local browser exist. This case study describes the engineering; source material remains a private research corpus.
AI-assisted English translation of the Chinese project record.