Blueprint-style drawing of a stack of scanned pages being read, sorted and filed into an index cabinet

The Secretary

A private document agent system. Bringing together every scan, document, or file I have. Everything gets read, sorted and indexed overnight, so I can find anything later just by asking.

Date

2026 – Present
Read it with your AI

Short on time? Have your AI read this page for you. Each button copies this exact prompt and opens your assistant. Paste it if it isn't filled in already.

Prompt

Read https://www.noahscott.org/work/secretary.md and give me the short version: what it is, what Noah did, and what stands out. Stick to what the page says.

Plain-text edition · llms.txt

Paperwork sucks and I hate not being able to find it WHEN I need it even more. So I built Secretary.

When I scan a physical document, drop in a file, or have Corvak add in additional documentation, it lands in my archive. Every night at 2:00 AM, Secretary picks up whatever is new, reads each page with a vision model running locally on my Mac Studio, chunks it with a multi-modal embedder, sorts it into the right category and indexes it. By morning it is searchable. Be it manually via the search console or by Corvak. The best part is that nothing leaves my own hardware.

To make this possible Secretary combines exact-word search with search by meaning and relational data, then re-ranks the results, so a vague description finds the right document even when I don’t remember a single word in it. It answers first and hands me the source document, forcing the system to have citations for every answer.

The first working version went live in early 2026, but it took a few weeks for the hard lesson to surface. For a stretch in the spring, the nightly filing quietly stopped working, and nothing told me. Results slowly degraded and it took a major retrieval failure to notice. The rebuilt system now self-repairs and learns from its failed retrievals.

So now I have a little code goblin that gets rewarded the more efficiently and effectively it spends its overnight time and gets me the correct answers when I ask.

More work

Leave a note

I’ll get back to you as soon as I can.