Lecturenote AI

Every semester in graduate coursework I repeated the same chore. The professor's slides live in a PDF or PPT, what was actually said in class lives only in a recording, and my handwriting lives somewhere else again. Come exam week, studying starts with manually stitching the three back together, and that stitching is clerical work, not studying. So I fixed the physical unit of a note first: one page equals one portrait A4, the top third a slide image, the bottom two thirds lined notes. Reproduce the paper format identically on screen, in print, and in the PDF export, and fill those lines automatically from the transcript. That was the original goal. The real problem showed up the moment it worked. Notes came out, but I had no way to judge whether they were the right notes. Whether a sentence attached to a slide actually came from that slide, or was invented by the model, was something I had to check by eye every time. The open gate on this project was never a missing feature; it was always this.

I built it alone, and I was its user that semester. The dogfooding was dense - 64 notes, 1,891 slides, and 5 real courses accumulated in the local database, so a new annoyance surfaced every week. The trap was that every annoyance looked like a missing feature: no calendar, no timetable, no practice questions. My hands kept moving toward features. Cost and dependency I kept low from the start. LLM calls try a local Ollama first and fall back to Claude on failure, and with no AI key at all the whole pipeline still runs end to end in mock mode. In early June I converged the product direction once through a six-persona roundtable, choosing "the semester graph" over "slide-to-transcript alignment quality" as the axis, and freezing investment in the latter while keeping its code. Looking back, that decision quietly supplied the excuse for never measuring alignment accuracy.

One real user: me. There are two accounts in the database, but one is mine and one is a dummy test account. An early strategy document set a beta recruitment target, but I found no trace of it being executed - no signup logs, no interview notes, no feedback issues. So the only validation this tool has is one semester of my own use.

If slides and transcript are aligned at slide granularity and the note draft is filled in automatically, the clerical work of re-stitching materials before an exam disappears and that time converts into actual studying.

• Upload a PDF/PPT/PPTX, render each slide as an image, and extract its text. Printouts with two slides per page are detected and split top/bottom, and outer white margins are cropped against the content box
• Three ways in for the transcript - upload audio and run Whisper STT, upload a transcript file, or paste text directly. Notes can be generated with no recording at all
• Chunk and embed the transcript, match it per slide, and show a preview of the matching before anything is generated. Cover and table-of-contents slides are excluded automatically
• The note editor - slide-by-slide pager, autosave on blur, and per-slide revision history shown as a diff with restore to any past version
• A year > semester > course tree with a calendar home. Exams are entered as a week ("midterm, week 8") rather than a date, and the date is derived backward from the semester start
• An iPad pen layer (pen, highlighter, eraser, pressure) stored as normalized coordinates, rendered to an image and passed through a vision model so handwriting joins the note as text
• PDF export that composites slide and note onto one page, a per-course cheat sheet PDF, practice-question generation, and RAG Q&A over the project's own corpus - plus the operational skeleton around it: Google SSO with admin approval, per-account data isolation, plan and quota gating, read-only share links, and a public status page sampling seven components every five minutes

• Fixed the note unit as "one portrait A4, top third slide, bottom two thirds lined notes" and held it identical on screen, in print, and in PDF - this is something people ultimately print, so a layout that only looks right on screen is worthless
• Kept the path that costs money out of the default - LLM calls try local Ollama first and fall back to Claude when it isn't reachable, and with no AI key at all the whole system runs in mock mode. Being able to walk the pipeline from upload to note generation without a key means development isn't tied to cost and network on every run
• Did not make audio mandatory - transcript upload and direct paste sat alongside it from the start, and later direct paste became the default. In practice this decision quietly pushed the original concept, slides paired with a recording, off to the side
• Took exams as a week number rather than a date - the date is derived from semester start plus weekday plus week, so a new semester only needs its start date corrected. The cost is that semester-start calculation became a single point of failure, and in August it failed exactly there
• Computed quota usage as "projects created this month" instead of a separate counter - if creation fails and rolls back, the usage disappears with it. That removes an entire bug class (a counter that forgets to decrement) structurally rather than by care
• Ran all 116 PRs through feature branches with no direct pushes to main, even working alone - it keeps a revert unit around, and I used it: one feature went in, got reverted three minutes later, and came back in a different shape half an hour after that

• When aligning the STT transcript to slide boundaries, an offset drift must not attach notes to the wrong slide
• Matching has to stay robust across audio edge cases - Whisper misrecognition, long silences, overlapping speakers
• The model must not invent content absent from the transcript - in a single day on June 15 I fixed JSON scaffolding leaking into notes, meta keys bleeding into body text, the same sentence being generated repeatedly, and the model echoing the raw transcript, then froze each fix into defensive parsers
• A slide passed over with no speech (an empty transcript segment) is left blank rather than having a note forced onto it
• Generation parameters must not break the output schema - repeat_penalty at 1.3 penalized even the necessarily repeating key name in a JSON array, so the model drifted to different keys. I lowered it to 1.15 and moved deduplication into its own function, separating sampling from schema stability
• Date boundaries in the week-number arithmetic - the fall default of "first Monday of September" wiped out the entire first week of September in a year where the 1st falls on a Tuesday. The commit records the reproduction condition, the wrong value, the fixed value, and that spring semester was unaffected
• Progress and time remaining must not lie to the user - the ETA was growing as time passed, and I rebuilt it as a countdown. In background jobs, the credibility of the number on screen breaks before the work itself does
• How to measure per-slide note accuracy and prevent regressions - without a metric this becomes "keep building without deciding," the open gate of this project, and three months on I still have no answer

• 246 commits (2026-06-02 to 2026-08-14), distributed as 236 in June, 0 in July, 10 in August
• 116 merged PRs, all through feature branches, with zero direct pushes to main
• 70 API endpoints, 17,462 tracked source lines, 14 database tables, 18 Alembic migrations
• Automated tests: 0. Zero test files, zero pytest/vitest/jest/playwright dependencies, no CI configuration
• Local dogfooding data - 64 notes, 1,891 slides, 5 courses, 2 accounts (mine plus a dummy test account)
• Deployments: 0. No public URL, private repository, no payment integration (plan and quota exist as schema only)
• The note-edit-rate (NSM) endpoint is built but has never been run to produce a figure - not measured. Transcript rows are 0, and saved practice questions, cheat sheets, and share links are 0 each: features that exist in code but have never accumulated data locally

The honest headline is that the product grew and the gate didn't move. A two-feature note generator started in June became, in three months, a study management tool with a year/semester/course tree, a calendar, a timetable, week-derived exam dates, practice questions, cheat sheets, pen input with OCR, accounts and quotas, share links, and a public status page. 246 commits, 116 PRs, 70 endpoints. It is the largest product in my repositories. And the open gate I wrote down when I first framed this project - what do we measure per-slide note accuracy with - got not one line of an answer in those three months. I even built the edit-rate endpoint, then never ran it and never recorded a number. With no metric I couldn't answer "are the notes better than last month," so I kept substituting a question I could answer: what should I build next. That is precisely the "keep building without deciding" failure I had warned myself about in writing. There's a sharper signal too. The transcript tables in the local database are empty, which suggests the recent 64 notes were effectively produced from slide text alone. The core of the original hypothesis, slides paired with a recording, is barely exercised in my own real usage. I spent three months widening the periphery of a hypothesis whose center I never validated, and the size numbers don't cover that.

• I wasn't postponing measurement; I was postponing a decision. Without a way to score note accuracy I couldn't judge whether what I had was good enough, and without that judgement the only available move was to add more. Next time I'll build the scoring rubric for 20 notes before generating 100.
• I justified zero tests with speed and paid for it with a date regression in August. On another project I pulled the time arithmetic into pure functions and pinned the boundary cases with tests; here I didn't. So the semester-start regression was fixed in code with nothing guarding it, and I still have no guarantee the same defect won't return. At minimum the week arithmetic and the slide-split decision should have been pure functions with tests.
• Choosing the semester graph over alignment quality as the product axis in the June roundtable is defensible as direction, but its side effect was a license to stop measuring alignment accuracy. Even when you move the axis, you should keep a quality bar on the axis you're leaving.