Agent Template
Receipt & Invoice Extractor
Photograph receipts or drop in PDF invoices and get a structured expense table you can export as CSV - the images are read on your device and never uploaded anywhere.
receipts
invoices
ocr
vision
expenses
privacy
ozzo
Sep 03, 2026
1 use
Preview Mode
This is a preview with sample data. The template uses placeholders like
which will be replaced with actual agent data.
About This Template
Receipt & Invoice Extractor is a browser-executable AI agent template built on AgentOp. It runs entirely in the browser using Python (via Pyodide) and can be deployed without a server — just download the generated HTML file and open it locally or host it anywhere.
Topics
receipts
invoices
ocr
vision
expenses
privacy
Template Preview
Template Metadata
- Slug
- receipt-extractor
- Created By
- ozzo
- Created
- Sep 03, 2026
- Usage Count
- 1
Tags
receipts
invoices
ocr
vision
expenses
privacy
Code Statistics
- HTML Lines
- 57
- CSS Lines
- 58
- JS Lines
- 431
- Python Lines
- 330
Source Code
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{{ agent_name }}</title>
<style>{{ css_code }}</style>
</head>
<body>
<main class="rx">
<header class="rx__head">
<h1>{{ agent_name }}</h1>
<p class="rx__sub">{{ description }}</p>
</header>
<section class="rx__intake">
<label class="rx__drop" for="receipt-file">
<input id="receipt-file" type="file"
accept="image/*,.pdf,.txt" multiple hidden>
<span>📷 Photograph or drop receipts & invoices
<small>images are read on-device; PDFs are parsed exactly</small></span>
</label>
<div class="rx__intake-actions">
<button id="rx-demo" type="button" class="rx__ghost">Try a sample receipt</button>
<button id="rx-csv" type="button" class="rx__ghost" disabled>Export CSV</button>
<button id="rx-clear" type="button" class="rx__ghost" disabled>Clear</button>
</div>
<div id="rx-status" class="rx__status" role="status"></div>
<div id="rx-progress" class="rx__progress" hidden><span id="rx-bar"></span></div>
</section>
<section id="rx-summary" class="rx__summary" hidden></section>
<section class="rx__tablewrap">
<table class="rx__table" id="rx-table" hidden>
<thead>
<tr>
<th>Vendor</th><th>Date</th><th>Category</th>
<th class="rx--num">Tax</th><th class="rx--num">Total</th><th>Source</th>
</tr>
</thead>
<tbody id="rx-rows"></tbody>
</table>
</section>
<section id="results-container" class="rx__chat" aria-live="polite"></section>
<form id="rx-form" class="rx__form" autocomplete="off">
<input id="rx-input" type="text"
placeholder="Ask about your receipts, e.g. how much did I spend on fuel?"
required disabled>
<button id="rx-send" type="submit" disabled>Ask</button>
</form>
</main>
<script>{{ js_code }}</script>
</body>
</html>
:root { color-scheme: light dark; }
* { box-sizing: border-box; }
body { margin: 0; font-family: system-ui, -apple-system, Segoe UI, Roboto, sans-serif;
background: #f6f7f9; color: #17181c; }
.rx { max-width: 900px; margin: 0 auto; padding: 24px 16px 96px; }
.rx__head h1 { font-size: 1.6rem; margin: 0 0 4px; }
.rx__sub { margin: 0 0 20px; color: #6b6f76; }
.rx__drop { display: block; border: 2px dashed #c9ccd3; border-radius: 12px;
padding: 22px; text-align: center; cursor: pointer; background: #fff;
color: #3d414a; transition: border-color .15s, background .15s; }
.rx__drop:hover { border-color: #0f9d76; background: #f2fbf8; }
.rx__drop small { display: block; margin-top: 6px; color: #8b9098; font-size: .8rem; }
.rx__intake-actions { display: flex; gap: 8px; flex-wrap: wrap; margin: 10px 0 0; }
.rx__ghost { padding: 7px 13px; border: 1px solid #d3d6dd; border-radius: 8px;
background: #fff; color: #3d414a; font-size: .85rem; cursor: pointer; }
.rx__ghost:hover:not(:disabled) { border-color: #0f9d76; color: #0f9d76; }
.rx__ghost:disabled { opacity: .5; cursor: not-allowed; }
.rx__status { font-size: .85rem; color: #6b6f76; margin: 10px 2px 0; min-height: 1.2em; }
.rx__progress { height: 4px; border-radius: 3px; background: #e4e6ea; overflow: hidden;
margin: 8px 2px 0; }
.rx__progress span { display: block; height: 100%; width: 0; background: #0f9d76;
transition: width .2s; }
.rx__summary { display: flex; flex-wrap: wrap; gap: 8px; margin: 16px 0 4px; }
.rx__chip { background: #fff; border: 1px solid #e4e6ea; border-radius: 999px;
padding: 6px 13px; font-size: .85rem; }
.rx__chip b { font-variant-numeric: tabular-nums; }
.rx__tablewrap { overflow-x: auto; }
.rx__table { width: 100%; border-collapse: collapse; margin: 12px 0 8px;
background: #fff; border: 1px solid #e4e6ea; border-radius: 10px;
font-size: .9rem; }
.rx__table th, .rx__table td { padding: 9px 12px; text-align: left;
border-bottom: 1px solid #eef0f3; }
.rx__table th { font-size: .72rem; text-transform: uppercase; letter-spacing: .04em;
color: #8b9098; }
.rx__table tr:last-child td { border-bottom: 0; }
.rx--num { text-align: right; font-variant-numeric: tabular-nums; }
.rx__guess { color: #b06a00; cursor: help; }
.rx__src { color: #8b9098; font-size: .8rem; }
.rx__chat { display: flex; flex-direction: column; gap: 10px; margin: 16px 0; }
.rx__msg { padding: 10px 14px; border-radius: 12px; max-width: 85%;
white-space: pre-wrap; line-height: 1.5; }
.rx__msg--user { align-self: flex-end; background: #0f9d76; color: #fff; }
.rx__msg--assistant { align-self: flex-start; background: #fff; border: 1px solid #e4e6ea; }
.rx__form { position: fixed; left: 0; right: 0; bottom: 0; display: flex; gap: 8px;
padding: 12px 16px; background: #fff; border-top: 1px solid #e4e6ea; }
.rx__form input { flex: 1; padding: 12px 14px; border: 1px solid #d3d6dd;
border-radius: 10px; font-size: 1rem; }
.rx__form button { padding: 0 20px; border: 0; border-radius: 10px; background: #0f9d76;
color: #fff; font-weight: 600; cursor: pointer; }
.rx__form button:disabled { background: #b4bcc0; cursor: not-allowed; }
@media (prefers-color-scheme: dark) {
body { background: #15161a; color: #ecedf1; }
.rx__drop, .rx__ghost, .rx__chip, .rx__table, .rx__msg--assistant, .rx__form {
background: #1f2027; border-color: #32343d; color: #ecedf1; }
.rx__table th, .rx__table td { border-color: #2a2c34; }
.rx__form input { background: #15161a; color: #ecedf1; border-color: #32343d; }
.rx__progress { background: #2a2c34; }
}
// Receipt & Invoice Extractor UI. The OCR engine, Pyodide and the answer model
// are injected by the generator; this wires the intake controls to them and
// does the one thing that has to happen in the browser: turning a whole receipt
// image into the single-line crops the OCR model actually expects.
(function () {
const MAX_WIDTH = 1400; // downscale wide photos before analysis
const MAX_LINES = 28; // OCR budget per receipt (~0.7 s per line warm)
const MIN_BAND_H = 6; // ignore specks and rule lines
const GAP_TOL = 2; // blank rows tolerated inside one text line
const TARGET_H = 48; // upscale short crops for the recognizer
const LOCAL_WINDOW = 25; // width of the local-contrast neighbourhood, px
const LOCAL_K = 14; // how much darker than its neighbourhood ink must be
const MIN_INK_FRACTION = 0.004; // floor under the adaptive row threshold
const $ = (id) => document.getElementById(id);
let pyReady = false;
let busy = false;
window.addMessage = function (type, content) {
const el = document.createElement('div');
el.className = 'rx__msg rx__msg--' + (type === 'user' ? 'user' : 'assistant');
el.textContent = content;
$('results-container').appendChild(el);
el.scrollIntoView({ behavior: 'smooth', block: 'end' });
return el;
};
const setStatus = (msg) => { $('rx-status').textContent = msg; };
function setProgress(done, total) {
const bar = $('rx-progress');
if (!total) { bar.hidden = true; return; }
bar.hidden = false;
$('rx-bar').style.width = Math.round((done / total) * 100) + '%';
}
function enableChat(on) {
$('rx-input').disabled = !on;
$('rx-send').disabled = !on;
}
// --- image -> line strips ------------------------------------------------
// The pinned OCR model reads ONE line of printed text per call, so a receipt
// photo must be cut into lines first. Photographs (as opposed to scans) drove
// all three choices below; each was measured against real receipt photos,
// where the naive version found 1-2 bands for a whole receipt and the OCR
// text was garbage as a result.
// 1. Ink is judged against a LOCAL horizontal neighbourhood, not the page
// mean: a phone photo has a lighting gradient, and a global threshold
// marks the shadowed side as ink until every line merges into one band.
function rowInkProfile(gray, w, h) {
const half = Math.floor(LOCAL_WINDOW / 2);
const profile = new Float32Array(h);
const prefix = new Float32Array(w + 1);
for (let y = 0; y < h; y++) {
const row = y * w;
for (let x = 0; x < w; x++) prefix[x + 1] = prefix[x] + gray[row + x];
let inked = 0;
for (let x = 0; x < w; x++) {
const a = Math.max(0, x - half);
const b = Math.min(w, x + half + 1);
if (gray[row + x] < (prefix[b] - prefix[a]) / (b - a) - LOCAL_K) inked++;
}
profile[y] = inked / w;
}
return profile;
}
// 2. The text/gap split comes from Otsu over the row-ink profile, so the cut
// adapts per photo instead of trusting one hard-coded fraction of width.
function otsuThreshold(profile) {
let peak = 0;
for (let i = 0; i < profile.length; i++) if (profile[i] > peak) peak = profile[i];
if (peak <= 0) return 0;
const BINS = 64;
const hist = new Float64Array(BINS);
for (let i = 0; i < profile.length; i++) {
hist[Math.min(BINS - 1, Math.floor((profile[i] / peak) * BINS))]++;
}
const value = (bin) => ((bin + 0.5) / BINS) * peak;
let grand = 0;
for (let b = 0; b < BINS; b++) grand += hist[b] * value(b);
let below = 0, sumBelow = 0, best = 0, bestVariance = -1;
for (let b = 0; b < BINS; b++) {
below += hist[b];
const above = profile.length - below;
if (below === 0 || above === 0) continue;
sumBelow += hist[b] * value(b);
const meanBelow = sumBelow / below;
const meanAbove = (grand - sumBelow) / above;
const variance = below * above * (meanBelow - meanAbove) * (meanBelow - meanAbove);
if (variance > bestVariance) { bestVariance = variance; best = value(b + 1); }
}
return best;
}
function findBands(profile, h) {
const threshold = Math.max(otsuThreshold(profile), MIN_INK_FRACTION);
const bands = [];
let start = -1, gap = 0;
for (let y = 0; y < h; y++) {
if (profile[y] >= threshold) { if (start < 0) start = y; gap = 0; }
else if (start >= 0) {
gap++;
if (gap > GAP_TOL) { bands.push([start, y - gap]); start = -1; gap = 0; }
}
}
if (start >= 0) bands.push([start, h - 1]);
return bands.filter(([top, bottom]) => bottom - top + 1 >= MIN_BAND_H);
}
// 3. A band far taller than its neighbours is a merged block (crumpled paper,
// touching lines) - re-cut it evenly rather than handing the recognizer a
// paragraph, which it answers with a single line of text.
function splitTallBands(bands) {
if (bands.length < 2) return bands;
const heights = bands.map(([t, b]) => b - t + 1).sort((x, y) => x - y);
const median = heights[Math.floor(heights.length / 2)] || 1;
const out = [];
for (const [top, bottom] of bands) {
const height = bottom - top + 1;
const parts = Math.round(height / median);
if (height > median * 2 && parts > 1) {
const step = height / parts;
for (let i = 0; i < parts; i++) {
out.push([Math.round(top + i * step), Math.round(top + (i + 1) * step) - 1]);
}
} else {
out.push([top, bottom]);
}
}
return out;
}
function lineStrips(img) {
const scale = Math.min(1, MAX_WIDTH / img.naturalWidth);
const w = Math.max(1, Math.round(img.naturalWidth * scale));
const h = Math.max(1, Math.round(img.naturalHeight * scale));
const page = document.createElement('canvas');
page.width = w; page.height = h;
const ctx = page.getContext('2d', { willReadFrequently: true });
ctx.drawImage(img, 0, 0, w, h);
const pixels = ctx.getImageData(0, 0, w, h).data;
const gray = new Float32Array(w * h);
let sum = 0;
for (let i = 0, p = 0; i < gray.length; i++, p += 4) {
gray[i] = pixels[p] * 0.299 + pixels[p + 1] * 0.587 + pixels[p + 2] * 0.114;
sum += gray[i];
}
// A white-on-black screenshot is inverted once here, so everything below
// can assume dark text on light paper.
const darkPage = sum / gray.length < 110;
if (darkPage) for (let i = 0; i < gray.length; i++) gray[i] = 255 - gray[i];
let keep = splitTallBands(findBands(rowInkProfile(gray, w, h), h));
// A receipt's two most valuable lines are the vendor (top) and the total
// (bottom), so an over-long receipt keeps both ends rather than the first
// MAX_LINES lines - which would drop the total every time.
if (keep.length > MAX_LINES) {
const head = Math.ceil(MAX_LINES / 2);
keep = keep.slice(0, head).concat(keep.slice(keep.length - (MAX_LINES - head)));
}
const strips = [];
for (const [top, bottom] of keep) {
const y0 = Math.max(0, top - 3);
const y1 = Math.min(h - 1, bottom + 3);
const bandH = y1 - y0 + 1;
const up = Math.max(1, Math.min(4, TARGET_H / bandH));
const crop = document.createElement('canvas');
crop.width = Math.round(w * up);
crop.height = Math.round(bandH * up);
const cctx = crop.getContext('2d', { willReadFrequently: true });
cctx.imageSmoothingQuality = 'high';
cctx.drawImage(page, 0, y0, w, bandH, 0, 0, crop.width, crop.height);
if (darkPage) { // recognizer wants dark-on-light
const strip = cctx.getImageData(0, 0, crop.width, crop.height);
for (let p = 0; p < strip.data.length; p += 4) {
strip.data[p] = 255 - strip.data[p];
strip.data[p + 1] = 255 - strip.data[p + 1];
strip.data[p + 2] = 255 - strip.data[p + 2];
}
cctx.putImageData(strip, 0, 0);
}
strips.push(crop.toDataURL('image/png'));
}
// Nothing segmented (a very low-contrast photo) - let the model see it whole.
return strips.length ? strips : [page.toDataURL('image/png')];
}
function loadImage(src) {
return new Promise((resolve, reject) => {
const img = new Image();
img.onload = () => resolve(img);
img.onerror = () => reject(new Error('could not decode this image'));
img.src = src;
});
}
const readAsDataURL = (file) => new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onload = () => resolve(reader.result);
reader.onerror = () => reject(new Error('could not read this file'));
reader.readAsDataURL(file);
});
async function ocrImage(src, label) {
const strips = lineStrips(await loadImage(src));
const lines = [];
for (let i = 0; i < strips.length; i++) {
setStatus('Reading ' + label + ' - line ' + (i + 1) + ' of ' + strips.length + '...');
setProgress(i, strips.length);
window.pyodide.globals.set('__strip', strips[i]);
try {
// One unreadable crop must not cost the whole receipt.
const text = await window.pyodide.runPythonAsync('await _ocr_line(__strip)');
if (text && text.trim()) lines.push(text.trim());
} catch (err) {
console.warn('[receipts] line ' + (i + 1) + ' failed', err);
}
}
setProgress(strips.length, strips.length);
return lines.join('\n');
}
async function textFor(file) {
if (/\.pdf$/i.test(file.name)) {
setStatus('Parsing "' + file.name + '"...');
const bytes = new Uint8Array(await file.arrayBuffer());
window.pyodide.globals.set('__pdf_bytes', bytes);
return await window.pyodide.runPythonAsync(
'_extract_pdf_text(bytes(__pdf_bytes.to_py()))'
);
}
if (/^image\//.test(file.type)) {
return await ocrImage(await readAsDataURL(file), '"' + file.name + '"');
}
return await file.text();
}
// --- table + summary -----------------------------------------------------
const money = (value, currency) =>
value === null || value === undefined
? '-'
: (currency ? currency + ' ' : '') + Number(value).toFixed(2);
async function refresh() {
const rows = JSON.parse(await window.pyodide.runPythonAsync('_rows_json()'));
const body = $('rx-rows');
body.textContent = '';
for (const row of rows) {
const tr = document.createElement('tr');
const cells = [
[row.vendor || '(unknown)', ''],
[row.date || '(no date)', ''],
[row.category || 'other', ''],
[money(row.tax, ''), 'rx--num'],
[money(row.total, row.currency), 'rx--num'],
[row.source || '', 'rx__src'],
];
cells.forEach(([text, cls], index) => {
const td = document.createElement('td');
td.textContent = text;
if (cls) td.className = cls;
// Flag a total that was not found on a line naming it.
if (index === 4 && row.total !== null && row.total_source !== 'total line') {
td.classList.add('rx__guess');
td.title = 'No "total" line found - this is the largest amount on the receipt.';
td.textContent = text + ' ?';
}
tr.appendChild(td);
});
body.appendChild(tr);
}
$('rx-table').hidden = rows.length === 0;
$('rx-csv').disabled = rows.length === 0;
$('rx-clear').disabled = rows.length === 0;
const totals = JSON.parse(await window.pyodide.runPythonAsync('_summary()'));
const summary = $('rx-summary');
summary.textContent = '';
summary.hidden = rows.length === 0;
const chip = (label, value) => {
const el = document.createElement('span');
el.className = 'rx__chip';
el.append(label + ' ');
const strong = document.createElement('b');
strong.textContent = value;
el.appendChild(strong);
summary.appendChild(el);
};
chip('Receipts', String(totals.count));
Object.entries(totals.by_currency).forEach(([code, value]) =>
chip('Total ' + code, value.toFixed(2)));
Object.entries(totals.by_category)
.sort((a, b) => b[1] - a[1]).slice(0, 4)
.forEach(([name, value]) => chip(name, value.toFixed(2)));
}
async function addReceipt(source, text) {
if (!text || !text.trim()) {
setStatus('No readable text in "' + source + '".');
return;
}
setStatus('Extracting fields from "' + source + '"...');
window.pyodide.globals.set('__src', source);
window.pyodide.globals.set('__text', text);
await window.pyodide.runPythonAsync('await _extract_receipt(__src, __text)');
await refresh();
enableChat(true);
setStatus('Added "' + source + '". Add more, or ask a question below.');
}
async function ingest(files) {
if (busy) return;
busy = true;
try {
for (const file of files) {
try { await addReceipt(file.name, await textFor(file)); }
catch (err) { setStatus('Could not read "' + file.name + '": ' + err); }
}
} finally {
busy = false;
setProgress(0, 0);
}
}
// --- sample receipt ------------------------------------------------------
// Drawn on a canvas and then put through the *same* OCR path, so the demo
// exercises the real pipeline rather than injecting text behind its back.
function sampleReceiptImage() {
const canvas = document.createElement('canvas');
canvas.width = 460; canvas.height = 420;
const ctx = canvas.getContext('2d');
ctx.fillStyle = '#ffffff';
ctx.fillRect(0, 0, canvas.width, canvas.height);
ctx.fillStyle = '#000000';
const lines = [
['CAFE MILANO', '600 26px monospace'],
['Bahnhofstrasse 12, Zurich', '18px monospace'],
['Date: 14/08/2026', '18px monospace'],
['', '18px monospace'],
['2x Espresso 7.00', '20px monospace'],
['1x Focaccia 8.50', '20px monospace'],
['', '18px monospace'],
['Subtotal 15.50', '20px monospace'],
['VAT 8.1% 1.26', '20px monospace'],
['TOTAL CHF 16.76', '600 22px monospace'],
];
let y = 48;
for (const [text, font] of lines) {
ctx.font = font;
if (text) ctx.fillText(text, 26, y);
y += 36;
}
return canvas.toDataURL('image/png');
}
// --- CSV -----------------------------------------------------------------
const csvCell = (value) =>
'"' + String(value === null || value === undefined ? '' : value).replace(/"/g, '""') + '"';
async function exportCsv() {
const rows = JSON.parse(await window.pyodide.runPythonAsync('_rows_json()'));
const header = ['vendor', 'date', 'category', 'currency', 'tax', 'total',
'total_source', 'source'];
const csv = [header.join(',')]
.concat(rows.map((row) => header.map((key) => csvCell(row[key])).join(',')))
.join('\n');
const url = URL.createObjectURL(new Blob([csv], { type: 'text/csv' }));
const link = document.createElement('a');
link.href = url;
link.download = 'receipts.csv';
link.click();
setTimeout(() => URL.revokeObjectURL(url), 1000);
}
// --- wiring --------------------------------------------------------------
$('receipt-file').addEventListener('change', async (event) => {
if (!pyReady) { setStatus('Still starting up - one moment...'); return; }
await ingest(event.target.files);
event.target.value = '';
});
$('rx-demo').addEventListener('click', async () => {
if (!pyReady || busy) { setStatus('Still starting up - one moment...'); return; }
busy = true;
try {
const text = await ocrImage(sampleReceiptImage(), 'the sample receipt');
await addReceipt('sample-receipt.png', text);
} catch (err) {
setStatus('Sample failed: ' + err);
} finally {
busy = false;
setProgress(0, 0);
}
});
$('rx-csv').addEventListener('click', exportCsv);
$('rx-clear').addEventListener('click', async () => {
await window.pyodide.runPythonAsync('_reset()');
await refresh();
enableChat(false);
setStatus('Cleared. Nothing left in this browser.');
});
$('rx-form').addEventListener('submit', async (event) => {
event.preventDefault();
const question = $('rx-input').value.trim();
if (!question || !pyReady) return;
window.addMessage('user', question);
$('rx-input').value = '';
enableChat(false);
try {
window.pyodide.globals.set('__q', question);
const answer = await window.pyodide.runPythonAsync('await process_user_query(__q)');
if (answer) window.addMessage('assistant', answer);
} catch (err) {
window.addMessage('assistant', 'Sorry, something went wrong: ' + err);
} finally {
enableChat(true);
$('rx-input').focus();
}
});
document.addEventListener('pyodide-ready', () => {
pyReady = true;
setStatus('Ready. Add a receipt photo or a PDF invoice to begin.');
});
})();
import json
import re
# How many extracted rows are shown to the model when answering questions.
MAX_ROWS_IN_PROMPT = [[[MAX_ROWS_IN_PROMPT|60]]]
# Hard cap on the facts sheet handed to the model (small local context windows).
MAX_SHEET_CHARS = [[[MAX_SHEET_CHARS|6000]]]
# How much receipt text the vendor/category classifier sees.
CLASSIFY_CHARS = [[[CLASSIFY_CHARS|700]]]
CATEGORIES = [
"groceries", "restaurant", "fuel", "travel", "lodging", "software",
"office", "utilities", "health", "other",
]
# One dict per extracted receipt (see _scan_receipt for the shape).
_RECEIPTS = []
_CURRENCY_SYMBOLS = {"$": "USD", "\u20ac": "EUR", "\u00a3": "GBP",
"\u20ba": "TRY", "\u00a5": "JPY"}
_CURRENCY_CODE_RE = re.compile(
r"\b(USD|EUR|GBP|TRY|CHF|JPY|CAD|AUD|SEK|NOK|DKK|PLN)\b"
)
# Money, and only money: 1.234,56 / 1,234.56 / 249.00 / $254.
#
# A bare integer is deliberately NOT an amount unless a currency symbol is
# attached to it. Receipts are full of long digit strings that are not money —
# UPCs, product codes, store and phone numbers — and browser testing on real
# photographed receipts showed exactly that failure: "#60101" and
# "04900005375" were picked up as the largest amount on the page and reported
# as the total. Requiring a two-digit decimal, thousands grouping, or an
# adjacent symbol is what separates "$2.18" from a barcode.
_AMOUNT_RE = re.compile(
r"(?<![0-9])"
r"(?:"
r"[$\u20ac\u00a3\u20ba\u00a5]\s?[0-9]{1,6}(?![0-9.,])" # symbol + whole units (JPY)
r"|[0-9]{1,3}(?:[.,\s][0-9]{3})+(?:[.,][0-9]{2})?" # grouped thousands
r"|[0-9]{1,6}[.,][0-9]{2}" # plain decimal amount
r")"
r"(?![0-9])"
)
_TOTAL_RE = re.compile(
r"grand total|total due|amount due|balance due|amount paid|to pay|\btotal\b"
r"|\btoplam\b|\bgesamt\b|\bimporte\b", re.I
)
_SUBTOTAL_RE = re.compile(r"sub\s?-?total|net total|\bnet\b", re.I)
_TAX_RE = re.compile(r"\b(vat|tax|gst|hst|kdv|iva|mwst|tva)\b", re.I)
_PERCENT_RE = re.compile(r"[0-9]+(?:[.,][0-9]+)?\s*%")
_MONTHS = {
"jan": 1, "feb": 2, "mar": 3, "apr": 4, "may": 5, "jun": 6,
"jul": 7, "aug": 8, "sep": 9, "oct": 10, "nov": 11, "dec": 12,
}
_ISO_DATE_RE = re.compile(r"(?<![0-9])(20[0-9]{2})[-/.]([01]?[0-9])[-/.]([0-3]?[0-9])(?![0-9])")
_DMY_DATE_RE = re.compile(r"(?<![0-9])([0-3]?[0-9])[-/.]([0-3]?[0-9])[-/.]((?:20)?[0-9]{2})(?![0-9])")
_NAMED_DATE_RE = re.compile(
r"(?<![0-9])([0-3]?[0-9])\s+([A-Za-z]{3,9})\.?\s+(20[0-9]{2})(?![0-9])"
)
def _parse_amount(raw):
"""Parse one money-ish token into a float, or None.
Handles both separator conventions: whichever of "." / "," comes last and
is followed by one or two digits is the decimal point; everything else is
thousands grouping.
"""
# Drop the currency symbol the amount pattern may have captured, plus any
# stray spacing; what is left must be digits and separators only.
s = re.sub(r"[^0-9.,]", "", str(raw))
if not s:
return None
decimal_at = max(s.rfind("."), s.rfind(","))
if decimal_at != -1 and len(s) - decimal_at - 1 in (1, 2):
whole = re.sub(r"[.,]", "", s[:decimal_at])
s = whole + "." + s[decimal_at + 1:]
else:
s = re.sub(r"[.,]", "", s)
try:
return round(float(s), 2)
except ValueError:
return None
def _amounts_in(line):
"""Every money-ish amount on one line.
Percentages are stripped first, so "VAT 8.1% 1.26" yields the 1.26 charged
and not the 8.1 rate.
"""
found = []
for match in _AMOUNT_RE.finditer(_PERCENT_RE.sub(" ", line)):
value = _parse_amount(match.group(0))
if value is not None:
found.append(value)
return found
def _normalize_date(text):
"""Return YYYY-MM-DD when the date is unambiguous, else the matched text.
Deliberately does NOT guess between 03/04/2026 readings — an expense table
that silently swaps day and month is worse than one that shows what it saw.
"""
match = _ISO_DATE_RE.search(text)
if match:
year, month, day = (int(g) for g in match.groups())
if 1 <= month <= 12 and 1 <= day <= 31:
return "%04d-%02d-%02d" % (year, month, day)
match = _NAMED_DATE_RE.search(text)
if match:
day, name, year = match.group(1), match.group(2)[:3].lower(), match.group(3)
if name in _MONTHS:
return "%s-%02d-%02d" % (year, _MONTHS[name], int(day))
match = _DMY_DATE_RE.search(text)
if match:
first, second, year = (int(g) for g in match.groups())
if year < 100:
year += 2000
if first > 12 and second <= 12:
return "%04d-%02d-%02d" % (year, second, first)
if second > 12 and first <= 12:
return "%04d-%02d-%02d" % (year, first, second)
return match.group(0) # ambiguous — report it verbatim
return ""
def _detect_currency(text):
match = _CURRENCY_CODE_RE.search(text)
if match:
return match.group(1).upper()
for symbol, code in _CURRENCY_SYMBOLS.items():
if symbol in text:
return code
return ""
def _guess_vendor(lines):
"""Receipts put the merchant at the top — take the first line that reads
like a name rather than an address, a date or a price."""
for line in lines[:6]:
letters = sum(character.isalpha() for character in line)
if letters < 3 or letters < len(line) / 3:
continue
if _TOTAL_RE.search(line) or _normalize_date(line):
continue
return line[:60]
return ""
def _find_total(lines):
"""(amount, how it was found) for the bottom-line total.
A line that names a total wins over a big number found anywhere, and the
LAST such line wins over earlier ones (grand totals come last).
"""
candidate = None
for line in lines:
if _SUBTOTAL_RE.search(line) or not _TOTAL_RE.search(line):
continue
amounts = _amounts_in(line)
if amounts:
candidate = max(amounts)
if candidate is not None:
return candidate, "total line"
everything = [value for line in lines for value in _amounts_in(line)]
if everything:
return max(everything), "largest amount"
return None, ""
def _find_tax(lines):
tax = None
for line in lines:
if not _TAX_RE.search(line):
continue
amounts = _amounts_in(line)
if amounts:
tax = max(amounts)
return tax
def _scan_receipt(text):
"""Deterministic extraction of one receipt's text. No model involved."""
lines = [" ".join(raw.split()) for raw in text.splitlines()]
lines = [line for line in lines if line]
total, total_source = _find_total(lines)
return {
"vendor": _guess_vendor(lines),
"date": _normalize_date(text),
"currency": _detect_currency(text),
"total": total,
"total_source": total_source,
"tax": _find_tax(lines),
"category": "",
"lines": len(lines),
}
def _first_json_object(reply):
"""Pull the first {...} block out of a model reply, or return None."""
start = reply.find("{")
end = reply.rfind("}")
if start == -1 or end <= start:
return None
try:
parsed = json.loads(reply[start:end + 1])
except (ValueError, TypeError):
return None
return parsed if isinstance(parsed, dict) else None
async def _classify(row, text):
"""Ask the model for the two fuzzy fields only: vendor name and category.
Best-effort by construction — any failure keeps the deterministic guess.
Amounts are never sent back through the model.
"""
prompt = (
"Read this receipt text and reply with ONLY a JSON object, no prose:\n"
'{"vendor": "<merchant name>", "category": "<one of: '
+ ", ".join(CATEGORIES) + '>"}\n\nRECEIPT TEXT:\n'
+ text[:CLASSIFY_CHARS]
)
reply = await agentop_llm.generate(prompt, "You extract fields as strict JSON.")
parsed = _first_json_object(reply or "")
if not parsed:
return row["vendor"], "other"
vendor = str(parsed.get("vendor") or row["vendor"]).strip()[:60]
category = str(parsed.get("category") or "").strip().lower()
return vendor or row["vendor"], category if category in CATEGORIES else "other"
async def _extract_receipt(source, text):
"""(JS-callable) Turn one receipt's text into a table row.
Underscore-prefixed so it is never exposed to the LLM as a tool. Returns
the row as JSON for the UI.
"""
row = _scan_receipt(text)
row["source"] = source
try:
row["vendor"], row["category"] = await _classify(row, text)
except Exception: # noqa: BLE001 — enrichment is optional, never fatal
row["category"] = row["category"] or "other"
_RECEIPTS.append(row)
return json.dumps(row)
def _rows_json():
"""(JS-callable) Every extracted row, for the table and the CSV export."""
return json.dumps(_RECEIPTS)
def _summary():
"""(JS-callable) Totals per currency and per category, for the summary bar."""
by_currency, by_category = {}, {}
for row in _RECEIPTS:
if row.get("total") is None:
continue
currency = row.get("currency") or "?"
by_currency[currency] = round(by_currency.get(currency, 0) + row["total"], 2)
category = row.get("category") or "other"
by_category[category] = round(by_category.get(category, 0) + row["total"], 2)
return json.dumps(
{"count": len(_RECEIPTS), "by_currency": by_currency, "by_category": by_category}
)
def _reset():
"""(JS-callable) Clear the table."""
_RECEIPTS.clear()
return "0"
def _facts_sheet():
"""The extracted table as text, capped for small context windows."""
rows = _RECEIPTS[-MAX_ROWS_IN_PROMPT:]
out = ["#. vendor | date | category | total | currency | tax | file"]
for index, row in enumerate(rows, 1):
out.append(
"%d. %s | %s | %s | %s | %s | %s | %s"
% (
index,
row.get("vendor") or "(unknown)",
row.get("date") or "(no date)",
row.get("category") or "other",
"(not found)" if row.get("total") is None else row["total"],
row.get("currency") or "?",
"-" if row.get("tax") is None else row["tax"],
row.get("source") or "",
)
)
return "\n".join(out)[:MAX_SHEET_CHARS]
async def process_user_query(query):
"""Answer questions about the extracted receipts, and only about them.
Overrides the default router: the data is a small structured table, so it
is handed over whole rather than retrieved (no embeddings needed).
"""
if not _RECEIPTS:
return (
"No receipts yet. Add a photo or a PDF invoice above (or press "
"Try a sample receipt) and I will pull out the vendor, date and total."
)
grounded_prompt = (
"Answer the QUESTION using ONLY the EXPENSE TABLE below, which was "
"extracted from the user's own receipts. Amounts in the table are exact "
"- never invent or re-estimate one, and say so if a figure is missing. "
"When you add figures up, show the arithmetic briefly.\n\n"
"EXPENSE TABLE:\n" + _facts_sheet() + "\n\nQUESTION: " + query
)
return await agentop_llm.generate(
grounded_prompt, globals().get("TEMPLATE_SYSTEM_PROMPT", "")
)
async def _ocr_line(image_url):
"""(JS-callable) Read one cropped line strip with the on-device OCR model."""
return (await agentop_ml.ocr(image_url)) or ""
def _extract_pdf_text(pdf_bytes):
"""(JS-callable) Extract text from a digital invoice PDF via pypdf."""
import io
from pypdf import PdfReader
reader = PdfReader(io.BytesIO(pdf_bytes))
return "\n\n".join((page.extract_text() or "") for page in reader.pages)
Name your agent
Based on . You'll get your own private copy to try and customize.