01New developmentWeb / AI2026.09 ~ 2026.10
AI Document and Data Analysis Platform
Built a full-stack platform that searches documents and answers questions using Gemma 4. Improved how it retrieves supporting evidence and evaluates the answers it produces.
- Scope
Full-stack development
Search and answer quality, processing performance
- Technology stack
- React
- TypeScript
- Tailwind CSS
- Vite
- TanStack Query
- Zustand
- Python
- Django
- MariaDB
- SQLGlot
- Gemma 4
- Ragas
- tokenizers
- Vitest
Requirements
A document-grounded AI feature needed trustworthy evaluations as well as useful answers. Long answers exceeded the evaluation input limit, and the evaluator sometimes rewrote statements in ways that changed what was being assessed.
Decisions and verification
Fix the evaluation targets and validate the verdicts
- Decision
- Fixed the target statements on the server for Ragas evaluation and matched the returned evaluation results by statement ID. Batched long inputs and checked quotations, supporting evidence, and answer coverage separately.
- Checks
- Completed evaluation of 25 claims extracted from an answer about a real document in five batches. Checks reject missing or duplicate IDs and invalid response types before malformed verdicts can be stored as valid results.
Separate model latency from application overhead
- Decision
- Profiled internal processing with fixed model responses, then removed repeated token counting and reused document pages already processed within the request.
- Checks
- After warmup, 20 runs before and after the change showed median internal processing fall from 2,969ms to 672ms. Model call counts stayed unchanged.
Results and delivery scope
Long-answer evaluations can finish, and repeated computation causes less internal delay. Answers appear first, followed by asynchronous Ragas evaluation, so users do not wait for evaluation to finish.
- Median internal application processing time
- 2,969ms → 672ms
- Compared application processing with fixed model responses: 20 runs per condition after warm-up, excluding model and network wait time.
- Completed evaluation of a long answer within the input limit
- 25 items evaluated
- Evaluated a Gemma 4 answer about a real document in 5 batches. The largest request used 8,164 tokens, below the 8,192-token input limit.
- No need to wait for quality evaluation
- Display the answer first, then evaluate it separately
- Users can read the answer before Ragas finishes. The next evaluation request is deferred while a new user question is being processed.
Implementation details and decisions
Problem
Sending only the matched passage to the model left out conditions and exceptions in nearby paragraphs. Questions with several parts sometimes returned source text instead of explanations, even when some parts could be answered. Evaluating long answers exceeded the model input limit, while repeated text-length calculations also delayed document analysis.
Decision
- A passage needed the surrounding paragraphs and referenced tables to be understood correctly. I included that context and kept track of which document supplied each piece of evidence.
- An exact quotation, an answer supported by the document and a response covering every part of the question are different checks. I tested them separately instead of treating one score as proof of all three.
- The evaluation model sometimes changed the statements it was asked to repeat. I assigned an ID to each fixed target statement and matched the returned evaluation results to the original items.
- To locate application bottlenecks, I needed to separate local processing from model response time. I used fixed model responses and compared the time spent on repeated calculations.
- Users did not need to wait for quality evaluation to read an answer that was already complete. I returned the answer first and evaluated it separately, deferring the next evaluation request while a new question was being processed.
Solution
- Used SQLGlot to compare query and aggregation conditions before and after AI-generated SQL repairs. Stopped execution when conditions changed or could not be confirmed as preserved.
- Expanded retrieval to include the paragraphs surrounding a match and the sections or tables it referenced. Reviewed missing evidence again and supplied original text and sources when generating the final answer.
- Ragas checks whether an answer is supported by the document, but long answers exceeded the model input limit. I split the content into batches and matched the result IDs to the original items, checking for missing or duplicate verdicts before calculating the score.
- Fixed requests for explanations being misclassified as requests for tables. Collected an answer and evidence IDs for each part of the question, then copied quotations directly from the corresponding source text. Kept supported answers and marked only the parts for which evidence was missing.
- Removed repeated token counting when merging tables and reused page-analysis results within the same request. Saved and displayed the answer first, then ran Ragas evaluation as a separate task so users did not have to wait for it to finish.
- Included surrounding dialogue when analyzing media to recover omitted subjects and reduce unnecessary abstention. Built navigation to the source document page or playback segment so users could inspect the evidence themselves.
Implementation details
- Built React and TypeScript interfaces, Django APIs, MariaDB storage and queries, document processing and Gemma 4 integration. Checked generated SQL against the user-selected data scope and validated table values, units and the source of each row.
- Connected the model tokenizer to count tokens in the actual request, including its output schema. Split long documents at sentence boundaries and tracked evidence omitted from final answer generation as unreviewed.
- Used Ragas Faithfulness to extract statements from an answer and calculate the proportion supported by document evidence. Ran evaluation in a separate process and discarded previous results when the answer, evidence or settings changed.
- Combined live-model checks with replay of saved failure responses. Separate tests with fixed responses checked for recurring failures, and I verified document and media processing independently.
Checks
- Used fixed model responses and measured 20 runs before and after the change, following warm-up. Median application processing fell from 2,969ms → 672ms for document analysis and 485ms → 312ms for selected-document search. Normal-path model-call counts remained unchanged.
- Extracted 25 items from a live long answer and evaluated them in 5 batches. The largest input was 8,164 tokens, with 6 calls including statement extraction. Automated tests also checked rejection of missing IDs, duplicate IDs and verdicts with invalid data types.
- Confirmed that live answers appeared before evaluation completed. Checked that the next evaluation request was deferred during a new question and that completed evaluation stages could be reused.
- Compared live answers with the source text and tested altered quotations, mismatched units and questions without supporting evidence. Browser checks covered source navigation, media playback and retention of input and selections.