EVALCARD
Independently built evaluation toolThe other projects began with external decision contexts. EVALCARD began with a question I kept asking myself: how do I judge whether an AI output is actually good, and how certain should I be about that judgment?
An independently built evaluation tool: a single HTML page that scores one model output using another model as judge.
Judging a single output leaves two bad options — eyeball it, which is fast and unrepeatable, or stand up an eval framework, which is disproportionate for one answer and one question.
A clear, accurate explanation that nails the core intuition behind self-attention and its scaling advantage over RNNs; loses a little for not noting attention’s quadratic cost.
One question, one answer, one optional reference.
Explain how a transformer model uses self-attention, and why it scales better than an RNN for long sequences.
A transformer uses self-attention so every token can attend to every other token in one step, producing weighted context vectors. Unlike an RNN, which processes tokens sequentially and struggles to carry information across long gaps, attention gives a direct O(1) path between any two positions, so all positions are computed in parallel. This parallelism is what lets transformers
Shown as the shipped interface presents it — a long response is clipped in the panel.
Self-attention computes pairwise relevance between all tokens; parallelizable; constant path length vs RNN linear path length.
Optional. When provided, the judge weighs the answer against it.
Five dimensions, each scored out of ten
Another model acts as judge. It reads the prompt, the output, and any reference, then returns one structured response: a score and a written justification for each of five dimensions, plus an overall verdict and a grade.
Judge model is selectable — three Claude models are offered.
A clear, accurate explanation that nails the core intuition behind self-attention and its scaling advantage over RNNs; loses a little for not noting attention’s quadratic cost.
The judge writes the verdict and the grade. The client adds the five dimension scores into the fifty-point total and colours the band. These are related outputs — not a statistically calibrated benchmark.
An LLM judge produces a structured opinion, not ground truth. The score becomes useful when its dimensions and reasoning remain visible.
A single HTML tool with no framework, package installation or backend.
EVALCARD has no application backend. The browser sends an evaluation directly to Anthropic, and a saved key is stored locally in the browser.
A working local, open-source tool — not a hosted product.
A later version explores evaluating observable agent trajectories.
- evidence faithfulness
- tool-use quality
- recovery and efficiency
- A/B comparison across observable traces
Planned — not part of v1. Observable traces only.
One score, made inspectable.
Independently built by Teja Padala. MIT licensed. Values shown are from one illustrative sample.