חיפוש אגנטי נגד חיפוש וקטורי לאגנטי קוד: אנחנו רצים את האבל — Braintrust

תקציר מקורי באנגליתSame accuracy. Four times the cost. What happened when Braintrust pitted vector search against agentic search for a coding agent. Jess Wang, developer advocate at Braintrust, starts with the basics of evals: why shipping on vibes fails, the four parts of an eval (dataset, task, scorer, experiments), and how observability and evals form a flywheel. Then she runs a real one. Using fix PRs from Microsoft's TypeScript Go repo, she has Claude Code find the buggy code with either agentic search (grep, find, reading files) or vector search over embeddings, scored by the repo's own test suite. Both reached the same accuracy, but vector search cost about four times as much because chunks lacked the surrounding context and the agent kept searching. In this talk: • What evals are made of, and why the