Firebase Agent Skills Get Eval-Driven Development: How Google Measures AI Coding Success

Firebase introduced Agent Skills earlier this year as a way to give AI coding agents structured cheat sheets for writing better Firebase code. Now the Firebase team has published a detailed look at how it builds and verifies those skills, and the answer is a discipline called eval-driven development: writing automated tests first, then iterating on the skills until the agent passes them.

Skills sit at the top of an agentic toolkit that also includes the Firebase CLI and MCP servers. CLIs are loved by agents for their straightforward terminal workflows, while MCP servers expose explicit tool definitions that let an agent understand exactly what a tool does and which arguments it requires. Skills act as the conductor, orchestrating both, and the team recommends installing the skills and the MCP server together for the most accurate results.

Why Skills Are Not Just Documentation

The distinction between an agent skill and official documentation matters. Documentation provides comprehensive reference material, while skills are highly focused, outcome-driven cheat sheets designed to enforce core principles, highlight best practices, and give preemptive guidance for tasks where agents are known to struggle. A skill can walk an agent through adding a feature that uses Firestore securely, from provisioning in the cloud to writing the app code, and there are even skills that review Firebase Security Rules to protect user data.

Skills also bridge a fundamental limitation of language models. Because LLMs are constrained by training data cut-off dates, they do not know about recent releases. A skill can explicitly instruct the agent to use the CLI or an MCP server tool to implement a feature or read the latest documentation, letting the model discover brand-new APIs on the fly instead of guessing based on stale knowledge.

That value proposition created a problem, though. As the skill catalog grew, the team faced a critical question: how do we know these skills are actually helping agents succeed? Writing instructions and hoping for the best was not good enough, so Firebase built a set of evals — automated prompt suites that score how well an agent completes specific critical user journeys, or CUJs.

Three Layers of Evaluation

The evaluation system works at three levels. Single-skill evaluations test the core functionality of one skill against five to ten specific cases per critical user journey, with strict expectations for what a successful answer looks like; one example expects the agent to suggest the npx-based Firebase CLI version command rather than the outdated firebase --version form. Skill activation evaluations test the frontmatter of each skill by running both positive and negative prompts, ensuring the agent activates the right skill when needed without over-activating on irrelevant requests.

• Single-skill evals verify a skill's guidance against concrete cases
• Activation evals check that agents trigger skills at the right moment
• End-to-end evals build and deploy real apps using every skill at once
• LLM-as-a-judge scoring determines whether implementations succeed

The ultimate test is the end-to-end multi-skill evaluation, where the agent receives all Firebase skills and is asked to create and deploy an app using multiple products. The published example is a Tetris clone with a global leaderboard backed by Firestore: the agent must generate a firebase.json with firestore and hosting blocks, implement Firebase Auth with Google sign-in, write a query for the top ten scores ordered descending, create security rules that let users write only their own records, and successfully deploy to Firebase Hosting.

The shift to eval-driven development changed how the team works. Instead of writing what they thought was important and hoping agents followed it, they now start with the test and iterate on the skills until the agent succeeds. The loop is beginning to extend beyond skills themselves, surfacing friction in the Firebase CLI and MCP servers as well, since a failing end-to-end evaluation may point at the tooling rather than the skill.

For Flutter and Dart developers building on Firebase, the practical takeaway is straightforward: the AI tooling that generates Firebase code is being held to measurable standards. Skills, CLI integration, and MCP servers are converging on a workflow where agents are not just encouraged to use Firebase but verified to use it correctly, against real projects and real deployment pipelines.

The approach also validates a broader movement in developer tooling. As agentic coding becomes a normal part of the delivery pipeline, the teams that treat their agent guidance as testable software rather than prose will find out quickly what works and what does not. Firebase's eval-driven development is a concrete example of that discipline in action, and its three-layer framework is a useful template for anyone shipping agent-facing tooling.