Snorkel AI

Some tweets/posts about "evals are dead" this weekend 🤯 What I think many mean to say: generic, off-the-shelf (OTS) evals are dead - and I agree! I'd liken this to people attacking a standardized test like the SAT - they're not actually attack the notion of systematically measuring and monitoring human performance (that would be absurd). They're questioning a certain set of generic (and potentially overrepresented) evals- likely in favor of a much more custom, nuanced way of doing evaluation. We're at a similar moment in the AI space: generic OTS evals and public benchmarks are going to have minimal (and naturally diminishing) value. Watching a set of pre-canned LLMAJ scores on a monitoring dashboard- no matter how slick - has close to zero additive value today. Quoting how your model/agents perform on public benchmarks - same, soon enough. Instead: successful AI products will rely on custom, specialized evals and benchmark datasets. Whether these are supported by formal platforms or less formalized error analysis/feedback loops - it's all about specialized evals. So are "evals dead"? Definitely not - in fact, we're only just getting started on the real way to do meaningful evaluation in AI!

Part 2 of our 5-part series on AI evaluation rubrics just dropped! 📚 The right tool for the job: an A-Z of rubrics In this post, we dive deeper into types of rubrics: - Dataset-level that applies to all prompts - Instance-specific rubrics designed along with a specific prompt. - We also talk about process evaluation (trace level) or outcome evaluation. - And we cover some notes about LLM-based evals vs. code-based evals. Read on!  https://lnkd.in/gsn6mfsE

Lots of chatter about agentic/RL simulation environments recently! Some key misconceptions (slightly caricatured): >> Building RL envs is easy, because you just code up a verifier quickly, and let the model do the tough data generation on its own! - Usually, this boils down to over-indexing on environments where verification is easy. - For example: you might need a chess expert to generate realistic expert gameplay traces, but anyone with a basic chess rulebook could verify a win easily. - However: there are many, many settings where verification is not at all trivial. The simplest examples are settings with nuanced, domain-specific evaluation rubrics (e.g. most real world enterprise settings). An extreme example being: verify whether a program will halt :) >> Building RL envs will get commoditized as the "standard" environments get rapidly solved. - RL environments effectively encode a complete product spec - including unique tools, data resources, constraints, rubrics/verifiers, and human/agent simulators - and as such, are as diverse as the space of all possible AI products. - Yes, certain generic RL envs will rapidly commoditize ('web browsing', 'computer OS') - but these are not the useful ones anyway! - The useful RL envs will be deeply domain- and product-specific – and will require corresponding human expertise and customization to build and evolve over time. >> RL (and RL envs) will be all that you need! - Current evidence suggests that RL / RL envs will be one part of the overall AI development loop- which will continue to require golden human annotations/traces for initial SFT; ongoing human evals; and more - Just like trial-and-error based learning is only one part of human learning, RL will likely be one tool/phase of many. In summary: - (1) Building the components of an RL environment is usually highly non-trivial - (2) RL envs effectively describe a product spec - there will be a wide range of unique ones, requiring deep product/domain expertise - (3) RL (and RL envs) will be one component of a rich ecosystem of tools for model learning, including human data, rubrics, evals, and more If interested in some of the work the Snorkel AI team is doing in partnership with leading LLM developers here- shoot us a note! It's an exciting time to build in this space :)