MLCommons

MedPerf's New WebUI Makes Federated Medical AI Benchmarking Accessible to Everyone 🏥 We're excited to announce that MedPerf now includes WebUI capabilities, extending our open-source orchestrator to the browser with an intuitive, visual interface. This is a significant step toward making federated benchmarking accessible to non-technical experts, including clinicians. The WebUI has already proven its value in the real world. In our ongoing glioblastoma clinical study (FL-PoST/FeTS 2.0), the interface enabled: ✅ Evaluation of AI models on 1,568 brain scans across 7 international sites ✅ Seamless asynchronous collaboration across time zones ✅ Validation of models trained on 10,208+ brain scans from 53 sites across 5 continents Dr. Evan Calabrese, lead principal investigator: "The familiar web-based interface enabled efficient collaboration across time zones without the need for live support, even for those with limited technical literacy... It's clear to me that this WebUI will be the future of MedPerf." By combining CLI power with browser convenience—while keeping data secure on local premises—we're making federated evaluation more accessible to the medical AI community. You can learn more in our blog post: https://hubs.ly/Q03VT_m40 Check out our full tutorial and join us in accelerating trusted medical AI: https://hubs.ly/Q03VV1NZ0 #MLCommons #MedicalAI #Healthcare #FederatedLearning #AIBenchmarking #HealthTech

Hey Folks if you are at NeurIPS in San Diego you should come hang out at our social session: "Nonprofits Working on Openness and Trust in AI" For those of you who know me know how much I love data and how important to all our ML/AI work. I will be speaking about The AI Alliance's work on the Open Trusted Data Initiative and how it builds on the work from others on the panel (Wikimedia Foundation, MLCommons) Wed. at 7-9 pm in Upper Level Ballroom 20D https://lnkd.in/g7Ajac2Y https://lnkd.in/gMVp6zb9 Slight spoiler - one of my points will be that Open Source AI requires Open Data

Encouraging to see #data mentioned as the first pillar of the #AIForScience strategy, with plans to make #scientific #data #AIready, build new #repositories with #standardised #metadata and #APIs, invest in #highvaludatasets, and improve the collection and sharing of underexplored data like negative results. While #FAIR is a solid baseline to build data infrastructure, there is much more to making data genuinely useful for #AI use cases, as we commented in our recent work at the Open Data Institute https://lnkd.in/eGGdsGPu. MLCommons #Croissant #metadata #standard for #AI #data is already used by all major #AI #data #repository, as well as the geospatial and biosciences communities. If you are interested in setting up a working group to discuss its application to other types of scientific #data, get in touch! https://lnkd.in/eVDmeEXX

If you’ll be at NeurIPS this week, come by and say hi! 👋 Wikimedia Enterprise along with MLCommons and The AI Alliance will be co-hosting at the official NeurIPS social — three nonprofits working at the intersection of openness, public interest, and trustworthy AI to bring together organizations committed to open knowledge, transparency, community governance, and equitable access in AI. If you’re looking for a space to connect, collaborate, and explore opportunities with nonprofits and other mission aligned orgs at NeurIPS come out and join us in the discussion as we explore what’s next in this evolving space. 𝗬𝗼𝘂’𝗹𝗹 𝗳𝗶𝗻𝗱 𝘂𝘀 𝗵𝗲𝗿𝗲: 📅 Wednesday, December 3rd 📍 San Diego Convention Center – Upper Level Ballroom 20D ⏰ 7pm to 9pm PT Our Principal PM, Christopher Petrillo, and Researcher, Chia Hwu, will both be co-hosting the event and joining speakers from across the nonprofit AI ecosystem: 𝗪𝗶𝗸𝗶𝗺𝗲𝗱𝗶𝗮 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 • Christopher Petrillo • Chia Hwu 𝗔𝗜 𝗔𝗹𝗹𝗶𝗮𝗻𝗰𝗲 • Steven Pousty, Ph.D. 𝗠𝗟 𝗖𝗼𝗺𝗺𝗼𝗻𝘀 • Neil Majithia If you’d like to meet with Chris or Chia, feel free to reach out to them to set up some time for a chat between Dec 3rd and the 8th, while they’re in San Diego. Hope to see you there! 👋 For details and the agenda, read more here 👉 https://lnkd.in/eUT64Zke

SPEAKER SPOTLIGHT: Dr. Shalaleh Rismani Dr. Rismani brings a critical perspective to AI development: systems thinking. As a researcher at Mila - Quebec AI Institute, she focuses on understanding and evaluating AI in real-world context — not just in isolation. At Endpoints, her session "Safety Is a Systems Problem: Understanding and Evaluating AI in Context This session is for anyone working on AI deployment, policy, or safety. Join us December 1 in San Diego: https://hubs.ly/Q03VYWDX0 #AISafety #Endpoints2025 #ResponsibleAI #MLCommons #SystemsThinking

The best conversations about AI don't happen in isolation.They happen when researchers share insights with practitioners. When competing companies find common ground. When academic rigor meets deployment reality. We're creating that space. MLCommons Endpoints | December 1, 2025 | San Diego Join us for one full day of collaboration on what matters most: AI agents & benchmarking • Risk & reliability • The AI economy • Token production efficiency • Safety as a systems problem • Data & benchmarking • The future of ML This is the conversation that shapes what comes next. With speakers including: David Kanter, Co-founder MLCommons James Goel, Qualcomm Dr. Lora Aroyo, Google DeepMind Dr. Sean McGregor, Digital Safety Research Institute Luis Oala, Dotphoton Dr. Shalaleh Rismani, Mila Neil Majithia, Open Data Institute The collective for a better AI future. 📍 San Diego | 🗓 December 1 | 🎟️ https://hubs.ly/Q03VYBn50 #MLCommons #Endpoints2025 #AIResearch #MachineLearning

SPEAKER SPOTLIGHT: Dr. Sean McGregor Dr. McGregor is a ML Safety Researcher and Founding Director at the Digital Safety Research Institute. He's the creator of the AI Incident Database and leads MLCommons' Agentic Workstream. At Endpoints, Sean will present "Bootstrapping Agents, Benchmarking, and Products -- All at Once" Traditional "research first, then build, then deploy" doesn't work anymore. The cycle is compressed. So how do we do this responsibly? If you're working on agents or safety-critical systems, this session is essential. December 1 | San Diego Register: https://hubs.ly/Q03VYJCX0 #AISafety #AIAgents #Endpoints2025 #MLCommons

⏰ 3 DAYS UNTIL ENDPOINTS! Have you registered yet? ✨ 10+ expert sessions ✨ Network with 125+ member companies Dec 1st, San Diego Secure your spot: https://hubs.ly/Q03VVjzz0 #MLCommons #Endpoints2025

Everyone's building AI. Few are thinking clearly about the economics. LuisOala of Dotphoton breaks it down at #Endpoints2025 https://hubs.ly/Q03Vx7hQ0