Dagster Labs

Our Founder and CTO, Nick Schrock, joined Prateek Joshit on the Infinite Curiosity Podcast to share insights on building and scaling developer tools that speak to developers. Heres the highlights: 🫰 Why assets beat tasks every time: When a downstream stakeholder sees wrong data, they don't think, "What workflow broke?" They think, "Why is this table wrong?" Building your orchestration around the language people actually use data assets, not abstract tasks, creates clearer communication and better debugging experiences. 💪 The hidden power of sales engineers: Nick's advice for anyone evaluating dev tools? "Skip the PMs and talk to the sales engineers without them in the room. They live with the product, see customer reactions, and know what's really working." They're the unsung heroes who bridge technical reality and business needs. 🫂 Community-first commercialization: We closed our first 20 customers in two months - all from our open source community. The key was being high-touch with users who already understood the value and needed enterprise features like SSO and integrations. 💻 Technical founders: stay technical: One mistake Nick admits? Stepping back too much from product decisions as the company grew. "You need to selectively micromanage what you care about. Good founders zoom in and out strategically - sometimes you go tactical, sometimes strategic." The conversation covers everything from GraphQL's three-month sprint at Facebook to why software engineering practices matter so much in data pipelines. Nick gets real about pricing mistakes, the reality of scaling from pure R&D to go-to-market, and why developer experience isn't optional for data tools. It's worth listening to if you're building in the data space or wondering how open-source companies actually become sustainable businesses. Link in the comments!

When it comes to Components, Engineers think they're writing code, but they're actually designing user experiences for other engineers. After analyzing hundreds of Dagster components, the ones that succeed follow three critical patterns: ✅ Focus on ONE user type (don't try to please everyone) ✅ Think about interfaces before writing any code ✅ Build small, modular pieces that compose together The biggest mistake is the "Swiss Army knife" component, which tries to do everything and ends up confusing everyone. We just published 6 battle-tested practices that separate components teams love from those that collect digital dust. The testing strategies alone will save you hours of debugging. Check out our latest blog from Dennis Hume today! Link in the comments

Data Drinks Happy Hour is starting soon! 🎉 Join the exhibitors below at 5pm onwards for some well deserved, relaxed networking with complimentary beers, soft drinks and tasty nibbles: Airbyte - stand N60, Atlan - I48, Carruthers and Jackson - B60, ClarityQ Agentic Data Analysis- D50, ClickHouse - K98, Collibra - N68, Dagster Labs - K100, DataGalaxy - P58, DataHub - K120, Snowflake - M40, Secoda - K110, Domo - P48, DQLabs - K80, Qlik - K88, Glean - F50, IP River Limited - A34, Jumpmind - L109, Mixpanel - C61, MotherDuck - B27, Omni - M30, Pentaho - K118, Quest Software - J80

Big Data LDN is finally here! 🇬🇧 It's been great to connect with our European friends. We're at Booth K100. Come say hi to the mad lads!

We’re hiring: Data Engineer at Version2.ai Version2.ai is the first AI research and design lab built exclusively for fundraising. Our mission is to help nonprofits break through the digital divide with technology that makes donor engagement more personal, scalable, and effective. As a Data Engineer, you’ll: •Design and scale cloud-native data pipelines that power our AI-driven donor engagement platform •Architect and maintain data lakes, warehouses, and lakehouses in AWS •Model data for analytics & ML, building feature stores and schemas for experimentation and insights •Implement observability frameworks that ensure clean, trusted, and reliable data •Collaborate with ML, backend, and product teams to deliver high-impact solutions We’re looking for: •2+ years of experience building and maintaining modern data pipelines using Python and SQL •Proficiency in AWS cloud services (S3, Glue, Redshift/Snowflake, Lambda, Step Functions) •Hands-on experience with orchestration frameworks like Dagster, Airflow, or Prefect •Strong data modeling and performance optimization skills across SQL and NoSQL systems A passion for applying technology to solve real-world problems in philanthropy This is your chance to join a mission-driven team working at the intersection of philanthropy and artificial intelligence—building scalable, high-impact technology that empowers nonprofits and reshapes the future of giving. Apply today: https://lnkd.in/eTcUnDKR #DataEngineering #NonprofitTech #AI #Hiring

Data engineers can have nice things too ✨ Lead Sales Engineer Christian Minich just released a demo of how Dagster transforms data teams from reactive firefighting to strategic building: 👁️ Single pane of glass for all data assets (no more tool archaeology) 🌴 Branch deployments that actually work for data pipelines 🩺 Asset health monitoring that catches issues before stakeholders do ⚙️ Legacy system observability without rip-and-replace migrations Teams spending 95% less time creating new pipelines (1-2 days → 1 hour) Stop treating your data platform like it's 2015. Modern problems need modern solutions. Check out the full demo today! Link in the comments

How to build AI-ready data pipelines with LangChain and Dagster Labs ⬇️ Step 1️⃣: Connect your data sources. Your LLM needs context from everywhere—Salesforce, Zendesk, your database. Use a data integration platform to sync 500+ sources automatically. No more brittle Python scripts that break every API update. Step 2️⃣: Set up smart orchestration. Deploy a pipeline that processes your data on schedule, handles failures gracefully, and transforms everything into an AI-readable format. Think: automated data flows that just work. Step 3️⃣: Build your vector store. With data flowing in, prepare it for your LLM: - Chunk documents into digestible pieces - Generate embeddings for semantic search - Store in a vector database for lightning-fast retrieval - Set up retrieval chains to find the right context for each query The magic happens when everything connects. Your AI assistant can now answer "What issues has this customer faced?" by searching across support tickets, CRM data, and documentation—instantly. Real teams are seeing: ✨ 50% faster support response times ✨ 30% fewer repetitive questions to data teams ✨ Real-time insights instead of day-old reports Want the full technical breakdown? We got you: https://bit.ly/42rMo6D

European data teams have been telling us: "We want to use Dagster+, but compliance means everything needs to stay in the EU." Dagster+ is now available in EU data centers. Your control plane, metadata, and operational data all stay within European boundaries. This removes a major blocker for teams that've been stuck with workarounds due to data residency requirements. Your data never leaves your infrastructure and only orchestration metadata flows to Dagster+. Bonus: having the control plane closer to your EU workloads means better performance and lower latency. This is a really exciting development, and I am happy that we can better serve our European users and customers! Read the full announcement. Link in the comments #DataEngineering #DataOrchestration #EU