Build an AI-powered system that can identify, classify, and analyze defects across different industrial domains (manufacturing, textiles, electronics, construction, etc.). The system uses a multi-stage AI pipeline combining Gemini AI for classification and either Hugging Face models or Gemini AI for detailed analysis.
- Uses Gemini 1.5 Flash to classify the defect type
- Identifies which of the 5 categories the defect belongs to
- Returns confidence score, severity level, and initial assessment
- Option A: If Hugging Face API key is available for the category:
- Uses specialized Hugging Face model for precise detection
- Returns bounding boxes, confidence scores, and specific defect locations
- Option B: If no Hugging Face API key (fallback):
- Uses Gemini AI for detailed analysis
- Provides comprehensive defect description, recommendations, and assessment
-
Surface Scratch Detection
- Input: Images of metal, wood, or plastic surfaces
- Detects: Scratches, abrasions, surface damage
- Use case: Automotive/metal parts inspection
- HF Model: Optional (facebook/detr-resnet-50)
-
Crack Detection in Concrete/Buildings
- Input: Drone images of walls, bridges, or roads
- Detects: Cracks, structural damage, fissures
- Use case: Civil engineering defect monitoring
- HF Model: Optional (facebook/detr-resnet-50)
-
Textile/Fabric Defect Detection
- Input: Fabric roll images
- Detects: Holes, missing threads, color irregularities
- Use case: Textile manufacturing QA
- HF Model: Optional (facebook/detr-resnet-50)
-
PCB (Printed Circuit Board) Defect Detection
- Input: PCB board images
- Detects: Soldering issues, missing components, trace damage
- Use case: Electronics manufacturing QA
- HF Model: Optional (facebook/detr-resnet-50)
-
Glass/Bottle Defect Detection
- Input: Industrial glass product images
- Detects: Cracks, bubbles, surface irregularities
- Use case: Glass manufacturing QA
- HF Model: Optional (facebook/detr-resnet-50)
VITE_GEMINI_API_KEY- Your Google Gemini API key (used for all classifications and fallback analysis)
Add these to .env file for enhanced detection capabilities:
VITE_HF_SURFACE_SCRATCH_KEY="" # For surface scratch detection
VITE_HF_CRACK_CONCRETE_KEY="" # For concrete crack detection
VITE_HF_TEXTILE_FABRIC_KEY="" # For textile defect detection
VITE_HF_PCB_BOARD_KEY="" # For PCB defect detection
VITE_HF_GLASS_BOTTLE_KEY="" # For glass defect detectionNote: If Hugging Face keys are not provided, the system automatically falls back to Gemini AI for detailed analysis.
- User uploads an image through the web interface
- Gemini AI classifies the defect category automatically
- System routes the image to the appropriate analysis method:
- If HF API key exists → Use specialized HF model
- If no HF API key → Use Gemini for detailed analysis
- Results are stored in Supabase database with full analysis
- User views comprehensive report with:
- Defect type and confidence score
- Number of defects detected
- Severity level (low/medium/high)
- Detailed findings and recommendations
- Visual annotations (if HF model provides bounding boxes)
- Frontend: React + TypeScript + Vite + Tailwind CSS + shadcn/ui
- Backend: Supabase (Database + Authentication)
- AI Models:
- Google Gemini 2.5 Flash (Classification + Fallback Analysis)
- Hugging Face Models (Optional specialized detection)
- Deployment: Ready for Vercel/Netlify
- Flexible AI Pipeline: Works with or without Hugging Face keys
- Multi-domain: Covers 5 different industrial use cases
- Production-ready: Includes authentication, database, and responsive UI
- Showcases integration skills: Combines multiple AI services intelligently
- Scalable architecture: Easy to add new defect categories
- Cost-effective: Falls back to Gemini when specialized models aren't available
- Clone the repository
- Install dependencies:
npm installorbun install - Set up environment variables in
.env:- Add your
VITE_GEMINI_API_KEY(required) - Optionally add Hugging Face API keys
- Add your
- Run development server:
npm run dev - Upload images and test the multi-stage detection system!