Skip to content

WhoamiI00/VISTA

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🛠 VISTA: Visual Inspection System for Tracking Anomalies

Core Idea

Build an AI-powered system that can identify, classify, and analyze defects across different industrial domains (manufacturing, textiles, electronics, construction, etc.). The system uses a multi-stage AI pipeline combining Gemini AI for classification and either Hugging Face models or Gemini AI for detailed analysis.


🔹 Architecture Overview

Stage 1: Classification (Gemini AI)

  • Uses Gemini 1.5 Flash to classify the defect type
  • Identifies which of the 5 categories the defect belongs to
  • Returns confidence score, severity level, and initial assessment

Stage 2: Detailed Analysis

  • Option A: If Hugging Face API key is available for the category:
    • Uses specialized Hugging Face model for precise detection
    • Returns bounding boxes, confidence scores, and specific defect locations
  • Option B: If no Hugging Face API key (fallback):
    • Uses Gemini AI for detailed analysis
    • Provides comprehensive defect description, recommendations, and assessment

🔹 Supported Defect Categories

  1. Surface Scratch Detection

    • Input: Images of metal, wood, or plastic surfaces
    • Detects: Scratches, abrasions, surface damage
    • Use case: Automotive/metal parts inspection
    • HF Model: Optional (facebook/detr-resnet-50)
  2. Crack Detection in Concrete/Buildings

    • Input: Drone images of walls, bridges, or roads
    • Detects: Cracks, structural damage, fissures
    • Use case: Civil engineering defect monitoring
    • HF Model: Optional (facebook/detr-resnet-50)
  3. Textile/Fabric Defect Detection

    • Input: Fabric roll images
    • Detects: Holes, missing threads, color irregularities
    • Use case: Textile manufacturing QA
    • HF Model: Optional (facebook/detr-resnet-50)
  4. PCB (Printed Circuit Board) Defect Detection

    • Input: PCB board images
    • Detects: Soldering issues, missing components, trace damage
    • Use case: Electronics manufacturing QA
    • HF Model: Optional (facebook/detr-resnet-50)
  5. Glass/Bottle Defect Detection

    • Input: Industrial glass product images
    • Detects: Cracks, bubbles, surface irregularities
    • Use case: Glass manufacturing QA
    • HF Model: Optional (facebook/detr-resnet-50)

🔹 API Configuration

Required:

  • VITE_GEMINI_API_KEY - Your Google Gemini API key (used for all classifications and fallback analysis)

Optional (Hugging Face Keys):

Add these to .env file for enhanced detection capabilities:

VITE_HF_SURFACE_SCRATCH_KEY=""    # For surface scratch detection
VITE_HF_CRACK_CONCRETE_KEY=""     # For concrete crack detection
VITE_HF_TEXTILE_FABRIC_KEY=""     # For textile defect detection
VITE_HF_PCB_BOARD_KEY=""          # For PCB defect detection
VITE_HF_GLASS_BOTTLE_KEY=""       # For glass defect detection

Note: If Hugging Face keys are not provided, the system automatically falls back to Gemini AI for detailed analysis.


🔹 How It Works

  1. User uploads an image through the web interface
  2. Gemini AI classifies the defect category automatically
  3. System routes the image to the appropriate analysis method:
    • If HF API key exists → Use specialized HF model
    • If no HF API key → Use Gemini for detailed analysis
  4. Results are stored in Supabase database with full analysis
  5. User views comprehensive report with:
    • Defect type and confidence score
    • Number of defects detected
    • Severity level (low/medium/high)
    • Detailed findings and recommendations
    • Visual annotations (if HF model provides bounding boxes)

🔹 Tech Stack

  • Frontend: React + TypeScript + Vite + Tailwind CSS + shadcn/ui
  • Backend: Supabase (Database + Authentication)
  • AI Models:
    • Google Gemini 2.5 Flash (Classification + Fallback Analysis)
    • Hugging Face Models (Optional specialized detection)
  • Deployment: Ready for Vercel/Netlify

🔹 Why This is Great

  • Flexible AI Pipeline: Works with or without Hugging Face keys
  • Multi-domain: Covers 5 different industrial use cases
  • Production-ready: Includes authentication, database, and responsive UI
  • Showcases integration skills: Combines multiple AI services intelligently
  • Scalable architecture: Easy to add new defect categories
  • Cost-effective: Falls back to Gemini when specialized models aren't available

🔹 Getting Started

  1. Clone the repository
  2. Install dependencies: npm install or bun install
  3. Set up environment variables in .env:
    • Add your VITE_GEMINI_API_KEY (required)
    • Optionally add Hugging Face API keys
  4. Run development server: npm run dev
  5. Upload images and test the multi-stage detection system!

About

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages