AI-assisted improvisation practice tool for actors. Based on the MyStoryKnight project. See the ImrovMate project page for more details

ImprovMate is an AI-powered improvisation practice tool designed to help actors refine their skills in a dynamic and engaging way. By leveraging advancements in artificial intelligence, particularly large language models (LLMs), ImprovMate provides a seamless and intuitive platform for individual improv training, addressing the limitations of traditional group-based methods.

The system uses multimodal input, such as an actor's audiovisual performance, to generate characters, objects, and scenarios that mimic the unpredictability of live performances. It tracks narrative elements throughout a session, ensuring coherence while reducing cognitive load. ImprovMate also incorporates structured exercises inspired by traditional improv training techniques, fostering creativity and supporting actors of all experience levels.

Key features of ImprovMate include:

• Dynamic Scenario Generation: AI-generated characters, objects, and plot points to simulate real-time improvisation challenges.
• Narrative Coherence: Tracks and maintains consistent plot threads throughout the session.
• Structured Training: Exercises based on proven improv techniques to enhance creativity and adaptability.
• Seamless Integration: Requires no additional equipment for speech or motion recording, ensuring minimal setup and overhead.
• Accessibility: Supports both traditional improv practice and AI-driven randomness to inspire new ideas.

ImprovMate is designed to empower actors by providing a flexible, creative, and effective tool for honing their improvisational skills, whether they are beginners or seasoned professionals.

• Docker Desktop

1. Clone the repository
2. Inside backend and frontend, copy .env.example to .env and fill in the necessary environment variables.
3. Run the following command in the root directory of the repository

docker-compose up

1. Open a web browser and navigate to http://localhost:3000
2. Access the backend API at http://localhost:5000/api

ImprovMate/
├── frontend                # Frontend code (React)
|  ├── Dockerfile           # Dockerfile for frontend
├── backend                 # Backend code (Flask)
|  ├── Dockerfile           # Dockerfile for backend
├── docker-compose.yml      # Docker compose file
└── README.md               # This file

In app.py, need to set the host to 0.0.0.0 to allow the container to access the host's network.

if __name__ == '__main__':
    app.run(host='0.0.0.0')

In vite.config.ts need to set the following settings:

import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";

// https://vitejs.dev/config/
export default defineConfig({
  base: "/",
  plugins: [react()],
  preview: {
    port: 8080,
    strictPort: true,
  },
  server: {
    port: 8080,
    strictPort: true,
    host: true,
    origin: "http://127.0.0.1:8080", // Correctly load assets
  },
});

1. Navigate to the backend directory
2. Deploy the backend using the following command

gcloud run deploy SERVICE --source .

3. Set the environment variables in the Google Cloud Console

• Make sure the active project is the correct one with the following commands

gcloud projects list
gcloud config get project
gcloud config set project PROJECT_ID

• The Cloud run service name should be improvmate-be.
• The region should be set to us-central1.

1. Navigate to the frontend directory
2. Make sure to update the .env file with the correct VITE_API_BASE_URL based on the backend service URL.
3. Deploy the frontend using the following commands

npm run predeploy
npm run deploy

4. This will deploy to the gh-pages branch of the repository.

@inproceedings{drago2025improvmate,
  author       = {Riccardo Drago and Yotam Sechayk and Mustafa Doga Dogan and Andrea Sanna and Takeo Igarashi},
  title        = {{ImprovMate: Multimodal AI Assistant for Improv Actor Training}},
  booktitle    = {Companion Publication of the 2025 ACM Designing Interactive Systems Conference (DIS ’25 Companion)},
  year         = {2025},
  publisher    = {Association for Computing Machinery},
  address      = {New York, NY, USA},
  doi          = {10.25145/3715668.3736363},
  isbn         = {979-8-4007-1486-3},
  url          = {https://doi.org/10.25145/3715668.3736363}
}