Note: UI screenshots are documented via Figma due to institutional security restrictions on the live environment.
EduSign is a mobile application built for the deaf and hard-of-hearing community, enabling teachers to upload learning videos that are automatically paired with synchronized Indonesian Sign Language (SIBI) animations — making educational content genuinely accessible without any manual effort.
1. Sign Language Video Synthesis Pipeline
When a teacher submits a YouTube link, the video ID is stored in Firebase. A local processing script — running on a team member's machine due to the multi-GB Whisper AI model being impractical to host on free cloud infrastructure — extracts a subtitle file (.SRT) from the video's audio.
The real complexity lies in preprocessing: every word in the transcript is morphologically reduced to its root form, then matched word-by-word against a local SIBI dataset where each clip is named after its corresponding word. These clips are stitched together and overlaid in the bottom-right corner of the screen in sync with the original video, so learners see both the lesson and its sign language translation simultaneously. Once processed, the SRT is pushed to Firebase Storage for downstream use by the quiz system.
2. AI-Powered Quiz Generation (My Core Contribution)
I built the complete quiz system and authentication layer. Teachers can create quizzes manually — inputting multiple-choice questions, options, and correct answers linked to a video ID — or trigger AI-generated quizzes in one click.
For AI generation, the video's SRT transcript is sent to the Groq API (running GPT OSS-120b), which returns 5 structured multiple-choice questions with answer choices and correct answers ready to store. The 5-question cap was a deliberate cost-control decision.
A critical security challenge: API keys cannot be embedded in Flutter client code. My solution was routing all Groq API calls through a Google Cloud Function connected to Firebase — the app triggers the function, which handles the request server-side and returns the quiz data safely, keeping the API key out of the client entirely.
I also implemented Google OAuth authentication, giving teachers and students a secure, frictionless login experience across the platform.
