Commit Graph

10 Commits

Author SHA1 Message Date
Claude 2301f033cd
Add Music Video Note Editor - Complete Interactive Piano/Keyboard Learning Platform
Created a comprehensive web application for music education with advanced features:

CORE FEATURES:

1. 🎵 YouTube Video Integration
   - Load any YouTube video with URL parsing
   - Playback controls (play/pause, seek ±5s)
   - Real-time timeline synchronization
   - Duration tracking and display

2. 🎹 WebMIDI Integration (Real-time MIDI Input)
   - Connect MIDI keyboards directly in browser
   - Live note visualization with velocity display
   - Support for multiple MIDI devices
   - Zero-latency feedback system
   - Portuguese note name translation
   - Device detection and status monitoring

3. 🎵 Audio Transcription Framework
   - Integration-ready for Spotify Basic Pitch
   - Support for Magenta Onsets & Frames
   - Demo mode with simulated transcription
   - Confidence threshold filtering
   - Automatic note detection from video audio
   - Production deployment instructions included

4. 🎯 Intelligent Practice Mode (Educational Tutor)
   - Real-time feedback while playing
   - Performance metrics tracking:
     * Correct/wrong note counting
     * Accuracy percentage
     * Streak tracking (consecutive correct notes)
     * Timing error analysis (±500ms tolerance)
     * Pitch accuracy detection
   - Dynamic canvas visualization:
     * Shows upcoming notes (5s lookahead)
     * Highlights current note to play
     * Progress bar
   - Gamification system:
     * Streak counter with fire emoji
     * Special achievements every 5 notes
     * Detailed final reports
   - Personalized improvement tips
   - Practice session statistics

5. 📝 Sheet Music Visualization (VexFlow)
   - Automatic musical score generation
   - Treble and bass clef support
   - Color-coded notes matching timeline
   - Configurable time signatures
   - PNG export capability
   - MIDI export framework

6. ✏️ Manual Note Editor
   - 12 chromatic notes (C through B, including sharps)
   - Octave selection (1-7)
   - Duration control (0.1-5 seconds)
   - Interactive timeline with colored blocks
   - Click notes to jump to video timestamp
   - Drag-free note deletion

7. 🎨 Pedagogical Color System
   Natural notes (light colors):
   - C: Light green | D: Light blue | E: Yellow
   - F: Light red | G: Light purple | A: Light orange | B: Pink

   Sharp notes (dark colors):
   - C#: Dark green | D#: Dark blue | F#: Dark red
   - G#: Dark purple | A#: Dark orange

8. 🤖 AI-Powered Educational Prompts
   - Automatic analysis of note patterns
   - Personalized tips for:
     * Hand positioning
     * Intensity and dynamics
     * Pedal usage
     * Rhythm and timing
     * General technique
   - Context-aware suggestions based on black key usage
   - Difficulty-adjusted recommendations

9. 💾 Complete Export System
   - Save full project (JSON with all data)
   - Load saved projects
   - Export notes to CSV
   - Export sheet music to PNG
   - Preserves video URL, notes, and educational content

CODE QUALITY & ARCHITECTURE:

- Modular architecture with separate handlers:
  * app.js - Main orchestration and state management
  * midi-handler.js - WebMIDI device management
  * transcription-handler.js - Audio to MIDI conversion
  * practice-mode.js - Educational tutor with gamification
  * score-renderer.js - VexFlow sheet music rendering

- Error handling throughout:
  * Try-catch blocks for all critical operations
  * Graceful degradation when features unavailable
  * User-friendly error messages
  * Console logging for debugging

- Safe null checks:
  * Optional chaining (?.) for property access
  * Existence checks before element manipulation
  * Type validation before operations

- Performance optimizations:
  * RequestAnimationFrame for smooth canvas rendering
  * Efficient DOM manipulation
  * Event delegation where appropriate
  * Cleanup of animation frames and listeners

- Responsive design:
  * Mobile-friendly layouts
  * Flexible grid systems
  * Touch-friendly buttons
  * Adaptive canvas sizing

TECHNOLOGIES & LIBRARIES:

External Libraries (CDN):
- VexFlow 4.2.2 (MIT) - Musical notation rendering
- WebMIDI.js 3.1.6 (MIT) - MIDI device support
- YouTube IFrame API - Video playback

Framework Ready For:
- Spotify Basic Pitch (Apache 2.0) - Audio transcription
- Magenta Onsets & Frames (Apache 2.0) - Piano detection
- ByteDance Piano Transcription - High-quality backend option

DOCUMENTATION:

- Comprehensive README with:
  * Quick start guide
  * Detailed feature explanations
  * Architecture blueprint (3 operational modes)
  * Integration guides for production
  * Troubleshooting section
  * Use cases for different audiences
  * Dataset recommendations (MAESTRO, ASAP)
  * Licensing information

- Code comments explaining:
  * Complex algorithms
  * Integration points
  * Configuration options
  * Future enhancement areas

EDUCATIONAL VALUE:

Three operational modes for different scenarios:
1. MIDI-First: Real-time practice with immediate feedback
2. Mic-First: For acoustic piano without MIDI
3. Hybrid Pro: Backend processing for complex transcriptions

Use cases:
- Teachers: Create interactive educational materials
- Students: Learn with real-time feedback and metrics
- YouTubers: Produce professional piano tutorials
- Musicians: Transcribe and analyze music from videos

This application transforms basic video annotation into a complete
educational platform with AI-powered features, professional-grade
tools, and comprehensive learning analytics.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-27 21:17:51 +00:00
Mahesh Murag 0877baec7e
Updates to README.md (#9) 2025-10-16 15:15:19 -04:00
Keith Lazuka b118d29ea4
Add Claude Claude instructions to the readme (#8) 2025-10-16 12:57:23 -04:00
Keith Lazuka 4d1e3f3f78
Add Claude Code Marketplace (#5) 2025-10-16 12:11:41 -04:00
Matt Piccolella 9b6100367b
Small tweak to blog link (#7) 2025-10-16 12:08:28 -04:00
Keith Lazuka 10e0fbe1a4
Add initial Agent Skills Spec (#2) 2025-10-16 11:32:56 -04:00
Keith Lazuka ec841043a5
Add 3rd Party notices (#4) 2025-10-16 08:32:27 -07:00
Matt Piccolella 67ada86353
Adding more details to README (#3)
* Adding details about document skills

* Adding partner skills section
2025-10-16 08:22:54 -07:00
Keith Lazuka 83291af582
Reorganize the example skills (#1)
* Rename to document-skills

* rearrange
2025-10-16 10:02:06 -04:00
Keith Lazuka 37292f37bf
init repo 2025-10-15 17:07:32 -04:00