cvondrick/soundnet
SoundNet learns sound representations from unlabeled video using a student-teacher training procedure that transfers visual model knowledge into the audio modality.
This research project from MIT/CSAIL learns rich natural sound representations by leveraging natural synchronization between vision and sound across two million unlabeled videos. The system uses a teacher-student training approach where discriminative visual knowledge from ImageNet and PlacesCNN models is transferred into sound representations. The 8-layer convolutional neural network is trained end-to-end to predict visual object and scene categories from audio, effectively learning audio features without explicit audio labels. Pre-trained models are provided for feature extraction and category recognition tasks.
Frequently asked
- What is cvondrick/soundnet?
- SoundNet learns sound representations from unlabeled video using a student-teacher training procedure that transfers visual model knowledge into the audio modality.
- Is soundnet open source?
- Yes — cvondrick/soundnet is open source, released under the MIT license.
- What language is soundnet written in?
- cvondrick/soundnet is primarily written in Lua.
- How popular is soundnet?
- cvondrick/soundnet has 466 stars on GitHub.
- Where can I find soundnet?
- cvondrick/soundnet is on GitHub at https://github.com/cvondrick/soundnet.