Realtime capabilities of madmom

[space-mob]

Hi @superbock
first and foremost merry xmas 🎅 and thank you for all the work you put into this library.
Over the last few days i've been doing some research regarding audio feature extraction and music information retrieval.
I'm especially interested in applications where future data is unknown, i.e., real-time / streamed processing.

I've gone through opened/closed issues and the old discussions and found several related threads:

• DBNBeatTracker on Stream Input
• online onset RNN detection
• process audio input from microphone?
• Documentation/Examples for online mode/real-time processing
• Real time beat detection
• etc.

From my understanding you have already integrated pyAudio in some of the processors, most of them neural network based.
To cut to the chase - i'm especially curious if the implementation based on your SuperFlux paper already works to extract an onset envelopes in real-time?

If not, what would need to be done to make it work? I assume we would need an additional layer to the pyAudio stream that would maybe have some kind of rolling buffer and handle the overlap?

best regards,
david

[superbock]

Hi David,

Sorry for the late reply!

No SuperFlux is not online ready yet. However, it is not too complicated to achieve this. The only missing part is to cache the previous spectrogram frame to be able to compute the difference of the current frame with it. If you are still interested in doing this I can give you more detailed hints on how to accomplish this.