Audio Signal Processing on the Propeller Demo Board

HemPeter · 2012-06-19 11:19

FFT powered scrolling musical note representation:

This is a demo that I've worked on for my dissertation this year. This project uses quite a few objects I've found and incorporated, including Heater's FFT.. http://forums.parallax.com/showthread.php?128292-Heater-s-Fast-Fourier-Transform.

Thanks for all the help I've received from the people on this forum. It took me a long time to get to this point, and I'm fairly happy with the result.
I hope that this will be of some interest to others. Hope it can be used for something awesome

I will be posting more objects on this thread, as I fully comment them up and manage to slim them down to be compiled with the PropTool (bst's compiler optimizations are deceptively useful)

Enjoy!

Pete

score_notation - Archive [Date 2012.06.19 Time 19.02].zip

cgracey · 2012-06-19 14:18

That's pretty neat! I listened to the kalimba music you recommended in the main file and the display definitely responded to it.

I think what you need to make here is a pitch detector, so you can affirmatively isolate any notes being played. I've never done this, and it's not supposed to be easy, but you'd look for several harmonics of the same fundamental before determining a note was present. You'd also want to see near-zero energy around those harmonic frequencies, or at least around some of them, so you don't trigger on broadband noise. It seems to me that maybe an FFT of an FFT would do this. If a log-scale-in-frequency Fourier is possible to compute, that would simplify things really nicely. Every bin could be a note. It would be better to do pitch detection, as well, from such a basis.

HemPeter · 2012-06-19 15:46

Hi Chip, thanks for responding.
I had a look into various algorithms for pitch detection and the constant Q transform sounds like what you described. My understanding is that you take log spaced samples from the input array and run FFT's on those, giving you log spaced outputs. I think the main issue-posing factor would be available RAM needed for real-time execution. There is a video of constant Q generating a spectrograph on the Propeller on the net, but I had no luck in contacting the author. I haven't found much in the way of understandable working examples (since I can't read Greek) so I decided to focus on getting something working for the demo

For now I'm happy to let go of the reins and if someone wants to pick them up and run then I'd be happy to see the result

HemPeter · 2012-06-19 15:54

Modified Microphone-to-VGA object to be a reciprocal frequency counter with note offset information.

This object is the basis for the backend of a VGA Frequency Counter Demo, also for my dissertation.

See if you can tune your whistle

Heater. · 2012-06-20 05:11

HemPeter,

Very interesting. Sadly I don't have a Prop with VGA capability around here so I can't see it in action. Any pics or videos?

Is your dissertation going to be available for us to read?

HemPeter · 2012-06-21 13:56

Hi Heater.

I spent today getting a video together, with muchos hassle on the behalf of youtube and their verification policy to allow videos over 15 mins.
So here's the link:
http://www.youtube.com/watch?v=E6JCXD6bbuc&hd=1

If you want to see your FFT in action skip to 14:00

The dissertation is available on the front page of the github repo I used during development. This page is also in the process of being updated:
http://petehemery.github.com/Final-Year-Project/Dissertation.pdf

Would be glad to hear your thoughts. I think the message at the end of the video sums it up. Hope you enjoy

Heater. · 2012-06-21 14:37

Oh bugger, now I am far away from where can even see the video. Only this phone to post with. Looking forward to getting back to civilization on monday to check this out.

Mark_T · 2012-06-24 04:05

HemPeter wrote: »

Hi Heater.

I spent today getting a video together, with muchos hassle on the behalf of youtube and their verification policy to allow videos over 15 mins.
So here's the link:
http://www.youtube.com/watch?v=E6JCXD6bbuc&hd=1

Excellent video - shows the strengths of the Prop and the object exchange nicely and looks like you've had a lot of fun

HemPeter · 2012-06-25 03:08

Hi Mark_T,

Thanks for your kind words

Glad you liked it, it's always nice to get positive feedback. Yes, fun..

there was a lot of swift keyboard/head interaction during development, but I'm happy with what I managed to achieve

More objects on the way, watch this space

HemPeter · 2014-04-01 17:59

So, after a bit of an absence, the demo video is back

http://www.youtube.com/watch?v=xQUO_jfl8SE

I'm afraid I let the ball drop somewhat on posting objects up here.

You can see all my work on the github pages:
http://petehemery.github.com/Final-Year-Project
https://github.com/PeteHemery/Final-Year-Project
https://github.com/PeteHemery/Final-Year-Project/tree/gh-pages

Wurlitzer · 2014-04-02 07:37

Quite interesting. It appeared (around the 16 min mark of the video) that it was able to detect simultaneous notes. If that is in fact correct, approx how many simultaneous note can be detected?

Heater. · 2014-04-02 11:56

HemPeter,

Well done on a great project and the video is wonderful.

@Wurlitzer

For the note detection HemPeter is using a Fast Fourier Transform (FFT).

With the FFT algorithm for N number of input samples you can distinguish N/2 different frequencies. At the same time.

Unless HemPeter modified my code it works on 1024 samples and can distinguish 512 different frequencies.

That of course is all theoretical. If you have 10,000 sample per second audio and you are analysing it 1024 samples at a time things get a bit muddied. Noise does not help.

Ultimately I can't answer the question about frequency resolution. Perhaps HemPeter can.

HemPeter · 2014-04-03 04:27

@Heater, thanks for the kind words

and thanks again for all your help during development.

@wurlitzer the demo in the video was sampling at 6kHz, with a filter cutting out frequencies higher than 1.5kHz
I hard coded the demo to look at some specific fft output array elements for values above a threshold. If a frequecy is present in the range of notes being looked for then it'll draw a line. It could potentially draw lines for all of them if there was a lot of noise. Theres a lot of processing when the drawing occurs so there could be a danger of stalling if there were a lot of lines being drawn for a long time..

The downside of this method is you do get some false positives when harmonics are present, so its only really useful for instruments that produce "clean" notes (not guitars that produce 2 or 3 harmonics when a string is plucked)

Hope that answers your question

Wurlitzer · 2014-04-03 11:29

Thanks HemPeter and Heater!

I was daydreaming about the possibility to capturing what notes were being played on a given piece (usually organ music). If no octave couplers (see below) were in play and all 10 fingers and both feet were playing something there would be 12 possible notes.

Octave coupler explanation (simplified) : Adding octave couplers simply allows a single key to play 1,2,3,4 notes with a fundamental and 3 additional notes each an octave higher than the last.

Most flue pipes have even and odd harmonics associated with them. A pipe with a stopper in the top only has odd harmonics. All bets are off on reed pipes however as they are rich in harmonics.

Audio Signal Processing on the Propeller Demo Board

Comments