Shop OBEX P1 Docs P2 Docs Learn Events
Speech Synthesis on the Propeller — Parallax Forums

Speech Synthesis on the Propeller

cgraceycgracey Posts: 14,256
edited 2009-06-30 15:27 in Propeller 1
Here is the data I used for the speech synthesis presentation·last night·at the UPEW.

This represents a lot of hard-won knowledge that's taken me years to accumulate. It's been·one of the most fascinating endeavors·of my life.·I hope a few of you out there will get some enjoyment from this. If you are interested enough to spend a few hours delving into the .zip below, reading the documentation, running the examples, and doing some experimentation of your own, you may get·the bug, yourself.

I wish you fun and learning.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


Chip Gracey
Parallax, Inc.
zip
zip
Propeller Speech Synthesis.zip
2M

Comments

  • DogPDogP Posts: 168
    edited 2009-06-29 02:15
    Wow! That's awesome! Or at least if the result is as good as I imagine it is after reading the document [noparse]:)[/noparse]. I'll definitely have to hook up some headphones to my proto board and test this out. Like you, this is also something that has interested me very much, so that's really cool to have something like this that I can try out and tweak to understand it better.

    It definitely would have been helpful last year when I was trying to do speech on an old video game system (in the end, I basically just made recordings, then converted those to a spectrograph and output the 4 strongest waves, frequently updating the output). I'll have to combine this with random parameters and get a modern day Q*Bert speech synthesis [noparse]:)[/noparse]. Ha... I love this quote from your doc: "Once mastered, though, you will have gained untold insight into the mechanics of speech that you’ll be eager to share with others who will be less interested"... so true, for just about everything I do [noparse];)[/noparse].

    DogP
  • Peter JakackiPeter Jakacki Posts: 10,193
    edited 2009-06-29 02:45
    "Hear Hear", good work Chip!

    I've just been playing with it and I was wondering if you had built up a speech object and tables so that you could pass byte/word parameters in a buffered output stream to it plus extend that table with macros. Obviously the examples are hard coded and I did just look at modifying some parts of it to hear the result, it's quite interesting. Probably what I'd like to do is build up some word/syllable tables so that I can tie this into a voice prompt application and pass actual word/character/syllable/phoneme parameters direct from the application.

    If I don't get too bogged down with actual work I might have some fun and proceed along these lines and see where I can build on this excellent vocal-tract micro-engine.

    *Peter*
  • BADHABITBADHABIT Posts: 138
    edited 2009-06-29 03:22
    It would be awesome if you could rotate the spectrum vs. time around a point selected by the mouse and view a 3d topographic model of it.

    Good Stuff.

    I have wanted an instruction set for the synthesis object ever since I first played with it. This is great. It combines super technical mumbo jumbo with simple easy to follow explanations of how to make that object really "sing".

    I have been wanting to do more than make the Singing Sevens say dirty words for·some time, now I can.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    BH skull.gif

    Post Edited (BADHABIT) : 6/29/2009 3:44:18 AM GMT
  • PhilldapillPhilldapill Posts: 1,283
    edited 2009-06-29 04:48
    I, too, got "the bug" a couple of semesters back - except in reverse. In one of my EE classes, we had a group project to design some software in MatLab which would do speech recognition - from the ground up. We also had the option of developing some external hardware filters, that would further help the program figure out what words were being said. Since we had to do all this from scratch, I had the chance to do massive amounts of research in the subject.

    It's really fascinating. If you've ever done some speech-to-text "writing", it seems simple enough, but boy you would be wrong. Speech recognition and speech synthesis is an absolutely fascinating field.

    Thank you, Chip, for your hardwork in producing this gem. [noparse]:)[/noparse]
  • cgraceycgracey Posts: 14,256
    edited 2009-06-29 05:28
    Peter Jakacki said...


    ...I've just been playing with it and I was wondering if you had built up a speech object and tables so that you could pass byte/word parameters in a buffered output stream to it plus extend that table with macros...
    Yes! I have made a simple recursive·executor in Spin which is very data-efficient. It's attached at the bottom of this message.


    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


    Chip Gracey
    Parallax, Inc.
    spin
    spin
    VocalData.spin
    4K
  • Peter JakackiPeter Jakacki Posts: 10,193
    edited 2009-06-29 05:49
    Curious though, I'm wondering whether perhaps you have modeled the vocal tract upon your voice in the way that you might speak to your 2 year old when you are playing monsters!

    "Mommyyyy, Dad's scaring me again" ...... "Chip, you keep that "talking prop head" away from our daughter!"

    Who says you can't combine work and play. Obviously they haven't worked with the Propeller chip.

    *Peter*
  • Ole Man EarlOle Man Earl Posts: 262
    edited 2009-06-29 05:50
    Burl...Pearl..........Earl, can you hear me now !
    Yup, and what a Great Expo it was ! Thanks.
  • BaggersBaggers Posts: 3,019
    edited 2009-06-29 15:22
    What an excellent read that was Chip, thanks [noparse]:)[/noparse] I shall look forward to having a play with doing some speech later Cheers [noparse]:)[/noparse]

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    http://www.propgfx.co.uk/forum/·home of the PropGFX Lite

    ·
  • Jimmy W.Jimmy W. Posts: 112
    edited 2009-06-29 17:46
    Peter Jakacki said...
    "Mommyyyy, Dad's scaring me again" ...... "Chip, you keep that "talking prop head" away from our daughter!"

    Hmm, you could examine the formants, nasal params, etc and get a pretty good idea of when the mouth should be doing I think, anyone for a prop head with moving lips?
  • Peter JakackiPeter Jakacki Posts: 10,193
    edited 2009-06-29 23:52
    Yeah, I think it's been done already, just take a look at the forum members list [noparse]:)[/noparse]

    *Peter*
  • Oldbitcollector (Jeff)Oldbitcollector (Jeff) Posts: 8,091
    edited 2009-06-30 14:12
    Chip's presentation on this was excellent...

    I learned much about speech and how the ear functions.
    Between that and some interesting discussion with Beau Monday
    during lunch, I have an interesting idea for a related project.

    I am starting to wonder if Chip reads biology and life science
    encyclopedias in his "spare" time for entertainment purposes. [noparse]:)[/noparse] [noparse]:)[/noparse]

    OBC

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    New to the Propeller?

    Visit the: The Propeller Pages @ Warranty Void.
  • sssidneysssidney Posts: 64
    edited 2009-06-30 15:27
    Jimmy W. said...
    Peter Jakacki said...
    "Mommyyyy, Dad's scaring me again" ...... "Chip, you keep that "talking prop head" away from our daughter!"

    Hmm, you could examine the formants, nasal params, etc and get a pretty good idea of when the mouth should be doing I think, anyone for a prop head with moving lips?

    Interesting. I have a talking head (2d) project driven by a prop that talks in synch to a wav file (played by the same prop) but it requires external data (stored in a data file) to determine the mouth movement timing associated with the phonemes. It would be cool to be able to drive the mouth movements directly from the talk track.

    Here's a link to a video of it talking:

    http://sidnancy.posterous.com/

    Post Edited (sssidney) : 6/30/2009 3:45:16 PM GMT
Sign In or Register to comment.