Voice Synthesis with Lip-syncing
Phil Pilgrim (PhiPi)
Posts: 23,514
A couple years back I attempted to produce a phonemic voice synthesizer which used Chip's excellent vocal tract object. My effort produced sounds that only a parent could love and pretend to understand, but it was fun trying. Now, in an effort to compound my folly, I've added a video mouth to the sound, which renders it no less garbled but somewhat more entertaining. (Actually, this addition has resulted from a spell of avoidance behavior that got out of hand. I'm really supposed to be documenting a new board right now.)
The mouth is formed in graphics.spin (with a severely reduced tile count) by the following polylines, in order from back to front:
····Tongue: a single, thin, red, horizontal line.
····Lower teeth: a single, thick, white horizontal line.
····Lower lip: three thick, red lines.
····Upper teeth: a single, thick, white horizontal line.
····Upper lip: four thick, red lines.
Note that the upper teeth are in front of the lower lip. This was done to make fricatives easy to show without requiring a special case. It also gives the mouth a somehwat buck-toothed appearance.
The number of mouth positions required for lip syncing is quite a bit fewer than the number of sounds produced. This is due to the fact that the tongue, which is mostly hidden, does a lot of the work in producing distinct vocalizations. The mouth positions that I used are derived from those presented here. Transistions between positions are performed by linearly interpolating the line segments' endpoints.
Attached is my original demo program, modified to include lip syncing. It will run on the Propeller Demo Board, with NTSC video out and audio coming from the right channel. It can be adapted for other boards by changing the pin assignments and clock parameters as necessary. It requires a clock speed of 80MHz to function and may work at higher speeds as well (which I haven't tried).
Enjoy! (Now I guess I'd better get back to the work I'm supposed to be doing.)
-Phil
Post Edited (Phil Pilgrim (PhiPi)) : 2/4/2009 11:52:37 PM GMT
The mouth is formed in graphics.spin (with a severely reduced tile count) by the following polylines, in order from back to front:
····Tongue: a single, thin, red, horizontal line.
····Lower teeth: a single, thick, white horizontal line.
····Lower lip: three thick, red lines.
····Upper teeth: a single, thick, white horizontal line.
····Upper lip: four thick, red lines.
Note that the upper teeth are in front of the lower lip. This was done to make fricatives easy to show without requiring a special case. It also gives the mouth a somehwat buck-toothed appearance.
The number of mouth positions required for lip syncing is quite a bit fewer than the number of sounds produced. This is due to the fact that the tongue, which is mostly hidden, does a lot of the work in producing distinct vocalizations. The mouth positions that I used are derived from those presented here. Transistions between positions are performed by linearly interpolating the line segments' endpoints.
Attached is my original demo program, modified to include lip syncing. It will run on the Propeller Demo Board, with NTSC video out and audio coming from the right channel. It can be adapted for other boards by changing the pin assignments and clock parameters as necessary. It requires a clock speed of 80MHz to function and may work at higher speeds as well (which I haven't tried).
Enjoy! (Now I guess I'd better get back to the work I'm supposed to be doing.)
-Phil
Post Edited (Phil Pilgrim (PhiPi)) : 2/4/2009 11:52:37 PM GMT
Comments
-Phil
Excellent work here though
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Cheers,
Simon
www.norfolkhelicopterclub.com
You'll always have as many take-offs as landings, the trick is to be sure you can take-off again
BTW: I type as I'm thinking, so please don't take any offence at my writing style
I nearly lost my first board to the kids when I played that for them! I once wrote them a program using MSagent(A text to speech talking teddy bear) They played with it for hours, and sometimes still do. But when they heard that voice, they said it sounded more like a computer than my program in visual basic. I had to explain to them that I was just goofing off and needed my board for other things. I'm just glad they didn't get the Miss'es involved or I'd of been out· building me another board to learn the propeller on.
Wild tangents seem to help a persons work be more prductive(all work and jack a dull boy kind of thing)
Great job Phil!