Android app that uses human voice to interact with the robot

silviu.lita

Hi all,

I've just finished working on an Android application which uses Speech-to-text and Text-to-speech to interact wth the robot.
All bi-directional interaction is based on voice, no buton clicks to command the robot.

What you need is:
- An typical Android device like a smart phone with bluetooth and speech-to-text, text-to-speech engines enabled
- A Basic STAMP or thelike with bluetooth module and a program that can receive and send serial data.

How it works:
- Install the Boebot.apk application on your phone
- Configure the MAC bluetoot address of the robot (it's assumed the the bluetooth devices are already paired)
- Configure a set of commands for the robtot in format "voice commands: byte command" to send to the robot. For example: "hello,hi:h" or "left:l" , etc.
- Confgure a set of replies from the robot in format "byte command: text to play". For example: "o: my name is Boe, awaiting your orders"
- Click a button and start speaking. No further interaction is required.
As you speak, voice commands will be mapped to robot commands and sent via bluetooth to your Basic STAMP program.
The communication uses the default 9600 baud rate.

I'm attaching the apk installation program for Android. To install, you need to enable installing programs from other sources in Settings\Security.

The Basic STAMP program can be anywhere between trivial (like the one below) and a complex one with plenty of commands and replies.
A simple one could look like:

' {$STAMP BS2}
' {$PBASIC 2.5}

BT_RX           PIN     0               ' RN-42 RX Pin
BT_TX           PIN     1               'RN-42 TX Pin

Baud            CON     84              ' 9600 bps

cmd       VAR     Byte            ' Command from BT
Main:
  HIGH BT_TX                               ' Initialize TX line
  PAUSE 100                             ' Allow time to settle
  DO
    DEBUG "Waiting serial...", CR
    SERIN BT_RX, Baud, [cmd]
    DEBUG ASC ? cmd, "d=", DEC cmd, CR
    IF cmd = 104 THEN    'if command is h (hi, hello)
      SEROUT BT_TX, Baud, ["o"] 'reply o => which will be played on the phone as "my name is Boe, awaiting your orders
    ENDIF
  LOOP

Hope you'll find it funny.
Cheers,
Silviu

ValeT

Cool Project!

Have you had it do anything when you give it commands, such as "go forward for 5 seconds" will make it move forward for 5 seconds?

silviu.lita

Hello,

In my setup, I use about 20 commands and 20 replies.
The robot can move on commands, use ping to go to closest objects, beam request colors using PAL, play music on speaker, dance , the Android app can already play music on command 'd', etc.
On the robot side, you can use SERIN with timeout so that you don't get stuck waiting for human cmd from the Android app.

The idea was to get the Boebot closer to Asimo.:)

The next thing I'm thinking about is a gripper. I'm however not happy with the Parallax one, it's too big and does not go with Ping mounting. Looking for something else.

Silviu

ValeT

Good luck on the project!