Voice control over desktop environment

Question

I would like to enable a system that can respond to user voice commands and run scripts according to the verbal command GIVEN.

Ideally, I would like for the computer to be listening constantly.

Once it hears the buzz word "Computer" it will activate and listen for commands for 10 seconds (or so).

Once active, and while it's listening, I would like for another set of buzz words to be able to run scripts.

For instance....

(Computer is idle) "Computer" (computer immediately listens) "Access new MESSAGES" (MESSAGES being buzz word *IGNORES "ACCESS NEW" -- computer runs a script and opens evolution or empathy). There could be one for browser, one for closing windows and showing desktop, or a bunch of stuff really.

Is this even possible?

Solution 1

Gnome-Voice-Control

Developed as part of Google Summer of Code 2007, this was an idea to develop some features that will improve the usability in the Gnome Desktop.

The goal is to implement a Desktop Voice Control System. The system consists in an application that will be monitoring the audio input(microphone) and when a significant audio signal has been detected, the software catches, processes and recognizes the signal and then executes the desired action over the Gnome Desktop. In a set of actions could include maximize, minimize, close the active window; open a specific program; switching from one desktop to another; among others. GnomeVoiceControl is implemented in C in conjunction with CMU Sphinx, which is an open source tool, created to convert speech to text.

Slide-show by the authors

Unfortunately it looks like it has not developed much beyond the basic "goals" - however there does appear to be some activity adding support for new languages such as Bulgarian and Turkish this year.

To install:

sudo apt-get install gnome-voice-control

Platypus

There is an interesting looking project call platypus - basically is a linux front-end to Dragon Naturally Speaking which runs in Wine.

The claim-to-fame for this application is "it can even launch Windows or Linux programs and scripts, e.g. "start terminal".

VEDICS

This is a newish project that says it works with the Unity interface.

The project page says it recognises "run [program name]" - so in theory you [program name] could be any script you want to run.

Solution 2

Google2Ubuntu

Google2Ubuntu is a tool that lets you control your computer using voice commands via the Google speech recognition API.

Run the below commands on terminal to install google2Ubuntu,

sudo add-apt-repository ppa:benoitfra/google2ubuntu
sudo apt-get update
sudo apt-get install google2ubuntu

Source

Solution 3

Simon Listens

This is an open source project aimed to replace mouse and/or keyboard by speech recognition initially developed for physically disabled people.

There is a ppa from where we can install the application: ppa:grasch-simon-listens/simon

Solution 4

Linux Speech Recognition

It is being developed to run under Linux distributions, it is very configurable and easy to extend, support custom dictionaries, context based recognition, dictation mode and configurable macros.

In order to install Linux Speech Recognition go to the project's GitHub page. Download the ZIP file and extract it.

Then install some dependencies:

sudo apt-get install sox python-argparse libsox-fmt-mp3 mutt xvkbd xautomation

Open the extracted folder (Palaver-master) in the terminal and execute the setup:

cd /~Path-to-folder~/Palaver-master/
./setup

Now it is just to assign a hotkey to enable recording, in order to do that go to System Settings... > Keyboard > Shortcuts > Custom Shortcuts click the Plus Button to add a shortcut, name it Voice Recognition and set the command to:

/~Path-to-folder~/Palaver-master/hotkey

Click apply, click over the shortcut and then assign a hotkey to activate Voice Recognition.

Notice that no stable build have been released yet, so don't expect commands to be fully functional at this stage, many phrases will be recognized, but no real action will be done.

View more solutions

Comments

[BorgDomination]

Thank you for the exceptionally fast response. I heard about that program and I installed it, but it doesn't seem to get past configuration -- it just says "config". I'm surprised there hasn't been much work done on this front. Is there maybe a windows equivilant that I could run with wine?

[Kendrick]

the older version of gnome voice that is in synaptic gave me the same results. I tried the latest from the website and it seemed to get further but I never did get it to work properly. the manual dependancys were a pain to.

[Forty-Two]

does simon only work with kde? It looks really awesome and I installed it but can't seem to get it working with gnome-shell. specially window management. If I want to show all windows for example, that is going to work differently in the kde env and gnome env right?

[Takkat]

Dependencies should be resolved when installing on Ubuntu <=12.04 from the ppa but I haven't tried recently, and also I have no idea how it may interfere with GNOME shell. There is no version (yet) for 12.10.

[user2740]

As of now, Palaver is dead: github.com/JamezQ/Palaver/issues/110#issuecomment-398646686

[Pablo Bianchi]

Gnome-Voice-Control: This project is unmaintained and has been archived.