Windows 10 Speech Recognition

Question

I want to create a WPF application in c# for windows 10. Now, the problem that i had with previous windows versions was that i'm italian and there isn't a support for speech recognition in italian. But now there is cortana. So, how can i use cortana's speech recognition engine for my application? If i simply use new SpeechRecognitionEngine(new CultureInfo("it-IT"))); it gives me an error, 'cause there isn't the simple recongition engine, so i have to use cortana's one. Hope you understood and sorry for my bad english. Thank you for your answer.

Solution

In order to use the new SpeechRecognition WinRT API released in windows 10, you're going to need to add support for WinRT APIs to your desktop C# application. This doesn't require converting the app to a Windows Store app, however, at least, for some parts. So far as I know, the new engine hasn't been backported to add support into System.Speech.SpeechRecognitionEngine, that still uses a legacy recognizer (I'll check with the speech team here and follow up in this post if I find more on that point.)

Based on the guidance taken from here and here, I was able to create a classic c# WPF app, and implement the following code:

private SpeechRecognizer reco;

    public MainWindow()
    {
        InitializeComponent();

        reco = new SpeechRecognizer();
        List<string> constraints = new List<string>();
        constraints.Add("Yes");
        constraints.Add("No");
        reco.Constraints.Add(new SpeechRecognitionListConstraint(constraints));
        IAsyncOperation<SpeechRecognitionCompilationResult> op = reco.CompileConstraintsAsync();
        op.Completed += HandleCompilationCompleted;
    }

    public void HandleCompilationCompleted(IAsyncOperation<SpeechRecognitionCompilationResult> opInfo, AsyncStatus status)
    {
        if(status == AsyncStatus.Completed)
        {
            System.Diagnostics.Debug.WriteLine("CompilationCompleted");
            var result = opInfo.GetResults();
            System.Diagnostics.Debug.WriteLine(result.Status.ToString());
        }
    }

In order to get this to compile, I added

  <PropertyGroup>
    <TargetPlatformVersion>10.0</TargetPlatformVersion>
  </PropertyGroup>

to the .csproj, and added Windows.Media and Windows.Foundation from the Project -> Add References -> Universal Windows -> Core section, and I also manually added references to

C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETCore\v4.5.1\System.Runtime.WindowsRuntime.dll

and

C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETCore\v4.5.1\System.Runtime.InteropServices.WindowsRuntime.dll

via the browse section of Add References.

You'll need to check the SpeechRecognizer.SupportedGrammarLanguages to retrieve the it-IT Language object to pass it to the Recognizer constructor, if your system isn't defaulting to it-IT already. (IF you installed an Italian version of windows 10, this should happen by default)

Now, my code snippet above only compiles a super simple grammar, it doesn't start recognition. You'll need to consult the rest of the Windows.Media.SpeechRecognition API for that, but it's along the same lines.

Comments

[pianka]

Thank you very much, but i have a little problem... my Project -> Add References -> Universal Windows -> Core section is empty and i can't add Windows.Media and Windows.Foundation. Where can i find them via the browse section?

[Andrew Pilley]

Hm, have you installed the Universal Windows App SDK via Visual studio's setup? It's not installed by default when you install VS2015, you may need to go back into VS's setup and add it if it's not installed (You can use 'modify' from the Add/remove Programs part of windows 10). Oh, and I just double-checked the rest of the solution, and made sure you can actually start recognition, and that does work (Even RecognizeWithUIAsync although the UI looks out of place over a WPF app :) )

[pianka]

i have reinstalled Visual Studio and Core section is empty again. It says "Unable to find refers in the Universal Windows SDK" (hope i have translated it correctly)

[Andrew Pilley]

Hm. I can't reproduce that problem. On a completely clean system, I installed VS, created a C# WPF project, added the PropertyGroup/TargetPlatformVersion section to the CSProj, and I was able to add the references just fine, even without the UWP SDK installed. I have deployed a UWP app to it via the remote debugger prior to this point however, so perhaps that installed something. Can you try creating and building a UWP app just to make sure that works?

[pianka]

yes it works, but i can't understand how to use this speech recognition API

[Andrew Pilley]

We have examples for using the API in the official windows samples here if you're looking for examples on how to set things up for different scenarios (dictation, grammars, etc).

[Arctic Vowel]

@AndrewPilley-MSFT The code you posted works well, but I tried to call the "RecognizeWithUIAsync" that you'd mentioned, but the compiler says "IAsyncAction is defined in an assembly that is not referenced, add reference to Windows". Do you know what I'm doing wrong ? MSDN says SpeechRecognizer is contained within Windows.Media.SpeechRecognition.dll.

[Andrew Pilley]

@AniruddhaVarma Hi. It sounds, on the face of it like you haven't added "using Windows.Foundation;" to your using statements.

[Arctic Vowel]

@AndrewPilley-MSFT Hi, thank you for your response. I added the first file (*.winmd) mentioned here blogs.windows.com/buildingapps/2017/01/25/… in addition to the two dll's from your answer, and everything (including UI) is working perfectly.

[Alan Wayne]

The above procedure was also needed for Visual Studio 2017 Community to create a WPF project using Speech to Text. :)