Video Capture output always in 320x240 despite changing resolution

Question

Ok I have been at this for 2 days and need help with this last part.

I have a Microsoft LifeCam Cinema camera and I use the .NET DirectShowLib to capture the video stream. Well actually I use WPFMediaKit, but I am in the source code of that dealing directly with the direct show library now.

What I have working is: - View the video output of the camera - Record the video output of the camera in ASF or AVI (the only 2 MediaType's supported with ICaptureGraphBuilder2)

The problem is: I can save it as a .avi. This works fine and at a resolution of 1280x720 but it saves the file in RAW output. Meaning it is about 50-60MB per second. Way too high.

Or I can switch it to .asf and it outputs a WMV, but when I do this the capture and the output both go to resolution 320x240.

In WPFMediaKit there is a function I changed because apparently with Microsoft LifeCam Cinema cameras a lot of people have this problem. So instead of creating or changing the AMMediaType you iterate through and then use that to call SetFormat.

        ///* Make the VIDEOINFOHEADER 'readable' */
        var videoInfo = new VideoInfoHeader();

        int iCount = 0, iSize = 0;
        videoStreamConfig.GetNumberOfCapabilities(out iCount, out iSize);

        IntPtr TaskMemPointer = Marshal.AllocCoTaskMem(iSize);


        AMMediaType pmtConfig = null;
        for (int iFormat = 0; iFormat < iCount; iFormat++)
        {
            IntPtr ptr = IntPtr.Zero;

            videoStreamConfig.GetStreamCaps(iFormat, out pmtConfig, TaskMemPointer);

            videoInfo = (VideoInfoHeader)Marshal.PtrToStructure(pmtConfig.formatPtr, typeof(VideoInfoHeader));

            if (videoInfo.BmiHeader.Width == DesiredWidth && videoInfo.BmiHeader.Height == DesiredHeight)
            {

                ///* Setup the VIDEOINFOHEADER with the parameters we want */
                videoInfo.AvgTimePerFrame = DSHOW_ONE_SECOND_UNIT / FPS;

                if (mediaSubType != Guid.Empty)
                {
                    int fourCC = 0;
                    byte[] b = mediaSubType.ToByteArray();
                    fourCC = b[0];
                    fourCC |= b[1] << 8;
                    fourCC |= b[2] << 16;
                    fourCC |= b[3] << 24;

                    videoInfo.BmiHeader.Compression = fourCC;
                   // pmtConfig.subType = mediaSubType;

                }

                /* Copy the data back to unmanaged memory */
                Marshal.StructureToPtr(videoInfo, pmtConfig.formatPtr, true);

                hr = videoStreamConfig.SetFormat(pmtConfig);
                break;
            }

        }

        /* Free memory */
        Marshal.FreeCoTaskMem(TaskMemPointer);
        DsUtils.FreeAMMediaType(pmtConfig);

        if (hr < 0)
            return false;

        return true;

When that was implemented I could finally view the captured video as 1280x720 as long as I set the SetOutputFilename to a MediaType.Avi.

If I set it to a MediaType.Asf it goes to 320x240 and the output is the same.

Or the AVI works and outputs in the correct format but does so in RAW video, hence a very large file size. I have attempted to add a compressor to the graph but with no luck, this is far out of my experience.

I am looking for 1 of 2 answers.

1. Recording the ASF at 1280x720
2. Adding a compressor to the graph so that the filesize of my outputted AVI is small.

Solution 1

I figured this out. So I am posting it here for any other poor soul who passes by wondering why it doesn't work.

1. Download the source of the WPFMediaKit, you are going to need to change some code.

2. Go to Folder DirectShow > MediaPlayers and open up VideoCapturePlayer.cs

3. Find the function SetVideoCaptureParameters and replace it with this:

   /// <summary>
   /// Sets the capture parameters for the video capture device
   /// </summary>
   private bool SetVideoCaptureParameters(ICaptureGraphBuilder2 capGraph, IBaseFilter captureFilter, Guid mediaSubType)
   {
       /* The stream config interface */
       object streamConfig;
   
       /* Get the stream's configuration interface */
       int hr = capGraph.FindInterface(PinCategory.Capture,
                                       MediaType.Video,
                                       captureFilter,
                                       typeof(IAMStreamConfig).GUID,
                                       out streamConfig);
   
       DsError.ThrowExceptionForHR(hr);
   
       var videoStreamConfig = streamConfig as IAMStreamConfig;
   
       /* If QueryInterface fails... */
       if (videoStreamConfig == null)
       {
           throw new Exception("Failed to get IAMStreamConfig");
       }
   
       ///* Make the VIDEOINFOHEADER 'readable' */
       var videoInfo = new VideoInfoHeader();
   
       int iCount = 0, iSize = 0;
       videoStreamConfig.GetNumberOfCapabilities(out iCount, out iSize);
   
       IntPtr TaskMemPointer = Marshal.AllocCoTaskMem(iSize);
   
   
       AMMediaType pmtConfig = null;
       for (int iFormat = 0; iFormat < iCount; iFormat++)
       {
           IntPtr ptr = IntPtr.Zero;
   
           videoStreamConfig.GetStreamCaps(iFormat, out pmtConfig, TaskMemPointer);
   
           videoInfo = (VideoInfoHeader)Marshal.PtrToStructure(pmtConfig.formatPtr, typeof(VideoInfoHeader));
   
           if (videoInfo.BmiHeader.Width == DesiredWidth && videoInfo.BmiHeader.Height == DesiredHeight)
           {
   
               ///* Setup the VIDEOINFOHEADER with the parameters we want */
               videoInfo.AvgTimePerFrame = DSHOW_ONE_SECOND_UNIT / FPS;
   
               if (mediaSubType != Guid.Empty)
               {
                   int fourCC = 0;
                   byte[] b = mediaSubType.ToByteArray();
                   fourCC = b[0];
                   fourCC |= b[1] << 8;
                   fourCC |= b[2] << 16;
                   fourCC |= b[3] << 24;
   
                   videoInfo.BmiHeader.Compression = fourCC;
                  // pmtConfig.subType = mediaSubType;
   
               }
   
               /* Copy the data back to unmanaged memory */
               Marshal.StructureToPtr(videoInfo, pmtConfig.formatPtr, true);
   
               hr = videoStreamConfig.SetFormat(pmtConfig);
               break;
           }
   
       }
   
       /* Free memory */
       Marshal.FreeCoTaskMem(TaskMemPointer);
       DsUtils.FreeAMMediaType(pmtConfig);
   
       if (hr < 0)
           return false;
   
       return true;
   }
   

Now that will sort out your screen display at what ever desired resolution you want, provided that your camera supports it.

Next you will soon figure out that this new correct capture you have isnt applied when writing the video to disk.

Since the ICaptureBuilder2 method only supports Avi and Asf (which is wmv) you need to set your mediatype to one of them.

hr = graphBuilder.SetOutputFileName(MediaSubType.Asf, this.m_fileName, out mux, out sink);

You will find that line in the SetupGraph function.

Asf will only output in 320x240, yet the Avi will output in the desired resolution, but uncompressed (meaning 50-60MB per second for a 1280x720 video feed), which is too high.

So that leaves you with 2 options

1. Figure out how to add a encoder (compression filter) to the Avi output

2. Figure out how to change the WMV profile

I tried 1, with no success. Mainly due to the fact this is my first time working with DirectShow and only just grasp the meaning of graphs.

But I was successful with #2 and here is how I did it.

Special thanks to (http://www.codeproject.com/KB/audio-video/videosav.aspx) I pulled out the needed code from here.

1. Create a new class in the same folder called WMLib.cs and place the following in it

Download the demo project from http://www.codeproject.com/KB/audio-video/videosav.aspx and copy and paste the WMLib.cs into your project (change the namespace as necessary)

1. Create a function in the VideoCapturePlayer.cs class

   /// <summary>
   /// Configure profile from file to Asf file writer
   /// </summary>
   /// <param name="asfWriter"></param>
   /// <param name="filename"></param>
   /// <returns></returns>
   public bool ConfigProfileFromFile(IBaseFilter asfWriter, string filename)
   {
       int hr;
       //string profilePath = "test.prx";
       // Set the profile to be used for conversion
       if ((filename != null) && (File.Exists(filename)))
       {
           // Load the profile XML contents
           string profileData;
           using (StreamReader reader = new StreamReader(File.OpenRead(filename)))
           {
               profileData = reader.ReadToEnd();
           }
   
           // Create an appropriate IWMProfile from the data
           // Open the profile manager
           IWMProfileManager profileManager;
           IWMProfile wmProfile = null;
           hr = WMLib.WMCreateProfileManager(out profileManager);
           if (hr >= 0)
           {
               // error message: The profile is invalid (0xC00D0BC6)
               // E.g. no <prx> tags
               hr = profileManager.LoadProfileByData(profileData, out wmProfile);
           }
   
           if (profileManager != null)
           {
               Marshal.ReleaseComObject(profileManager);
               profileManager = null;
           }
   
           // Config only if there is a profile retrieved
           if (hr >= 0)
           {
               // Set the profile on the writer
               IConfigAsfWriter configWriter = (IConfigAsfWriter)asfWriter;
               hr = configWriter.ConfigureFilterUsingProfile(wmProfile);
               if (hr >= 0)
               {
                   return true;
               }
           }
       }
       return false;
   }
   

2. In the SetupGraph function find the SetOutputFileName and below it put

   ConfigProfileFromFile(mux, "c:\wmv.prx");

3. Now create a file called wmv.prx on your c: drive and place the relevant information in it.

You can see a sample of a PRX file from the demo project here: http://www.codeproject.com/KB/audio-video/videosav.aspx (Pal90.prx)

And now enjoy your .wmv file outputted at the right size. Yes I know the code I placed in was rather scrappy but I will leave it up to you to polish it up.

Solution 2

Lifecam is known for unobvious behavior with setting capture format (more specifically, falling back to other formats). See prevoius discussions which are likely to suggest you a solution:

• Can't make IAMStreamConfig.SetFormat() to work with LifeCam Studio
• IAMStreamConfig settings are ignored by stream - Microsoft LifeCam Cinema records only on 640x480
• Can Microsoft LifeCam record at a frame rates lower than 30 fps?

Comments

[leppie]

You are likely hitting a limitation of the codec. Try 640x480.

[Adam]

I've tried many resolutions, the AVI changes accordingly but not the ASF.

[Stephen Chung]

@Adam, if you are compressing WMV, you'll need to query the ASF interfaces on the WMV compression filter to load a "profile" (which basically is an XML file on disk) which contains the resolution, compression modes etc. The WMV compressor defaults (I believe) to 320x240, so it scales everything down to that size.

[Adam]

Hi. Thanks for responding. I have noticed your name around the place as I searched Google for answers. I have read all of the above links and many more. One of the above links is where I got the ability to record in 1280x720 as per my code above. However I am still stuck on either adding a compression filter or getting ASF to support the correct resolution. I have tried things like adding the Matroska filter // MKV Guid guid = new Guid("1E1299A2-9D42-4F12-8791-D79E376F4143"); Type comtype = Type.GetTypeFromCLSID(guid);

[Adam]

Continued ... IBaseFilter matroska_mux = (IBaseFilter)Activator.CreateInstance(comtype); hr = graphBuilder.RenderStream(PinCategory.Capture, MediaType.Video, m_captureDevice, matroska_mux, mux);

[Adam]

And just to add, I successfully installed the matroskamux.ax and the filter is created successfully. It just crashes when I add it in the compressor part of the Renderstream

[Adam]

DirectShow is very new to me so please elaborate on any answers given, it would be much appreciated.

[Roman R.]

I see you figured it out and fixed via WM ASF Writer. By setting the profile, you enable it to scale the video internally, so the camera itself might be capturing in a better or worse resolution/mode. Looks like you are happy with the video capture, so it works quite nice and the mode is good.

[Adam]

Just thought I would add that originally via the information I found it looked to be a Microsoft LifeCam issue. So considering hours of my time is worth more than $49 for a new webcam, I bought a Logitech HD Webcam C510. It still had the same issue. It must be common with the newer HD webcams.

[Adam]

Thanks Roman. Yes all working finally. The quality and resolutions are great as well.

[Stephen Chung]

I'm sure there are some calls on the interface that allows you to load a profile (.prx) file directly instead of having to read it into a string first. I've used it before when compressing a video stream into WMV using the WM ASF Writer filter, but it was a long time ago, and I can't remember it.

[Eternal21]

Thanks a lot for taking the time to post your solution. Had to use the prx profile workaround as well. It's amazing how crappy the support for video capture is in .NET.