Why AVCaptureSession output a wrong orientation?

Question

So, I followed Apple's instructions to capture video session using AVCaptureSession: http://developer.apple.com/iphone/library/qa/qa2010/qa1702.html. One problem I'm facing is that even though the orientation of the camera / iPhone device is vertical (and the AVCaptureVideoPreviewLayer shows a vertical camera stream), the output image seems to be in the landscape mode. I checked the width and height of imageBuffer inside imageFromSampleBuffer: of the sample code, and I got 640px and 480px respectively. Does anyone know why this's the case?

Thanks!

Solution 1

Take a look at the header AVCaptureSession.h. There is a definition for an enum called AVCaptureVideoOrientation that defines various video orientations. On the AVCaptureConnection object there is a property called videoOrientation that is a AVCaptureVideoOrientation. You should be able to set this to change the orientation of the video. You probably want AVCaptureVideoOrientationLandscapeRight or AVCaptureVideoOrientationLandscapeLeft.

You can find the AVCaptureConnections for the session by looking at the outputs for the session. The outputs have a connections property that is an array of connections for that output.

Solution 2

Y'all are making this difficult.

In the DidOutputSampleBuffer, simply change the orientation before you grab the image. It's mono, but you have

    public class OutputRecorder : AVCaptureVideoDataOutputSampleBufferDelegate {    
        public override void DidOutputSampleBuffer (AVCaptureOutput captureOutput, CMSampleBuffer sampleBuffer, AVCaptureConnection connection)
        {
            try {
                connection.videoOrientation = AVCaptureVideoOrientation.LandscapeLeft;

in objC it's this method

- ( void ) captureOutput: ( AVCaptureOutput * ) captureOutput
   didOutputSampleBuffer: ( CMSampleBufferRef ) sampleBuffer
      fromConnection: ( AVCaptureConnection * ) connection

Solution 3

I made a simple one-line modification to the imageFromSampleBuffer to correct the orientation problem (see my comment in the code under "I modified ..."). Hope it helps someone because I spent too much time on this.

// Create a UIImage from sample buffer data
- (UIImage *) imageFromSampleBuffer:(CMSampleBufferRef) sampleBuffer  {
    // Get a CMSampleBuffer's Core Video image buffer for the media data
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); 
    // Lock the base address of the pixel buffer
    CVPixelBufferLockBaseAddress(imageBuffer, 0); 

    // Get the number of bytes per row for the pixel buffer
    void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer); 

    // Get the number of bytes per row for the pixel buffer
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); 
    // Get the pixel buffer width and height
    size_t width = CVPixelBufferGetWidth(imageBuffer); 
    size_t height = CVPixelBufferGetHeight(imageBuffer); 

    // Create a device-dependent RGB color space
    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); 

    // Create a bitmap graphics context with the sample buffer data
    CGContextRef context1 = CGBitmapContextCreate(baseAddress, width, height, 8, 
                                                 bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);

    // Create a Quartz image from the pixel data in the bitmap graphics context
    CGImageRef quartzImage = CGBitmapContextCreateImage(context1); 
    // Unlock the pixel buffer
    CVPixelBufferUnlockBaseAddress(imageBuffer,0);

    // Free up the context and color space
    CGContextRelease(context1); 
    CGColorSpaceRelease(colorSpace);

    // Create an image object from the Quartz image
    //I modified this line: [UIImage imageWithCGImage:quartzImage]; to the following to correct the orientation:
    UIImage *image =  [UIImage imageWithCGImage:quartzImage scale:1.0 orientation:UIImageOrientationRight]; 

    // Release the Quartz image
    CGImageRelease(quartzImage);

    return (image);
}

Solution 4

Here is a right sequence:

AVCaptureVideoDataOutput *videoCaptureOutput = [[AVCaptureVideoDataOutput alloc] init];

if([self.captureSession canAddOutput:self.videoCaptureOutput]){
    [self.captureSession addOutput:self.videoCaptureOutput];
}else{
    NSLog(@"cantAddOutput");
}

// set portrait orientation
AVCaptureConnection *conn = [self.videoCaptureOutput connectionWithMediaType:AVMediaTypeVideo];
[conn setVideoOrientation:AVCaptureVideoOrientationPortrait];

Solution 5

For instance:

AVCaptureConnection *captureConnection = <a capture connection>;
if ([captureConnection isVideoOrientationSupported]) {
    captureConnection.videoOrientation = AVCaptureVideoOrientationPortrait;
}

The default appears to be AVCaptureVideoOrientationLandscapeRight.

See also QA1744: Setting the orientation of video with AV Foundation.

View more solutions

Comments

[cheesus]

Property 'videoOrientation' not found on object of type 'AVCaptureSession *'

[Jim True]

this works for me, i added it to myAVCaptureConnection handler. It rotated the image as expected but the video is mirrored. I see that the isMirrored is deprecated so unclear how to fix the mirroring stuff b/c the new mirroring methods has no documentation on usage.

[freyaariel]

@Jim: Since mirrored in AVCaptureVideoPreviewLayer has been deprecated, you should use AVCaptureConnection’s videoMirrored property instead. It is documented, at least in the header files, where mirrored and its friends also explicitly refers you to videoMirrored and the matching capture connection methods.

[Jim True]

it looks like that feature isn't available in iOS sadly

[freyaariel]

@Jim: No way, dude! It’s right here in the online documentation as well: developer.apple.com/library/ios/documentation/AVFoundation/… ʘ‿ʘ What made you think it’s not available on iOS?

[GabCas]

@cheesus videoOrientation is on the AVCaptureConnection, not the AVCaptureSession

[Evgeniy S]

Thanks, for me it helps. note the method taken from: stackoverflow.com/questions/8924299/…

[Reid]

You 'da man! I could not figure this out as I'm not using an AVCaptureConnection...this did the trick. Thanks!

[Pochi]

@ReidBelton you cannot not be using AVCaptureConnection, its automatically created when you have an input and an output, you just gotta retrieve it and change the orientation.

[drewish]

Tried this and I get 'AVCaptureConnection' does not have a member named 'VideoOrientation'

[drewish]

It was a capitalization issue. Should have been videoOrientation so edited the answer.

[Duck]

just one thing I don't understand. When I use the camera with my ipad landscape I have to invert the videoOrientation. For example: if my ipad is UIDeviceOrientationLeft I have to do connection.videoOrientation = AVCaptureVideoOrientationLandscapeRight and the same for the other landscape, or the image will be upside down. Why the video orientation has to be inverted in terms of device orientation is one more confuse thing thanks to Apple.

[spfursich]

This worked for me. Like this code shows, it's important to add the output to the capture session before changing these properties. AVCaptureVideoOrientationLandscapeRight with videoMirrored = true gave me the same effect that you get from using the standard camera app with front facing camera.

[rptwsthi]

But where should i set this property in AVCaptureConnection? Inside the delegate method? something like: - (void) captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection { [connection setVideoOrientation:AVCaptureVideoOrientationPortrait];

[loretoparisi]

I have an CGBitmapContextCreateImage: invalid context 0x0 error

[gonglong]

@rptwsthi you can set just after you add output into session captureSession.addOutput(videoOutput) let connection = videoOutput.connectionWithMediaType(AVFoundation.AVMediaType‌​Video) connection.videoOrientation = .Portrait captureSession.startRunning()

[user3344977]

This should be marked as the correct answer. You can also use: connection.videoMirrored = true

[Roi Mulia]

Works! Thank you SIR!

[Andrzej Filipowicz]

Works. Just remember to set the orientation asynchronously on the main queue, because this method is called on a special serial queue.

[Sami Samhuri]

@SpaceDog If you look at the definitions of AVCaptureVideoOrientation and UIDeviceOrientation you'll see that the definitions of the landscape orientations are reversed in these enums. Why it's like this is beyond me, but it is documented. I agree it's confusing and assume there's some legacy reason they differ.

[Kartick Vaddadi]

Doesn't this re-compress the image, reducing its quality? Also the EXIF and embedded thumbnail (if any) will be wrong.

[RawMean]

@Gaddafi no, there is no compression or compression in this code

[mylogon]

Doesn't this just change the orientation of the local conn object, rather than the one belonging to self.videoCaptureOutput?

[ibisum]

The constants you are looking for are defined in kCGImagePropertyorientation... these come from the TIFF/EXIF spec: awaresystems.be/imaging/tiff/tifftags/orientation.html

[Koen.]

Can you explain why this would work? Because when I "just try this", it doesn't

[zyc]

Did you call the private startLiveVideo() func? What the problem did you get?

[KnowNothing]

@spfursich what you metioned is very very inconspicuous but important.

[alex bird]

@SpaceDog It is because they are planes facing in opposite directions out of opposite sides of the phone. The left of the rear camera sensor is described as if you were looking at it from its front, i.e. the lens, but if your point of reference is the screen on the front of the device then it is reversed and this is on the right. For the front camera these are more intuitive because the sensor is in the same plane as the screen. It is confusing, but probably best to keep naming consistent in case phones don't always look the way they do today.

[famfamfam]

swift version thanks