convert CMSampleBufferRef to UIImage
Solution 1
Your best bet will be to set the capture video data output's videoSettings
to a dictionary that specifies the pixel format you want, which you'll need to set to some variation on RGB that CGBitmapContext can handle.
The documentation has a list of all of the pixel formats that Core Video can process. Only a tiny subset of those are supported by CGBitmapContext. The format that the code you found on the internet is expecting is kCVPixelFormatType_32BGRA
, but that might have been written for Macs—on iOS devices, kCVPixelFormatType_32ARGB
(big-endian) might be faster. Try them both, on the device, and compare frame rates.
Solution 2
You can try this code.
-(UIImage *) screenshotOfVideoStream:(CMSampleBufferRef)samImageBuff
{
CVImageBufferRef imageBuffer =
CMSampleBufferGetImageBuffer(samImageBuff);
CIImage *ciImage = [CIImage imageWithCVPixelBuffer:imageBuffer];
CIContext *temporaryContext = [CIContext contextWithOptions:nil];
CGImageRef videoImage = [temporaryContext
createCGImage:ciImage
fromRect:CGRectMake(0, 0,
CVPixelBufferGetWidth(imageBuffer),
CVPixelBufferGetHeight(imageBuffer))];
UIImage *image = [[UIImage alloc] initWithCGImage:videoImage];
CGImageRelease(videoImage);
return image;
}
It work for me.
Benoît Freslon
French Game Designer, Game Developer Unity, iOS. Indie Game developer Freelance. Blog : Video Game Creation (French) Portfolio : http://www.benoitfreslon.com
Updated on June 04, 2022Comments
-
Benoît Freslon almost 2 years
I'm captuting video with AVCaptureSession. But I would like to convert the captured image to an UIImage.
I found some code on Internet:
- (UIImage *) imageFromSampleBuffer:(CMSampleBufferRef) sampleBuffer { NSLog(@"imageFromSampleBuffer: called"); // Get a CMSampleBuffer's Core Video image buffer for the media data CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); // Lock the base address of the pixel buffer CVPixelBufferLockBaseAddress(imageBuffer, 0); // Get the number of bytes per row for the pixel buffer void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer); // Get the number of bytes per row for the pixel buffer size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); // Get the pixel buffer width and height size_t width = CVPixelBufferGetWidth(imageBuffer); size_t height = CVPixelBufferGetHeight(imageBuffer); // Create a device-dependent RGB color space CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); // Create a bitmap graphics context with the sample buffer data CGContextRef context = CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst); // Create a Quartz image from the pixel data in the bitmap graphics context CGImageRef quartzImage = CGBitmapContextCreateImage(context); // Unlock the pixel buffer CVPixelBufferUnlockBaseAddress(imageBuffer,0); // Free up the context and color space CGContextRelease(context); CGColorSpaceRelease(colorSpace); // Create an image object from the Quartz image UIImage *image = [UIImage imageWithCGImage:quartzImage]; // Release the Quartz image CGImageRelease(quartzImage); return (image); }
But I got some errors:
Jan 17 17:39:25 iPhone-4-de-XXX ThinkOutsideTheBox[2363] <Error>: CGBitmapContextCreate: invalid data bytes/row: should be at least 2560 for 8 integer bits/component, 3 components, kCGImageAlphaPremultipliedFirst. Jan 17 17:39:25 iPhone-4-de-XXX ThinkOutsideTheBox[2363] <Error>: CGBitmapContextCreateImage: invalid context 0x0 2013-01-17 17:39:25.896 ThinkOutsideTheBox[2363:907] image <UIImage: 0x1d553f00> Jan 17 17:39:25 iPhone-4-de-XXX ThinkOutsideTheBox[2363] <Error>: CGContextDrawImage: invalid context 0x0 Jan 17 17:39:25 iPhone-4-de-XXX ThinkOutsideTheBox[2363] <Error>: CGBitmapContextGetData: invalid context 0x0
EDIT: I also use the UIImage to get the rgb color:
-(void) captureOutput:(AVCaptureOutput*)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection*)connection { UIImage* image = [self imageFromSampleBuffer:sampleBuffer]; unsigned char* pixels = [image rgbaPixels]; double totalLuminance = 0.0; for(int p=0;p<image.size.width*image.size.height*4;p+=4) { totalLuminance += pixels[p]*0.299 + pixels[p+1]*0.587 + pixels[p+2]*0.114; } totalLuminance /= (image.size.width*image.size.height); totalLuminance /= 255.0; NSLog(@"totalLuminance %f",totalLuminance); }