AVCaptureVideoPreviewLayer (camera preview) freezes/stuck after moving to background and back

Question

Cant figure this one out. Everything works fine when the app is active, and than sometimes when i move the app to the background(pressing the home button) and than going back, the previewlayer freezes/stuck. Im using viewWillAppear and viewDidAppear for the set up. This is how i set everything up :

  var backCamera = AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo)
  var global_device : AVCaptureDevice!
  var captureSession: AVCaptureSession?

override func viewWillAppear(animated: Bool) {
super.viewWillAppear(animated)

captureSession = AVCaptureSession()
        captureSession!.sessionPreset = AVCaptureSessionPresetPhoto
        CorrectPosition = AVCaptureDevicePosition.Back
        for device in backCamera {
            if device.position == AVCaptureDevicePosition.Back {
                global_device = device as! AVCaptureDevice
                CorrectPosition = AVCaptureDevicePosition.Back
                break
            }
        }


        configureCamera()
        var error: NSError?
        var input = AVCaptureDeviceInput(device: global_device, error: &error)


        if error == nil && captureSession!.canAddInput(input) {
            captureSession!.addInput(input)

            stillImageOutput = AVCaptureStillImageOutput()
            stillImageOutput!.outputSettings = [AVVideoCodecKey: AVVideoCodecJPEG]
            if captureSession!.canAddOutput(stillImageOutput) {
                captureSession!.addOutput(stillImageOutput)

                previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
                var bounds:CGRect = camera_Preview.layer.bounds
                previewLayer?.videoGravity = AVLayerVideoGravityResizeAspectFill
                previewLayer?.bounds = bounds
                previewLayer?.position = CGPointMake(CGRectGetMidX(bounds), CGRectGetMidY(bounds))
                camera_Preview.layer.addSublayer(previewLayer)
                self.view.bringSubviewToFront(camera_Preview)
                self.view.bringSubviewToFront(nan_view)

                captureSession!.startRunning()


            }
        }

ViewDidAppear :

  var previewLayer: AVCaptureVideoPreviewLayer?


override func viewDidAppear(animated: Bool) {
        super.viewDidAppear(animated)
        previewLayer!.frame = camera_Preview.bounds
    }

Solution 1

For future readers: this is the correct process to setting up camera inside your app.

First of all, thanks for the folks above that took their time and tried help me. They both direct me into the correct direction. Although Bill was mistaken about the viewDidLoad theory, he did gave the solution Apple Project.

This setting up camera - the correct way - is little more complicated than I thought, follow the documentation gave me excellent results. So for the Objective-C coders:

Objective C cam project

Swift cam project

About Andrea answer, he did said some great pointers you should consider when you are creating this kind of app. Check them out - they are highly relevant (most of the things he said inside the Apple project as well).

Solution 2

Roi,

I think your problem is that you are doing all the session setup and such in the viewWillAppear. Lets say that the captureSession, and the previewLayer were both alloc'd and working correctly. Now, you put app into the background and bring back.

You will immediately try to create a new captureSession, and a new previewLayer. I suspect that the old ones and the new ones are getting tangled up.

In the Apple AVCam example they do the setup in the viewDidLoad. That way it is only done once.

You should move all your setup stuff to a method and then call the method from the viewDidLoad.

bill

Solution 3

Just a quick update In 2017 if someone suffering overthinking such that thing,

Do the same thing but change your

stillImageOutput!.outputSettings = [AVVideoCodecKey: AVVideoCodecJPEG]

replace it with

stillImageOutput!.outputSettings = [((kCVPixelBufferPixelFormatTypeKey as NSString) as String):NSNumber(value:kCVPixelFormatType_32BGRA)]

it will solve the issue. If not, then come back here and write down :))

Solution 4

Swift 4 Solution

You have to delete the camera input when the user enters the background, then restore it when they return. Take a look at this code:

override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)
         createCameraPreview() //Call the setup of the camera here so that if the user enters the view controller from another view controller, the camera is established. 
         notificationCenter() //Call the notification center function to determine when the user enters and leaves the background. 
    }



 func notificationCenter() {
            NotificationCenter.default.addObserver(self, selector: #selector(willResignActive), name: .UIApplicationWillResignActive , object: nil)
                NotificationCenter.default.addObserver(self, selector: #selector(openedAgain), name: .UIApplicationDidBecomeActive, object: nil)
        }

 @objc func openedAgain() {
     createCameraPreview() // This is your function that contains the setup for your camera. 
    }

    @objc func willResignActive() {
        print("Entered background")
        let inputs = captureSession!.inputs
        for oldInput:AVCaptureInput in inputs {
            captureSession?.removeInput(oldInput)
        }
    }

Swift 4.2:

   override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)
            setupCamera()
            //createCameraPreview() //Call the setup of the camera here so that if the user enters the view controller from another view controller, the camera is established.
            notificationCenter() //Call the notification center function to determine when the user enters and leaves the background.
    }



    func notificationCenter() {
        NotificationCenter.default.addObserver(self, selector: #selector(willResignActive), name: UIApplication.willResignActiveNotification , object: nil)
        NotificationCenter.default.addObserver(self, selector: #selector(openedAgain), name: UIApplication.didBecomeActiveNotification, object: nil)
    }

    @objc func openedAgain() {
        setupCamera() //This is your function that contains the setup for your camera.
    }

    @objc func willResignActive() {
        print("Entered background")
        let inputs = captureSession.inputs
        for oldInput:AVCaptureInput in inputs {
            captureSession.removeInput(oldInput)
        }
    }

Solution 5

I think that there are different things that could cause the problem:

1. You should wrap all the configuration that you are doing in a -beginConfiguration and -commitConfiguration block of code. Each time you setup something in the session it will takes time to do it. Wrapping your configuration code between those method will guarantee that all the changes are committed in one shot, reducing the overall session creation time
2. Pausing the session is a good when you go to the background. Register your class as an observer to UIApplicationDidEnterBackground and UIApplicationWillEnterForeground to pause and start again the session.
3. You create the session in -viewWillAppear each time this method is called you create a session but, is not really clear from your code if you get rid of it. You should separate and balance the session creation and destroy. Provide a -setupSession and a -tearDownSession methods. Make sure that the setup is called only if there is no active session and make sure that when you don't need the session anymore you get rid of it by calling the teardownSession. In SWIFT you na use a @lazy variable and destroy the session in deinit() or -viewWillDisappear.
4. It could be very useful to create or use a SYNC queue creating and destroying a session is an intensive task and you usually prefer to put it on a background queue, furthermore helps to synchronize all the methods that involve the session. Creating your own sync queue will guarantee for example the sync between a setup session call and tear down call, one is called only when the other one finish.

I undestand that is huge refactor, but I'm this way I'm pretty sure that you will experience less problems in the future.

View more solutions

Comments

[Roi Mulia]

Bill, thanks for your respond . Does going out and in of the app retrigger the WillAppear etc methods?

[Roi Mulia]

I actually just checked it , put a breakpoint in viewwillAppear , Does Not seems to trigger again when i re-open the app

[Bill Johnson]

Right you are. ViewWillAppear does not appear to be fired on minimize and return. However, if you seque to another view controller it will be. Could that be part of the trouble?

[Roi Mulia]

You mean if im going out after i moved from the original view?

[Bill Johnson]

If you are in a navigation controller and push to a second screen, and hit the back button, you will cause the trouble I was describing. Your capture session and preview layer will get re-created. You might want to move capture session creation to the viewDidLoad.

[Roi Mulia]

Im not doing that. Im staying at the same view.. Any suggestions?

[Bill Johnson]

The only thing that I can think of is to start logging. Put a log right before the captureSession = AVCaptureSession(). That will tell for sure if you are calling twice. Also, can you share the code in your configureCamera()?

[Roi Mulia]

I will edit it soon. But the thing is that im going in and out if the app(home button), nothing else. And if i press the take pictures button, it saves the correct output(even if it stucked)

[Bill Johnson]

If you are going into the background, you should stop your session to save battery. Its probably already stopping anyway and that could be part of the problem.

[Roi Mulia]

Im make some tests and feedback you

[Roi Mulia]

Bill , still i cant find a cause to this problem.. any new suggestions? in the configureCamera there is nothing but checking if the flash is available or not . Thanks , Roi

[Roi Mulia]

Hey! I used Apple based document to fix my code. I did added most of the things you said, is it necessary doing the tearDown part? Because i didn't and it seems working now , thanks!

[kiran]

The link for "Objective C cam project" is dead. The answer is not useful for anyone now. Can you please update the link or somehow show the solution in the answer?

[Lepidopteron]

Link is updated. For future: In case the link is broken again, the Project is called "AVCam-iOS" - google finds this in an instant or have a look on developer.apple.com for it.

[Anurag Sharma]

@BillJohnson that link is no more!

[Duck]

Things could be so easy if Apple had decent documentation. Throwing up sample code or poorly written documentation as they always do is not doing any good for developers. Imagine a world without stackoverflow and not a single line of code could be created.

[Gizmodo]

Do you know the reason for this?