Understanding necessity of Android VSYNC signals

Question

I'm trying to get a better understanding of the Android display subsystem, but one item that's still confusing to me is how VSYNC signals are handled, and why so many exist in the first place.

Android is designed to use VSYNC at its core, but there are multiple VSYNC signals that it employs. Via https://source.android.com/devices/graphics/implement.html in the "VSYNC Offset" section, there is a flow diagram which diagrams three VSYNC signals: HW_VSYNC_0, VSYNC, and SF-VSYNC. I understand that HW_VSYNC is used to update the timing in DispSync, and that VSYNC and SF-VSYNC are used by the apps and surfaceflinger, but why are these individual signals necessary at all? Furthermore, how do the offsets impact these signals? Is there a timing diagram available anywhere which better explains this?

Thanks for any help you can offer.

Solution

To understand this stuff, it's best to start with the System-Level Graphics Architecture document, taking particular note of The Need for Triple-Buffering section and the associated diagram (which ideally would be an animated GIF). The sentence that begins, "If the app starts rendering halfway between VSYNC signals" is talking specifically about DispSync. Once you've read that, hopefully the DispSync section of the device graphics doc makes more sense.

Most devices don't have DispSync offsets configured, so there is really only one VSYNC signal. In what follows I'm assuming DispSync is enabled.

The hardware only provides one VSYNC signal, corresponding to the primary display refresh. The others are generated in software by the SurfaceFlinger DispSync code, firing at fixed offsets from the actual VSYNC. Some clever software is used to keep the timings from slipping out of phase.

The signals are used to trigger SurfaceFlinger composition and app rendering. If you follow the section in the architecture document, you can see that this establishes two frames of latency between when the app renders its content, and when the content appears on the screen. Think of it like this: given three occurrences of VSYNC, the app draws at V0, the system does composition at V1, and the composed frame is sent to the display at V2.

If you're trying to track touch input, perhaps moving a map around under the user's finger, any latency will be seen by the user as sluggish touch response. The goal is to minimize the latency to improve the user experience. Suppose we delayed the events slightly, so the app draws at V0.5, we composite at V1.2, and then swap to the display at V2. By offsetting the app and SF activity we reduce the total latency from 2 frames to 1.5, as shown below.

That's what DispSync is for. In the feedback diagram on the page you linked, HW_VSYNC_0 is the hardware refresh for the physical display, VSYNC causes the app to render, and SF_VSYNC causes SurfaceFlinger to perform composition. Referring to them as "VSYNC" is a bit of a misnomer, but on an LCD panel referring to anything as "VSYNC" is probably a misnomer.

The "retire fence timestamps" noted in the feedback loop diagram refers to a clever optimization. Since we're not doing any work on the actual hardware VSYNC, we can be slightly more efficient if we turn the refresh signal off. The DispSync code will instead use the timestamps from retire fences (which is a whole other discussion) to see if it is falling out of sync, and will temporarily re-enable the hardware signal until it's back on track.

Edit: you can see how the values are configured in the Nexus 5 boardconfig. Note the settings for VSYNC_EVENT_PHASE_OFFSET_NS and SF_VSYNC_EVENT_PHASE_OFFSET_NS.

Comments

[Shookit]

fadden, thanks for your help once again. Just two questions: 1) Does this timing chart I made make sense? Original (2 frames of latency): i.imgur.com/4E2cD9l.png DispSync: i.imgur.com/p7LdmXF.png 2) Is it possible to draw and compose in a single frame if you reduce the offsets enough (at risk of reducing render time)?

[fadden]

The diagrams look good. It's possible to draw and compose in one VSYNC interval (typically 16.7ms), but in practice it's hard to hit those deadlines regularly, so attempting to do that often yields janky results. Note that SurfaceFlinger composition is very fast if everything fits into overlays, but if it falls back to GLES composition then it will take longer, especially if SF and the app end up competing for GPU resources. The most accurate visualization of DispSync is systrace output from a device where it's enabled, e.g. the Nexus 5.

[Shookit]

Is there a hardware reason that a board wouldn't support DispSync? Or is it just more testing that the manufacturer has to do to implement? I'm testing on the N4, and I noticed that neither of the VSYNC_EVENT_PHASE_OFFSET_NS flags are defined for the N4 (nor is RUNNING_WITHOUT_SYNC_FRAMEWORK). These just default to 0 in that case, disabling DispSync?

[fadden]

We had hoped the OEMs would be able to provide out-of-phase VSYNC signals, but that didn't work out, so a software solution was used instead. Very few devices had DispSync enabled last I checked. RUNNING_WITHOUT_SYNC_FRAMEWORK is separate but related -- it refers to the fence system described in the "Explicit synchronization" section of the graphics doc linked from your question. Nvidia tegra3 boards set this, e.g. android.googlesource.com/device/asus/grouper/+/lollipop-rele‌​ase/… . They were using the older "implicit sync" approach, though that might be fixed now.

[Shookit]

Got it; I really appreciate your time. I'll organize this all into your answer when I have a few spare cycles. One quick verification: if DispSync isn't being used, then does HW_VSYNC_0 immediately then trigger VSYNC-app and VSYNC-sf, or do those still get triggered by a timer in software? Or what ultimately causes those signals to fire?

[fadden]

If DispSync is not enabled, the hardware VSYNC triggers both SF and the app. It should be noted that not all apps are VSYNC-driven. Framework UI runs off of Choreographer callbacks, so animations happen on VSYNC, but many apps (especially games) just throw frames out to a SurfaceView as fast as they can. They're still VSYNC-limited, because they'll block waiting for the BufferQueue to empty, but they're not explicitly sleeping on VSYNC. (Honestly, the graphics arch doc could be 10x the size and still not cover all the ins and outs.)