How to determine number of GPU cores being utilized for a process?

Question

As we know, if there are 4 cores in CPU, then maximum utilization would be 400%, and if the total utilization is 200% then it will be analysed as 2 cores has been used up among 4 cores.

Thus in case of GPUs, the maximum utilization is 100%, and if the total utilization is 80%, then how to analyse how many cores has been utilized?

For nvidia graphics, nvidia-smi can be used to see nvidia GPU utilization.

For intel graphics, intel_gpu_time/intel-gpu-top can be used to see iGPU utilization.

But both of them are not showing number of gpu cores being utilized. Is there any way to do that?

I am using Linux. Both Ubuntu 14.04 and Centos 7.2

Solution

On GPUs, 100% utilization means that all cores are basically executing instructions. It is running at peak performance when all of those instructions are fused-multiply-add operations: a = a + b * c, which most current GPUs can do using only a single instruction.

When you are writing a program that performs computation on the GPU (using CUDA or OpenCL), you distribute the work in a so-called grids of blocks of threads (CUDA terminology). It is up to the GPU to schedule all these threads (in 'warps' of 32 threads) to keep all GPU cores busy. I don't know how familiar you are with this subject, but this introduction might be an interesting read.

An example explains how this relates to utilization. Let's say they GPU is idle and then you start an application that launches a kernel only for a single thread block, but with sufficient computations within this block. The GPU will schedule the thread block onto on of the streaming multiprocessors (SMs = group of 128 cores) on the GPU. In case of for example the Nvidia GTX 1080, which has 20 SMs, that would result in an utilization of only (1/20 * 100%) = 5%.

Given this basic knowledge of GPU-computing, you can run it through the Nvidia Visual Profiler (for CUDA applications) or through CodeXL for (OpenCL applications) to see those thread configurations for any kernel that the application uses to reason about utilization of the GPU. But that's not all, this tool is invaluable to see exactly what kind of operations (and how efficient) the GPU is executing.

Comments

[debonair]

gpu-utlization is confusing term. for nvidia-smi, it gives percentage of total time your one or more multiprocessors are being used. It does not mean percentage of total cores in use.

[eloperdev88]

an updated intro to CUDA devblogs.nvidia.com/parallelforall/…