How to verify the usage of the GPU? - cntk

Question

How can I verify that CNTK is using the GPU? I have installed the CNTK-2-0-beta7-0-Windows-64bit-GPU-1bit-SGD binaries on my machine. But, when I try to run this from Python:
from cntk.device import set_default_device, gpu
set_default_device(gpu(0))
I get:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-8-eca77b3090be> in <module>()
1 from cntk.device import set_default_device, gpu
----> 2 set_default_device(gpu(0))
C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py34\lib\site-packages\cntk\device.py in gpu(device_id)
74 :class:`~cntk.device.DeviceDescriptor`: GPU device descriptor
75 '''
---> 76 return cntk_py.DeviceDescriptor.gpu_device(device_id)
77
78 def use_default_device():
ValueError: Specified GPU device id (0) is invalid.
Adding some more information today:
This is the result from running NVidia_smi.exe
C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi.exe
Thu Jan 12 20:38:30 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 369.61 Driver Version: 369.61 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GPU WDDM | 0000:01:00.0 Off | N/A |
| N/A 51C P0 2W / N/A | 864MiB / 1024MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
After restarting the kernel in a Jupyter Notebook, I get:
import cntk as C
if C.device.default().type() == 0:
print('running on CPU')
else:
print('running on GPU')
running on CPU
However today I was able to run:
from cntk.device import set_default_device, gpu
set_default_device(gpu(0))
import cntk as C
if C.device.default().type() == 0:
print('running on CPU')
else:
print('running on GPU')
running on GPU
Should the GPU be the default on a GPU machine, or do you need to explicitly set it?

Answer 1

This sounds like an intermittent failure. This can happen on some laptops such as the Surface Book which have two GPUs, one from NVIDIA and an integrated one, and the laptop has shutdown the NVIDIA GPU to conserve energy, e.g. when it is running on battery.
Regarding default behavior, by default CNTK will choose the best available device and if it is a GPU it will lock it so no other process can use it. If you explicitly use set_default_device(gpu(0)) then the GPU won't get locked and other processes can use it.