How to run Tensorflow on CPU
Solution 1
You can apply device_count
parameter per tf.Session
:
config = tf.ConfigProto(
device_count = {'GPU': 0}
)
sess = tf.Session(config=config)
See also protobuf config file:
tensorflow/core/framework/config.proto
Solution 2
You can also set the environment variable to
CUDA_VISIBLE_DEVICES=""
without having to modify the source code.
Solution 3
If the above answers don't work, try:
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
Solution 4
For me, only setting CUDA_VISIBLE_DEVICES
to precisely -1
works:
Works:
import os
import tensorflow as tf
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
if tf.test.gpu_device_name():
print('GPU found')
else:
print("No GPU found")
# No GPU found
Does not work:
import os
import tensorflow as tf
os.environ['CUDA_VISIBLE_DEVICES'] = ''
if tf.test.gpu_device_name():
print('GPU found')
else:
print("No GPU found")
# GPU found
Solution 5
The environment variable solution doesn't work for me running tensorflow 2.3.1. I assume by the comments in the github thread that the below solution works for versions >=2.1.0.
From tensorflow github:
import tensorflow as tf
# Hide GPU from visible devices
tf.config.set_visible_devices([], 'GPU')
Make sure to do this right after the import with fresh tensorflow instance (if you're running jupyter notebook, restart the kernel).
And to check that you're indeed running on the CPU:
# To find out which devices your operations and tensors are assigned to
tf.debugging.set_log_device_placement(True)
# Create some tensors and perform an operation
a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
c = tf.matmul(a, b)
print(c)
Expected output:
2.3.1
Executing op MatMul in device /job:localhost/replica:0/task:0/device:CPU:0
tf.Tensor(
[[22. 28.]
[49. 64.]], shape=(2, 2), dtype=float32)
Related videos on Youtube
Alexander R Johansen
Updated on August 06, 2022Comments
-
Alexander R Johansen over 1 year
I have installed the GPU version of tensorflow on an Ubuntu 14.04.
I am on a GPU server where tensorflow can access the available GPUs.
I want to run tensorflow on the CPUs.
Normally I can use
env CUDA_VISIBLE_DEVICES=0
to run on GPU no. 0.How can I pick between the CPUs instead?
I am not intersted in rewritting my code with
with tf.device("/cpu:0"):
-
Crashalot over 7 yearsSomeone said running neural nets on CPUs after the training phase is as efficient as running them on GPUs -- i.e., only the training phrase really needs the GPU. Do you know if this is true? Thanks!
-
Crashalot over 7 yearsSomeone said running neural nets on CPUs after the training phase is as performant as running them on GPUs -- i.e., only the training phrase really needs the GPU. Do you know if this is true? Thanks!
-
Thomas over 7 years@Crashalot: This is not true. Look for various benchmarks for interference, CPUs are an order of magnitude slower there too.
-
Crashalot over 7 years@Thomas thanks. suggestions on which benchmarks to consider? probably also varies on workload and nature of the neural nets, right? apparently the google translate app runs some neural nets directly on smartphones, presumably on the cpu and not gpu?
-
P-Gn almost 7 yearsThat doesn't work for me (tf1.1). The solution of fabrizioM does.
-
Girishkumar almost 7 years@fabrizioM, a toy example will be more useful.
-
Nandeesh almost 7 years@Thomas, what I have observed is after the training is done and the model is frozen, doing predictions using the frozen model on CPU (around 3 ms) is slightly faster than on GPU (around 7 ms).
-
Nandeesh almost 7 yearsIsn't it better to use
CUDA_VISIBLE_DEVICES
environment variable instead of changing the config in the code? -
Ivan Aksamentov - Drop almost 7 years@Nandeesh I guess it depends on your needs. So far there are at least 53 people who feel more into environment variables and 35 who prefer to set number of devices in code. The advantage of first is simplicity and of another is more explicit control over (multiple) sessions from within the python program itself (that zero is not necessary to be hardcoded, it can be a variable).
-
lahwran over 6 yearsHow do you hide it from tensorflow without hiding it from cuda? ie, I want something besides tensorflow in the same script to have access to the gpu
-
Guilherme de Lazari over 6 yearsThis did not work for me. :/ set the environment variable but tensorflow still uses the GPU, I'm using conda virtual env, does this make a diference?
-
Davidmh over 6 years@Crashalot it depends on the nature of the network. For example, RNNs can be faster on CPUs for small batch sizes due to its sequential nature. CNNs will still benefit from a GPU in inference mode, but since you only need to run them once per example, a CPU may be fast enough for many practical purposes.
-
Dammi over 5 yearsI prefer this one as sometimes you maybe want to run two networks simultaneously, a small network on CPU, while the bigger one is on GPU.
-
CMCDragonkai over 5 yearsHow does this configuration interact with
$CUDA_VISIBLE_DEVICES
? -
Bs He over 5 yearsHowever, if say we have no GPU devices, then does this solution still works? Or we have to use
{device_count = {'GPU':0}}
-
Admin about 5 yearsKeep finding GPU. This may have worked on earlier versions of TF but not anymore.
-
Admin about 5 yearsJust for the record, the first option doesn't seem to work anymore.
-
Zaccharie Ramzi over 4 yearsDo you know how to adapt this to tensorflow 2.0, as there is no more session or configproto?
-
kentavv about 4 years
CUDA_VISIBLE_DEVICES="" ./learn_tf.py
works well on a Fedora 31, TensorFlow 2.1.0, and CUDA 10.2 system. CUDA_VISIBLE_DEVICES can also be a comma separated list of device ids. Effects entire program and so does not help if you want to have CUDA code and CPU-only TensorFlow in one program. -
Nicolas M. almost 4 yearsWorks also for tf 2.X when using
tf.keras.Sequential
models. -
mboss almost 4 years@Crashalot: for some datasets run with shallow models, the CPU is much faster even in the training phase because we can avoid the transfer time from memory to GPU on each batch. Some data feeding/preloading techniques can reduce this gap, but it sometimes still persists. 95% of the time the GPU is faster, but sometimes it's worth checking performance on the CPU.
-
Crashalot almost 4 years@mboss thanks for the answer! is it possible to describe the cases where the CPU is faster or comparable in speed?
-
m3h0w over 3 yearsThis indeed doesn't work anymore: check out my lower answer for something that works from 2.3.1 up.
-
JamesAng about 3 yearshi, does not work for me...I'm using tensorflow-gpu 2.4.1
-
Nermin about 3 yearsIs there a way to do this without tensorflow invoking the error message "CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected" ?
-
Vinson Ciawandy almost 3 yearsWorking with tensorflow and pytorch in one script, this approach help me to disable cuda on tensorflow but still make the pytorch use cuda. I believe this answer deserved more votes.
-
jona almost 3 yearsThis works for tensorflow 2.5. But, I think my GPU for tensorflow 2.5 is no longer usable in the current environment after running the command. **(I tried the recommended way above and it doesn't work)
-
Bruno Feroleto almost 3 yearsA potential advantage of this solution is that it doesn't rely on a variable that explicitly mentions CUDA and which might as such be reserved to specific devices. For example, it works on my Apple Silicon Mac.
-
ker2x over 2 yearsBest solution, thx (because I'm on a Silicon too :D)
-
agata about 2 yearsHow to use this tensorflow-cpu with tf.io.decode_image?
-
madhur about 2 yearsworks for me like a charm. In Jupyter notebook, just follow these steps (based on above comment) - Restart the Kernel --> put this line just beneath the tensorflow: import tf.config.set_visible_devices([], 'GPU') --> run your script
-
Olympia almost 2 yearsthis finally worked for me on tensorflow 2.7.0, thanks!