I'm using TPU through Google Colab and GCP, and want to dump XLA IR. But I have seen the xla doc in github xla index, and it only shows the way while the backend is CPU or GPU.
I have tried using XLA_FLAGS="--xla_dump_hlo_as_text --xla_dump_to=/content/iir/" TF_XLA_FLAGS=--tf_xla_cpu_global_jit to run a CPU-targeted program and get dumped hlo file. I have also tried capture_tpu_file and can only get ir for each operator in 'op_profile' page. So is there a way to dump XLA IR for the whole program when the backend is TPU?
Thank you!
Jay
Unfortunately there isn't a way to dump/access the XLA IR on Cloud TPUs at the moment, since the XLA_FLAGS need to be set on the TPU server.
Related
I want to perform distributed training on Amazon SageMaker. The code is written with TensorFlow and similar to the following code where I think CPU instance should be enough:
https://github.com/horovod/horovod/blob/master/examples/tensorflow_word2vec.py
Can Horovod with TensorFlow work on non-GPU instances in Amazon SageMaker?
Yeah you should be able to use both CPU's and GPU's with Horovod on Amazon SageMaker. Please follow the below example for the same
https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-python-sdk/tensorflow_script_mode_horovod/tensorflow_script_mode_horovod.ipynb
So I am using Google Colab because I have some functions I need to execute that take far too long on my cpu. I have set the runtime to the GPU accelrator, however when I run the cell, I still get this message: 'Warning: You are connected to a GPU runtime, but not utilizing the GPU'.
I understand that this means the code I am running is just using my cpu. However using my cpu, the function takes hours to execute. This is why I want to utilise Colab's GPU, however, even when I change runtime, it still uses my cpu... How do I specifically force Colab to utilise the GPU for executing a certain cell/function in Colab???
Edit: I have just found out apparently Colab uses GPU only when the package being used is a package specifically made for GPU usage. Is there some sort of external package I can use that forces a function to find a GPU to use before executing the function?
Edit: (The package I am using for the long calculation is Network X if that makes any difference)
Check out cuGraph, which lets you do the same graph calculations on the gpu as networkx. A medium post on compatibility between cuGraph and networkx graphs.
You only need to do a couple of things to get cuGraph working on Google Colab. As the Google Colab demo from this medium post suggests:
Use pynvml to confirm Colab allocated you a Tesla T4 GPU.
Install most recent Miniconda release compatible with Google Colab's Python install (3.6.7)
Install RAPIDS libraries
Copy RAPIDS .so files into current working directory, a workaround for conda/colab interactions
Update env variables so Python can find and use RAPIDS artifacts
!wget -nc https://github.com/rapidsai/notebooks-
extended/raw/master/utils/rapids-colab.sh
!bash rapids-colab.sh
import sys, os
sys.path.append('/usr/local/lib/python3.6/site-packages/')
os.environ['NUMBAPRO_NVVM'] = '/usr/local/cuda/nvvm/lib64/libnvvm.so'
os.environ['NUMBAPRO_LIBDEVICE'] = '/usr/local/cuda/nvvm/libdevice/'
And then you can do the same calculations on the gpu:
pagerank = cugraph.pagerank(G)
instead of
pagerank = nx.pagerank(G)
I've coded a Neural Network from scratch in Python and I am using Google Colaboratory to train it. However, if I enable GPU or TPU acceleration, the training is not faster.
When you search for examples online, all of them use Tensorflow and other libraries, and their training times are shorter with GPU than without it.
Am I doing it correctly or am I missing something and the GPU is not being used?
Just enabling GPU or TPU won't help your problem, you need to explicitly code them to run on GPU if you are not using any frameworks or libraries.
I have a USB TPU and would like to use it as LOCAL RUNTIME in Google Colab.
I was not able to find any resources on this topic.
You can use a local Runtime (local Jupyter) and it is explained here :
https://research.google.com/colaboratory/local-runtimes.html
Do I need to install all the TPU libraries in my local Jupyter and then connect to local Jupyter as local runtime to start using my USB TPU in Colab?
I'm not familiar with Google Colab, but looks like it allows you to expose your model on your hardware. You'll then need to locate your model in order to run inference with it. There are multiple ways that you can choose to run it which are all listed here:
https://coral.withgoogle.com/docs/edgetpu/inference/
I have previously asked if it is possible to run tensor flow with gpu support on a cpu. I was told that it is possible and the basic code to switch which device I want to use but not how to get the initial code working on a computer that doesn't have a gpu at all. For example I would like to train on a computer that has a NVidia gpu but program on a laptop that only has a cpu. How would I go about doing this? I have tried just writing the code as normal but it crashes before I can even switch which device I want to use. I am using Python on Linux.
This thread might be helpful: Tensorflow: ImportError: libcusolver.so.8.0: cannot open shared object file: No such file or directory
I've tried to import tensorflow with tensorflow-gpu loaded in the uni's HPC login node, which does not have GPUs. It works well. I don't have Nvidia GPU in my laptop, so I never go through the installation process. But I think the cause is it cannot find relevant libraries of CUDA, cuDNN.
But, why don't you just use cpu version? As #Finbarr Timbers mentioned, you still can run a model in a computer with GPU.
What errors are you getting? It is very possible to train on a GPU but develop on a CPU- many people do it, including myself. In fact, Tensorflow will automatically put your code on a GPU if possible.
If you add the following code to your model, you can see which devices are being used:
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
This should change when you run your model on a computer with a GPU.