I already found its workaround inside Docker. But, in my case, I am running TensorFlow inside virtualenv so that I could run my Jupyter notebook to make a code and to run it.
But I also needed to run the Tensorboard. How can I run two web application inside virtualenv? I have never run two things at the same time. If I want to, I don't know how to run it in a background.
I found a simple solution. Just open another shell and run virtualenv / Jupyter notebook after it is already running a Tensorboard in other shell.
Related
I had a notebook running on Google Colab, which would use several shell commands such as:
!accelerate launch train_dreambooth.py
Since I am running this notebook in a Managed Notebook from GCP (Jupyter Lab), I only get the output of the command when the command has finished executing. I also seem to be getting a less verbose output.
Does somebody know what is going on, and what can I do? It's a long running command and I need to see live progress. Thanks!
My problem is the following: I want to run a Jupyter notebook on my remote desktop and access it via my laptop elsewhere. I have accomplished this, but I can't use my GPU for tensorflow because the GPU-supported version is only installed in my custom, non-base environment. Even though all of my installed jupyter kernels are available, it seems things don't work right unless I run 'jupyter notebook' from within the correct activated conda environment (says "no GPU" even though I select as the kernel the one where tensorflow-gpu is installed).
Is there a simple way of running jupyter notebook from within that environment by a batch script? I also need it to run the notebook on a secondary drive.
I could of course just start up the server while at home and then access it using the token, but that's a little clumsy.
I've found a solution. On windows, in %AppData%\Roaming\Microsoft\Windows\Start Menu\Programs\Anaconda3, there are shortcuts for various Anaconda-related programs, including Jupyter notebook for each environment.
The shortcut for Jupyter notebook for my given env is
`E:\Software\Anaconda3\python.exe E:\Software\Anaconda3\cwp.py E:\Software\Anaconda3\envs\tf E:\Software\Anaconda3\envs\tf\python.exe E:\Software\Anaconda3\envs\tf\Scripts\jupyter-notebook-script.py "%USERPROFILE%".
I modified this to end in '"E:" --no-browser' instead of the userprofile bit and made that into a script. Now when I SSH into the computer and run this script, the notebook is within the correct environment and I have access to my GPU, all on the correct drive, E.
I am using Jupyter Notebook to help debug some issues I'm having moving between JSON and pandas. The specific application isn't important.
The important part is that I needed to use pandas.json_normalize() which apparently first showed up in pandas version 1.0.3. I was confused when Jupyter said it doesn't exist. I did a version check and got:
In[]: pd.__version
Out[]: 0.25.2
This is not the version of python installed in either my base environment or the conda environment that Jupyter Notebook is running in or that the app is running in. Version checks in both environments in Anaconda Prompt (outside of Jupyter Notebook) confirm this.
What is going on here? Looking around I haven't seen a good answer, but it does appear that other people have had the same issue --- Jupyter defaulting to pandas 0.25.2 for some reason.
It seems that your Notebook is using a different kernel/environment than what you want.
run this in the notebook to see which environment you are using
! which python
or try
import sys
print(sys.executable)
which would show you which environment it's using, if you have env named venv then you will get something like.
/home/your_home_directory/anaconda3/envs/venv/bin/python
If you don't care about all of that and you just want to update the pandas that it's using then copy that path and do this.
! pip install --upgrade pandas
Note that this will also depend on which version of python you are using
For the Encoding categorical data data section, the first time I ran it, I got an error because I didn't have sklearn installed. After I installed it and re-run, I got In [ * ]. No matter how many times I run it, it's not executing. Basically, after an error is made, all lines can't be executed again.
One thing I found out is that this Jupyter Notebook is under tensorflow environment. This is what I did from the beginning:
1.Create tensorflow environment, then install tensorflow
2.Import the libraries, it didn't work because matplotlib not installed by default
3.Then install matplotlib
From my own troubleshooting, this is where the problem begins. So I tried to uninstall matplotlib, it's fine again as shown.
Now even there's an error, I can still re-run the line and the execution is fine because the number will be updated instead of [ * ].
Then the problem arises again because if I keep running it, it will turn into [*] again after maybe 3rd or 4th times. The thing is if I do the same thing with Jupyter notebook under root environment(no tensorflow installed, matplotlib is installed), I can keep re-running it for as many times and there's no problem(the number will be changed). So what's wrong with it?
It seems the problem came from tensorflow python 3.6. I tried many different methods to troubleshoot. Lastly I removed tensorflow environment, and reinstalled as python 3.5, Jupyter notebook is working fine now. It won't stop executing. Does 3.6 have to be fixed?
I can't be able to run the tensorflow code with GPU when I ran it from a jupyter notebook. Same code runs no problem, if I ran in a python script.
I followed the main installation link:
https://www.tensorflow.org/install/install_windows
Also tried:
http://bailiwick.io/2017/11/05/tensorflow-gpu-windows-and-jupyter/
No problems outside notebook when I ran in a python script file.
Most likely the problem is similar to this:
Tensorflow not running on GPU in jupyter notebook
More specifically my test:
I can see both devices CPU and GPU via python a script
I can see only CPU via notebook
Thanks a lot for any help in advance!
Very late, but short answer:
Here is a Tutorial on how to set up a GPU-based Jupyterlab instance with Docker (which makes the installation faster).
I hope this helps!
I removed all existing environments and created a new one, which resolved the issue.
(Also, I had to apply the following to get around an issue caused by removed environments:
https://github.com/jupyter/notebook/issues/2301
)