Now I can't mount colab because of too many files on 'My Drive' when use the command:
from google.colab import drive
drive.mount('/content/gdrive')
So shall I delete all files on 'My Drive' by command or other sets?
Related
I want to run M-file (Octave) in Colab. first, I upload all file in Google Drive with the following address:
/content/drive/MyDrive/Colab/Corroded/hybrid/FAEICA_ANN/
Then I used the following code to run my file:
!apt install octave
from google.colab import drive
drive.mount('/content/drive')
import sys
import os
py_file_location = "/content/drive/MyDrive/Colab/Corroded/hybrid/FAEICA_ANN"
sys.path.append(os.path.abspath(py_file_location))
!octave -W MainANN_FAEICA.m
But I got:
error: no such file, '/content/MainANN_FAEICA.m'
error: source: error sourcing file '/content/MainANN_FAEICA.m'
Although I added the path, Colab cannot recognize the file.
You've added the directory to Python's system search path, not to Octave's search path.
The simplest way to run this file is to change the working directory to where the file is, then run the file:
os.chdir(py_file_location)
!octave -W MainANN_FAEICA.m
I have a file that's stored on the Bioinformatics project folder of my google drive. I want to load and unpack this file gbm_tcga_pub2013.tar.gz.
from google.colab import drive
drive.mount('/content/gdrive')
path='/content/gdrive/MyDrive/Bioinformatics project'
!tar -xvzf path/gbm_tcga_pub2013.tar.gz
Traceback:
tar: path/gbm_tcga_pub2013.tar.gz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar is a bash command. path is a Python variable. There might be a way of mixing them, but this is not it. In this case, tar command is looking for path/gbm_tcga_pub2013.tar.gz directory, which obviously does not exist. The following should work:
!tar -xvzf /content/gdrive/MyDrive/Bioinformatics project/gbm_tcga_pub2013.tar.gz
I have uploaded a folder of google drive with all of my files .It's structure is like -
MyFolder
-Images
-train.py
-classify.py
-Facenet.py
I mounter the folder by following instruction on How to Upload Many Files to Google Colab?
In my computer i simply go to MyFolder , i open the terminal and i run python train.py .How to do same thing in google colab ? I have uploaded MyFolder on google drive.
Edit : After mounting i changed my directory to MyFolder (credits : Google colab changing directory) .I have runned train.py , it's still running .I hoper everything works fine . So now all i want to know that all changes caused after running the script will be stored in MyFolder of drive itself ?
My problem is solved .Two links mentioned are enough.
First upload MyFolder on google drive .Then go to google colab (using same gmail account).Copy and paste the following script and run on google colab .(see How to Upload Many Files to Google Colab?)
# Install a Drive FUSE wrapper.
# https://github.com/astrada/google-drive-ocamlfuse
!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse
# Generate auth tokens for Colab
from google.colab import auth
auth.authenticate_user()
# Generate creds for the Drive FUSE library.
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}
# Create a directory and mount Google Drive using that directory.
!mkdir -p My Drive
!google-drive-ocamlfuse My Drive
!ls My Drive/
# Create a file in Drive.
!echo "This newly created file will appear in your Drive file list." > My Drive/created.txt
After than you can use command %cd Drive/ to move to the drive and then command %cd MyFolder to move inside Myfolder .see Google colab changing directory .You can run terminal commands of Linux using % or ! at the beginning of you command statement.to train the data run the corresponding command .
!python train.py
I have a directory named ML in my drive: which has some of my colab-notebooks and some csv files which I want to load in my colab-notebook.
But , when I use !pwd to find out the current folder, its output is as follows.
!pwd
/content
When I use !ls .. The directory structure is as follows:
bin/
boot/
colabtools/
content/
datalab/
dev/
etc/
gpu-tensorflow-1.9.0-cp27-cp27mu-linux_x86_64.whl
gpu-tensorflow-1.9.0-cp36-cp36m-linux_x86_64.whl
home/
lib/
lib64/
media/
mnt/
opt/
proc/
root/
run/
sbin/
srv/
sys/
tensorflow-1.9.0-cp27-cp27mu-linux_x86_64.whl
tensorflow-1.9.0-cp36-cp36m-linux_x86_64.whl
tf_deps/
tmp/
tools/
usr/
var/
I am unable to !cd to my directory which is drive/ML , since I don't know the path, and the tree structure and thus, load my csv files.
use %cd instead of !cd, like this:
%cd drive/ML
In such cases we need to mount our google-drive and it will be shown as
below:
os.listdir()
/content
And,
os.listdir('/content')
datalab drive
If the google-drive isn't mounted the /content directory will only contain the datalab folder.
To mount your Google Drive:
from google.colab import drive
drive.mount('/content/drive')
For further information refer here
Afterwards, to change directory use:
import os
os.chdir("drive/My Drive/ML")
I don't understand how colab works with directories, I created a notebook, and colab put it in /Google Drive/Colab Notebooks.
Now I need to import a file (data.py) where I have a bunch of functions I need. Intuition tells me to put the file in that same directory and import it with:
import data
but apparently that's not the way...
I also tried adding the directory to the set of paths but I am specifying the directory incorrectly..
Can anyone help with this?
Thanks in advance!
Colab notebooks are stored on Google Drive. But it is run on another virtual machine. So, you need to copy your data.py there too. Do this to upload data.py through Colab.
from google.colab import files
files.upload()
# choose the file on your computer to upload it then
import data
Now google is officially providing support for accessing and working with Gdrive at ease.
You can use the below code to mount your drive to Colab:
from google.colab import drive
drive.mount('/gdrive')
%cd /gdrive/My\ Drive/{location you want to move}
To easily upload a local file you can use the new Google Colab feature:
click on right arrow on the left of your screen (below the Google
Colab logo)
select Files tab
click Upload button
It will open a popup to choose file to upload from your local filesystem.
To upload Local files from system to collab storage/directory.
from google.colab import files
def getLocalFiles():
_files = files.upload()
if len(_files) >0:
for k,v in _files.items():
open(k,'wb').write(v)
getLocalFiles()
So, here is how I finally solved this. I have to point out however, that in my case I had to work with several files and proprietary modules that were changing all the time.
The best solution I found to do this was to use a FUSE wrapper to "link" colab to my google account. I used this particular tool:
https://github.com/astrada/google-drive-ocamlfuse
There is an example of how to set up your environment there, but here is how I did it:
# Install a Drive FUSE wrapper.
!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null
!apt-get update -qq 2>&1 > /dev/null
!apt-get -y install -qq google-drive-ocamlfuse fuse
# Generate auth tokens for Colab
from google.colab import auth
auth.authenticate_user()
# Generate creds for the Drive FUSE library.
from oauth2client.client import GoogleCredentials
creds = GoogleCredentials.get_application_default()
import getpass
!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL
vcode = getpass.getpass()
!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret}
At this point you'll have installed the wrapper and the code above will generate a couple of links for you to authorize access to your google drive account.
The you have to create a folder in the colab file system (remember this is not persistent, as far as I know...) and mount your drive there:
# Create a directory and mount Google Drive using that directory.
!mkdir -p drive
!google-drive-ocamlfuse drive
print ('Files in Drive:')
!ls drive/
the !ls command will print the directory contents so you can check it works, and that's it. You now have all the files you need and you can make changes to them with no further complications. Remember that you may need to restar the kernel to update the imports and variables.
Hope this works for someone!
you can write following commands in colab to mount the drive
from google.colab import drive
drive.mount('/content/gdrive')
and you can download from some external url into the drive through simple linux command wget like this
!wget 'https://dataverse.harvard.edu/dataset'