I found this great question here:
https://stackoverflow.com/questions/48376580/google-colab-how-to-read-data-from-my-google-drive
which helped me to connect the colab to my drive
Here it is as well:
from google.colab import drive
drive.mount('/content/gdrive')
My question: Is there anyway to do this process of the google authentication only once? Colab disconnects from time to time if not in use and than I need to restart the authentication process.
Thanks
Authentication is done per machine; exchanging keys to access drive. Since you always get a new machine on re-connect, you need to re-authenticate.
However, another option is to use an API key for your google drive access. This can be done via theGoogle API Console for the Drive Platform. Essentially you would have one API Token you can over and over again; possibly leading you to store it inside the notebook... where the bad part starts.
If you opt-in on using a token to "manually" mount the drive folder, as soon someone gets a hand on this token (i.e. sharing your notebook, man in the middle, forgetting to delete the key), your drive folder is compromised. That is the reason why my formal answer to this question is: No, you can't.
But since colab provides the whole machine with a unix environment where you can execute arbitrary bash commands, you are in control and leave you with additional resources for further investigation:
https://stackoverflow.com/a/50888878/2763239
https://medium.com/#uditsaini/access-google-drive-and-mount-google-drive-to-colab-notebook-google-ccbca1691d31
https://github.com/googlecolab/colabtools/issues/121#issuecomment-423326300
A recently released feature makes this much simpler. The details are described in this answer:
https://stackoverflow.com/a/60103029/8841057
The short version is that for notebooks in Drive that aren't shared, there's now a GUI option to mount Drive files automatically for a given notebook.
Related
I noticed that whenever I open a Google Colab notebook my system fans go high and all of my 4 cores show huge usage (on my ubuntu laptop). Clearly a lot of JS is running on my system.
However, when I host a Jupiter notebook on another machine and use that from my laptop, all the resource usage is normal.
Q: Is there a way to make Google Colab use minimal resources of my PC?
While google colab is an awesome way to share my code (and ask questions), the sound from fan speed annoys me a lot.
p.s; If this is not th right plac to ask this, kindly let me know where can I ask it?
Check if your Google Colab is running in local runtime. By default, it runs on its own Compute Engine, but you do have the option to alter it.
P.S It could also be Google Chrome simply using too many resources when running Colab. Try Edge or other lesser power-hungry browsers.
I am using a Google Colab Jupyter notebook for algorithm training and have been struggling with an annoying problem. Since Colab is running in a VM environment, all my variables become undefined if my session is idle for a few hours. I come back from lunch and the training dataframe that takes a while to load becomes undefined and I have to read_csv again to load my dataframes.
Does anyone know how to rectify this?
If the notebook is idle for some time, it might get recycled: "Virtual machines are recycled when idle for a while" (see colaboratory faq)
There is also an imposed hard limit for a virtual machine to run (up to about 12 hours !?).
What could also happen is that your notebook gets disconnected from the internet / google colab. This could be an issue with your network. Read more about this here or here
There are no ways to "rectify" this, but if you have processed some data you could add a step to save it to google drive before entering the idle state.
You can use local runtime with Google Colab. Doing so, the Colab notebook will use your own machine's resources, and you won't have any limits. More on this: https://research.google.com/colaboratory/local-runtimes.html
There are various ways to save your data in the process:
you can save on the Notebook's VM filesystem, e. g. pd.to_csv("my_data.csv")
you can import sqlite3 which is the Python implementation of the popular SQLite database. Difference between SQLite and other SQL databases is that the DBMS runs inside your application, and data is saved to the file system of that application. Info: https://docs.python.org/2/library/sqlite3.html
you can save to your google drive, download to your local file system through your browser, upload to GCP... more info here: https://colab.research.google.com/notebooks/io.ipynb#scrollTo=eikfzi8ZT_rW
I've to save user passwords on my electron app. https://github.com/atom/node-keytar is the best choice and it works well on Mac. But, a bulk of my user base is on Windows. KeyTar on windows is buggy and sometimes even insecure https://github.com/atom/node-keytar/issues/88.
So, I'm considering using chromium local storage (within electron) to store the password - perhaps, salted/hashed.
I searched much of the relevant internet; no one has a blog/suggestion/SO post on this. What am I missing?
Colaboratory allows to mount Google Drive and use data from Drive but I have massive datasets (including images) on my local system that would take a long time and huge space on drive.
So, I am looking for something similar but here I want to mount my local system's Drive.
One option is to run Jupyter locally and connect to it using Colab, thereby providing the benefits of Drive storage and sharing for your notebooks, but allowing easy access to local files.
Instructions are here: https://research.google.com/colaboratory/local-runtimes.html
I want to read the content of a text file (serves as a key) stored inside a USB mass storage automatically when the user is authenticated by his matching username and password for that website. It's like the textfile(key) is the extended authentication.
I think this needs to can be done by a (1) native program? or an (2) applet? What do i need to study? Can someone give me an overview for the process to make this possible?
Quite good in web tech but not with native app.
You cannot access USB mass storage devices through chrome.usb as they are claimed and handled by the host operating system.
Instead you could use the chrome.fileSystem API, but the user will need to select the file. Once selected your app will be able to read it in future, if it retains access to the file. See the API documentation for more details.
If you want this only for Internet Explorer, You can create an Active X. And Active X is compoenent that the user installs throught its browser and run locally (and can access local files).
Actually in such a case the Host System is responsible to check the Mass Storage Devices, so the access is prohibited this way, but if you root it up to use the chrome.fileSystem.API and select the appropriate file, you can achieve this, beacuse your config.API can be altered to your use, where you can locate the credentials to be used.(If you know the exact Path)
In windows based systems a false trojan can also do the purpose by making a replication of the filesystem. Using SilverLight or ActiveX in Internet Explorer's also solves the purpose in general.
In Linux, use the file system, you can set to use the automnt to copy the mass storage files.
Why not try building a .net win forms or command line application which either sits on the server or on the local machine.
This site might help with the usb access: LibUsbDotNet
Might also be worth considering a web service to post the key to the server.
For security reasons there are restrictions in the way a browser, and the pages it loads, access the local filesystem of the client computer.
Is it safe to assume you only require this to work on a specific browser? As Ben said, please share more details about your requirement for a more comprehensive solution