I'm using Colab Pro and I have no issue with the RAM when I'm using either GPU or TPU. The only problem is that my running usually takes more than 12 hours and it looks like Colab automatically stops (with no error) after 12 hours. I've reached out to their support and got no response (this is strange enough for itself that how/why Google does not have proper support for paid Colab Pro technical, not billing, issues!) I wonder if there is any way that the runtime limit can be increased to more than 12 hours? Or, has anyone experienced a similar situation or has any contact I can reach out?
A similar situation is described here, but since there was no question asked, I'm creating this post.
Answering this question here: the runtime limit is just simply part of the Google Pro plan as also stated on their website. For now, I had to upgrade my plan to Google Colab Pro+ to be able to 1) have longer runtime and 2) have background execution when I close my browser.
Is this ideal? No! Do I hate to spend $50 a month on Google Pro+ as a student? Yes! But that is just how Colab Pro plans work. As the [wise] man says: "it is what it is!"
Related
I subscribed to Colab Pro and I have always been able to get >1 instance of High Ram Non_GPU sessions. But today, I found this has been severely restricted.
I got "Too many session" popup when I tried a 2nd instance.
For the sole instance, I get only 12gb RAM even configuring for High RAM.
I am aware there had been pricing changes. Their provisioning policy is so vague that I am not sure if this is a violation of the terms. I have sent my "feedback" to alert them of the memory issue.
Anyone out there with similar experience? Colab has been one of the cheapest option despite awkwardness and inconvenience. But if it is no longer cost competitive, probably won't live with all the cons that go with it. I would greatly appreciate if someone can post other alternative (besides Kaggle).
In the last week or two I have seen frequent disconnects while trying to run a lengthy training run. A month or two ago this seemed to be working pretty reliably. My code has definitely changed but those internal details seem unrelated to the operation of Colab.
(On the other hand, I did switch my local machine from an Intel MacBook Pro running Big Sur to an M1 (Apple Silicon) MacBook Pro running Monterey. I assume that does not matter to Colab running in the cloud, via a Chrome browser.)
I see two kinds of disconnects:
There are “faux disconnects” which seem like false positives from
the disconnect detector. These last less than a second, then the
computation continues apparently unscathed. A black notification
slides up from the lower left corner of then window, then slides
back. See a link to a video of this below.
Then there are “real disconnects.” I start a computation that I
expect to run for several hours. I see “faux disconnects” happen
frequently. But less than an hour into the computation, I find
the Colab window idle, no status information, and a Reconnect button
in the upper right corner.
Link to video. I started this session around 1:03 pm. This video was recorded at 1:35 pm. Normally the training session should have run for several hours. Instead it died at 1:52 pm (~50 minutes into the run). See some additional comments in an issue at GitHub.
Can anyone help me understand how to get past this? I am currently unable to make progress in my work because I cannot complete a training run before my Colab runtime decides to disconnect.
Edit:
FYI: since once a “real disconnect” happens it is too late to look at the (no longer connected) runtime's log, and since this seems to run for about an hour before disconnecting, I saved a log file when a run was about 10 minutes in.
Edit on August 1, 2022:
My real problem is the “real disconnect” on my real Colab notebook. But my notebook is overly complicated, so not a good test case. I tried to make a small test case, see Colab notebook: DisconnectTest.ipynb. It contains a generic NIST-based Keras/TensorFlow benchmark from the innertubes. I made a screen grab video of the first 2.5 minutes of a run. While this run completes OK — that is, there are no “real disconnects” — it had several “faux disconnects.” The first one is at 1:36. These seem fairly benign, but they do disrupt the Resources panel on the right. This makes it hard to know if the source of the “real disconnect” has anything to do with exhausting resources.
As I described in a parallel post on Colab's Issue #2965 on Github, this appears to be “some interaction between Colab and Chrome (and perhaps macOS Monterey (Version 12.5.1), and perhaps M1 Apple Silicon). Yet Colab seems to work fine on M1/Monterey/Safari.”
As described there, a trivial Colab example fails on Chrome browser but works fine on Safari.
I can not connect to a GPU on my Colab Pro+ for more than 5 days. I suppose this must be a bug.
I understand that GPUs are allocated dynamically, but if someone pays Pro+ it would be essential to get a smaller GPU at least. I can not even develop anything atm, because I can't run specific code without a GPU.
10 days of this month I was not using Colab (holidays), then some days of use and now 5 days with the error message "Cannot connect to GPU backend".
Trying to contact the support is a nightmare: I only get automated answers for 5 days in a row. It's an interesting platform but if there is a problem you are screwed and your 50$/month are just wasted.
I am running a deep learning training program on my colab notebook which will cost about 10hours. If i close my browser, will it be shutdown by google before it ends as expected? Or will the last output be saved coorectly in my Drive?
I suggest you to look here and here. Basically, the code should keep running, but after some time (around 90 minutes) of idle activity, the notebook should be cut off, so I assume that what you suggest is not viable. Maybe you could try to launch the script in the morning and interact with it every 20-30 minutes to prevent it going to idle. Also, consider using Google Colab pro (faster GPUs and longer runtimes, but never longer that 24 hours)
The simple answer to that question is a solid no. Your session will go ahead and continue executing or will stay idle, as stated in the #SilentCloud 's Answer above it will go for about
90 Minutes [With CPU]
30 Minutes [With GPU]
The reason I say 30 Minutes with GPU is that I have personally tested that and it appears to be this number, as do use on a rather regular basis.
You can make a simple bot on Your Machine using pyautogui in order to go ahead and do some random stuff if for some reason it makes more economical sense, or you are not interested in Google Colab Pro Subscription.
Run with Browser Closed
If you want a seamless experience with the browser window effectively closed and having access to GPU's that are much more better and faster, I would recommend the Colab Pro + Subscription.
But the Scripting Idea is there, and your mileage may vary.
Recently google colab consumes too much of internet data . Approx 4GB in 6 hours of training for single notebook . What can be the issue ?
Yes I have the same issue. It normally works fine but, there is sudden spike in the internet data. Check this. In the process it wasted 700 Mb in just 20 minutes, and I have mobile internet, so this creates a problem sometimes. Didn't find the answer but it seems like there is some kind of synchronization going on between the browser and the colab platform.
One thing you could do is to open the notebook in Playground mode as shown in this link How to remove the autosave option in Colab. This only happens because of the fact that Colab is saving everytime and there is a constant spike in the network. It becomes difficult when you use only mobile data. So, it is a safe option to open the notebook in Playground mode, so that the synchronization doesn't continue as usual.