I want to use Google word2vec (GoogleNews-vectors-negative300.bin)
I downloaded it from https://code.google.com/archive/p/word2vec/
When I load it, the memory errors occured
(Process finished with exit code 139 (interrupted by signal 11: SIGSEGV))
from gensim.models.word2vec import Word2Vec
embedding_path = "data/GoogleNews-vectors-negative300.bin"
word2vec = Word2Vec.load_word2vec_format(embedding_path, binary=True)
print word2vec
I use ubuntu 16.04 / GTX-1070(8gb) / Ram(16gb).
How can I fix it?!
A SIGSEGV is an error occurring when the process tries to access a particular segment in memory that it does not have permission on.
So you should check permissions and, by debugging, see which memory location gives you the error.
This way you could understand if another program is interfering.
The problem might also be CUDA related as #TheM00s3 suggested
Related
I am adding profiler on my custom method on Odoo v15.0
I have referred below doc for code profiling
https://www.odoo.com/documentation/15.0/developer/howtos/profilecode.html
Below is the syntax i am using
from odoo.tools.profiler import profile
#profile
#api.model
def mymethod(...):
My code
But on execution of code i am getting below error on terminal
"ImportError: cannot import name 'profile' from 'odoo.tools.profiler'"
To debug the issue i have deep dived in base code of "/odoo/tool/profiler.py".
But unable to locate any wrapper or function called profiler.
What is correct way to use profiling using "Log a method" strategy on odoo v15.0?
Goto path and make sure you have this file for line-by-line code profiling looks like you don't have this file
From the front end in debug mode open enable profiling, this will give you all the information per user
I'm using Google's Natural Language API to run sentiment analysis on text blocks, and according to some instructions I'm following, I need t be using the latest version of google-cloud-language.
So I'm running this at the start of my Colab notebook.
!pip install --upgrade google-cloud-language
When I get to the end of that, it requires me to restart the runtime, which means I can't automatically run this along with my entire code, instead having to manually attend to the runtime restart.
This SO post touches on the topic, but only offers the 'crash' solution, and I'm wondering if anything else is available now 3 years later.
Restart kernel in Google Colab
So I'm curious if there's any workaround, or way to permanently upgrade google-cloud-language to avoid that?
Thank you for any input.
Here's the NL code I'm running, if helpful.
# Imports the Google Cloud client library
from google.cloud import language_v1
# Instantiates a client
client = language_v1.LanguageServiceClient()
def get_sentiment(text):
# The text to analyze
document = language_v1.Document(
content=text,
type_=language_v1.types.Document.Type.PLAIN_TEXT
)
# Detects the sentiment of the text
sentiment = client.analyze_sentiment(
request={"document": document}
).document_sentiment
return sentiment
dfTW01["sentiment"] = dfTW01["text"].apply(get_sentiment)
I am trying to save my model by using tf.keras.callbacks.ModelCheckpoint with filepath as some folder in drive, but I am getting this error:
File system scheme '[local]' not implemented (file: './ckpt/tensorflow/training_20220111-093004_temp/part-00000-of-00001')
Encountered when executing an operation using EagerExecutor. This error cancels all future operations and poisons their output tensors.
Does anybody know what is the reason for this and the workaround for this?
Looks to me that you are trying to access the file system of your host VM from the TPU which is not directly possible.
When using the TPU and you want to access files in e.g. GoogleColab you should place it within:
with tf.device('/job:localhost'):
<YOUR_CODE>
Now to your problem:
The local host acts as parameter server when training on TPU. So if you want to checkpoint your training, the localhost must do so.
When you check the documention for said callback, you cann find the parameter options.
checkpoint_options = tf.train.CheckpointOptions(experimental_io_device='/job:localhost')
checkpoint = tf.keras.callbacks.ModelCheckpoint(<YOUR_PATH>, options = checkpoint_options)
Hope this solves your issue!
Best,
Sascha
I'm trying to run a tensorflow for pre-annotation in cvat.
I can start the docker container, option in the menue shows up.
However, after selection of the model i get the error:
Could not infer model for the task 10
Error: Request failed with status code 400. "No labels found for tf annotation".
It seems that i need to specify some labels, in which format do i have to configure them though ?
Documentation seems sparse on this one. Maybe someone on here knows something?
Also if some stackoverflow wizzard with a lot of reputation could create the tag cvat i would be very happy :)
I am working on Google Colab environment to create a Siamese network using Keras. I have used this code from GitHub. But I get an error when I tried to run the pickle.dump code:
with open(os.path.join(save_path,"train.pickle"), "wb") as f:
pickle.dump((X,c), f)
The error : OverflowError: cannot serialize a bytes object larger than 4 GiB
So, I used Use pickle with protocol=4
pickle.dump((X,c), f, protocol=4)
but the session stopped during running this code and I got this message "Session crash for an unknown reason " and Your session crashed after using all available RAM
How can I solve this problem?
My guess is that your runtime is crashing out of memory.
I was able to pickle 4 GB of data, but it required ~8G of memory in Python to do so.
You can view the runtime logs with 'View runtime logs' for the Runtime menu. That often has hints about crashes. In this case, it reports many large allocations.
Example:
The sessions manager will show memory. In my case, without doing anything else:
I tried this and its working for me
import pickle
pickle_out = open("train.pickle","wb")
pickle.dump((X,c), pickle_out)
pickle_out.close()