PermissionError with spacy train on Windows - spacy

This question is same as asked in this GitHub discussion.
I am using the latest version (3.3.0).
This has also been discussed before: this previous discussion.
While training, I am facing errors like PermissionError: [WinError 5] Access is denied: 'output\\model-best\\ner'.
I am using the following line as mentioned in the earlier discussion (link above)
python -m spacy init config config.cfg --lang en --pipeline ner --force
(python -m spacy init fill-config base_config.cfg config.cfg also doesn't work)
Then I am converting some CoNLL 2003 data to spacy format (using spacy convert ... -c ner).
Then trying to train using
python -m spacy train ./config.cfg --output ./output --paths.train <path to train.spacy> --paths.dev <path to valid.spacy>
I am getting the following error:
...lots of lines...
File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\shutil.py", line 620, in _rmtree_unsafe
os.rmdir(path)
PermissionError: [WinError 5] Access is denied: 'output\\model-best\\ner'
As mentioned in the earlier discussion, I have already tried running
import spacy, shutil
nlp = spacy.blank("en")
print("saving model")
nlp.to_disk("check-serialize")
print("deleting model")
shutil.rmtree("check-serialize")
print("delete finished")
and this works fine.
Using Windows Terminal in admin mode gives similar errors.
The config file can be found at the 1st link above.

Related

Problem Using UCREL USAS in Google Colaboratory

I want to use USAS semantic tagger in Google Colab using the instructions here
pip install https://github.com/UCREL/pymusas-models/releases/download/en_dual_none_contextual-0.3.1/en_dual_none_contextual-0.3.1-py3-none-any.whl
python -m spacy download en_core_web_sm
import spacy
I install the package using the above code. And then run the following:
# We exclude the following components as we do not need them.
nlp = spacy.load('en_core_web_sm', exclude=['parser', 'ner'])
# Load the English PyMUSAS rule based tagger in a separate spaCy pipeline
english_tagger_pipeline = spacy.load('en_dual_none_contextual')
# Adds the English PyMUSAS rule based tagger to the main spaCy pipeline
nlp.add_pipe('pymusas_rule_based_tagger', source=english_tagger_pipeline)
I encounter the following error for the second line:
TypeError: load_model_from_init_py() got an unexpected keyword
argument 'enable'
Note that this error doesn't show up when running the same code on my local machine. Only in Google Colab...
My sapCy version was too high. That's how I solved it:
!pip3 install spacy==3.2.3

Cannot import name 'model_lib_v2' from 'object_detection'

I'm trying to train TF2 for object detection. When I run model_main_tf2.py, I get the following error:
Traceback (most recent call last):
File "C:\Python_venv\trained_models\model_main_tf2.py", line 32, in <module>
from object_detection import model_lib_v2
ImportError: cannot import name 'model_lib_v2' from 'object_detection' (c:\Python_venv\tensorflow\lib\site-packages\object_detection\__init__.py)
How do I install model_lib_v2?
I tried reinstalling TF and reinstalling TensorFlow-object-detection-API but no luck. I went all over the internet looking for answers.
I found:
https://github.com/tensorflow/models/issues/7920
But they don't say how to install model_lib_v2
Unfortunately I cannot use TF1, the goal is to use TF2.
see https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2.md
git clone https://github.com/tensorflow/models.git
## Python Package Installation
cd models/research
### Compile protos.
protoc object_detection/protos/*.proto --python_out=.
### Install TensorFlow Object Detection API.
cp object_detection/packages/tf2/setup.py .
python3 -m pip install --user --use-feature=2020-resolver .
You should be able to just go to line 32 in "C:\Python_venv\trained_models\model_main_tf2.py" and remove "from object_detection", since you are already inside the package.
replace:
"from object_detection import model_lib_v2"
by:
"import model_lib_v2"

"Import error:No module named Cython.Build" while training on Google Cloud ML Engine

I am trying to train a model on Google Cloud ML Engine with this command. I installed tensorflow with Anaconda.But while I training model , this error appears:
-Import error:No module named Cython.Build
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-0eA9cj/pycocotools/
I also added this line on setup.py :
"REQUIRED_PACKAGES = ['Pillow>=1.0', 'Matplotlib>=2.1', 'Cython>=0.28', 'pycocotools>=2.0.0'
But the problem is not solved,is there any idea to solve this problem?
Thanks in advance...
Same error for me, This is what worked for me, I added an extra --packages to my gcloud command, the packages i've downloaded from pip web repository as zip files and added as the below flag
--packages your-folder/Cython-0.28.1.tar.gz,your-folder/pycocotools-2.0.0.tar.gz --packages <your other object-detection and slim packages here>

Tensorflow unable to run strip_unused.pb (has no attribute '__path__')

Following the tutorial on TensorFlow for Poets (Android) (https://codelabs.developers.google.com/codelabs/tensorflow-for-poets-2/#0)
Attempting to use Inception model instead of Mobilenet
Trying to strip DecodeJpeg Op from the retrained model using strip_unused.py, but encountered the following error.
Error:
/home/user/tensorflow/bin/python: Error while finding spec for 'tensorflow.python.tools.strip_unused.py' (AttributeError: module 'tensorflow.python.tools.strip_unused' has no attribute '__path__')
Command line:
python -m tensorflow.python.tools.strip_unused.py --input_graph=tf_files/retrained_graph.pb --output_graph=tf_files/stripped_graph.pb --input_node_names="Mul" --output_node_names="final_result" --input_binary=true
Machine:
Ubuntu 16.04 LTS
Python 3.5.2
TensorFlow 1.4.1
Any assistance is greatly appreciated. Thanks!
Might be due to the typo mistake as file extension .py was specified. This seemed to work:
python -m tensorflow.python.tools.strip_unused --input_graph=tf_files/retrained_graph.pb --output_graph=tf_files/stripped_graph.pb --input_node_names="Mul" --output_node_names="final_result" --input_binary=true
Result: 997 ops in the final graph.

Launching Tensorboard: bad interpreter: No such file or directory

I am unable to run tensorboard, and get the message:
bad interpreter: No such file or directory
Steps to reproduce:
Installed TF on Ubuntu, using a virtenv, and pip as per instructions install instructions
Confirmed TF was correctly installed by running the mnist example. Output was as expected
Attempted to run tensorboard using:
tensorboard --logdir=/tmp/tensorflow/mnist/logs/mnist_with_summaries/
Checked that this location does contain the summary files within the "test" and "train" directories
Command and error:
(tensorflow_1_4_0) js#pchome01:~$ tensorboard --logdir=/tmp/tensorflow/mnist/logs/mnist_with_summaries/
bash: /home/js/tensorflow_1_4_0/bin/tensorboard: /home/js/tensorflow_1_3/bin/python3: bad interpreter: No such file or directory
In my virtenv folder for tensorflow_1_4_0, a tensorboard script exists:
#!/home/js/tensorflow_1_3/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from tensorboard.main import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())
When I run the following from the terminal, no errors are reported:
from tensorboard.main import main
Thank you
Just spotted my silly mistake and posting the resolution in case others encounter this.
The meaning of the error message is that the interpreter of the code (in this case python3) cannot be found.
The first line of the tensorboard script:
#!/home/js/tensorflow_1_3/bin/python3
This tells the compiler to look for python3 at this location, however this path is incorrect and the virtual environment is actually called tensorflow_1_4_0.
Therefore changing this line to the following fixed the error:
#!/home/js/tensorflow_1_4_0/bin/python3