I'm attempting to run the Dask-MPI "Getting Started" (http://mpi.dask.org/en/latest/) example in a fresh Anaconda environment.
I set up an environment using
conda create -n dask-mpi -c conda-forge python=3.7 dask-mpi
conda activate dask-mpi
Inside the environment, I run
mpirun -np 4 dask-mpi --scheduler-file ./scheduler.json
Then, from a python interpreter on the same machine (and in the same folder), I run
from dask.distributed import Client
client = Client(scheduler_file='/path/to/scheduler.json')
This results in the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/distributed/client.py", line 712, in __init__
self.start(timeout=timeout)
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/distributed/client.py", line 858, in start
sync(self.loop, self._start, **kwargs)
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/distributed/utils.py", line 331, in sync
six.reraise(*error[0])
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/six.py", line 693, in reraise
raise value
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/distributed/utils.py", line 316, in f
result[0] = yield future
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/distributed/client.py", line 954, in _start
yield self._ensure_connected(timeout=timeout)
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/distributed/client.py", line 1015, in _ensure_connected
timedelta(seconds=timeout), self._update_scheduler_info()
File "/home/nleaf/anaconda3/envs/dask-mpi/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
tornado.util.TimeoutError: Timeout
The terminal that I ran dask-mpi from does not have any output which would indicate that something is trying to connect. I have verified that the port in question, 8786, is open. I've also verified via debugger that the client is getting the correct address from the scheduler file.
I've tried this in quite a few different environments and on a few different machines, including a fresh Ubuntu 18.04 docker container. I'm completely at a loss for what steps I might be missing.
It turns out this was due to an error in newer versions of dask.distributed (1.25.3) which broke the behavior of dask-mpi. This seems to be fixed as of dask-mpi 1.0.3 (https://github.com/dask/dask-mpi/releases/tag/1.0.3).
Related
I am running a relation extraction spacy model on google colab , It works when I use !spacy project run all or !spacy project run train_cpu but when I run !spacy project run train_gpu it returns following error:
================================= train_gpu =================================
Running command: /usr/bin/python3 -m spacy train configs/rel_trf.cfg --output training --paths.train data/train.spacy --paths.dev data/dev.spacy -c ./scripts/custom_functions.py --gpu-id 0
ℹ Saving to output directory: training
ℹ Using GPU: 0
=========================== Initializing pipeline ===========================
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.7/dist-packages/spacy/__main__.py", line 4, in <module>
setup_cli()
File "/usr/local/lib/python3.7/dist-packages/spacy/cli/_util.py", line 71, in setup_cli
command(prog_name=COMMAND)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/typer/main.py", line 497, in wrapper
return callback(**use_params) # type: ignore
File "/usr/local/lib/python3.7/dist-packages/spacy/cli/train.py", line 45, in train_cli
train(config_path, output_path, use_gpu=use_gpu, overrides=overrides)
File "/usr/local/lib/python3.7/dist-packages/spacy/cli/train.py", line 72, in train
nlp = init_nlp(config, use_gpu=use_gpu)
File "/usr/local/lib/python3.7/dist-packages/spacy/training/initialize.py", line 41, in init_nlp
nlp = load_model_from_config(raw_config, auto_fill=True)
File "/usr/local/lib/python3.7/dist-packages/spacy/util.py", line 531, in load_model_from_config
validate=validate,
File "/usr/local/lib/python3.7/dist-packages/spacy/language.py", line 1784, in from_config
raw_config=raw_config,
File "/usr/local/lib/python3.7/dist-packages/spacy/language.py", line 794, in add_pipe
validate=validate,
File "/usr/local/lib/python3.7/dist-packages/spacy/language.py", line 652, in create_pipe
raise ValueError(err)
ValueError: [E002] Can't find factory for 'transformer' for language English (en). This usually happens when spaCy calls `nlp.create_pipe` with a custom component name that's not registered on the current language class. If you're using a Transformer, make sure to install 'spacy-transformers'. If you're using a custom component, make sure you've added the decorator `#Language.component` (for function components) or `#Language.factory` (for class components).
Available factories: attribute_ruler, tok2vec, merge_noun_chunks, merge_entities, merge_subtokens, token_splitter, doc_cleaner, parser, beam_parser, entity_linker, ner, beam_ner, entity_ruler, lemmatizer, tagger, morphologizer, senter, sentencizer, textcat, spancat, textcat_multilabel, relation_extractor, en.lemmatizer
I used both following installations (interchangeably) in case the GPU wasn't called correctly:
!pip install -U spacy[cuda101]
#!pip install -U spacy-nightly --pre
You haven't installed spacy-transformers. The easiest way to do this is probably to spacy download en_core_web_trf.
I would recommend you check the install quickstart again - I don't think spacy-nightly has been updated since v3 was released almost a year ago. Also check the Discussions FAQ - it's been a while since we've heard reports of it, but a while ago you had to specifically not install cupy (that is, not use pip install spacy[cuda101]) in order to get GPU support on Colab.
I am trying to install Tensorflow from source using Bazel on Raspberry pi. I am following the official documentation as given here. When I run the ./configure in Tensorflow directory after completing all the steps written for Bazel, I get the following error
/home/cvit/bin/bazel: line 88: /home/cvit/.bazel/bin/bazel-real: cannot execute binary file: Exec format error
/home/cvit/bin/bazel: line 88: /home/cvit/.bazel/bin/bazel-real: Success
Traceback (most recent call last):
File "./configure.py", line 1552, in <module>
main()
File "./configure.py", line 1432, in main
check_bazel_version('0.15.0')
File "./configure.py", line 450, in check_bazel_version
curr_version = run_shell(['bazel', '--batch', '--bazelrc=/dev/null', 'version'])
File "./configure.py", line 141, in run_shell
output = subprocess.check_output(cmd)
File "/usr/lib/python2.7/subprocess.py", line 223, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['bazel', '--batch', '--bazelrc=/dev/null', 'version']' returned non-zero exit status 1
I didn't put the user flag in the bazel installation. So, I think this might be bazelrc error so I tried to set $PATH=$BAZEL/bin but nothing happened.
Please give any suggestion !!
Probably the problem is that the non appropriate version of bazel is installed.
Run bazel version in the tensorflow directory, and see if there is an error.
If there is a problem with bazel version, then check out the .baselversion file, and if it contains a version that isn't installable with apt, then dowload the installer from https://github.com/bazelbuild/bazel/releases then install it, else install with apt.
After that everything should work fine.
While using Apache beamIO for preprocessing data, snappy library was a good to have module for compression but looks like the file transformation doesnt seems to work as it cannot find the crc32 compress function in the library! Im using snappy-0.5.2 version
the error looks like this -
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
ERROR:root:Exception at bundle <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7f1dd1d60e50>, due to an exception.
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/direct/executor.py", line 312, in call
side_input_values)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/direct/executor.py", line 347, in attempt_call
evaluator.process_element(value)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/direct/transform_evaluator.py", line 551, in process_element
self.runner.process(element)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/common.py", line 390, in process
self._reraise_augmented(exn)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/common.py", line 388, in process
self.do_fn_invoker.invoke_process(windowed_value)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/common.py", line 281, in invoke_process
self._invoke_per_window(windowed_value)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/common.py", line 307, in _invoke_per_window
windowed_value, self.process_method(*args_for_process))
File "/usr/local/lib/python2.7/dist-packages/apache_beam/typehints/typecheck.py", line 63, in process
return self.wrapper(self.dofn.process, args, kwargs)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/typehints/typecheck.py", line 81, in wrapper
result = method(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/iobase.py", line 965, in process
self.writer.write(element)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/filebasedsink.py", line 299, in write
self.sink.write_record(self.temp_handle, value)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/filebasedsink.py", line 129, in write_record
self.write_encoded_record(file_handle, self.coder.encode(value))
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/tfrecordio.py", line 235, in write_encoded_record
_TFRecordUtil.write_record(file_handle, value)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/tfrecordio.py", line 97, in write_record
struct.pack('<I', cls._masked_crc32c(encoded_length)), #
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/tfrecordio.py", line 77, in _masked_crc32c
crc = crc32c_fn(value)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/tfrecordio.py", line 43, in _default_crc32c_fn
_default_crc32c_fn.fn = snappy._crc32c # pylint: disable=protected-access
AttributeError: 'module' object has no attribute '_crc32c' [while running 'WriteTrainData/Write/WriteImpl/WriteBundles']
If any one could help me to use snappy with tensorflow correctly!
Thank you
I just hit this issue; I think it is due to Beam being a little careless about versions of optional test-dependencies (in this case, tensorflow and python-snappy).
The problematic code:
import snappy
snappy._crc32c
works in python-snappy version 0.5.1 but not in 0.5.2 (the latest version).
I got these Beam tests passing by installing python-snappy 0.5.1 via:
pip install \
--upgrade --ignore-installed \
python-snappy==0.5.1 \
--global-option=build_ext \
--global-option="-I/usr/local/include" \
--global-option="-L/usr/local/lib"
On OSX I need the three --global-option flags otherwise it doesn't find my snappy headers (symptom: errors about #include <snappy-c.h>) and library files, which brew install snappy placed in /usr/local/include and /usr/local/lib, respectively.
The bits before that seem necessary to override pip's default of wanting to give me the latest version.
I have Ubuntu 16.04 server environment and was trying to install CNTK on it. While I was trying to install pip install in an environment section I get the following error.
I succesfully ran below 2 steps:
$ conda create --name cntk-py34 python=3.4 numpy scipy h5py jupyter
$ activate cntk-py35
But when I try to install the cntk whl file I get an error:
$ pip install https://cntk.ai/PythonWheel/CPU-Only/cntk-2.0.beta15.0-cp35-cp35m-linux_x86_64.whl
========error==================
Exception:
Traceback (most recent call last):
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/commands/install.py", line 335, in run
wb.build(autobuilding=True)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/req/req_set.py", line 620, in _prepare_file
session=self.session, hashes=hashes)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/download.py", line 821, in unpack_url
hashes=hashes
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/download.py", line 659, in unpack_http_url
hashes)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/download.py", line 853, in _download_http_url
stream=True,
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/_vendor/requests/sessions.py", line 488, in get
return self.request('GET', url, **kwargs)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/download.py", line 386, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/_vendor/requests/sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/_vendor/requests/sessions.py", line 596, in send
r = adapter.send(request, **kwargs)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/_vendor/cachecontrol/adapter.py", line 37, in send
cached_response = self.controller.cached_request(request)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/_vendor/cachecontrol/controller.py", line 111, in cached_request
resp = self.serializer.loads(request, cache_data)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/_vendor/cachecontrol/serialize.py", line 114, in loads
return getattr(self, "_loads_v{0}".format(ver))(request, data)
File "/home/ubuntu/.conda/envs/cntk-py35/lib/python3.5/site-packages/pip/_vendor/cachecontrol/serialize.py", line 176, in _loads_v2
cached = json.loads(zlib.decompress(data).decode("utf8"))
MemoryError
Any ideas???
Thanks in advance!
In your question, you mention two different Python versions (3.4 and 3.5). Also, the Anaconda Environment activation needs to be sourced. Assuming you already satisfied the OpenMPI dependency (cf. https://github.com/Microsoft/CNTK/wiki/Setup-Linux-Python), can you try one of these:
# For a Python 3.4 based setup
conda create --name cntk-py34 python=3.4 numpy scipy h5py jupyter
source activate cntk-py34
pip install https://cntk.ai/PythonWheel/CPU-Only/cntk-2.0rc1-cp34-cp34m-linux_x86_64.whl
# For a Python 3.5 based setup
conda create --name cntk-py35 python=3.5 numpy scipy h5py jupyter
source activate cntk-py35
pip install https://cntk.ai/PythonWheel/CPU-Only/cntk-2.0rc1-cp35-cp35m-linux_x86_64.whl
I recently installed a new verion of Anaconda 3 4.2 on a windows laptop.
All the packages work fine , but pandas never worked for me from day 1.
So i thought of uninstalling and installing a new version of pandas 0.19
While using pip:
C:\Users\>python -m pip install --user pandas
Collecting pandas
Using cached pandas-0.19.2-cp35-cp35m-win_amd64.whl
Exception:
Traceback (most recent call last):
File "D:\Anaconda3-4.2\lib\site-packages\pip\basecommand.py", line 215, in main
status = self.run(options, args)
File "D:\Anaconda3-4.2\lib\site-packages\pip\commands\install.py", line 335, in run
wb.build(autobuilding=True)
File "D:\Anaconda3-4.2\lib\site-packages\pip\wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "D:\Anaconda3-4.2\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "D:\Anaconda3-4.2\lib\site-packages\pip\req\req_set.py", line 620, in _prepare_file
session=self.session, hashes=hashes)
File "D:\Anaconda3-4.2\lib\site-packages\pip\download.py", line 821, in unpack_url
hashes=hashes
File "D:\Anaconda3-4.2\lib\site-packages\pip\download.py", line 663, in unpack_http_url
unpack_file(from_path, location, content_type, link)
File "D:\Anaconda3-4.2\lib\site-packages\pip\utils\__init__.py", line 599, in unpack_file
flatten=not filename.endswith('.whl')
File "D:\Anaconda3-4.2\lib\site-packages\pip\utils\__init__.py", line 499, in unzip_file
fp = open(fn, 'wb')
PermissionError: [Errno 13] Permission denied: C:\\Users\\AppData\\Local\\Temp\\pip-build-h5ip5q8f\\pandas\\pandas/io/tests/data/blank.xls
While using conda :
An unexpected error has occurred.
Please consider posting the following information to the
conda GitHub issue tracker at:
https://github.com/conda/conda/issues
Current conda install:
platform : win-64
conda version : 4.3.8
conda is private : False
conda-env version : 4.3.8
conda-build version : 2.0.2
python version : 3.5.2.final.0
requests version : 2.12.4
root environment : D:\Anaconda3-4.2 (writable)
default environment : D:\Anaconda3-4.2
envs directories : D:\Anaconda3-4.2\envs
package cache : D:\Anaconda3-4.2\pkgs
channel URLs : https://repo.continuum.io/pkgs/free/win-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/win-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/win-64
https://repo.continuum.io/pkgs/pro/noarch
https://repo.continuum.io/pkgs/msys2/win-64
https://repo.continuum.io/pkgs/msys2/noarch
config file : None
offline mode : False
user-agent : conda/4.3.8 requests/2.12.4 CPython/3.5.2 Windows/7
Windows/6.1.7601
D:\Anaconda3-4.2\Scripts\conda-script.py install pandas
Traceback (most recent call last):
File "D:\Anaconda3-4.2\lib\site-packages\conda\exceptions.py", line 617, in conda_exception_handler
return_value = func(*args, **kwargs)
File "D:\Anaconda3-4.2\lib\site-packages\conda\cli\main.py", line 137, in _main
exit_code = args.func(args, p)
File "D:\Anaconda3-4.2\lib\site-packages\conda\cli\main_install.py", line 80, in execute
install(args, parser, 'install')
File "D:\Anaconda3-4.2\lib\site-packages\conda\cli\install.py", line 347, in install
execute_actions(actions, index, verbose=not context.quiet)
File "D:\Anaconda3-4.2\lib\site-packages\conda\plan.py", line 837, in execute_actions
execute_instructions(plan, index, verbose)
File "D:\Anaconda3-4.2\lib\site-packages\conda\instructions.py", line 258, in execute_instructions
cmd(state, arg)
File "D:\Anaconda3-4.2\lib\site-packages\conda\instructions.py", line 118, in UNLINKLINKTRANSACTION_CMD
txn = UnlinkLinkTransaction.create_from_dists(index, prefix, unlink_dists, link_dists)
File "D:\Anaconda3-4.2\lib\site-packages\conda\core\link.py", line 121, in create_from_dists
for dist, pkg_dir in zip(link_dists, pkg_dirs_to_link))
File "D:\Anaconda3-4.2\lib\site-packages\conda\core\link.py", line 121, in <genexpr>
for dist, pkg_dir in zip(link_dists, pkg_dirs_to_link))
File "D:\Anaconda3-4.2\lib\site-packages\conda\gateways\disk\read.py", line 71, in read_package_info
index_json_record = read_index_json(extracted_package_directory)
File "D:\Anaconda3-4.2\lib\site-packages\conda\gateways\disk\read.py", line 94, in read_index_json
with open(join(extracted_package_directory, 'info', 'index.json')) as fi:
FileNotFoundError: [Errno 2] No such file or directory: 'D:\\Anaconda3-4.2\\pkgs\\pandas-0.19.2-np111py35_1\\info\\index.json
The problem is that you are trying to install Pandas as a regular user, and therefore cannot change the contents of Anaconda installation folder (where Python installed packages live). You need to run CMD or PowerShell (whichever you are using) as an administrator. Right-click on its shortcut in the start menu and click "Run as administrator", then run again the same command.
Try to use Anaconda package manager - conda instead of pip:
C:\> conda install pandas