New to Scipy.
I've installed scipy and numpy, but ween I run the scipy.test() I get many test fails.
I tried to find some common reasons and I have found several:
>>> import scipy
>>> scipy.test()
Running unit tests for scipy
NumPy version 1.6.2
NumPy is installed in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy
SciPy version 0.10.1
SciPy is installed in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy
Python version 2.7.3 (default, Apr 19 2012, 00:55:09) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]
nose version 1.1.2
...
AssertionError:
Arrays are not almost equal to 7 decimals
...
AssertionError:
Not equal to tolerance rtol=1e-10, atol=1e-305
...
AssertionError:
Arrays are not almost equal to 4 decimals
...
AssertionError:
Max |adiff|: 92783.6
Max |rdiff|: 0.353942
Sounds to me like MPI problem or something, but I would't really know.
Any help would be greatly appreciated.
Best, Jonathan.
-------- EDIT ---------
Tried it with scipy.test(verbose=10).
Most tests seems to work fine.
The fails starts with an ERROR. this is what I get:
ERROR: Failure: DeprecationWarning (
The scipy.maxentropy module is deprecated in scipy 0.10, and scheduled to be
removed in 0.11.
If you are using some of the functionality in this module and are of the
opinion that it should be kept or moved somewhere - or you are even interested
to maintain/improve this whole module - please ask on the scipy-dev mailing
list.
The logsumexp function has already been moved to scipy.misc.)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/loader.py", line 390, in loadTestsFromName
addr.filename, addr.module)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 39, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/importer.py", line 86, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/maxentropy/__init__.py", line 19, in <module>
DeprecationWarning)
DeprecationWarning:
The scipy.maxentropy module is deprecated in scipy 0.10, and scheduled to be
removed in 0.11.
If you are using some of the functionality in this module and are of the
opinion that it should be kept or moved somewhere - or you are even interested
to maintain/improve this whole module - please ask on the scipy-dev mailing
list.
The logsumexp function has already been moved to scipy.misc.
======================================================================
FAIL: test_asum (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/lib/blas/tests/test_blas.py", line 58, in test_asum
assert_almost_equal(f([3,-4,5]),12)
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 467, in assert_almost_equal
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals
ACTUAL: 0.0
DESIRED: 12
======================================================================
FAIL: test_dot (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/lib/blas/tests/test_blas.py", line 67, in test_dot
assert_almost_equal(f([3,-4,5],[2,5,1]),-9)
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 467, in assert_almost_equal
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals
ACTUAL: 0.0
DESIRED: -9
======================================================================
FAIL: test_nrm2 (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/lib/blas/tests/test_blas.py", line 78, in test_nrm2
assert_almost_equal(f([3,-4,5]),math.sqrt(50))
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 467, in assert_almost_equal
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals
ACTUAL: 0.0
DESIRED: 7.0710678118654755
======================================================================
FAIL: test_basic.TestNorm.test_overflow
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_basic.py", line 581, in test_overflow
assert_almost_equal(norm(a), a)
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 451, in assert_almost_equal
return assert_array_almost_equal(actual, desired, decimal, err_msg)
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 846, in assert_array_almost_equal
header=('Arrays are not almost equal to %d decimals' % decimal))
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 677, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals
(mismatch 100.0%)
x: array(-0.0)
y: array([ 1.00000002e+20], dtype=float32)
======================================================================
FAIL: test_basic.TestNorm.test_stable
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_basic.py", line 586, in test_stable
assert_almost_equal(norm(a) - 1e4, 0.5)
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 467, in assert_almost_equal
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals
ACTUAL: -10000.0
DESIRED: 0.5
======================================================================
FAIL: test_basic.TestNorm.test_types
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_basic.py", line 568, in test_types
assert_allclose(norm(x), np.sqrt(14), rtol=tol)
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 1213, in assert_allclose
verbose=verbose, header=header)
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 677, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=2.38419e-06, atol=0
(mismatch 100.0%)
x: array(1.0842021724855044e-19)
y: array(3.7416573867739413)
======================================================================
FAIL: test_asum (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_blas.py", line 99, in test_asum
assert_almost_equal(f([3,-4,5]),12)
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 467, in assert_almost_equal
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals
ACTUAL: 0.0
DESIRED: 12
======================================================================
FAIL: test_dot (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_blas.py", line 109, in test_dot
assert_almost_equal(f([3,-4,5],[2,5,1]),-9)
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 467, in assert_almost_equal
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals
ACTUAL: 0.0
DESIRED: -9
======================================================================
FAIL: test_nrm2 (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_blas.py", line 127, in test_nrm2
assert_almost_equal(f([3,-4,5]),math.sqrt(50))
File "/Library/Python/2.7/site-packages/numpy/testing/utils.py", line 467, in assert_almost_equal
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals
ACTUAL: 0.0
DESIRED: 7.0710678118654755
----------------------------------------------------------------------
Ran 5101 tests in 56.140s
FAILED (KNOWNFAIL=12, SKIP=42, errors=1, failures=9)
<nose.result.TextTestResult run=5101 errors=1 failures=9>
I am really sorry to say I have no idea what the errors mean.
The Scipy failures seem typical of what comes out from using a wrong (= ABI incompatibility with Veclib) Fortran compiler on OSX. The Numpy failures are pretty likely unrelated to these --- difficult to say without seeing them.
The mess with binary interface incompatibilities and Fortran compilers on OSX is unfortunate and applies to all Fortran code, but not very easily fixable by Scipy developers --- except that if an incorrect Fortran compiler is detected, the build should be made to fail with an informative error message...
You may find these useful: http://projects.scipy.org/scipy/ticket/1476 http://projects.scipy.org/scipy/ticket/1496
Can you run scipy.test(verbose=10), please? This will print more output and help in finding ot what's wrong.
Looks like you are running into exactly the situation described in this mailing list.
In the discussion it is mentioned that this might relate to a bug in SciPy. There is a bug report describing your situation. You may want to follow this report.
Related
I know there are many questions about tensordot, and I've skimmed some of the 15 page mini-book answers that people I'm sure spent hours making, but I haven't found an explanation of what axes=2 does.
This made me think that np.tensordot(b,c,axes=2) == np.sum(b * c), but as an array:
b = np.array([[1,10],[100,1000]])
c = np.array([[2,3],[5,7]])
np.tensordot(b,c,axes=2)
Out: array(7532)
But then this failed:
a = np.arange(30).reshape((2,3,5))
np.tensordot(a,a,axes=2)
If anyone can provide a short, concise explanation of np.tensordot(x,y,axes=2), and only axes=2, then I would gladly accept it.
In [70]: a = np.arange(24).reshape(2,3,4)
In [71]: np.tensordot(a,a,axes=2)
Traceback (most recent call last):
File "<ipython-input-71-dbe04e46db70>", line 1, in <module>
np.tensordot(a,a,axes=2)
File "<__array_function__ internals>", line 5, in tensordot
File "/usr/local/lib/python3.8/dist-packages/numpy/core/numeric.py", line 1116, in tensordot
raise ValueError("shape-mismatch for sum")
ValueError: shape-mismatch for sum
In my previous post I deduced that axis=2 translates to axes=([-2,-1],[0,1])
How does numpy.tensordot function works step-by-step?
In [72]: np.tensordot(a,a,axes=([-2,-1],[0,1]))
Traceback (most recent call last):
File "<ipython-input-72-efdbfe6ff0d3>", line 1, in <module>
np.tensordot(a,a,axes=([-2,-1],[0,1]))
File "<__array_function__ internals>", line 5, in tensordot
File "/usr/local/lib/python3.8/dist-packages/numpy/core/numeric.py", line 1116, in tensordot
raise ValueError("shape-mismatch for sum")
ValueError: shape-mismatch for sum
So that's trying to do a double axis reduction on the last 2 dimensions of the first a, and the first 2 dimensions of the second a. With this a that's a dimensions mismatch. Evidently this axes was intended for 2d arrays, without much thought given to 3d ones. It is not a 3 axis reduction.
These single digit axes values are something that some developer thought would be convenient, but that does not mean they were rigorously thought out or tested.
The tuple axes gives you more control:
In [74]: np.tensordot(a,a,axes=[(0,1,2),(0,1,2)])
Out[74]: array(4324)
In [75]: np.tensordot(a,a,axes=[(0,1),(0,1)])
Out[75]:
array([[ 880, 940, 1000, 1060],
[ 940, 1006, 1072, 1138],
[1000, 1072, 1144, 1216],
[1060, 1138, 1216, 1294]])
It takes nearly three hours for the simulator to get pass the initialization point when I try to run a PARSEC benchmark in full system mode, only to be met with the following output:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "build/X86/python/m5/main.py", line 457, in main
exec(filecode, scope)
File "configs/example/fs.py", line 396, in <module>
Simulation.run(options, root, test_sys, FutureClass)
File "configs/common/Simulation.py", line 726, in run
exit_event = benchCheckpoints(options, maxtick, cptdir)
File "configs/common/Simulation.py", line 269, in benchCheckpoints
exit_event = m5.simulate(maxtick - m5.curTick())
File "build/X86/python/m5/simulate.py", line 180, in simulate
return _m5.event.simulate(*args, **kwargs)
RuntimeError: bad_function_call
I don't even know where to begin solving this problem. Does anyone know what might be causing this issue and how to solve it? Thanks.
I want to view statistics of my model by saving my graph to a file then running graph_metrics.py.
I have tried a few different things to write the file, my best effort is:
tf.train.write_graph( session.graph_def, ".", "my_graph", as_text=True )
But here's what happens:
$ python ./util/graph_metrics.py --noinput_binary --graph my_graph
Traceback (most recent call last):
File "./util/graph_metrics.py", line 137, in <module>
tf.app.run()
File ".virtualenv/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run
sys.exit(main(sys.argv))
File "./util/graph_metrics.py", line 85, in main
FLAGS.batch_size)
File "./util/graph_metrics.py", line 109, in calculate_graph_metrics
input_tensor = sess.graph.get_tensor_by_name(input_layer)
File ".virtualenv/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2531, in get_tensor_by_name
return self.as_graph_element(name, allow_tensor=True, allow_operation=False)
File ".virtualenv/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2385, in as_graph_element
return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
File ".virtualenv/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2427, in _as_graph_element_locked
"graph." % (repr(name), repr(op_name)))
KeyError: "The name 'Mul:0' refers to a Tensor which does not exist. The operation, 'Mul', does not exist in the graph."
Is there a complete working example of saving a graph, then analyzing it with graph_metrics.py?
This process seems to involve a magic incantation that I haven't yet discovered.
The error you're hitting is because you need to specify the name of your own input node with --input_layer= (it just defaults to Mul:0 because that's what we use in one of our Inception models):
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/graph_metrics.py#L51
The graph_metrics script is still very much a work in progress unfortunately, and you may hit problems with shape inference, but hopefully this should get you past the initial hurdle.
Firstly, I know very very little about Python, Xwindows, Matplotlib or GTK. I am trying to run a tool called SpliceGrapher which uses the above. I get an (ugly, sorry) error:
/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/__init__.py:57: GtkWarning: could not open display
warnings.warn(str(e), _gtk.Warning)
/home/my/bin/lib64/python2.6/site-packages/matplotlib/backends/backend_gtk.py:52: GtkWarning: gdk_cursor_new_for_display: assertion `GDK_IS_DISPLAY (display)' failed
cursors.MOVE : gdk.Cursor(gdk.FLEUR),
Traceback (most recent call last):
File "/home/my/bin/SpliceGrapher-0.2.0/scripts/view_splicegraphs.py", line 28, in <module>
from pylab import *
File "/home/my/bin/lib64/python2.6/site-packages/pylab.py", line 1, in <module>
from matplotlib.pylab import *
File "/home/my/bin/lib64/python2.6/site-packages/matplotlib/pylab.py", line 264, in <module>
from matplotlib.pyplot import *
File "/home/my/bin/lib64/python2.6/site-packages/matplotlib/pyplot.py", line 95, in <module>
new_figure_manager, draw_if_interactive, _show = pylab_setup()
File "/home/my/bin/lib64/python2.6/site-packages/matplotlib/backends/__init__.py", line 25, in pylab_setup
globals(),locals(),[backend_name])
File "/home/my/bin/lib64/python2.6/site-packages/matplotlib/backends/backend_gtkagg.py", line 10, in <module>
from matplotlib.backends.backend_gtk import gtk, FigureManagerGTK, FigureCanvasGTK,\
File "/home/my/bin/lib64/python2.6/site-packages/matplotlib/backends/backend_gtk.py", line 52, in <module>
cursors.MOVE : gdk.Cursor(gdk.FLEUR),
RuntimeError: could not create GdkCursor object
If anyone has any idea what I can do, that would be great.
I am on a Linux (RedHat) system, through puTTy using Xming. I have X11 configured and xeyes shows the little eyes that follow my mouse so I know thats working.
New error:
16:00:46 view_splicegraphs.py Started
Traceback (most recent call last):
File "/home/bmoran/bin/SpliceGrapher-0.2.0/scripts/view_splicegraphs.py", line 164, in <module>
g = getFirstGraph(f)
File "/home/bmoran/bin/lib64/python2.6/site-packages/SpliceGrapher/SpliceGraph.py", line 371, in getFirstGraph
result = SpliceGraphParser(f, **args).next()
File "/home/bmoran/bin/lib64/python2.6/site-packages/SpliceGrapher/SpliceGraph.py", line 1351, in __init__
self.loadFromFile()
File "/home/bmoran/bin/lib64/python2.6/site-packages/SpliceGrapher/SpliceGraph.py", line 1424, in loadFromFile
raise ValueError("Graph feature found before graph header at line %d" % lineNo)
ValueError: Graph feature found before graph header at line 1
/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/__init__.py:57: GtkWarning: could not open display
Tells you your script does not have an access to your X session.
Can you post here your script?
Did you access the machine with ssh -XC ... ?
on the shell when you issue echo $DISPLAY what do you see?
When you add in your script print os.getenv("DISPLAY") do you see the same result?
one more note ... I don't know Xming but I can recommend you to try MobaXterm, it has a builtin ssh and X11 server compiled for Windows, and my guess is that it will solve your X problem.
I am attempting to execute the following within python:
from pandas import *
tickdata = read_csv('/home/user/sourcefile.csv',index_col=0,parse_dates='TRUE')
The csv files has rows that look like:
2011/11/23 23:56:00.554389,1165.2500
2011/11/23 23:56:02.310943,1165.5000
2011/11/23 23:56:05.564009,1165.2500
On pandas .7, this executes fine. On pandas .8.0rc2, I get the error below. Because I have .7 and .8 installed on two different systems, I have not ruled out a dependency or python version difference. Any ideas on how to get this to execute under .8 are appreciated.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pandas-0.8.0rc2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 225, in read_csv
return _read(TextParser, filepath_or_buffer, kwds)
File "/usr/local/lib/python2.7/dist-packages/pandas-0.8.0rc2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 192, in _read
return parser.get_chunk()
File "/usr/local/lib/python2.7/dist-packages/pandas-0.8.0rc2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 728, in get_chunk
index = self._agg_index(index)
File "/usr/local/lib/python2.7/dist-packages/pandas-0.8.0rc2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 846, in _agg_index
if try_parse_dates and self._should_parse_dates(self.index_col):
File "/usr/local/lib/python2.7/dist-packages/pandas-0.8.0rc2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 874, in _should_parse_dates
return i in to_parse or name in to_parse
TypeError: 'in <string>' requires string as left operand, not int
I fixed the parser bug shown in the stack trace that you pasted. However, I'm wondering whether your date column is named "TRUE" or did you mean to just pass a boolean? I haven't dug through pandas history but I know that in 0.8 we are supporting much more complex date parsing behavior as part of the time series API so here we're interpreting the string value as a column name.
I've reported the bug on GitHub (best place for bug reports):
https://github.com/pydata/pandas/issues/1544
Should have a resolution today or tomorrow.