I have a data in mat file (observations and features) and i want to load it into numpy 2D array. I dont want to convert it into csv first and then load csv into numpy.
Use scipy's loadmat (API-docs).
The docs should be sufficient to get you going, but make sure to read the notes.
There is also the io-tutorial with some examples.
Related
Suppose there exists a numpy array, data. I am trying to do the equivalent of the following
cv2.imwrite(filename, data)
with open(filename, 'rb') as fp:
data_compressed = filename.read()
without having to write to a file. Is there a way to convert numpy array to its equivalent PNG/JPG... representation without having to write to a file and read it as binary?
As Miki pointed out, imencode(...) is the solution.
I use image_transport pkg in ROS to transport numpy arrays (from OpenCV) to a listener written in python.
Now I want to convert a numpy array (from OpenCV) to Open3D Image data structure. How can I write this conversion? Is there any method that Open3D already have to finish this job?
I have a large collection of NumPy arrays saved on disk. I would like to read them efficiently and concurrently with the training. I can't load them all into memory at once - the data set is too large.
Additionally, it would be nice to apply some user defined transforms on the fly. Also it would be nice to be able to read them from C++, not just Python.
I believe CNTK does not have this capability now, am I correct?
Currently, we don't have build-in numpy reader. However, you have multiple options:
Read the numpy data in batches and feed them to the trainer, here an example that read images into numpy array and feed it to the trainer:
https://github.com/Microsoft/FERPlus
What the data inside your numpy array? Can you convert it to a format readable by one of the CNTK readers?
I want to save the result of TfidfVectorizer in sklearn.feature_extraction.text into a text file for future use. As I found, it is a sparse matrix of type ''. However when I try to save it using the following code
np.savetxt('Feature_TfIdf.txt', X_Tfidf, fmt='%2.6f')
I get an error like this
IndexError: tuple index out of range
Use joblib.dump or sklearn.externals.joblib.dump for this. NumPy doesn't get SciPy sparse matrices.
Simple example:
np.save('TfIdf.pkl',tfidf)
I manage to solve the problem by converting the sparse matrix to full matrix and then save matrix and save the results. This approach however is not useful for large arrays so it is better to save the matrix in .pkl format.
I have two sliced Numpy arrays stored in memory, say a and b, shape 1*480*640, how can I add them to an existing HDF5 file in PyTables? The existing HDF5 file has three arrays: n1,n2,n3, I want the new HDF5 file to have 5 arrays: n1,n2,n3,a,b . Thx!
You can use createArray on the file object to do that (or if you need compression use createCArray/createEArray):
File.createArray('/','a', numpyArray)