Suppose there exists a numpy array, data. I am trying to do the equivalent of the following
cv2.imwrite(filename, data)
with open(filename, 'rb') as fp:
data_compressed = filename.read()
without having to write to a file. Is there a way to convert numpy array to its equivalent PNG/JPG... representation without having to write to a file and read it as binary?
As Miki pointed out, imencode(...) is the solution.
Related
I try to implement this code, using colab:
x = np.load('data_sample.npy',allow_pickle=True)
stacked_x = np.concatenate([x,x,x],1)
stacked_x.shape
but my sample was an image in jpg format, so I wasn't able to convert these images to .npy to use it.
actually, I tried this code:
import numpy as np
array = np.asarray('image.JPG')
print(array.tobytes())
print(array)
x = np.load(array,allow_pickle=True)
stacked_x = np.concatenate([x,x,x],1)
stacked_x.shape
print(x)
but I got:
and when I use this code :
x = np.load(array.tobytes(),allow_pickle=True)
I got:
so, any suggestion to solve this, precisely to convert .jpg to .npy?
Look at what your code does:
In [57]: np.asarray('image.JPG')
Out[57]: array('image.JPG', dtype='<U9')
In [58]: np.asarray('image.JPG').tobytes()
Out[58]: b'i\x00\x00\x00m\x00\x00\x00a\x00\x00\x00g\x00\x00\x00e\x00\x00\x00.\x00\x00\x00J\x00\x00\x00P\x00\x00\x00G\x00\x00\x00'
That's just playing with the string. It's not doing anything with a file named "image.JPG". You need to use some sort of image processing package to first load the file, converting it from the compressed format to a 3d array.
np.load is used to load a file that was created by np.save('file.npy'). It doesn't make sense to give that array variable as the file name. It won't load a jpg file.
np.load('file.npy',allow_pickle=True)
I'm trying to use Python to parse mxnet params into plain text. The code looks like the below. But the parsing result is not plain string, but some encoded text looks like this, "... \xaa>\x0f\xed\x8e>\xaf!\x8f>g ..." Could anybody give me some tips on it? Thanks a lot!
...
param_file = 'resnet-50-0000.params'
with open(param_file, 'rb') as f:
net_params = f.read()
...
The parameters are binary files. If you want to read them as plain text you need to decode them first as a dictionary of parameter_name->NDArray, that you can convert them to numpy. From numpy you can convert it to a list and then process it as a list (of lists) of scalar.
import mxnet as mx
params = mx.nd.load('resnet-50-0000.params')
for k, param in params.items():
print(k)
print(param.asnumpy().tolist())
Well, it seems like a couple of similar questions were asked here in stack overflow, but none of them seem like answered correctly or properly, nor they described the exact examples.
I have a problem with saving array or list into hdf5 ...
I have a several files contains list of (n, 35) dimensions, where n may be different in each file. Each of them can be saved in hdf5 with code below.
hdf = hf.create_dataset(fname, data=d)
However, if I want to merge them to make in 3d the error occurs as below.
Object dtype dtype('O') has no native HDF5 equivalent
I have no idea why it turns to dtype object, since what I have done is only this
all_data = list()
for fname in file_list:
d = np.load(fname)
all_data.append(d)
hdf = hf.create_dataset('all_data', data=all_data)
How can I save such data?
I tried a couple of tests, and it seems like all_data turns to dtype with 'object' when I change them with
all_data = np.array(all_data)
Which looks it has the similar problem with saving hdf5.
Again, how can I save such data in hdf5?
I was running into a similar issue with h5py, and changing the type of the NumPy array using array.astype worked for me (I believe this changes the type from dtype('O') to the data type you specify). Please see the code snippet below:
import numpy as np
print(X.dtype)
--> dtype('O')
print(X.astype(np.float64).dtype)
--> dtype('float64')
When I ran h5.create_dataset with this data type conversion, I was able to successfully create a h5 dataset. Hope this helps!
ONE ADDITIONAL UPDATE: I believe the NumPy object type 'O' is created when the NumPy array itself has mixed element types (e.g. np.int8 and np.float32).
dtype('O') stands for object. In my case I had a list of lists where the lengths were different and got the same error. If you convert it to a numpy array numpy warns Creating an ndarray from ragged nested sequences. h5 files can't handle this type of data for more info see this post
This error comes when I use:
with h5py.File(peakfilename, 'w') as pfile: # saves the data
pfile['peakY'] = np.array(X)
pfile['peakX'] = np.array(Y)
However when I used dtype when saving the arrays... the problem went away... I guess h5py is not able to create datasets from undefined data types.
with h5py.File(peakfilename, 'w') as pfile: # saves the data
pfile['peakY'] = np.array(X, dtype=np.float32)
pfile['peakX'] = np.array(Y, dtype=np.float32)
I have a data in mat file (observations and features) and i want to load it into numpy 2D array. I dont want to convert it into csv first and then load csv into numpy.
Use scipy's loadmat (API-docs).
The docs should be sufficient to get you going, but make sure to read the notes.
There is also the io-tutorial with some examples.
my question is related to this but I can't get that solution to work and didn't want to add my own scenario to the old question.
I have a 2D float numpy array, am running python 3.5.1 with numpy 1.10.4, and am trying to write out the array with
numpy.savetext(filename, arrayname, delimiter = ',')
which works beautifully with a 1D array.
I've tried the solution from the referenced post
with open(filename, 'ab') as f:
numpy.savetext(f, arrayname, delimiter = ',')
to no avail. Actually, I've tried this without the delimiter as well as with 'w', 'wb, 'a' and with formatting arguments, and always get the same error message:
TypeError: Mismatch between array dtype ('float64') and format specifier.
I need to write this 2D array to a file which will be read later into a panda dataframe (have been using read.csv). I understand this may be an issue with numpy.savetxt, so I'm looking for an alternative.
Please try a minimal example and post the result, since the following works for me:
import numpy as np
array1=np.array([[1,2],[3,4]])
np.savetxt('file1.txt', array1 , delimiter = ',')
file content:
1.000000000000000000e+00,2.000000000000000000e+00
3.000000000000000000e+00,4.000000000000000000e+00
I had the same error message - until I finally realized that the type of my output actually was a list, not a numpy array!