I tried to convert my text file into NetCDF (nc) file with the help of the youtube link I shared here. I cannot open this nc file in GrADS. I guess the reason is that I cannot add metadata or something into nc file with these lines of codes.
I would therefore like to improve the code in my hand so that I can open it up in other platforms. I need to open this NetCDF file in RCMES so I can carry out quantile mapping bias correction operations.
I am also open to suggestion for other ways/programming languages/platforms to perform this conversion task.
Below is the code I used.
import netCDF4 as nc
import numpy as np
import panda
import numpy as np
import pandas as pd
import xarray
# here csv file is converted into pandas dataframe
df = pd.read_csv('C:/Users/Asus/Documents/ArcGIS/ArcGIS Copy/evaporation/Downscaling Files/netcdfye dönecek csvler/Aydin_cnrm_Prec_rcp451.txt')
df
#converting pandas dataframe into xarray
xr = df.to_xarray()
xr
#lastly from xarray to nc file conversion
xr.to_nc('Aydin_cnrm_Prec_rcp451.nc')
Instead of using Python for creating a Netcdf file from a ASCII/txt file I tried using cdo that I installed on Ubuntu.
The following lines of code solved the problem
cdo -f nc input,r1x1 tmp.nc < Aydin_cnrm_Prec_rcp45.txt
cdo -r -chname,var1,prec -settaxis,1980-01-01,00:00:00,1mon tmp.nc prec.nc
Related
I'm trying to generate graphs for all the columns from a excel file but I'm having one error. My goal is getting this graphs in png files and then download it. Let me give you some context: I'm reading a csv, for each column, I'm trying to use a for to use .value_counts() and then create a graph once the graph is generated saving this one in a png file with the number of the index my code is this one:
import pandas as pd
from google.colab import files
from matplotlib import pyplot as plt
df = pd.read_excel('columns.xlsx')
for i in df.columns:
print(i)
#i.value_counts().plot(kind="bar", figsize=(15,7), color="#61d199")
df[i].value_counts().plot(kind="bar", figsize=(15,7), color="#61d199")
#plt.savefig('viz_movies.png')
Error in this code line:
df[i].value_counts().plot(kind="bar", figsize=(15,7), color="#61d199")
Error:
index 0 is out of bounds for axis 0 with size 0
Also I want to add in the for the names of the files something like this:
for index, number in enumerate(numbers):
plt.savefig('index.png') #use the index as a name
This is the image of the error on my Colab when I used pd.dtye to pd_data
This is the image of the error on my Colab when I used np.dtye to pd_data
I have loaded one csv file to my Colab note by two diffrent way. By pd.read_csv() and np.loadtxt(). And I have assigned these two in nd_data and pd_data ,repectively. After that I printed the shape of each data. At this point I've got two diffrent shape even though I loaded the same csv file.
My question is why I've got two diffrent shape by loading the same data.
this is the link to ThoraricSurgery.csv file which I've used.
'''
from google.colab import drive
drive.mount('/content/drive')
import pandas as pd
pd_data = pd.read_csv('/content/drive/MyDrive/딥러닝과실습1/ThoraricSurgery.csv')
print(pd_data.shape)
print(type(pd_data))
import numpy as np
nd_data = np.loadtxt('/content/drive/MyDrive/딥러닝과실습1/ThoraricSurgery.csv', delimiter=",")
print(nd_data.shape)
print(type(nd_data))this is the mentioned result
'''
I'm hoping to get some help here. I'm trying to create some simple bar/line graphs from a csv file, however, it gives me an empty graph until I open this csv file manually in excel and change the data type to numeric. I've tried changing the data type with pd.to_numeric but it still gives an empty graph.
The csv that I'm trying to visualise is web data that I scraped using Beautiful Soup, I used .text method do get rid of all of the HTML tags so maybe it's causing the issue?
Would really appreciate some help. thanks!
Data file: https://dropmefiles.com/AYTUT
import numpy
import matplotlib
from matplotlib import pyplot as plt
import pandas as pd
import csv
my_data = pd.read_csv('my_data.csv')
my_data_n = my_data.apply(pd.to_numeric)
plt.bar(x=my_data_n['Company'], height=my_data_n['Market_Cap'])
plt.show()
Your csv file is corrupt. There are commas at the end of each line. Remove them and your code should work. pd.to_numeric is not required for this sample dataset.
Test code:
from matplotlib import pyplot as plt
import pandas as pd
my_data = pd.read_csv('/mnt/ramdisk/my_data.csv')
fig = plt.bar(x=my_data['Company'], height=my_data['Market_Cap'])
plt.tick_params(axis='x', rotation=90)
plt.title("Title")
plt.tight_layout()
plt.show()
I want python to take ANY .xls file from given location and save it as .xlsx with original file name? How I can do that so anytime I paste file to location it will be converted to xlsx with original file name?
import pandas as pd
import os
for filename in os.listdir('./'):
if filename.endswith('.xls'):
df = pd.read_excel(filename)
df.to_excel(??)
Your code seems to be perfectly fine. In case you are only missing the correct way to write it with the given name, here you go.
import pandas as pd
import os
for filename in os.listdir('./'):
if filename.endswith('.xls'):
df = pd.read_excel(filename)
df.to_excel(f"{os.path.splitext(filename)[0]}.xlsx")
A possible extension to convert any file that gets pasted inside the folder can be implemented with an infinite loop, for instance:
import pandas as pd
import os
import time
while True:
files = os.listdir('./')
for filename in files:
out_name = f"{os.path.splitext(filename)[0]}.xlsx"
if filename.endswith('.xls') and out_name not in files:
df = pd.read_excel(filename)
df.to_excel(out_name)
time.sleep(10)
Using Python can I open a text file, read it into an array, then save the file as a NetCDF?
The following script I wrote was not successful.
import os
import pandas as pd
import numpy as np
import PIL.Image as im
path = 'C:\path\to\data'
grb = [[]]
for fn in os.listdir(path):
file = os.path.join(path,fn)
if os.path.isfile(file):
df = pd.read_table(file,skiprows=6)
grb.append(df)
df2 = pd.np.array(grb)
#imarray = im.fromarray(df2) ##cannot handle this data type
#imarray.save('Save_Array_as_TIFF.tif')
i once used xray or xarray (they renamed them selfs) to get a NetCDF file into an ascii dataframe... i just googled and appearantly they have a to_netcdf function
import xarray and it allows you to treat dataframes just like pandas.
so give this a try:
df.to_netcdf(file_path)
xarray slow to save netCDF