Plotting mean and standard dev values on skyplot using astropy from hdf5 file - matplotlib

I am trying to create a skyplot(using astropy) containing mean and standard dev values from a hdf5 file. The link to the data is https://wwwmpa.mpa-garching.mpg.de/~ensslin/research/data/faraday2020.html (Faraday Sky 2020).
I have programmed the following code until now, where data is read from the hdf5 file to ggl and ggb, after which values are converted to galactic coordinates (l and b) in gb and gl. I need these values to be plotted in the skyplot.
from astropy import units as u
from astropy.coordinates import SkyCoord
import matplotlib.pyplot as plt
import numpy as np
import h5py
dat = []
ggl=[]
ggb=[]
with h5py.File('faraday2020.hdf5','r') as hdf:
print(list(hdf.keys()))
faraday_sky_mean = hdf['faraday_sky_mean'][:]
faraday_sky_std = hdf['faraday_sky_std'][:]
print(faraday_sky_mean.shape, faraday_sky_mean.dtype)
print(f'Max Mean={max(faraday_sky_mean)}, Min Mean={min(faraday_sky_mean)}')
print(faraday_sky_std.shape, faraday_sky_std.dtype)
print(f'Max StdDev={max(faraday_sky_std)}, Min StdDev={min(faraday_sky_std)}')
ggl = faraday_sky_mean.tolist()
print(len(ggl),type(ggl[0]))
ggb = faraday_sky_std.tolist()
print(len(ggb),type(ggb[0]))
gl = ggl * u.degree
gb = ggb * u.degree
c = SkyCoord(l=gl,b=gb, frame='galactic', unit = (u.deg, u.deg)) #,
l_rad = c.l.wrap_at(180 * u.deg).radian
b_rad = c.b.radian
###
plt.figure(figsize=(8,4.2))
plt.subplot(111, projection="aitoff")
plt.title("Mean and standard dev", y=1.08, fontsize=20)
plt.grid(True)
P1=plt.plot(l_rad, b_rad,c="blue", s=220, marker="h", alpha=0.7) #,
plt.subplots_adjust(top=0.95, bottom=0.0)
plt.xlabel('l (deg)', fontsize=20)
plt.ylabel('b (deg)', fontsize=20)
plt.subplots_adjust(top=0.95, bottom=0.0)
plt.show()
However, I am getting the following error:
'got {}'.format(angles.to(u.degree)))
ValueError: Latitude angle(s) must be within -90 deg <= angle <= 90 deg, got [1.12490771 0.95323024 0.99124631 ... 4.23648627 4.28821608 5.14498169] deg
Please help me on how to fix this.

This is an extension to my previous answer. The original post wanted to plot Mean and Standard Deviation of the Faraday Sky 2020 data on an astropy skyplot. The referenced data source (from Radboud University) only included the mean and standard deviation. The associated longitude and latitude coordinates were obtained from the NASA Goddard LAMBDA-Tools site. The code below shows how to merge the data from both files into a single HDF5 file. For convenience, links to the data sources are repeated here:
Link to the Faraday Sky 2020 data
Link to the HEALPix Pixel Coordinates
The resulting file is named: "faraday2020_with_coords.h5".
from astropy.io import fits
import h5py
fits_file = 'pixel_coords_map_ring_galactic_res9.fits'
faraday_file = 'faraday2020.hdf5'
with fits.open(fits_file) as hdul, \
h5py.File(faraday_file,'r') as h5r, \
h5py.File('faraday2020_with_coords.h5','w') as h5w:
arr = hdul[1].data
dt = [('LONGITUDE', float), ('LATITUDE', float), \
('faraday_sky_mean', float), ('faraday_sky_std', float) ]
ds = h5w.create_dataset('skyplotdata', shape=(arr.shape[0],), dtype=dt)
ds['LONGITUDE'] = arr['LONGITUDE'][:]
ds['LATITUDE'] = arr['LATITUDE'][:]
ds['faraday_sky_mean'] = h5r['faraday_sky_mean'][:]
ds['faraday_sky_std'] = h5r['faraday_sky_std'][:]

I see why you are having problems plotting this data. The data in the linked file (faraday2020.hdf5) is only the mean and standard deviation of the reconstructed Faraday sky. See this note on the linked page: "All maps are presented in Galactic at a HEALPix resolution of Nside=512 and are stored in RING ordering scheme. The units are rad/m2." In other words, you need to get the skyplot coordinates from another source.
A little Googling found the coordinates on the NASA Goddard LAMBDA-Tools site: HEALPix Pixel Coordinates. Specifically, you want this file for NSide=512 / Galactic / Ring Pixel Ordering: pixel_coords_map_ring_galactic_res9.fits
So, first problem solved. Next you need to read the FITS formatted file to get the coordinates. Astropy has the 'fits' module to do that. See code below.
from astropy.io import fits
from astropy import units as u
from astropy.coordinates import SkyCoord
import matplotlib.pyplot as plt
import h5py
filename='pixel_coords_map_ring_galactic_res9.fits'
with fits.open(filename) as hdul:
print(hdul.info())
arr = hdul[1].data
print(arr.shape)
# Returns:
# (3145728,)
print(arr.dtype)
# Returns:
# dtype((numpy.record, [('LONGITUDE', '>f4'), ('LATITUDE', '>f4')]))
ggl = arr['LONGITUDE'][:].tolist()
ggb = arr['LATITUDE'][:].tolist()
gl = ggl * u.degree
gb = ggb * u.degree
c = SkyCoord(l=gl,b=gb, frame='galactic', unit = (u.deg, u.deg))
l_rad = c.l.wrap_at(180 * u.deg).radian
b_rad = c.b.radian
The code above gives you l_rad and b_rad for your skyplot coordinates. Next, you need to merge in the code I gave you earlier to read the Farady Sky Mean and StdDev.
with h5py.File('faraday2020.hdf5','r') as hdf:
faraday_sky_mean = hdf['faraday_sky_mean'][:]
faraday_sky_std = hdf['faraday_sky_std'][:]
Finally, plot both sets of data with matplotlib. I changed the plot to use a scatterplot, color coding the markers with c=faraday_sky_mean (the mean values). You can do the same with faraday_sky_stddev to get the standard deviation values.
plt.figure(figsize=(8,4.2))
plt.subplot(111, projection="aitoff")
plt.title("Mean", y=1.08, fontsize=20)
plt.grid(True)
# P1=plt.plot(l_rad, b_rad,c="blue", marker="h", alpha=0.7) #, s=220)
P2 = plt.scatter(l_rad, b_rad, s=20, c=faraday_sky_mean, cmap='hsv')
plt.subplots_adjust(top=0.95, bottom=0.0)
plt.xlabel('l (deg)', fontsize=20)
plt.ylabel('b (deg)', fontsize=20)
plt.subplots_adjust(top=0.95, bottom=0.0)
plt.show()
print('DONE')
Put it all together, and you will get the images below. I think this is accurate (but know nothing about astrophysics, so not 100% sure). This should get you pointed in the right direction. Good luck.

Related

Cartopy: coastlines() and contourf() interfering

I'm trying to migrate from Basemap to Cartopy looking demo examples. I have a simple code using both coastlines() and contourf(). I can get both separately but not simultaneously. The data set is a netcdf file containing the sea surface temperature data of the west Med. The code is:
import numpy as np
from netCDF4 import Dataset
import cartopy
import matplotlib.pyplot as plt
# DATA
data = Dataset('20190715.0504.n19.nc','r')
lon = data.variables['lon'][:]
lat = data.variables['lat'][:]
sst = data.variables['mcsst'][0,:,:].squeeze()
xxT,yyT = np.meshgrid(lon,lat)
# PLOT
fig = plt.figure(figsize=(10, 5))
ax1 = fig.add_axes([0.01,0.01,0.98,0.98],projection=cartopy.crs.Mercator())
ax1.coastlines()
#ax1.contourf(xxT,yyT,sst)
ax1.set_extent([16.5, -15.0, 35.0, 46.5])
plt.show()
With this code I get:
If I use:
#ax1.coastlines()
ax1.contourf(xxT,yyT,sst)
ax1.set_extent([16.5, -15.0, 35.0, 46.5])
I get a white rectangle.
If I use:
#ax1.coastlines()
ax1.contourf(xxT,yyT,sst)
ax1.set_extent([16.5,-15.0,35.0,46.5],crs=cartopy.crs.Mercator())
I get the contoured data.
But with both:
ax1.coastlines()
ax1.contourf(xxT,yyT,sst)
ax1.set_extent([16.5,-15.0,35.0,46.5],crs=cartopy.crs.Mercator())
the contour is ok ! but without coastlines. And if finally
ax1.coastlines()
ax1.contourf(xxT,yyT,sst)
ax1.set_extent([16.5,-15.0,35.0,46.5])
only coastlines are shown, not contour !. I try to understand how I have to proceed because problems arose when trying to include this into a GUI with options show/hide for coatlines, features, etc. Just in case I'm using Python 3.7.4, Cartopy 0.17, proj4 5.2, matplotlib 3.1.1. Thanks !
Thanks to swatchai suggestion, although, I still don't understand why I need to use the transform keyword with the specific PlateCarree projection keyword, the code works fine if:
fig = plt.figure(figsize=(10, 5))
ax1 = fig.add_axes([0.01, 0.01, 0.98, 0.98],projection=cartopy.crs.Mercator())
ax1.coastlines('10m')
ax1.set_extent([16.5, -15.0, 35.0, 46.5])
ax1.contourf(xxT,yyT,sst,transform=cartopy.crs.PlateCarree())
Here the result:

How to control the axis units in a map made with astropy and matplotlib?

When using astropy and matplotlib to create a map, the units in the right ascension axis are deg/min/sec, instead of h/m/s. I do not find an easy way in astropy to select the units h/m/s.
For example, if I try to reproduce the map of the Horsehead nebula as in the documentation of astropy.wcs, I get a R.A. axis in deg/min/sec.
The code is simply:
from matplotlib import pyplot as plt
from astropy.io import fits
from astropy.wcs import WCS
from astropy.utils.data import get_pkg_data_filename
filename = get_pkg_data_filename('tutorials/FITS-images/HorseHead.fits')
hdu = fits.open(filename)[0]
wcs = WCS(hdu.header)
fig = plt.figure()
fig.add_subplot(111, projection=wcs)
plt.imshow(hdu.data, origin='lower', cmap=plt.cm.viridis)
plt.xlabel('RA')
plt.ylabel('Dec')
plt.show()
It is supposed to produce this:
correct units
but I get that:
wrong units
You can use:
ax = fig.gca()
ra = ax.coords[0]
ra.set_format_unit('hour')
e.g. as specified here: http://docs.astropy.org/en/stable/visualization/wcsaxes/controlling_axes.html
However, when I ran the same example, it defaulted to hours, so I'm not sure what configuration you have set that defaulted to degrees instead.

Cartopy AzimuthalEquidistant projection: zooming into a region and coastlines

I am trying to plot some data on an AzimuthalEquidistant projection using cartopy. However, it gives me a couple of problems. First the coastlines no longer show for this type of projection. Not sure if this is my code or a Cartopy problem. I also notice that if I use a ccrs.PlateCarree() transform in the pcolormesh command the coastlines do show but then, presumably, my data is on the wrong type of prejection?
Second I would prefer if the axis boarder was circular after plotting the data, is it possible to use set_extent or some similar function to do this?
The code below should reproduce the problems, the circle shows how I would like the boarder to look.
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
import matplotlib.patches as mpatches
clat = 55.0
clon = -8.0
lons = np.arange(clon-15,clon+16,0.5)
lats = np.arange(clat-15,clat+16,0.5)
d = np.random.rand(lons.shape[0],lats.shape[0])
trans = ccrs.AzimuthalEquidistant(central_latitude=clat, central_longitude=clon)
ax = plt.axes(projection=trans)
ax.coastlines(resolution='10m')
CB=ax.pcolormesh(lons-0.25, lats-0.25, d.T,
cmap=plt.cm.viridis, alpha=0.5,
transform=trans)#ccrs.PlateCarree())
p1 = mpatches.Circle((clon,clat), radius=15, color='k', lw=5, fill=False,
transform=trans)
ax.add_patch(p1)
If the data you are plotting is in latitude/longitude coordinates then the correct value for the transform keyword is indeed ccrs.PlateCarree(). This is common gotcha for new users. The transform argument tells cartopy what coordinates your data are in, and is completely independent of the projection you want to plot onto.
To make the plot circular you'll need to set the boundary yourself. The Cartopy documentation have a couple of examples of this: http://scitools.org.uk/cartopy/docs/latest/examples/always_circular_stereo.html and http://scitools.org.uk/cartopy/docs/latest/examples/star_shaped_boundary.html.

How do I match the projection of my cartopy map with that of a shapefile?

I am trying to synthesise the projections of a coastlines() map with that of a shapefile, whose .prj file says:
GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",
SPHEROID["WGS_1984",6378137.0,298.257223563]],
PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]
My attempt is:
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
from cartopy.io import shapereader
# set up a map with coastlines around Auckland:
plt.figure(figsize=(10, 10))
platecarree = ccrs.PlateCarree(globe=ccrs.Globe(datum='WGS84'))
ax = plt.axes(projection=platecarree)
extent = [174.25, 175.25, -37.5, -36.5]
ax.set_extent(extent)
ax.coastlines('10m',color='red')
# read in shapefile and plot the polygons:
shp2 = shapereader.Reader('auckland_geology_wgs84gcs.shp')
formations = shp2.records()
for formation in formations:
# plot water blue, and all other rocks yellow
if formation.attributes['MAIN_ROCK'] == b' ':
ax.add_geometries(formation.geometry, ccrs.PlateCarree(),facecolor='blue',alpha=.1)
else:
ax.add_geometries(formation.geometry, ccrs.PlateCarree(), facecolor='yellow',alpha=.1)
plt.show()
I tried giving the globe parameter in my platecarree definition the radius and inverse flattening from the prj file, but I didn't see any change to the output if I set or even varied those numbers.
In addition, with the defined "platecarree" projection (with the call to the globe with WGS84) as the crs in the add_geometries calls, my output is blank.
As is, the result looks to me like a projection mismatch
I've tried to reproduce your problem using QGIS and data downloaded from Natural Earth (10m coastlines) and from GADM (NZ adm0 level). It looks like the NE10m coastlines are the culprit ! The GADM aligns perfectly with your geology layer, while the NE10m is off (and deformed). screenshot of QGIS with Geological map & coastlines

How to change pyplot.specgram x and y axis scaling?

I have never worked with audio signals before and little do I know about signal processing. Nevertheless, I need to represent and audio signal using pyplot.specgram function from matplotlib library. Here is how I do it.
import matplotlib.pyplot as plt
import scipy.io.wavfile as wavfile
rate, frames = wavfile.read("song.wav")
plt.specgram(frames)
The result I am getting is this nice spectrogram below:
When I look at x-axis and y-axis which I suppose are frequency and time domains I can't get my head around the fact that frequency is scaled from 0 to 1.0 and time from 0 to 80k.
What is the intuition behind it and, what's more important, how to represent it in a human friendly format such that frequency is 0 to 100k and time is in sec?
As others have pointed out, you need to specify the sample rate, else you get a normalised frequency (between 0 and 1) and sample index (0 to 80k). Fortunately this is as simple as:
plt.specgram(frames, Fs=rate)
To expand on Nukolas answer and combining my Changing plot scale by a factor in matplotlib
and
matplotlib intelligent axis labels for timedelta
we can not only get kHz on the frequency axis, but also minutes and seconds on the time axis.
import matplotlib.pyplot as plt
import scipy.io.wavfile as wavfile
cmap = plt.get_cmap('viridis') # this may fail on older versions of matplotlib
vmin = -40 # hide anything below -40 dB
cmap.set_under(color='k', alpha=None)
rate, frames = wavfile.read("song.wav")
fig, ax = plt.subplots()
pxx, freq, t, cax = ax.specgram(frames[:, 0], # first channel
Fs=rate, # to get frequency axis in Hz
cmap=cmap, vmin=vmin)
cbar = fig.colorbar(cax)
cbar.set_label('Intensity dB')
ax.axis("tight")
# Prettify
import matplotlib
import datetime
ax.set_xlabel('time h:mm:ss')
ax.set_ylabel('frequency kHz')
scale = 1e3 # KHz
ticks = matplotlib.ticker.FuncFormatter(lambda x, pos: '{0:g}'.format(x/scale))
ax.yaxis.set_major_formatter(ticks)
def timeTicks(x, pos):
d = datetime.timedelta(seconds=x)
return str(d)
formatter = matplotlib.ticker.FuncFormatter(timeTicks)
ax.xaxis.set_major_formatter(formatter)
plt.show()
Result:
Firstly, a spectrogram is a representation of the spectral content of a signal as a function of time - this is a frequency-domain representation of the time-domain waveform (e.g. a sine wave, your file "song.wav" or some other arbitrary wave - that is, amplitude as a function of time).
The frequency values (y-axis, Hertz) are wholly dependant on the sampling frequency of your waveform ("song.wav") and will range from "0" to "sampling frequency / 2", with the upper limit being the "nyquist frequency" or "folding frequency" (https://en.wikipedia.org/wiki/Aliasing#Folding). The matplotlib specgram function will automatically determine the sampling frequency of the input waveform if it is not otherwise specified, which is defined as 1 / dt, with dt being the time interval between discrete samples of the waveform. You can can pass the option Fs='sampling rate' to the specgram function to manually define what it is. It will be easier for you to get your head around what is going on if you figure out and pass these variables to the specgram function yourself
The time values (x-axis, seconds) are purely dependent on the length of your "song.wav". You may notice some whitespace or padding if you use a large window length to calculate each spectra slice (think- the individual spectra which are arranged vertically and tiled horizontally to create the spectrogram image)
To make the axes more intuitive in the plot, use x- and y-axes labels and you can also scale the axes values (i.e. change the units) using a method similar to this
Take home message - try to be a bit more verbose with your code: see below for my example.
import matplotlib.pyplot as plt
import numpy as np
# generate a 5Hz sine wave
fs = 50
t = np.arange(0, 5, 1.0/fs)
f0 = 5
phi = np.pi/2
A = 1
x = A * np.sin(2 * np.pi * f0 * t +phi)
nfft = 25
# plot x-t, time-domain, i.e. source waveform
plt.subplot(211)
plt.plot(t, x)
plt.xlabel('time')
plt.ylabel('amplitude')
# plot power(f)-t, frequency-domain, i.e. spectrogram
plt.subplot(212)
# call specgram function, setting Fs (sampling frequency)
# and nfft (number of waveform samples, defining a time window,
# for which to compute the spectra)
plt.specgram(x, Fs=fs, NFFT=nfft, noverlap=5, detrend='mean', mode='psd')
plt.xlabel('time')
plt.ylabel('frequency')
plt.show()
5Hz_spectrogram: