I have the following code:
hist house1 if house1 >0 & house1 <200000, bin(25) fraction by(Year)
graph export house1.png, replace
I would like to iterate it substituting house1 with car1 and bed1 without copy-pasting the code and substituting, or at least something like:
var = "house1"
hist var if house1 >0 & house1 <200000, bin(25) fraction by(Year)
graph export var.png, replace
So that I can change just the value assigned to var.
A simple foreach loop will work:
foreach x in house1 car1 bed1 {
display "hist `x' if `x' >0 & `x' <200000, bin(25) fraction by(Year)"
display "graph export `x'.png, replace"
}
hist house1 if house1 >0 & house1 <200000, bin(25)fraction by(Year)
graph export house1.png, replace
hist car1 if car1 >0 & car1 <200000, bin(25)fraction by(Year)
graph export car1.png, replace
hist bed1 if bed1 >0 & bed1 <200000, bin(25)fraction by(Year)
graph export bed1.png, replace
Here x is a local macro that gets the values specified in foreach.
Note that the display command is used for illustration and is not necessary.
Related
I wrote a script to convert the EELS map to EELS line scan data, and it works well with DM 2.0. I can deal with it as directly collected EELS line scan data with DM2.0. But it does not work with DM 3.0 and the above version. It looks DM 3.0 still recognizes it as an EELS map file. DM3.0 still tried to generate elemental maps with multiple windows from it not generate line scan profiles with one single window and said the display type is incorrect. Not sure what code/command I need to add to fit the DM 3.0 and above versions. Appreciate any suggestions/comments.
image source
source := getFrontImage()
number sizeX,sizeY,sizeZ
source.Get3Dsize(sizeX,sizeY,sizeZ)
Result( "Original size:"+ sizeX +"; "+ sizeY+"; "+sizeZ+""+"\n" )
image sum
number regionsizeX = 1
number regionsizeY = sizeY
number row,col
Result( "new size:"+ regionsizeX +"; "+ regionsizeY+"; "+row+""+row+" "+"\n" )
sum := RealImage("Line Scan of [0,0,"+regionSizeY+","+regionSizeX+"]",4,sizeX/regionSizeX,sizey/regionsizeY,sizeZ)
//sum := ImageClone(source)
sum = 0
for (row=0;row<regionsizeY;row++) for (col=0;col<regionSizeX;col++)
{
OpenAndSetProgressWindow("Doing sub-block","x = "+col," y = "+row)
sum += Slice3(source,col,row,0,0,sizeX/regionSizeX,regionsizeX,1,sizeY/regionSizeY,regionSizeY,2,sizez,1)
}
OpenAndSetProgressWindow("","","")
ImageCopyCalibrationFrom(sum, source)
sum.setdisplaytype (1)
sum.SetStringNote( "Meta Data:Format", "Spectrum image" )
sum.SetStringNote( "Meta Data:Signal", "EELS" )
showimage(sum)
I'm also a bit confused by your terminology. When you write "Convert a Map into a LineScan" do you mean:
a) Convert a 3D Spectrum-Image (xy scan, one spectral dimension) into a 2D Line-Scan Spectrum-Image (one spatial dimension, one spectral dimension)
or
b) Convert a 2D Map (xy scan, one value) in a 1D Line-Trace (one spatial dimension, one value per point) ?
I suppose you mean a) and answer to that.
I'm surprised if/that your script would work without issues in GMS 2.
Your final (supposedly line-scan SI) data is still a 3D dataset with the dispersion running in Z-direction. This is not the typical LineScan SI data format (which is dispersion in X, spatial dimension in Y, no Z dimension).
Am I right in thinking that you want to "collapse" your 3D data along the y-dimension (by summing) ?
If so, what you want to do is:
// Get Input
image src3D := GetFrontImage()
number sizeX,sizeY,sizeZ
if ( 3 != src3D.ImageGetNumDimensions() ) Throw( "Input not 3D")
src3D.Get3Dsize(sizeX,sizeY,sizeZ)
// Optional: Use Rect-ROI on image to specify area
// If no selection, will return full FOV
number t,l,b,r
src3D.GetSelection(t,l,b,r)
// Prepare output (for summing 3D of rect-selection along y)
// NB: 2D container has:
// X dimension (spatial) along Y
// Z dimension (energy) along X
number nSpatial = r - l
number nSpectral = sizeZ
number eOrig, eScale, sOrig, sScale
string eUnit, sUnit
src3D.ImageGetDimensionCalibration(0, sOrig, sScale, sUnit, 0)
src3D.ImageGetDimensionCalibration(2, eOrig, eScale, eUnit, 0)
string name
if ( nSpatial != sizeX )
name = "Y-projection of [" + t + "," + l + "," + b + "," + r + "] over " + (b-t) + " rows"
else
name = "Y-projection over " + sizeY + " rows"
image dst2D := RealImage( name, 4, nSpectral, nSpatial )
dst2D.ImageSetDimensionCalibration(0, eOrig, eScale, eUnit, 0)
dst2D.ImageSetDimensionCalibration(1, sOrig, sScale, sUnit, 0)
// Copy Tags (contains necessary meta tags! Meta Data Format & Signal)
dst2D.ImageGetTagGroup().TagGroupCopyTagsFrom( src3D.ImageGetTagGroup() )
// Display (with captions)
dst2D.ShowImage()
dst2D.ImageGetImageDisplay(0).ImageDisplaySetCaptionOn(1)
number doFAST = 0
if ( !doFAST )
{
// Perform actuall summing (projection) by summing "line by line"
// into the LinePlot SI. Note the flipping of input and output dimensions!
for( number y = t; y<b; y++ )
{
number lineNumber = y - t
dst2D.slice2( 0,0,0, 0,nSpectral,1, 1,nSpatial,1 ) += src3D.slice2( l,y,0, 2,nSpectral,1, 0,nSpatial,1)
}
}
else
{
// Alternative (faster) projection. Use dedicated projection command.
image proj := src3D[l,t,0,r,b,nSpectral].Project(1) // Outcome of projectsion is having x=x and y=z, so need flip axis
dst2D = proj.slice2(0,0,0, 1,nSpectral,1, 0,nSpatial,1 ) // Flip axis
}
// Display (with captions)
dst2D.ShowImage()
dst2D.ImageGetImageDisplay(0).ImageDisplaySetCaptionOn(1)
Note that iterating using slice blocks is fast, but not as fast as the dedicated 'Project' command available in latest GMS versions. The example uses either, but lines #51-56 might not be available in older GMS.
Edit to address comment below:
Other relevant meta data for spectra is also found in the tags. For EELS, in particular the collection & convergence angle as well as the HT is of importance. You can find out about the tag-path by checking the tags of a properly acquired EELS spectrum.
Or, you can find out about their tag-paths by "converting" an empty 1D line-plot into an EELS spectrum and then attempting a quantification. You will get the prompt to fill in the data. After doing so, check the tags of the image:
i'm trying to make the object parallel to the z - axis by using bpy.ops.transform.rotate(value=90.0, axis=(1,0,0)) but all i got is this
enter image description here
import bpy
ungu = bpy.data.materials.new('Ungu')
ungu.diffuse_color=(0.6,0.1,0.3)
for i in range (5) :
x = i*2
y = 0
z = 0
bpy.ops.mesh.primitive_plane_add(location=(x,y,z))
ob=bpy.context.object
ob.name='PLANE'
mymesh=ob.data
ob.scale=((0.5,3,2))
#aplikasikan warna ungu ke objek mesh
mymesh.materials.append(ungu)
bpy.ops.transform.rotate(value=90.0, axis=(1,0,0))
so what number should i put in value parameter?
the value argument should be in radians but you are using degrees. I am not sure which version of Blender you are using, but acording to docs for Blender 2.82a https://docs.blender.org/api/current/bpy.ops.transform.html the transform function should be called like this:
bpy.ops.transform.rotate(value=3.14/2, orient_axis='X')
Try to find the code with function to set/change the slice label for a line plot in DM script, but could not find it.
I mean the codes could change the label "slice 1" to "ages" in a line plot with DM script.
The command you seek is ImageDisplaySetSliceLabelByID and the example how to use it is:
image img := RealImage("Slice Demo",4,100,3)
img.ShowImage()
img = icol*(irow+random())
imageDisplay disp = img.ImageGetImageDisplay(0)
disp.ImageDisplayChangeDisplayType(3)
disp.LinePlotImageDisplaySetLegendShown(1)
OKDialog("Now rename slices")
number nSlices = disp.LinePlotImageDisplayCountSlices()
for( number i=0; i<nSlices; i++ )
{
object id = disp.ImageDisplayGetSliceIDByIndex( i )
disp.ImageDisplaySetSliceLabelByID( id, "entry #"+(i+1) )
}
The examples section on Lineplot-displays in the F1 help is generally a good starting point for these things:
I'm using word2vec to represent a small phrase (3 to 4 words) as a unique vector, either by adding each individual word embedding or by calculating the average of word embeddings.
From the experiments I've done I always get the same cosine similarity. I suspect it has to do with the word vectors generated by word2vec being normed to unit length (Euclidean norm) after training? or either I have a BUG in the code, or I'm missing something.
Here is the code:
import numpy as np
from nltk import PunktWordTokenizer
from gensim.models import Word2Vec
from numpy.linalg import norm
from scipy.spatial.distance import cosine
def pattern2vector(tokens, word2vec, AVG=False):
pattern_vector = np.zeros(word2vec.layer1_size)
n_words = 0
if len(tokens) > 1:
for t in tokens:
try:
vector = word2vec[t.strip()]
pattern_vector = np.add(pattern_vector,vector)
n_words += 1
except KeyError, e:
continue
if AVG is True:
pattern_vector = np.divide(pattern_vector,n_words)
elif len(tokens) == 1:
try:
pattern_vector = word2vec[tokens[0].strip()]
except KeyError:
pass
return pattern_vector
def main():
print "Loading word2vec model ...\n"
word2vecmodelpath = "/data/word2vec/vectors_200.bin"
word2vec = Word2Vec.load_word2vec_format(word2vecmodelpath, binary=True)
pattern_1 = 'founder and ceo'
pattern_2 = 'co-founder and former chairman'
tokens_1 = PunktWordTokenizer().tokenize(pattern_1)
tokens_2 = PunktWordTokenizer().tokenize(pattern_2)
print "vec1", tokens_1
print "vec2", tokens_2
p1 = pattern2vector(tokens_1, word2vec, False)
p2 = pattern2vector(tokens_2, word2vec, False)
print "\nSUM"
print "dot(vec1,vec2)", np.dot(p1,p2)
print "norm(p1)", norm(p1)
print "norm(p2)", norm(p2)
print "dot((norm)vec1,norm(vec2))", np.dot(norm(p1),norm(p2))
print "cosine(vec1,vec2)", np.divide(np.dot(p1,p2),np.dot(norm(p1),norm(p2)))
print "\n"
print "AVG"
p1 = pattern2vector(tokens_1, word2vec, True)
p2 = pattern2vector(tokens_2, word2vec, True)
print "dot(vec1,vec2)", np.dot(p1,p2)
print "norm(p1)", norm(p1)
print "norm(p2)", norm(p2)
print "dot(norm(vec1),norm(vec2))", np.dot(norm(p1),norm(p2))
print "cosine(vec1,vec2)", np.divide(np.dot(p1,p2),np.dot(norm(p1),norm(p2)))
if __name__ == "__main__":
main()
and here is the output:
Loading word2vec model ...
Dimensions 200
vec1 ['founder', 'and', 'ceo']
vec2 ['co-founder', 'and', 'former', 'chairman']
SUM
dot(vec1,vec2) 5.4008677771
norm(p1) 2.19382594282
norm(p2) 2.87226958166
dot((norm)vec1,norm(vec2)) 6.30125952303
cosine(vec1,vec2) 0.857109242583
AVG
dot(vec1,vec2) 0.450072314758
norm(p1) 0.731275314273
norm(p2) 0.718067395416
dot(norm(vec1),norm(vec2)) 0.525104960252
cosine(vec1,vec2) 0.857109242583
I'm using the cosine similarity as defined here Cosine Similarity (Wikipedia). The values for the norms and dot products are indeed different.
Can anyone explain why the cosine is the same?
Thank you,
David
Cosine measures the angle between two vectors and does not take the length of either vector into account. When you divide by the length of the phrase, you are just shortening the vector, not changing its angular position. So your results look correct to me.
Can anyone help me with parameters for SetGeoTransform? I'm creating raster layers with GDAL, but I can't find description of 3rd and 5th parameter for SetGeoTransform. It should be definition of x and y axis for cells. I try to find something about it here and here, but nothing.
I need to find description of these two parameters... It's a value in degrees, radians, meters? Or something else?
The geotransform is used to convert from map to pixel coordinates and back using an affine transformation. The 3rd and 5th parameter are used (together with the 2nd and 4th) to define the rotation if your image doesn't have 'north up'.
But most images are north up, and then both the 3rd and 5th parameter are zero.
The affine transform consists of six coefficients returned by
GDALDataset::GetGeoTransform() which map pixel/line coordinates into
georeferenced space using the following relationship:
Xgeo = GT(0) + Xpixel*GT(1) + Yline*GT(2)
Ygeo = GT(3) + Xpixel*GT(4) + Yline*GT(5)
See the section on affine geotransform at:
https://gdal.org/tutorials/geotransforms_tut.html
I did do like below code.
As a result I was able to do same with SetGeoTransform.
# new file
dst = gdal.GetDriverByName('GTiff').Create(OUT_PATH, xsize, ysize, band_num, dtype)
# old file
ds = gdal.Open(fpath)
wkt = ds.GetProjection()
gcps = ds.GetGCPs()
dst.SetGCPs(gcps, wkt)
...
dst.FlushCache()
dst = Nonet
Given information from the aforementioned gdal datamodel docs, the 3rd & 5th parameters of SatGeoTransform (x_skew and y_skew respectively) can be calculated from two control points (p1, p2) with known x and y in both "geo" and "pixel" coordinate spaces. p1 should be above-left of p2 in pixelspace.
x_skew = sqrt((p1.geox-p2.geox)**2 + (p1.geoy-p2.geoy)**2) / (p1.pixely - p2.pixely)`
y_skew = sqrt((p1.geox-p2.geox)**2 + (p1.geoy-p2.geoy)**2) / (p1.pixelx - p2.pixelx)`
In short this is the ratio of Euclidean distance between the points in geospace to the height (or width) of the image in pixelspace.
The units of the parameters are "geo"length/"pixel"length.
Here is a demonstration using the corners of the image stored as control points (gcps):
import gdal
from math import sqrt
ds = gdal.Open(fpath)
gcps = ds.GetGCPs()
assert gcps[0].Id == 'UpperLeft'
p1 = gcps[0]
assert gcps[2].Id == 'LowerRight'
p2 = gcps[2]
y_skew = (
sqrt((p1.GCPX-p2.GCPX)**2 + (p1.GCPY-p2.GCPY)**2) /
(p1.GCPPixel - p2.GCPPixel)
)
x_skew = (
sqrt((p1.GCPX-p2.GCPX)**2 + (p1.GCPY-p2.GCPY)**2) /
(p1.GCPLine - p2.GCPLine)
)
x_res = (p2.GCPX - p1.GCPX) / ds.RasterXSize
y_res = (p2.GCPY - p1.GCPY) / ds.RasterYSize
ds.SetGeoTransform([
p1.GCPX,
x_res,
x_skew,
p1.GCPY,
y_skew,
y_res,
])