I am attempting to use AsterixDB (which uses SQL++) to join two sets together via an sql query. In one dataset, I have a series of points in the form of latitude and longitude. The other dataset is geometries for zip codes. I am trying to append the relevant zip code to the first dataset based on whether the point exists in the zip code or not.
The query is below as well as the schema for each dataset
use csv;
select sett.lat, sett.long, zip.g
from csv_set as sett
left join csv_zipset as zip
on st_contains(zip.g, st_make_point(sett.lat, sett.long));
create type csv_type as {
id:uuid,
...
lat: double,
long: double
};
create type csv_ziptype as {
id: uuid,
g:geometry
};
This is the error I am facing:
ERROR: Code: 1 "java.lang.NullPointerException: Cannot invoke \"org.apache.hyracks.control.nc.io.FileHandle.close()\" because \"fHandle\" is null"
I have tried adding null checks for both the point and geometry with no luck.
I have also validated that st_make_point is working properly, and st_contains works when I pass it a fixed geometry which leads me to believe that this is an issue with the geometry.
Any help is much appreciated
After exhausting more options I came to the realization that there were multiple types in my dataset: polygon, multipolygon, linestring, and geometryCollection. It seems that AsterixDB doesn't yet have the ability to compute st_contains with geometry collections. Once I removed those entries from the dataset the query was able to complete successfully.
I have an Excel file and there are two columns in it, I want to combine them, but one of them is in datetime form and the other is object (actually time). What I want to do is convert the object one to datetime format.enter image description here
I've tried everything I can think of but I keep getting an error.
Edit :enter image description here
import pandas as pd
dataFrame = pd.read_excel('/content/drive/MyDrive/Colab Notebooks/data.xlsx')
dataFrame.head()
output:
enter image description here
and my error
enter image description here
If I'm understanding? You'd want to split "Time" column on space and take 0 index. Finally use .cat to concatenate the string columns together. Next .pop old columns and finally wrap it all in to_datetime.
df["Time"] = df["Time"].str.split(r"\s+").str[0]
df["Datetime"] = pd.to_datetime(df.pop("Date").astype(str).str.cat(df.pop("Time"), sep=" "))
I'd like to concat or merge or append/join two csv files with the same indix ID but different extensions on the same ID. The data are grouped by ID also. The 1st file looks like this:
ID,year,age
810006862,2000,49
810006862,2001,
810006862,2002,
810006862,2003,52
810023112,2003,27
810023112,2004,28
810023112,2005,29
810023112,2006,30
810033622,2000,24
810033622,2001,25
and the 2nd file looks like this:
ID,year,from1,to1
810006862,2002,15341,15705
810006862,2003,15706,16070
810006862,2004,16071,16436
810006862,2005,,
810023112,2000,14610,14975
810023112,2001,14976,15340
810023112,2003,15825,16523
810033622,2000,13211,14876
810033622,2001,14761,14987
I have set index of ID for both files after reading it to dataframe, and then concat them together, but it gets error message of "ValueError: Shape of passed values is (25, 2914), indices imply (25, 251)"
I've tried the following codes:
sp = pd.read_csv('sp1.csv')
sp = sp.set_index('ID')
op = pd.read_csv('op1.csv')
op = op.set_index('ID')
ff = pd.concat([sp, op], join = 'outer', sort = False, axis = 1)
I've also tried concat the two files together without setting up index, and the result seemed having correct rows, but the horizontal values were incorrect related.
I've also tried merge as well, but it came with many unnecessary duplicated rows within each group. Since each group has different year and age, I found quite difficult to delete those newly generated rows using this method.
full = pd.merge(sp, op, on = 'ID', how = 'outer', sort = False)
Maybe somebody can suggest ways to easily delete these duplicates, and this will also work for me, because the merged file became so huge! Thanks in advance!
Expected results would be including all different values from both csv files. It is somewhat like this:
ID,year,age,from1,to1
810006862,2000,49,,
810006862,2001,,,
810006862,2002,,15341,15705
810006862,2003,52,15706,16070
810006862,2004,,16071,16436
810006862,2005,,,
810023112,2000,,14610,14975
810023112,2001,,14976,15340
810023112,2003,27,15825,16523
810023112,2004,28,,
810023112,2005,29,,
810023112,2006,30,,
810033622,2000,24,13211,14876
810033622,2001,25,14761,14987
I've searched online for similar posts along for quite some time, but unable to solve my problem. Can anybody offer any clue how to do this? Thanks a lot!
My User_ID has several images and with this piece of code I receive the largest IMAGE_NR .
max({$<USER_ID = {'8638087'}> } IMAGE_NR)
Every image number is linked to an IMAGE_ID. How do I get this?
In words:
Give me IMAGE_ID where IMAGE_NR = largest IMAGE_NR of my USER_ID
I tried following that doesn't work:
sum({$<max({<USER_ID={'8638087'}> } IMAGE_NR)>} IMAGE_ID)
Thank you for any thoughts!
Here is a step-by-step solution.
You mention that this already brings back the correct IMAGE_NR:
max({$<USER_ID = {'8638087'}> } IMAGE_NR)
Now you want the IMAGE_ID. The logic is:
=only({CORRECT_IMAGE_NR_SET_ANALYSIS} IMAGE_ID)
I generally prefer to avoid search mode (double quotes inside element list) and instead use a higher evaluation level on the top calculation, so I would recommend:
$(=
'only({<IMAGE_NR={'
& max({$<USER_ID = {'8638087'}> } IMAGE_NR)
& '}>} IMAGE_ID)'
)
This would also provide you a nice preview formula in the formula editor, something like:
only({<IMAGE_NR={210391287}>} IMAGE_ID)
I didn't test it, but i believe it's something like :
only({<IMAGE_NR={'$(=max(IMAGE_NR))'}, USER_ID={'8638087'}>} IMAGE_ID)
If the dimension of the table is USER_ID then this will return the IMAGE_ID of the maximum IMAGE_NR per USER_ID. Should work / return results for any dimension, but then will have to be read as maximum IMAGE_NR per whatever that dimesion is. The minus on IMAGE_NR is to get the largest/last sorted value
firstsortedvalue(IMAGE_ID,-IMAGE_NR)
If you're using it in a text box with no dimension then you can also add the set analysis
firstsortedvalue({<USER_ID={'8638087'}>} IMAGE_ID,-IMAGE_NR)
I have been using
=Sum(Fields!Figure, "Dataset1")
+ Sum(Fields!Figure, "Dataset2")
Now this gives me a result in a text box.
However, i need to sum together the result from 4 different tablix datasets.
I have tried this below but as soon as i try to sum more that 2 datasets it errors.
=Sum(Fields!Figure, "Dataset1")
+ Sum(Fields!Figure, "Dataset2")
+ Sum(Fields!Figure, "Dataset3")
+ Sum(Fields!Figure, "Dataset4")
This gives me (#error). I am assuming i am missing some code somewhere but i have looked online and cant find anything anywhere.
Can anyone help?
Thanks
I'm fairly certain the field is ended with .value:
=SUM(Fields!TextBoxName.Value, "Dataset1")
+SUM(Fields!TextBoxName.Value, "Dataset2")
...