I have some documents stored in Raven, which have a CreationDate property, of type "DateTimeOffset".
I have been attempting to get these documents returned in a query from C#, and they are never returned if I use the CreationDate in the query criteria.
After watching the Raven console, I saw the query being issued was:
Query: (FeedOwner:25eb541c\-b04a\-4f08\-b468\-65714f259ac2) AND ( Creati
onDate:[20120524120000000 TO NULL] AND CreationDate:{* TO 20120525120000000})
I ran this query directly against HTTP, and changed the date format to:
Query: (FeedOwner:25eb541c\-b04a\-4f08\-b468\-65714f259ac2) AND ( Creati
onDate:[2012-05-24120000000 TO NULL] AND CreationDate:{* TO 2012-05-25120000000})
And now it works - it returns my documents which DEFINITELY fall within the range.
Is Raven generating the incorrect date format for lucene? If so, how do I fix this?
Notes:
Yes I need time zone support
Yes I need Time aswell as Date in my index.
Thanks
[EDIT]
Err... I just changed my entities to use DateTime, just for giggles... and it still fails to return the data... whats going on??? Im using RavenDB.Server.1.2.2002-Unstable
Adam,
You are using RavenDB Server 1.2 pre release bits with the RavenDB Client 1.0 Stable.
Those are incompatible.
Related
I'm new with Airflow and I'm currently stuck on an issue with the Bigquery operator.
I'm trying to execute a simple query on a table from a given dataset and copy the result on a new table in the same dataset. I'm using the bigquery operator to do so, since according to the doc the 'destination_dataset_table' parameter is supposed to do exactly what I'm looking for (source:https://airflow.apache.org/docs/stable/_api/airflow/contrib/operators/bigquery_operator/index.html#airflow.contrib.operators.bigquery_operator.BigQueryOperator).
But instead of copying the data, all I get is a new empty table with the schema of the one I'm querying from.
Here's my code
default_args = {
'owner':'me',
'depends_on_past':False,
'start_date':datetime(2019,1,1),
'end_date':datetime(2019,1,3),
'retries':10,
'retry_delay':timedelta(minutes=1),
}
dag = DAG(
dag_id='my_dag',
default_args=default_args,
schedule_interval=timedelta(days=1)
)
copyData = BigQueryOperator(
task_id='copyData',
dag=dag,
sql=
"SELECT some_columns,x,y,z FROM dataset_d.table_t WHERE some_columns=some_value",
destination_dataset_table='dataset_d.table_u',
bigquery_conn_id='something',
)
I don't get any warnings or errors, the code is running and the tasks are marked as success. It does create the table I wanted, with the columns I specified, but totally empty.
Any idea what I'm doing wrong?
EDIT: I tried the same code on a much smaller table (from 10Gb to a few Kb), performing a query with a much smaller result (from 500Mb to a few Kb), and it did work this time. Do you think the size of the table/the query result matters? Is it limited? Or does performing a too large query cause a lag of some sort?
EDIT2: After a few more tests I can confirm that this issue is not related to the size of the query or the table. It seems to have something to do with the Date format. In my code the WHERE condition is actually checking if a date_column = 'YYYY-MM-DD'. When I replace this condition with an int or string comparison it works perfectly. Do you guys know if Bigquery uses a particular date format or requires a particular syntax?
EDIT3: Finally getting somewhere: When I cast my date_column as a date (CAST(date_column AS DATE)) to force its type to DATE, I get an error that says that my field is actually an int-32 (Argument type mismatch). But I'm SURE that this field is a date, so that implies that either Bigquery stores it as an int while displaying it as a date, or that the Bigquery operator does some kind of hidden type conversion while loading tables. Any idea on how to fix this?
I had a similar issue when transferring data from other data sources than big-query.
I suggest casting the date_column as follows: to_char(date_column, 'YYYY-MM-DD') as date
In general, I have seen that big-query auto detection schema is often problematic. The safest way is to always specify schema before executing its corresponding query, or use operators that support schema definition.
I have been encountering a very strange error while working on a postgre server. I have a column which contains times as varchar in the format HH24MMSS and I have to convert it to a timestamp using to_timestamp(varchar, 'HH24MMSS'). However, it returns me wrong values and I cannot understand why. Here are a few examples:
Select to_timestamp('111111', 'HH24MMSS')
-> 11:00:11
Select to_timestamp('000000', 'HH24MMSS')
-> 00:00:00
Select to_timestamp('122317', 'HH24MMSS')
-> 12:06:49
Here A Short Overview:
Unfortunately, I cannot offer more detailed information about the server since I am not the admin nor the one maintaining it. I only know that it is Postgre 9.2. I really do not understand, what is wrong here. I would appreciate any hint or help.
Thanks for the comments. The "MM" is part of the problem and explains why the minutes have not been translated correctly. However, using the correct function with 'MI' instead of 'MM' still delivers weird values. For some reasons, postgre automatically adds 6:32 to add time:
enter image description here
I am getting an error message when I query a specific table in my data set that has a nullable timestamp field. In the BigQuery web tool, I run simple query, e.g.:
SELECT * FROM [reztrack.201401] LIMIT 100
The result I get is: Error: Schema changed for Timestamp field date
Example Job ID: esiteisthebomb:job_6WKi7ZhSi8D_Ewr8b5rKV-a5Eac
This is the exact same issue that was noted here: Error: Schema changed for Timestamp field.
Also logged this under: https://code.google.com/p/google-bigquery/issues/detail?id=307 but I was unsure since it said we should be logging everything in Stackoverlfow.
Any information on how to fix this for this or other tables would be greatly appreciated.
Note: The original answer states to contact google support, but Google support for BigQuery was moved to StackOverflow. Therefore I assume that means to open it as a new question in hopes the engineers will respond.
BigQuery recently improved the representation of its internal timestamp format (there had previously been a lot of cases where timestamps broke in strange ways and this change should fix that). Your table still was using the old timestamp format, and you tickled a bug in the old format when schemas changed (in this case, the field went from REQUIRED to OPTIONAL).
We have an automated process that coalesces tables to make their storage more efficient. I scheduled this to run over your table, and have verified that it has rewritten your table using the new timestamp format.
You should now be able to query this field of your table without further problems.
When I run a NamedQuery in NHibernate, if I set a DateTimeOffset using SetParameter("name", date) or SetParameter("name", date, new DateTimeOffsetType()), it loses the time offset information when sending the query ('2011-01-01T14:00:00.00').
Does anyone know where/how I can tell NHibernate to use the full date string?
Sorry, there was another problem going on with my query. The date time string output was just in NHProfiler. Using regular query logging showed that it is sending the correct time offset information.
My Lucene/Solr database contains a date column (created_at) that I need to use as a condition in a query.
I'm new to RoR and assume that RoR automatically uses its own date object upon anyObject.save, and that Solr in turn reindexes that column in its own way.
Regardless, the date is in this format: "2008-06-03 11:15:20"
I can write a quick parser to parse my query string into the above format, but when I query
Object.find(keyword:foo created_at >= '2008-06-03 11:15:20')
Solr throws a parsing error. I've tried several standard variations on this without success. Any suggestions?
I hate to ask the obvious, but have you checked the solr docs for the query language for dates?
http://wiki.apache.org/solr/SolrQuerySyntax
and experience suggests that ">=" is not a valid solr operator. You can do range queries on date fields, but using the correct format with your example query would be:
Object.find("keyword:foo AND created_at:[2008-06-03T11:15:20.000Z TO *]")