I want to implement min and max date validation(i.e. min date should be less than max date).
I'm using v1.0.0-rc.2 version and I searched the solution, but not found for this version.
Note: I can't change the plugin(version) as it has been used for the entire project.
Please provide me solution only for materialize "v1.0.0-rc.2" version.
Related
I'm trying to query 10th, 25th, 75th percentile for each row in druid from an integer column value. I came across some solutions ( http://druid.io/docs/latest/development/extensions-core/datasketches-quantiles.html ) but not sure how they can be implemented. Can somebody explain it in simpler terms?
If you're using druid SQL, it's pretty easy (once you load the extension). Eg, you can use
SELECT APPROX_QUANTILE_DS(myNum, .25) FROM myData
to get the 25th percentile of myNum. (If you search for 'approx_quantile_ds' on this doc page, there's also a link to a known issue.)
For native queries, I'm not sure offhand, but maybe this will help: https://www.druidforum.org/t/quantile-calculation/4929
I've been trying to learn pandas in a lab class. One part of our lab manual goes over generating time-based indices with the date_range function. The class's lab manual says
The freq parameter accepts a variety of string representations, referred to as offset aliases. See Table 1.3 for a sampling of some of the options. For a complete list of the options, see http://pandas.pydata.org/pandas-docs/stable/timeseries.html#offset-aliases.
I checked through the 'offset-alias' and 'anchored offsets' sections of the online documentation. Most of the entries in table 1.3 can be understood from those two sections.
However, the last entry of the table is "WOM-3FRI" The table says this corresponds to a frequency of every 3rd Friday of the month. I have no idea how to deduce that from the online documentation. It looks like "WOM" is being used as the alias and "3FRI" is being used as an anchor. But, "WOM" is not listed as an alias in the online documentation. So, I'm struggling to make sense of what's happening here.
One hypothesis I have is that this is some sort of operation.
The online documentation and my lab book have a couple examples where prepending a number in front of an alias increases the length of a the period by that number. So, '2' operates in a way so that '2M' creates a frequency of every 2 months. Similarly, '5' operates in a way so that '5Y' creates a frequency of every 5 years. Does 'O' somehow operate in a way that the offset alias 'XOY' gives the xth sub-period of period Y? For example, would "MOY-5" give the 5th month of the year? Would "DOY-7FRI" give the 7th Friday of the year?
Another hypothesis I have is that "MOA" is a new-fangled alias, and "3FRI" is an anchor for it. However, the documentation online does not list "MOA". I checked, and it was pandas 0.23.4 documentation. My lab machine is running version 0.23.4, and it can handle "WOM-3FRI" just fine. Have they just not updated the documentation yet?
Would anyone could clear up the method/theory behind creating "WOM-3FRI"?
Lab manual with Table 1.3: http://www.acme.byu.edu/wp-content/uploads/2018/10/Pandas4.pdf
I did a little more digging. It looks like "WOM" is just an undocumented offset alias. Source: https://github.com/pandas-dev/pandas/issues/2289#issuecomment-269616457
read pandas DateOffsets:
WeekOfMonth - 'WOM' - the x-th day of the y-th week of each month
And see example here exercise
Create a DateTimeIndex consisting of the third Thursday in each month for the years 2015 and 2016.
pd.date_range('2015-01-01', '2016-12-31', freq='WOM-3THU')
I need to add decorators that will represent from 6 days ago till now.
how should I do it?
lets say the date is realative 604800000 millis from now and it's absolute is 1427061600000
#-604800000
#1427061600000
#now in millis - 1427061600000
#1427061600000 - now in millis
Is there a difference by using relative or absolute times?
Thanks
#-518400000--1
Will give you data for the last 6 days (or last 144 hours).
I think all you need is to read this.
Basically, you have the choice of #time, which is time since Epoch (your #1427061600000). You can also express it as a negative number, which the system will interpret as NOW - time (your #-604800000). These both work, but they don't give the result you want. Instead of returning all that was added in that time range, it will return a snapshot of your table from 6 days ago....
Although you COULD use that snapshot, eliminate all duplicates between that snapshot and your current table, and then take THOSE results as what was added during your 6 days, you're better off with :
Using time ranges directly, which you cover with your 3rd and 4th lines. I don't know if the order makes a difference, but I've always used #time1-time2 with time1<time2 (in your case, #1427061600000 - now in millis).
I have a scenario in which I have an object which has an availability property assosiated with it. I have encoded the dates in a month as a 32 bit binary with 1 for available and 0 for not available. Now I want to search for objects that are available on a range of dates. How will I best do it with lucene?
Maybe a better way to store that would be as:
available_on=20111028
available_on=20111029
where the date is encoded as an integer, and one field for each date that is available. Then you can use a NumericRangeQuery to search the availability range.
Failing that, I guess you could write a filter to step through each value used for your bitfield and pick out the ones with one of the requisite bits set.
I have a solr instance containing documents which have a 'startTime' field ranging from last month to a year from now. I'd like to add a boost query/function to boost the scores of documents whose startTime field is close to the current time.
So far I have seen a lot of examples which use rord to add boosts to documents whom are newer but I have never seen an example of something like this.
Can anyone tell me how to do it please?
Thanks
If you're on Solr 1.4+, then you have access to the "ms" function in function queries, and the standard, textbook approach to boosting by recency is:
recip(ms(NOW,startTime),3.16e-11,1,1)
ms gives the number of milliseconds between its two arguments. The expression as a whole boosts scores by 1 for docs dated now, by 1/2 for docs dated 1 year ago, by 1/3 for docs dated 2 years ago, etc.. (See http://wiki.apache.org/solr/FunctionQuery#Date_Boosting, as Sean Timm pointed out.)
In your case you have docs dated in the future, and those will get assigned a negative score by the above function, so you probably would want to throw in an absolute value, like this:
recip(abs(ms(NOW,startTime)),3.16e-11,1,1)
abs(ms(NOW,startTime)) will give the # of milliseconds between startTime and now, guaranteed to be nonnegative.
That would be a good starting place. If you want, you can then tweak the 3.16e-11 if it's too agressive or not agressive enough.
Tangentially, the ms function will only work on fields based on the TrieDate class, not the classic Date and LegacyDate classes. If your schema.xml was based on the example one for Solr 1.4, then your date field is probably already in the correct format.
You can do date math in Solr 1.4.
http://wiki.apache.org/solr/FunctionQuery#Date_Boosting