i was searched about catchup on airflow documentatuon.
but i still don't understand what the purpose of this API.
catchup (bool) – Perform scheduler catchup (or only run latest)? Defaults to True
thanks
You'll find an expanded explanation in the documents about scheduling backfill and catchup.
Let me try to expand on it with an example.
Assume this calendar for January this year:
January 2018
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31
Let's say you add a DAG on the 23rd with start_date=datetime(2018, 1, 1) and schedule_interval='0 0 * * MON'.
With catchup=True on first parsing the DAG the scheduler will immediately recognize that the periods 1-1 to 1-8, 1-8 to 1-15, 1-15 to 1-22 have closed and passed. It would schedule a DAG run for execution_date 2018-01-01 starting when you add the DAG on 23rd. If there are max_active_runs > 2 it would also schedule a DAG run for 2018-01-08 and 2018-01-15.
With catchup=False on first parsing the DAG the scheduler will still recognize that the same periods have closed and passed. But it would schedule a DAG run for execution_date 2018-01-15 only, starting when you add the DAG on 23rd. IE it would run the most recent closed period first, and not run any prior periods. The next run would then be 2018-01-22 starting at 2018-01-29T00:00:00±scheduler_lag. But if after the 2018-01-15 run completed, you paused the DAG, and then unpaused it on 2018-01-29T09:00, the scheduler would see that there are prior dag runs, and that the most recent period's start time is well past, it would not run a catchup run of this missed period.
Related
I have a problem to manipulate my data. I have table like this
customer_id day_checkin month_checkin status_checkin
001 1 10 start check-in
001 2 10 check-in consecutively
001 3 10 check-in consecutively
001 4 10 check-in consecutively
001 6 10 start check-in
001 7 10 check-in consecutively
001 8 10 check-in consecutively
The question I want to answer is, what is the average user check-in in a row? from the example table above, this user successfully checked-in 4 days in a row (day 1 to 4). Then this user didn't check-in on day 5, which made him repeat the check-in cycle again. On the 6th day, he checked-in until the 8th day without skipping, which in this cycle he managed to check-in 3 days in a row (day 6 to 8).
I'm having trouble how to calculate it automatically from postgresql and get the average from that. I hope to get some tips from all of you for creating dashboard in Metabase. Thank you
I am trying to append some text in /var/log/messages output whenever the timestamp between the two consecutive log is different such as :
previous log: 00:01:59 and current log 00:02:00
or
previous log:00:01:49 and current log 00:01:50
above substring of timestamp if different in consecutive log, append some message to $0.
You may run below command it is working for 1 minute, needed it for 10 sec.
tail -f /var/log/messages |awk '{split($3,a,":");split($3,b,"");new_time=a[1]":"a[2]":"b[1]; if(prev_time==new_time) print $0; else print "10 Second group is over, starting new: "$0" "prev_time " "new_time } {split($3,a,":");split($3,b,"");prev_time=a[1]":"a[2]":"b[1]}'
Required result is modification of above command to print same message in 10 second gap of logs , currently its doing for 1 minute. I have used split() to capture 'HH:MM:S" not "HH:MM:SS",so whenever privious 'HH:MM:S" and current 'HH:MM:S"differ , print the message "10 Second group is over, starting new: $0". Not sure what is the mistake here.
In short, currently its working when a minute changes, I need it when second changes from 39 to 40th sec or 09 sec to 10 sec. NOT 11 sec to 12 sec. HH:MM:SS , S marked in bold needed to be changed.
Sample lines:
Jan 23 15:09:54 foo bar
Jan 23 15:10:04 bla bla
this is the general idea:
$ for((i=35;i<45;i++)); do echo Jan 23 00:01:$i; done |
awk '{split($3,a,":"); print $0, (p!=(c=int(a[3]/10))?"<<<":""); p=c}'
Jan 23 00:01:35 <<<
Jan 23 00:01:36
Jan 23 00:01:37
Jan 23 00:01:38
Jan 23 00:01:39
Jan 23 00:01:40 <<<
Jan 23 00:01:41
Jan 23 00:01:42
Jan 23 00:01:43
Jan 23 00:01:44
first part is the test data generated for the script since you didn't provide enough. There is spurious first line match, which can be eliminated with NR>1 condition but I don't think that's critical.
I'm using dojox/calendar/Calendar in the "month" view, and I'd like to adjust the start/end days of the month to match up with my company's fiscal or marketing calendars.
In these Fiscal/Marketing Calendar Month-views, I'd like to see something like this (note the square brackets indicating the beginning and ending of the fiscal month):
S M T W T F S
19 20 [21 22 23 24 25
26 27 28 29 30 1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19] 20 21 22 23
Basically I want a normal Gregorian calendar, but with the month shifted to put a non-standard date as the beginning and end of the month-view. I see various "alternate calendars" such as dojox/date/hebrew, but I'm not certain as to whether implementing my own "date" object is the right way to proceed.
How can I do this?
I'm sure this has been done before. I've checked a couple of projects on github, such as calendar master, but they are all more complicated than what I need.
Oct 2013
S M T W T F S
30 1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31 1 2 3
Is there a best approach to do this? To layout days in a way that will match the day of the week for that month?
This will be made in a tableview, but I'm not looking for code, just the logic behind it. Unless there's a really good solution out there already.
Thank you!
The logic could be something like this (pseudo code):
get the weekday of the first day of the month in question
determine the number of days of the previous month
fill the first line
with leading final days of previous month, if any
the remaining days for the first week of the month in question
fill all other lines
fill the rest of the last line
with the first days of the following month, if any space left
I have a daily sales report query and it have 2 columns like
days sales
1 12
2 65
3 25
...
30 24
but when I want to print it there is a lots of free spaces on paper, so I want to seperate query with a percentage (like % 33)
and result will be like 3 x 2 columns for one paper. and it will be more comfortable for me.
days sales days sales days sales
1 12 11 21 21 5
2 65 12 53 22 18
3 25 13 0
...
10 45 20 12 30 55
Any way to do this with DevEx Grid?
this is the view which I get
and I dont want such kind of empty paper for couple of records..
You will not find an easy way to achieve the desired result using XtraGrid, because it is not intended for reporting. I suggest that you create reports using another product: XtraReports.