I have the following lines in a file. The first columns look like this (these are dates and time):
May 29 23:14:39
Dec 20 19:45:15
Nov 3 13:15:19
Sep 8 10:34:15
Mar 9 18:39:20
Jan 17 19:34:59
I would like to use awk to sort it by today's date. For example today is November 03 (Nov 3). Tomorrow it will be November 04 (Nov 4). The dates will be there accordingly as the days change and the file changes along. Now I would like the first line to be the today's date all the time / always. Is that even possible to sort it out like this using either awk, sed and the like?
Alphabetical sort does not work because it does 1,2,3,4 and so on and if today is January 20 for example, my first line will be January 1 anyway (not January 20) because it will do alphabetical order / sort.
Would appreciate any help / suggestions / pointers. Many thanks in advance.
P.S. Let me edit this as asked by Cyrus.
Well, the following code actually works, but... sed '/reject/!d' file.txt | sort -r -k2'... It goes like this: November 1, November, 2, November 3, and then October 31. I guess it grabs numbers like so 1,2,3,31. If I could get it to solve this it would also work. Thanks.
My desired output is to sort it by date with the current today's date to be the first line in my file all the time. However, the code above would also work for me if I could get it to count November 1, November 2, November 3, November 4 (instead of November 1, November 2, November 3, October 31).
P.S.S. That's another edit as per Ed's Morton request.
The dates are all there. I do not need to add anything. The only requirement is for today's date line to be the first one in that file all the time and then sort it out backwards in descending order. For example today is November 05, 2022 and this is the first line in the file like this Nov 5 12:45:89. Then all the other lines are for November 4, 3, 2, 1. Then all the other lines are for October 31, 30, 29, 28 and so on. It is supposed to go backwards and it has to start from the current date, that is from "today" all the time. For example tomorrow the first line has to be Nov 6 and everything else backwards. Then the day after tomorrow the first line has to be Nov 7 and everything else backwards and so on and so forth. I do not need to truncate anything. I do not need to add anything. I do not need to delete anything. All the data has to stay there and go backwards starting from the "current" day, whether it is today, tomorrow, after tomorrow and so on. I hope it's clear enough. Thanks.
It sounds like this, using any POSIX compliant versions of the tools, might be what you want but without expected output in the question it's a guess:
$ cat tst.sh
#!/usr/bin/env bash
awk -v today="$(date +'%F')" '
BEGIN {
OFS = "\t"
split(today,d,/-/)
year = d[1]
}
{
mthNr = index(" JanFebMarAprMayJunJulAugSepOctNovDec",$1) / 3
date = sprintf("%04d-%02d-%02d", year, mthNr, $2)
}
date <= today {
print date, $3, $0
}
' "${#:--}" |
sort -rk1,2 |
cut -f3-
$ ./tst.sh file
Nov 3 13:15:19
Sep 8 10:34:15
May 29 23:14:39
Mar 9 18:39:20
Jan 17 19:34:59
The above uses a DSU approach to solve the problem. It's sorting on both date and time so for the same date the output is similarly ordered by the time that day.
Splunk cron job does not provide a way for running scheduled queries on last and first business days of a month.
In that case I need to provide the data in the query and pass it to alerting condition.
For example:
I need to check each last business day if a file has been delivered. So an ideal alerting condition is:
count < 1 AND is_last_business_day=1
Here I need to check if the query was executed on the last business day.
I wonder if someone can help with a query to check if it is the last business day and pass it as a field. So it can be used in alert conditions.
| eval Date=now(),dayofmonth=strftime(Date,"%d"),weekday=strftime(Date,"%w"),lastDayOfMonth=strftime(relative_time(now(),"#month-1d"),"%d")
| eval is_last_business_day=case(lastDayOfMonth-dayofmonth<7 AND (weekday IN(1,2,3,4,5)),1,True(),0)
Using this statement would work.
it adds the fields Date,dayofmonth,weekday and lastDayOfMonth.
then uses these to calculate if its a last BusinessDay.
Which in this I have defined as the last weekday in the month.
You can stack eval statements by separating them with commas, feel free to put this all in one line or split them up for readability
After trying many things and also using some parts of the query from Daniel here is my solution.
I think this can be improved and may become a shorter query. I tried to write a query that is not the shortest but easy to understand.
The | stats count in the beginning is just to give eval values in the query output even without any results. So you can test the query without any additional conditions or waiting for some dataset results.
I have tested this query with multiple dates and it is working.
| stats count
| eval mydate=strptime("07/28/2022", "%m/%d/%Y")
| eval today_weekday=strftime(mydate,"%w")
| eval today_number=strftime(mydate,"%d")
| eval is_today_business_day=if(today_weekday>=1 and today_weekday<=5,1,0)
``` caluclate last day of month ```
| eval last_day_number=strftime(relative_time(now(),"#month-1d"),"%d")
``` tomorrow for checking if it is a business day ```
| eval tomorrow=relative_time(mydate,"+1d#d")
| eval tomorrow_day=strftime(tomorrow,"%w")
| eval tomorrow_number =strftime(tomorrow,"%d")
| eval is_tomorrow_business_day=if(tomorrow_day>=1 and tomorrow_day<=5,1,0)
| eval is_tomorrow_day_one=if(tomorrow_number=1,1,0)
| eval is_today_in_last_3days=if(today_number>last_day_number-3,1,0)
``` is today a business day AND is today in the last 3 days of month and ( is tomorrow not a businessday OR is tomorrow day one of next month ```
| eval is_today_last_business_day=if(is_today_business_day=1 and is_today_in_last_3days=1 and (is_tomorrow_business_day!=1 OR is_tomorrow_day_one=1),1,0)
| eval Date=now(),dayofmonth=strftime(Date,"%d"),weekday=strftime(Date,"%w"),lastDayOfMonth=strftime(relative_time(now(),"#month-1d"),"%d")
| eval daystillend=lastDayOfMonth-dayofmonth, daystillnextweekday=case(weekday=5,2,True(),0)
| eval is_last_business_day=case(DaysTillEOM<=DaysTillNextWeekDay AND (weekday IN(1,2,3,4,5)),1,True(),0)
New Answer to this question to fix the old one's issues
Approach is get number of days till end of month, and number of days till next weekday
We compare the number of days till end of month and days till next weekday to see if this is the last weekday before end of month (filtering out saturday and sunday too)
I'd like to seek help with my access database. i would like to distribute value (format on currency) into multiple field base on date start and date end.
for example on:
if field Spend Amount value is - $10,000
if field start date is - Jan-2018
if Field end date is - April-2018
then the Jan field will be $2,500
and Feb field should be $2,500
Mar field also $2,500
april field also $2,500
basically the amount should be equally divided base on start month and end month category
appreciate your help please
The expression to put in your update can look something like this:
Amount/(DateDiff("m",StartDate,EndDate)+1)
This will divide 10.000 with 4. The 4 comes from the DateDiff that counts the number of months between the start and end (which results in 3) plus one. Make sure to test edge cases like Nov 30 to Dec 1, leap years and other problematic combinations.
Edit, clarification - this is not VBA. Put it in an update statement
i need your help on this
i need to pull birthdays between two dates irrespective of year
i use the below query
$sql = "SELECT * FROM family_member WHERE DATE_FORMAT(dob, '%c-%d') BETWEEN DATE_FORMAT('2013-".$from_month."-".$from_day."', '%c-%d') AND DATE_FORMAT('2013-".$to_month."-".$to_day."', '%c-%d') order by MONTH(dob), DAYOFMONTH(dob)";
the query works well if i give the
- start date as Nov 6 &
- End date as Dec 13
but the query returns zero records if i give
- start date as Sept 6 &
- end date as Dec 13
it works at certain scenarios. can you please let me know the issue i need to correct
This is due to the fact, that September is the 9th month and "09" and "9" are not the same string. Use %m instead of %c
Edit
Some explanation as requested: For the month range of September to December your comparison range was 9-06 to 12-06. Now remembering, that this is a string comparison, e.g. 10-25 is NOT bigger than 9-06, meaning the BETWEEN clause will produce no meaningful results. If you chose 2-digit months, you end up comparing 09-06 to 10-25, which works as expected.
I need to create a CRON job that will run on the last day of every month.
I will create it using cPanel.
Any help is appreciated.
Thanks
Possibly the easiest way is to simply do three separate jobs:
55 23 30 4,6,9,11 * myjob.sh
55 23 31 1,3,5,7,8,10,12 * myjob.sh
55 23 28 2 * myjob.sh
That will run on the 28th of February though, even on leap years so, if that's a problem, you'll need to find another way.
However, it's usually both substantially easier and correct to run the job as soon as possible on the first day of each month, with something like:
0 0 1 * * myjob.sh
and modify the script to process the previous month's data.
This removes any hassles you may encounter with figuring out which day is the last of the month, and also ensures that all data for that month is available, assuming you're processing data. Running at five minutes to midnight on the last day of the month may see you missing anything that happens between then and midnight.
This is the usual way to do it anyway, for most end-of-month jobs.
If you still really want to run it on the last day of the month, one option is to simply detect if tomorrow is the first (either as part of your script, or in the crontab itself).
So, something like:
55 23 28-31 * * [[ "$(date --date=tomorrow +\%d)" == "01" ]] && myjob.sh
should be a good start, assuming you have a relatively intelligent date program.
If your date program isn't quite advanced enough to give you relative dates, you can just put together a very simple program to give you tomorrow's day of the month (you don't need the full power of date), such as:
#include <stdio.h>
#include <time.h>
int main (void) {
// Get today, somewhere around midday (no DST issues).
time_t noonish = time (0);
struct tm *localtm = localtime (&noonish);
localtm->tm_hour = 12;
// Add one day (86,400 seconds).
noonish = mktime (localtm) + 86400;
localtm = localtime (&noonish);
// Output just day of month.
printf ("%d\n", localtm->tm_mday);
return 0;
}
and then use (assuming you've called it tomdom for "tomorrow's day of month"):
55 23 28-31 * * [[ "$(tomdom)" == "1" ]] && myjob.sh
Though you may want to consider adding error checking since both time() and mktime() can return -1 if something goes wrong. The code above, for reasons of simplicity, does not take that into account.
There's a slightly shorter method that can be used similar to one of the ones above. That is:
[ $(date -d +1day +%d) -eq 1 ] && echo "last day of month"
Also, the crontab entry could be update to only check on the 28th to 31st as it's pointless running it the other days of the month. Which would give you:
0 23 28-31 * * [ $(date -d +1day +%d) -eq 1 ] && myscript.sh
What about this one, after Wikipedia?
55 23 L * * /full/path/to/command
For AWS Cloudwatch cron implementation (Scheduling Lambdas, etc..) this works:
55 23 L * ? *
Running at 11:55pm on the last day of each month.
Adapting paxdiablo's solution, I run on the 28th and 29th of February. The data from the 29th overwrites the 28th.
# min hr date month dow
55 23 31 1,3,5,7,8,10,12 * /path/monthly_copy_data.sh
55 23 30 4,6,9,11 * /path/monthly_copy_data.sh
55 23 28,29 2 * /path/monthly_copy_data.sh
You could set up a cron job to run on every day of the month, and have it run a shell script like the following. This script works out whether tomorrow's day number is less than today's (i.e. if tomorrow is a new month), and then does whatever you want.
TODAY=`date +%d`
TOMORROW=`date +%d -d "1 day"`
# See if tomorrow's day is less than today's
if [ $TOMORROW -lt $TODAY ]; then
echo "This is the last day of the month"
# Do stuff...
fi
For a safer method in a crontab based on #Indie solution (use absolute path to date + $() does not works on all crontab systems):
0 23 28-31 * * [ `/bin/date -d +1day +\%d` -eq 1 ] && myscript.sh
Some cron implementations support the "L" flag to represent the last day of the month.
If you're lucky to be using one of those implementations, it's as simple as:
0 55 23 L * ?
That will run at 11:55 pm on the last day of every month.
http://www.quartz-scheduler.org/documentation/quartz-1.x/tutorials/crontrigger
#########################################################
# Memory Aid
# environment HOME=$HOME SHELL=$SHELL LOGNAME=$LOGNAME PATH=$PATH
#########################################################
#
# string meaning
# ------ -------
# #reboot Run once, at startup.
# #yearly Run once a year, "0 0 1 1 *".
# #annually (same as #yearly)
# #monthly Run once a month, "0 0 1 * *".
# #weekly Run once a week, "0 0 * * 0".
# #daily Run once a day, "0 0 * * *".
# #midnight (same as #daily)
# #hourly Run once an hour, "0 * * * *".
#mm hh Mday Mon Dow CMD # minute, hour, month-day month DayofW CMD
#........................................Minute of the hour
#| .................................Hour in the day (0..23)
#| | .........................Day of month, 1..31 (mon,tue,wed)
#| | | .................Month (1.12) Jan, Feb.. Dec
#| | | | ........day of the week 0-6 7==0
#| | | | | |command to be executed
#V V V V V V
* * 28-31 * * [ `date -d +'1 day' +\%d` -eq 1 ] && echo "Tomorrow is the first today now is `date`" >> ~/message
1 0 1 * * rm -f ~/message
* * 28-31 * * [ `date -d +'1 day' +\%d` -eq 1 ] && echo "HOME=$HOME LOGNAME=$LOGNAME SHELL = $SHELL PATH=$PATH"
Set up a cron job to run on the first day of the month. Then change the system's clock to be one day ahead.
I found out solution (On the last day of the month) like below from this site.
0 0 0 L * ? *
CRON details:
Seconds Minutes Hours Day Of Month Month Day Of Week Year
0 0 0 L * ? *
To cross verify above expression, click here which gives output like below.
2021-12-31 Fri 00:00:00
2022-01-31 Mon 00:00:00
2022-02-28 Mon 00:00:00
2022-03-31 Thu 00:00:00
2022-04-30 Sat 00:00:00
00 23 * * * [[ $(date +'%d') -eq $(cal | awk '!/^$/{ print $NF }' | tail -1) ]] && job
Check out a related question on the unix.com forum.
You can just connect all answers in one cron line and use only date command.
Just check the difference between day of the month which is today and will be tomorrow:
0 23 * * * root [ $(expr $(date +\%d -d '1 days') - $(date +\%d) ) -le 0 ] && echo true
If these difference is below 0 it means that we change the month and there is last day of the month.
55 23 28-31 * * echo "[ $(date -d +1day +%d) -eq 1 ] && my.sh" | /bin/bash
What about this?
edit user's .bashprofile adding:
export LAST_DAY_OF_MONTH=$(cal | awk '!/^$/{ print $NF }' | tail -1)
Then add this entry to crontab:
mm hh * * 1-7 [[ $(date +'%d') -eq $LAST_DAY_OF_MONTH ]] && /absolutepath/myscript.sh
In tools like Jenkins, where usually there is no support for L nor tools similar to date, a cool trick might be setting up the timezone correctly. E.g. Pacific/Kiritimati is GMT+14:00, so if you're in Europe or in the US, this might do the trick.
TZ=Pacific/Kiritimati \n H 0 1 * *
Result: Would last have run at Saturday, April 30, 2022 10:54:53 AM GMT; would next run at Tuesday, May 31, 2022 10:54:53 AM GMT.
Use the below code to run cron on the last day of the month in PHP
$commands = '30 23 '.date('t').' '.date('n').' *';
The last day of month can be 28-31 depending on what month it is (Feb, March etc). However in either of these cases, the next day is always 1st of next month. So we can use that to make sure we run some job always on the last day of a month using the code below:
0 8 28-31 * * [ "$(date +%d -d tomorrow)" = "01" ] && /your/script.sh
Not sure of other languages but in javascript it is possible.
If you need your job to be completed before first day of month node-cron will allow you to set timezone - you have to set UTC+12:00 and if job is not too long most of the world will have results before start of their month.
If the day-of-the-month field could accept day zero that would very simply solve this problem. Eg. astronomers use day zero to express the last day of the previous month. So
00 08 00 * * /usr/local/sbin/backup
would do the job in simple and easy way.
Better way to schedule cron on every next month of 1st day
This will run the command foo at 12:00AM.
0 0 1 * * /usr/bin/foo
Be cautious with "yesterday", "today", "1day" in the 'date' program if running between midnight and 1am, because often those really mean "24 hours" which will be two days when daylight saving time change causes a 23 hour day. I use "date -d '1am -12 hour' "