Can anyone please provide me a step by step procedure for scheduling a pentaho job/transformation using crontab.
As per my knowledge first you need to create shell script to run pentaho jobs or transformation, whatever it is.
Once you create a shell file then you have to call this file with the required arguments, if any.
Then you can schedule this scripting file using crontab.
0 5,17 * * * /scripts/script.sh
Related
I have data coming from an API in JSON format which I then run a few functions/transformations in python and then insert the data in to an SQL database through pandas & SQLalchemy.
Now, how do I automatically do this at the end of every day, without having to open up the script and run it manually?
You can use crontab on a server (or your Linux/Mac laptop but it will not run the script if its turned off of course).
You can run crontab -e to edit the crontab file. Add something like the following to run your script everyday at 11 PM:
0 23 * * * ~/myscript.py
crontab guru is a useful resource to try different schedule expressions.
I have a small sql query(which uses Wbexport utility) that I need to run daily. I want to automate this task so I am thinking to write a batch file and schedule it using windows task scheduler. Could someone tell me how to run a sql query/.sql file from command line in SQL Workbench/J?
This is documented in the manual
You need to put the query you want to run in a .sql script and then pass the name of the script on the commandline:
java -jar sqlworkbench.jar -script=run_export.sql -profile=...
or
SQLWorkbench64.exe -script=run_export.sql -profile=...
There are various ways to define the connection, either through the -profile parameter or by specifying the complete connection information. When using the profile parameter you need to make sure that the profiles are stored in the default location or specify the location where the profiles are stored.
I have a an sql script that creates a database, and want to know how to run this from a batch file at a command prompt.
Do i create a batch file with a few lines of code pointing to the location of the .sql file, or create a new batch file containing the contents of the .sql file?
I've had a look at a couple of related questions, but can't seem to see a clear answer.
Thanks :)
You could create a batch file and use the -i flag with sqlcmd.exe, where -i sets the path to the .sql file you want to run:
sqlcmd.exe -i F:\wherever\the\file\is
See http://msdn.microsoft.com/en-us/library/ms162773.aspx for a full list of the flags and this post, How to use sqlcmd to create a database.
is quite easy, you only need 3 things:
SQL Client Path/
Script Path/
SQL Conection data/
Then you can run it directly from the batch without any complex script - let me know if you have the info I can arrange it.
regards
Suppose I am having 100 sql files and I need to execute all the files one by one in sequence. Is there any approach to do this with out executing the scripts manually?
You can write a bat file to execute them using sqlcmd Utility
Write a shell script or similar to run them sequentially.
We've had great success with the SQL Deploy tool by SSW Australia.
It's not free - but worth every penny, and saves you so much time, it pays for itself in no time at all!
(I have no affiliation with SSW Australia other than being a happy user of SQL Deploy)
Pipe the dir /b > foo.txt output to a file
Add sqlcmd at the start of each line etc using a decent text editor like notepad++
You can use PowerShell to do this. The following blog post describes such a script. As part of the foreach a pipe is used to sort the files in the manner that you want to process them. In this example it is being sorted by descending alphabetical file name, but you can also do it by other attributes, such as the date the file was created.
Also the following blog post describes how to run all the .sql files in a directory like the above linked post, but without the use of PowerShell
Assuming your files are named something like this:
001_my_script.sql
002_another_script.sql
003_foo_script.sql
004_bar_script.sql
You can do the following at the command line:
copy *.sql /a my_big_script.sql
And then run the resulting file as one script (via sqlcmd or Management Studio).
Suppose I have wrote script Table_ABC.sql which creates table ABC. I have created many such scripts for each of required tables. Now i want to write a script that call all of these script files in a sequence so basically I want another script file createTables.sql. Mysql provides option to execute a script file from "mysql" shell application but could find some command like exec c:/myscripts/mytable.sql. Please tell me if there is any command that can be written in sql script itself to call other one in latest mysql versions or alternative for same.
Thanks
You can use source command. So your script will be something like:
use your_db;
source script/s1.sql;
source script/s2.sql;
-- so on, so forth