I'd prefer to use raw SQL ( Mainly select + insert ) instead of O/R mapper, because it would be difficult to perform queries.
(The RDBMS is postgres9.4)
So the question is
Can I use raw SQL for logic part in Airflow?
You can create tasks that run raw SQL in a Postgres database directly using the PostgresOperator. You will need to set up your Postgres database as a Connection object in order for Airflow to know how to connect to the database.
Related
We are using django and sql, so we have written query in sql , here my request is how can I convert sql query to django query.Please let me know
You can use ModelName.objects.raw() to run SQL queries which will return model Instances.
Refer: https://docs.djangoproject.com/en/2.2/topics/db/sql/
or to execute directly from Database import connections from django.db.
Refer:https://docs.djangoproject.com/en/2.2/topics/db/sql/#executing-custom-sql-directly
Note: Django ORM is secure than normal SQL. If using raw SQL queries read docs on how to make raw SQL queries secure.
I'm trying to implement simple queries like SELECT * FROM TABLE_X WHERE XID = #id, but the problem that I'm having is that these queries would run on different databases (SQL Server and Oracle) for different application instances.
How to do it without to have to write each database a new set of queries?
Dapper is really closed to the database, and allow you to leverage pure sql tricks specific for a specific database. In my opinion you should use a query object pattern, so you will have an interface in front of each extraction /commit that would possibly change for SQL/Oracle.
I've downloaded the code of SqlMapper.cs and hacked SetupCommand to check if the command is from Oracle or SQL Server.
That was what I did:
if (cnn.GetType().Name.ToLowerInvariant().Contains("oracle"))
{
sql = sql.Replace('#', ':');
}
We have 40+ Tables present in SQL SERVER DB and we need to copy the data to an IBM DB2 database. What methods do you recommend to accomplish this?
My ANALYSIS:
BCP and Data Import - The team is trying to avoid any BCP files
Write Stored procedure and use LINKED Server in SQL and insert the data in DB2
SSIS Packages to move data.
Please let us know if you have any better way to approach this issue.
Have you considered Information Integration, that is known in DB2 as federation? you can do a select in SQL Server directly from DB2, and with this feature you can define a cursor and then just use the LOAD command.
I have two DB's one is feed by filtered data from another, now i'm using perl script witch executes query on foreign DB, stores a result in a csv file, and loads it to local DB using \COPY sytnatx
Is there a way to write plpgsql function witch will connect to foreign DB and load filtered data in local DB ( I know it can be done in ie. plperl, but i search more "native" way )
And there is the DBI-LINK that supports much more databases :)
Currently, PostgreSQL has dblink, but it only supports connecting to other PostgreSQL instances - not any other database, sadly.
I would recommend PL/Proxy, which is significantly easier to use - just write the desired stored procedure on the target database (with some minor caveats, like not using enumerated types), and declare the same function on the source, PL/Proxy will handle the communications. It is the basis for Skype's distributed database architecture and is production-ready.
I'd like to write stored procedures in pgSQL that dynamically generate web-ready data. I need a pure SQL to HTML or SQL to XML gateway. Oracle has OWA. In Oracle you can setup a RAC frontend to a SAN and connect a large set of OWA hosts to your RAC so you
layer your web requests and spread your queries.
What is the PostgreSQL or MySQL equivalent? I'm not looking at getting the data out of the DB, then processing it via Python or Ruby. Is something like
this even possible in PostgreSQL?
From experience, stored SQL procedures are better at moving/calculating large datasets than piping the SQL query/cursor/proc result set to a middleware Python/Ruby/Perl/PHP
that then process the data and send it to the web browser.
PostgreSQL has a host of functions that allow you to convert from tables/schemas/queries to XML ( http://www.postgresql.org/docs/8.4/static/functions-xml.html ) as well as a built-in XML datatype ( http://www.postgresql.org/docs/8.4/static/datatype-xml.html ).
Unfortunately, I can't give much more information than that as I'm not an XML guy but hopefully these references will make sense to you. ;)