We have a python scripts that parses the sql script (select, insert) to get the source and target columns while inserting data from one table to another. Right now we parse only SQL queries. Now when we want some Spark SQL queries to be parsed using the same model, will the structure of the queries change from SQL to Spark SQL?
Related
I am working on PowerBi and use SQL server as database. I used views or direct tables as source to PowerBi . My views are simple select queries with simple joins. I am not finding any scope for query optimizations. Query execution takes time in SQL and table has millions of data increasing day by day.
Now I am thinking to use impala as well as SQL server. I am getting clean data from Rapidminer. I didn't use impala before. So I have some doubts. Please answer if you can. I have zero knowledge of impala.
Can we create connection between rapid miner and impala? then what will be the steps? google give me some steps which is difficult to understand.
Can we create connection between impala and sql?
Can we create view on impala and create joins in views? I know we can create view as well as joins in impala. But my question is can we create it together?
suppose SQl and impala connection is made then suppose I have one table from impala and one table from sql server management studio. can I join both tables in impala? for this can we create connection between impala and sql server management studio?
5.Can I use all tables or views created in sql to impala (after making connection between sql and impala). That means my tables or views are in sql. but I am fetching data in impala.
All tables stored in sql server. can I do join operation on these tables in impala.
7.Can I make views in impala using tables which are stored in sql
8.Can I create all tables in impala and do etl operation like sum, add, dateadd in impala
9.Can I create all tables in impala and do etl operation like sum, add, dateadd in power query
10.Can I create views from sql and put it in impala table. and use in power query
Can I create all tables and views with joins in impala?
12.How can I optimise my query in sql and if I run same query for same data in impala then my execution time will reduce or not?
My SQL query is like this
create view as test
select * from table a
inner join table b on a.id=b.id
inner join table c on b.name=c.name
go
output is 3000000 row. increasing day by day
also instead of using view I use table directly. but execution time is not decreasing.
I'm working on Sql server to Snowflake migration project,So i pointed ssrs reports to Snowflake data source and converting sql queries as per snowflake,but i'm not able to get how can we write queries for parameterized reports.Example select * from Student where Std_id=#id,want to convert to snowflake query.
You can use SQL variables:
https://docs.snowflake.com/en/sql-reference/session-variables.html
or Snowflake Scripting Variables:
https://docs.snowflake.com/en/developer-guide/snowflake-scripting/variables.html
I think the SQL variables would be helpful in your case, but Snowflake Scripting variables would be more similar to your SQL Server # variables.
I am looking to see if the capability is there to have a custom SSMS sql query imported in SPSS (Statistical Package for the Social Sciences). I would want to build syntax that generates this query as my new dataset that I can then continue my scripted analysis. I see the basic query capability of one table from a Sql Server but I would like to create a query that joins to many tables. I anticipate the query to be a bit complex with many joins and perhaps data transformations.
Has anybody had experience or a solution to this situation?
I know I could take the query and make a table of it that SPSS can then connect to but my data changes daily and I would need a job in another application to refresh this table before my SPSS syntax would pull it and I would like to eliminate that first step by just having the query that grabs the data at the beginning of my syntax.
Ultimately I am looking to build out my SPSS syntax and schedule it in the Production Facility to run daily.
I have created Relational Model in Oracle SQL Data Modeler with all tables and relations. I would like to generate a DDL to use it in Oracle SQL Developer and work with some SQL queries.
Before generating DDL I would like to fill all the tables with data, so I can put DDL with all data for tables prepared in it.
So, how can fill tables with data in Oracle SQL Data Modeler?
Tables can be filled with data in SQL Oracle Developer, filling it before in Data modeler is not needed.
SO heres what I am trying to do.
I have a table which has 2 columns - QC_Check and Query. For each QC_Check I have a query. There are several records like this.
Is there a way using SQL transformation that, I can fetch the SQL query stored in the Query column to Informatica, run the queries in Teradata and get the results stored somewhere.
Although, I have not tried it myself, this should be possible using SQL transformation in Query mode with Dynamic SQL Queries.
Use the table with Query column as a source. Create a SQL transformation with Query mode. Connect the Query column to the SQL transformation.
Write ~Query_Port~ in the SQL editor in the SQL transformation:
If you want to capture the results from your query, you have to configure output ports for columns you retrieve from the database.