In my use case, I am trying to migrate SQL based traditional data warehousing application into big data infrastructure. I chose Hive and i think it is doing well. However, what I could not find is that stored procedures. Is it available in hive? I am using apache Hue to write and execute Hive query.
No, Stored Procedures are not yet available. However, there are plans to be available in the future.
Please refer to HPL/SQL, I am looking for same solution but not try yet.
Related
I need to create recurring job in which I have to map two SQL Server databases which are on two different servers. I need to check the data mismatch in both the tables in regular intervals because new data keeps on adding every second.
I am thing to use anyone of the ETL tool like kettle pentaho which will actually do the data mapping. Do we have any other better option to handle this scenario.
This seems a ETL job approach, as long as you're using SQL Server I would recommend you use SSIS, is the Microsoft ETL tool. Of course you can use Pentaho and I think it will work very good also.
Another approach would be use linked servers and a job, writing the script as a stored procedure, but in my opinion this is not a recommended way to address the problem (SSIS or any ETL tool is so much versatile).
I have a requirement to move some of the existing frontend applications running Teradata as the backend to Google BigQuery. One of the common pattern used in these frontend applications is to call a Macro in Teradata, based on different input selected by users. Considering BigQuery doesn't have a way to create a macro entity, how can I replace this and have the frontend calling BigQuery to execute something similar. Connection to BigQuery is through ODBC/JDBC or java services.
A macro in Teradata is just a way to execute multiple SQL statements as a single request, which is in turn treated as a single transaction. It also allows you to parameterize your query.
If your new DB backend supports it, you can convert the macros into stored procedures / functions. Otherwise, you can pull out the individual SQL statements from the macro and try to run them together as a single transaction.
These links may be helpful: Functions,
DML
Glancing at the documentation, it looks like writing a function may be your best bet: "There is no support for multi-statement transactions."
You can look at Bigquery scripting which is in Beta - https://cloud.google.com/bigquery/docs/reference/standard-sql/scripting#bigquery-scripting for migrating your macros from Teradata. With this release you can write procedures where you can define all your business logic and then execute the procedure using a CALL statement.
Thanks,
Jayadeep
As mentioned above:
A macro in Teradata is just a way to execute multiple SQL statements
as a single request, which is in turn treated as a single transaction.
It also allows you to parameterize your query.
Having said that, you just need to do the migrating part from teradata, here you can find the guide to do this, and answering your question, the connection is made through JDBC whose drivers are tdgssconfig.jar and terajdbc4.jar.
I want to design Web UI which fetches data from HDFS. I want to generate some reports using this data which is stored in HDFS. I have my own custom reports format. I am writing REST API's to fetch data. But running HIVE queries gives latency issues Hence I want different approach for this, I could think of two.
Using IMPALA to create tables. But I am not sure about REST support for IMPALA.
Using HIVE but instead of MR use SPARK as execution engine. .
spark-job-server provides REST support, and fetch data with SPARK-SQL.
Which of the approach will be suitable or is there any better approach for this?
Please can anyone help as I am very new in this.
I'd prefer to choose impala if latency is the main consideration. It's dedicated to SQL processing on hdfs and does it well. About REST api and the application logic you are achieving, this seems to be a good example
I am investigating a legacy app that uses an Oracle 8i database in a test environment, specifically trying to find out what tables are accessed for read, insert, update or delete when the user performs an app function.
What is the best/easiest way to do this? Can I simply get a list of all sql statements sent to the database? Can I see when stored procedures are called?
Having little experience with Oracle but getting help from a DBA, I'm thinking I should either use a trace or look at the redo log with LogMiner, but how?
Thanks!
What you could do is to harvest the sql's from v$sql. If the SQL's are properly written - using bind variables - you should be able to catch most of the statements in a table for this. I currently have no running v8 at hand but this should be possible.
In order to get most of them, you probably need to repeat the harvesting during the various workloads that run on the database.
I need to migrate data (including tables, procedures, schema) from Sql Server 2005 to Oracle 10g. Can anyone suggest me a way to do it? Thanks.
Take a look at the Oracle Migration Workbench
SwisSQL have a commercial tool that will do that. It can go from and to several different database vendors. I've used it before to go from SQL Server to Oracle and it works well.
The best way to work around is by using ORACLE GATEWAY First create tables, views, stored procedures and triggers in ORACLE and then using ORACLE GATEWAY you can directly insert the data from SQLSERVER to ORACLE.
But you need to be cautious about data types. For migrating data from Image, TEXT you need to use *DBMS_Sql*
One of the possible options is to use SQL Developer. It is an Oracle free tool, which migrates both database objects and data. However, you need to have high level of knowledge in database migration, in particular experience of working with Oracle and SQL Server. Sometimes the results of automatic migration with SQL Developer require significant manual adjustments.