I am currently updating a java-based web application which allows database developers to create stored procedure regression test suites for database testing.
Currently, for test setup, execution and clean-up stages, the user is provided with text boxes where they are able to enter SQL code which is executed by the isql command.
I would like to extend the application to use DB Unit’s DatabaseOperation methods to provide more ways to setup the state of the database than just SQL statements. The main reason for using Db Unit rather than just SQL statements is to be able to create and store xml and xls DataSets on a server where they can be associated with their test cases and used for data setup.
My question is:
How can I provide users with the functionality of the DBUnit DatabaseOperation methods from a web interface?
I have considered:
Creating a simple programming language and a parser to read some simple syntax involving the DB Unit method names which accept a parameter being the file location to an xml or xls DataSet. I was thinking of allowing the user to register the files they need with the web app which would catalogue them and provide each file with an identifier which could passed as a parameter to the methods in this simple programming language.
Creating an XML DTD which provides the user with the ability to specify operations and parameters. If I went this approach, how can I execute the methods and their parameters that I parse from the XML document?
Creating a table in the database which stores the method and a FK relation to a catalogued DataSet file, however I don’t think this would be good solution due to the fact that data entry would be tedious.
Thanks for your help.
This actually seems like rather simple problem when I think about it again.
DBUnit has plugins for Maven and Ant integration which run tests written in XML in the Maven POM file.
I'm going to take a similar approach and go ahead with the XML option using the Xerces-J parser and create a collection of Operation, Export and Compare objects which are run in order.
Related
I have quite old application with current database (on MSSQL but it does not matter). I scripted it completely with all required static data. Now I want to introduce DB change only via update scripts. So each function, each SP will be placed in stand-alone file and all schema update scripts will be stored in files named like 'SomeProduct01_0001' what means that this script belongs to product SomeProduct, sprint 1 and it is first schema update script.
I know that each script must be absolutely re-runnable, but anyway I want to have functionality to combine these scripts into one based on DB version (stored in DB table).
What common best practices there is to handle bunches of update scripts?
What is better - implement version anylyzis in collector
(bat or exe file) or add some SQL header to each file? From other point of view I am already have version - it will consist of sprint identifier and script identifier, not sure that it is ok to duplicate this information in script header.
How to skip file content if user tries to apply it to newer database but keep
availability combine this script with any other to perform updates
of other old database?
How to avoid database conflicts if combined scripts operates columns/table which still does not exists in database but will be created byt this script (for example, in line 10 table created and in line 60 it is used in trigger or constraint, as I know script will not be validated)? Maybe wrap in EXEC('') entire script? What I need to escape besides sigle quote characters?
UPD: As David Tanzer asnwered it is better to use ready solutions for DB migrations, so it may be best solution for cases like mine. It was not and answer exactly for my question, but it is suitable for new solutions.
You don't have to implement this yourself, there are tools that do it. Take a look at dbmaintain, it provides almost exactly the functionality you described:
http://www.dbmaintain.org/overview.html
I know of and have worked with several teams who use it to manage their database schemas in all their environments: Development, Testing, Staging and Production.
http://flywaydb.org/ seems to be another tool to do this, and it has even more features. They even have a comparison of multiple tools on their homepage (including dbmaintain)
In Grails, how can plain SQL/DDL be used to create / drop tables in the same manner they would be if one were using GORM / ORM?
For example, when using GORM / ORM, the tables used for persistence are regularly created/dropped, and inserted into, during the runtime of Integration Tests, and execution of the application.
I know there is a way to do this using just Groovy as shown in the example named "Advanced Usage" here, but I'm looking for something more along the lines of being built into the framework already, something where I can specify an SQL file with DDL to be loaded.
I'm looking for something more along the lines of being built into the framework already, something where I can specify an SQL file with DDL to be loaded.
As far as I know, there is no such support built-into Grails, so you'll have to write it yourself. Luckily it shouldn't be too difficult. Here's an implementation plan:
Store your DDL file in the conf directory
In Bootstrap.groovy, dependency-inject the DataSource Spring bean
In the init closure of Bootstrap.groovy use the DataSource to get a Connection to the DB
Using the Connection, create the database and execute the SQL statements in the DDL file against it
In the destroy closure of Bootstrap.groovy, drop the database
Here is what I have:
1) a simple sql file given to me that creates tables and fills them with data
2) a simple sql file that contains PL/SQL procedures I've written for displaying/manipulating the tables
The goal is to create some sort of user interface that allows a student to login, view their transcript, withdraw from classes, etc.
I am using sqlplus. I have procedures that do all the required displaying/manipulating. I am successful at creating a simple command line UI with sqlplus, but the problem arrives when I need to get user input inside a loop (for allowing them to see course information for any number of courses until they wish to go back to the main menu). After doing research, I learned that this would be the job of something like PHP, C, etc. Unfortunately, I am not proficient in any language required, and setting up extensions and such have proven to be extremely complicated.
I am capable of learning the necessary techniques to complete this, but I do not know which direction to go in. What is easiest to implement a simple UI? Should I use PHP? C? C++? Is there some sort of program out there that automatically creates simple UIs given database data?
Oracle Apex is perfect for your situation. You can easily create web based forms and reports to perform all CRUD operations. Plus, it's free to use with any licensed Oracle database: Oracle Application Express
In my application, user can specify the format of their file. Based on user input we dynamically create SSIS package.
http://lakshmik.blogspot.com/2005/05...eate-ssis.html
Dynamically created SSIS package is used for processing user's files.
We want to evaluate Pentaho-Kettle for this requirement. Is this possible with Kettle to dynamically create ETL jobs based on user's inputs?
If not Pentaho, is there any Java ETL tool which allows use to dynamically create ETL jobs?
I dont know about others, but this is traditionally quite tricky in Kettle but people have done it in various ways.
The best option for this is the (brand new) inject step which lets you do really clever stuff with metadata - but it only works for some basic steps. I think it will do what you want, read about it in Matt Casters (PDI Creator and god) blog here:
http://www.ibridge.be/?s=inject&submit=Go
If that doesnt work; then your other options are to go down the generic field name route (nasty) or dynamically generate the transformation. This is easier than it sounds - but you will need to get much more heavily involved in the Java side than usual for an ETL tool.
It is possible, and not very hard.
You can use the Kettle API to dynamically create transformations that can do anything Kettle does. The GUI designer uses the API to create transformations, so anything you can do with the GUI, you can do thru the API.
If you look in the 'test' source tree you will find lots of examples of how to create transformations dynamically.
For instance, will I be able to create an application that allows users to create and modify existing types at runtime? Will I be able to persist instances of those types in SQL without having to worry about the user who adds 100,000 records and expects a (really) fast query on them?
Think SharePoint Content Types... but on steroids. Oslo steroids - Possible or not?
That would be awesome!
In the demos, they create the new extent, and then a (updatable) view with the old name.
But I haven't heard about a feature that would automatically merge existing data into the new structure, though. For now, they suggest using SQL Server Integration Services for that part - but then it's a DB-Admin task.
Regarding performance, after MSchema is compiled to SQL statements, it's all plain SQL Server performance.