PLease help me out.
I am currently migrating DTS 2000 packages to SQL 2005 version. I have a package which has a VB script in it. The job of this VB script is to download data from AVAYA server and update some templates. Post which it will select specific cells in the excel templates and update some more templates and run a stored procedure to further update a table.
I know that this can be replaced by SSIS scripting. But I have come to know that office side automation is not recomended anymore from SQL server.
So what is the best possible solution. This is a strict requirement that I update those excel templates from SSIS.
I saw this question over on dba.SE and it seemed an interesting approach. By installing the ACE OLEDB driver, not a full blown Office install, you can use any of your familiar DML (INSERT, UPDATE, DELETE) statements to modify values in Excel. I've used this approach to query Excel like a table before but never thought to try to modify the contents.
After installing the driver, you simply create an OLE DB Connection to the Excel resource and then use an Execute SQL Task to fire off your updates. Syntax to add the value DOB to cell D2 on Sheet1 would be
INSERT INTO [Sheet1$D2:D2] VALUES ('DOB')
Other examples from the wordpress article
INSERT INTO [NameOfExcelSheet] VALUES('firsttextcol', 2, '4/11/2009');
-- [I assume First Column is char field, 2nd col is integer, 3rd is Date]
DELETE FROM [NameOfExcelSheet] Where secondintcol=2;
UPDATE [NameOfExcelSheet] SET secondintcol = 3 where firsttextcol = ‘firsttextcol’;
References
http://www.winautomation.com/forum/updating-excel-through-sql-statement
http://yoursandmyideas.wordpress.com/2011/02/05/how-to-read-or-write-excel-file-using-ace-oledb-data-provider/
http://bidn.com/blogs/KeithHyer/bidn-blog/2475/updating-a-single-excel-cell-using-ssis
Related
Currently We have developed a system for a manual work they have been doing using many excel files.
Is there a best practice for data migration? because I wanted to use backend language like .net to do the validation and insert into tables rather than using SQL to do migration.
Total record in excel is around 12K rows but for many tables so its not needed consider a lot about performance and it is only one time.
I would add a few calculated columns in Excel that would generate SQL Insert / Update scripts. Something like ="INSERT INTO table (column) VALUES ('"&A1&"');"
Then just copy calculated column and run it through SQL client. I used to have a macros to run it directly from Excel through OLEDB that would highlight failed expressions and store SQL Exceptions next to them.
That way the data can be easily tidied, corrected and SQL re-run as needed.
How can I export all of my rows in a table to sql script in Microsoft SQL Server 2005 and then import them to another database?
Thanks in advance
If you moving it to another sql db you can right click the database you want and choose tasks -> generate scripts. That will launch a a wizard - follow along, choose the option to script all tables and data. Then execute that script in the new db(assuming that you've already created one with the same name)
If you can't find a data import/export tool that will work in your particular circumstances, it's possible to write plain SQL SELECT queries that will generate SQL INSERT statements. In this way it's possible to "export" all your data to a script file that can be run against the destination database. It's kind of an ugly hack, but it's simple and it works if you don't have a lot of data to move. See my answer to this question for details: Export SQL Server 2005 query result to SQL INSERT statement?
Note that this method assumes that the destination table already exists. But it's pretty straightforward to generate table creation scripts, as J Cory's answer has already shown.
There's a command line tool available to dump your data from particular tables into a SQL script that be executed against a different database:
http://blog.sqlauthority.com/2007/11/16/sql-server-2005-generate-script-with-data-from-database-database-publishing-wizard/
I don't believe SQL Management Studio Express supports data scripting (as your screenshot on J Cory's answer shows), but the full version does support that feature. In either case, the command line tool should accomplish what you need.
Is it possible to search and replace all occurrences of a string in all columns in all tables of a database? I use Microsoft SQL Server.
Not easily, though I can thing of two ways to do it:
Write a series of stored procedures that identify all varchar and text columns of all tables, and generate individual update statements for each column of each table of the form "UPDATE foo SET BAR = REPLACE(BAR,'foobar','quux')". This will probably involve a lot of queries against the system tables, with a lot of experimentation -- Microsoft doesn't go out of its way to document this stuff.
Export the entire database to a single text file, do a search/replace on that, and then re-import the entire database. Given that you're using MS SQL Server, this is actually the easier approach. Microsoft created the Microsoft SQL Server Database Publishing Wizard for other reasons, but it makes a fine tool for exporting all of the tables of a SQL Server database as a text file containing pure SQL DDL and DML. Run the tool to export all of the tables for a database, edit the resulting file as you need, and then feed the file back to sqlcmd to recreate the database.
Given a choice, I'd use the second method, as long as the DPW works with your version of SQL Server. The last time I used the tool, it met my needs (MS SQL Server 2000 / 2005) but it had some quirks when working with database Roles.
In MySQL, you can do it very easily like this:
update [table_name] set [field_name] = replace([field_name],'[string_to_find]','[string_to_replace]');
I have personally tested this successfully on a production server.
Example:
update users set vct_filesneeded = replace(vct_filesneeded,'.avi','.ai');
Ref: http://www.mediacollege.com/computer/database/mysql/find-replace.html
A good starting point for writing such a query is the "Search all columns in all the tables in a database for a specific value" stored procedure. The full code is at the link (not trivial, but copy/paste it and use it, it just works).
From there on it's relatively trivial to amend the code to do a replace of the found values.
Is it possible to perform a bulk insert into an MS-SQL Server (2000, 2005, 2008) using the RODBC package?
I know that I can do this using freebcp, but I'm curious if the RODBC package implements this portion of the Microsoft SQL API and if not, how difficult it would be to implement it.
check out the new odbc and DBI packages. DBI::dbWriteTable writes around 20,000 records per second... Much much faster than the Row Inserts from RODBC::sqlSave()
You're probably looking for ?sqlSave which uses a parametrized INSERT INTO query (taking place in one operation) when you set Fast=True.
Now You can use dbBulkCopy from the new rsqlserver package:
A typical scenario:
You create a matrix
you save it as a csv file
You call dbBulkCopy to read fil and insert it using internally bcp tool of MS Sql server.
This assume that your table is already created in the data base:
dat <- matrix(round(rnorm(nrow*ncol),nrow,ncol)
id.file = "temp_file.csv"
write.csv(dat,file=id.file,row.names=FALSE)
dbBulkCopy(conn,'NEW_BP_TABLE',value=id.file)
Using RODBC, the fastest insert we've been able to create (260 million row insert) looks like the following (in R pseudo code):
ourDataFrame <- sqlQuery(OurConnection, "SELECT myDataThing1, myDataThing2
FROM myData")
ourDF <- doStuff(ourDataFrame)
write.csv(ourDF,ourFile)
sqlQuery(OurConnection, "CREATE TABLE myTable ( la [La], laLa [LaLa]);
BULK INSERT myTable FROM 'ourFile'
WITH YOURPARAMS=yourParams;")
If you're running this from between servers, you need a network drive that the R server can write to (e.g. one server with permissions for writing to the DB uses Rscript to productionalize the code), and the SQL Server can read from.
From everything I can find, there is NO solution for bulk insert to MySQL and nothing that works with SSIS which is why Microsoft is including in-database analytics with SQL Server 2016 after buying Revolution R Analytics.
I tried to comment on the previous answer but don't have the reputation to do it.
The rsqlserver package needs to run with rClr and neither of those packages are well-behaved, especially because rsqlserver's INSERT functions have poor data type handling. So if you use it, you'll have no idea what you're looking at in the SQL table as much of the information in your data.frame will have been transformed.
Considering the RODBC package has been around for 15 years, I'm pretty disappointed that no one has created a bulk insert function...
Our n2khelper package can use bcp (bulkcopy) when it is available. When not available it falls back to multiple INSERT statements.
You can find the package on https://github.com/INBO-Natura2000/n2khelper
Install it with devtools::install_git("INBO-Natura2000/n2khelper") and look for the odbc_insert() function.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
The only thing I don't have an automated tool for when working with SQL Server is a program that can create INSERT INTO scripts. I don't desperately need it so I'm not going to spend money on it. I'm just wondering if there is anything out there that can be used to generate INSERT INTO scripts given an existing database without spending lots of money.
I've searched through SQL Server Management Studio Express with no luck in finding such a feature. If it exists in SSMSE then I've never found it.
This web site has many useful scripts including generating inserts.
You can utilise sp_msforeachtable with it to generate for an entire DB.
Update: There is built-in functionality to script data as INSERTs in SQL Server Management Studio 2008(onwards).
SQL Server 2008:
Right-click on your database in SSMS, select Tasks –> Generate Scripts, ensure your database is highlighted and click next. Scroll down the options list to the “Table/View Options” section, and change “Script Data” to True.
SQL Server 2012:
Right-click on your database in SSMS, select Tasks –> Generate Scripts. Choose the tables you want to generate scripted INSERTs for, then click Next. Choose where you want to send the output to (such as a new Query Window), and then click the Advanced button (which for some reason is in the Save to File Section!). Scroll down the options list to the “Types of data to script” option and change it to either "Data only" or "Schema and data". Click Next twice.
you can also use this add-in for SSMS that provides this functionality:
http://www.ssmstoolspack.com/
it also provide other useful features as well.
Theres a stored proc called "sp_generateinserts" you can google for it. It will take a table and convert it into insert statements.
Not sure about the express edition but the normal edition of SMSS, you can right click on a tab and script the table as select, insert update to a new window, clipboard or file.
You could also look at MyGeneration or CodeSmith as code generators. I believe they are free and should have soem templates that given a DB will create a bunch of stored procs for you.
If you're just looking to insert test data, and have fewer than 64K rows to insert, you could use Excel (that's what I do).
For example, if you put values in cells A1, B1, and C1, then entered the following formula in D1, you'd get a usable insert statement:
="INSERT INTO TEST (col1, col2, col3) VALUES ("&a1&","&b1&","&c1&");"
Then just fill down and you can modify the data any time you want.
Not many people know this but you can use sub sonic (http://subsonicproject.com/) to script both the database structure and the data inside it.
The commands are pretty simple ( i use the command line ) eg.
sonic.exe scriptdata /server testserver /db testdb .
This will script all the data into insert statements for you :) , nice and clean.You and also hook this into visual studio if you want to make it easier and not use the command line.
Try DBSourceTools. http://dbsourcetools.codeplex.com
It has a facility to generate insert scripts for any table in your database.
Microsoft has a free app you can install.
http://www.microsoft.com/downloads/en/results.aspx?freetext=Microsoft+SQL+Server+Database+Publishing+Wizard&displaylang=en&stype=s_basicdownload here
You can generate the insert into statement from this stored procedure.
http://raresql.com/2011/12/20/how-to-generate-insert-statements-from-table-data-using-sql-server/
But normally we are generating insert into statement like this :
Insert into [table] (...) values (...)
But, In this procedure, you can generate insert into statement like this
Insert into [table]
select * from [table] Union ALL
select * from [table]
So, you can view the data before insertion.
Imran
In SQL Server Management Studio Express, right click on a table in the Object Explorer sidebar and select "Script Table As / INSERT To / New Query Editor Window".