I am working in a system which gets reloaded frequently. When the system is reloaded a lot of files can be deleted or changed back to a previous stable build version (it's a dev environment). The only thing that doesn't get reloaded is a database.
I have been tasked with inserting and deleting .sql scripts into the environments so that if required someone can run them at a given time. Usually 2 weeks after the files have been added to the system.
Initially I had thought of creating a directory to keep the files, and make a table where one of the columns keeps the path to these .sql files. This would allow for someone to query the database and using the path they could execute a desired script. The problem is the environment constantly reloading which could result in losing files.
Am I correct in assuming that the correct way to approach this is to use BLOB datatype to store the .sql files? The database isn't reloaded so no files would be lost. I am new to SQL and I'm unsure what is the correct approach to this. Would Varchar also work? Is one datatype more efficient than the other?
I wouldn't put SQL scripts in a SQL database, it doesn't feel right.
Sql Scripts usually go in a git repo, in order to be versioned and have better visibility.
Related
I have a homework project in PL/SQL, we have to create database (.sql files).I have main directory with subdirectories "tables", "views", "procedures", "triggers", they contain .sql files that create associated objects for database. I know the correct order to run those files to create entire database from them. We have to send zip with them.
I want to create single .sql, or some script, or something alike that can run all those .sql files in correct order. Just login to DataGrip or anything like that, double click, and create tables, than views etc. (in order specified by me).
What is the proper way to do that? I'm quite sure there must be a way other than doing it by hand or copying everything into one huge .sql file (which I did and it works, but is ugly as hell).
Our company is in the process of adapting TFS for source repository and project management. I am in charge of database part of the project. We are using SQL Server 2008 R2, Visual Studio 2012 and TFS Online. We have a database that is used by several of our applications. So far I have been the only one handling any change to this database. As the company is expending we are going to have multiple dev teams. So I am planning to save the database as as SSDT project to TFS.
At the moment I am maintaining my database like the following:
I have separate folders for UDFs, Stored Procedures, and Config.
Under these folders I have subfolders for each objects. For example, for stored procedures I have subfolders for each stored procedure which contains the SQL script to create the SP. The config folder contains any script similar to SSDT's post deployment script (for example, populating static data).
The SQL script contains code to drop the procedure and create it.
I have a c# app to concatenate all the SQL files into one single SQL file. Let's call it the FINAL script. When creating FINAL script I can specify version number which adds an update statement to update the version table on the database.
FINAL script is made available for customers to download and execute on the database. So the script mainly contains any add/edit to SPs, UDFs, and static data. It does not touch any existing data (data entered by user) in most cases.
As a newbie to TFS and SSDT I am not exactly sure how this can be done using SSDT/TFS or if there is better way of doing something similar. So far what I have understood about SSDT and TFS is:
I can import an existing database to SSDT project.
This will create scripts for all objects including tables.
I can easily do a publish of the database to a local server or to a server I have access to.
Things that seem confusing so far:
How do I supply clients with my latest update script? I am thinking of manually including the FINAL script to the SSDT project but there must be better way of doing it.
How do I publish the changes to a copy of the database without the loss of any user-entered data? My guess is when publishing the tables get created. I can take care of the static data but I am not sure how to handle data entered by users.
May be there is something fundamentally wrong in my understanding of this whole thing. That is why I am here... :)
You want to pull your DB into a SQL Project. Maintain all of your changes there. This tells your system what the schema of your database should be. From there, I'd generate the dacpac files (through building the project) and provide those to your clients along with having them install the SSDT tools that include SQLPackage. They can run SQLPackage to make changes to their database to handle the schema changes automatically. This will bring their database in line with your schema, no matter how far off it might be.
I'd also create a publish profile for them to use. This lets you control some of the settings.
You can choose to not drop any objects not in your project
You can choose to ignore users/permissions
You can set an option to not allow changes if there would be data loss.
You can wrap everything in a transaction so a failed update rolls back
If you give them a batch file to run, you can specify an output file or a Diff report, or have them generate their own script to do the update.
I blogged about this at http://schottsql.blogspot.com/2013/10/all-ssdt-articles.html
(or http://schottsql.blogspot.com/search/label/SSDT if that doesn't work well). That will take you through some basics of why you might want to use SQL Projects, creating them, maintaining them, and publishing the changes to an existing database.
Please note:
I am a game programmer, so backend development isn't my forte. There are times, however, where I work with our database at my job. Please don't shoot me if my question is ridiculous.
Is there a way to create a local mySQL file and access it through PHP or C#?
I know you can make a local webpage on your machine (pretty much for testing purposes) and access multiple locally created files.
I assume that something similar would work with mySQL. (Are the login credentials also stored within the file?) I remember seeing a few online tutorials where it offered a download for both PHP and the database file, but I can't seem to find them now.
I've searched for this, but all the relevant results involved downloading mySQL and hosting a server which is a bit more than I wanted to do.
So if its possible to create a local mySQL, how do you do so?
The tools I intend on using while doing this:
PHP/JQUERY/HTML and C#
For MyISAM tables, inside the MySQL data directory there is one directory per database which contains several (usually three) files per table. For InnoDB tables, they are all contained in several files directly inside the data directory.
The location of the MySQL data directory is usually set in my.cnf using the datadir parameter.
The login credentials are stored in a special database called "mysql" which is in that data directory like any other database.
However, you have to install and run MySQL to access those files. You cannot access them with PHP or any other client API alone. If you want to do such a thing, better use SQLite.
MySQL is a database engine, u need to install that before you can use it. Unlike SQLite which stores it's database in files. Maybe that is something more of your liking. And I know there are library that supports SQLite for PHP, not sure about the rest.
SQLite you don't need to install anything.
MySQL can be used an an embedded database, but you will need to contact them in order to purchase a copy of it.
I was wondering if there is a way to automatically append to a script file all the changes I am making to my columns, tables, relationships etc...
The thing is I am doing a lot of different changes on a TEST db and the idea will be to apply this change script when I move the test db to production... hence keeping production data but applying all schema and object changes.
Is there an easy way to do this? Can it also migrate database diagram changes?
I have seen how you can create a change script each time I do a change but this means I have to copy and paste into a master file. Actually pretty easy!
I was just wondering if I was missing something?
Do not make changes to the test server using the UI. Write scripts and keep them under source control. You can test your scripts starting from backups of the live data and you can tune yoru scripts untill they achieve the desired result. Then you can check in the scripts for reference and later apply them on the live server. See this article Version Control and Your Database.
BTW, check out the SSMS toolpack, I think it may do what you want (I'm not sure). My advice stand none the less: version your schema, use explicitly created/saved scripts, use source control.
There's no way to directly generate a "delta" script in SSMS.
However, if every time you publish changes, you script out the entire database, including data, to SQL using the SQL Server Database Publishing Wizard you should be able to extract diffs between the versions and get your deltas that way.
If money is no object, you can purchase Visual Studio Team System Database Architect edition and use its fantastic database comparison tools to generate and version control exactly the diffs you want.
Try using TableDiff , that came with SQL Server 2005.
SQL Server 2005 TableDiff Utility
tablediff Utility
We have the process where when a developer gets done with a change, they then script it out and check it into Subversion. In Subversion we have a folder for Tables, Stored Procs, Data, etc. They script it out so it is repeatable (i.e. don’t insert the new data if it is already there.) This is important to do anyway so you keep the history of changes for a given object in the database.
In the past, we would just enter each of the files that we wanted scripted out into a text file (i.e. FileListV102.txt). When we were ready to make a release we would do “get latest” on all of the files (from VSS back then.) We then had a simple utility that would read the “file list” file and open each of those files in turn concatenating them into an output file. That is pretty easy to code.
We outgrew that and now we have a release management tools (which can be found here and will be on sale mid September), that takes all of the files and creates a big SQL script file out of it. It does it in the order that you would expect based on the folder names – so files found in the "Tables" folder are done before those in the "Data" folder, etc.
Either way, once you are done you have a big SQL script file that you can then apply to a fresh copy of production and that is what you test against.
I know I'm way late to the party, but I just wanted to add that there are tens of third party products out there. Some are very good, some are very cheap or free, and some are a mixture. I listed 22 here:
http://bertrandaaron.wordpress.com/2012/04/20/re-blog-the-cost-of-reinventing-the-wheel/
We have been using a relatively new software called Kal Admin.
It has Change Management feature and let distributing selected changes to other databases very easily. We used to do it by comparing two databases but it not satisfy our need for change tracking.
BTW Kal Admin has Metadata and data compare capabilities as well.
I have a VB project that runs on SQL SERVER 2005, while making the setup file for it, how do I include the DB?
You don't
Typically you have a DB generation script that is run either as part of setup or as part of first run of application
You also need to consider migrations (changes to DB when new releases of your application are published)
Consider using MigratorDotNet or RikMigrations to solve these problems in a seperate installer/upgrade program if you are still using VB6
I disagree, you could include the database. Simply distribute the .MDF file with your application.
Of course, the setup application would have know how to attach the database to an existing SQL Server RDBMS.
Both methods given in the above answers will work. I have tried them both. However using a
db generation script reduces the size of the final deployment files considerably. I would launch the script on the first run of the application and not in the setup itself.
I will second jack on this one.
From my experience of using installs that require an actual database file tend to have more issues then when updating or on first install when running scripts. As jack mentioned another bonus is reduced file size.
You can create who database scripts by right clicking on the required database, and selecting the script database option. Note however this will only create the tables and fields and not replicate any data.