I've searched for what should be a seemingly simple thing, and I can't find a reference to this issue anywhere. I have a very simple Azure Sync setup... one master database that needs to do a one way sync to three client databases.
It's only syncing three tables right now (all fields), and there's no filtering at all. I've verified that the schema on all four databases is identical and my sync process returns with no errors, but when I check the data, it's only ever updated one of the three client databases.
Like I say, this should be really simple. I've tried clearing the tables and re-adding them, even deleting the whole Sync Group, but no matter what I do, only one database updates. Any idea what I could be missing, or does Azure only allow one table to be sync'd?
Related
Currently I have 3 (same code base apps) with it's own databases and own unique data. Were moving towards doing multi tenancy in rails, after a couple of prototype testing we've decided to go for a shared tenancy. My only biggest problem is that, each databases have their own data with unique ids and etc. How would it be possible to merge them either via sql command/dump or rails script that way they will have their own account_id + keep all data integrity?
Absolutely doable. It depends on a lot of details.
Basically I would
Make a full backup of all three.
Prep each database to hold compatible data (no duplicates).
Select one to be the new master.
Dump the other two (data only).
Hack the dump, to make sure. Typical COPY statements in dumps are just fine.
Restore data from the two additional database on top of existing data in the master.
Make sure all sequences are set properly.
Run vaccumdb -fz master.
I have a structured database and software to handle it and I wanted to setup a demo version based off of a simple template version. I'm reading through some resources on temporary tables but I have questions.
What is the best way to go about cloning a "temporary" database while keeping a clean list of databases?
From what I've seen, there are two ways to do this - temporary local versions that are terminated at the end of the session, and tables that are stored in the database until deleted by the client or me.
I think I would prefer the 2nd option, because I would like to be able to see what they do with it. However, I do not want add a ton of throw-away databases and clutter my system.
How can I a) schedule these for deletion after say 30 days and b) if possible, keep these all under one umbrella, or in other words, is there a way to keep them out of my main list of databases and grouped by themselves.
I've thought about having one database and then serving up the information by using a unique ID for the user and 'faux indexes' so that it appears as 1,2,3 instead of 556,557,558 to solve B. I'm unsure how I could solve A, other than adding a date and protected columns and having a script that runs daily and deletes if over 30 days and not protected.
I apologize for the open-ended question, but the resources I've found are a bit ambiguous.
These aren't true temp tables in the sense that your DBMS knows them. What you're looking for is a way to have a demo copy of your database, probably with a cut-down data set. It's really no different from having any other non-production copy of your database.
Don't do this on your production database server.
Do not do this on your production database server.
Script the creation of your database schema. Depending on the DBMS you're using, this may be pretty easy. If you've got a good development/deployment/maintenance process for your system, this should already exist.
Create your database on the non-production server using the script(s) generated in the previous step. Use an easily-identifiable naming convention, like starting the database name with demo.
Load any data required into the tables.
Point the demo version of your app (that's running on your non-production servers) at this new database.
Create a script/process/job which looks at your database server and drops any databases that match your demo DB naming convention and were created more than 30 days ago.
Without details about your actual environment, people can't give concrete examples/sample code/instructions.
If you cannot run a second, independent database server for these demos, then you will have to make do with your production server. This is still a bad idea because of potential security exposures and performance impact on your production database (constrained resources).
Create a complete copy of your database (or at least the schema, with a reduced data set) for each demo.
Create a unique set of credentials for each of these demo databases. This account should have access to only its demo database.
Configure the demo instance(s) of your application to connect to the demo database
Here's why I'm pushing so hard for separate databases: If you keep copying your "demo" tables within the database, you will have to update your application code to point at those tables each time you do a new demo. Once you start doing this, you're taking a big risk with your demos - the code you keep changing isn't really the application you're running in production anymore. And if you miss one of those changes, you'll get unexpected results at best, and mangling of your production data at worst.
I want to create a web application that is supposed to contain a lot of data. I want to ask if anyone of you have ever met a system that contained two databases - main and archive. I want to create a mechanism that will move old data from main database to archive database in order to unload it. For instance, when I have a table of user accounts, I want to move the ones that weren't used for, say, more than three months to an archive database. Having this done, main database may be significantly unloaded so I expect it to work faster. However such mechanism has to work in two directions - not only migrating from main to archive but also from archive to main db in order to allow user's to "refresh" their accounts. Of course in such scenario I will use GUID's instead of BIGINT's as PRIMARY KEY. What do you think about it? Is such concept right or I shouldn't bother about it and assume that there should be only one database? Thanks in advance.
Having archive database never hurts, but usually it's used for restoring or reporting. I think in most cases partitioning will serve your purpose better. Also, many RDMS systems propose different solutions out of the box, like database clustering, mirroring, etc.
I have a database server with few main databases, and few dozens of small ones.
These small databases are kind of intermediary/staging databases for data import from various sources into main database. Data import is a daily task. They are all quite similar in structure as the implementation of these data imports are similar, so basically they have a configuration tables, which define mapping, conversions etc, and the data tables, which contain the results of the import.
Some time ago there have been only the handful of small ones, but now I have more then 20 of them will grow further with the number of supported data feeds.
I have just migrated all the server environment to SQL Server 2008, and having some time now for clean-up/refactoring, I am thinking to merge all of data-import databases into just one database, and use database schema to separate them.
Question-0: Any other ideas for the described situation?
Question-1: Shall I change from a separate database to a separate schema?
Question-2: !!!: Any tricky thing to be careful about in database schema implementation?
Edit-1: highlighted question-2 as the most 'unanswered' currently.
In your instance, I would probably put merge the databases into one. I don't really see a reason to have them separated, and merging them will reduce the amount of work you have to do to support backups etc. If you were importing data from a data source once and then never using the staging tables again, I could see the reason to bring up separate databases to handle the data transformation. Since you use these tables on an ongoing basis, I would much rather keep them together so that I only have to go to one place to find the full end to end state of the production data and the data load states.
2008 is really good at handling database partitioning too, if the db gets too large, or you need to separate data for security reasons you get the benefit of having a single db with the advantages like having several smaller ones. You won't get that with multiple smaller dbs.
When we migrated we had a very similar situation and I ended up moving everything into one some-what large Importing database like you have hinted towards. We did not, however, separate them using schemas.
Because the database is the unit of referential integrity and backup, if you are bringing in large amounts of data for staging which does not need to be backed up on the same schedule, it might be easiest to keep it in a separate DB.
You can use a single DB with multiple file groups and different backups, but it will require a lot more design.
The basic factors this will depend on are: recovery model, backup objectives, usage patterns and amount of effort to design and maintain your file group design.
All the prior answers work for me, particularly your comment about selectively combining databases -- if some are very busy, very large, or process sensitive data, you might want to keep them separate, or in separate groupings. This would make it easier to configure backups/restores and disk/drive allocation (give the busy ones their own set of spindles).
Like possibly most database developers, I have dealt almost exclusively with objects in the dbo schema, but I have done some recent work with other schemas. The main gotcha I've encountered is remembering to always specify the schema when referring to any database object. Never assume that any given connection will reference an object in the schema you want it to--always be clear and precise!
I would put all your import staging tables in one database separate from your regular production databse as the backup needs may be very different. This database should also contains things like your configuration management for SSIS packages, any logging tables, any import metadata tables (we keep track of every run of the imports and the status of that run as well as a bazillion other things about the import like the filename, the normal file size, etc. Comes in handy for researching problems and for adding checks to the processing. We usea a schema that is by client and then an additional schema for objects realted to the importing/exporting process (logs, meta data etc.)
Most questions about tenancy are centered around multi-tenancy database design issues. I want to know about single tenancy but multiple applications. The software I'm developing allows for a single user to create, from a single code base, multiple applications(I call them "sections"):
user could create a blog inside domain.com/application-blog1, another blog on domain.com/application-blog2.
I've already decided for a single database for everything But I am undecided whether or not I should use multiple tables for different application instances or the same table, maybe with a "sectionId" field to distinguish between them.
I'm using mysql and myisam tables. Could storing everything inside the same table lead to locking issues in the case of having many application instances running?
What's your experience on the subject?
I don't think people will typically use multiple tables in the same database. If you have multiple instances of the same application, you often have separate databases - typically only if new instances are created administratively, rather than through end-user actions. In this case, you'ld put the name of the database into a configuration file, and have the software connect to the right database.
In your case, I would go for the single-schema single-database approach, using sectionIds. This really is the same as multi-tenancy, perhaps minus the need to do access control.
You will of course have locking across concurrent transactions. However, this should never cause problems, since transactions for different sections won't operate on records in a conflicting manner (except when new sections are created - you'll probably have another table telling you what sections you have).