In our company we using many DB servers in different cities. Sometimes data in one server should be synchronized with another. For example, in table "Monitor" values "status" and "date" may be updated very often. My problem is when theese values updated in server A, they also should be updated in server B:
Update Monitor set(date='2013-06-13')
and then
'Update Monitor set(status=4)'
in server A udating of both values is sucsessfull, but in server B (usualy with highest loading) somtimes, in approx. 0.03% cases updated only value date and status is stil old. Can anybody explain, is it possible in DB server with high loading?
It's hard to explain without looking at the boxes, logs and workload each is doing; there are a thousand things that would cause server "B" to miss data, including table and row locks, requests dropped by the network, unfinished transactions and the like. To find out exactly, you'd have to turn on the logging and compare the requests on "A" versus "B". The first thing I'd do, however, would be to look for errors in the SQL logs.
But in general keeping database synchronized across regions is do-able using existing technologies available in MS and Oracle. One scenario involves using a master, central db to receive all requests. It then distributes inserts, updates, delete and queries out to the regional DBs using SSIS or regular DB connectivity over a WAN.
Here's a high-level guide to the technology solution available in SQL Server.
http://msdn.microsoft.com/en-us/library/hh868047.aspx
You were probably looking for a simple answer, but I don't think there is one.
Related
I want to understand the best approach for SQL Server architecture on production environment.
Here is my problem:
I have database which has on average around 20,000 records being inserted every second in various tables.
We have reports also implemented for the same, now what's happening is whenever reports is searched by user, performance of other application steeps down.
We have implemented
Table Partitioning
Indexing
And all other required things.
My question is: can anyone suggest an architecture that have different SQL Server databases for reports and application, and they can sync themselves online every time when new data is entered in master SQL Server?
Some what like Master and Slave Architecture. I understand Master and Slave architecture, however need to get more idea around it.
Our main tables are having around 40 millions rows (table partitioning done)
In SQL Server 2008R2 you have database mirroring and replication available, which will keep two databases in sync.
A schema which is efficient for OLTP is unlikely to be efficient for large volume reporting. The 'live' and 'reporting' databases should have different schema with an ETL process moving data from one to the other. I'd would like to negotiate with the business just how synchronised the reporting database needs to be. If the reports are processing large amounts of data they will take some time to run so a lag in data replication will not be noticed, I would suggest. In extremis you could construct a solution using Service Broker to move the data and processing on the reporting server to distribute it amonst the reporting tables.
The numbers you quote (20,000 inserts per second, 40 millions rows in largest table) suggests a record doesn't reside in the DB for long. You would have a significant load performing DELETEs. Optimising these out of peak hours could be sufficient to solve your problems.
We have an AS400 mainframe running our DB2 transactional database. We also have a SQL Server setup that gets loaded nightly with data from the AS400. The SQL Server setup is for reporting.
I can link the two database servers, BUT, there's concern about how big a performance hit DB2 might suffer from queries coming from SQL Server.
Basically, the fear is that if we start hitting DB2 with queries from SQL Server we'll bog down the transactional system and screw up orders and shipping.
Thanks in advance for any knowledge that can be shared.
Anyone who has a pat answer for a performance question is wrong :-) The appropriate answer is always 'it depends.' Performance tuning is best done via measure, change one variable, repeat.
DB2 for i shouldn't even notice if someone executes a 1,000 row SELECT statement. Take Benny's suggestion and run one while the IBM i side watch. If they want a hint, use WRKACTJOB and sort on the Int column. That represents the interactive response time. I'd guess that the query will be complete before they have time to notice that it was active.
If that seems unacceptable to the management, then perhaps offer to test it before or after hours, where it can't possibly impact interactive performance.
As an aside, the RPG guys can create Excel spreadsheets on the fly too. Scott Klement published some RPG wrappers over the Java POI/HSSF classes. Also, Giovanni Perrotti at Easy400.net has some examples of providing an Excel spreadsheet from a web page.
I'd mostly agree with Buck, a 1000 row result set is no big deal...
Unless of course the system is looking through billions of rows across hundreds of tables to get the 1000 rows you are interested in.
Assuming a useful index exists, 1000 rows shouldn't be a big deal. If you have IBM i Access for Windows installed, there's a component of System i Navigator called "Run SQL Scripts" that includes "Visual Explain" that provides a visual explanation of the query execution plan. View that you can ensure that an index is being used.
On key thing, make sure the work is being done on the i. When using a standard linked table MS SQL Server will attempt to pull back all the rows then do it's own "where".
select * from MYLINK.MYIBMI.MYLIB.MYTABE where MYKEYFLD = '00335';
Whereas this format sends the statement to the remote server for processing and just gets back the results:
select * from openquery(MYLINK, 'select * from mylib.mytable where MYKEYFLD = ''00335''');
Alternately, you could ask the i guys to build you a stored procedure that you can call to get back the results you are looking for. Personally, that's my preferred method.
Charles
I am new to sql server .I have a sql server and I have 2nd sql server(backupserver).I want to copy data from sql server to 2nd one and all my transactions create update delete must be reflected in the 2nd server immediately in real time.I have very large data in my table assuming million of rows.How can I achieve this.I dont have previalge to use third party tools.
You specify that you need data to be reflected on the second server immediately, in real time.
This requirement will add latency to every transaction that occurs on your primary server, since every action will need to be committed on the secondary server before it can be committed on your primary server.
In addition to the latency, this requirement will also likely reduce your availability. If the secondary server is no longer able to successfully commit transactions from the primary, then the primary can no longer commit transactions either, and your system is down.
For more information about these constraints, refer to the extensive discussion around this topic(CAP theorem).
If you're OK with these restrictions, you might consider using Synchronous Database Mirroring (High-Safety Mode).
If you're not OK with these restrictions, please adjust / clarify your requirements.
You may try to get some help from these two references:
Replication in MS SQL Server
SQL Server Replication
On a side note an important thing you should need to plan before doing replication is the Replication model which you will use for your replication.
There is a list of Replication model which you can use:-
Peer-to-peer model
Central publisher model
Central publisher with remote distributor model
Central subscriber model
Publishing subscriber model
Each one of the above has its own advantages. Check them as per your need.
Also to add to it there are three types of Replication:-
Transactional replication
Merge replication
Snapshot replication
Check out this tutorial on SQL Server 2008 replication:
Tutorial: Replicating Data Between Continuously Connected Servers
Since you said "... reflected in the 2nd server immediately in real time", that means you want to use Transactional Replication. You still may only choose to replicate certain tables.
Does the "millions of rows" represent some kind of history? If so, consider the risks that Michael mentioned... and whether you need all the history in the 2nd server, or just current / recent activity. If it's just current/recent, it may be safer and less of a system drain, to write something in T-SQL or SSIS, for a job to execute that loops, reads, and copies the data.
That could be done with linked servers and triggers... but the risks Michael mentions, about preventing the primary server from committing transactions, are as much or more a concern with triggers... that you can avoid with your own T-SQL/SSIS + job.
Hope that helps...
I'm in a satellite office that needs to pull some data from our main office for display on our intranet. We use MS SQL Server in both locations and we're planning to create a linked server in our satellite office pointing to the main office. The connection between the two is a VPN tunnel I believe (does that sound right? What do I know, I'm a programmer!)
I'm concerned about generating a lot of traffic across a potentially slow connection. We will be getting access to a SQL view on the main office's server. It's not a lot of data (~500 records) once the select query has run, but the view is huge (~30000 records) without a query.
I assume running a query on a linked server will bring back only the results over the wire (and not the entire view to be queried locally). In that case the major bottleneck is most likely the connection itself assuming the view is indexed, etc. Are there any other gotchas or potential bottlenecks (maybe based on the way I structure queries) that I should be aware of?
From what you explained your connection is likely to be the bottleneck.
Also, you might also consider caching data at the satellite location.
The decision will depend on the following:
- how many rows and how often data are updated in the main database
- how often you need to load the same data set at satellite location
Two edge examples:
Data is static or relatively static - inserts only in main DB. In satellite location users often query the same data again and again. In this case it would make sense to cache the data locally at satellite location.
Data is volatile, a lot of updates or/and deletes. Users in satellite location rarely query data and when they do, it is always different where condition. In this case it doesn't make sense to cache. If connection is slow and there are often changes you might end up never being at sync with the main DB.
Another advantage of caching is that you can implement data compression, which will alleviate bad effect of slow connection.
If you chose to cache at local location there are a lot of options, but this I believe would be another topic.
[Edit]
About compression: You can use compressed transaction log shipping. In SQL 2008 compression is supported in Enterprise edition only. In SQL 2008 R2 it is available starting Standard version. http://msdn.microsoft.com/en-us/library/bb964719.aspx .
You can implement custom compression before you ship transaction logs, using any compression library you like.
I have a database server that currently has two databases, call them [A] and [B].
[A] is the standard database used for my application
[B] is used by another application (that puts considerable load on the server), however, we use a few tables sparingly (that are shared between my application and the primary application that utilizes [B]). Recently the use of [B] has been increasing and it's causing long wait periods and timeouts in my application.
I'm open to alternative ideas, but at the moment the quickest potential solution I've come up with is to get a second database server and move [A] to that. However, I do need access to some of the tables in [B] - ideally with as little code changes as possible.
We were thinking a setup something like:
Current:
DB Server 1 {[A],[B]} (SQL Server 2005)
New Setup
DB Server 1 {[B]} (SQL Server 2005)
DB Server 2 {[A], [B]} (SQL Server 2008)
Where in the new setup, DBServer2.[B] is a linked table (kind of) to DBServer1.[B]
I looked into linked databases, from my understanding (limited) those work as an alias to a database on another server, which is almost what we want, except it adds the extra server qualifier, and it works on the db level. I'd like to do something like this, but ideally without the server qualifier and on a table level.
So for example, [B] has tables Users and Events. The Users table is updated weekly by batch, and we use it often, so we'd like to have a local copy on the new DBServer2. However, Events we use far less often and needs to be real-time between the two servers. In this case, it would be great to have Events as a linked table.
Further, it would be fantastic if we could use the same markup for queries. Today to query one db from the other we'd do something like
select * from b.events join a.dates
We'd like to continue that, except have the database server know that when we touch events it's really located at dbserver1.b.events.
Does that make sense? I confuse myself sometimes.
Thanks for any help
~P
You can use synonyms for linked objects - http://msdn.microsoft.com/en-us/library/ms177544%28SQL.90%29.aspx
This unfortunately only works for single objects, you CANNOT make a synonym for linkedserver.databasename and then reference synDBName.table.
"Alternative Idea" from me since you said you are open...
How about looking into the cause of the slowness? What is the "Load" you are measuring?
How are your disks laid out on the server?
Maybe you could use more memory, another CPU or Some SQL tuning?
Fixing your issues with a software or hardware fix MAY be faster than getting a server and doing all the installs and then working through the integration problems you may run into.