Is a million users hitting one file on the server the same as a million users hitting a million different files?
It depends on the web server.
However, static files are likely to be cached in memory, so I would assume that 1 million users accessing the same file will be faster.
Depends on server configuration and software. You'll need to measure.
Related
I'm migrating an openldap database backend from HDB to MDB. To migrate I repopulate all data on the DB. Write performance is 3x faster which is great but searching has a very low perfomance. In HDB I get responses in a second but with MDB it takes 4 seconds (some timeouts also!). I test several flags for mdb but it only improve write operations.
My database has a million of entries and every entry has many attributes. A single search could return 13K of data (or more). I try to make index of almost all attributes accessed more than one time per query. But the only index that seems to impact the performance is "objectClass".
Some ideas or documents?
Thanks in advance!
Have a dedicated DB server with 16 cores and 200+ GB of memory.
Use tempdb a lot but it typically stays under 4 GB
Recently added a dedicated SSD stripe for tempdb
Based on this page should create multiple files
Optimizing tempdb Performance
Understand multiple row files.
Here is my question:
Should I also create multiple tempdb log files?
It does say "create one data file for each CPU".
So my thought is that data means row (not log) files.
No, as with all databases, SQL server can only use one log file at a time so there is no benefit at all in having multiple log files.
The best thing you can do with log files really is keep them on separate drives to the data files as they have different IO requirements, pre-size them so they don't have to auto grow and if they do have to autogrow, make sure they do so at a sensible level to manage the number of virtual log files that are created inside them.
I have a database with 200 million records and I need to support 200 write tps. How many partitions do you recommend to use?
One. Don't bother. Partitions will slow you down for writes
It's far more important for writes to have a dedicated, fast volume for your transaction log file (the LDF file) for that database alone. Don't add log files either: one LDF on one volume only.
This is because of write ahead logging: One and Two. Simply, a data page may not be written to disk immediately, but your associated log entry must be confirmed as written for any given transaction
my website-hoster is allowing me unlimited amount of MS SQL databases - but each database may only be a max of 3GB.
My Users database is separate from my Main database. The Main db contains several tables consisting of strings and numbers (no blobs), for example: "Messages", "BugReports", "UserOptions"
I am allowing offline messages between users - when a message can't be delivered it is stored in the "Messages" table. This works great, but I worry: in the long term, what happens when the website gets a lot of traffic and the database nears 3GB ?
Should I make the "Messages" and "BugReports" tables separate databases instead ?
If you store them in a separate database, they will still fill that one up, wouldn't they? So you'll need to shard the Messages table across a number of databases.
If you 're site is so successful to get enough traffic that you run into the 3Gb limit, then look at changing the provider to someone that allows bigger databases, or a dedicated host, or collocation. Yes, you can argue that if you make your app work on 3 GB shards you can scale easier later, but you'll spend a lot of effort to get this working to surpass such an artificial limit. Spend you effort on features useful to your end users instead. With a good design, 3Gb should go a long mile.
Host MyPhpAdmin on ur server. I'm not sure who your hosting with but a lot of them offer unlimites disk space. That way you will have unlimited databases with unlimited capacity.
I have read that it is a good idea to have one file per CPU/CPU Core so that SQL can more efficiently stream data to and from the disks. Ok, I can see the benefit if they are on different spindles, but what if I only have one spindle (4 drives in Raid 10) for my data files (.mdf and .ndf), will I still benefit from splitting the data files (from just the .mdf file to a .mdf and several .ndf files)? Same goes for the log file, although I see no benefit to it as the data has to be written serially and you're limited by the spindle's sequential write speed...
FYI, this is in regards to SQL Server 2005/2008...
Thanks.
The recommendation for multiple tempdb data files is definitely not about IOPS. It is about contention on the allocation pages (GAM, SGAM, PFS) in tempdb. SQL 2005+ doesn't require as big of a load on these pages, but contention still occurs. Not all system require a 1 file to 1 core mapping. Most sytems will perform well with 1 file to 2 or 4 cores. Having too many files adds overhead for managing the files. A good recommendation is to start with 1:4 or 1:2 and increasing if contention continues. Don't go above 1:1.
For other databases, this is not recommended.
And yes, only 1 log file ... always.
8 Steps to better Transaction Log throughput:
Create only ONE transaction log file.
Even though you can create multiple
transaction log files, you only need
one... SQL Server DOES not "stripe"
across multiple transaction log files.
Instead, SQL Server uses the
transaction log files sequentially.
Misconceptions around TF 1118:
Why is the trace flag not required so
much in 2005 and 2008? In SQL Server
2005, my team changed the allocation
system for tempdb to reduce the
possibility of contention. There is
now a cache of temp tables. When a new
temp table is created on a cold system
(just after startup) it uses the same
mechanism as for SQL 2000. When it is
dropped though, instead of all the
pages being deallocated completely,
one IAM page and one data page are
left allocated, and the temp table is
put into a special cache. Subsequent
temp table creations will look in the
cache to see if they can just grab a
pre-created temp table 'off the
shelf'. If so, this avoids accessing
the allocation bitmaps completely. The
temp table cache isn't huge (I think
it's 32 tables), but this can still
lead to a big drop in latch
contention in tempdb.
So the answer is NO to both questions. Log striping was never an issue, and one-NDF-per-CPU is largely a myth, one that will take a very long time to die out. Multiple files IMHO make sense only if you can stripe IO (separate LUNs). Multiple filegroups though make sense, but not for IO reasons, for administrative purposes: piecemeal restores and archive read-only filegroups.
Still good. This is not about IOPS - it is about SQL Server BLOCKING a file for certain operations. mostly when file extends are allocated to a table / index. If you do a lot of inserts / updates, multiple files basically mean another thread will block another file, not wait on the first one.
So, this is not really about IOPS loads, it is about a blocking behavior.