I restored a db with a .bak of the production db, on the development environment. Then I truncated a few, big tables so the space used became the half, but the datafile size obviously is now the same. Usually I shrink the datafile and then set the initial size to value slightly bigger than the used space. This time it seems I cannot do both things. I tried to increase the size and it works, but when I shrink it returns to the previous size I upgraded from. What can I do?
Related
We are trying to upload 80 GB of data in 2 host servers each with 48 GB RAM(in total 96GB). We have partitioned table too. But even after partitioning, we are able to upload data only upto 10 GB. In VMC interface, we checked the size worksheet. The no of rows in the table is 40,00,00,000 and table maximum size is 1053,200,000k and minimum size is 98,000,000K. So, what is issue in uploading 80GB even after partitioning and what is this table size?
The size worksheet provides minimum and maximum size in memory that the number of rows would take, based on the schema of the table. If you have VARCHAR or VARBINARY columns, then the difference between min and max can be quite substantial, and your actual memory use is usually somewhere in between, but can be difficult to predict because it depends on the actual size of the strings that you load.
But I think the issue is that the minimum size is 98GB according to the worksheet, meaning if any nullable strings are null, or any not-null strings would be an empty string. Even without taking into account the heap size and any overhead, this is higher than your 96GB capacity.
What is your kfactor setting? If it is 0, there will be only one copy of each record. If it is 1, there will be two copies of each record, so you would really need 196GB minimum in that configuration.
The size per record in RAM depends on the datatypes chosen and if there are any indexes. Also, VARCHAR values longer than 15 characters or 63 bytes are stored in pooled memory which carries more overhead than fixed-width storage, although it can reduce the wasted space if the values are smaller than the maximum size.
If you want some advice on how to minimize the per-record size in memory, please share the definition of your table and any indexes, and I might be able to suggest adjustments that could reduce the size.
You can add more nodes to the cluster, or use servers with more RAM to add capacity.
Disclaimer: I work for VoltDB.
I have a situation where initially the space of temp db is 8mb only.I have created a temp table say #ABC. Filled with some data.Now space of temp db is 400 mb. after i dropped #ABC the temp db is not getting to 8mb.It has still 400 mb space.what i need to get it to 8mb.
I am asking this question because i am facing a situation where temp db completely takes all memory of hard drive(500GB) after deleting the temp table's in ssrs.I really need to solve this and i need to restart the sql server each time to release the space.
Thank you for any help.
Assuming you mean the operating systems files holding tempdb, these do not Auto Shrink by default and most people would think that was a bad idea.
If your process needs all that space to do it's work, then you should give it sufficient space to do so. Shrinking will only postpone the issue as next time the process runs it will need to get that same amount of space again. It might fail or go into a death spiral if free space has shrunk inbetween runs.
I have a database that needs to support many INSERTs, UPDATEs and DELETEs.
The database keeps live data for last 3 months. The aount of data is huge - the size of the database is around 0.5TB.
Each day I run a process that archives older than 3 months data. Of course, deleting the data won't free up the resulted space. To reclaim the disk space I need to (auto)shrink the database. This of course is followed by (auto)grow when more database space is neded.
On the other hand we know that shrinking the database is evil!
One solution that crossed my mind is to set a fixed size for my database from 0.5TB to 0.8TB or even 0.9TB or whatever hoping that the free space resulted after archiving the data (that is delete older data) will be reused by the database later.
Can anyone confirm that this way I can avoid the need for (auto)shrink + (auto)growth? Of course, we assume that the fixed size is big enough.
Why don't you just set up a scheduled task to shrink the database every n days.
Fixed size is risky - easy to fill the database up, and block live access.
I have an SSIS package which suppose to take 100,000 records loop on them and for each one save the details to few tables.
It's working fine, until it reaches somewhere near the 3000 records, then the visual studio crashes. At this point devenv.exe used about 500MB and only 3000 rows were processed.
I'm sure the problem is not with a specific record because it always happens on different 3K of records.
I have a good computer with 2 GIG of ram available.
I'm using SSIS 2008.
Any idea what might be the issue?
Thanks.
Try increasing the default buffer size on your data flow tasks.
Example given here: http://www.mssqltips.com/sqlservertip/1867/sql-server-integration-services-ssis-performance-best-practices/
Best Practice #7 - DefaultBufferMaxSize and DefaultBufferMaxRows
As I said in the "Best Practices #6", the execution tree creates
buffers for storing incoming rows and performing transformations. So
how many buffers does it create? How many rows fit into a single
buffer? How does it impact performance?
The number of buffer created is dependent on how many rows fit into a
buffer and how many rows fit into a buffer dependent on few other
factors. The first consideration is the estimated row size, which is
the sum of the maximum sizes of all the columns from the incoming
records. The second consideration is the DefaultBufferMaxSize property
of the data flow task. This property specifies the default maximum
size of a buffer. The default value is 10 MB and its upper and lower
boundaries are constrained by two internal properties of SSIS which
are MaxBufferSize (100MB) and MinBufferSize (64 KB). It means the size
of a buffer can be as small as 64 KB and as large as 100 MB. The third
factor is, DefaultBufferMaxRows which is again a property of data flow
task which specifies the default number of rows in a buffer. Its
default value is 10000.
Although SSIS does a good job in tuning for these properties in order
to create a optimum number of buffers, if the size exceeds the
DefaultBufferMaxSize then it reduces the rows in the buffer. For
better buffer performance you can do two things. First you can remove
unwanted columns from the source and set data type in each column
appropriately, especially if your source is flat file. This will
enable you to accommodate as many rows as possible in the buffer.
Second, if your system has sufficient memory available, you can tune
these properties to have a small number of large buffers, which could
improve performance. Beware if you change the values of these
properties to a point where page spooling (see Best Practices #8)
begins, it adversely impacts performance. So before you set a value
for these properties, first thoroughly testing in your environment and
set the values appropriately.
You can enable logging of the BufferSizeTuning event to learn how many
rows a buffer contains and you can monitor "Buffers spooled"
performance counter to see if the SSIS has began page spooling. I
will talk more about event logging and performance counters in my next
tips of this series.
I was wondering why my CouchDB database was growing to fast so I wrote a little test script. This script changes an attributed of a CouchDB document 1200 times and takes the size of the database after each change. After performing these 1200 writing steps the database is doing a compaction step and the db size is measured again. In the end the script plots the databases size against the revision numbers. The benchmarking is run twice:
The first time the default number of document revision (=1000) is used (_revs_limit).
The second time the number of document revisions is set to 1.
The first run produces the following plot
The second run produces this plot
For me this is quite an unexpected behavior. In the first run I would have expected a linear growth as every change produces a new revision. When the 1000 revisions are reached the size value should be constant as the older revisions are discarded. After the compaction the size should fall significantly.
In the second run the first revision should result in certain database size that is then keeps during the following writing steps as every new revision leads to the deletion of the previous one.
I could understand if there is a little bit of overhead needed to manage the changes but this growth behavior seems weird to me. Can anybody explain this phenomenon or correct my assumptions that lead to the wrong expectations?
First off, CouchDB saves some information even for deleted revisions (just the ID and revision identifier), because it needs this for replication purposes.
Second, inserting documents one at a time is suboptimal because of the way the data is saved on disk (see WikiPedia), this could explain the superlinear growth in the first graph.