We had recently upgraded our 8 host MarkLogic cluster from ML8 to ML9 (9.0-8.2). We enabled telemetry for our cluster and we did see around 1 MB data is getting uploaded to telemetry in ErrorLog.txt. We had a system /tmp space of 5GB but whereas in Memory,Disk, Swap space requirements (https://docs.marklogic.com/9.0/guide/installation/intro#id_11335) it says
"System temp space sizing - when using telemetry allow for 20 GB maximum in system temp space, although normal usage will likely be less than 100 MB."
My concern is do we face some performance issues if we didn't increase /tmp to 20 GB?
Related
I know that more number of caches open too many file descriptors and consume more resources.
Is there a recommendation on number of caches per ignite/GridGain cluster?
Is there a recommendation on number of caches vs number of nodes vs OS configuration (CPU, RAM)
We have 45 caches (PARTITIONED) and the system configuration is 4 CPU and 60GB RAM on each node. It is a 3 node cluster.
Current data storage size is 2GB and Data is expected to grow 1.5-2 TB in next one year.
We are frequently getting "Too many open files" error.
First of all there's nothing wrong with increasing the limit for file descriptors on the OS level. You can use the ulimit utility for that.
Another option is to leverage cache groups, it will make caches share some structures including files.
I'm currently doing some cleanup on an Azure environment and just wanted to check calculations. I have about 55 Databases on an elastic server, which is currently sitting at 3.64TB of the maximum 4TB. Having a look at each of the databases within this pool, I can see that they have their own Used / Allocated / Maximum sizes. Each of these is ranging between 0.1% and 80% of their allocated 250GB. Is the allocated size of the elastic pool dependent on the maximum sizes of each of the databases within that elastic pool? IE if I took a database that is using 1GB of 250GB and reduced the maximum size of this database within the elastic pool from the default 250GB down to 20GB, would it have any positive/negative implications? If anyone can suggest any good resources for azure environment maintenance plans it would be greatly appreciated as I'm coming from an AWS background.
Your allotted area will immediately expand; you do not need to be concerned; this is typical. You'll see that occupied space is always close to available space. The key is when the total amount of allotted and utilised storage space approaches the maximum storage size.
If the database size reaches the maximum size, use the following statement to raise the maximum size or use Azure portal to alter the maximum size.
ALTER DATABASE AzureDB2 MODIFY (EDITION='STANDARD', MAXSIZE= 50 GB)
The database receives a certain amount of log space depending on the tier. When you specify the storage size in vCore, you are given a fixed amount of space for your logs. If you pick 1TB of storage, for example, 300GB is set aside for logs.
I have a Docker image that is running as a Fargate task. I am curious to know how AWS bills for the use of it. Currently I have a hard limit of 1GB and a soft limit of 512MB. If I bump the hard limit up to 2GB to avoid memory issue in certain cases, will I be charged for 2GB all the time or only the period that the container needs it? For most of time my application does not even need 512MB but occasionally it needs 2GB.
Visit here for pricing details
https://aws.amazon.com/fargate/pricing/
The lowest vCPU is 0.25 which provides memory upto 2 GB and is charged based on the CPU utilized.
I'm looking at Redis backed up rdb files for a web application. There are 4 such files (for 4 different redis servers working concurrently), sizes being: 13G + 1.6G + 66M + 14M = ~15G
However, these same 4 instances seem to be taking 43.8GB of memory (according to new relic). Why such a large discrepancy between how much space redis data takes up in mem vs disk? Could it be a misconfiguration and can the issue be helped?
I don't think there is any problem.
First of all, the data is stored in compressed format in rdb file so that the size is less than what it is in memory. How small the rdb file is depends on the type of data, but it can be around 20-80% of the memory used by redis
Another reason your memory usage could be more than the actual usage(you can compare the memory from new relic to the one obtained from redis-cli info memory command) is because of memory fragmentation. Whenever redis needs more memory, it will get the memory allocated from the OS, but will not release it easilyly(when the key expires or is deleted). This is not a big issue, as redis will ask for more memory only after using the extra memory that it has. You can also check the memory fragmentation using redis-cli info memory command.
I would like to use an EBS volume with data on it that I've been working with in an Ubuntu AMI in a RedHat 6 AMI. The issue I'm having is that RedHat says that the volume does not have a valid partition table. This is the fdisk output for the unmounted volume.
Disk /dev/xvdk: 901.9 GB, 901875499008 bytes
255 heads, 63 sectors/track, 109646 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/xvdk doesn't contain a valid partition table
Interestingly, the volume isn't actually 901.9 GB but 300 GB.. I don't know if that means anything. I am very concerned about possibly erasing the data in the volume by accident. Can anyone give me some pointers for formatting the volume for RedHat without deleting its contents?
I also just checked that the volume works in my Ubuntu instance and it definitely does.
I'm not able to advise on the partition issue as such, other than stating that you definitely neither need nor want to format it, because formatting is indeed a (potentially) destructive operation. My best guess would be that RedHat isn't able to identify the file system currently in use on the EBS volume, which must be advertized by some means accordingly.
However, to ease with experimenting and gain some peace of mind, you should get acquainted with one of the major Amazon EBS features, namely to create point-in-time snapshots of volumes, which are persisted to Amazon S3:
These snapshots can be used as the starting point for new Amazon EBS
volumes, and protect data for long-term durability. The same snapshot
can be used to instantiate as many volumes as you wish.
This is detailed further down in section Amazon EBS Snapshots:
Snapshots can also be used to instantiate multiple new volumes, expand
the size of a volume or move volumes across Availability Zones. When a
new volume is created, there is the option to create it based on an
existing Amazon S3 snapshot. In that scenario, the new volume begins
as an exact replica of the original volume. [...] [emphasis mine]
Therefore you can (and actually should) always start experiments or configuration changes like the one you are about to perform by at least snapshotting the volume (which will allow you to create a new one from that point in time in case things go bad) or creating a new volume from that snapshot immediately for the specific task at hand.
You can create snapshots and new volumes from snapshots via the AWS Management Console, as usual there are respective APIs available as well for automation purposes (see API and Command Overview) - see Creating an Amazon EBS Snapshot for details.
Good luck!.