Using the excellent https://github.com/laurent22/rsync-time-backup I backup a Mac to a drive in Windows. To achieve this, I FUSE mount the Windows drive to the Mac
I see rsync-time-backup
correctly finds the previous backup folder (backup/day-x) so that it can hard link when it creates a new backup folder (backup/day-y)
correctly transfers only the changed files (Total file size: 4.1G bytes vs Total transferred file size: 100M bytes)
But here are the odd things I found:
when I run WinDirStat to find the total size of the entire backup directory "backup/", I expected 4.1G
4G (for day-x) and
100M (for day-y).
Instead, I found 8.1G
4G (for day-x) and
4.1G (for day-y).
when I run ls -lHi, on the Mac Terminal, I don't see the link count greater than 1
So, why is it that even though only a fraction of the files are transferred, the size of the Windows drive is huge?
Situation: Server uses XFS filesystem, block size 4kB. There are a lot of small files.
Result: Some directories take 2+GB space, but actual file size is less then 200MB.
Solution: Change XFS block size to least possible, 512B. I am aware it means more overhead and some performance loss, I haven't been able to find out how much.
Question: How to do it?
I am aware XFS uses xfsdump to backup data. So lets presume /dev/sda2 is actual XFS filesystem I want to change and /mnt/export is where I want to dump it using xfsdump. Manual says that dumped blocksize has to be same as restored blocksize and -b parameter "Specifies the blocksize, in bytes, to be used for the dump." . I am bit worried because manual also says "The default block size is 1Mb"
Is this then correct way to dump my complete filesystem (in this case, /dev/sda2 is mounted to /home)?
xfsdump -b 512 -f /mnt/export /home
If the above command is correct, what I need to do to correctly get my files back, only with 512B blocksize? My guess is reformat /dev/sda2 to 512B blocksize using
mkfs.xfs /dev/sda2 -b 512
and then use
xfsrestore -f /mnt/export /home
but I am not sure how and there isn't a good way to test and see if I am right.
So I did more research and came up with this:
xfsdump -f /mnt/export_file /home
umount /home
mkfs.xfs -b size=1024 /dev/sda2 -f
#Minimum block size for CRC enabled filesystems is 1024 bytes.
mount /home
xfsrestore -f /mnt/export_file /home
As you can see, I was unable to change blocksize to 512B, because of minimum requirement for CRC enabled filesystems, but otherwise, complete success.
I didn't concider I have to unmount the filesystem first. You won't be able to do that if you are logging in as someone who has homedir in /home, so log in as root.
I have been receiving alerts regarding the disk space utilization and would like to increase disk space but not sure where the increased usage occurs. The following alert appears:
'Rabbit-Disk-Alert' threshold
Description: Average Disk utilization during the past 15 minutes exceeds
75%
Now when I log on the server and run
df -h
It shows the drive that is getting full but I do not know how to find the directory or files that are causing this issue. Is there a way to diagnose this or determine the root of the alert?
Use the du command to find the directory taking up space:
cd /var
du -hs *
More than likely a sub-directory of /var is the culprit. The above command will show you which sub-directory (or directories) takes up the most space, and you can change to those directories, re-run du -hs *, and continue "downward".
I have my sandbox created using vboxmanage cli tool which created vmdk file.
I converted it from .vmdk to .vdi, as i wanted to compress it. then i attached this newly created .vdi file.
Now i want to detach the vmdk file as i don't want it any more.
Can you please suggest me what should be the command for the same?
List of my HDDs are:
$ vboxmanage list hdds
UUID: f3b90783-abe7-4549-91aa-39aa6161f103
Parent UUID: base
State: created
Type: normal (base)
Location: /home/ankit/VirtualBox VMs/asr-vm/asr-sandbox-3.5.0-lb1404x64-disk1.vmdk
Storage format: VMDK
Capacity: 65536 MBytes
UUID: 1b9a4eff-f1bd-4c4e-ae48-8f11b8003244
Parent UUID: base
State: created
Type: normal (base)
Location: /home/ankit/VirtualBox VMs/asr-vm/asr-sandbox-3.5.0-lb1404x64-disk1.vdi
Storage format: VDI
Capacity: 65536 MBytes
the solution for this is to attach that medium with emptydrive or none. that means for that particular port and device you are attaching an emptydrive or none.
vboxmanage storageattach "asr-vm" --storagectl "SATA" --port 1 --medium none
or
vboxmanage storageattach "asr-vm" --storagectl "SATA" --port 1 --medium emptydrive
As per docs medium none is a better option.
For anyone wondering why none is a better option than emptydrive for the --medium flag, the VboxManage docs have this to say:
--medium
Specifies what is to be attached. The following values are supported:
none: Any existing device should be removed from the given slot.
emptydrive: For a virtual DVD or floppy drive only, this makes the device slot behave like a removeable drive into which no media has been inserted.
So given we want to detach (or "remove" it), none seems like the correct option.
Source:
https://www.virtualbox.org/manual/ch08.html#vboxmanage-storageattach
This is a reponse to Mehrdad Hedayati comment to the accepted answer.
I have virtualbox-4.1.0 with centos-5.6 installed in. Now I would like to extend the VM image, which I previously allocated for virtual machine installation, it was 8Gb, that's not enough now. Is there a way to extend the partition without loosing information?
Actually in centos I have one root fs with my home dir etc. so this partition eventually would need to be resized.
Thanks in advance for suggestions!
It can be done in 4 steps :
Clone your VMDK to VDI format with VBoxManage Tool.
Resize the disk to create free space with VBoxManage Tool.
Modify the filesystem to allocate free space for your drive with GParted.
Add created disk space to Linux FileSystem.
The detailed steps are below (tested with "Virtual Machine CentOS 6.4" and "VirtualBox 4.2.18");
Observe disk format of the virtualbox file, if it is not *.vdi, convert disk format from *.wmdk to *.vdi. Open windows terminal:
$ VBoxManage clonehd --format VDI "path_of_wmdk_file" "path_of_vdi_file"
Resize disk size of vdi file. Open windows terminal. For example to do disk size ~500 GB => 512000;
$ VBoxManage modifymedium "path_of_vdi_file" --resize 512000
Choose *.vdi file instead of *.wmdk file as disk
Virtual Machine -> Settings -> Storage -> Controller : SATA (Right Click on *.wmdk file) -> Remove Attachment -> Add HardDisk and choose newly created *.vdi file
Download "gparted-live-x.xx.x-x-ixxx.iso" file from http://gparted.sourceforge.net/download.php. Mount this iso file as CD.
Virtual Machine -> Settings -> Storage -> Controller IDE (Right Click) -> Add CD/DVD -> Select gparted-live-x.xx.x-x-ixxx.iso file
Run virtual machine, Virtual Machine will boot from this CD. Choose default values with pressing "Enter", "Enter" ... until Gpart ISO GUI starts. Select tool gpart program and start.
Extend disk size as below;
Right click on partitions and if "possible" click on "Disable Active Partion".
Extend Partition as much as possible from GUI (for this case 500GB).
Right click the partition which is disabled and select "Enable Active Partion".
Apply and wait until the operations finished.
Shut down virtual machine.
Unmount gparted-live-x.xx.x-x-ixxx.iso.
Virtual Machine -> Settings -> Storage-> Controller IDE (Right Click on gparted-live-x.xx.x-x-ixxx.iso) -> Remove Attachement
Start the virtual machine.
Open linux terminal and login as root. Run commands below;
lvm vgdisplay
=> Free PE / Size 122880 / 480.00 GiB
lvm lvdisplay /dev/VolGroup/lv_root
=> Current LE 3978
Calculate the sum of the values above. In this case : 122880 + 3978 = 126858 <- will be used in the next command
lvm lvresize -l 126858 /dev/VolGroup/lv_root
resize2fs /dev/VolGroup/lv_root
lsblk
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
sda 8:0 0 500G 0 disk
+¦sda1 8:1 0 500M 0 part /boot
L¦sda2 8:2 0 499.5G 0 part
+¦VolGroup-lv_root (dm-0) 253:0 0 480G 0 lvm /
L¦VolGroup-lv_swap (dm-1) 253:1 0 4G 0 lvm [SWAP]
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Check whether the filesystem extended or not with creating a huge file:
fallocate -l 480G /test_file
Remove the test file of course:
rm -rif /test_file
I'm using a CentOS7 virtualbox, and I finally enlarged my partition /dev/mapper/centos-root - gparted doesn't work for me because I do not have a desktop on CentOS7 virtualbox.
Power off your CentOS virtual machine
Go into the directory of your *.vdi image. If you don't know where it is, look at your Virtualbox Manager GUI virtualbox -> settings -> storage -> *.vdi -> location e.g. mine is located under ~/VirtualBox VMs/CentOS7/CentOS.vdi
Back up your image just in case anything goes wrong
$ cp CentOS7.vdi CentOS7.backup.vdi
Resize your virtual storage size, e.g. 200 GB
$ VBoxManage modifyhd CentOS7.vdi --resize 204800
Power on your CentOS virtual machine, issue
$ sudo fdisk -l
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 209715199 104344576 8e Linux LVM
Use fdisk utility to delete/create partitions
$ sudo fdisk /dev/sda
You are in the fdisk utility interactive mode, issue following commands: (mostly just follow the default recommendation)
d - delete a partition
2 - select a partition to delete (/dev/sda2 here)
n - create a new partition
p - make it a primary partition
2 - make it on the same partition number as we deleted
<return> - set the starting block (by default)
<return> - set end ending block (by default)
w - write the partition and leave the fdisk interactive mode
Reboot your CentOS machine
$ sudo reboot
Resize the physical volume and verify the new size
$ sudo pvresize /dev/sda2
$ sudo pvscan
Take a look at your logical mapping volume to see what volume you want to enlarge, in my case, /dev/mapper/centos-root
Resize the file system by adding -r option, it will take care of resizing for you
$lvextend -r -l +100%FREE /dev/mapper/centos-root
Resize the file system:
$resize2fs /dev/mapper/fedora-root
For CentOS 7: $xfs_growfs /dev/mapper/fedora-root
Last check:
$df -h.
Reference:https://blog.jyore.com/2013/06/virtualbox-increase-size-of-rhelfedoracentosscientificos-guest-file-system/#comment-2879
I found this nugget at the link following. I worked perfect for me and only took 5 seconds.
As of VirtualBox 4 they added support for expansion.
VBoxManage modifyhd filename.vdi --resize 46080
That will resize a virtual disk image to 45GB.
https://superuser.com/questions/172651/increasing-disk-space-on-virtualbox
It can be done with 3 steps :
cloning your VMDK to VDI format using VBoxManage
resize the disk to create free space using VBoxManage
modify the filesystem to allocate free space to your drive, using GParted
Don't forget the last part, otherwise, you will have unallocated free space and your disk will still appear as full.
I wrote a tutorial here to resize your VM Disk
At some point the VBoxManage utility changed the syntax a little bit. It's worth noting that this doesn't work on all vm types so beware if you have a *.vmdk. An example command to change your HD to 40GB is:
$ VBoxManage modifyhd MyVirtualImage --resize 40960
For reference I am on VirtualBox 4.2.1, Mac 10.8.2
From the VirtualBox FAQ:
You have to clone the data from the VDI you already have to a new, larger VDI and expand the partition(s). You can use tools like CloneZilla to clone the virtual hard drive to the bigger one and Gparted to increase the partition size. Another method is to use CloneVDI by mpack and clone the VDI with a larger size, then resize the partition(s) with Gparted.
Since 4.0.0, you can use VBoxManage modifyhd --resize to resize the max size of the VDI. You can only make it bigger. After that, use Gparted to increase the partition size inside the VDI.
Alternatively you could attach another VDI and mount your /home there.
Search for CloneVDI tool on the Oracle VirtualBox forums. It worked for me and is a much easier GUI based program for anyone nervous at the command line. Also allows conversion from fixed to dynamic mode which VBoxManage.exe doesn't support (yet).
https://forums.virtualbox.org/viewtopic.php?f=6&t=22422
I spent a long time googling and finally got it working for me. ( before I found this ) And I wanted a place to save my work
use vboxmanage to add space to the disk image
use gparted to resize so all space is used
use blivet-gui to create a new volume :
Below is commands I copied from terminal in fedora :
dnf install blivet-gui
blivet-gui
lvremove /dev/mapper/fedora00-00
lvextend -l +100%FREE /dev/mapper/fedora00-root