Firstly i have 1 TB HDD & now i put external SSD of 256 GB & extend RAM from 4 to 8 GB So firstly my C drive is 470 GB & Now that old C drive is new D drive with storage of 470 GB & the SSD of 256 GB is new C drive
So my question is that if i fromat that new D drive of 470 GB which is older C drive
If the windows you are booting is now actually installed on your new C drive, and you have nothing you care about on your D drive you can go ahead and format it
Related
As you can see in the below command I have assigned a total of 500 GB disk space to my VM. But I am seeing 14.4 GB actual space available to the disk and once it gets used completely. I got an error there isn't much space to use? How to extend space for /dev/mapper/centos-root.
I am using VMware ESXi and using centOS for this VM.
[root#localhost Apr]# fdisk -l
Disk /dev/sda: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00064efd
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 33554431 15727616 8e Linux LVM
Disk /dev/mapper/centos-root: 14.4 GB, 14382268416 bytes, 28090368 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/centos-swap: 1719 MB, 1719664640 bytes, 3358720 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 byte
Execute bellow Steps to increase your Linux Disk after adding space in VMWare :
Step 1 - update partition table
fdisk /dev/sda
Press p to print the partition table to identify the number of partitions.
Press n to create a new primary partition.
Press p for primary.
Press 3 for the partition number, depending on the output of the partition table print.
Press Enter two times.
Press t to change the system's partition ID.
Press 3 to select the newly creation partition.
Type 8e to change the Hex Code of the partition for Linux LVM.
Press w to write the changes to the partition table.
Step 2 - Restart the virtual machine.
Step 3 - verify that the changes were saved
fdisk -l
Step 4 - convert the new partition to a physical volume
pvcreate /dev/sda3
Step 5 - extend the physical volume (centos is your VG name, if not use your VG name in place of centos )
vgextend centos /dev/sda3
Step 6 - extend the Logical Volume (500G is the size you want to add , if not use the right size in place of 500G)
lvextend -L+500G /dev/mapper/centos-root
Step 7 - expand the ext filesystem online
resize2fs /dev/mapper/centos-root
extend disk without reboot
echo 1 > /sys/block/sda/device/rescan
echo 1 > /sys/block/sdb/device/rescan
echo 1 > /sys/block/nvme0n1/device/rescan_controller
partprobe
gdisk fix warnging
parted change partion size
## parted can executed as command line. but this is very dangerous
parted -s /dev/sdb "resizepart 2 -1" quit
parted -s /dev/sdb "resizepart 3 100%" quit
resizepart 3 100%
pvresize /dev/sda3
lvextend -l +100%FREE cs/root
xfs_growfs /dev/cs/root
Loading 1500 images of size (1000,1000,3) breaks the code and throughs kill 9 without any further error. Memory used before this line of code is 16% of system total memory. Total size of images direcotry is 7.1G.
X = np.asarray(images).astype('float64')
y = np.asarray(labels).astype('float64')
system spec is:
OS: macOS Catalina
processor: 2.2 GHz 6-Core Intel Core i7 16 GB 2
memory: 16 GB 2400 MHz DDR4
Update:
getting the bellow error while running the code on 32 vCPUs, 120 GB memory.
MemoryError: Unable to allocate 14.1 GiB for an array with shape (1200, 1024, 1024, 3) and data type float32
You would have to provide some more info/details for an exact answer but, assuming that this is a memory error(incredibly likely, size of the images on disk does not represent the size they would occupy in memory, so that is irrelevant. In 100% of all cases, the images in memory will occupy a lot more space due to pointers, objects that are needed and so on. Intuitively I would say that 16GB of ram is nowhere nearly enough to load 7GB of images. It's impossible to tell you how much you would need but from experience I would say that you'd need to bump it up to 64GB. If you are using Keras, I would suggest looking into the DirectoryIterator.
Edit:
As Cris Luengo pointed out, I missed the fact that you stated the size of the images.
I am very newbie in OrangePI PC. I have installed it by dd on my macOS, and I have tried installing a Raspbian image which downloaded from the orangepi.org in Windows as well, after installation when I check free disk space it is showing:
root#orangepi:~# df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 3.4G 2.7G 474M 86% /
/dev/root 3.4G 2.7G 474M 86% /
devtmpfs 374M 0 374M 0% /dev
tmpfs 101M 188K 101M 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 201M 0 201M 0% /run/shm
/dev/mmcblk0p1 41M 4.9M 37M 12% /boot
I have installed it on 32G flash drive. But when I check it through fdisk command it shows 32G as a disk size:
root#orangepi:~# sudo fdisk -l
Disk /dev/mmcblk0: 32.0 GB, 32010928128 bytes
4 heads, 16 sectors/track, 976896 cylinders, total 62521344 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x34605ba5
Device Boot Start End Blocks Id System
/dev/mmcblk0p1 40960 124927 41984 83 Linux
/dev/mmcblk0p2 124928 7170047 3522560 83 Linux
root#orangepi:~#
How to fix this?
This solved my problem (solution is taken from here):
root#orangepi:~# fdisk /dev/mmcblk0
Command (m for help): p
Disk /dev/mmcblk0: 15.8 GB, 15804137472 bytes
4 heads, 16 sectors/track, 482304 cylinders, total 30867456 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x34605ba5
Device Boot Start End Blocks Id System
/dev/mmcblk0p1 40960 124927 41984 83 Linux
/dev/mmcblk0p2 124928 7170047 3522560 83 Linux
Command (m for help): d
Partition number (1-4): 2
Command (m for help): n
Partition type:
p primary (1 primary, 0 extended, 3 free)
e extended
Select (default p): p
Partition number (1-4, default 2): 2
First sector (2048-30867455, default 2048): 124928
Last sector, +sectors or +size{K,M,G} (124928-30867455, default 30867455):
Using default value 30867455
Command (m for help): w
Then quit (command q), reboot. You will then be able to use resize:
resize2fs /dev/root
When we use spark on yarn for non-streaming apps, we generally get the allocated memory to match the number of executors times memory per executor. When doing streaming apps, the allocated memory is immediately pushed to the limit (total memory) as shown in the yarn console.
With this set of parameters
--driver-memory 2g --num-executors 32 --executor-memory 500m
total memory 90G, memory used 85.88G
total vcores 64, vcores used 33
you would expect a basis of 32 * 1 G (500m + overhead) + driver memory or around 34 G, and 33 vcores (32 workers + 1 driver)
question:
is the 64 vcore due to the requirement of 2 core pairs for streaming connection and processing?
how did the estimated 34 G get pushed to 85.88 G? is this always true that with streaming apps, yarn gives it all it has?
In the neo4j-wrapper.conf file I see this:
# Java Heap Size: by default the Java heap size is dynamically
# calculated based on available system resources.
# Uncomment these lines to set specific initial and maximum
# heap size in MB.
#wrapper.java.initmemory=512
#wrapper.java.maxmemory=512
Does that mean that I should not worry about -Xms and -Xmx?
I've read elsewhere that -XX:ParallelGCThreads=4 -XX:+UseNUMA -XX:+UseConcMarkSweepGC would be good.
Should I add that on my Intel® Core™ i7-4770 Quad-Core Haswell 32 GB DDR3 RAM 2 x 240 GB 6 Gb/s SSD (Software-RAID 1) machine?
I would still configure it manually.
Set both to 12 GB and use the remaining 16GB for memory mapping in neo4j.properties. Try to match it to you store file sizes