I've been using Google Colab with the GPU backend. On December when I used it, the disk size for the GPU backend was more than 300 GB. Now running df -h on the virtual machine shows this:
Filesystem Size Used Avail Use% Mounted on
overlay 69G 33G 33G 50% /
tmpfs 64M 0 64M 0% /dev
tmpfs 6.4G 0 6.4G 0% /sys/fs/cgroup
/dev/sda1 75G 37G 39G 49% /opt/bin
tmpfs 6.4G 12K 6.4G 1% /var/colab
shm 5.9G 4.0K 5.9G 1% /dev/shm
tmpfs 6.4G 0 6.4G 0% /proc/acpi
tmpfs 6.4G 0 6.4G 0% /proc/scsi
tmpfs 6.4G 0 6.4G 0% /sys/firmware
Do you know if something has changed? I searched the web for news about this but couldn't find any. Before, the overlay filesystem was 359 GB.
Thanks in advance for any clues.
Bests,
B.
It seems it is a new issue. I found this on Github: https://github.com/googlecolab/colabtools/issues/919.
What's ironic about this problem is that one proposed solution is to mount Google Drive. Therefore I bought 200 GB of Google Drive. However, the disk size is still an issue. Apparently once Google Drive is mounted it starts to cache files in /root/.config/Google/DriveFS/[uniqueid]/content_cache. The cache has no control over its size, it does not delete or replace anything, it just accumulates, and it takes all the disk making the code crash. :(
Related
I have a debian VM deployed at BlueMix, and I want to increase the size of the hard drive mounting a BlockStorage Device.
I followed the instructions on the new Beta BlockStorage Service and created a volume, and then attached it to the VM as a new device, but seems that although the volume is attached to the VM; is not automatically mounted.
I tryed several ways to mount it, but I did not find it the correct way. In fact, I even tryed to clone the line that came on the fstab refering to the root device mounted (I suspected that the additional volume should be similar) but it did not work (even broke the reboot of my machine)... So.. Can someone please advice me how to mount the BlockStorage Bluemix Service on the VM Machine ?
THks!
By attaching a volume you've essentially done the equivalent of plugging a raw, physical hard disk into your system. Before you can mount it you'll have to format it with a filesystem known by your OS.
After attaching the device you should be able to see the raw block device, for example with the lsblk command:
[mysys]# lsblk
sr0 11:0 1 416K 0 rom
vda 252:0 0 20G 0 disk
--vda1 252:1 0 20G 0 part /
vdb 252:16 0 25G 0 disk
Typically vda is your root device, so in this example the additional device is vdb with 25GB.
Now you can create a filesystem with the mkfs command, for example:
[mysys]# mkfs.ext4 /dev/vdb
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1638400 inodes, 6553600 blocks
...
mkfs supports different filesystems, so you might want to check the man pages on the system you're using (man mkfs).
Now all that's left is to create a mount point and mount the new filesystem:
[mysys]# mkdir /mnt/test
[mysys]# mount /dev/vdb /mnt/test
The additional space is now available:
[mysys]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 20G 946M 18G 5% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/vdb 25G 172M 24G 1% /mnt/test
I have a running Aerospike server with about 36,000 records in a single set. All I'm storing is a few bins for the set. I have configured my aerospike.conf file to persist data on disk as well:
namespace default {
replication-factor 2
memory-size 4G
default-ttl 0
storage-engine device {
file /opt/aerospike/data/default.dat
filesize 2T
data-in-memory true
}
}
The problem I'm having is that my /opt/aerospike/data/default.dat file is listed in my system as about 2TB:
/opt/aerospike/data# ls -lh
total 10M
-rw------- 1 root root 2.0T Jun 5 19:01 default.dat
My questions are:
Why does this .dat file have to be 2TB when the data I'm using in Aerospike is minimal for now?
My hard drive limit is 78GB, so why isn't my Ubuntu system not giving me out of drive space errors?
System disk space looks fine:
df -h --total
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 79G 4.2G 72G 6% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 2.0G 8.0K 2.0G 1% /dev
tmpfs 395M 424K 395M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 2.0G 0 2.0G 0% /run/shm
none 100M 0 100M 0% /run/user
total 83G 4.2G 76G 6% -
Anyone have any ideas?
Well,
Aerospike uses sparse file (or kind of) for storing its file storage.
So when you specified the filesize 2T in namespace config, Aerospike will create a sparse file of size 2TB that you are seeing.
http://en.wikipedia.org/wiki/Sparse_file
So, its just a file with some metadata and not real 2TB data. Once the file content has actually filled your disk, you will see the usual disk full errors on both your system as well as in Aerospike.
I'd like to know what the ideal Aerospike namespace configuration is for a mini (staging) server on Ubuntu 12.04 with 1 GB ram and 1 GHz CPU
Some requirements:
1. I'd like to persist data permanently on disk (not using it as a cache).
2. I'm only using a single node
3. I don't want a limit on the filesize of my data
Here's my current config snippet I'm using:
namespace default {
replication-factor 1
memory-size 1G
default-ttl 0 # not sure if this is for cache or disk
storage-engine device {
file /opt/aerospike/data/default.dat
filesize 2T
data-in-memory true
}
}
Thanks
Aerospike doesn't cache data-in-memory. If data-in-memory is set to true then all of your data must fit in RAM.
On a single node you will not be affected by the replication-factor parameter.
Aerospike has a limit of 2 TiB per file, but you can create multiple files of this size and Aerospike will distribute data across them. When going through a filesystem, having multiple files often helps. Also if you are going to use a filesystem then you may wan to look into disabling atime when mounting the disks.
default-ttl is how long the server will keep a record after it is written by default (can be overridden by your application). A default-ttl of 0 mean to never expire or evict data.
Example config with multiple files:
namespace default {
replication-factor 1
memory-size 1G
default-ttl 0 # (This applies to the primary index)
storage-engine device {
file /opt/aerospike/data/file0.dat
file /opt/aerospike/data/file1.dat
file /opt/aerospike/data/file2.dat
file /opt/aerospike/data/file3.dat
file /opt/aerospike/data/file4.dat
file /opt/aerospike/data/file5.dat
filesize 2T
data-in-memory true
}
}
My goal is to implement APC for opcode cache for a drupal 6 production site.
I have so far tested APC with several php files with and without including other php files with include_once.
Also tried to tweak the apc.ini values for shm_size, apc.include_once_override and apc.stat.
Restarted apache every time.
Resulting in apc.php not showing any changes in any values. (except of course the changed apc.ini values are shown as they should)
Every time i refresh the apc.php test page, the start time resets as the current time showing uptime 0 minutes.
apc.php -testpage shows:
General Cache InformationAPC Version 3.1.9
PHP Version 5.2.10
APC Host xxxx.xx.xx
Server Software Apache/2.2.3 (CentOS)
Shared Memory 1 Segment(s) with 128.0 MBytes
(mmap memory, pthread mutex Locks locking)
Start Time 2011/07/26 11:53:56
Uptime 0 minutes
File Upload Support 1
Cached Files 0 ( 0.0 Bytes)
Hits 1
Misses 1
Request Rate (hits, misses) 2.00 cache requests/second
Hit Rate 1.00 cache requests/second
Miss Rate 1.00 cache requests/second
Insert Rate 0.00 cache requests/second
Cache full count 0
Cached Variables 0 ( 0.0 Bytes)
Hits 0
Misses 0
Request Rate (hits, misses) 0.00 cache requests/second
Hit Rate 0.00 cache requests/second
Miss Rate 0.00 cache requests/second
Insert Rate 0.00 cache requests/second
Cache full count 0
apc.cache_by_default 1
apc.canonicalize 1
apc.coredump_unmap 0
apc.enable_cli 0
apc.enabled 1
apc.file_md5 0
apc.file_update_protection 2
apc.filters
apc.gc_ttl 3600
apc.include_once_override 0
apc.lazy_classes 0
apc.lazy_functions 0
apc.max_file_size 16
apc.mmap_file_mask /tmp/apcphp5.095eRm
apc.num_files_hint 1024
apc.preload_path
apc.report_autofilter 0
apc.rfc1867 0
apc.rfc1867_freq 0
apc.rfc1867_name APC_UPLOAD_PROGRESS
apc.rfc1867_prefix upload_
apc.rfc1867_ttl 3600
apc.serializer default
apc.shm_segments 1
apc.shm_size 128M
apc.slam_defense 0
apc.stat 0
apc.stat_ctime 0
apc.ttl 7200
apc.use_request_time 1
apc.user_entries_hint 4096
apc.user_ttl 7200
apc.write_lock 1
Host Status Diagrams:
Free: 128.0 MBytes (100.0%) Hits: 1 (50.0%)
Used: 20.3 KBytes (0.0%) Misses: 1 (50.0%)
Detailed Memory Usage and Fragmentation:
Fragmentation: 0%
phpinfo shows:
Server API CGI/FastCGI
APC:
Version 3.1.9
APC Debugging Enabled
MMAP Support Enabled
MMAP File Mask /tmp/apcphp5.JkKDk7
Locking type pthread mutex Locks
Serialization Support php
Revision $Revision: 308812 $
Build Date Jul 21 2011 14:31:12
I followed these steps to find if suexec settings would prevent caching:
http://www.litespeedtech.com/support/forum/showthread.php?t=4189
[root#host /]# ps -ef|grep lsphp
root 20402 17833 0 11:21 pts/0 00:00:00 grep lsphp
[root#host /]# ps -waux
root 17833 0.0 0.1 5004 1484 pts/0 S 10:39 0:00 bash
..indicates that there is no lsphp running on the host
also I read the following article and comments, concluding that in my case the problem is not the suexec as the user apache is the httpd process owner
http://www.brandonturner.net/blog/2009/07/fastcgi_with_php_opcode_cache/
also suexec command is not recognized when logged and launced as root # host
also i'm almost confident that there is no cPanel running on the host to check if a setting there would reset the running cache process at some interval
This leaves me with few clues where to head next.
I tried to set (with chown and chgrp) apache as the owner of the apc.php file and some test php files resulting in 500 server error.
Is there a way to check if the file permissions prevent the apc stay running?
I'm tremendously grateful for any suggestions or help.
Can you give your php.ini settings for APC ?
You must restart httpd to take setting change into account.
Try to change max file size
apc.max_file_size = 20M
You are alowing 128M of ram, it's quite low for big php applications like we have today (a single wordpress uses 32M)
apc.shm_size 128M
increase it also
One of my sites is growing and I'm having scalability issues. My knowledge with this new software is rather small, my hosting company doesn't have a clue either.
Shared Memory is not working since variables are not cached between requests, is there a way to make this work? At the moment the script is relying on Memcached but there is the TCP/IP overhead.
Will PHP-FPM automatically solve this problem ? Since it comes bundled with PHP 5.3.3 maybe an upgrade of PHP will suffice ?
== OUTPUT FROM apc.php ==
General Cache Information
APC: 3.0.19
PHP: 5.2.14
Server: Apache/2.2.16 (Unix) mod_ssl/2.2.16 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4 mod_fcgid/2.3.5
Shared Memory: 1 Segment(s) with 128.0 MBytes
(mmap memory, pthread mutex locking)
File Cache Information
Cached Files 91 ( 4.2 MBytes)
Hits 245
Misses 92
Request Rate (hits, misses) 0.41 cache requests/second
Hit Rate 0.30 cache requests/second
Miss Rate 0.11 cache requests/second
Insert Rate 0.11 cache requests/second
Cache full count 0
User Cache Information (PROBLEM!!!!)
Cached Variables 0 ( 0.0 Bytes)
Hits 0
Misses 0
Request Rate (hits, misses) 0.00 cache requests/second
Hit Rate 0.00 cache requests/second
Miss Rate 0.00 cache requests/second
Insert Rate 0.00 cache requests/second
Cache full count 0
Runtime Settings
apc.cache_by_default 1
apc.coredump_unmap 0
apc.enable_cli 0
apc.enabled 1
apc.file_update_protection 2
apc.filters
apc.gc_ttl 3600
apc.include_once_override 0
apc.max_file_size 10M
apc.mmap_file_mask /tmp/apc.RqsiCE
apc.num_files_hint 1024
apc.report_autofilter 0
apc.rfc1867 0
apc.rfc1867_freq 0
apc.rfc1867_name APC_UPLOAD_PROGRESS
apc.rfc1867_prefix upload_
apc.shm_segments 1
apc.shm_size 128
apc.slam_defense 0
apc.stat 1
apc.stat_ctime 0
apc.ttl 7200
apc.user_entries_hint 4096
apc.user_ttl 7200
apc.write_lock 1
What are you using in your PHP code to try storing and fetching variables? Are you sure you're using APC to store variables? Try to create a sample code to check out if you're using everything properly.
Try to enable apc.enable_cli
I've worked with APC with fastCGI - php-cgi and php-fpm