Could NFS be mounted on one of openstack swift storage node? - nfs

Question

Currently, I got one CentOS in virtual box with openstack swift running which is installed by SAIO.
Question: Data would not stored in the node which I mount nfs.
For example, I create a container by curl, the output returns 201. But there is no data in the mounted directory. I refer to file /var/log/swift/proxy.error, and it shows
Error Insufficient storage balabala...
Would anyone help figure why this happens and how to fix it?
Thanks in advance!
Sure, there are some logs about the mounted node.
By following the SAIO guide, the configurations remain almost the same except the ${USER}. I use 'osddev' for the user name and group name.
/dev/sdb1 on /mnt/sdb1 type xfs (rw,noatime,nodiratime,seclabel,attr2,nobarrier,inode64,logbufs=8,noquota)
And I got 4 directories under /mnt/sdb1 named 1, 2, 3 and 4 respectively.
I mounted nfs on the directory '2',
I mount like this:
mount.nfs 192.168.0.1:/mnt/path/to/mount /mnt/sdb1/2
and the output of command 'mount' is like
192.168.0.1:/mnt/path/to/mount on /mnt/sdb1/2 type nfs4(rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=...,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.2,local_lock=none,addr=192.168.0.1)
The structure of the folder of 'mnt' is like this:
/mnt/sdb1
/1
/node
/sdb1
/accounts
/containers
/objects
/tmp
/2
/node
/sdb2
/containers
/tmp
/3
/node
/sdb3
/accounts
/containers
/objects
/tmp
/4
/node
/sdb4
/accounts
/containers
/objects
/tmp
When the mount was done, I tried to create a new container, and it returned 201 created, and there was a db file name '48ce59400b16f806fe2fee7e40e236as.db' and another file named '48ce59400b16f806fe2fee7e40e236as.db.pending' under the directory /mnt/sdb1/2/node/sdb2/containers/291/6ab/48ce59400b16f806fe2fee7e40e236as which was the same as other directoies under /mnt/sdb1.
When I tried to create a new object, there was 'objects' directory under /mnt/sdb1/2/node/sdb1. But in other directories like /mnt/sdb1/1 or /mnt/sdb1/3, the 'objects' directory existed.
So I checked out the error log under /var/log/swift.
And I found that in logs 'proxy.error' and 'storage2.error', there were some errors, I'll list them below:
proxy.error:
Jun 8 17:26:33 localhost proxy-server: Started child 4024
Jun 8 17:27:04 localhost proxy-server: STDERR: (4024) wsgi starting up on http://127.0.0.1:8080/
Jun 8 17:28:22 localhost proxy-server: STDERR: (4024) accepted ('127.0.0.1', 57718)
Jun 8 17:28:22 localhost proxy-server: STDERR: 127.0.0.1 - - [08/Jun/2017 09:28:22] "GET /auth/v1.0 HTTP/1.1" 200 356 0.004022 (txn: tx016fa30128e74197af806-00593918b6)
Jun 8 17:29:39 localhost proxy-server: STDERR: (4024) accepted ('127.0.0.1', 57721)
Jun 8 17:29:41 localhost proxy-server: ERROR Insufficient Storage 127.0.0.1:6020/sdb2 (txn: tx45826ac5bc284bd8b15a6-0059391903)
Jun 8 17:29:41 localhost proxy-server: STDERR: 127.0.0.1 - - [08/Jun/2017 09:29:41] "PUT /v1/AUTH_test/annecontainer/annefile3 HTTP/1.1" 201 254 1.474424 (txn: tx45826ac5bc284bd8b15a6-0059391903)
storage2.error
Jun 8 17:26:31 localhost account-server: Started child 4004
Jun 8 17:26:31 localhost container-server: Started child 4007
Jun 8 17:26:32 localhost object-server: Started child 4016
Jun 8 17:27:03 localhost object-server: STDERR: (4016) wsgi starting up on 127.0.0.1:6020/
Jun 8 17:27:03 localhost account-server: STDERR: (4004) wsgi starting up on 127.0.0.1:6022/
Jun 8 17:27:03 localhost container-server: STDERR: (4007) wsgi starting up on 127.0.0.1:6021/
Jun 8 17:29:39 localhost object-server: STDERR: (4016) accepted ('127.0.0.1', 43279)
Jun 8 17:29:40 localhost object-server: STDERR: ERROR:root:Filesystem at 9 does not support xattr#012Traceback (most recent call last):#012 File "/home/osddev/swift/swift/obj/diskfile.py", line 150, in write_metadata#012 metastr[:xattr_size])#012 File "/usr/lib64/python2.7/site-packages/xattr-0.9.1-py2.7-linux-x86_64.egg/xattr/init.py", line 185, in setxattr#012 return xattr(f).set(attr, value, options=options)#012 File "/usr/lib64/python2.7/site-packages/xattr-0.9.1-py2.7-linux-x86_64.egg/xattr/init.py", line 78, in set#012 return self._call(_setxattr, _fsetxattr, name, value, 0, options | self.options)#012 File "/usr/lib64/python2.7/site-packages/xattr-0.9.1-py2.7-linux-x86_64.egg/xattr/init.py", line 58, in _call#012 return fd_func(self.value, *args)#012 File "/usr/lib64/python2.7/site-packages/xattr-0.9.1-py2.7-linux-x86_64.egg/xattr/lib.py", line 106, in _fsetxattr#012 raise error()#012 File "/usr/lib64/python2.7/site-packages/xattr-0.9.1-py2.7-linux-x86_64.egg/xattr/lib.py", line 48, in error#012 raise IOError(errno, strerror)#012IOError: [Errno 95] Operation not supported
Jun 8 17:29:41 localhost container-server: STDERR: (4007) accepted ('127.0.0.1', 45775)
Jun 8 17:29:41 localhost object-server: STDERR: 127.0.0.1 - - [08/Jun/2017 09:29:41] "PUT /sdb2/957/AUTH_test/annecontainer/annefile3 HTTP/1.1" 507 263 1.300362 (txn: tx45826ac5bc284bd8b15a6-0059391903)
Jun 8 17:29:41 localhost container-server: STDERR: 127.0.0.1 - - [08/Jun/2017 09:29:41] "PUT /sdb2/291/AUTH_test/annecontainer/annefile3 HTTP/1.1" 201 120 0.044787 (txn: tx45826ac5bc284bd8b15a6-0059391903)
So far, I realized that nfs does not support xattr. Would that caused the issue?
What I want to do is use swift-on-file to store data in the mounted directory(by nfs) like /mnt/sdb1/2.

Answer 1

Objects are stored as binary files on the filesystem with metadata stored in the file’s extended attributes (xattrs). This requires that the underlying filesystem choice for object servers support xattrs on files. Some filesystems, like ext3, have xattrs turned off by default.
Source: https://docs.openstack.org/developer/swift/overview_architecture.html