I'm writing a python script to delete files on MacOS, and I run into SIP protected files. I know the presence of st_flags more than likely mean I can't delete the file. Like here:
>>> os.stat(f).st_flags
524288
But I'm curious to know what that actually means. I looked in stat.h and see:
/*
* Definitions of flags stored in file flags word.
*
* Super-user and owner changeable flags.
*/
#define UF_SETTABLE 0x0000ffff /* mask of owner changeable flags */
#define UF_NODUMP 0x00000001 /* do not dump file */
#define UF_IMMUTABLE 0x00000002 /* file may not be changed */
#define UF_APPEND 0x00000004 /* writes to file may only append */
#define UF_OPAQUE 0x00000008 /* directory is opaque wrt. union */
/*
* The following bit is reserved for FreeBSD. It is not implemented
* in Mac OS X.
*/
/* #define UF_NOUNLINK 0x00000010 */ /* file may not be removed or renamed */
#define UF_COMPRESSED 0x00000020 /* file is compressed (some file-systems) */
/* UF_TRACKED is used for dealing with document IDs. We no longer issue
* notifications for deletes or renames for files which have UF_TRACKED set. */
#define UF_TRACKED 0x00000040
#define UF_DATAVAULT 0x00000080 /* entitlement required for reading */
/* and writing */
/* Bits 0x0100 through 0x4000 are currently undefined. */
#define UF_HIDDEN 0x00008000 /* hint that this item should not be */
/* displayed in a GUI */
/*
* Super-user changeable flags.
*/
#define SF_SUPPORTED 0x001f0000 /* mask of superuser supported flags */
#define SF_SETTABLE 0xffff0000 /* mask of superuser changeable flags */
#define SF_ARCHIVED 0x00010000 /* file is archived */
#define SF_IMMUTABLE 0x00020000 /* file may not be changed */
#define SF_APPEND 0x00040000 /* writes to file may only append */
#define SF_RESTRICTED 0x00080000 /* entitlement required for writing */
#define SF_NOUNLINK 0x00100000 /* Item may not be removed, renamed or mounted on */
I just dont quite see how it adds up to 524288. I mean I kinda get it, like permission bits, 1 or more in the 6th position from the right should mean SF_NOUNLINK is set, but where is the 5 coming from? 2 in the 5th position from the right means SF_IMMUTABLE is set, and 2nd position is 8, which is UF_DATAVAULT, which makes sense. The other values in positions 3-4, and the value of 5 (from right) I dont understand. Any pointers as how to read this?
I am trying to build Mono on Solaris 10 and have ran into an issue with error "Posix system lacks support for recursive mutexes". I am trying to build this with gcc 3.4.3 and I have installed gmake, gar, granlib, and gstrip to replace their Solaris alternatives. I have found a possible solution, but I can not locate the blog that references this line "1) patch /usr/lib/pkgconfig/gthread-2.0.pc to replace the -mt option (see Jonel's blog)" located at https://lists.dot.net/pipermail/mono-list/2007-January/034101.html. The blog is no longer active. Does anyone have any ideas what the patch they are referring to may be? Thanks in advance.
Solaris 10 does support recursive mutexes.
Per the Solaris 10 pthread_mutexattr_settype man page:
PTHREAD_MUTEX_RECURSIVE
A thread attempting to relock this mutex without first unlocking it
will succeed in locking the mutex. The relocking deadlock that can
occur with mutexes of type PTHREAD_MUTEX_NORMAL cannot occur with
this type of mutex. Multiple locks of this mutex require the same
number of unlocks to release the mutex before another thread can
acquire the mutex. A thread attempting to unlock a mutex that another
thread has locked will return with an error. A thread attempting to
unlock an unlocked mutex will return with an error. This type of mutex
is only supported for mutexes whose process shared attribute is
PTHREAD_PROCESS_PRIVATE.
Also per the man page, the required compile/link options, and the proper #include statement:
cc –mt [ flag... ] file... –lpthread [ library... ]
#include <pthread.h>
Note the addition of -lpthread. I suspect the blog you're referring to said to replace -mt with -mt -lpthread.
(A bit more research into this problem, prodded by the Unix question How to install .net Mono on Solaris 11 (source code compile)? produced this answer, which I'm repeating here.)
The exact value that you set _XOPEN_SOURCE to doesn't matter. Mono is incorrectly defining the _XOPEN_SOURCE_EXTENDED macro:
case "${host}" in
*solaris* )
AC_MSG_CHECKING(for Solaris XPG4 support)
if test -f /usr/lib/libxnet.so; then
CPPFLAGS="$CPPFLAGS -D_XOPEN_SOURCE=600"
CPPFLAGS="$CPPFLAGS -D__EXTENSIONS__"
CPPFLAGS="$CPPFLAGS -D_XOPEN_SOURCE_EXTENDED=1" <-- WRONG
LIBS="$LIBS -lxnet"
The _XOPEN_SOURCE_EXTENDED macro does not exist in either the POSIX 6 or the POSIX 7 standard.
Even Linux agrees that _XOPEN_SOURCE_EXTENDED should not be defined here. Per the Linux feature_test_macros man page:
_XOPEN_SOURCE_EXTENDED
If this macro is defined, and _XOPEN_SOURCE is defined, then expose definitions corresponding to the XPG4v2 (SUSv1) UNIX extensions (UNIX 95). Defining _XOPEN_SOURCE with a value of 500 or more also produces the same effect as defining _XOPEN_SOURCE_EXTENDED. Use of _XOPEN_SOURCE_EXTENDED in new source code should be avoided.
Since defining _XOPEN_SOURCE with a value of 500 or more has the same effect as defining _XOPEN_SOURCE_EXTENDED, the latter (obsolete) feature test macro is generally not described in the SYNOPSIS in man pages.
Note the precise wording:
If this macro (_XOPEN_SOURCE_EXTENDED) is defined, and _XOPEN_SOURCE is defined, then expose definitions corresponding to the XPG4v2 (SUSv1) UNIX extensions (UNIX 95). ...
Defining _XOPEN_SOURCE to any value while also defining _XOPEN_SOURCE_EXTENDED results in XPG4v2, and that's NOT the XPG6 necessary to get recursive mutexes.
You're likely running into this check in the Solaris 11 /usr/include/sys/feature_tests.h:
/*
* It is invalid to compile an XPG3, XPG4, XPG4v2, or XPG5 application
* using c99. The same is true for POSIX.1-1990, POSIX.2-1992, POSIX.1b,
* and POSIX.1c applications. Likewise, it is invalid to compile an XPG6
* or a POSIX.1-2001 application with anything other than a c99 or later
* compiler. Therefore, we force an error in both cases.
*/
#if defined(_STDC_C99) && (defined(__XOPEN_OR_POSIX) && !defined(_XPG6))
#error "Compiler or options invalid for pre-UNIX 03 X/Open applications \
and pre-2001 POSIX applications"
#elif !defined(_STDC_C99) && \
(defined(__XOPEN_OR_POSIX) && defined(_XPG6))
#error "Compiler or options invalid; UNIX 03 and POSIX.1-2001 applications \
require the use of c99"
#endif
_XPG6 gets defined earlier in the file, in this block:
/*
* Use of _XOPEN_SOURCE
*
* The following X/Open specifications are supported:
*
* X/Open Portability Guide, Issue 3 (XPG3)
* X/Open CAE Specification, Issue 4 (XPG4)
* X/Open CAE Specification, Issue 4, Version 2 (XPG4v2)
* X/Open CAE Specification, Issue 5 (XPG5)
* Open Group Technical Standard, Issue 6 (XPG6), also referred to as
* IEEE Std. 1003.1-2001 and ISO/IEC 9945:2002.
*
* XPG4v2 is also referred to as UNIX 95 (SUS or SUSv1).
* XPG5 is also referred to as UNIX 98 or the Single Unix Specification,
* Version 2 (SUSv2)
* XPG6 is the result of a merge of the X/Open and POSIX specifications
* and as such is also referred to as IEEE Std. 1003.1-2001 in
* addition to UNIX 03 and SUSv3.
*
* When writing a conforming X/Open application, as per the specification
* requirements, the appropriate feature test macros must be defined at
* compile time. These are as follows. For more info, see standards(5).
*
* Feature Test Macro Specification
* ------------------------------------------------ -------------
* _XOPEN_SOURCE XPG3
* _XOPEN_SOURCE && _XOPEN_VERSION = 4 XPG4
* _XOPEN_SOURCE && _XOPEN_SOURCE_EXTENDED = 1 XPG4v2
* _XOPEN_SOURCE = 500 XPG5
* _XOPEN_SOURCE = 600 (or POSIX_C_SOURCE=200112L) XPG6
*
* In order to simplify the guards within the headers, the following
* implementation private test macros have been created. Applications
* must NOT use these private test macros as unexpected results will
* occur.
*
* Note that in general, the use of these private macros is cumulative.
* For example, the use of _XPG3 with no other restrictions on the X/Open
* namespace will make the symbols visible for XPG3 through XPG6
* compilation environments. The use of _XPG4_2 with no other X/Open
* namespace restrictions indicates that the symbols were introduced in
* XPG4v2 and are therefore visible for XPG4v2 through XPG6 compilation
* environments, but not for XPG3 or XPG4 compilation environments.
*
* _XPG3 X/Open Portability Guide, Issue 3 (XPG3)
* _XPG4 X/Open CAE Specification, Issue 4 (XPG4)
* _XPG4_2 X/Open CAE Specification, Issue 4, Version 2 (XPG4v2/UNIX 95/SUS)
* _XPG5 X/Open CAE Specification, Issue 5 (XPG5/UNIX 98/SUSv2)
* _XPG6 Open Group Technical Standard, Issue 6 (XPG6/UNIX 03/SUSv3)
*/
/* X/Open Portability Guide, Issue 3 */
#if defined(_XOPEN_SOURCE) && (_XOPEN_SOURCE - 0 < 500) && \
(_XOPEN_VERSION - 0 < 4) && !defined(_XOPEN_SOURCE_EXTENDED)
#define _XPG3
/* X/Open CAE Specification, Issue 4 */
#elif (defined(_XOPEN_SOURCE) && _XOPEN_VERSION - 0 == 4)
#define _XPG4
#define _XPG3
/* X/Open CAE Specification, Issue 4, Version 2 */
#elif (defined(_XOPEN_SOURCE) && _XOPEN_SOURCE_EXTENDED - 0 == 1)
#define _XPG4_2
#define _XPG4
#define _XPG3
/* X/Open CAE Specification, Issue 5 */
#elif (_XOPEN_SOURCE - 0 == 500)
#define _XPG5
#define _XPG4_2
#define _XPG4
#define _XPG3
#undef _POSIX_C_SOURCE
#define _POSIX_C_SOURCE 199506L
/* Open Group Technical Standard , Issue 6 */
#elif (_XOPEN_SOURCE - 0 == 600) || (_POSIX_C_SOURCE - 0 == 200112L)
#define _XPG6
#define _XPG5
#define _XPG4_2
#define _XPG4
#define _XPG3
#undef _POSIX_C_SOURCE
#define _POSIX_C_SOURCE 200112L
#undef _XOPEN_SOURCE
#define _XOPEN_SOURCE 600
#endif
My valgrind runs reports errors like this
Memcheck: mc_main.c:8292 (mc_pre_clo_init): Assertion 'MAX_PRIMARY_ADDRESS == 0x1FFFFFFFFFULL' failed.
What does this mean? Is it a valgrind internal error or an error from my program?
This is a valgrind internal error. This is very weird, as this failed
assertion is a self check done very early on.
You should file a bug on valgrind bugzilla, reporting all the needed
details (version, platform, ...)
From the Valgrind source code (from git HEAD)
/* Only change this. N_PRIMARY_MAP *must* be a power of 2. */
#if VG_WORDSIZE == 4
/* cover the entire address space */
# define N_PRIMARY_BITS 16
#else
/* Just handle the first 128G fast and the rest via auxiliary
primaries. If you change this, Memcheck will assert at startup.
See the definition of UNALIGNED_OR_HIGH for extensive comments. */
# define N_PRIMARY_BITS 21
#endif
/* Do not change this. */
#define N_PRIMARY_MAP ( ((UWord)1) << N_PRIMARY_BITS)
/* Do not change this. */
#define MAX_PRIMARY_ADDRESS (Addr)((((Addr)65536) * N_PRIMARY_MAP)-1)
...
tl_assert(MAX_PRIMARY_ADDRESS == 0x1FFFFFFFFFULL);
So it looks like something has been changed that shouldn't have been.
to be able to run my eSata Sheevaplug with Debian Wheezy I had to upgrade U-Boot to the DENX version.
As step-by-step guide I used this read from Martin Michlmayr. I did the upgrade using screen and a USB stick at the plug.
The upgrade went good and after resetting I got the plug started with the new version.
Marvell>> version
U-Boot 2013.10 (Oct 21 2013 - 21:06:56)
Marvell-Sheevaplug - eSATA - SD/MMC
gcc (Debian 4.8.1-9) 4.8.1
GNU ld (GNU Binutils for Debian) 2.23.52.20130727
Marvell>>
In the guide is written to set machid environment variable and MAC address.
But unfortunatly saveenv fails due to bad blocks in the NAND. I tried different versions of U-Boot also the one provided by NewIT. All behave the same way.
Marvell>> setenv machid a76
Marvell>> saveenv
Saving Environment to NAND...
Erasing NAND...
Skipping bad block at 0x00060000
Writing to NAND... FAILED!
There are some blocks marked as bad, which might be normal - by NewIT.
Marvell>> nand info
Device 0: nand0, sector size 128 KiB
Page size 2048 b
OOB size 64 b
Erase size 131072 b
Marvell>> nand bad
Device 0 bad blocks:
00060000
00120000
00360000
039c0000
0c300000
10dc0000
1ac40000
1f1c0000
Has someone a clue what the problem is and what I need to change to be able saving environment variables in u-boot?
Thanks,
schibbl
Due to configuration of environment variable storage at NAND, the sector size of 128k and a bad block mapping the environment variable storage adress it is not possible to write env to NAND.
Marvell>> nand bad
Device 0 bad blocks:
00060000
...
include/configs/sheevaplug.h which points perfectly to the bad block.
/*
* max 4k env size is enough, but in case of nand
* it has to be rounded to sector size
*/
#define CONFIG_ENV_SIZE 0x20000 /* 128k */
#define CONFIG_ENV_ADDR 0x60000
#define CONFIG_ENV_OFFSET 0x60000 /* env starts here */
Because of unused sector 0x80000 to 0x9FFFF I moved env storage there.
/*
* max 4k env size is enough, but in case of nand
* it has to be rounded to sector size
*/
#define CONFIG_ENV_SIZE 0x20000 /* 128k */
#define CONFIG_ENV_ADDR 0x80000
#define CONFIG_ENV_OFFSET 0x80000 /* env starts here due to bad block */
Beware! We have to ensure our compiled u-boot.kwb is less then 384k. Otherwise we will write u-boot to bad block marked memory and will brick the device.
Best way to recompile with custom env address, is to use Michlmayrs sources, which includes patches for mmc and e-sata support.
I'd like to try using valgrind to do some heap corruption detection. With the following corruption "unit test":
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
int main()
{
char * c = (char *) malloc(10) ;
memset( c, 0xAB, 20 ) ;
printf("not aborted\n") ;
return 0 ;
}
I was suprised to find that valgrind doesn't abort on error, but just produces a message:
valgrind -q --leak-check=no a.out
==11097== Invalid write of size 4
==11097== at 0x40061F: main (in /home/hotellnx94/peeterj/tmp/a.out)
==11097== Address 0x51c6048 is 8 bytes inside a block of size 10 alloc'd
==11097== at 0x4A2058F: malloc (vg_replace_malloc.c:236)
==11097== by 0x400609: main (in /home/hotellnx94/peeterj/tmp/a.out)
...
not aborted
I don't see a valgrind option to abort on error (like gnu-libc's mcheck does, but I can't use mcheck because it isn't thread safe). Does anybody know if that is possible (our code dup2's stdout to /dev/null since it runs as a daemon, so a report isn't useful and I'd rather catch the culprit in the act or closer to it).
There is no such option in valgrind.
Consider adding a non-daemon mode (debug mode) into your daemon.
http://valgrind.org/docs/manual/mc-manual.html#mc-manual.clientreqs 4.6 explains some requests from debugged program to valgrind+memcheck, so you can use some of this in your daemon to do some checks at fixed code positions.