It looks like in OS X 10.11 El Capitan, dtruss and dtrace can no longer do what they're meant to do. This is the error I get when I try to run sudo dtruss curl ...:
dtrace: failed to execute curl: dtrace cannot control executables signed with restricted entitlements
I've come across people noticing this problem but so far no solutions.
Is there a way to fix this or work around this?
Following up to Alexander Ushakov and Charles' answers:
Once you csrutil enable --without dtrace, there is an alternative to copying the binary: run the binary in one Terminal window and trace the Terminal process itself in another Terminal window.
In the first terminal window, find its PID:
$ echo $$
1154
In the second terminal window, begin the trace:
$ sudo dtruss -p 1154 -f
Back, in the first terminal window, run the process you want to trace:
$ ls
At this point, you should see the trace in the second window. Ignore the entries for the PID you are tracing (e.g., 1154), and the rest are for the process (and its descendants) you are interested in.
1154/0x1499: sigprocmask(0x3, 0x7FFF53E5C608, 0x0) = 0x0 0
1154/0x1499: sigprocmask(0x1, 0x7FFF53E5C614, 0x7FFF53E5C610) = 0x0 0
3100/0xa9f3: getpid(0x7FFF82A35344, 0x7FFF82A35334, 0x2000) = 3100 0
3100/0xa9f3: sigprocmask(0x3, 0x10BE32EF8, 0x0) = 0x0 0
For those who want to dtrace system shipped binary after csrutil disable, copyit to a directory that is not "restricted", for example, /tmp
CC#~ $ csrutil status
System Integrity Protection status: disabled.
CC#~ $ cp /bin/echo /tmp
CC#~ $ sudo dtruss /tmp/echo
SYSCALL(args) = return
thread_selfid(0x0, 0x0, 0x0) = 46811 0
csops(0x0, 0x0, 0x7FFF51B6CA20) = 0 0
issetugid(0x0, 0x0, 0x7FFF51B6CA20) = 0 0
shared_region_check_np(0x7FFF51B6A918, 0x0, 0x7FFF51B6CA20) = 0 0
stat64("/usr/lib/dtrace/libdtrace_dyld.dylib\0", 0x7FFF51B6BEA8, 0x7FFF51B6CA20 = 0 0
See #J.J's comment: https://apple.stackexchange.com/questions/208762/now-that-el-capitan-is-rootless-is-there-any-way-to-get-dtrace-working/224731#224731
As Andrew notices it's because of System Integrity Protection, also known as "rootless".
You can disable it completely or partially (enable just dtrace with some limitations).
Completely disable SIP
Although not recommended by Apple, you can entirely disable System
Integrity Protection on you Mac. Here's how:
Boot your Mac into Recovery Mode: reboot it and hold cmd+R until a progress bar appears.
Go to Utilities menu. Choose Terminal there.
Enter this command to disable System Integrity Protection:
$ csrutil disable
It will ask you to reboot — do so and you're free from SIP!
Partially disable SIP
Fortunately, SIP is not monolithic: it's built from many different
modules we can disable/enable separately.
Repeat steps 1 and 2 from «Completely disable SIP» section above. Now
in Terminal enter these commands:
$ csrutil clear # restore the default configuration first
$ csrutil enable --without dtrace # disable dtrace restrictions *only*
Reboot and enjoy your OS again.
Dtrace starts to work but you're still unable to attach dtrace to restricted processes
I would post this as a comment but I'm not allowed.
Disabling SIP is not necessary. Just copy the binary to an alternate location and it works just fine:
$ sudo dtruss ping google.com
dtrace: system integrity protection is on, some features will not be available
dtrace: failed to execute ping: dtrace cannot control executables signed with restricted entitlements
$ sudo cp $(which ping) .
$ sudo dtruss ./ping google.com
dtrace: system integrity protection is on, some features will not be available
SYSCALL(args) = return
PING google.com (172.217.10.78): 56 data bytes
^C
$ csrutil status
System Integrity Protection status: enabled.
For binaries that can still function normally after being copied, this is the best option as it captures the entire lifetime of the process and doesn't require disabling any protections.
Looks like completely disabling SIP still blocks dtruss for restricted processes:
$ /usr/bin/csrutil status
System Integrity Protection status: disabled.
$ sudo dtruss /bin/echo "blah"
dtrace: failed to execute /bin/echo: dtrace cannot control executables signed with restricted entitlements
$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.11.2
BuildVersion: 15C50
See my answer on related question "How can get dtrace to run the traced command with non-root priviledges?" [sic].
DTrace can snoop processes that are already running. So, start a background process which waits 1sec for DTrace to start up (sorry for race condition), and snoop the PID of that process.
sudo true && \
(sleep 1; ps) & \
sudo dtrace -n 'syscall:::entry /pid == $1/ {#[probefunc] = count();}' $! \
&& kill $!
Full explanation in linked answer.
Related
System release: CoreOS 2135.5.0
Kernel: 4.19.50-coreos-r1
System install way: vmware
When I change the port in the sshd.service，it displays:
CoreOS-234 ssh # echo "Port 10000" >> /usr/share/ssh/sshd_config ;systemctl mask sshd.socket;systemctl enable sshd.service;systemctl restart sshd.service
-bash: /usr/share/ssh/sshd_config: Read-only file system
The file system that you are working in is currently in Read-only mode. Remounting the file system to read-write should resolve the issue. You will need to have root privilages:
$ mount -o remount,rw /
Occasionally the reason your file system will be running in read-only mode is due to Kernel issues, therefore there may be further problems with the system that will need to be debugged. Regarding the Kernel errors you may want to have a look at the following link: https://unix.stackexchange.com/questions/436483/is-remounting-from-read-only-to-read-write-potentially-dangerous?rq=1
In coreos /usr is designed to be a read-only file system, Remounting /usr is theoretically feasible, but is not officially recommended
You can refer to this
I use the following command to solve this problem
sudo sed -i '$a\Port=60022' /etc/ssh/sshd_config && \
sudo systemctl mask sshd.socket && \
sudo systemctl enable sshd.service && \
sudo systemctl start sshd.service
I am trying to run a redis image in gke. It works except I get the dreaded "Transparent Huge Pages" warning:
WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
Redis is currently too slow to be useful... So I tied turning off THP:
sheena#gke-projectwaxd-cluster-default-pool-23593a74-wxrv ~ $ cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
sheena#gke-projectwaxd-cluster-default-pool-23593a74-wxrv ~ $ echo never > /sys/kernel/mm/transparent_hugepage/enabled
-bash: /sys/kernel/mm/transparent_hugepage/enabled: Permission denied
sheena#gke-projectwaxd-cluster-default-pool-23593a74-wxrv ~ $ sudo echo never > /sys/kernel/mm/transparent_hugepage/enabled
-bash: /sys/kernel/mm/transparent_hugepage/enabled: Permission denied
These permission errors are disconcerting. Redis wants THP off so it can work properly.
I did a little digging and found that google uses a special os image that makes /sys/ a read-only path. There's an alternative image that's based on Debian 7. It got me all excited but in the end I have exactly the same problem.
So how do I stop redis from being effected by THP on Google container engine?
It's not like I'm doing something unique here. Running databases in containers is pretty normal. And it's pretty normal for a database to malfunction when THP is enabled. So... what am I missing here?
Your command is slightly incorrect: echo runs as root but the redirection itself (>) runs as user so it can't write /sys/.
The following command works fine both on container-vm (debian based) and gci (chromeos based):
sudo sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'
Persisting this setting on container-vm
Add this kernel command line parameter into /etc/default/grub (don't forget to run sudo update-grub and sudo reboot afterwards):
GRUB_CMDLINE_LINUX="... transparent_hugepage=never"
Persisting this setting on gci
First, using the cloud console copy the instance template that is in use by the node pool.
Second, under metadata change the value for userdata:
#cloud-config
write_files:
- path: /etc/systemd/system/hugepage.service
permissions: 0644
owner: root
content: |
[Unit]
Description=Disable THP
[Service]
Type=oneshot
ExecStart=/bin/sh -c "echo never > /sys/kernel/mm/transparent_hugepage/enabled"
[Install]
WantedBy=kubernetes.target
...
runcmd:
- ...
- systemctl enable hugepage.service
- systemctl start kubernetes.target
Third, change the instance template to the newly created one:
gcloud compute instance-groups managed set-instance-template \
gke-YOUCLUSTER-YOURPOOL-grp \
--template=YOURNEWTEMPLATENAME \
--zone=...
Forth, recreate the instace(s):
gcloud compute instance-groups managed recreate-instances \
gke-YOUCLUSTER-YOURPOOL-grp \
--zone=... \
--instances=...
The instances will loose all data and come up with THP disabled. All new instances will have THP disabled as well (in this node pool).
Background:
I'm trying to figure out how to use ConfigFS to set up an HID device on BeagleBoneBlack.
I found the following example (www.isticktoit.net/?p=1383) on the web and tried it. The sample runs on a Raspberry Pi Zero. However, the sample does not work on my BBB. The following is the script that I wrote and which is executed as root. The script attempts to define a keyboard hid device.
#!/bin/bash
cd /sys/kernel/config/usb_gadget/
modprobe libcomposite
modprobe usb_f_hid
mkdir -p isticktoit
cd isticktoit
echo 0x1d6b > idVendor # Linux Foundation
echo 0x0104 > idProduct # Multifunction Composite Gadget
echo 0x0100 > bcdDevice # v1.0.0
echo 0x0200 > bcdUSB # USB2
mkdir -p strings/0x409
echo "fedcba9876543210" > strings/0x409/serialnumber
echo "Tobias Girstmair" > strings/0x409/manufacturer
echo "iSticktoit.net USB Device" > strings/0x409/product
mkdir -p configs/c.1/strings/0x409
echo "Config 1: ECM network" > configs/c.1/strings/0x409/configuration
echo 250 > configs/c.1/MaxPower
# Add functions here
pwd
mkdir -p functions/hid.xyz
echo 1 > functions/hid.xyz/protocol
echo 1 > functions/hid.xyz/subclass
echo 8 > functions/hid.xyz/report_length
echo -ne \\x05\\x01\\x09\\x06\\xa1\\x01\\x05\\x07\\x19\\xe0\\x29\\xe7\\x15\\x00\\x25\\x01\\x75\\x01\\x95\\x08\\x81\\x02\\x95\\x01\\x75\\x08\\x81\\x03\\x95\\x05\\x75\\x01\\x05\\x08\\x19\\x01\\x29\\x05\\x91\\x02\\x95\\x01\\x75\\x03\\x91\\x03\\x95\\x06\\x75\\x08\\x15\\x00\\x25\\x65\\x05\\x07\\x19\\x00\\x29\\x65\\x81\\x00\\xc0 > functions/hid.xyz/report_desc
ln -s functions/hid.xyz configs/c.1/
# End functions
ls /sys/class/udc > UDC
The error that I get is "ls: write error: Devicew or resource busy".
I am running Debian Jessie - Linux version 4.4.9-ti-r25
I did an lsmod and libcomposite and usb_f_hid are loaded.
The usb device controller, musb-hdrc-0.auto, is loaded.
Questions:
How can I tell which device is busy?
Where can I find the USB configfs defect bug list for BBB.
Is there a logging file and enabling parameter that would give me a clue as to what is happening?
Thanks for any help
David Glaser
The problem you are likely having with the beaglebone black is the cdc_acm driver. It is difficult to remove (well, not really now that I KNOW how to do it) if you don't know how because the steps aren't just laid out to find anywhere yet. I found this: https://media.defcon.org/DEF%20CON%2023/DEF%20CON%2023%20presentations/DEFCON-23-Phil-Polstra-One-device-to-Pwn-them-all.pdf
which led me to the following solution
#!/usr/bin/env bash
function checkModule(){
MODULE="$1"
if lsmod | grep "$MODULE" &> /dev/null ; then
echo "$MODULE" found.
return 0
else
echo "$MODULE" not found.
return 1
fi
}
if which 'systemctl' | grep "systemctl" &> /dev/null ; then
systemctl stop serial-getty#ttyGS0.service >/dev/null
fi
if checkModule "g_serial" == 0; then
modprobe -r g_serial
fi
if checkModule "usb_f_acm" == 0; then
modprobe -r usb_f_acm
fi
if ! checkModule "libcomposite" == 0; then
modprobe libcomposite
fi
basically, it stops the serial-getty service which allows you to remove the g_serial device and then this allows you to remove usb_f_acm. This in turn removes the libcomposite device, which you actually want to keep. Once this is done, you can likely do all the things you needed to do. I got a nice HID keyboard working this way (well, okay I guess its a KeygleBone Black now)... It is pretty dirt simple once you understand ALL of the pieces, but I'm having a little trouble tearing my device back down. I might not need to eventually, but I'd like to be able to do that and it seems that certain directories cannot be removed (namely the "strings" directories that I've created). This means I can't really fully tear down the device, but maybe I only need to:
echo "" > /sys/kernel/config/usb_gadget/my_gadget/UDC
to actually tear it down. I haven't worked that part out yet. There are also some C libraries, but I've got a bunch of python scripts that I want to use and I don't yet have python wrappers for those. But that probably isn't too much work.
I didn't want to forget to mention, that I tried to throw the above script into rc.local so the beaglebone black I am using would be "HID ready" on boot. There are probably better locations and methods to do this, but I just wanted to use rc.local because the above is a script, rc.local is a script, it should run on boot... But it doesn't... You have to make sure to make rc.local runable ( chmod 755 /etc/rc.local ) as well as modifying the default shell it wants to run (well, it always runs bash, but its method for running bash is the "POSIX" method, and that doesn't seem to work, so you have to force it to run bash in non-POSIX mode with:
#!/usr/bin/bash
Again, there are probably other better methods (I was lazy here and, well, I'm just old school), especially if your device is going to be an IoT device or anything linked to the net, so you might want to consider something else if you need this script to run on boot.
I did stupidly leave out one thing: I made sure the beaglebone black doesn't present its usual "disk" portion as well. I would put the details here, but frankly, those I'd have to track back down. I basically googled around for how to disable the beaglebone black disk. It isn't hard and amounted to me moving some file to another name so it doesn't find the "USB disk" configuration on boot. You can also change a line in the uboot config somewhere I believe, but I didn't really want to do that.
Found the file: /var/local/bb_usb_mass_storage.img
Well, it might be bbg_usb_mass_storage.img if it is a beaglebone green, but I just moved this file so it wouldn't present the mass storage device. That should allow you to do what you want.
When I run
strace -f strace /bin/ls
to know how strace work it failed with
ptrace(PTRACE_TRACEME, 0, 0, 0) = -1 EPERM (Operation not permitted)
even with root account.
It there any solution for this?
Docker
When running strace within Docker container, to enable ptrace, run with SYS_PTRACE param:
docker run -it --cap-add SYS_PTRACE ubuntu
See: Running Strace in Docker.
ptrace system call is limited only one tracing application per process.
man ptrace:
EPERM The specified process cannot be traced. This could be because the tracer has insuffi‐
cient privileges (the required capability is CAP_SYS_PTRACE); unprivileged processes
cannot trace processes that they cannot send signals to or those running set-user-
ID/set-group-ID programs, for obvious reasons. Alternatively, the process may already
be being traced, or (on kernels before 2.6.26) be init(1) (PID 1).
This means only a debug application can attach to same process. When you done strace -f you tell it to attach all process started by program debugged. In your case strace call fork to create a new process and setups the new process for debugging using ptrace system call. Then it calls exec with parameters you provide to the call. This then start strace again which tries to do fork and ptrace again. But the second ptrace fails with EPERM because first strace has already attached to the process.
Running first strace without -f parameter allows you to trace the first thread from second strace while second strace is tracing the ls.
strace strace -f ls
There is -b to detach from lwp when a specific syscall is made but it only supports execve. If there was a ptrace call support it would be perfect. That means strace either needs a small patch to support ptrace call.
Alternative potential hacks include preloaded library which implements detaching with some trickery.
Better alternative would be using tracing tool systemtap or trace-cmd which can use kernel provided tracing infrastructure instead of ptrace.
I mention this and more helpful tips in a recent blog post about strace.
You need to enable support for gdb, strace, and similar tools to attach to processes on the system.
You can do this temporarily by running command to set a setting proc:
sudo bash -c 'echo 0 > /proc/sys/kernel/yama/ptrace_scope'
You can persist that setting between system reboots by modifying /etc/sysctl.d/10-ptrace.conf and setting kernel.yama.ptrace_scope = 0.
If your system does not have /etc/sysctl.d/10-ptrace.conf, you can modify /etc/sysctl.conf and set kernel.yama.ptrace_scope = 0.
I'm trying to automate some things on remote Linux machines with bash scripting on Linux machine and have a working command (the braces are a relict from cmd concatenations):
(ssh -i /path/to/private_key user#remoteHost 'sh -c "echo 1; echo 2; echo 3; uname -a"')
But if an ampersand is concatenated to execute it in background, it seems to execute, but no output is printed, neither on stdout, nor on stderr, and even a redirection to a file (inside the braces) does not work...:
(ssh -i /path/to/private_key user#remoteHost 'sh -c "echo 1; echo 2; echo 3; uname -a"') &
By the way, I'm running the ssh client dropbear v0.52 in BusyBox v1.17.4 on Linux 2.4.37.10 (TomatoUSB build on a WRT54G).
Is there a way to get the output either? What's the reason for this behaviour?
EDIT:
For convenience, here's the plain ssh help output (on my TomatoUSB):
Dropbear client v0.52
Usage: ssh [options] [user#]host[/port][,[user#]host/port],...] [command]
Options are:
-p <remoteport>
-l <username>
-t Allocate a pty
-T Don't allocate a pty
-N Don't run a remote command
-f Run in background after auth
-y Always accept remote host key if unknown
-s Request a subsystem (use for sftp)
-i <identityfile> (multiple allowed)
-L <listenport:remotehost:remoteport> Local port forwarding
-g Allow remote hosts to connect to forwarded ports
-R <listenport:remotehost:remoteport> Remote port forwarding
-W <receive_window_buffer> (default 12288, larger may be faster, max 1MB)
-K <keepalive> (0 is never, default 0)
-I <idle_timeout> (0 is never, default 0)
-B <endhost:endport> Netcat-alike forwarding
-J <proxy_program> Use program pipe rather than TCP connection
Amendment after 1 day:
The braces do not hurt, with and without its the same result. I wanted to put the ssh authentication to background, so the -f option is not a solution. Interesting side note: if an unexpected option is specified (like -v), the error message WARNING: Ignoring unknown argument '-v' is displayed - even when put in background, so getting output from background processes generally works in my environment.
I tried on x86 Ubuntu regular ssh client: it works. I also tried dbclient on x86 Ubuntu: works, too. So this problem seems to be specific to the TomatoUSB build - or inside the "dropbear v0.52" was an unknown fix between the build in TomatoUSB and the one Ubuntu provides (difference in help output is just the double-sized default receive window buffer on Ubuntu)... how can a process know if it was put in background? Is there a solution to the problem?
I had the similar problem on my OpenWRT router. Dropbear SSH client does not write anything to output if there is no stdin, e.g. when run by cron. I presume that & has the same effect on process stdin (no input).
I found some workaround on author's bugtracker. Try to redirect input from /dev/zero.
Like:
ssh -i yourkey user#remotehost "echo 123" </dev/zero &
It worked for me as I tried to describe at my blog page.