From what I read osquery is used for querying / reading the system information.
By any chance it has facility to modify the system state like killing the process or deleting a registry key ??
I am using osqueryi commands like select * form users before diving in programatically.
Generally not.
osquery itself aims to not change anything in the filesystem. The main distribution has no mechanisms that would do that. (Except, of course, it's local state files)
osquery extensions, however, can be written to do whatever the extension author desires. Further, osquery supports the idea of "writeable tables" which extensions may use to present a simpler interface.
Check out https://blog.trailofbits.com/2018/05/30/manage-your-fleets-firewalls-with-osquery/ for a writable table example.
Related
First of all, I don't mean version control such as git.
I do use git locally but, I'm trying to determine the best way to do back-ups of source code (as well as other app assets) in case of hardware failure or such.
I was thinking I could set up a script to tar my project folders, and encrypt them with gpg. I would then save the encrypted tar to external hard drives and to 1 or more off-site locations using a service such as amazon drive or dropbox.
Currently, I'm a sole developer so my thinking was that this method should be okay. But I wanted to get some input to make sure I'm doing this the best/most reliable way possible.
If there is a better approach to this that may be more applicable to small teams, then please let me know, as I'm more than happy to do the extra work implementing the approach.
There are much of ways of doing that.
But, if you always work local and you need a simple way of doing that, you may take a look at run scripts if some specific usb device is plugged in.
Meaning that a simple backup script with tar would run if you plug in your specific backup hdd.
Take a look at udev rules in linux.
udev is a generic device manager running as a daemon on a Linux system and listening (via a netlink socket) to uevents the kernel sends out if a new device is initialized or a device is removed from the system. The udev package comes with an extensive set of rules that match against exported values of the event and properties of the discovered device. A matching rule will possibly name and create a device node and run configured programs to set up and configure the device.
Take a look at these posts:
https://unix.stackexchange.com/questions/65891/how-to-execute-a-shellscript-when-i-plug-in-a-usb-device
&
https://askubuntu.com/questions/401390/running-a-script-on-connecting-usb-device
If you plan to go further, to extend the team or even to keep your code for a while in other words, if you want to be professional, I would go with a scalable and reliable tool designed for this: use a real backup and restore tool and don't use scripts. A lot of people, small (and even not so small) companies are doing it and they end up in trouble: maintenance, scalabolity, update, and so on.
There are plenty of backup & restore tools for different purposes and/or platforms, prices and so on. https://en.wikipedia.org/wiki/List_of_backup_software would be a good start :)
Cheers
Werlan
Is it possible to execute a method as a different user in Linux (or SELinux specifically)? The programs that I have run in individual sandboxes, each with a different user and process id. I have a situation where I have to execute a branch of code as a different user and with different process id to prevent the access of the memory and disk space of the code that's spawning it.
If not possible, can you throw some light on how much of the kernel code has to be changed to achieve it? (I understand its subjective. Alternatively, if you can suggest what and how to go about it, that will be much helpful).
Protecting some resources from other codes executing on the same machine is precisely what lead to the process and UID invention.
If you are searching for a mechanism that looks like a simple function call, I would say it's impossible because it requires the memory to be shared between the caller and the callee. However, using fork/exec (or wrappers like system()) will give you some isolation as long as you deal with parameters/results using system objects like program parameters or pipes.
Although, the fact that *nix user is meant to protect processes from one-another, requires that an explicit relationship be built between two users to have one user act on behalf of the other.
Actually, you may want to:
define a sudoers policy which gives the right to your first user to run a command (or a particular command) as the second one.
use popen() (or system()) in your first program to call the less privileged code.
if any, pass the parameters and parse the result from stdout
As an extra, you may use the same binary for both executions, this way, all the code can be at the same location.
I am interested in creating a routine that would query the currently running cache processes and then write this information to a file. How could this be done in Cache 2008.2?
PERFMON might be what you're looking for. That's app with it's own UI, but you can call it's functions directly too, as an API.
Check the Cache docs for "Cache Monitoring Guide". That will give you links to PERFMON docs, as well as docs for other system monitoring tools.
You might find something useful in the Class Reference, under packages %SYSTEM, %SYS, and %Monitor.
For some process info you might need to shell out to the OS. In that case check into the $ZF function. That will let you invoke os-level commands from within Cache.
Oh, and you might want to consider saving the process data within the Cache DB, rather than dumping it out to a file. That is, create a Persistent Class with Properties corresponding to each process attribute that you want to capture, then write code to create, populate, and save instances of that class, taking the data from PERFMON or whatever other source you choose.
If you do that you can use Cache SQL to generate whatever kind of report you need. (Cache will automatically generate a SQL Table corresponding to your Persistent Class.) Cache supports ODBC, so you can use an external tool like Crystal Reports or Access for that part.
Obviously that will be more work than just echoing data to a file, but some kind of structure will be needed if you're going to do anything interesting with the information.
I have a particular file I want to monitor for file read attempts by all applications on OSX. I'd like to be able to interrupt the requests so I could decide which applications have permission to read the file and which don't (by querying the user, or checking a cache of user responses). Is this possible with the OSX API? If not, is it even possible to get a list of which applications or processes do read a file?
I'm not saying there's no way to do this, but what #Jonathan is talking about isn't it.
That API is for tracking the creation, change, and destruction of files. Notably this tool is used by things like Spotlight to watch activity on the filesystem for new, interesting files.
But, wisely, reading isn't one of the events it tracks.
And even if reading WAS tracked, it is still the wrong mechanism, as it's a notification system after the fact, not in line with the call itself.
I seriously doubt what you want is possible the way you describe it.
With Access Control Lists, you can limit access at the user level (Fred can read the file, but Bob can not). This is a setting on the file itself. But there's no mechanism to allow Bobs App1 to read a file, while Bobs App2 can not, since there's really no formal mechanism of "application identity" beyond the command to executed, or whatever the program "says" its name is (both of which can be spoofed if motivated enough).
However, feel free to crawl the Darwin sources -- no doubt the answer is buried in there somewhere near the open(2) call.
EDIT, regarding comment.
What are you trying to do? What's the overall context?
Another thing that you may want to try is to use FUSE.
FUSE is a utility that let's you have "user space filesystems". People use FUSE for many purposes, like reading NTFS volumes, or mounting remote system via SSH.
They have a simple example, that gives you a skeleton that you can fill in for your purposes.
For most of the use cases, you'll simple defer to the system. However, for OPEN you will add your logic. Then you could point your FUSE utility at a directory, and "mount it". Then all of the files below that directory can use your new behavior.
I'm still not sure how you will identify Apps by name, but if it's not a real "security" issue, just for local control, I imaging you can come up with something. Activity Monitor has apps names, so they must be available, and FUSE will be running within the process space (I think), rather than through some external mechanism.
All that said, I think FUSE is your best bet, but it's probably not appropriate if you want to do this to "any file" with no preparation by the user (like not installing FUSE). If you wanted to do "any file", your FUSE system would need to be mounted at root, and then you'll simply have a full "clone" of the filesystem, with those files from the normal root "unprotected", while those from your new FUSE root will be protected. So, if someone wanted to NOT use your FUSE system, the real file is readily available to them through the actual file location.
If not, is it even possible to get a list of which applications or processes do read a file?
The command-line tool fs_util allows you to monitor filesystem activity, including reading.
The writings of Amit Singh should come in very handy. He explored the API that provides FileSystem events a few years ago, and provided a sample tool that allows you to intercept FS events. It's open source!
If i remember his conclusion properly, their isn't an official API, but you can use apple's tools to achieve what you want.
I'm running a very computationally intensive scientific job that spits out results every now and then. The job is basically to just simulate the same thing a whole bunch of times, so it's divided among several computers, which use different OSes. I'd like to direct the output from all these instances to the same file, since all the computers can see the same filesystem via NFS/Samba. Here are the constraints:
Must allow safe concurrent appends. Must block if some other instance on another computer is currently appending to the file.
Performance does not count. I/O for each instance is only a few bytes per minute.
Simplicity does count. The whole point of this (besides pure curiosity) is so I can stop having every instance write to a different file and manually merging these files together.
Must not depend on the details of the filesystem. Must work with an unknown filesystem on an NFS or Samba mount.
The language I'm using is D, in case that matters. I've looked, there's nothing in the standard lib that seems to do this. Both D-specific and general, language-agnostic answers are fully acceptable and appreciated.
Over NFS you face some problems with client side caching and stale data. I have written an OS independent lock module to work over NFS before. The simple idea of creating a [datafile].lock file does not work well over NFS. The basic idea to work around it is to create a lock file [datafile].lock which if present means file is NOT locked and a process that wants to acquire a lock renames the file to a different name like [datafile].lock.[hostname].[pid]. The rename is an atomic enough operation that works well enough over NFS to guarantee exclusivity of the lock. The rest is basically a bunch of fail safe, loops, error checking and lock retrieval in case the process dies before releasing the lock and renaming the lock file back to [datafile].lock
The classic solution is to use a lock file, or more accurately a lock directory. On all common OSs creating a directory is an atomic operation so the routine is:
try to create a lock directory with a fixed name in a fixed location
if the create failed, wait a second or so and try again - repeat until success
write your data to the real data file
delete the lock directory
This has been used by applications such as CVS for many years across many platforms. The only problem occurs in the rare cases when your app crashes while writing and before removing the lock.
Why not just build a simple server which sits between the file and the other computers?
Then if you ever wanted to change the data format, you would only have to modify the server, and not all of the clients.
In my opinion building a server would be much easier than trying to use a Network file system.
Lock File with a twist
Like other answers have mentioned, the easiest method is to create a lock file in the same directory as the datafile.
Since you want to be able to access the same file over multiple PC the best solution I can think of is to just include the identifier of the machine currently writing to the data file.
So the sequence for writing to the data file would be:
Check if there is a lock file present
If there is a lock file, see if I'm the one owning it by checking that its content has my identifier.
If that's the case, just write to the data file then delete the lock file.
If that's not the case, just wait a second or a small random length of time and try the whole cycle again.
If there is no lock file, create one with my identifier and try the whole cycle again to avoid race condition (re-check that the lock file is really mine).
Along with the identifier, I would record a timestamp in the lock file and check whether it's older than a given timeout value.
If the timestamp is too old, then assume that the lock file is stale and just delete it as it would mea one of the PC writing to the data file may have crashed or its connection may have been lost.
Another solution
If you are in control the format of the data file, could be to reserve a structure at the beginning of the file to record whether it is locked or not.
If you just reserve a byte for this purpose, you could assume, for instance, that 00 would mean the data file isn't locked, and that other values would represent the identifier of the machine currently writing to it.
Issues with NFS
OK, I'm adding a few things because Jiri Klouda correctly pointed out that NFS uses client-side caching that will result in the actual lock file being in an undetermined state.
A few ways to solve this issue:
mount the NFS directory with the noac or sync options. This is easy but doesn't completely guarantee data consistency between client and server though so there may still be issues although in your case it may be OK.
Open the lock file or data file using the O_DIRECT, the O_SYNC or O_DSYNC attributes. This is supposed to disable caching altogether.
This will lower performance but will ensure consistency.
You may be able to use flock() to lock the data file but its implementation is spotty and you will need to check if your particular OS actually uses the NFS locking service. It may do nothing at all otherwise.
If the data file is locked, then another client opening it for writing will fail.
Oh yeah, and it doesn't seem to work on SMB shares, so it's probably best to just forget about it.
Don't use NFS and just use Samba instead: there is a good article on the subject and why NFS is probably not the best answer to your usage scenario.
You will also find in this article various methods for locking files.
Jiri's solution is also a good one.
Basically, if you want to keep things simple, don't use NFS for frequently-updated files that are shared amongst multiple machines.
Something different
Use a small database server to save your data into and bypass the NFS/SMB locking issues altogether or keep your current multiple data files system and just write a small utility to concatenate the results.
It may still be the safest and simplest solution to your problem.
I don't know D, but I thing using a mutex file to do the jobe might work. Here's some pseudo-code you might find useful:
do {
// Try to create a new file to use as mutex.
// If it's already created, it will throw some kind of error.
mutex = create_file_for_writing('lock_file');
} while (mutex == null);
// Open your log file and write results
log_file = open_file_for_reading('the_log_file');
write(log_file, data);
close_file(log_file);
close_file(mutex);
// Free mutex and allow other processes to create the same file.
delete_file(mutex);
So, all processes will try to create the mutex file but only the one who wins will be able to continue. Once you write your output, close and delete the mutex so other processes can do the same.