We have a legacy COBOL application based on OpenVMS for-which we do not have a clear idea of configuration. In this context, by "configuration" I am talking about:
Which executable files comprise the application;
Which pristine source files correspond to which executable files.
It may seem odd that 1 above is something that is not known, but over time what has happened is that executables have "come and gone" (and many still remain used). The knowledge of which executable files constitute the application as it exists today is not known since knowledge of which executables are no longer required has been lost in time. In practical terms, the team faithfully compiles all source code files and deploy the resultant executables despite the fact that there are obviously programs that are no longer used.
It goes without saying that there is no formal configuration management process and the source code is not kept in a version control system. Since the application runs on OpenVMS, the corresponding Files-11-based file system keeps older versions of files (including source files) and this has long been the excuse for not putting the application source into a version control system (despite the reasons for using a VCS extending far beyond merely having a record of previous versions).
There are a number of ways in which the configuration can be determined, of course, but I'd like to start with a first "small step", that is: determine the set of executables that comprise the application. At this point I should mention that the executable components of the application are not limited to OpenVMS images, but also DCL command files. I would like to:
Log all invocations of images that reside in a certain directory or set of directories;
Log all invocations of command files that reside in a certain directory or set of directories.
If we run this logging on our production system over an extended period of time, say two months, we can get a pretty good idea of what the application comprises. Together with user consultation, we'll be able to confirm the need for the executable files that aren't being called.
I think I have an idea of how to do 1 above, although I'm not sure of the specifics, that is, use SET/AUDIT. The second part, at this stage, I have no idea of how to do.
So, the main criterion for this effort is that as little of the existing system be affected in order to gain the above information. Due to the question mark around the configuration (and the complete lack of automated tests), changing anything is a nerve-wracking undertaking.
Using operating-system-level services like SET/AUDIT would allow one to get to know what's being run without the need to change source and/or recompile anything. So, my question is a multi-parter:
Is this the optimal way to do this on OpenVMS?
What would I need to do to restrict SET/AUDIT to only monitor images in a particular directory?
How would I log command file invocation without changing the .COM source files?
What should I expect in terms of performance degradation as a result of logging such information?
Ad 2., 3.
I would try security auditing with ACLs. From a a privileged account, something like ...
Make sure ACL auditing is enabled:
$ show audit
should show
System security audits currently enabled for:
...
ACL
...
If it doesn't, enable it with
$ set audit/audit/enable=acl
and then you may want to disable it when you are done with
$ set audit/audit/disable=acl
Set audit ACLs on all the wanted files:
$ set sec/acl=(audit=security,access=success+execute) [.app]*.com
$ set sec/acl=(audit=security,access=success+execute) [.app]*.exe
and you may want to delete the ACLs when you are done with
$ set security/acl=(audit=security,access=success+execute)/delete [.app]*.com
$ set security/acl=(audit=security,access=success+execute)/delete [.app]*.exe
You can check what ACLs are set with:
$ show security [.app]*.*
Run you application ...
Get the results from the audit file
$ analyze/audit [vms$common.sysmgr]security.audit$journal/sel=access=execute/full/since=17:00/out=app.log
Check your report for your files:
$ pipe type app.log |search sys$pipe "File name", ,"Access requested"
File name: _EMUVAX$DUA0:[USER.APP]NOW.COM;1
Access requested: READ,EXECUTE
Auditable event: Object access
File name: _EMUVAX$DUA0:[USER.APP]ECHO.EXE;1
Access requested: READ,EXECUTE
$
Sorry, I have no answer for 1. and 4.
It would help to know the OpenVMS Version (e.g. 6.2, 7.3-2, 8.4...) and the architecture (Vax, Alpha,Itanium).
Recent OpenVMS versions have great sda extensions
http://h71000.www7.hp.com/doc/84final/6549/6549pro_ext1.html
or
http://de.openvms.org/Spring2009/05-SDA_EXTENSIONS.pdf
such as LNM to check the logical names used by a process, PCS for PC sampling of a process, FLT to check the faulting behavior of applications, RMS for RMS data structures, PERF only for Itanium performance tracing, PROCIO for the reads and writes for all files opened by a process
Post a
dir sys$share:*sda.exe
so that we know which Sda extensions are available for you.
You can always check what a process with a pid of 204002B4 does with
$ ana/sys
set proc/id=204020b4
sh process /channel
exam #pc
and repeat while the process moves on.
Related
My program deploys with a configuration option that I've chosen to expose as a feature. This option can be one of two values.
Each configuration changes a set of settings files. They have different input file names (for the sake of example, let's call it option1-config20-lv80.xml), but should be installed to the configuration directory as config20-lv80.xml. Each option has a prefix that should be stripped like that, which also means only one of these options can be selected for install at a time. However, even with conditions preventing the install of one feature when the other is selected, my output is littered with:
LGHT0204: ICE30: The target file 'config20-lv80.xml' is installed in 'path' by to different components... This breaks component reference counting.
How can I give my users the option to choose between these configuration options and get around my ICE30 issues without any negative side effects?
I saw an similar question answered, but I'm not 100% sure how to implement it in wix#, or if there are other ways open to me to achieve my goal without disabling ICE30 validation or creating 2 installers.
A bit rushed, have a look...
Milk & Honey Winnie: In cases like this I prefer to install both files with different names using two different components and then switch between them with an option shown in the application itself. On launch or in the preferences. Makes deployment simpler, it is already complex (section "The Complexity of Deployment"). The linked answer you refer to can work technically, as can more hacky approaches.
Alternatives: I have a long answer here on different ways to install settings files: Create folder and file on Current user profile, from Admin Profile ranging from eliminating the whole file and using internal defaults, to downloading settings files from the network or just relying on clouded web-service settings retrieval from a database. Not 100% match, but maybe give it a skim?
A related issue is when you have a settings file that regular users can't write to. This is a list of approaches for eliminating that condition: System.UnauthorizedAccessException while running .exe under program files.
First of all, I don't mean version control such as git.
I do use git locally but, I'm trying to determine the best way to do back-ups of source code (as well as other app assets) in case of hardware failure or such.
I was thinking I could set up a script to tar my project folders, and encrypt them with gpg. I would then save the encrypted tar to external hard drives and to 1 or more off-site locations using a service such as amazon drive or dropbox.
Currently, I'm a sole developer so my thinking was that this method should be okay. But I wanted to get some input to make sure I'm doing this the best/most reliable way possible.
If there is a better approach to this that may be more applicable to small teams, then please let me know, as I'm more than happy to do the extra work implementing the approach.
There are much of ways of doing that.
But, if you always work local and you need a simple way of doing that, you may take a look at run scripts if some specific usb device is plugged in.
Meaning that a simple backup script with tar would run if you plug in your specific backup hdd.
Take a look at udev rules in linux.
udev is a generic device manager running as a daemon on a Linux system and listening (via a netlink socket) to uevents the kernel sends out if a new device is initialized or a device is removed from the system. The udev package comes with an extensive set of rules that match against exported values of the event and properties of the discovered device. A matching rule will possibly name and create a device node and run configured programs to set up and configure the device.
Take a look at these posts:
https://unix.stackexchange.com/questions/65891/how-to-execute-a-shellscript-when-i-plug-in-a-usb-device
&
https://askubuntu.com/questions/401390/running-a-script-on-connecting-usb-device
If you plan to go further, to extend the team or even to keep your code for a while in other words, if you want to be professional, I would go with a scalable and reliable tool designed for this: use a real backup and restore tool and don't use scripts. A lot of people, small (and even not so small) companies are doing it and they end up in trouble: maintenance, scalabolity, update, and so on.
There are plenty of backup & restore tools for different purposes and/or platforms, prices and so on. https://en.wikipedia.org/wiki/List_of_backup_software would be a good start :)
Cheers
Werlan
My goal is to write to a file (that the user whenever the user launches an application, such as FireFox) and timestamp the event.
The tricky part is having to do this from the kernel (or a module loaded onto the kernel).
From the research I've done so far (sources listed below), the execve system call seemed the most viable. As it had the filename of the process it was handling which seemed like gold at the time, but I quickly learned that it wasn't as useful as I thought since this system call isn't limited to user-related operations.
So then I thought of using ps -ef as it listed all the current running processes and I would just have to filter through which ones were applications opened by the user.
But the issue with that method is that I would have to poll every X seconds so, it has the potential to miss something if the user launched and closed an application within the time that I didn't call ps -ef.
I've also realized that writing to a file would be a challenge as well, since you don't have access to the standard library from the kernel. So my guess for that would be making use of proc somehow to allow the user to actually access the information that I'm trying to log.
Basically I'm running out of leads and I'd greatly appreciate it if anyone could point me in the right direction.
Thanks.
Sources:
http://tldp.org/LDP/lkmpg/2.6/html/x978.html (not very recent)
https://0xax.gitbooks.io/linux-insides/content/SysCall/syscall-4.html
First, writing to a file or reading a real file from the kernel is a bad idea which is not used in the kernel. There is of course VFS files, like /sys/fs or /proc, but this is a special case and this is allowed.
See this article in Linux Journal,
"Driving Me Nuts - Things You Never Should Do in the Kernel" by Greg Kroach-Hrtman
http://www.linuxjournal.com/article/8110
Every new process that is created in Linux, adds an entry under /proc,
as /proc/pidNum, where pidNum is the Process ID of the new process.
You can find out the name of the new application which was invoked simply by
cat /proc/pidNum/cmdline.
So for example, if your crond daemon has pid 1336, then
$cat /proc/1336/cmdline
will give
cron
And there are ways to monitor adding entries to a folder in Linux.
I have a very large tar file(>1GB) that needs to be checked out and is a precondition for executing any tests.
I cannot have dedicated build server for my tests since tests are going to be executed on slave machines which are disposable.
Checking out a file(>1GB) is not optimal since in this case test execution time would increase because of precondition.What is the best optimal way of solving this problem?
I would dedicate a location on the slaves for that file.
Then in your tests, check if the file is in that location. If not, check it out and move it there. Since this location is outside your normal work area it won't get cleaned, and the file will stay there for the next test execution to use, and you won't need to check it out again.
Of course if the file changes you have to clear those caches. A first option would be to do this manual, alternative you can create a hash of the file and keep that hash in the cash and in your version control. You would then compare only the hashes, and only if those change you would check out the file.
Of course this requires that you have the ability to checkout all the rest of your code without the big file. How to do that obviously depends on the version control system in use.
Where can I find a well-respected reference that details the proper handling of PID files on Unix?
On Unix operating systems, it is common practice to “lock” a program (often a daemon) by use of a special lock file: the PID file.
This is a file in a predictable location, often ‘/var/run/foo.pid’. The program is supposed to check when it starts up whether the PID file exists and, if the file does exist, exit with an error. So it's a kind of advisory, collaborative locking mechanism.
The file contains a single line of text, being the numeric process ID (hence the name “PID file”) of the process that currently holds the lock; this allows an easy way to automate sending a signal to the process that holds the lock.
What I can't find is a good reference on expected or “best practice” behaviour for handling PID files. There are various nuances: how to actually lock the file (don't bother? use the kernel? what about platform incompatibilities?), handling stale locks (silently delete them? when to check?), when exactly to acquire and release the lock, and so forth.
Where can I find a respected, most-authoritative reference (ideally on the level of W. Richard Stevens) for this small topic?
First off, on all modern UNIXes /var/run does not persist across reboots.
The general method of handling the PID file is to create it during initialization and delete it from any exit, either normal or signal handler.
There are two canonical ways to atomically create/check for the file. The main one these days is to open it with the O_EXCL flag: if the file already exists, the call fails. The old way (mandatory on systems without O_EXCL) is to create it with a random name and link to it. The link will fail if the target exists.
As far as I know, PID files are a convention rather than something that you can find a respected, mostly authoritative source for. The closest I could find is this section of the Filesystem Hierarchy Standard.
This Perl library might be helpful, since it looks like the author has at least given thought to some issues than can arise.
I believe that files under /var/run are often handled by the distro maintainers rather than daemons' authors, since it's the distro maintainers' responsibility to make sure that all of the init scripts play nice together. I checked Debian's and Fedora's developer documentation and couldn't find any detailed guidelines, but you might be able to get more info on their developers' mailing lists.
See Kerrisk's The Linux Programming Interface, section 55.6 "Running Just One Instance of a Program" which is based on the pidfile implementation in Stevens' Unix Network Programming, v2.
Note also that the location of the pidfile is usually something handled by the distro (via an init script), so a well written daemon will take a command line argument to specify the pidfile and not allow this to be accidentally overridden by a configuration file. It should also gracefully handle a stale pid file by itself (O_EXCL should not be used). fcntl() file locking should be used--you may assume that a daemon's pidfile is located on a local (non-NFS) filesystem.
Depending on the distribution, its actually the init script that handles the pidfile. It checks for existence at starting, removes when stopping, etc. I don't like doing it that way. I write my own init scripts and don't typically use the stanard init functions.
A well written program (daemon) will have some kind of configuration file saying where this pidfile (if any) should be written. It will also take care to establish signal handlers so that the PID file is cleaned up on normal, or abnormal exit, whenever a signal can be handled. The PID file then gives the init script the correct PID so it can be stopped.
Therefore, if the pidfile already exists when starting, its a very good indicator to the program that it previously crashed and should do some kind of recovery effort (if applicable). You kind of shoot that logic in the foot if you have the init script itself checking for the existence of the PID, or unlinking it.
As far as the name space, it should follow the program name. If you are starting 'foo-daemon', it would be foo-daemon.pid
You should also explore /var/lock/subsys, however that's used mostly on Red Hat flavors.
The systemd package on Red Hat 7 provides a man page daemon(7) with the header line "Writing and packaging system daemons."
This man page discusses both "old style" (SysV) and "new style" (systemd) daemonization. In new style, systemd itself handles the PID files for you (if so configured to do so). However, in old style, the man page has this to say:
In the daemon process, write the daemon PID (as returned by getpid())
to a PID file, for example /run/foobar.pid (for a hypothetical daemon
"foobar") to ensure that the daemon cannot be started more than once.
This must be implemented in race-free fashion so that the PID file is
only updated when it is verified at the same time that the PID
previously stored in the PID file no longer exists or belongs to a
foreign process.
You can also read this man page online.