After using crashplan for a while, I noticed that several files aren't being backed up. The files are synced via unison (through cygwin) with another PC and while the *nix permissions are copied correctly, the mirrored file does not have SYSTEM as a user (in windows). Therefore, crashplan can't back it up. Both client and server are running cygwin.
What's the best solution? Can I copy this permission as well with unison? Can I do it with a script (in cygwin or cmd)?
Thanks
Sander
EDIT: To fix it short term I ran an icacls command, but I'm still looking for a way to copy the ACLs via unison whilst syncing.
Relevant section from the Unison Manual:
Permissions
Synchronizing the permission bits of files is slightly tricky when two different filesytems are involved (e.g., when synchronizing a Windows client and a Unix server). In detail, here's how it works:
When the permission bits of an existing file or directory are changed, the values of those bits that make sense on both operating systems will be propagated to the other replica. The other bits will not be changed.
When a newly created file is propagated to a remote replica, the permission bits that make sense in both operating systems are also propagated. The values of the other bits are set to default values (they are taken from the current umask, if the receiving host is a Unix system).
For security reasons, the Unix setuid and setgid bits are not propagated.
The Unix owner and group ids are not propagated. (What would this mean, in general?) All files are created with the owner and group of the server process.
Related
I'm trying to install bit vise ssh client but its not installing and throwing an exception as this.
Exception caught:
Failed to create directory "C:\Program Files(x86)\Common Files\Bitvise"
CreateDirectory() failed: windows error 5: Access is denied.
My system is 64 bit, I know bitvise has one version which supports both 64 and 32bit.
I also tried "run as Administrator", still same exception. Could anyone tell me the procedure to install it properly !
Logging: Always create an MSI log for debugging when encountering any deployment problems. See that link for hints on interpreting the log file content. Search for "value 3" first of all:
msiexec.exe /i C:\Path\Your.msi /L*vx! C:\Your.log
In general: check vendor web sites and / or user forums to figure out details on known issues. It could be a permission issue on your TEMP folder.
Emergency Approach: Use a clean virtual machine to get the software running. Try different OS-versions. Just for a heartbeat in a pinch. Or try someone else's computer. Obvious yes, but try it if you can.
Keep in mind that "very clean" virtuals (there is absolutely nothing on there - just a fresh OS) could lack certain runtimes that might be "taken for granted" and hence missing from an installer. VCRuntime, .NET versions and such. Just in case you see mysterious errors there too.
First Checks: A simplified, generic check-list for deployment issues:
AD / Group Policies: Corporate environments could have group policies and restrictions preventing the installation of anything at all. Check that first.
Installation Media: Re-download installation media to ensure its integrity.
Corrupted by Malware: Note that malware or other factors can corrupt downloaded files, but more commonly they are destroyed in-transit.
Wrong Bitness: The setup could be the wrong bitness (x64 on 32 bit system) or architecture such as Itanium (incompatible with normal x64 systems). Or even the wrong OS (zip file wrappers etc...).
Corrupted / Quarantined by Scanners: Security suites, firewalls, corporate blocks and the likes can cause problems (separate issue below - not sure if anti-virus programs try to clean binaries anymore? Block they certainly do).
Incomplete Download: Launching before download is fully finished (premature launch) is a classic weirdness - error messages are generally ok, but can be misleading. Remember to allow anti-virus scanners to complete their post-download scan. This can take much longer than you think (they hash the file, check their site, etc...).
Download Mirror Issue: Sometimes the download comes from a number of download servers, some of which could be corrupted or contain faulty media or be misconfigured. Download again - check with virustotal.com and repeat a few times to verify. Have your colleague in another office download? Different mirror likely (automatic load-balancing - when you can't pick another server yourself).
Network Problems (LAN): When you have problems, try to copy installation files to a local location (the desktop will do) to eliminate any LAN network issues as the source of your deployment problem. If there are network problems file copy might fail with a proper warning message? Network related fallacies. More towards bottom.
Missing Runtimes: A few, very core-runtimes can make setups fall over. This is particularly common on virtual machines that are "fresh" and basic.
Examples would be: VCRedist (in particular), .NET, Powershell, etc...
Lacking and more advanced components such as IIS, MSSQL, .NET Core, Java, etc... can also make some badly authored setups fall over.
Admin Rights: Ensure you have real admin rights on the box in question. In other words you are logged on using a real administrator account. Avoid "run-as" if you have a failure to look at. Try a real login.
Reboot: Just to try the obvious. Reboot and allow the PC to "settle down after reboot". This means you ensure that Windows Update hasn't started installing - or something else that was set to start pending the next reboot. PCs that are seldomly rebooted can have a lot going on after a reboot - some try to "reboot twice" - or even several times - to make sure all locks and blocks from "stuff that is happening" are released. Make sure to allow update operations to finish before rebooting once, twice or more (wait for reboot prompt). Virtual machines that are reverted to a previous state can be a nightmare when it comes to things that automatically start to update and cause confusion and problems.
Clean Slate: If you don't reboot, close down all applications before running your setup. This sorts out various locks and blocking happenstances. Preferably reboot first and run the setup the first thing you do when the machine is back up again. Again: give the machine enough time to be idle - everything started (services and such - and no updates installing).
Disk Space & Integrity: Ensure available free disk space AND that there are no errors on disk. The very small SSD and NVME disks of the last few years have made this problem more acute again.
Different user: Try installing as a different and real admin user. The important thing here is that this is a different admin account than you first tried (user account profile issues). So, in other words log in as a real admin user and don't just use "run as" (create a new account if you need to). An example of a problem could be someone who has messed up their user profile shell folder settings so that the directory table resolution of MSI fails. Another user profile would normally be unaffected and still work OK.
ACL - Access Control: Very often access denied can be related to custom NTFS ACL configuration that is erroneous. This can lead to weird error messages during installation. In corporate environments - with application packagers adapting installers - ACLs are sometimes modified extensively to tighten security. I have seen this a lot, but there are also other sources of ACL changes such as system administrator scripts, malware and I saw issues after a security fix from Windows Update a few years ago. Tightened security can trigger a lot of errors previously unseen in software that should "know better".
Malware check: Run anti-virus or Windows Defender to verify that you don't have a malware issue on your box. Additionally check the installation media with https://www.virustotal.com/ to ensure it is not malware itself! (the setup.exe could be infected, or the whole product could be malware outright - never know).
Security Software Interference: Anti-virus, firewalls, scanners and other security products can be overactive and block access to a folder or a resource so it looks like it is an ACL permission issue. Disable temporarily if possible when required. Do anti-virus software still try to fix binaries in the age of digital certificates? I am not sure. Always check installation file using virustotal.com.
Localized Setups: Sometimes setups made for other languages than English - or rather another language than the original setup (could be any language) - fail on systems with other languages installed and in use. Try on a clean virtual with the "setup-expected language". Problems like these indicate VERY bad setup design (hard coded localized paths, incorrect server paths or addresses due to translation errors, etc...) - but due to QA resources they are not uncommon. In essence the main-language version is generally (in almost all cases) put through better testing.
Mount Points: Some disks have mounted drives in folders and such things - this can cause some seriously weird problems. Try on a clean virtual with no drama-settings.
NTFS / FAT32: (Somewhat edge-case). It is no longer possible to install Windows 10 on a FAT32 drive - with the limitations that strike (no ACL permissions, max 4gb files, no journaling and such). However, the setup could be redirected to a non-system FAT32 partition or some other disk format. This could trigger security problems (no ACL permissions), but should not generally create any access denied issues - barring any custom actions trying to apply ACL permissions and failing (this might degrade gracefully by now, I don't know). However there are file size limitations in FAT32 disks (4gb) that might actually trigger errors these days for huge setups (games, video files, etc...). Note that downstream Windows OSs might still allow FAT32 system partitions. And finally - and importantly - FAT32 is not a journaling file system. This means data corruption can easily happen without self-correction.
Flagged Downloaded File: In newer versions of Windows downloaded files are flagged as "This file came from another computer and might be blocked to help protect this computer". See screenshot below. Read more details about the feature here and Digital signatures, false positives, tagged downloaded file. Just make sure your file does NOT have this flag (I do not have a complete overview of all problems that can result from this):
Odds and Ends: There are additional things such as setups being to old to install properly (they don't handle modern Windows features well - you can try to run the setup in compatibility mode by enabling this in the property page for the setup.exe file itself) and older Installshield setups had lots of DCOM-based installscript engine issues and such things. Other setup vendors have their own problems - and quite a few of them for older setups. Brand new stuff, and ancient stuff - always surprises.
Network Problems (LAN): This is mentioned above in the "Installation Media" section. You can copy files locally to try to eliminate LAN network problems as a source of problem (SAMBA problems, network overload and packet loss, interfering scanners, timeouts, etc...). You might get a real error message if you try to copy local. Try to download file directly from the Internet vendor site to the desktop as a test. Network related fallacies.
Update: Incompatibilities: It happens that software can't co-exist properly. These situations can be rather obvious (COM version incompatibilities, setups designed to detect existing software and prevent themselves from installing, setups in different language versions quarreling, etc...) or quite hard to work out (deep-seated driver problems, hardware peculiarities, anti-virus false positives or otherwise unsolvable problems). Make sure you test your setup on a clear virtual whenever you have problems. You can also use that as a "solution" if it works - have people run incompatible software on virtuals - obviously one of the key use cases for virtuals (there are many others).
Full Check List: See Section "Generic Tricks? in this answer for more.
Smartscreen issues: Digital signatures, false positives, tagged downloaded file
Running the pdf2htmlEX.exe Windows binary from the command prompt works as expected. While, running the pdf2htmlEX Windows binary in a wrapper (.Net in my case) I received an error like the one below.
__tmp_font1.ttf is not in a known format (or uses features of that format fontfo
rge does not support, or is so badly corrupted as to be unreadable)
Cannot load font C:\Users\admin\AppData\Local\Temp\pdf2htmlEX-5RLDCX/__tmp_fo
nt1.ttf
This is a pretty ambiguous error, and appears to be frequent among users when using the windows binary version.
Apparently Lu Wang wasn't able to offer a solution for Windows users, as all posts related are marked 'insufficient info'. Unfortunately, the pdf2htmlEX project is also archived, and no new comments can be added, so I'm adding this information here in the hope that this may help someone else in the future.
In my scenario, the library is called via an ASP.Net wrapper method using System.Diagnostics.Process to convert uploaded files into HTML versions. The Pdf2htmlEX library would work without issue from the Command Prompt, and for some reason, would also work perfectly in my development environment, but not in a production environment (Both of which are Windows Server 2012R2).
My first assumption, and correctly so, was that there was a permissions issue. Pdf2htmlEX uses FontForge internally to handle fonts, and one or both use the Windows Temp directory by default to store resource files used in the creation of the HTML and/or other files. And, I 'believe' although not confirmed, that it also may use the active user's %USERPROFILE%\AppData\Local\Temp folder...
When running test commands from Command Prompt, you are operating under your user context, and everything your user can do, Pdf2htmlEX can do. So everything works as expected.
In a server environment, the process is operating under the ApplicationPoolIdentity, a special IIS user type with limited permissions. Here it failed for me. While, I'd see folders and files created in the Windows Temp folder, they couldn't be opened by Pdf2HtmlEX to create the end files elsewhere.
Solution: (there may be other solutions for your individual case)
In my case, adding a new system user, adding that user to the Users group, and then setting the IIS worker process to that account resolved the issue. The reason I believe, is that the Users group has read/write access to the Windows Temp directory, and potentially other required areas of the system required for Pdf2htmlEX to complete.
I am doing an intensive computing project with a super old C program. The program requires a library called Sun Performance Library which is a commercial ware. Instead of purchasing the library by myself, I am running the program by logging onto a Solaris machine in our computer lab with the ssh command, while the working directory to store output data is still on my local Mac.
Now, a problem just occurred: the program uses large amount of disk space to save some intermediate results and the space on my local Mac is quickly filled (50 GB for each user prescribed by the administrator). These results are necessary for the next stage of computing and I cannot delete any of them before it finally produce the output data. Therefore, I have to move the working directory to an external hard drive in order to continue. Obviously,
cd /Volumes/VOLNAME
is not the correct way to do it because the remote machine will give me a prompt saying
/Volumes/VOLNAME: No such file or directory.
So, what is the correct way to do it?
sshfs recently added support for "slave mode" which allows you to do this. Assuming you have sshfs on Solaris (I'm not sure about this), the following command (ran from your Mac) will do what you want: dpipe /usr/lib/openssh/sftp-server = ssh SOLARISHOSTNAME sshfs MACHOSTNAME:/Volumes/VOLNAME MOUNTPOINT -o slave
This will result in the MOUNTPOINT directory on the server being mounted to your local external drive. Note that I'm not sure whether macOS has dpipe. If it doesn't, you can replace it with one of the equivalent solutions at How to make bidirectional pipe between two programs?. Also, if your SFTP server binary is somewhere else, substitute its path.
The common way to mount a remote volume in Solaris is via NFS, but that usually requires root permissions.
Another approach would be to make your application read its data from stdin and output its results to stdout, without using the file system directly. Then you could just redirect the data from/to your local machine through ssh. For instance:
ssh user#host </Volumes/VOLNAME/input.data >/Volumes/VOLNAME/output.data
After successfully mounting the directory (ZFS remote storage) from one of the server, I'm getting an "Operation not permitted" error when I try changing the ownership of the directory. I'm using the following command:
To mount the remote directory:
mount -t nfs 10.1.32.33:/dir/temp/tools /home/materials
After mounting the directory, the contents are belongs to nobody:nobody
I want to change ownership so I can run the installer inside the directory.
I'm using the command below to change ownership but it's not working:
chown -R otm:otm materials/
I can always upload the file to the server without using the ZFS storage, however I want to start making a central installer repository so I don't need to upload the files/installers for future server install. I appreciate your help guys.
NFS servers by default do not allow root access to files - root is normally mapped to "nobody".
See "root squash":
Root squash[2][3] is a reduction of the access rights for the remote
superuser (root) when using identity authentication (local user is the
same as remote user). It is primarily a feature of NFS but may be
available on other systems as well.
This problem arises when a remote file system is shared by multiple
users. These users belong to one or multiple groups. In Unix, every
file and folder normally has separate permissions (read, write,
execute) for the owner (normally the creator of the file), for the
group to which the owner belongs, and for the "world" (all other
users). This allows restriction of read and write access only to the
authorized users while in general the NFS server must also be
protected by firewall.
A superuser has more rights than an ordinary user, being able to
change the file ownership, set arbitrary permissions, and access all
protected content. Even users that do need to have root access to
individual workstations may not be authorized for the similar actions
on a shared file system. Root squash reduces rights of the remote
root, making one no longer superuser. On UNIX like systems, root
squash option can be turned on and off in /etc/exports file on a
server side.
After implementing the root squash, the authorized superuser performs
restricted actions after logging into an NFS server directly and not
just by mounting the exported NFS folder.
In general, you DO NOT want to disable root squash unless you REALLY know what you're doing as there are serious security issues you can create if you do that. And since you didn't even know it exists...
(And that mention of /etc/exports is an extremely limited statement that is wrong on many systems - like Solaris.)
If I open a file in my C/C++/Java code using a pathname that goes to an nfs directory, how the does the read and write syntax work with NFS being stateless and all? I have tried but cant find an example code accessing NFS mounted files. My current understanding is that it is the job of the NFS client to keep state (like read and write pointer) and the application uses the same syntax.
A related question is regarding VFS and UFS. Are all files in a current unix machine accessed through their vnodes first and then (depending on local vs remote) inode or rnode structures?
NFS (short of file locking) is no different than local storage to user-level applications. It might be slower, or it might drop out unexpectedly, but that can happen to local storage too. That's probably why you can't find specific NFS-centric example code.