Set apache documentRoot to symlink (for easy deployment) - apache

we are looking for a way to point our Apache DocumentRoot to a symlink.
E.g. DocumentRoot /var/www/html/finalbuild
finalbuild should point to a folder somewhere like /home/user/build3
when we move a new build to /home/user/build4 we want to use a shell script that changes the symbolic link "finalebuild" to this new directory /home/user/build4 and do an apache graceful restart to have a new web application version up and running with little risk.
What's the best way to create this symlink and to change this link afterwards using the shell script?

We're using capistrano to employ a similar setup. However, we've run into a few problems:
After switching to the setup, things appeared to be going fine, but then we started noticing that after running cap deploy, even though the symlink had been changed to point toward the head revision, the browser would still show the old pages, even after multiple refreshes and appending different GET parameters.
At first, we thought it was browser caching, so for development we disabled browser caching via HTTP headers, but this didn't change anything. I then checked to make sure we weren't doing full-page caching server-side, and we weren't. But I then noticed that if I deleted a file in the revision the symlink used to point to, we would get a 404, so Apache was serving up new pages, but it was still following the "old symlink" and serving the pages up from the wrong directory.
This is on shared hosting, so I wasn't able to restart Apache. So I tried deleting the symlink and creating a new one each time. This seemed to work sometimes, but not reliably. It worked probably 25~50% of the time.
Eventually, I found that if I:
removed the existing symlink (deleting it or renaming it);
made a page request, causing Apache to attempt to resolve the symlink but find it missing (resulting in a 404)
then created a new symlink to the new directory
it would cause the docroot to be updated properly most of the time. However, even this isn't perfect, and about 2-5% of the time, when the deploy script ran wget to fetch a page right after renaming the old symlink, it would return the old page rather than a 404.
It seems like Apache is either caching the filesystem, or perhaps the mv command only changed the filesystem in memory while Apache was reading from the filesystem on disk (doesn't really make any sense). In either case, I've taken up someone's recommendation to run sync after the symlink changes, which should get the filesystem on disk in sync with memory, and perhaps the slight delay will also help the wget to return a 404.

I've used symlinks as the apache DocumentRoot in production with no graceful restart necessary. In general, the idea should work. A 403 error probably indicates a permissions error unrelated to the symlink changing. An added wrinkle that you would want to add is making the symlink switch atomic so the symlink always exists. That is to say, at no time is the symlink nonexistent, even for a moment.
The solution to this problem is to effect the change by creating a new symlink and then renaming it over the old symlink. On Unix-like systems, renaming is an atomic operation, and thus the symlink “change” will be atomic too. By hand, the process looks like this:
$ ln -s new current_tmp && mv -Tf current_tmp current

Related

Apache: DocumentRoot set on symlink, points on the actual file?

I'm building a deployment script and something seems wrong in the way Apache deals with my symlink.
I have a folder containing:
foo_svn: folder (my current website)
foo_tmp: folder (a copy)
web: symlink (pointing on foo_svn)
My DocumentRoot is set on web.
Before svnupdating foo_svn, I backup the contents in foo_tmp and I replace the symlink so that it points on foo_tmp.
Then I update foo_svn: let's say index.php has been modified. The website should be the same, since it now points on foo_tmp and that folder hasn't changed. And yet the page displayed is the new one, as if Apache looked directly for foo_svn/index.php instead of web/index.php.
In the vhost config file, DocumentRoot is still .../web, same as in the <Directory>. The symlink web still points on foo_tmp, and foo_tmp/index.php is still the old version.
EDIT: Digging a little deeper, it looks like there's something really weird with some kind of cache. Hard-reloading my page two times, sometimes it points first on foo_tmp, then on foo_svn, then on foo_tmp (then it stabilizes). Sometimes it just takes some time for the page to accept the new version (even hard-reloading). Sometimes it shows the new page in foo_svn too early, because the symlink is still pointing on the previous version in foo_tmp. I'm going insane over this, it's like Apache has a kind of "symlink cache," I don't know.

can't open fossil repo over web

I've been strugling for a couple of days with this problem, but can't seem to fix it, I think I'm almost there.... but... not quite :(
This is where I am at.
I'm on a headless debian server, running virtualmin / webmin for creating my domains / users etc. I don't know if this will mess things up, but I'm happy to modify the config files manually (via webmin or via ssh/vim).
I am attempting to run fossil as a cgi service over apache.
its an internal site, named as homeserver.net I can reach the default pages just fine, and add in and create links etc as I want to.
Please note that the solution to my problem is at the end of the question.
so the files are located on disk at, which tallys up with my apache document root
/home/homeserver/www
I would like to run fossil to have both the internal site, and later on and dev work that I practice on in separate files. So I created a new directory for these repositories.
/home/homeserver/repos/web/site.fossil
/home/homeserver/repos/dev/ [no repository yet!]
reading the instructions on the fossil page I have inserted a short cgi file called 'fos_repo.cgi' that reads as.
#!/usr/bin/fossil
directory: /home/homeserver/repos
notfound: http://www.homeserver.net/site404.htm
when I open the link to
www.homeserver.net/cgi-bin/fos_repo.cgi
I get redirected to the 404 page that I have written. So the script is clearly being read and working.
From reading the fossil pages I understand that I should be able to use the following link to open/access the repo.
www.homeserver.net/cgi-bin/repos/web/site
I'm not sure why this isn't working...
so far I have tried the following.
I opened the repository from the cli, and had the server run in the background
fossil server site.fossil &
I though maybee the file should have been inside the main repo directory, not inside a sub directory, so I moved it... it now lives in
/home/homeserver/repos/site.fossil
I tried creating an alias to the file in apache
Alias /home/homeserver/repos/web/site.fossil /home/homeserver/www/repos
When I browse to
www.homeserver.net/repos/site
I get nothing, but going to
www.homeserver.net/repos/site.fossil
will attempt to downloaded the file (which is a binary)
so I think I'm getting somewhere, but I'm not sure what I'm missing.
I've used fossil before, but I ran it as a local server, and started it up as and when I needed it.
I'm running it like this so as I can eventually push the site out to a live VPS (maybe even finish up hosting the fossil site on the VPS also).
ps I really liked fossil when I used it before, and loved the whole integrated wiki and bug tracker, and the fact I could simply copy the file to my external drive to do a backup. Personally don't really want to change to something else, but if I have to....
thanks in advance.
David
Edit: trying other options.
So I thought I would try the single repository method shown on the fossil page, so adjusted my cgi script accordingly.
Now when I navitage to : www.homeserver.net/cgi-bin/fos_repo.cgi I get the following message returned
SQLITE_CANTOPEN: cannot open file at line 30276 of [f5b5a13f73]
SQLITE_CANTOPEN: os_unix.c:30276: (21) open(/home/homeserver/repos)
however if I ssh to the server an start it manually with
fossil server site.fossil
I can get to the server with www.homeserver.net:8081
So I either have a problem with my SQLite usage in apache or something else wrong. Plesse help
Solution
So for reasons of simplicity I've decided that using a single cgi file for each repo is what I am going to go with.
My initial directory structure was as follows:
/home/homeserver/www
/home/homeserver/www/repos
/home/homeserver/www/repos/web # for web site development
/home/homeserver/www/repos/dev # for other development
I think part of my problem was that I was hoping that having the directory: pont to my repos/ location fossil would find the site.fossil file (located in repos/web) and the dev.fossil file (located in repos/deb).
Obviously this didn't work.
The reason I wanted it too look like this was for separation of the information on my system.
For some reason I had decided that pointing fossil as repos/ would give me a nice fossil style front page and links to my repositories automatically. However After having used the directory: version and getting the following error message
Unable to find or open the project repository
I realised that I was still going to need to write my front page to the repositories, and that my expectation was a little too much.
So I've decided to run with a single cgi file pointing to each repo that i need to make.
Instead of
www.homeserver.net/cgi-bin/repos/web/site
try
www.homeserver.net/cgi-bin/repos.cgi/index
Reading your ( very long ) question again, I suggest trying
www.homeserver.net/cgi-bin/fos_repos.cgi/index

Server not servering random files

I currently run on Ubuntu 14.04.1 LTS. I'm not a server admin by any means, and just really have basic setup/config knowledge. I got a virtual box so I could learn more. Unfortunately, I just took a bit of a leap and screwed everything up.
I'm running Apache2, PHP, MySQL. This morning, I got a prompt saying I could update my server. I wasn't thinking and decided to go through with it. In addition to completely screwing up my server settings, I'm pretty sure I've lost data. But first I need to get my server running properly again (and probably never update without paying someone to do it for me).
I got most things up and running, but now when I go to my site (gamersplane.com), some paths end up 404'ing, and going to the Apache 404 instead of hitting the htaccess. The way I have it setup, the htaccess should get all files (barring things like css, js, images, xml, etc), and direct it all to a file called dispatch.php which processes the url and determines what to do/load. Basically, some stuff goes to the htaccess, other stuff isn't. I'm not sure why.
Mainly, files directly under /javascript won't load, but files in subfolders of /javascript do. In addition, going to /tools (folder exists) works, but /tools/cards (folder doesn't exist) doesn't. Strangely though, /forum (folder exists) and /forum/3 (doesn't exist) both work. I can't figure out when things do or don't work (systematically).
apache.conf: http://pastebin.com/A3J4bXE3
gamersplane.conf: http://pastebin.com/TAD06h59
Update
I found out about package called javascript-common which forces /javascript to search elsewhere. I uninstalled the package, and JavaScript is now loading. I'm told I need the package, but I don't know why.
This still doesn't solve why some other paths don't work.

Apache Following Old Symbolic Link After Updating The Link

I have apache pointing to a symbolic link for a website. I'm using capistrano to deploy the code so it updates the symbolic link to point to the new release
After capistrano updates the symbolic link to the new release directory apache uses the previous release directory.
The weird thing is it doesn't happen all the time. After a deploy everything either works fine or apache follows the previous link until i restart or reload apache.
any ideas?
This may depend on Apache httpd holding a handle on the old symlink. As long as that is the case the old data (symlink) is not thrown away in Unix based filesystems.
It seems you are unable to predict whether this is the case. Maybe use of lsof can help. What could also help is to trigger a graceful restart of httpd.

Is there a way to check if a directory exists in Apache configuration files?

Is there a way to include configuration settings in Apache based on if a directory exists? Basically I have a portable hard drive that I transport between work and home that has some stuff I'm developing on it. I only want the Apache config to load a particular virtual host if the folder exists.
Since Apache 2.4.34 you can now use <IfFile>...</IfFile> which will check to see if a file exists. There's more details on the <IfFile> page.
No, there seems to be no direct way to do this.
The only thing that might be a solution is the IfDefine directive. You can define defines using the -d parameter to when the server is started.
The parameter-name argument is a define as given on the httpd command line via -Dparameter-, at the time the server was started.
You might be able to check for the existence of a directory in a batch or bash file, and set the -d parameter accordingly.
Whether that is an option, will depend on how your server is started from the portable hard drive.
I've come up with a solution that seems to work for Linux and OS X, and it hinges on "mountpoints". It might be possible to emulate it within Windows, as well, but you would probably have to get creative with FUSE and/or Cygwin.
If you create an empty folder in your home directory, such as "/Users/username/ExtraVhosts", you can add an apache directive to "Include /Users/username/ExtraVhosts/*".
Then, when you insert your thumb drive, you can mount somewhere and then use mountpoint "binding" to cross-link the ExtraVhosts folder to a folder on the mobile device.
An OS X example:
I have a thumb drive called 'Cherrybomb'
When I insert it, it always gets mounted to /Volumes/Cherrybomb
I can then use bindfs (sudo port install bindfs) to mount a subfolder of it, like so:
sudo bindfs /Volumes/Cherrybomb/Projects/vhosts /Users/username/ExtraVhosts
Then I can restart apache to read in the updated configuration:
sudo /opt/local/apache2/bin/apachectl restart
At that point, it's just a matter of adding entries in /etc/hosts for server aliases to get picked up.
The linux equivalent would be using the "--bind" parameter of the mount command.
One caveat: This makes it difficult to quickly unmount the USB drive, since it is always marked as "in use" by apache. Here's a removal procedure:
Close all open files and terminal sessions that are using the drive (the present-working-directory in terminal can cause unmount issues)
Stop apache: sudo /opt/local/apache2/bin/apachectl stop
umount /Users/username/ExtraVhosts
Then you can either unmount it graphically or manually (umount /Volumes/Cherrybomb).
If your work and home machines mount the drive to different locations, you could have multiple vhosts folders - home_vhost, work_vhost, etc - and use that in the binding step.
I hope this helps someone out :)
If you point apache to the mountpoint only there shouldn't be an issue. Just don't point Directory directives to directories within the drive.
eg, if you mount /dev/somedisk /mnt/somevhost, the
/mnt/somevhost directory will be there whether or not you have the drive mounted and apache will start. Apache doesn't care if the directory is empty so a <Directory "/mnt/somevhost"/> won't cause server to not start if the drive isn't mounted.
Work with UNIX not against it :-p This solution should be sufficient for development.