I've been referring to http://toroid.org/ams/git-website-howto as a starting point for a production web server running on a managed VPS. The VPS runs cPanel and WHM, and it will eventually host multiple client websites, each with its own cPanel account (thus, each with its own Linux user and home directory from which sites are served). Each client's site is a separate Git repository.
Currently, I'm pushing each repository via SSH to a bare repo in the client's home folder, e.g. /home/username/git/repository.git/. As per the tutorial above, each repo has been configured to checkout to another directory via a post-receive hook. In this case, each repo checks out to its own /home/username/public_html (the default DocumentRoot for new cPanel accounts), where the files are then served by Apache. While this works, it requires me to set up (in my local development environment) my remotes like this:
url = ssh://username#example.com/home/username/git/repository.git/
It also requires me to enter the user's password every time I push to it, which is less than ideal.
In an attempt to centralize all of my repositories in one folder, I also tried pushing to /root/git/repository.git as root and then checking out to the appropriate home directory from there. However, this causes all of the checked-out files to be owned by root, which prevents Apache from serving the site, with errors like
[error] [client xx.xx.xx.xx] SoftException in Application.cpp:357: UID of script "/home/username/public_html/index.php" is smaller than min_uid
(which is a file ownership/permissions issue, as far as I can tell)
I can solve that problem with chown and chgrp commands in each repo's post-receive hook--however, that also raises the "not quite right" flag in my head. I've also considered gitosis (to centralize all my repos in /home/git/), but I assume that I'd run into the same file ownership problem, since the checked-out files would then be owned by the git user.
Am I just approaching this entire thing the wrong way? I feel like I'm completely missing a third, more elegant solution to the overall problem. Or should I just stick to one of the methods I described above?
It also requires me to enter the user's password every time I push to it, which is less than ideal
It shouldn't be necessary if you publish your public ssh key to the destintion account ".ssh/authorized_keys" file.
See also locking down ssh authorized keys for instance.
But also the official reference Pro Git Book "Setting Up the Server".
Related
I created an application that runs on a localhost server using expressjs. And I also bought a domain.
I'm wondering if there is a way to take that localhost server and turn it into a real shared server
I tried once to use a hosting service like hostgator but I still don't know how I can turn the express app into a real website.
I have no experience with any web development services so please don't tell me to use ....... whatever because I will have no idea what that is.
For one thing it is not clear how your website actually works: if it is only express does it generate HTML or is it purely JSON passed to browser clients via get requests (to each their own).
There are so many options as to how you might do this: one of the best options is to first make sure your server runs on Docker. Find a tutorial on YouTube/google/Stack Overflow/Blogs on how to run your Express server with docker. If you do that you can deploy it to a Container manager like Google/Amazon/Digital Ocean. If this seems hard to you there are other options.
Presumably you run your server with something like npm start. This guide can show you how to do essentially that but on a cloud computer.
Before you begin make sure that you're locally working server is checked in to a cloud Git provider like Github, GitLab, Bitbucket, etc.
Since Amazon AWS, and Google Cloud have free tier or options for hosting for free for a certain amount of time (AWS 1 year) or for a certain amount of money (Google Cloud). These two seem like viable place to start.
If you find the option that you'd like you'll need to:
create an account
Create a server (choose a cheap one especially initially like mice/small/cheap etc).
Find a tutorial on how to "SSH" into that server (which basically means remotely control the terminal on that server). Google actually makes this fairly easy there's a big button that says SSH into this server.
Once you've logged into that Computer you'll be able to run the same commands you probably normally do on your home computer:
The computer you'll be getting is likely to be a virtual Linux Computer probably something like Linux Ubuntu. Find a tutorial on how to get git and node installed there (but it's something like sudo apt-get update && sudo apt-get install git node).
Once you have git and node try mkdir www and cd into that: mkdir www && cd project (This isn't critical but conventional.)
Copy the link that allows you to "Clone your repo using HTTPS" (there's a link at the top right of your GitHub (or others) repo that allows you to do that. You'll need to enter your password
Now all the files that you had on your computer are on this new computer.
Next you'll have to probably npm i to install your dependent NPM packages. (This assumes you properly used .gitignore to prevent GitHub from being filled with extra copies of your npm packages.)
Now you should be able to run your code as usual: npm run start
If all those steps work you'll want to get something that will run these "forever" like https://www.npmjs.com/package/forever npm i -g forever or even better: https://www.npmjs.com/package/pm2 will allow you to continuously run your express server.
Finally, you'll need to configure this server on AWS/Google/whatever service you're using to push traffic coming in on port 80 and 443 to port 3000 and open traffic to all. And depending on the service you chose that's different so find a tutorial for doing just that part.
This will only allow people across the internet to see your service on an AWS URL or a google URL. But it's a good chance to make sure everything works perfectly. Once you're happy with everything associate your purchased domain with that special AWS/Google domain. You can do that on the AWS side, or the GoDaddy/NameCheap/where-ever you bought your domain side.
For the docker option you can download aws-cli tools and upload your built docker container to AWS and have it available. Find a tutorial to do that.
Essentially your question is very broad so I sometimes brushed over some details, but this is essentially what you have to do.
So, I am new to the GitLab server. Now, what I want to achieve is this:
Allow access to repositories only on certain ssh-keys. There are a limited no of machines and a limited no of users, so if a user adds an ssh-key outside these sets of keys, the repo should not clone there. Because my team size is small, I am okay if I only add those public keys to the account.
I am fine with the idea of ssh access but currently, as an admin, I lose the freedom to conveniently track or choose which all ssh-keys can access my repo. Can I disable users from adding ssh keys?
Is there any other way to ensure this? Would instead of having ssh enabled access HTTPS with whitelisting IP-enabled access work?
GitLab was, in the beginning (2011) based upon gitolite, but switched to its own mechanism in 2013.
Nowadays, it is best to declare a GitLab project private and add users to said project: that way you won't have to manage SSH or HTTPS access: any user who is not part of that project won't be able to see it/clone it (HTTPS or SSH).
In other words, repository access is no longer based on SSH keys (not for years), but is based on project visibility.
The OP adds:
even if a user is part of a project, he should only be able to clone the project on certain remote machines.
That is not a Git or GitLab feature, which means you need:
to restrict Git protocols on GitLab to SSH only
change the gitlab-shell SSH forced command script in order to allow commands only coming from some IPs
There is access to group by IP address restriction feature, since GitLab 12.0 (June 2019), but... only in GitLab Ultimate (meaning: "not free").
I wish to run some untrusted code using chroot.
However, many claim that chroot is not a security feature and can easily be broken out of.
Therefore my question is how does apps like https://ideone.com/ manage to run untrusted code quickly and securely. Also if chroot can be broken out of, couldn't it be possible to break out of chroot in https://ideone.com/ .
I'm not sure what untrusted code you're referring to, but chroot just changes the apparent file structure -- theoretically you can't see above a certain level. But symbolic links can still work so if your chrooted directory has a symlink to a directory above, chroot doesn't do you any good.
It's also possible for applications can get access to resources through other applications. Some clever hackers know how to exploit running apps, but if you set up the ideone environment as root, well, anything is possible.
More theoretically, you could install a master application that has full access to the file system. Then you run code in a chrooted environment. If that master app is running and listening it can relay resources to your chrooted application.
Quickly? Sure... Securely... well... with my example the master app is a gatekeeper but it's still security through trust of the master app.
I'm trying to convert some projects at work from subversion to git. The projects are websites and our current subversion setup uses davfs to mount the repository and point apache's document root there. This way apache in dev runs the code currently checked-into the svn repository.
mount:
mount.davfs http://code.repository/svn/site.com /mnt/davfs/site.com
httpd.conf:
ServerName site.com
DocumentRoot /mnt/davfs/site.com
I'm looking for a way to mimic this setup with git. But, from what I understand, mounting a git repository (yes, our git repo is accessed over http) this way will result in the git repository internals showing up as the docroot and not the code itself.
example:
ls /mnt/davfs/gitrepository
Parent Directory
HEAD
branches/
config
description
hooks/
info/
objects/
refs/
Does anyone know if there is a way to achieve the desired effect?
Thanks!
If you want to be able to browse the code, you should be using something like gitweb. If you want to push/pull from the repo, then the internals should be showing up as the docroot.
In a bare repository (the kind that you would use for such a central repo, since you generally don't want to push to non-bare repos), there is no actual checkout of the code files on disk, the only things in that bare repo are the "git internals".
If you want to get a copy of the code on the server out of the repository, you probably want to use git archive - possibly in a post-receive hook if you want it to run every time new code is pushed to the repository. See the following man pages for details:
http://www.kernel.org/pub/software/scm/git/docs/git-archive.html
http://git-scm.com/docs/githooks
Well the git repo is another beast. If you want to browse source code, you need something like gitweb. Mind you, if you just pointed to a svn repo, you'd be looking at internals too; The /mnt/davfs/site.com is probably hosted with apache mod-dav-svn which does something similar as gitweb would do.
You'll want to look at gitweb or competition. Gitweb is IMHO the simplest to setup
https://git.wiki.kernel.org/index.php/Gitweb
For sharing your repository (to make it clonable, e.g.) just serve the tree as static HTTP pages (as docroot directly) because davfs will not (reliably) make it possible for others to push to your repo anyway.
Pushing would be done using the Smart HTTP server, git-daemon or over ssh
I'm running an Apache web server and was wondering what's the best way to deploy changes (from github) to the web server?
/var/www/ right now is only writable by root.
Should I have my git project directly in /var/www/? (so it creates /var/www/.git/?)
However, when I need to run commands (i.e. sudo git push) wouldn't work (since my ssh keys are not under sudo).
Would I be better off making /var/www/ writable by myself (and not just root)? Or should I add ssh keys to the root user? Or should I do something else entirely?
Thanks.
I use rsync to sync the contents of my local machine with the server, and if you're just deploying to one server, then it's pretty simple (and Capistrano is overkill.). I put the following aliases in ~/.bash_profile:
alias eget='rsync -avie ssh matt#example.com:sites/example.com/www/ ~/Projects/example/example.com/www/ --exclude .DS_Store --exclude ".git*" --delete-after'
alias edep='rsync -avuie ssh ~/Projects/example/example.com/www/ matt#example.com:sites/example.com/www/ --exclude .DS_Store --exclude ".git*" --delay-updates --delete-after'
Then, from the git repo on my local machine. I do:
git commit -am 'commit some changes'
git pull --rebase # pull any new changes from remote (--rebase prevents an unnecessary merge commit.)
eget -n # confirm which files I've changed
If it looks fishy, I could do eget without the -n and then just do a git diff -w. Then, I could do git checkout -- path/to/file for the files I want to keep my changes for. Then, I commit the changes that were on the server that I didn't get yet. This would only happen if the files on the server are changing in a different way than from deployments. Otherwise, you know that your local version is always more up to date than the files on the server and so don't have to worry about overwriting things on the server that you don't yet have on your local. Continue...
edep -n # just see what files will be deployed/updated/etc.
edep # looks good. Deploy for real.
Done!
Check out the rsync(1) Mac OS X Manual Page for more info.
Another option is to use the Git post-receive hook. But, you'll have to install Git on the server to do that. Also, I recommend putting the .git directory outside of your public www directory for security & cleanliness reasons. You can do this with the Git core.worktree configuration option. For example, from ~/git/example.com.git, do git init --bare; git config core.worktree ~/sites/example.com/. That makes ~/git/example.com.git like the .git dir for ~/sites/example.com/.
Create a central repository, use the branching of git to create different branches for different purposes, and never serve all of your repository publicly, nor should you ever serve your .git directory publicly (since that's the same as serving everything you ever did with the code or put in the repository to the public). Off the top of my head, here are the steps I recommend, from my own experience:
Create a central/hub repository for the code. (optional, but recommended. Even better is using github.com for your central repository). You can then check out local copies for local deployments, e.g. when you want to recreate the site on your laptop. Not necessary, but very convenient and makes sure your site is portable. You can have a staging repository and staging branch for development purposes. You can also have a repo and branch for production purposes.
Create a explicitly public directory in the repository that is not the root of the repository: E.g. Create a /www/ or /served/ or /public/ directory within the repository. This is stuff that will be publicly available and indexable by search engines, so be careful what goes in there. Assume that anything that goes in there is public knowledge, cached for eternity, and will be the target of security vulnerability attacks (because that could easily be the truth).
Create the dev repository: git clone the central repository on the server (e.g. cd /home/tchal then git clone git#github.com:tchalvak/ninjawars.git ) , though ideally in a folder that has shared permissions for your developer group.
Create a symbolic link for you development site: cd /var/www/ , ln -s /url/to/shared/repository/public/ nickNameForDevSiteHere , creating a symbolic link to only the served/public files of the site, creating a simple development level site. (optional, but recommended). In that manner, the dev site can easily be accessible via some ip and a nickname, e.g. http://10.0.1.123/publicdevelopmentsitenickname without the need of a real domain name.
Specify the live & deployed code commit. You may well want to create a live-branch for whatever code is currently "live", just be aware that this branch will probably have to be forcibly overwritten periodically, e.g. git branch live-branch git push -f origin live-branch. Consider it a snapshot of your code, and not a branch that will stay stable.
When you're sure that the dev site has been tested well enough, either deploy the live-branch code manually, via a custom deploy script, or use a distinct repository with the live-branch checked out in it, serving only the explicitly public content, similar to the dev site.
create a virtualhost in apache for the domain name. For example, you could use something like:
<VirtualHost *>
ServerName greatdomain.com
ServerAlias www.greatdomain.com
DocumentRoot /srv/greatdomain/www/
</VirtualHost>
That's a huge topic, so if you're not clear on all of the details, I recommend getting into further research of setting up a virtualhost in apache.
Point your DNS for the domain name at the ip of the server.
In summary, you can pretty easily use git to deploy all of your code using specific-to-each-deployment-type branches. Won't help with syncing, for example, databases between deployments, but that is a step that you could figure out after you have things running, as a second tier of deploying a site, and do it manually in the meantime.