How to create git remote repository using script RHEL 7 - git-remote

I apologize if this question has been asked but I could not find any answers when I did a search.
I am trying to create a GIT remote repository that is being hosted on our server (123.123.com) instead of gitlab.com using a script. I have found really great scripts that connected via https that I was able to tweak and get it to work in my vm # home but that configuration is user to gitlab.com.
I don't know how to tweak the script to replace things like "content-type json" .... or find the API for the server. So, I think ideally the below script written by ckalima is what I would like to use but I'm a noob to GIT and definitely scripting on a whole. https://gist.github.com/ckalima/1364886
I replaced all the variables and it does not work. In the end, it says it creates the repo but it really doesn't. Sorry for the long post, thanks in advance. Joybells
# remgit.sh
# Creates a remote git repository from the current local directory
# Configuration
# Replace SSH_USERNAME, SSH_HOST, SSH_GIT_PATH with your details
USER=SSH_USERNAME
HOST=SSH_HOST
GIT_PATH=SSH_GIT_PATH
REPO=${PWD##*/}
GIT_REMOTE_URL=ssh://$USER#$HOST/$GIT_PATH/$REPO
echo "-------------------------------------------"
echo "------ Building New Git Repository --------"
echo "-------------------------------------------"
# Setup remote repo
echo "--"
echo "-- Creating bare remote repo at:"
echo "-- $USER#$HOST/$GIT_PATH/$REPO"
echo "--"
ssh $USER#$HOST 'mkdir '$GIT_PATH'/'$REPO' && cd '$GIT_PATH'/'$REPO' && git --bare init && git --bare update-server-info && cp hooks/post-update.sample hooks/post-update && chmod a+x hooks/post-update && touch git-daemon-export-ok'
# Configure local repo
echo "--"
echo "-- Initializing local repo & pushing to remote"
echo "--"
touch .gitignore
git init
git add .
git commit -m 'initial commit'
git push --all $GIT_REMOTE_URL
git remote add origin $GIT_REMOTE_URL
git config branch.master.remote origin
git config branch.master.merge refs/heads/master
git fetch
git merge master
git branch -a
echo "--"
echo "-- Your new git repo '$REPO' is ready and initialized at:"
echo "-- $USER#$HOST/$GIT_PATH/$REPO"
echo "--"

Related

Github authentication error when trying to push on command line (SSH key added in Github)

I have generated and added my SSH key into Github.
Now I'm trying to write a script which generates a new private repository in Github and upload intiial files. User will be prompted for the name of the new repository.
The code that I have is given below. The problem is that when trying to push files into Github I get the following error
ssh: Could not resolve hostname github.com:rongardF: Name or service not known
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Why do I get this error? Is there anything else that I have to do?
The code:
#echo off
set /p name="Enter new project name: "
touch README.md
touch .gitignore
echo __Previews>>.gitignore
echo History>>.gitignore
echo Project Outputs *>>.gitignore
echo * Logs *>>.gitignore
git config --global user.email "r****n#gmail.com"
git config --global user.name "R** F****"
curl -u r***F:p*****d https://api.github.com/user/repos -d "{\"name\":\"%name%\",\"private\":true}"
git init
git add .
git commit -m "Initial commit"
git remote add origin git#github.com:r***F/%name%.git
git push -u origin master
NB: The names obviously don't include "*" symbol, this is just for blurring out my details.
git remote add origin git#github.com:r***F/%name%.git
For testing, you can check if this syntax would work in your case
git remote add origin ssh://git#github.com/r***F/%name%.git
Check also if a simple ssh -Tv git#github.com works (meaning if it display a Welcome message at the end)
Thanks for the suggestion #VonC
I got it to work by adding the following line before the push command:
git remote set-url origin https://github.com/r****F/%name%.git
Not sure why, but this works. So the snippet of the final code is:
...
...
git init
git add .
git commit -m "Initial commit"
git remote add origin git#github.com:r****dF/%name%.git
git remote set-url origin https://github.com/r****F/%name%.git
git push -u origin master

Azure DevOps private Linux agent - YAML pipeline checkout failing - can fix with "git config --global --unset http.extraHeader" but not early enough

We have an issue regarding the following:
Azure DevOps Linux Private Agent
Possible issue with corrupt / stale bearer token
Can be fixed with this command by logging onto the box but this is not convenient: git config --global --unset http.extraHeader
Can be fixed with this command as part of script in YAML: git config --global --unset http.extraHeader but not early enough. (See next comment below).
I can't run this command early enough in the pipeline YAML to clear the header as checkout is not controlled by me.
It generally only happens if a previous run fails at some point on the same private agent
Syncing repository: test-project-azure-workspace (Git)
git version
git version 2.26.0
git lfs version
git-lfs/2.10.0 (GitHub; linux amd64; go 1.13.4)
git config --get remote.origin.url
git clean -ffdx
git reset --hard HEAD
HEAD is now at 5f9fd24 sql mi
git config gc.auto 0
git config --get-all http.https://xxxxxxx#dev.azure.com/xxxxxxx/xxxxxxx/_git/test-project-azure-workspace.extraheader
git config --get-all http.proxy
git config http.version HTTP/1.1
git -c http.extraheader="AUTHORIZATION: bearer ***" -c http.proxy="http://10.XXX.XXX.XX:80" fetch --force --tags --prune --progress --no-recurse-submodules --unshallow origin
* Couldn't find host dev.azure.com in the .netrc file; using defaults
Here is the code that can remedy the symptom:
- script: |
echo '======================================================================'
echo 'list all of git config values for your convenience:'
echo '======================================================================'
git config --list
echo '======================================================================'
existing_header=$(git config --get http.extraHeader)
if [ ${#existing_header} -gt 0 ]
then
echo 'We found the http.extraHeader'
echo 'un-setting extra header: http.extraHeader 🔥'
git config --global --unset http.extraHeader
else
echo 'no extra header: http.extraHeader was not found. Nothing to unset 👍'
fi
condition: always()
workingDirectory: '$(Agent.BuildDirectory)/s'
displayName: 'Remove Git Authentication'
Obviously, I am treating the symptom and not the cause so any pointers as to what is causing this would be a great help.
You can create a "cleaning" job at the start of your pipeline that uses the checkout task with none as the repo to checkout. This will allow you to then run the cleaning script.
- job: cleanAgent
steps:
- checkout: none
- script: |
echo "Put your git cleaner here"
A better option is really to clean the agent after you are done by running a cleaning task with a condition of always() at the end of your pipeline. This can be hard if other people also use the agents and don't clean-up though.
- job: cleanupAgents
condition: always()
steps:
- script: |
echo "Put your git cleaner here"

GitLab CI denies access to push using a deploy key with write access

I added a deploy key with write access to my GitLab repository. My .gitlab-ci.yml file contains:
- git clone git#gitlab.domain:user/repo.git
- git checkout master
- git add myfile.pdf
- git commit -m "Generated PDF file"
- git push origin master
The deploy key works when cloning the repository.
Pushing is not possible, even if the deploy key has write access.
remote: You are not allowed to upload code.
fatal: unable to access 'https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx#domain/user/repo.git/': The requested URL returned error: 403
I just encountered the same problem and saw this question without answer, so there is my solution.
Problem
The problem is caused by the fact that the remote url used by git to push the code is in the form http(s)://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx#git.mydomain.com/group/project.git.
This url is using http(s) protocol so git doesn't use the ssh deploy key that you setup.
Solution
The solution is to change the push url of the remote origin so it matches ssh://git#git.mydomain.com/group/project.git.
The easiest way to do so is to use the predefined variable CI_REPOSITORY_URL.
Here is an example of code doing so by using sed:
# Change url from http(s) to ssh
url_host=$(echo "${CI_REPOSITORY_URL}" | sed -e 's|https\?://gitlab-ci-token:.*#|ssh://git#|g')
echo "${url_host}"
# ssh://git#git.mydomain.com/group/project.git
# Set the origin push url to the new one
git remote set-url --push origin "${url_host}"
Also, those using docker executor may want to verify the SSH host key as suggested by the gitlab documentation on deploy keys for docker executor.
So I give a more complete example for docker executor.
The code is mainly from gitlab documentation on ssh deploy keys.
In this example, the private deploy key is stored inside a variable named SSH_PRIVATE_KEY.
create:push:pdf:
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "${SSH_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- git config --global user.email "email#example.com"
- git config --global user.name "User name"
- gitlab_hostname=$(echo "${CI_REPOSITORY_URL}" | sed -e 's|https\?://gitlab-ci-token:.*#||g' | sed -e 's|/.*||g')
- ssh-keyscan "${gitlab_hostname}" >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
script:
- git checkout master
- git add myfile.pdf
- git commit -m "Generated PDF file"
- url_host=$(echo "${CI_REPOSITORY_URL}" | sed -e 's|https\?://gitlab-ci-token:.*#|ssh://git#|g')
- git remote set-url --push origin "${url_host}"
- git push origin master

Capistrano "Permission denied (publickey)." error message

I know that this problem has been asked many times, but I can't get it sorted (I'm a beginner).
What I'm trying to do is to deploy my rails application to my production server using capistrano. I stored my project on a directory on gitlab. Everything was working perfectly until I moved my application in an other gitlab repository (git#gitlab.com:myusername/xxxxxx.git).
I think I set up my deploy.rb file accordingly :
set :application, "xxxxxx"
set :user, "yyyyy"
set :repository, "git#gitlab.com:myusername/xxxxxx.git"
But when I try to deploy it, I get the permission error :
[xxxxxx.com] executing command
[xxxxxx.com] env PATH=/home/kar/.rbenv/shims:/home/kar/.rbenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin sh -c 'if [ -d /var/www/xxxxxx/shared/cached-copy ]; then cd /var/www/xxxxxx/shared/cached-copy && git fetch -q origin && git fetch --tags -q origin && git reset -q --hard 97ff4f45240a680c1d278325d7ac1871536c8091 && git clean -q -d -x -f; else git clone -q git#gitlab.com:myusername/xxxxxx.git /var/www/xxxxxx/shared/cached-copy && cd /var/www/xxxxxx/shared/cached-copy && git checkout -q -b deploy 97ff4f45240a680c1d278325d7ac1871536c8091; fi'
** [xxxxxx.com :: err] Permission denied (publickey).
** [xxxxxx.com :: err] fatal: The remote end hung up unexpectedly
Could you please propose me some tests to find out from where the issue comes ?
Is there any key to add on my server ?
Thanks a lot for your help.
Here's the capistrano 3 plugin that is created solely for the purpose of troubleshooting problems like this one: capistrano-ssh-doctor.
The plugin outputs a report with found issues and suggested next steps.

How to commit a Git repo to an empty repo SVN server?

I have setup an empty svn on a server and I have been working on locally making commits along the way. Now I wish to commit my repo to an svn server. For this I tried:
git-svn checkout http://remote.svn.server.com
git-svn dcommit
Git complains that:
Use of uninitialized value in concatenation (.) or string at /usr/bin/git-svn line 411.
Committing to ...
Unable to determine upstream SVN information from HEAD history
Since I started on my local computer first, and the repo online is empty, I can't find any info on how to make this work.
I needed something like this recently and the process is relatively straightforward.
There's good tutorial by Brandon Dimcheff, "Commit a linear git history to subversion" (replaces old broken link), which these steps are based on.
As of Git version 1.6.3 these are the steps:
$ svnadmin create svn_repository
$ svn mkdir -m "Initial setup" file:///full/path/to/svn_repository/trunk
$ mkdir gitrepo && cd gitrepo
$ git init
$ echo 'Hello from Git' > file.txt
$ git add file.txt
$ git commit -m "Hello from Git"
$ git svn init --trunk=trunk file:///full/path/to/svn_repository/
$ git svn fetch
$ git branch -a # Lists remotes/trunk
$ git rebase --onto remotes/trunk --root master
# => Applying: Hello from Git etc.
$ git svn dcommit
# => Committing to ... Committed r2 ... etc
You can do a svn checkout of svn_repository now and see your Git repo.
Here is what I would do:
git-svn clone http://remote.svn.server.com otherdir
Then in other dir pull the changes locally from your previous dir. Then you should have a git repo that is "connected" via git-svn and you should be able to use dcommit on it.
This might also be a useful read.