How does git-svn handle line endings? - git-svn

I'm pretty happy with how Git itself handles line endings, via core.autocrlf, core.eol + gitattributes (Tim's post is excellent).
I have a Windows Git repo that has autocrlf set to true. So, all text files are stored in the repo as LF and live in the working directory as CRLF. This repo was cloned from an SVN repo, which we still use to push from / pull to (the SVN repo is our central, blessed repo for triggering CI etc).
But I don't know how git-svn handles line endings during the push / pull operations.
Can anyone explain what git-svn does in this instance?

I'm interested in this too. Supposing you have a repo that was created via git svn clone, I think you could break it down into three different questions:
Does any git newline normalization/alteration happen at git svn fetch time, in moving commits from svn to the git repo?
Does any git newline normalization/alteration happen at git commit time [i.e. during a normal local git commit to a repo with svn remotes]? How about merge/rebase time?
Does any git newline normalization/alteration happen at git svn dcommit time, in pushing/replaying/whatever git commits against svn?
I'd love to hear what's theoretically supposed to be true for these questions, but for now I did a little experiment that seems to show that there is no newline normalization in case #1 at least:
rem We'll make a svn repo with CRLF newlines, clone it into git with
rem autocrlf enabled, and try to see if that results in LF-only newlines
rem getting stored in the git repo
cd c:\code
rem Step 1. Prepare SVN repo with CRLF type newlines.
rem The pre-1.4 flag is to prevent an error during git clone.
svnadmin create --pre-1.4-compatible svnrepo
svn checkout file:///C:/code/svnrepo svnworking
cd svnworking
echo "First line" > file.txt
echo "Second line" >> file.txt
echo "Third line" >> file.txt
rem NOTE: At this point file.txt has CRLF newlines
svn add file.txt
svn commit -m "Add file.txt"
rem NOTE: At this point file.txt still has CRLF newlines
cd ..
rem Step 2. Clone the svn repo into git and inspect work copy newline type
git svn clone file:///C:/code/svnrepo gitrepo
rem The following outputs true on my machine
git config --get core.autocrlf
cd gitrepo
rem The following also outputs true on my machine
git config --get core.autocrlf
git svn fetch
rem NOTE: At this point file.txt (git working dir copy) has CRLF newlines
rem Step 3. Disable autocrlf to inspect repo's inner newline type
rem Use the following and my editor to set core.autocrlf to false:
git config --edit --local
rem This now prints false:
git config --get core.autocrlf
git checkout .
rem NOTE: At this point file.txt (git working dir copy) still has CRLF newlines
del file.txt
git checkout .
rem NOTE: Even after explicitly deleting the old one and checking out again,
rem file.txt still has CRLF newlines
If git newline conversion had taken place during my git svn pull, in contrast, then I would expect file.txt to have LF-only newlines at the end of all this.
Here's a sanity check that step 3 above actually implements a valid test of whether the repo has LF-only newlines:
rem We'll a git repo with core.autocrlf on, then switch it off to
rem pull out a file
rem The following outputs true
git config --get core.autocrlf
git init gitcrtest
cd gitcrtest
rem The following still outputs true
git config --get core.autocrlf
echo "First line" > file.txt
echo "Second line" >> file.txt
echo "Third line" >> file.txt
git add file.txt
git commit -m "Add file.txt"
rem NOTE: At this point file.txt (git working dir copy) has CRLF newlines
rem Use the following to set core.autocrlf to false
git config --edit --local
git checkout .
rem NOTE: Now file.txt (git working dir copy) has LF-only newlines
In summary: Based on the above, it seems that when git-svn pulls from svn, the svn commits get added to the git commit graph without any crlf translation, even when autocrlf is enabled. That is, whatever type of newline your files have in your svn repo, they're also going to have in your git clone thereof. (But your git working copy may have different newline types.)
Note that this is pretty consistent with the discussion of end-of-line normalization in "git help attributes"; there normalization is presented as something that happens either with commands that pull stuff from the repo into your working directory (e.g. checkout or merge) or with commands that move things from your working directory into the index/repo (e.g. add or commit). "Git svn fetch" doesn't seem to do either of those things, so it makes sense that no end-of-line normalization would happen at that time. I'm fuzzier about what dcommit does, so I'm not sure whether to expect end-of-line normalization at that time.
Note there's an additional wrinkle if SVN's svn:eol-style property is set on your repo/machine. I think the SVN default is to not do end-of-line conversions on its end, but I'm not 100% sure.
Update: For a real-world svn->git migration perspective on newlines, see also Tim Abell's description thereof. CRLF newlines were not converted to LF-only newlines by git-svn, with non-ideal results if git's automatic end-of-line normalization was left on. Solutions were to normalize the line endings in git or to disable end-of-line-normalization.

Related

Get SVN URL of removed git-svn file

I would like to track a removed file as far back in history as possible, while using git-svn on a subdirectory of the SVN repository.
Using git log --full-history -- path/to/removed_file.py, I can get see the history starting with the time the file was moved into the subdirectory I checked out using git-svn.
I can see which SVN revision that was in the git-svn commit message postfix, so I would now like to use svn log <full_url>#revision to see the rest of the history.
I know that I could use git svn info --url path/to/existing_file.py to see the required full SVN url, but what is a quick (ideally scriptable) way of getting the SVN URL of a file that is no longer in the repository?
To git, it doesn't matter much that a file foo/bar.py is removed in HEAD — as long as you have it in history, you can view every past version of it.
For clarity of concreteness, I'll take this git-svn repo from the LLVM project as an example. There, the file docs/todo.rst has been deleted in svn revision 308987, git commit fb572868… and is absent in master.
Let's first init a local clone.
$ git clone https://github.com/llvm-mirror/lnt && cd lnt
Cloning into 'lnt'...
...
$ git svn init https://llvm.org/svn/llvm-project/lnt/trunk
$ git update-ref refs/remotes/git-svn refs/remotes/origin/master
$
$ #-- ask svn info of anything to check setup and/or force laziness
$ git svn info --url README.md
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map.91177308-0d34-0410-b5e6-96231b3b80d8 ...
r154126 = 3c3062527ac17b5fac440c55a3e1510d0ab8c9d9
r154135 = 82a95d29ac7d25c355fbd0898a44dc3e71a75fd8
...
r374687 = 446f9a3b651086e87684d643705273ef78045279
r374824 = 8c57bba3687ada10de5653ae46c537e957525bdb
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.91177308-0d34-0410-b5e6-96231b3b80d8
https://llvm.org/svn/llvm-project/lnt/trunk/README.md
So it gives back the README.md URL as expected. Now let's try the case of a deleted file:
$ git svn info --url docs/todo.rst
svn: 'docs/todo.rst' is not under version control
Fails, just like you say. man git-svn says that info Does not currently support a -r/--revision argument.
OK then, let's try emulating what it does, first by hand.
https://llvm.org/svn/llvm-project/lnt/trunk/README.md?r=374824 — this is the URL for given file at given revision.
Our vanished docs/todo.rst is available at https://llvm.org/svn/llvm-project/lnt/trunk/docs/todo.rst?p=308986 Notice the decrement: per git show fb572868 | grep git-svn-id, docs/todo.rst is already deleted in r308987 — so we request r308986.
On to scripting it... rather simple job.
git-svn-oldinfo () {
relfname="$1"
git log -n1 -- "$relfname" \
| awk '/git-svn-id:/ {sub(/#/, " ", $2); print $2}' \
| { read baseurl rev; echo "${baseurl}/${relfname}?p=$((rev-1))"; }
}
#-- test:
$ git-svn-oldinfo docs/todo.rst
https://llvm.org/svn/llvm-project/lnt/trunk/docs/todo.rst?p=308986
Quick-n-dirty but tested — you're welcome to adjust & extend as needed.
Edit
Despite git log being a "porcelain" command (i.e. not really designed for scripting), it's quite possible to parse out the filenames from it too, if you're to query by globs like **/removed_file.py:
git-svn-oldinfo-glob () {
fileglob="$1"
git log -n1 --stat --format=oneline -- "$fileglob" \
| { read commit msg; \
read fullname _remainder_dummy; \
git cat-file -p $commit \
| tail -n1 \
| awk '/git-svn-id:/ {sub(/#/, " ", $2); print $2}' \
| { read baseurl rev; echo "${baseurl}/${fullname}?p=$((rev-1))"; } \
}
}
#-- test:
$ git-svn-oldinfo-glob '**/todo.rst'
https://llvm.org/svn/llvm-project/lnt/trunk/docs/todo.rst?p=308986
Take it with a grain of salt: it'll probably break in hilarious ways or output garbage if the glob matches multiple files, non-removed files, files with whitespace in the name, etc.
As always, check out man git-log and customize as needed.

trying to push file into bitbucket repo, but showing wrong repo

Normally I open a bash prompt inside my Test folder. I then git add, commit, and push origin the file and it goes into my Test folder in bitbucket. Now somehow my Test folder instead of showing .../Test (Development), it shows another repo, .../Test (Review). I do not know why it changed. How can I get (Review) to be (Development)?
In git there are pretty much three stages. When pressing git status you probably get a similar few to this with many more files:
# On branch review
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# modified: file.txt
#
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: file2.txt
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# file3.txt
file.txt on top has staged changes. These will go into the next commit when you do git commit.
file2.txt has unstaged changes. This file is tracked in the repository but the changes will not be added to the next commit. Only if you git add this file will it get staged.
file3.txt is an untracked file. You have to add it with git add which will automatically put it into the staged area. Next time you will make changes to it you will find it in the unstaged area like file2.txt
from this situation git checkout master gives:
error: Your local changes to the following files would be overwritten by checkout:
file2.txt
Please, commit your changes or stash them before you can switch branches.
Aborting
This is probably what you get too. Git noticed that you made changes in the tracked file file2.txt but you didn't specify what to do with them. Similarly I suspect that you made changed to those '50 or so files' and now git doesn't know what to do.
Either add them to your commit and do a commit:
git add <files>
git commit -m "did some work"
or drop the changes:
git checkout <files>
Then they will return to the way they were at the last commit.
You can also add some files and drop others, or even do partial adds with git add -p.
Check the changes you made with git diff.
After this is resolved you can switch branches again with git checkout <branchname>.
Without more information on your branch structure in your bitbucket and your commit history it is hard to say what you can push to where.

how to revert changes only for one file with git-svn?

I've corrupted one file and I'd like to revert it back. My project is using git-svn.
So how can I revert this one particular file? Or even better if I could view whole change set of this file.
Detailed steps would be appreciated.
git revert SHA1_OF_FAULTY_COMMIT
add back the changes but don't commit
git cherry-pick -n SHA1_OF_FAULTY_COMMIT
Modify what needs to be modified, e.g.
git reset HEAD file_that_should_not_have_been_modified
Commit
git commit -m "to_be_merged"
Squash the two commits, put a meaningful comment.
git rebase -i HEAD~2
Review your changes, it should only contains the modification on the single file you:
git show
You can now push that to svn
git svn dcommit

Enabling SVN tracking in a git clone of a git-svn clone that didn't start at revision 1?

Suppose I have a git svn clone that I created like this:
$ mkdir foo
$ cd foo
$ git svn clone -s -r 100:HEAD http://svn.example.com/project
and then I git clone it like this:
$ cd ..
$ git clone foo bar
$ cd bar
So now bar has foo as its origin.
How do I make bar track the original SVN server? The git-svn man page provides an example of the case where the original git svn clone includes the full revision history, but this doesn't seem to cover the case that the original git svn clone doesn't start from r1. When I try the various things I see described, what git svn fetch does is always start a disconnected history starting at r1.

How to commit a Git repo to an empty repo SVN server?

I have setup an empty svn on a server and I have been working on locally making commits along the way. Now I wish to commit my repo to an svn server. For this I tried:
git-svn checkout http://remote.svn.server.com
git-svn dcommit
Git complains that:
Use of uninitialized value in concatenation (.) or string at /usr/bin/git-svn line 411.
Committing to ...
Unable to determine upstream SVN information from HEAD history
Since I started on my local computer first, and the repo online is empty, I can't find any info on how to make this work.
I needed something like this recently and the process is relatively straightforward.
There's good tutorial by Brandon Dimcheff, "Commit a linear git history to subversion" (replaces old broken link), which these steps are based on.
As of Git version 1.6.3 these are the steps:
$ svnadmin create svn_repository
$ svn mkdir -m "Initial setup" file:///full/path/to/svn_repository/trunk
$ mkdir gitrepo && cd gitrepo
$ git init
$ echo 'Hello from Git' > file.txt
$ git add file.txt
$ git commit -m "Hello from Git"
$ git svn init --trunk=trunk file:///full/path/to/svn_repository/
$ git svn fetch
$ git branch -a # Lists remotes/trunk
$ git rebase --onto remotes/trunk --root master
# => Applying: Hello from Git etc.
$ git svn dcommit
# => Committing to ... Committed r2 ... etc
You can do a svn checkout of svn_repository now and see your Git repo.
Here is what I would do:
git-svn clone http://remote.svn.server.com otherdir
Then in other dir pull the changes locally from your previous dir. Then you should have a git repo that is "connected" via git-svn and you should be able to use dcommit on it.
This might also be a useful read.