`git svn dcommit` failing on a branch - branch

I have been using git-svn to communicate with my company’s svn repo for a while now without any major headaches.
Today, the β€œheadache”-part changed dramatically:
I’ve been working on master/trunk pretty exclusively, and needed to merge most (but not all!) of those change-sets into a new svn-branch, that originated from a pre-existing svn-branch.
Basically this:
πŸ’---πŸ’©---πŸ’©---πŸ’©--1πŸ’--1πŸ’---πŸ’©--1πŸ’---πŸ’©---πŸ’©--1πŸ’--1πŸ’--1πŸ’---πŸ’© master/trunk
\
\
2πŸ’--2πŸ’--2πŸ’--2πŸ’--2πŸ’ versioned-release
Should have become this:
πŸ’---πŸ’©---πŸ’©---πŸ’©--1πŸ’--1πŸ’---πŸ’©--1πŸ’---πŸ’©---πŸ’©--1πŸ’--1πŸ’--1πŸ’---πŸ’© master/trunk
\
\
2πŸ’--2πŸ’--2πŸ’--2πŸ’--2πŸ’ versioned-release
\
\
1πŸ’--1πŸ’--1πŸ’--1πŸ’--1πŸ’--1πŸ’ new-versioned-release
Where πŸ’© are commits that shouldn’t be in the new-versioned-release, and xπŸ’ the wanted commits from the respective branches x.
So I did the following:
git checkout -b versioned-release-svn remotes/versioned-release
git svn branch new-versioned-release -m "Preparing for merge of XXX"
git checkout -b new-versioned-release-svn remotes/new-versioned-release
git cherry-pick ... for every 1πŸ’, resolving any conflicts on the way.
Because I wanted to be sure I was really going to target the correct branch on the repo, I then ran git svn dcommit --dry-run which did not yield any errors or warnings, but told me…
Committing to svn://username#$repo-host/$repo-name/$path/branches/new-versioned-release ...
…followed by a couple of diff-tree lines.
So I attempted to omit the --dry-run and half way through the commits ended up with…
Item already exists in filesystem: File already exists: filesystem '/data/subvroot/$repo-name/db', transaction '20856-g3m', path '/$path/branches/new-versioned-release/some-directory' at /usr/libexec/git-core/git-svn line 862
…and a bunch of unstaged changes.
Apart from the obvious β€” β€œWTF?!?” and β€œHow do I get out of this mess without losing everything I did?” β€” I have two questions:
Assuming I was back to before git svn dcommit: How do I get my local branch dcommit to its planned destination?
By now it seems obvious, that this wasn’t the right way to achieve what I wanted…but how should I have done it, instead?
Everything I found for the error-message, that somehow resembled my situation, so far was this other stack overflow question and the proposed solution of β€œsomehow […] to blow away the .git/svn metadata directory” doesn’t resonate quite that well with me…

Someone just up–voted my old question, so I thought I’d share how I do that nowadays.
It works really quite well.
Assuming the git repository has been created using
git svn clone \
--prefix svn/ \
--stdlayout \
svn://username#$repo-host/$repo-name/$path
$git_repo_name
change into the git repo, and there run
git checkout svn/versioned-release
git svn branch new-versioned-release
This will result in the following history on the SVN server:
πŸ’---πŸ’©---πŸ’©---πŸ’©--1πŸ’--1πŸ’---πŸ’©--1πŸ’---πŸ’©---πŸ’©--1πŸ’--1πŸ’--1πŸ’---πŸ’© trunk
\
\
2πŸ’--2πŸ’--2πŸ’--2πŸ’--2πŸ’ versioned-release
\
\
3⭐️ new-versioned-release
Now I’d run
git checkout svn/new-versioned-release
git checkout -b new-versioned-release
# resulting in the following **local** history:
#
# πŸ’---πŸ’©---πŸ’©---πŸ’©--1πŸ’--1πŸ’---πŸ’©--1πŸ’---πŸ’©---πŸ’©--1πŸ’--1πŸ’--1πŸ’---πŸ’© master (tracks 'svn/trunk')
# \
# \
# 2πŸ’--2πŸ’--2πŸ’--2πŸ’--2πŸ’--3⭐️ new-versioned-release (tracks 'svn/new-versioned-release')
This is the foundation for achieving what I wanted.
There is one additional commit, because branching in SVN doesn’t work the same way as in Git: creating a branch always means a new revision, (aka commit) and that’s where the 3⭐️ comes from. It doesn’t really matter, but it’s there.
I can now git cherry-pick all the 1πŸ’s, ending up with this local history:
πŸ’---πŸ’©---πŸ’©---πŸ’©--1πŸ’--1πŸ’---πŸ’©--1πŸ’---πŸ’©---πŸ’©--1πŸ’--1πŸ’--1πŸ’---πŸ’© master (tracks 'svn/trunk')
\
\
2πŸ’--2πŸ’--2πŸ’--2πŸ’--2πŸ’--3⭐️--1πŸ’--1πŸ’--1πŸ’--1πŸ’--1πŸ’--1πŸ’ new-versioned-release (tracks 'svn/new-versioned-release')
When I now git svn dcommit while sitting on new-versioned-release in git, the history on the SVN server looks like what I wanted to end up with:
πŸ’---πŸ’©---πŸ’©---πŸ’©--1πŸ’--1πŸ’---πŸ’©--1πŸ’---πŸ’©---πŸ’©--1πŸ’--1πŸ’--1πŸ’---πŸ’© trunk
\
\
2πŸ’--2πŸ’--2πŸ’--2πŸ’--2πŸ’ versioned-release
\
\
3⭐️--1πŸ’--1πŸ’--1πŸ’--1πŸ’--1πŸ’--1πŸ’ new-versioned-release
The only difference being that additional 3⭐️ from creating the third SVN branch.

Related

Get SVN URL of removed git-svn file

I would like to track a removed file as far back in history as possible, while using git-svn on a subdirectory of the SVN repository.
Using git log --full-history -- path/to/removed_file.py, I can get see the history starting with the time the file was moved into the subdirectory I checked out using git-svn.
I can see which SVN revision that was in the git-svn commit message postfix, so I would now like to use svn log <full_url>#revision to see the rest of the history.
I know that I could use git svn info --url path/to/existing_file.py to see the required full SVN url, but what is a quick (ideally scriptable) way of getting the SVN URL of a file that is no longer in the repository?
To git, it doesn't matter much that a file foo/bar.py is removed in HEAD β€” as long as you have it in history, you can view every past version of it.
For clarity of concreteness, I'll take this git-svn repo from the LLVM project as an example. There, the file docs/todo.rst has been deleted in svn revision 308987, git commit fb572868… and is absent in master.
Let's first init a local clone.
$ git clone https://github.com/llvm-mirror/lnt && cd lnt
Cloning into 'lnt'...
...
$ git svn init https://llvm.org/svn/llvm-project/lnt/trunk
$ git update-ref refs/remotes/git-svn refs/remotes/origin/master
$
$ #-- ask svn info of anything to check setup and/or force laziness
$ git svn info --url README.md
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map.91177308-0d34-0410-b5e6-96231b3b80d8 ...
r154126 = 3c3062527ac17b5fac440c55a3e1510d0ab8c9d9
r154135 = 82a95d29ac7d25c355fbd0898a44dc3e71a75fd8
...
r374687 = 446f9a3b651086e87684d643705273ef78045279
r374824 = 8c57bba3687ada10de5653ae46c537e957525bdb
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.91177308-0d34-0410-b5e6-96231b3b80d8
https://llvm.org/svn/llvm-project/lnt/trunk/README.md
So it gives back the README.md URL as expected. Now let's try the case of a deleted file:
$ git svn info --url docs/todo.rst
svn: 'docs/todo.rst' is not under version control
Fails, just like you say. man git-svn says that info Does not currently support a -r/--revision argument.
OK then, let's try emulating what it does, first by hand.
https://llvm.org/svn/llvm-project/lnt/trunk/README.md?r=374824 β€” this is the URL for given file at given revision.
Our vanished docs/todo.rst is available at https://llvm.org/svn/llvm-project/lnt/trunk/docs/todo.rst?p=308986 Notice the decrement: per git show fb572868 | grep git-svn-id, docs/todo.rst is already deleted in r308987 β€” so we request r308986.
On to scripting it... rather simple job.
git-svn-oldinfo () {
relfname="$1"
git log -n1 -- "$relfname" \
| awk '/git-svn-id:/ {sub(/#/, " ", $2); print $2}' \
| { read baseurl rev; echo "${baseurl}/${relfname}?p=$((rev-1))"; }
}
#-- test:
$ git-svn-oldinfo docs/todo.rst
https://llvm.org/svn/llvm-project/lnt/trunk/docs/todo.rst?p=308986
Quick-n-dirty but tested β€” you're welcome to adjust & extend as needed.
Edit
Despite git log being a "porcelain" command (i.e. not really designed for scripting), it's quite possible to parse out the filenames from it too, if you're to query by globs like **/removed_file.py:
git-svn-oldinfo-glob () {
fileglob="$1"
git log -n1 --stat --format=oneline -- "$fileglob" \
| { read commit msg; \
read fullname _remainder_dummy; \
git cat-file -p $commit \
| tail -n1 \
| awk '/git-svn-id:/ {sub(/#/, " ", $2); print $2}' \
| { read baseurl rev; echo "${baseurl}/${fullname}?p=$((rev-1))"; } \
}
}
#-- test:
$ git-svn-oldinfo-glob '**/todo.rst'
https://llvm.org/svn/llvm-project/lnt/trunk/docs/todo.rst?p=308986
Take it with a grain of salt: it'll probably break in hilarious ways or output garbage if the glob matches multiple files, non-removed files, files with whitespace in the name, etc.
As always, check out man git-log and customize as needed.

How do I clone a git repo from a local svn repo

I want to learn to use git-svn. I have an svn local repository on my disk that I've checked out a while ago using something like this:
svn co http://myserver.com/mysvnrepo/trunk/ /mysvnrepo/
ls -a /mysvnrepo/
. .. .svn foo bar
This /mysvnrepo/ is HUGE, so I want to avoid re-downloading or copying the files at all costs.
I'm wondering if there's a way to git clone this local repo without downloading / copying anything (because it's already there).
I have this which seems to be what I'm looking for, but when I do that it doesn't quite give me what I expect.
cd /mysvnrepo/
git svn clone file://mysvnrepo/
ls /mysvnrepo/
. .. .git .svn foo bar
git status
# On branch master
#
# Initial commit
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# .svn/
# foo/
# bar/
I would expect git to detect foo and bar as "versioned and up-to-date".
According to the docs it seems that I need to use git svn init because git svn clone runs a fetch, which I certainly don't want. So I tried
git svn init --trunk=file:///mysvnrepo/
...but no luck.
I'm completely new to git, so my confusion is off-the-charts... am I doing something utterly wrong?
Thanks in advance
You cannot take a subversion snapshot and convert it into a git repository.
It sounds like you are trying to avoid a lengthy initialization of the git repository from svn: which ordinarily will try to ready your entire history. This can be done in another way, by limiting the fetch to recent history depending on how much history is relevant to you:
git svn clone -s -r 12334:HEAD https://svn.host.org/repo
Where 12334 is the earliest svn revision you are interested in and assuming that the repo is laid out in a standard svn way with branches and tags.

git-svn won't rebase or dcommit

I've no idea exactly how this came to pass but my git-svn is now fubar.
ERROR from SVN:
Transaction is out of date: File '/proj/some/file.java' is out of date
No changes between 61f2c8cfa5881aab335edc2c0807f288c420f624 and refs/remotes/svn/git-svn
Resetting to the latest refs/remotes/svn/git-svn
ERROR: Not all changes have been committed into SVN, however the committed
ones (if any) seem to be successfully integrated into the working tree.
Please see the above messages for details.
There was a similar question on SO before, but neither solution worked for me - there are no merge conflicts, and moving aside .git/svn and running git svn fetch did not change the error from git svn dcommit. Running fetch did have one interesting statement:
M proj/some/file.java
... [merge list] ...
Last fetched revision of refs/remotes/svn/git-svn was r8104, but we are about to fetch: r8102!
Not sure what that means.
Edit:
When I looked at the tree in SourceTree, I could see that the SVN was "ahead" of the master branch and that commits after r8102 were "duplicated" in both trees. Reverting the master to before r8102 and attempting to move forward again did not resolve the problem.
Edit 2:
The tree looks something like this:
+-------D---E---F---G remote/svn/git-svn
/
A---B---C---D---E master
Before I attempt the suggested git rebase remotes/svn/git-svn master by #Ben what should I expect? There are a lot of commits (B & C here) that aren't apparing in the remote, but are in the master, I noticed that the duplicates seem to be somehow related to my switching between different tunneling methods. I need to protect the svn here, this is more for my personal interest (and future avoidance of problems).
From my .subversion/config:
[tunnels]
wlan = /usr/bin/ssh -F /dev/null -l petri -t -A wifi-gateway /usr/bin/ssh -F /dev/null -l svnremote -A
wan = /usr/bin/ssh -F /dev/null -l petri -t -A extern-gateway /usr/bin/ssh -F /dev/null -l svnremote -A
This results in different svn commit comments like
git-svn-id: svn+wan://dev/svn/v9/trunk#8102 b9636cbc-3124-4d49-8704-a9669b6dc9bf
I'm wondering if there isn't some bad reference id problems here...

Why does git-svn dcommit leave duplicate commits in my git repo? Can I stop it doing that?

My typical git-svn workflow is:
git checkout -b story-xyz
git commit -a -m "work"
git commit -a -m "more work"
git checkout master
git svn fetch
git merge remotes/trunk
git checkout story-xyz
git rebase master (sometimes with -i)
git checkout master
git merge story-xyz
At this point I have my master and story-xyz branches pointing to the same commit, one or more commits ahead of remotes/trunk. Everything since remotes/trunk is in one linear history.
last svn commit [remotes/trunk] <--- work <--- more work [master, story-xyz]
I then run
git svn dcommit
I expected to see the commits between remotes/trunk and master become Subversion revisions, and end up with a single linear history with remotes/trunk, master and story-xyz all pointing to the latest revision, like so:
last svn commit <--- work <--- more work [master, story-xyz, remotes/trunk]
My Subversion revisions go in fine, but I end up with a two-branched structure. The common root of the branch is the Subversion HEAD before I committed. Both branches contain the same series of commits, in the sense that they contain the same diffs. The branch story-xyz is at the head of one branch, remotes/trunk and master at the other:
last svn commit <--- work <--- more work [master, remotes/trunk]
|
\- work <--- more work [story-xyz]
The git commits that I had before running git svn dcommit are on the lower branch (story-xyz), with my git commit messages, git user name and email, and git commit timestamps. The commits on the upper branch are new git commits. They use my Subversion username, the timestamp when I ran the dcommit, and the commit messages have the git-svn-id field appended to them.
This is all OK, and I can carry on working. The problem is that I look in gitk and see what looks like an unmerged branch story-xyz. It's pretty hard to tell the difference between a story branch that I have merged back into master, and one that I haven't. The most obvious way to spot it is the duplicate commit messages. I could delete the story-xyz branch, but that feels like I'm not using git properly and I've lost some of my history.
Am I missing something that would stop git-svn from doing this? Or is this just one of the ways that interacting with Subversion dilutes the power and freedom of git?
I don't think you're really missing anything. You might be doing some unnecessary work, though. In this case, you have two pointers to the "more work" commit, and you are asking git-svn to move one of them. The other one still stays where it is.
You don't really need the master branch. Git-svn doesn't care about what branch you are dcommiting. IIRC, it uses the first svn-remote it can find among the ancestors of the current commit.
I'll offer another version of the workflow:
git checkout -b story-xyz remotes/trunk
git commit -a -m "work"
git commit -a -m "more work"
git svn fetch
git rebase remotes/trunk (with -i, perhaps)
git svn dcommit
This should give you a tree without the extra branch. You need to be careful with fast-forward merges, though.

How to commit a Git repo to an empty repo SVN server?

I have setup an empty svn on a server and I have been working on locally making commits along the way. Now I wish to commit my repo to an svn server. For this I tried:
git-svn checkout http://remote.svn.server.com
git-svn dcommit
Git complains that:
Use of uninitialized value in concatenation (.) or string at /usr/bin/git-svn line 411.
Committing to ...
Unable to determine upstream SVN information from HEAD history
Since I started on my local computer first, and the repo online is empty, I can't find any info on how to make this work.
I needed something like this recently and the process is relatively straightforward.
There's good tutorial by Brandon Dimcheff, "Commit a linear git history to subversion" (replaces old broken link), which these steps are based on.
As of Git version 1.6.3 these are the steps:
$ svnadmin create svn_repository
$ svn mkdir -m "Initial setup" file:///full/path/to/svn_repository/trunk
$ mkdir gitrepo && cd gitrepo
$ git init
$ echo 'Hello from Git' > file.txt
$ git add file.txt
$ git commit -m "Hello from Git"
$ git svn init --trunk=trunk file:///full/path/to/svn_repository/
$ git svn fetch
$ git branch -a # Lists remotes/trunk
$ git rebase --onto remotes/trunk --root master
# => Applying: Hello from Git etc.
$ git svn dcommit
# => Committing to ... Committed r2 ... etc
You can do a svn checkout of svn_repository now and see your Git repo.
Here is what I would do:
git-svn clone http://remote.svn.server.com otherdir
Then in other dir pull the changes locally from your previous dir. Then you should have a git repo that is "connected" via git-svn and you should be able to use dcommit on it.
This might also be a useful read.