KDE SVN2GIT "WARN: Branch ... in repository ... doesn't exist at revision ... -- did you resume from the wrong revision?" Can't continue - kde-plasma

I'm trying to migrate an 11GB SVN repo with over than 24k revisions inside to a single GIT repository.
I did a single file dump of the SVN using svnrdump command and load it into my local SVN server, placed on my MacBook machine.
I downloaded the svn2git from the https://github.com/svn-all-fast-export/svn2git repository.
Due to differences in the way how SVN and GIT handle tags, I used a merged-branches-tags.rules from the svn2git sample directory, which look like this (I've removed comments):
create repository myproject
end repository
match /trunk/
repository myproject
branch master
end match
match /(branches|tags)/([^/]+)/
repository myproject
branch \2
end match
Then I used a docker image solution as described in the documentation (in my console it was a single line. I've did split it to clarify what I was doing):
docker run --rm -it \
-v /Users/me/work/SVN/dest:/workdir \
-v /Users/me/work/svnServer/repositories/my_svn_repo:/tmp/svn \
-v /Users/me/work/SVN/svn2git/samples:/tmp/conf \
svn2git /usr/local/svn2git/svn-all-fast-export \
--rules /tmp/conf/merged-branches-tags.rules \
--add-metadata --svn-branches --debug-rules --svn-ignore --empty-dirs \
/tmp/svn/
During the first try I got an error between revisions 12600 and 126001:
Exporting revision 12601 /tags/7.0M0p0000 was copied from /tags rev 12600
rev 12601 /tags/7.0M0p0000/ matched rule: "/tmp/conf/merged-branches-tags.rules:28 /(branches|tags)/([^/]+++++
)/" exporting.
.WARN: SVN reports a "copy from" # 12601 from /tags # 12600 but no matching rules found! Ignoring copy, treating as a modification
WARN: Transaction: "7.0M0p0000" is not a known branch in repository "myproject"
Going to create it automatically
add/change dir ( /tags/7.0M0p0000 -> "7.0M0p0000" "" )
+++++
WARN: Branch "7.0M0p0000" in repository "myproject" doesn't exist at revision 12601 -- did you resume from the wrong revision?
Failed to write to process: Process crashed for repository myproject
6223345 modifications from SVN /tags/7.0M0p0000/ to myproject/7.0M0p0000%
I've check it and in the rev 12601 there there is a new tag named as "7.0M0p0000", which I'm going to import as a branch and which wasn't in the repo in rev 12600.
Do you have any ideas what can I do to fix that and continue my migration?
Any help will be really appreciated.

After a further investigation, it turns out that the mentioned "7.0M0p0000" tag was created in the rev 12601 as a copy of all the tags from rev 12600.
I've found it in the dump file, created using this command:
svnrdump dump -r 12600:12601 --incremental http://xxx.xxx.xxx.xxx/svn/my_repo > my_repo.dump
There was an entry:
Revision-number: 12601
...
Node-path: tags/7.0M0p0000
Node-kind: dir
Node-action: add
Node-copyfrom-rev: 12600
Node-copyfrom-path: tags
It seems that KDE's svn2git is unable to deal with such cases (which was probably done by mistake).
The only solution I found was completely skip this tag by adding a match to my merged-branches-tags.rules file (order of match expressions is important):
match /tags/7.0M0p0000/
min revision 12600
max revision 12606
end match
...
match /(branches|tags)/([^/]+)/
repository myproject
branch \2
end match

Related

Get SVN URL of removed git-svn file

I would like to track a removed file as far back in history as possible, while using git-svn on a subdirectory of the SVN repository.
Using git log --full-history -- path/to/removed_file.py, I can get see the history starting with the time the file was moved into the subdirectory I checked out using git-svn.
I can see which SVN revision that was in the git-svn commit message postfix, so I would now like to use svn log <full_url>#revision to see the rest of the history.
I know that I could use git svn info --url path/to/existing_file.py to see the required full SVN url, but what is a quick (ideally scriptable) way of getting the SVN URL of a file that is no longer in the repository?
To git, it doesn't matter much that a file foo/bar.py is removed in HEAD — as long as you have it in history, you can view every past version of it.
For clarity of concreteness, I'll take this git-svn repo from the LLVM project as an example. There, the file docs/todo.rst has been deleted in svn revision 308987, git commit fb572868… and is absent in master.
Let's first init a local clone.
$ git clone https://github.com/llvm-mirror/lnt && cd lnt
Cloning into 'lnt'...
...
$ git svn init https://llvm.org/svn/llvm-project/lnt/trunk
$ git update-ref refs/remotes/git-svn refs/remotes/origin/master
$
$ #-- ask svn info of anything to check setup and/or force laziness
$ git svn info --url README.md
Rebuilding .git/svn/refs/remotes/git-svn/.rev_map.91177308-0d34-0410-b5e6-96231b3b80d8 ...
r154126 = 3c3062527ac17b5fac440c55a3e1510d0ab8c9d9
r154135 = 82a95d29ac7d25c355fbd0898a44dc3e71a75fd8
...
r374687 = 446f9a3b651086e87684d643705273ef78045279
r374824 = 8c57bba3687ada10de5653ae46c537e957525bdb
Done rebuilding .git/svn/refs/remotes/git-svn/.rev_map.91177308-0d34-0410-b5e6-96231b3b80d8
https://llvm.org/svn/llvm-project/lnt/trunk/README.md
So it gives back the README.md URL as expected. Now let's try the case of a deleted file:
$ git svn info --url docs/todo.rst
svn: 'docs/todo.rst' is not under version control
Fails, just like you say. man git-svn says that info Does not currently support a -r/--revision argument.
OK then, let's try emulating what it does, first by hand.
https://llvm.org/svn/llvm-project/lnt/trunk/README.md?r=374824 — this is the URL for given file at given revision.
Our vanished docs/todo.rst is available at https://llvm.org/svn/llvm-project/lnt/trunk/docs/todo.rst?p=308986 Notice the decrement: per git show fb572868 | grep git-svn-id, docs/todo.rst is already deleted in r308987 — so we request r308986.
On to scripting it... rather simple job.
git-svn-oldinfo () {
relfname="$1"
git log -n1 -- "$relfname" \
| awk '/git-svn-id:/ {sub(/#/, " ", $2); print $2}' \
| { read baseurl rev; echo "${baseurl}/${relfname}?p=$((rev-1))"; }
}
#-- test:
$ git-svn-oldinfo docs/todo.rst
https://llvm.org/svn/llvm-project/lnt/trunk/docs/todo.rst?p=308986
Quick-n-dirty but tested — you're welcome to adjust & extend as needed.
Edit
Despite git log being a "porcelain" command (i.e. not really designed for scripting), it's quite possible to parse out the filenames from it too, if you're to query by globs like **/removed_file.py:
git-svn-oldinfo-glob () {
fileglob="$1"
git log -n1 --stat --format=oneline -- "$fileglob" \
| { read commit msg; \
read fullname _remainder_dummy; \
git cat-file -p $commit \
| tail -n1 \
| awk '/git-svn-id:/ {sub(/#/, " ", $2); print $2}' \
| { read baseurl rev; echo "${baseurl}/${fullname}?p=$((rev-1))"; } \
}
}
#-- test:
$ git-svn-oldinfo-glob '**/todo.rst'
https://llvm.org/svn/llvm-project/lnt/trunk/docs/todo.rst?p=308986
Take it with a grain of salt: it'll probably break in hilarious ways or output garbage if the glob matches multiple files, non-removed files, files with whitespace in the name, etc.
As always, check out man git-log and customize as needed.

Error when cloning a svn repository using git-svn

I am trying to migrate a SVN repo to a git. For this I am using git-svn tool. I'm running the command:
git svn clone [SVN repo URL] --no-metadata -A authors-transform.txt --stdlayout ~/temp
and I'm getting the following error:
Name does not refer to a filesystem directory: Can’t get entries of non-directory at /Applications/Xcode.app/Contents/Developer/usr/share/git-core/perl/Git/SVN/Ra.pm line 312.”
Line 312 of that script is:
$reporter->finish_report($pool);
So I understand that finish_report is failing, but I don't understand why.
That particular section in perl/Git/SVN/Ra.pm is seven years old and part of the initial split of git-svn.perl
Try instead subgit (in its free version, for a one-shot import): it should be more robust.

How do I clone a git repo from a local svn repo

I want to learn to use git-svn. I have an svn local repository on my disk that I've checked out a while ago using something like this:
svn co http://myserver.com/mysvnrepo/trunk/ /mysvnrepo/
ls -a /mysvnrepo/
. .. .svn foo bar
This /mysvnrepo/ is HUGE, so I want to avoid re-downloading or copying the files at all costs.
I'm wondering if there's a way to git clone this local repo without downloading / copying anything (because it's already there).
I have this which seems to be what I'm looking for, but when I do that it doesn't quite give me what I expect.
cd /mysvnrepo/
git svn clone file://mysvnrepo/
ls /mysvnrepo/
. .. .git .svn foo bar
git status
# On branch master
#
# Initial commit
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# .svn/
# foo/
# bar/
I would expect git to detect foo and bar as "versioned and up-to-date".
According to the docs it seems that I need to use git svn init because git svn clone runs a fetch, which I certainly don't want. So I tried
git svn init --trunk=file:///mysvnrepo/
...but no luck.
I'm completely new to git, so my confusion is off-the-charts... am I doing something utterly wrong?
Thanks in advance
You cannot take a subversion snapshot and convert it into a git repository.
It sounds like you are trying to avoid a lengthy initialization of the git repository from svn: which ordinarily will try to ready your entire history. This can be done in another way, by limiting the fetch to recent history depending on how much history is relevant to you:
git svn clone -s -r 12334:HEAD https://svn.host.org/repo
Where 12334 is the earliest svn revision you are interested in and assuming that the repo is laid out in a standard svn way with branches and tags.

`git svn dcommit` failing on a branch

I have been using git-svn to communicate with my company’s svn repo for a while now without any major headaches.
Today, the “headache”-part changed dramatically:
I’ve been working on master/trunk pretty exclusively, and needed to merge most (but not all!) of those change-sets into a new svn-branch, that originated from a pre-existing svn-branch.
Basically this:
🍒---💩---💩---💩--1🍒--1🍒---💩--1🍒---💩---💩--1🍒--1🍒--1🍒---💩 master/trunk
\
\
2🍒--2🍒--2🍒--2🍒--2🍒 versioned-release
Should have become this:
🍒---💩---💩---💩--1🍒--1🍒---💩--1🍒---💩---💩--1🍒--1🍒--1🍒---💩 master/trunk
\
\
2🍒--2🍒--2🍒--2🍒--2🍒 versioned-release
\
\
1🍒--1🍒--1🍒--1🍒--1🍒--1🍒 new-versioned-release
Where 💩 are commits that shouldn’t be in the new-versioned-release, and x🍒 the wanted commits from the respective branches x.
So I did the following:
git checkout -b versioned-release-svn remotes/versioned-release
git svn branch new-versioned-release -m "Preparing for merge of XXX"
git checkout -b new-versioned-release-svn remotes/new-versioned-release
git cherry-pick ... for every 1🍒, resolving any conflicts on the way.
Because I wanted to be sure I was really going to target the correct branch on the repo, I then ran git svn dcommit --dry-run which did not yield any errors or warnings, but told me…
Committing to svn://username#$repo-host/$repo-name/$path/branches/new-versioned-release ...
…followed by a couple of diff-tree lines.
So I attempted to omit the --dry-run and half way through the commits ended up with…
Item already exists in filesystem: File already exists: filesystem '/data/subvroot/$repo-name/db', transaction '20856-g3m', path '/$path/branches/new-versioned-release/some-directory' at /usr/libexec/git-core/git-svn line 862
…and a bunch of unstaged changes.
Apart from the obvious — “WTF?!?” and “How do I get out of this mess without losing everything I did?” — I have two questions:
Assuming I was back to before git svn dcommit: How do I get my local branch dcommit to its planned destination?
By now it seems obvious, that this wasn’t the right way to achieve what I wanted…but how should I have done it, instead?
Everything I found for the error-message, that somehow resembled my situation, so far was this other stack overflow question and the proposed solution of “somehow […] to blow away the .git/svn metadata directory” doesn’t resonate quite that well with me…
Someone just up–voted my old question, so I thought I’d share how I do that nowadays.
It works really quite well.
Assuming the git repository has been created using
git svn clone \
--prefix svn/ \
--stdlayout \
svn://username#$repo-host/$repo-name/$path
$git_repo_name
change into the git repo, and there run
git checkout svn/versioned-release
git svn branch new-versioned-release
This will result in the following history on the SVN server:
🍒---💩---💩---💩--1🍒--1🍒---💩--1🍒---💩---💩--1🍒--1🍒--1🍒---💩 trunk
\
\
2🍒--2🍒--2🍒--2🍒--2🍒 versioned-release
\
\
3⭐️ new-versioned-release
Now I’d run
git checkout svn/new-versioned-release
git checkout -b new-versioned-release
# resulting in the following **local** history:
#
# 🍒---💩---💩---💩--1🍒--1🍒---💩--1🍒---💩---💩--1🍒--1🍒--1🍒---💩 master (tracks 'svn/trunk')
# \
# \
# 2🍒--2🍒--2🍒--2🍒--2🍒--3⭐️ new-versioned-release (tracks 'svn/new-versioned-release')
This is the foundation for achieving what I wanted.
There is one additional commit, because branching in SVN doesn’t work the same way as in Git: creating a branch always means a new revision, (aka commit) and that’s where the 3⭐️ comes from. It doesn’t really matter, but it’s there.
I can now git cherry-pick all the 1🍒s, ending up with this local history:
🍒---💩---💩---💩--1🍒--1🍒---💩--1🍒---💩---💩--1🍒--1🍒--1🍒---💩 master (tracks 'svn/trunk')
\
\
2🍒--2🍒--2🍒--2🍒--2🍒--3⭐️--1🍒--1🍒--1🍒--1🍒--1🍒--1🍒 new-versioned-release (tracks 'svn/new-versioned-release')
When I now git svn dcommit while sitting on new-versioned-release in git, the history on the SVN server looks like what I wanted to end up with:
🍒---💩---💩---💩--1🍒--1🍒---💩--1🍒---💩---💩--1🍒--1🍒--1🍒---💩 trunk
\
\
2🍒--2🍒--2🍒--2🍒--2🍒 versioned-release
\
\
3⭐️--1🍒--1🍒--1🍒--1🍒--1🍒--1🍒 new-versioned-release
The only difference being that additional 3⭐️ from creating the third SVN branch.

Incomplete Clone Using Git-Svn

I have an Svn repository at http://svn.domain.com/project that is structured as follows:
trunk/
build_file_1.xml
build_file_2.xml
project_root/
branches/
cc2.10/
cc3.00/
..
cc3.5/
jira-labs-39/
tags/
studio-2.10.0.0/
studio-2.10.0.1/
...
studio-3.4.1.0/
I want to clone the trunk and branches, but I'm only getting the trunk and the first branch. I'm using this command to clone:
git svn clone http://svn.domain.com/project working-dir --trunk=trunk --branches=branches --prefix=svn/
What I end up with is this:
$ git br -r
svn/cc2.10
svn/trunk
I need to do some work on one of the other branches, but can't figure out what I'm doing wrong. Can someone point me in the right direction?
UPDATE
I just noticed the following error at the end of the output stream:
merge-base 7c552afeaba8194137acb95e642a2222db801dad c40b790b610dc43da93de5328832b1f852a14ef2: command returned error: 1
I assume that error is aborting the clone before it's complete, but I can't find any reference to the error or what it means in order to debug.
So the problem appears to be in the fact that Git-Svn tries to being cloning from one directory above the requested start point. Using the --no-minimize option fixed that.
git svn clone http://svn.domain.com/project working-dir -s --no-minimize-url