I have one main repo that contains multiple submodules that each have active development.
main_proj/
- sub1/
- sub2/
- sub3/
It is not uncommon for development in the main project to be performed in lock-step with one or more of the submodules.
1234abc main_proj commit 1
2345bcd main_proj commit 2
fedc987 submodule update1 for sub1 <-- only updates submodule hash
3456cde main_proj commit 3
edcb876 submodule update2 for sub1 <-- only updates submodule hash
aaaa001 submodule update1 for sub2 <-- updates main_proj AND submodule hash at same time
4567def main_proj commit 4
5678ef0 main_proj commit 5
cccc999 submodule update for sub1, sub2 and sub3 <-- updates submodule hashes
6789f01 main_proj commit 6
I want to run an interactive rebase on main_proj and rearrange the last two or three commits. Some of those commits, however, update the submodule hashes and therefore, causes merge conflicts:
Auto-merging sub1
CONFLICT (submodule): Merge conflict in sub1
error: could not apply 8792e483... Take 1
Resolve all conflicts manually, mark them as resolved with "git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".
Could not apply 8792e483... Take 1
Since the interactive rebase was intended to only work on files under main_proj, I will just re-apply the "submodule hash" commits once the work is done, so my ultimate commit will be pointing at all the currect submodule HEADs.
Is there a command during interactive rebase that I need to use in order to allow the commit to continue with "dirty" changes to the submodules?
Related
I am developing a kotlin script which executes code on the platform which it is running on. Platform code is called using this method from the script:
fun exec(command: String, vararg arguments: String, runLive: Boolean = !isDebug): OutputStream {
val allArgs = arguments.joinToString(" ")
if (runLive) {
val process = Runtime.getRuntime().exec(command, arguments)
val exitCode = process.waitFor()
if (exitCode != 0) {
val platformError = String(BufferedInputStream(process.errorStream).readAllBytes(), Charset.defaultCharset())
throw IllegalStateException("Execution of '$command $allArgs' failed with exit code $exitCode!\n$platformError")
}
return process.outputStream
} else {
println("$command $allArgs")
return object : OutputStream() {
override fun write(b: Int) {
println("dummy $b")
}
}
}
}
In the script, I try to get all the tags for a git repository using this call:
exec(command = "git", "tag", runLive = true)
When the command fails, how can process.errorStream be read? The output now is not readable and the script failure says:
java.lang.IllegalStateException: Execution of 'git tag' failed with exit code 1!
at Snap_tag_main.exec(snap-tag.main.kts:75)
at Snap_tag_main.<init>(snap-tag.main.kts:64)
Not all commands use the error stream when displaying errors. The solution is to bring both the input stream as well as the output stream like this:
if (exitCode != 0) {
val platformMessages = """
${String(BufferedInputStream(process.inputStream).readAllBytes(), Charset.defaultCharset())}
${String(BufferedInputStream(process.errorStream).readAllBytes(), Charset.defaultCharset())}
""".trimIndent().trim()
throw IllegalStateException("Execution of '$command $allArgs' failed with exit code $exitCode!\n$platformMessages\n---")
}
The failure message now looks like
java.lang.IllegalStateException: Execution of 'git tag' failed with exit code 1!
usage: git [--version] [--help] [-C <path>] [-c <name>=<value>]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
[--super-prefix=<path>] [--config-env=<name>=<envvar>]
<command> [<args>]
These are common Git commands used in various situations:
start a working area (see also: git help tutorial)
clone Clone a repository into a new directory
init Create an empty Git repository or reinitialize an existing one
work on the current change (see also: git help everyday)
add Add file contents to the index
mv Move or rename a file, a directory, or a symlink
restore Restore working tree files
rm Remove files from the working tree and from the index
examine the history and state (see also: git help revisions)
bisect Use binary search to find the commit that introduced a bug
diff Show changes between commits, commit and working tree, etc
grep Print lines matching a pattern
log Show commit logs
show Show various types of objects
status Show the working tree status
grow, mark and tweak your common history
branch List, create, or delete branches
commit Record changes to the repository
merge Join two or more development histories together
rebase Reapply commits on top of another base tip
reset Reset current HEAD to the specified state
switch Switch branches
tag Create, list, delete or verify a tag object signed with GPG
collaborate (see also: git help workflows)
fetch Download objects and refs from another repository
pull Fetch from and integrate with another repository or a local branch
push Update remote refs along with associated objects
'git help -a' and 'git help -g' list available subcommands and some
concept guides. See 'git help <command>' or 'git help <concept>'
to read about a specific subcommand or concept.
See 'git help git' for an overview of the system.
---
at Snap_tag_main.exec(snap-tag.main.kts:82)
at Snap_tag_main.<init>(snap-tag.main.kts:66)
Building on this question, I have a workflow where I'm constantly making PRs on top of PRs to make it easier for others to review my work. Goal is to have smaller PR sizes. So I often end up with situations like the following:
G--H--I <-- branch3
/
D--E--F <-- branch2
/
A--B--C <-- branch1
/
M <-- master
And so on for N branches after branch3. The problem is, after I squash and merge branch1, I have to manually rebase branches 2, 3...N:
G--H--I <-- branch3
/
D--E--F <-- branch2
/
A--B--C
/
M--S <-- master, origin/master (branch1 changes are squashed in S)
In the above case, I have to run:
git checkout branch2
git rebase --onto master (SHA-1 of C)
git checkout branch3
git rebase --onto branch2 (SHA-1 of F)
And so on...
Is there a way to automate this process by rebasing all branches automatically with a script? What I can't figure out is a way to automatically detect the correct SHA-1 to pass as parameter for each rebase.
There are a couple of fundamental problems, or maybe one fundamental problem, depending on how you look at it. That is:
branches do not have parent/child relationships, and/or
branches, in the sense you mean the word, don't exist. All that we have are branch names. The branches themselves are mirages, or something. (This doesn't really seem like the right way to look at it, but it helps shake one loose from the more rigid view of branches that most non-Git systems take.)
Let's start with a question that seems straightforward, but because Git is Git, is actually a trick question: which branch holds commits A-B-C?
Is there a way to automate this process by rebasing all branches automatically with a script? What I can't figure out is a way to automatically detect the correct SHA-1 to pass as parameter for each rebase.
There isn't a general solution to this problem. If you have exactly the situation you have drawn, however, there is a specific solution to your specific situation—but you'll have to write it yourself.
The answer to the trick question is that commits A-B-C are on every branch except master. A branch name like branch3 just identifies one particular commit, in this case commit I. That commit identifies another commit, in this case, commit H. Each commit always identifies some previous commit—or, in the case of a merge commit, two or more previous commits—and Git simply works backwards from the end. "The end" is precisely that commit whose hash ID is stored in the branch name.
Branch names lack parent/child relationships because every branch name can be moved or destroyed at any time without changing the hash ID stored in each other branch. New names can be created at any time too: the only constraint on creating a new name is that you must pick some existing commit for that name to point-to.
The commits have parent/child relationships, but the names do not. This leads to the solution to this specific situation, though. If commit Y is a descendant of commit X, that means there's some backwards path where we start at Y and can work our way back to X. This relationship is ordered—mathematically speaking, it forms a partial order over the set of commits—so that X ≺ Y (X precedes Y, i.e., X is an ancestor of Y), then Y ≻ X (Y succeeds X: Y is a descendant of X).
So we take our set of names, translate each name to a commit hash ID, and perform these is-ancestor tests. Git's "is-ancestor" operator actually tests for ≼ (precedes or is equal to), and the is-equal case occurs with:
...--X <-- name1, name2
where both names select the same commit. If that could occur we would have to analyze what our code might do with that case. It turns out that this usually doesn't require any special work at all (though I won't bother proving this).
Having found the "last" commit—the one for which every commit comes "before" the commit in question—we now need to do our rebase operation. We have:
G--H--I <-- branch3
/
D--E--F <-- branch2
/
A--B--C
/
M--S <-- master, origin/master (branch1 changes are squashed in S)
just as you showed, and we know that S represents the A-B-C sequence because we picked commit C (via the name branch1) when we made S. Since the last commit is commit I, we want to copy—as rebase does—every commit from D through I, with the copies landing after S. It might be best if Git didn't move any of these branch names at all, during the copying operation, and we can get that to happen using Git's detached HEAD mode:
git checkout --detach branch3 # i.e., commit `I`
or:
git checkout <hash-of-I> # detach and get to commit `I`
or:
git switch --detach ... # `git switch` always requires the --detach
which gets us:
G--H--I <-- branch3, HEAD
/
D--E--F <-- branch2
/
A--B--C
/
M--S <-- master, origin/master
We now run git rebase --onto master branch1 if the name branch1 is still available, or git rebase --onto master <hash-of-C> if not. This copies everything as desired:
G--H--I <-- branch3
/
D--E--F <-- branch2
/
A--B--C
/
M--S <-- master, origin/master
\
D'-E'-F'
\
G'-H'-I' <-- HEAD
Now all (?) we need to do is go back through those same sets of branch names and count how far they are along the chain of original commits. Because of the way Git works—backwards—we'll do this starting from wherever they end and working backwards to commit C. For this particular drawing, that's 3 for branch2 and 6 for branch3. We count how many commits we copied as well, which is also of course 6. So we subtract 3 from 6 for branch2, and 6 from 6 for branch3. That tells us where we should move those branch names now: zero steps back from I' for branch3, and three steps back from I' for branch2. So now we make one last loop through each name and re-set each name as appropriate.
(Then we probably should pick some name to git checkout or git switch to.)
There are some challenges here:
Where did we get this set of names? The names are branch1, branch2, branch3, and so on, but in reality they won't be so obviously related: why do we move branch fred but not branch barney?
How did we know that branch1 is the one that we shouldn't use here, but should use as the "don't copy this commit" argument to our git rebase-with-detached-HEAD?
How exactly do we do this is-ancestor / is-descendant test?
This question actually has an answer: git merge-base --is-ancestor is the test. You give it two commit hash IDs and it reports whether the left-hand one is an ancestor of the right-hand one: git merge-base --is-ancestor X Y tests X ≼ Y. Its result is its exit status, suitable for use in shell scripts with the if built in.
How do we count commits?
This question also has an answer: git rev-list --count stop..start starts at the start commit and works backwards. It stops working backwards when it reaches stop or any of its ancestors. It then reports a count of the number of commits visited.
How do we move a branch name? How do we figure out which commit to land on?
This one is easy: git branch -f will let us move an existing branch name, as long as we do not have that name currently checked-out. As we are on a detached HEAD after the copying process, we have no name checked-out, so all names can be moved. Git itself can do the counting-back, using the tilde and numeric suffix syntax: HEAD~0 is commit I', HEAD~1 is commit H', HEAD~2 is commit G', HEAD~3 is commit F', and so on. Given a number $n we just write HEAD~$n, so git branch -f $name HEAD~$n does the job.
You still have to solve the first two questions. The solution to that will be specific to your particular situation.
Worth pointing out, and probably the reason no one has written a proper solution for this—I wrote my own approximate solution many years ago but abandoned it many years ago as well—is that this whole process breaks down if you don't have this very specific situation. Suppose that instead of:
G--H--I <-- branch3
/
D--E--F <-- branch2
/
A--B--C <-- branch1
/
M <-- master
you begin with:
G--H--I <-- branch3
/
D--E--F <-- branch2
/
A--B--C <-- branch1
/
M <-- master
This time, ending at commit I and copying all commits that reach back through, but do not include, commit C fails to copy commit F. There is no F' to allow you to move branch name branch2 after copying D-E-G-H-I to D'-E'-G'-H'-I'.
This problem was pretty major, back in the twenty-aughts and twenty-teens. But git rebase has been smartened up a bunch, with the newfangled -r (--rebase-merges) interactive rebase mode. It now has almost all the machinery for a multi-branch rebase to Just Work. There are a few missing pieces that are still kind of hard here, but if we can solve the first two problems—how do we know which branch names to multi-rebase in the first place—we could write a git multirebase command that would do the whole job.
I am orchestrating an automatic delivery process using GIT, Bamboo, Ansible and Liquibase.
I am having some issues trying to use Liquibase rollback feature. Basically I have the same files, my change set master and the version files (each having its own rollback section), in two different places, say an "upgrade" folder and a "rollback" folder. Even though the files are the same, the rollback simply does not work. Illustrating:
+ deployment_folder
+ update
- changeset-master.xml
- changeset-1.0.0.xml
- changeset-1.0.1.xml
+ rollback
- changeset-master.xml
- changeset-1.0.0.xml
- changeset-1.0.1.xml
The files have exactly the same content.
Running liquibase updates and tagging is fine:
$>liquibase --username=USR --password=*** --classpath=./ojdbc7.jar --driver=oracle.jdbc.driver.OracleDriver --url=jdbc:oracle:thin:#host:port:SID --changeLogFile=update/changeset-master.xml update
$>liquibase --username=USR --password=*** --classpath=./ojdbc7.jar --driver=oracle.jdbc.driver.OracleDriver --url=jdbc:oracle:thin:#host:port:SID --changeLogFile=update/changeset-master.xml tag 1.0.0
$>liquibase --username=USR --password=*** --classpath=./ojdbc7.jar --driver=oracle.jdbc.driver.OracleDriver --url=jdbc:oracle:thin:#host:port:SID --changeLogFile=update/changeset-master.xml update
$>liquibase --username=USR --password=*** --classpath=./ojdbc7.jar --driver=oracle.jdbc.driver.OracleDriver --url=jdbc:oracle:thin:#host:port:SID --changeLogFile=update/changeset-master.xml tag 1.0.1
However, when trying to rollback from 1.0.1 to 1.0.0 using the change set master in rollback folder it says "Liquibase Rollback Successful" but the changes are not rolled back. the rollbackSQL command also does not display any relevant SQL statement other than the DATABASECHANGELOGLOCK updates.
$>liquibase --username=USR --password=*** --classpath=./ojdbc7.jar --driver=oracle.jdbc.driver.OracleDriver --url=jdbc:oracle:thin:#host:port:SID --changeLogFile=rollback/changeset-master.xml rollback 1.0.0
Looks like the file has to be exactly the same (for checksum I suppose), which is a show blocker in my case, where I have to constantly pull versions from my source control system, so the files will never be "the same", although they have the same content. Is there any way to disable this verification in Liquibase? Currently I am using Liquibase 3.4.2.
Liquibase rollback doesn't work this way, you need to specify rollback instructions in <rollback> section of your script and execute it with rollback command: http://www.liquibase.org/documentation/rollback.html
UPD: Liquibase uses id based on changeset file name, change id and author id to identify changes. If you apply <change id="1" author="dbf">...</change> from file f1.xml it's 'virtual' id is <f1.xml, 1, dbf>. When you run f2.xml with the same content for rollback the id it calculates is <f2.xml, 1, dbf> so it doesn't match the first id and nothing is rolled back.
Currently, we have the following in our CVS Repository :
Module1
|
|
+-----A
|
+-----B
We want o restructure this module such that the sub directories A and B appears as high level modules. What I could do is to check Module1 out and then pull A and B out and then do a fresh cvs add for A and B individually, thus making them new cvs modules. But I am sure if I do this, I am going to lose the history as well as I would have to remove all internal CVS folders under A and B.
Q1: So is there a way to restructure this and retain the history?
What I essentially am trying to do is to filter out access between A and B.
So -
Q2: Is there a way to set up security so that certain users can check out Module1/A only and not Module1/B ? and vice-versa?
Q1: So is there a way to restructure this and retain the history?
Like you wrote in your comment, if you have sys privs you can mv modules around the repository and keep the history of all the files below A and B but in doing so, you lose the history that /A used to be Module1/A and /B used to be in Module1/B (not to mention build scripts probably break now). Subversion resolves this for you by offering the move (or rename) command which remembers the move/rename history of a module.
Q2: Is there a way to set up security so that certain users can check out Module1/A only and not Module1/B ? and vice-versa?
There sure is, used group permissions. From this page,
http://www.linux.ie/articles/tutorials/managingaccesswithcvs.php
Here's the snip I'm referring to in case that page ever goes away
To every module its group
We have seen earlier how creating a
cvsusers group helped with the
coordination of the work of several
developers. We can extend this
approach to permit directory level
check-in restrictions.
In our example, let's say that the
module "cujo" is to be r/w for jack
and john, and the module "carrie" is
r/w for john and jill. We will create
two groups, g_cujo and g_carrie, and
add the appropriate users to each - in
/etc/group we add
g_cujo:x:3200:john,jack
g_carrie:x:3201:john,jill>
Now in the repository (as root), run
find $CVSROOT/cujo -exec chgrp g_cujo {} \;
find $CVSROOT/carrie -exec chgrp g_carrie {} \;
ensuring, as before, that all
directories have the gid bit set.
Now if we have a look in the
repository...
john#bolsh:~/cvs$ ls -l
total 16
drwxrwsr-x 3 john cvsadmin 4096 Dec 28 19:42 CVSROOT
drwxrwsr-x 2 john g_carrie 4096 Dec 28 19:35 carrie
drwxrwsr-x 2 john g_cujo 4096 Dec 28 19:40 cujo
and if Jack tries to commit a change
to carrie...
jack#bolsh:~/carrie$ cvs update
cvs server: Updating .
M test
jack#bolsh:~/carrie$ cvs commit -m "test"
cvs commit: Examining .
Checking in test;
/home/john/cvs/carrie/test,v <-- test
new revision: 1.2; previous revision: 1.1
cvs [server aborted]: could not open lock file
`/home/john/cvs/carrie/,test,': Permission denied
jack#bolsh:~/carrie$
But in cujo, there is no problem.
jack#bolsh:~/cujo$ cvs update
cvs server: Updating .
M test
jack#bolsh:~/cujo$ cvs commit -m "Updating test"
cvs commit: Examining .
Checking in test;
/home/john/cvs/cujo/test,v <-- test
new revision: 1.2; previous revision: 1.1
done
jack#bolsh:~/cujo$
The procedure for adding a user is now
a little more complicated that it
might be. To create a new CVS user, we
have to create a system user, add them
to the groups corresponding to the
modules they may write to, and (if
you're using a pserver method)
generate a password for them, and add
an entry to CVSROOT/passwd.
To add a project, we need to create a
group, import the sources, change the
groups on all the files in the
repository and make sure the set gid
on execution bit is set on all
directories inside the module, and add
the relevant users to the group.
There is undoubtedly more
administration needed to do all this
than when we jab people with a pointy
stick. In that method, we never have
to add a system user or a group or
change the groups on directories - all
that is taken care of once we set up
the repository. This means that an
unpriveleged user can be the CVS admin
without ever having root priveleges on
the machine.
When I have a history
-rev 100, merge
-- rev 95.3
-- rev 95.2
-- rev 95.1
-rev 99
-rev 98
and I try bzr uncommit -r 95.3.. it writes an error message. How can I fix the problem?
Error is
bzr: ERROR: exceptions.TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
Traceback (most recent call last):
File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 846, in run_bzr_catch_errors
return run_bzr(argv)
File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 797, in run_bzr
ret = run(*run_argv)
File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 499, in run_argv_aliases
return self.run(**all_cmd_args)
File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 3694, in run
local=local)
File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 3717, in _run
revno = revision[0].in_history(b).revno + 1
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
bzr 1.5 on python 2.5.2 (linux2)
arguments: ['/usr/bin/bzr', 'uncommit', '-r', '11955.2.32..']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'pl_PL.UTF-8'
plugins:
bzrtools /usr/lib/python2.5/site-packages/bzrlib/plugins/bzrtools [1.5.0]
gtk /usr/lib/python2.5/site-packages/bzrlib/plugins/gtk [0.94.0]
interactive /home/adi/.bazaar/plugins/interactive [1.2.0]
launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [unknown]
rebase /home/adi/.bazaar/plugins/rebase [0.3.0]
*** Bazaar has encountered an internal error.
Please report a bug at https://bugs.launchpad.net/bzr/+filebug
including this traceback, and a description of what you
were doing when the error occurred.
This is really just a guess, but I suspect that the revision spec isn't accepting 95.3 as a revision.
The merge will have been committed to your branch as revision 100, so you should be able to just do:
bzr uncommit
or:
bzr uncommit -r 100
If you want to merge in changes 95.1 and 95.2 (part of the original merge), you will probably have to do:
bzr uncommit -r 100
bzr merge -r 95..97 /path/to/merge/source
or something like that.
If you need to refer to the 95.3, I'd recommend having a read of the Bazaar revision spec documentation to see if that helps. You may need a more explicit revision number such as revno:95.3 (which probably won't work) or revno:97:/path/to/merge/source (which might work). Most of that was a guess though...
You can't uncommit to dotted revision like 95.1.3, because this revision is not in your mainline history. You can use pull command instead to get your tree to desired revision:
bzr pull . -r95.1.3 --overwrite
The main difference between uncommit and pull is: uncommit leaves changes after revision to which you uncommit in the working tree, while pull simply overwrite your working tree with files of specified revision.
Also if you need to uncommit to the state just before merge you need to use bzr uncommit without specifying revisions if r100 is the last revision in the branch.