I am trying to add submodules support to a project of mine, however I seem to be unable to use JGit to read specific commits or to log, while commandline git can do this just fine, despite the detached head that the submodule is in (as far as by design of how submodules work).
I have code like this, with "fuse-jna" being the submodule:
FileRepositoryBuilder builder = new FileRepositoryBuilder();
Repository repository = builder.setGitDir(new File("fuse-jna/.git"))
.readEnvironment() // scan environment GIT_* variables
.findGitDir() // scan up the file system tree
.build();
ObjectId lastCommitId = repository.resolve("39c1c4b78ff751b0b9e28f4fb35148a1acd6646f");
Iterable<RevCommit> commits = new Git(repository).log().add(lastCommitId).call();
for(RevCommit commit : commits) {
System.out.println("Commit: " + commit.getId());
}
with this I get:
org.eclipse.jgit.errors.MissingObjectException: Missing commit 39c1c4b78ff751b0b9e28f4fb35148a1acd6646f
at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:149)
at org.eclipse.jgit.revwalk.RevWalk.getCachedBytes(RevWalk.java:883)
at org.eclipse.jgit.revwalk.RevCommit.parseHeaders(RevCommit.java:145)
at org.eclipse.jgit.revwalk.RevWalk.markStart(RevWalk.java:279)
at org.eclipse.jgit.api.LogCommand.add(LogCommand.java:330)
at org.eclipse.jgit.api.LogCommand.add(LogCommand.java:189)
at org.dstadler.jgitfs.util.JGitHelperTest.testGitLinkRepository(JGitHelperTest.java:638)
However on the commandline in the "fuse-jna" directory
git diff 39c1c4b78ff751b0b9e28f4fb35148a1acd6646f~
works just fine, so it is possible to do this.
Is there some other way how to do this with JGit? Or is support for submodules not fully there yet?
Seems the trick is to not open the repository of the submodule like a normal Git repository, but use the JGit class SubmoduleWalk and the parent-repository to do this!
The following did work for me after playing around with it some more:
Repository subRepo = SubmoduleWalk.getSubmoduleRepository(repository, "fuse-jna");
Related
TL;DR
In Puppet Enterprise, how do I run a manifest (testpp.pp) from a task or plan (not Bolt).
plan base_windows::testplan (
TargetSpec $targets,
Optional[String] $contents = undef,
String $filename,
){
$apply_prep($targets)
$apply_results = apply($targets, '_catch_errors' => true) {
class { 'base_windows::testpp': }
}
$apply_results.each | $result | {
notice($result.report)
}
}
apply_prep seems to succeed, but apply is failing with the following error:
{
"msg" : "Evaluation Error: Unknown function: 'report'. (file: /opt/puppetlabs/server/data/orchestration-services/code/environments/development/modules/base_windows/plans/testplan.pp, line: 16, column: 19)",
"kind" : "bolt/plan-failure",
"details" : {
"class" : "Bolt::PAL::PALError"
}
}
If I change the code to:
plan base_windows::testplan (
TargetSpec $targets,
Optional[String] $contents = undef,
String $filename,
){
apply_prep($targets)
$apply_results = apply($targets, '_catch_errors' => true) {
# Is this how to call a class? I cannot find an example.
class { 'base_windows::testpp': }
}
$apply_results.each |$result| {
$target = $result.target.name
if $result.ok {
out::message("${target} returned a value: ${result.value}")
} else {
out::message("${target} errored with a message: ${result.error.message}")
}
}
}
The plan tells me it has failed, but there are no errors in the node's report. In fact, there is no entry for the time the plan was executed.
I cannot find any examples on how to call a class from a plan, so the above apply() is a guess, based on this documentation.
I have installed the puppetlabs_reboot module and successfully ran a plan using it, therefore, I conclude my system is set up correctly, it's just my code that is wrong.
Background
I may be going about this all wrong, so here is some background to the problem. Currently, I have a series of manifests that install various packages from the public Chocolatey repository depending on a node's classification. Package definitions are stored in Hiera data and each package' version is set to latest. At the end of the Package{} resource, some manifests include a reboot.
These manifests are used to provision new nodes and keep existing nodes up-to-date with the latest package version.
The Puppet agent is set to run once per hour and if the source package is updated in the Chocolatey repo, on the next Puppet run, the manifest will update the package, rebooting the node, if required.
Goal
New nodes are provisioned with the latest package version.
Prevent package updates at undetermined times on existing nodes.
Continue to allow Puppet agent runs every hour.
Make use of existing manifests.
Ideas
Split out the package{} code from the profile manifest and place them in tasks / plans, allowing packages to be updated out-of-hours.
Specify the actual package version in Hiera. Although this is more declarative and idempotent, it means keeping an eye on over 100 package version. I guess it would be fairly simple to interrogate the Chocolatey repos with code to pull the latest version number, but even so I am no better off.
Create a task with a script that runs choco upgrade all, however, the next Puppet run would revert package versions according to the version defined in Hiera, meaning Hiera still needs to be kept up-to-date.
Problems
As per the main crux of this question, how do I run manifests (classes) from plans? If I understand correctly, tasks are for ad-hoc scripts, whereas plans can run tasks and manifests. As a lot of time has been invested in writing manifests, I would prefer not to rewrite all my manifests as scripts.
I am confused by the Puppet documentation as it seems to switch between PE and Bolt syntax. I am using Puppet Enterprise where Puppet says they don't recommend using Bolt but their examples seem to site Bolt commands.
No errors in the node' report. apply_prep() reports executed successfully, albeit taking far longer to execute than puppetlabs_reboot module, but apply() results in a failure, but nothing is logged in the node's reports.
Using puppetlabs_reboot module as a reference, it appears their plan uses a bunch of tasks. It appears that they don't use apply() to run their reboot{} class. Is this not duplicating the work?
If anyone has any suggestions or ideas, I'd be grateful if you could share.
I've got it to work. The class I was trying to run, required parameters that I hadn't provided!
plan base_windows::testplan (
TargetSpec $targets,
Optional[String] $contents = undef,
String $filename,
){
apply_prep($targets)
$apply_results = apply($targets, '_catch_errors' => true) {
class { 'base_windows::testpp':
filename => $filename,
contents => $contents,
}
}
}
# Output the whole result_set in the PE console
return $apply_results
I found this out using the logs.
Turn on debug level logging in /etc/puppetlabs/puppetserver/logback.xml (root level="debug")
Tail the following logs:
tail -f /var/log/puppetlabs/bolt-server/bolt-server.log
tail -f /var/log/puppetlabs/puppetserver/puppetserver.log | grep -B 5 -A 5 'testplan'
tail -f /var/log/puppetlabs/orchestration-services/orchestration-services.log
i try to Checkout a Remotebranch via LibGitSharp. In git itself you use this comands:
git fetch origin
git checkout -b test origin/test
in newer Versions it is just:
git fetch
git checkout test
So i tried this Code:
repo.Fetch("origin");
repo.Checkout("origin/" + Name);
The Fetch and Checkout runs without any problems but there is no copy of the Remotebranch.
Does anyone have an idea to Checkout the Remote with other methods?
My alternative would be to create the Branch in the Repository and push it into Remote:
Branch newBranch = repo.Branches.Add(Name, repo.Branches["master"].Commits.First());
repo.Network.Push(newBranch);
but i get this Exception:
The branch 'Test1' ("refs/heads/Test1") that you are trying to push does not track an upstream branch.
maybe i could set the Branch to an upstream Branch, but i don't know how.
Edit: I haven't explained it properly, so I try to describe it better what the Fetch and Checkout does in my program. The Fetch command is performed correctly. Now if i use the checkout command it should be create an local Branch of the Remotebranch, but it doesn't. I've also tried repo.Checkout(name), without "origin/" ,but it cast an Exception: No valid git object identified by '...' exists in the repository.
If I correctly understand your question, you're willing to create a local branch which would be configured to track the fetched remote tracking branch.
In other words, once you fetch a repository, your references contains remote tracking branches (eg. origin/theBranch) and you'd like to create a local branch bearing the same name (eg. theBranch).
The following example should demonstrate how to do this
const string localBranchName = "theBranch";
// The local branch doesn't exist yet
Assert.Null(repo.Branches[localBranchName]);
// Let's get a reference on the remote tracking branch...
const string trackedBranchName = "origin/theBranch";
Branch trackedBranch = repo.Branches[trackedBranchName];
// ...and create a local branch pointing at the same Commit
Branch branch = repo.CreateBranch(localBranchName, trackedBranch.Tip);
// The local branch is not configured to track anything
Assert.False(branch.IsTracking);
// So, let's configure the local branch to track the remote one.
Branch updatedBranch = repo.Branches.Update(branch,
b => b.TrackedBranch = trackedBranch.CanonicalName);
// Bam! It's done.
Assert.True(updatedBranch.IsTracking);
Assert.Equal(trackedBranchName, updatedBranch.TrackedBranch.Name);
Note: More examples can be found in the BranchFixture.cs test suite.
I've been trying to create a new repository on a remote GraphDB server using RDF4J, but I'm having problems.
This runs, but is seemingly not correct
HTTPRepositoryConfig implConfig = new HTTPRepositoryConfig(address);
RepositoryConfig repoConfig = new RepositoryConfig("test", "test", implConfig);
Model m = new
However, based on the info I get from "edit repository" in the workbench, the result doesn't look right. All the values are empty, except for id and title.
This fails
I tried to copy the settings from an existing repository that I created on the workbench, but that failed with:
org.eclipse.rdf4j.repository.config.RepositoryConfigException:
Unsupported repository type: owlim:MonitorRepository
The code for that attempt is inspired by the one found here . Except that the config file is based on an existing repo, as explained above. I also tried to config file provided in the example, but that failed aswell:
org.eclipse.rdf4j.repository.config.RepositoryConfigException:
Unsupported Sail type: graphdb:FreeSail
Anyone got any tips?
UPDATE
As Henriette Harmse correctly pointed out, I should have provided my code, not simply linked to it. That way I might have discovered that I hadn't done a complete copy after all, but changed the important first bits that she points out in her answer. Full code below:
String address = "serveradr";
RemoteRepositoryManager repositoryManager = new RemoteRepositoryManager( address);
repositoryManager.initialize();
// Instantiate a repository graph model
TreeModel graph = new TreeModel();
InputStream config = Rdf4jHelper.class.getResourceAsStream("/repoconf2.ttl");
RDFParser rdfParser = Rio.createParser(RDFFormat.TURTLE);
rdfParser.setRDFHandler(new StatementCollector(graph));
rdfParser.parse(config, RepositoryConfigSchema.NAMESPACE);
config.close();
// Retrieve the repository node as a resource
Resource repositoryNode = graph.filter(null, RDF.TYPE, RepositoryConfigSchema.REPOSITORY).subjects().iterator().next();
// Create a repository configuration object and add it to the repositoryManager
RepositoryConfig repositoryConfig = RepositoryConfig.create(graph, repositoryNode);
It fails on the last line.
ANSWERED #HenrietteHarmse gives the correct method in her answer below. The error is caused by missing dependencies. Instead of using RDF4J directly, I should have used the graphdb-free-runtime.
There are a number of issues here:
(1) RepositoryManager repositoryManager = new LocalRepositoryManager(new File(".")); will create a repository where ever your Java application is running from.
(2) Changing to new LocalRepositoryManager(new File("$GraphDBInstall/data/repositories")) will cause the repository to be created under the control of GraphDB (assuming you have a local GraphDB instance) only if GraphDB is not running. If you start GraphDB after running your program, you will be able to see the repository in GraphDB workbench.
(3) What you need to do is get the repository manager of the remote GraphDB, which can be done with RepositoryManager repositoryManager = RepositoryProvider.getRepositoryManager("http://IPAddressOfGraphDB:7200");.
(4) In the way you have specified the config, you cause the RDF graph config to be lost. The correct way to specify it is:
RepositoryConfig repositoryConfig = RepositoryConfig.create(graph, repositoryNode);
repositoryManager.addRepositoryConfig(repositoryConfig);
(5) A minor issue is that GraphUtil.getUniqueSubject(...) has been deprecated, for which you can use something like the following:
Model model = graph.filter(null, RDF.TYPE, RepositoryConfigSchema.REPOSITORY);
Iterator<Statement> iterator = model.iterator();
if (!iterator.hasNext())
throw new RuntimeException("Oops, no <http://www.openrdf.org/config/repository#> subject found!");
Statement statement = iterator.next();
Resource repositoryNode = statement.getSubject();
EDIT on 20180408:
(5) Or you can use the compact option as #JeenBroekstra suggested in the comments:
Models.subject(
graph.filter(null, RDF.TYPE, RepositoryConfigSchema.REPOSITORY))
.orElseThrow(() -> new RuntimeException("Oops, no <http://www.openrdf.org/config/repository#> subject found!"));
EDIT on 20180409:
For convenience I have added the complete code example here.
EDIT on 20180410:
So the actual culprit turned out to be an incorrect pom.xml. The correct version is as below:
<dependency>
<groupId>com.ontotext.graphdb</groupId>
<artifactId>graphdb-free-runtime</artifactId>
<version>8.4.1</version>
</dependency>
I believe I just had the same issue. I used the example code from GraphDB Free for running with RDF4J as a remote service and ran into the same exception as you (Unsupported Sail type: graphdb:FreeSail). Henriette Harmse's answer does not directly address this issue but one should follow the suggestions given there to avoid running into issues later. In addition, based on a look into the RDF4J code you need the following dependency in your pom.xml file (assuming GraphDB 8.5):
<dependency>
<groupId>com.ontotext.graphdb</groupId>
<artifactId>graphdb-free-runtime</artifactId>
<version>8.5.0</version>
</dependency>
This seems to be because there is some kind of service loading going on with META-INF, which I frankly am not familiar with. Maybe someone can provide more details in the comments. The requirement for adding this dependency in also seems to be absent from the instructions, so if this works for you, please let me know. Others who followed the same steps we did should be able to resolve this issue as well then.
We have a custom build tool which is dependent on the ivy functionality to resolve dependencies. The configuration of the dependencies is not an ivy.xml file, but a custom configuration that allows for.. well, irrelevant. The key is that we're using ivy programmatically.
Given a dependency (group id, artifact id, version), we create a ModuleRevisionId:
ModuleRevisionId id = ModuleRevisionId.newInstance(orgName, moduleName, revisionName);
followed by a ModuleDescriptor. This is, I'm guessing, where I'm not convincing enough to inform ivy that I want both the target library jar file as well as the sources. I'm just not sure what a DependencyConfiguration is vs. just a 'configuration' when creating a ModuleDescriptor.
DefaultModuleDescriptor md
= new DefaultModuleDescriptor(
ModuleRevisionId.parse("org#standalone;working"),
"integration",
new java.util.Date());
DefaultDependencyDescriptor mainDep
= new DefaultDependencyDescriptor(id, /* force = */ true);
mainDep.addDependencyConfiguration("compile", "compile");
mainDep.addDependencyConfiguration("compile", "sources");
md.addDependency(mainDep);
md.addConfiguration(new Configuration("compile"));
md.addConfiguration(new Configuration("sources"));
Nor do I really understand the above vs. RetrieveOptions vs. ResolveOptions.
I need a drink.
Ok, so it took a while, but I finally wrapped my head around some of this.
// define 'our' module
DefaultModuleDescriptor md
= new DefaultModuleDescriptor(ModuleRevisionId.parse("org#standalone;working"),
/* status = */ "integration",
new java.util.Date());
// add a configuration to our module definition
md.addConfiguration(new Configuration("compile"));
// define a dependency our module has on the (third party, typically) dependee module
DefaultDependencyDescriptor mainDep = new DefaultDependencyDescriptor(md, dependeeModuleId, /* force = */ true, false, true);
mainDep.addDependencyConfiguration("compile", "default");
mainDep.addDependencyConfiguration("compile", "sources");
// define which configurations we want to resolve (only have 1 in this case anyway)
ResolveOptions resolveOptions = new ResolveOptions();
String[] confs = new String[] {"compile"};
resolveOptions.setConfs(confs);
resolveOptions.setTransitive(true); // default anyway
resolveOptions.setDownload(true); // default anyway
ResolveReport report = ivy.resolve(md, resolveOptions);
This pulls down both the default jar as well as the sources target. Note that ivy has an issue where it won't transitively pull sources, though it will transitively pull 'main' jars. So you only get the sources for immediate dependency defined here, not the sub dependencies.
One other weakness I'm trying to figure out is this assumes the target dependency has a 'sources' configuration. I'd rather tell it to get any artifacts of type sources/source/src. Haven't figured that one out yet.
I'm using the v3 API and managed to list repos/trees/branches, access file contents, and create blobs/trees/commits. I'm now trying to create a new repo, and managed to do it with "POST user/repos"
But when I try to create blobs/trees/commits/references in this new repo I get the same error message. (409) "Git Repository is empty.". Obviously I can go and init the repository myself through the git command line, but would rather like if my application did it for me.
Is there a way to do that? What's the first thing I need to do through the API after I create an empty repository?
Thanks
Since 2012, it is now possible to auto initialize a repository after creation, according to this blog post published on the GitHub blog:
Today we’ve made it easier to add commits to a repository via the GitHub API. Until now, you could create a repository, but you would need to initialize it locally via your Git client before adding any commits via the API.
Now you can optionally init a repository when it’s created by sending true for the auto_init parameter:
curl -i -u pengwynn \
-d '{"name": "create-repo-test", "auto_init": true}' \
https://api.github.com/user/repos
The resulting repository will have a README stub and an initial commit.
Update May 2013: Note that the repository content API now authorize adding files.
See "File CRUD and repository statistics now available in the API".
Original answer (May 2012)
Since it doesn't seems to be supported yet ("GitHub v3 API: How to create initial commit for my shiny new repository?", as aclark comments), you can start by pushing an initial empty commit
git commit --allow-empty -m 'Initial commit'
git push origin master
That can be a good practice to initialize one's repository anyway.
And it is illustrated in "git's semi-secret empty tree".
If you want to create an empty initial commit (i.e. one without any file) you can do the following:
Create the repository using the auto_init option as in Jai Pandya's answer; or, if the repository already exists, use the create file endpoint to create a dummy file - this will create the branch:
PUT https://api.github.com/repos/USER/REPO/contents/dummy
{
"branch": "master",
"message": "Create a dummy file for the sake of creating a branch",
"content": "ZHVtbXk="
}
This will give you a bunch of data including a commit SHA, but you can discard all of it since we are about to obliterate that commit.
Use the create commit endpoint to create a commit that points to the empty tree:
POST https://api.github.com/repos/USER/REPO/git/commits
{
"message": "Initial commit",
"tree": "4b825dc642cb6eb9a060e54bf8d69288fbee4904"
}
This time you need to take note of the returned commit SHA.
Use the update reference endpoint to make the branch point to the commit you just created (notice the Use Of The ForceTM):
PATCH https://api.github.com/repos/USER/REPO/git/refs/heads/master
{
"sha": "<the SHA of the commit>",
"force": true
}
Done! Your repository has now one branch, one commit and zero files.
2023 January: The 2. step of Konamiman's solution did not work for me, unless I deleted the dummy file with the contents api:
DELETE https://api.github.com/repos/USER/REPO/contents/dummy
{
"branch": "master",
"message": "Delete dummy file",
"sha": "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"
}
(This sha is for the empty content "".)
After deleting the dummy file somehow 4b825dc642cb6eb9a060e54bf8d69288fbee4904 becomes assignable to a new commit.
It looks like over time this changed. It feels bad to rely on undocumented API behavior. :(