I have a usecase where I need to be able to inspect Git repositories as part of a web service and the average repo size will be very large - 1GB+ due to being used for video game projects. I need to do simple actions such as listing the revision history, etc.
Right now I'm implementing it via API calls to the remote Git host services (Github, Bitbucket, etc). This works okay, however there are some great Git projects like GitVersion that only work with real Git repos, that use libGit2sharp, and I cannot easily write a work around for.
I feel like this'll be a longshot, but I was wondering if anyone has discussed or begun work upon an implementation of libGit2sharp that works with the major Git hosts via their API's. Obviously not all actions available in libGit2 will work with an API interface, but at least most read-only actions should be.
If this is an entirely new feature request - I'd like to get the opinion of someone with knowledge of the libGit2sharp codebase about how difficult such a feature request would be to implement.
Git only specifies the network protocol for fetching, pushing and creating an archive. Nothing else can be done via the Git protocol (and providers will likely disable the archive so they can leverage their existing caching solutions).
If this is an entirely new feature request - I'd like to get the opinion of someone with knowledge of the libGit2sharp codebase about how difficult such a feature request would be to implement.
This feature would be out of scope and impossible as Git does not provide a way to perform these tasks.
Once you're trying not to do Git, then you're out of the Git world into each provider's API. Trying to replicate Git operations and git commands on top of each provider's API is a whole project unto itself, and one which is likely to get you to hit these provider's API limits, as in-depth analysis of the repositories is not generally why they provide these services.
Not to mention that looking up each necessary object over HTTP would be extremely slow and you'd likely not gain anything over grabbing a Gigabyte or two from the network.
But if all you need is a few questions that can be easily answered from the APIs themselves (say, latest commit and its relationship to different branches), and you do need the logic in GitVersion, then you're probably better off making its history analysis pluggable so you can put in the data from your API lookups.
I'm not familiar with how GitVersion makes its decisions, but if it doesn't just want references and their relationships to each other and the tags, but rather it wants to look at the repositories themselves, and you do need it rather than just replicate some of its logic, I would recommend to download the repositories and perform all the analysis there. It'll be a much more efficient use of time to rent a bit of disk space from some provider than try to fit each individual provider's API into some idealised version of a git command where you then still need to figure out the edge cases of both the command and its API you're using.
Related
Currently I'm creating an open source project and I want people to participate in creating the backend code, a simple API. May sound newbie, but ive been told that I should never make public my backend code, for security reasons. Then how can people collaborate making the API if they can't make it public. They should secretly send me the code or there's no security issues by doing so.
Thank you very much.
No, it is not a security issue itself. There are lots of good well-secure open source backends and libs to write backends are published in open public repositories.
And the opposite, if you deploy non-secure backend (i.e. with bugs, old exploited libs, etc.), even the privacy of your source code will not help you. Public source code will make it faster to break into, but it does not make it less secure comparing with the private code.
With the release of Gitlab 11.7 in January 2019, we get the new key feature Publish releases for your projects. I want precisely what the screenshot on that page shows and I want to be able to download compiled binaries using the releases API.
I can do it manually. Of course, instructions for the manual approach can be found here on stack overflow. The problem I need help with is doing it as part of a CI/CD pipeline, which is not covered by the answers one can find easily.
The release notes contain a link to the documentation, which states:
we recommend doing this as one of the last steps in your CI/CD release pipeline.
From this I gather it's possible. However, the only approach I can imagine is using the GitLab API just as I do, when I create releases manually. When one wants to access the GitLab API one has essentially three options for authentication, according to the fine manual: OAUTH2 tokens, personal access tokens and session cookies. Consequently I would need a method for having either of these available in my CI/CD pipeline, with sufficient privileges. Solutions for this problem are an ongoing discussion with lots of contributions, but virtually no tangible progress in recent years.
So, how does one create releases as one of the last steps in one's CI/CD release pipeline?
Storing my personal access key with API access in a CI/CD variable or even a file in the repo is not an option for obvious reasons.
They've put up a blog post explaining how to do this:
https://about.gitlab.com/blog/2020/05/07/how-gitlab-automates-releases/
They've created a tool (gitlab-releaser) to help with this task. Basically you create a new step, where you use a docker image that provides this tool, and then call the tool with the proper parameters.
release_upload:
image: registry.gitlab.com/gitlab-org/release-cli:v0.1.0
script:
- gitlab-releaser create --name="My Release" --description="My Release description"
Does anyone know how to push tickets from one Trac instance to another?
The problem that I'am trying to solve is the following:
Our company is doing some development for big international firm (let's call it CompanyX) that has everything behind VPN's. We have our Trac, hosted at our firm, which we use for management of all our projects. CompanyX also uses Trac, and since the developers from CompanyX cannot use our Trac for tracking bugs, requests and issues, they use their own. The reason is that their security policy is very restrictive with no Internet access to our server, and nothing can be done about that.
The problem is that we are also forced to use THEIR Trac because they prefer to communicate everything through it internally, and they expect from us to conform to their workflow as well. And for that purpose we have to connect to their VPN via some IE java plugin client from Juniper (which does not remember passwords) and every time have to configure whatnot, just to see a ticket or two on weekly basis which is really tiresome.
Since the communication is mainly one directional, from client to our firm, with no real interaction, I was wondering is there an EASY way to just push (or even sync) the tickets and their updates from client's Trac to our Trac server which would satisfy their outsourced security provider?
(It is not possible for us to touch the Trac's source on their server, so by EASY, I mean some plugin or script or something similar which would be easily accepted by their admin)
There is TicketImportPlugin that can import tickets from csv or excel files. The opposite is to export tickets as csv or tsv files via link on the bottom of a ticket page.
TicketMoverPlugin is able to move tickets from one Trac instance to another.
You should be able to do something using the XMLRPC plugin. You can script up an application that queries your client's Trac for tickets (using ticket.query()), then grabs the ticket details (ticket.get()) and posts them to your Trac in a new ticket (ticket.create()).
This would require both Trac instances to install that plugin and create a trac account that has the XML_RPC permission. You will have to make sure that your client finds this acceptable. Since it's a pre-packaged plugin that you can enable on a per-account basis, it shouldn't disrupt their normal workflow very much.
Apache Ace documentation refers about RepositoryTool.jar that can be used to manage Repository. But I could not find this tool in the Apache ACE distribution. Where can I download this tool?
The page you're referring to is part of the old site (the new one is located at http://ace.apache.org), and refers to tooling you probably shouldn't be using anymore: it has been used before there were other ways to interact with the repository, mainly for development purposes.
Depending on your needs, you can use the repository in a number of ways,
If you need to programmatically read and write the repositories (remember that they're only XML), use the HTTP API available for that.
You can do the same thing from code, see Repository and its implementations.
If you want to edit 'meaningful' ACE data (such as linking distributions and targets), use the Client REST API. This is probably the option you want.
I know you use the C based networking API to do FTP communication but I'd prefer to use something a little higher level. I've seen a few Objective-C based wrappers but I'm not sure what to use. I don't need that complex of FTP interaction. Its just the typical create/delete dirs, upload/download files... What do you recommend?
Edit:
Here is one that looks promising but I can't get it to compile for the iPhone SDK
The ConnectionKit
This may help, but you may reconsider design for reasons stated by bbum:
http://code.google.com/p/s7ftprequest/
The reason why you can't find much in the way of useful FTP client software is because FTP isn't used much any more and is generally actively discouraged from use.
Without great care, it is quite easy to create big old security holes when using FTP (when I ran a consulting company, the 3 times we had infected machines were all because of FTP server security holes or exploits -- one time, the damned HP copier's FTP server was the attack vector!).
FTP is also inefficient unless carefully configured.
I would encourage you to use an HTTP based protocol. WebDAV allows you to do basically anything FTP can do, but does so over an HTTP channel. Thus, it'll work through proxies and the like. Heck -- HTTP has become so ubiquitous that pretty much everything works with HTTP.
And, of course, there are plenty of good HTTP client APIs built for Objective-C.
Obviously, if FTP is a requirement for your project, this answer won't help you much....
After not finding anything that works well I decided I'd go ahead and follow Apple's tutorial on how to do FTP. It sure is a PITA but at least it does work. I'm defiantly going to support WebDAV in the first revision my app, and eventually perhaps some other transfer methods later on. I think I'm going to consider releasing this open source after I get FTP & WebDAV working good, since there is no reason why you should have to do this much low level work to do such a basic and ambiguous task as FTP these days.
I've implemented FTP file download and upload, directory create and directory list download through the regular FTP possibilities in the iPhone SDK. Note: you'd be passing the login name and password as part of every FTP request unsecured. Apparently no apparent connection to the FTP server is maintained at the app level, like with a real FTP client app, that I haven't been able to find for iPhone yet. If you're interested in the source code please let me know through e-mail.
GoldRaccoon isn't mentioned and can be found on GitHub. I use that library and it works very well (besides it didn't support FTP rename)