Is it possible to move a Conan package from conan-center to a private registry on an air-gapped network?
I'm working with an air-gapped network and would like to "export" open source packages from conan-center and move them to my private remote on an air gapped network.
Related
Error Screenshot While Trying To Use npm install telegraf in termux shared Storage
I'm more familiar with the Python ecosystem at this point and have a question about how I can do something with npm that I'm used to doing with pip.
Let's say I have a wheel for a particular Python package, as well as a wheel file for each of the Python package's dependencies. And let's say I have all these wheel files in a folder called /path/to/wheel/files. To install this package and all of its dependencies, I could run something like pip install /path/to/wheel/files/*.whl --no-deps, where --no-deps keeps me from having to install the various dependencies in the proper order.
Does npm have an equivalent to this? I'm using npm-offline-packager to create a tarball that contains a Node package (as its own tarball) and all of its dependencies (as their own tarballs). I know I can tell npm install to install a particular tarball. However, when I do this, it tries pulling in the required dependencies from the online NPM registry instead of pulling in the dependencies from the tarballs I already have.
Ideally, I'd like npm install to use the tarballs to add the main package to my project's package.json while adding the package's dependencies to my project's package-lock.json. And of course, I'd also like the main package and all its dependencies to be installed to my project's node_modules directory as well.
TL;DR Does npm have something equivalent to pip install /path/to/wheel/files/*.whl --no-deps?
I'm responding to my own question here, but note that my answer is only applicable to my particular use case and may not be applicable in general.
For my use case, I have access to two computers: one that has access to the internet and one that doesn't. For the machine that doesn't have access to the internet, I was attempting to use Verdaccio as a way of creating a self-hosted NPM registry. However, publishing packages to Verdaccio wasn't working because it kept trying to pull in the package's dependencies from the public NPM repository. The solution was to remove all references to "npmjs" in Verdaccio's config file (which, for me, Verdaccio created at ~/.config/verdaccio/config.yaml).
So, in case anyone needs to do development on a machine that doesn't have access to the internet, the process for setting up Verdaccio looks something like this:
On the machine that has access to the internet, create an NPM project using npm init (I called my project "verdaccio_runner"). The reason I did this is because, without already having an NPM registry on the machine that doesn't have access to the internet, it was hard doing a global install of Verdaccio.
Run npm install verdaccio to install Verdaccio to the NPM project that was created in the previous step.
Transfer this project over to the machine that doesn't have access to the internet.
Once it's transferred over, run Verdaccio from the project like this: npx verdaccio.
Quit out of Verdaccio.
Remove all references to "npmjs" from the config file that Verdaccio created (again, mine was at ~/.config/verdaccio/config.yaml).
Run Verdaccio again to pull in those changes.
Tell NPM where your private registry is: npm config set registry http://localhost:4873/.
Add yourself as a user by running npm adduser and by then filling out the information you're prompted for.
And the process for publishing packages to Verdaccio on a machine that doesn't have access to the internet looks like this:
For the package you want to install, on the machine that has access to the internet, run npo fetch <package name> --no-cache (assuming you've already done a global install of npm-offline-packager on the machine that has internet access).
Bring the tarball that npo created for you over to the machine that doesn't have internet access.
Untar the tarball.
From the directory that's created, run for file in ./*.tgz; do npm publish $file; done.
The published packages can now be npm installed to projects on the machine that doesn't have internet access.
Note: in order for Verdaccio to be accessible to other machines on the private network, I also had to add the following to Verdaccio's config file:
listen:
0.0.0.0:4873
I have a Python project that uses Poetry for dependency management.
I want to install a Python module from a private registry.
I know I can add it to the pyproject.toml file in a given source like this:
[[tool.poetry.source]]
name = "private-pipy-registry"
url = "https://my-private-pipy-registry"
but this registry url differs whether my project is run in review or production. I could add both review and production sources to the .toml file but this is not a suitable solution either.
I tried to use the poetry.config command line to add a private source.
This works when I want to publish this Python package to a private registry:
poetry config repositories.my-repo "https://my-url/my-pypi/simple"
peotry publish --repository my-repo --build -u my-username -p my-password
but does not work when I want to install a dependency from a private registry as I first described:
poetry config repositories.my-repo "https://my-url/my-pypi/simple"
poetry config http-basic.repo my-username my-password
poetry update
It returns: Because my-new-lib depends on my-private-lib (0.1.2) which doesn’t match any versions, version solving failed.
Do you know if I am missing something, if it is a known issue and if there is any solution? Thanks.
EDIT: I solved it by unsetting any repositories that were listed in poetry config --list, running poetry config repositories.my-repo --unset and resetting these repositories config
I have a private repo on github with dependencies on other private github repos and npm install installs all the dependencies without problems.
I installed Yarn (on MacOS) and tried to run yarn in the command line, but it failed cloning the private dependencies with the following error:
ssh: Could not resolve hostname github: nodename nor servname provided, or not known
fatal: Could not read from remote repository.
How do I make yarn install the modules? Thanks!
Yarn#0.16.0 works by replacing the url of type
"private-test": "git+ssh://git#github.com:ramasilveyra/private-test.git#d6c5789"
with
"private-test": "git+ssh://git#github.com/ramasilveyra/private-test.git#d6c5789"
Replace : colon with / to make it work.
For now Yarn doesn't support installation from private package and private github repo
There is a workaround provided by Milos Ivanovic:
If you add the following to your ~/.ssh/config file:
Host github.com
User git
you can force all logins to github.com via SSH to use the user git by >default, and this makes yarn able to clone from
private repositories when using the ssh://github.com//
source format.
Fortunately this is just a matter of time before yarn support private repo on github, as there is already a pull request for it.
My node.js app uses a number of private shared modules hosted in git repos. I use git URLs like below in the dependencies block in package.json:
"xxx-accountMgr": "git+ssh://git#github.xxx.ibm.com:xxx/lib-account-mgr.git",
when "cf push" this errors during npm install on ssh:
npm ERR! git clone --template=/home/vcap/.npm/_git-remotes/_templates --mirror ssh://git#github.xxx.ibm.com/ipaas/lib-account-mgr.git /home/vcap/.npm/_git-remotes/ssh-git-github-xxx-ibm-com-xxx-lib-account-mgr-git-bf65c10c: ssh: Could not resolve hostname github.xxx.ibm.com: Name or service not known
i expected this as I haven't configured the ssh key in bluemix. is that possible? if not, what's the alternative to install modules from a private repo in bluemix?
If you are downloading a private module hosted on Git, you should be able to use https protocol (with creds) to access it.
There is a trick that could help avoid this issue if that is not an option for you:
1) Package private modules with your application (in node_modules)
2) Move the private modules to devDependencies in package.json, rather than dependencies, so that your local dev workflow is unaffected.
npm install will install dependencies and devDependencies. By
default, Bluemix will only install dependencies
Step 2 is necessary because even if you package the private node_modules with your application, staging for your application will fail because npm still tries to access your private repo to validate the dependency.
Also, if you had a .cfignore file ignoring the entire node_modules directory, that would have to be changed to ignore only the public modules.
If the private repo requires Github authentication to access the shared mdoules, Bluemix won't be able to access them. You can use a command such as git clone https://github.com/repo/etc but that will require the files to be accessible without authentication.
An alternative could be to manual install the files in your repo prior to using cf push so they are available. This is not a great solution but it will solve the problem in the short term.
The problem here is that Bluemix cannot reach back into the corporate network which is apparently where your github repo lives.
It has nothing to do with authentication, although what the others say here is accurate for publically accessible git repositories
This is a workaround what works for me is to use npmjs private modules. On one hand it will work and on the other hand it is really easy to manange versions, and reuse code. Of course you'll need to make some minor changes, but is totally worth it.
Upgrade your npmjs account to use private modules: https://www.npmjs.com/private-modules
On your computer log in to npmjs:
npm login
Publish your modules
Copy your npmrc file to your project:
cp ~/.npmrc /path/to/your/project
npm install your_module --save
Enjoy!
Be aware that if you ever change your password the token inside .npmrc will be revocated.
The token is not derived from your password password, but changing
your password will invalidate all tokens. The token will be valid
until the password is changed. You can also invalidate a single token
by logging out on a machine that is logged in with that token.
Source: https://docs.npmjs.com/private-modules/ci-server-config#checking-in-your-npmrc
You can also use cfnpm module (https://www.npmjs.com/package/cfnpm) it is designed to deal with private package in cliud foundry