Is it possible to include a file outside Python package in wheel/bdist/sdist? - setup.py

I am the author of KiKit. KiKit is a tool using KiCAD's Python API. KiKit also serves as a plugin for KiCAD. To register a plugin for KiCAD, you have to place a Python file in a specific location (e.g., /usr/share/kicad/scripting/plugins/ or ~/.kicad_plugins/ - see KiCAD doc for details).
I distribute KiKit as a python package via pip so my users can just type pip install kikit and not care about anything else. I would like to be able to register the plugin within the installation step so the users don't have to perform this step manually.
I know that wheel does not support arbitrary code execution thus no post-installation script is possible. But I wasn't able to find out if it is possible to include a file that will be installed on a location outside the root of the package. If so, how can I specify it in setup.py?
PS: I am aware of the option to provide an extra script, e.g., register-kikit, which would register the plugin. But I would like to include it in the installation step so there is a lesser chance of the user forgetting to do so.

Related

How does rpmbuild compute which fields to generate?

When comparing different RPM files, I've noticed that not all of them expose the same header tags. So there must be some logic that activates/deactivates creation of some of them.
One example is the build time and host. I've stumbled upon two RPM specs. Neither mentions anything that looks at all like a specification or switch to provide the information. Still, one of them is generated with Build Time and Build Host fields, the other isn't (I am not permitted to post either one).
I am aware of the new _buildhost macro. The RPM version used to generate both is insufficient to use it. Both packages get created from a list of Sources, as far as I can see. The one that doesn't display the build information gets built using CMake/CPack, the other uses rpmbuild directly, that's the only information I have about serious difference.
Both are defined as Group: AddOn. So far, I haven't found any remotely definite resources about what groups are valid, or their meanings. Only thing I found was the list of deprecated groups in Fedora. I'd be more interested in a list of supported ones, but wasn't successful so far.
Resources I've found until now (omitting the pointless ones):
Max RPM Package Building Page, RedHat blog-ish tutorial, The RPM build guide, The actual RPM tags documentation, The RPM packaging guide
Unfortunately, none of the above provide the information I'm looking for.
"Give a man a fish" question: How can I suppress creation of Build Time or Build Host in rpm 4.11, be it in spec syntax or in usage of rpmbuild?
"Teach a man how to fish" question: Is there any documentation about what header tags get created with which settings?
You can use Mock for building rpm (recommended anyway). and use config_opts['hostname'] = 'my.own.hostname'.
Mock will call sethostname() in the chroot.
This is the only way how to do it AFAIK.
rpmbuild should honor SOURCE_DATE_EPOCH - but I never used it.
You can set environment variable using:
config_opts['environment']['SOURCE_DATE_EPOCH'] = 'foo'

Why are the source file names not human readable?

I installed Perl6 with rakudobrew and wanded to browse the installed files to see a list of hex-filenames in ~/.rakudobrew/moar-2018.08/install/share/perl6/site/sources as well as ~/.rakudobrew/moar-2018.08/install/share/perl6/sources/.
E.g.
> ls ~/.rakudobrew/moar-2018.08/install/share/perl6/sources/
09A0291155A88760B69483D7F27D1FBD8A131A35 AAC61C0EC6F88780427830443A057030CAA33846
24DD121B5B4774C04A7084827BFAD92199756E03 C57EBB9F7A3922A4DA48EE8FCF34A4DC55942942
2ACCA56EF5582D3ED623105F00BD76D7449263F7 C712FE6969F786C9380D643DF17E85D06868219E
51E302443A2C8FF185ABC10CA1E5520EFEE885A1 FBA542C3C62C08EB82C1F4D25BE7B4696F41B923
522BE83A1D821D8844E8579B32BA04966BAB7B87 FE7156F9200E802D3DB8FA628CF91AD6B020539B
5DD1D8B49C838828E13504545C427D3D157E56EC
The files contain the source of packages but this does not feel very accessible. What is the rational for that?
In Perl 6, the mechanism for loading modules and caching their compilations is pluggable. Rakudo Perl 6 comes with two main mechanisms for this.
One is a file-system based repository, and it's used with things like -Ilib. This resolves modules simply using paths on disk. Whenever a module loaded, it first has to check that the modules sources have not changed in order to re-compile them if so. This is ideal for development, however such checks take time. Furthermore, this doesn't allow for having multiple versions of the same module available and picking the one matching the specification in the use statement. Again, ideal for development, when you just want it to use your latest changes, but less so for installation of modules from the ecosystem.
The other is an installation repository. Here, specific versions of modules are installed and precompiled. It is expected that all interactions with such a repository will be done through the API or tools using the API (for example, zef locate Some::Module). It's assumed that once a specific version of a module has been installed, then it is immutable. Thus, no checks need to be done against source, and it can go straight to loaded the compiled version of the module.
Thus, the installation repository is not intended for direct human consumption. The SHA-1s are primarily an implementation convenience; an alternative scheme could have been used in return for a bit more effort (and may well be used in the future). However, the SHA-1s do also create the appearance of something that wasn't intended for direct manipulation - which is indeed the case: editing a source file in there will have no effect in the immediate, and probably confusing effects next time the compiler is upgraded to a new version.

How do you make it so that cpack doesn't add required libraries to an RPM?

I'm trying to convert our build system at work over to cmake and have run into an interesting problem with the RPMs that it generates (via cpack): It automatically adds all of the dependencies that it thinks your RPM has to its list of required libraries.
In general, that's great, but in my case, it's catastrophic. Unfortunately, the development packages that we build end up getting installed with one our home-grown tool that uses rpm to install them in a separate RPM database from the system one. It's stupid, but I can't change it. What this means is that all of the system libraries that any normal library will rely on (like libc or libpthread) aren't in the RPM database that is being used with our development packages. So, if an RPM for one of our development packages lists system libraries as being required, then we can't install it, as rpm will think that they're not installed (since they're listed in the normal database rather than the one that it's being told to use when installing our packages). Our current build stuff handles this just fine, because it doesn't list any system libraries as dependencies in the RPMs, but cpack automatically populates the RPM's list of required libraries and puts the system libraries in there. I need a way to stop it from doing so.
I tried setting CPACK_RPM_PACKAGE_REQUIRES to "", but that has no effect. The RPM cpack generates still ends up with the system libraries listed as being required. All I can think of doing at this point is to copy the RPM cpack generator and hack it up to do what I want and use that instead of the standard one, but I'd prefer to avoid that. Does anyone have any idea how I could get cpack to stop populating the RPM with required libraries?
See bottom of
http://www.rpm.org/max-rpm/s1-rpm-depend-auto-depend.html
The autoreqprov Tag — Disable Automatic Dependency Processing
There may be times when RPM's automatic dependency processing is not desired. In these cases, the autoreqprov tag may be used to disable it. This tag takes a yes/no or 0/1 value. For example, to disable automatic dependency processing, the following line may be used:
AutoReqProv: no
EDIT:
In order to set this in cmake, you need to do set(CPACK_RPM_PACKAGE_AUTOREQPROV " no"). The extra space seems to be required in front of (or behind) the no in order for it to work. It seems that the RPM module for cpack has a bug which makes it so that it won't let you set some its variables to anything shorter than 3 characters long.
To add to Mark Lakata's answer above, there's a snapshot of the "Maximum RPM" doc
http://www.rpm.org/max-rpm-snapshot/s1-rpm-depend-auto-depend.html
that also adds:
The autoreq and autoprov tags can be used to disable automatic processing of requirements or "provides" only, respectively.
And at least with my version of CPackRPM, there seems to be similar variables you can set e.g.
set(CPACK_RPM_PACKAGE_AUTOREQ " no")
to only disable the automatic dependency processing of 'Requires'.

Creating an rpm package from source tarball - but tarball contains no spec file, just an install script

During my time as a SysAdmin, I have encountered applications that provide no rpm packages to install on redhat-based distributions - only a source tarball. The source tarball does not provide a spec file which would simplify the process of rpm package creation. Instead, the source tarball only provides a bash/ksh/ script which must be executed as root to install the application on the system.
I have tried to create an rpm package which essentially runs the install script to perform the install script. I have also tried to do the right thing by attempting to rpmbuild the package as a non-root user, and modifying the install script as best i can to ensure that the script's install dirs refer to rpm enviromentals/macros. But with a complicated install script that is a few hundred lines long, which also calls other scripts into play... well, I was bound to fail in this endeavour.
Is there a better way to package up such .spec-less source tarballs? Would a better solution be to:
somehow take a snapshot of the system before the application install
install the source tarball with the provided install script
take a snapshot of the system after the installation, and determine the changes/additions made by the installation
put the list of changes/additions in a spec file, and create an rpm package this way?
Any useful/relevant/instructive/amusing/profound input and advice to this problem will be much appreciated.
Thank you in advance
I think the best solution is to request the upstream to provide a spec file for the applications. Another way could be request experience package maintainer to package the application. Or you can probably explore to checkinstall as it tracked the install file using Makefile. If you want to package yourself, you should read this link as it provides explanation and with many examples.
I've seen many scripts like you mention, and I suspect I can tell you with certainty the company selling you these horrible installs.
Your best bet is to request a proper installable package. Read up on why a package is better than a configure;make;make-install and is better than this install.sh nonsense. Armed with that, even if it's talking points, try and get them into the 3rd millennium . It will be hard, but ultimately most rewarding.
Barring that, you will need that package built (and by now you will have read why). Unfortunately, there are some apps and vendors whose payload can't be packaged, because they compile, query the destination host, look for a license, compile a bit more, etc. My bias here is well-honed.

QuickBuild: How can I create a builder to open a tarball package (tar.gz) whose name will change with each version?

I'm using PMEase QuickBuild to perform automated builds of our Maven2 projects and a nightly sanity test to ensure nothing is broken.
The test needs to untar packages which are created by the automated Maven2 projects. The problem is that the package names change frequently due to project versions being incremented all the time.
Does anyone know how I can configure QuickBuild to pick up the version (ideally from the POM file of the individual components), if this is possible at all?
I don't know if this is an option for you but it looks like you can do it the other way around. Quoting Build with Maven:
Control build version
If you want to control the build
version from QuickBuild side, please
follow below steps:
Change the POM file and define the project version as
${buildVersion}. Do not forget to
commit the file into your SCM after
change.
Define a build property like below when define the Maven build
step:
buildVersion=${build.version}
There are maybe other options but I must admit that my knowledge (zero) of QuickBuild is very limited
I created a work around to this issue by having QuickBuild execute a shell script which did the untarring by using wildcards, similar to the following (to avoid computing the exact version):
tar xzf filename-*.tar.gz
I couldn't figure out how to do this in QuickBuild, so I offloaded the work to the shell script.