We are working on a a system that uses cloud runner, where we have a tech like Spring + Gradle plus Mongo.
The system is containerized and runs on cloud-run in gcp. However, gcp has a hard limit of 2gb on container size, which we are trying to fit into, as of now.
Upon deeper investigation, I found that, the gradle wrapper that we use downloads at least 170mb extra than what we needed.
It includes following -
It contains documentation, which is not needed while running a build via wrapper.
It does not delete the zip file after extracting the same..
Together it counts to 270 mb, which quite big for us.. What I want to know is, Is there any wrapper configuration OOTM that will help me avoid these extra files being downloaded on our system?
It seems you used the Gradle distribution type "all", which includes source code and the Gradle documentation (e.g., for IDE support -- source).
Since you run the Gradle wrapper in the cloud, you probably do not require IDE support: Use distribution type "bin". At least in latest versions of Gradle (version 7) this is the default, but you can still be explicit to make sure:
# gradle wrapper --gradle-version 7.0.2 --distribution-type bin
The size difference is about 200 MB:
# du -hs ~/.gradle/wrapper/dists/gradle-7.0.2-{all,bin}
438M ~/.gradle/wrapper/dists/gradle-7.0.2-all
229M ~/.gradle/wrapper/dists/gradle-7.0.2-bin
Gradle still keeps the zip-file, so you will have to delete that manually.
For us, this what we went with.
Although we used gradle wrapper, We also made sure that we install relavent gradle wrapper upfront on the machine, so that when we run gradlew command, we dont download it again.
When you download gradle wrapper while building image, you have liberty to delete the file from the filesystem while building container image. So you can free up some space.
Next step is you can squash the image to remove all unnecessary layers. How can i remove layers from my docker image?
Related
When I performed a release build of Chromium in a Windows environment according to the following procedure, the file size was 48GB.
https://chromium.googlesource.com/chromium/src/+/main/docs/windows_build_instructions.md
However, the file installed using the Chrome installer is about 700MB.
Is there a build option to output the minimum required files for browsing?
The build options currently in use are:
gen gen out/Default --args="is_debug=false enable_nacl=false symbol_level=0 blink_symbol_level=0 is_component_build=true"
autoninja -C out/Default chrome
You are using is_component_build flag which will build many shared libraries to avoid the long linkage time, if it's not specified. You can just remove that flag, then it will be automatically set to false. That should minimize the number of files output during the compilation process.
Also your build folder includes debug symbols and other files thereby increasing the build folder's size to a few GBs. Generally even for highly optimized release build, it's greater than 20 GB and for debug build it's around 80-100 GB.
So, you should not be comparing the build folder size with the installation folder size. Also, mini_installer will greatly reduce the size of your Chromium fork while distributing your app. So you should consider making an installer for your Chromium fork for distribution to end-users.
Due to:
extensive usage of code auto-generation (notably for the .isa instruction and decoder sublanguage)
scons symlinking sources into the build tree, and in particular symlinking src/arch/ARCH to src/ARCH
it becomes very hard to setup Eclipse to work well with gem5.
Has anyone managed to achieve a good setup? Especially interested in Linux hosts.
I have achieved a very good setup with the following horrendous hack: define the Eclipse project inside the build/ARM directory itself!
This is done by creating a "New makefile project with existing code" in the build directory. You will also want to fix the C++ standard library as mentioned at: How to solve "Unresolved inclusion: <iostream>" in a C++ file in Eclipse CDT?
This works amazingly because the way the gem5 build system works as of May 2020, the build/ARM directory contains exactly the final compilation tree, including all the source symlinks and autogenerated code,.
This setup is not perfect because there is still some C++ stuff in build/ outside of ARM, e.g. ext, but 99% of the time we don't care about those, and I can perfectly navigate key autogenerated code such as instructions and decoder.
I then just build via the command line normally with scons.
Humongous autogenerated files like exec-ns.cc.inc turn on Eclipse's large file limited mode. Notably, if you want to jump to a definition, Ctrl + click does not work for me, so I just copy the symbol of interest and Ctrl + Shift + T to go to its definition.
I don't usually bother to try GDB step debug gem5 through Eclipse and use it mostly for code navigation, since GDB Dashboard tends to work well enough for me, and I often need to do new log collection runs and I sometimes use reverse debugging when things get serious.
But I have tested step debugging through Eclipse, and it did work normally (no reason not to I guess), you just have to set it up in exactly the same was as for any other C++ application with a Makefile, i.e. basically tell eclipse the Binary name and the desired CLI on the debug configurations.
You have of course as usual to choose between a gem5.debug debug build or a gem5.opt build, where the .debug build is about 10-20x slower but gives greater debug visibility. In general, if the time to reach the point of interest in a debug build starts to annoy you however, what you tend to really want is to do use reverse debugging.
This is also mentioned at: https://cirosantilli.com/linux-kernel-module-cheat/#gem5-eclipse-configuration
Tested in Eclipse 2020-03.
I made changes to Intellij Community Edition (ce). I can compile and run those changes from within the IntelliJ editor. That launches a second instance of IntelliJ ce which is running from classes containing my changes. What I want to do is just run those changes without having to first load the source, compile and run from within IntelliJ.
Netbeans made this easy by just producing an executable as a result of the build. With Intellij, it's not at all clear what has to be done. I have tried the following-
using the Run configuration Intellij itself uses to run the altered classes- this includes setting the working directory , main class, vm options and classpath. Actually, this doesn't work for reasons unknown to me.
on someone's suggestion, running dist.gant in build. This blows up with very many errors which are not helpful (no class def found errors which indicate some confusion on Intellij's part on classpaths somewhere)
Running WinLauncher.exe under bin gives the error message that it can't find VM options file (although it's in bin, (and also for good measure under bin/win with the other files which are co-located with vmoptions in the intellij directory structure for Intellij proper. )
ALl this is just harder than it should be. The solution is to provide an executable as a result of the build and place it in a predictable location.
Has anyone ever actually DONE what I am trying to do- make changes to the community source then use the resultant editor not as a project you're working with in IntelliJ but as the Intellij editor you're working through?
FOLLOW UP
User60561 had the correct answer. Just to mop up the details, in artifacts, there is a compressed file (win.zip for Windows, mac.zip for Mac etc.). In order to run your snapshot, you have to unzip this archive (after which it will have the same name, minus the zip extension) then go into folder "bin". There you'll see two executables: idea.exe and idea64.exe, for 32 and 64 bit versions, respectively. Clicking on these runs your snapshot.
Adjusting contents of the files idea.exe.vmoptions and idea64.exe.vmoptions lets you set the VM parameters to suit yourself, typically people might want to give the VM more memory through the -Xmx value.
It seems straightforward:
To build the distribution archive of IntelliJ IDEA Community Edition, execute build.xml Ant build script in the root directory of the source code. The results of the build execution can be found at out/artifacts.
https://github.com/JetBrains/intellij-community#building
So download ant, and run ant in the directory that you have it stored in. Make sure to use the commandline to launch ant in order to make sure everything is working correctly.
Initially execute getPlugins.bat/sh, then:
Use update.bat/sh according to it's instructions
Or
Click on: Main Menu | Build | IntelliJ IDEA CE build
Copy content of intellij-community\out\deploy (lib, plugins folder) into existing IJ installation (sometimes it is better to delete existing folders if they contain older dependencies or when the installation was of Ultimate version)
We bundle the Java 6 JRE with our application installer so that it can be run on any machine, but this makes the application a little bit heavier. So we are planning to reduce the size of the JRE. If anyone has done this sort of task, can you please provide guidance to move forward with this?
Look at the README file in the JRE directory. The 'Optional Files and Directories' section lists a number of files that can be removed from the Oracle/Sun JRE if you are packaging it with your application.
I use an Ant buildfile to copy the JRE from the system install location to the package directory when creating an installation. Put the list of files you want excluded in a separate file and use the 'excludesfile' attribute to load this list:
<copy todir="${deployed_jre_dir}">
<fileset dir="${system_jre_dir}" excludesfile="jre_excludes.properties"
</copy>
Sample jre_excludes.properties file:
# per the README from the JRE, these files are for the browser plugin and are not needed otherwise
#bin/javaw.exe
bin/javaws.exe
bin/javacpl.exe
bin/jucheck.exe
bin/jusched.exe
bin/wsdetect.dll
bin/NPJPI*.dll
bin/NPJava*
bin/NPOJI610.dll
bin/RegUtils.dll
bin/axbridge.dll
bin/deploy.dll
bin/jpicom.dll
bin/javacpl.cpl
bin/jpiexp.dll
bin/jpinscp.dll
bin/jpioji.dll
bin/jpishare.dll
lib/deploy.jar
lib/plugin.jar
lib/javaws.jar
lib/javaws/messages*
lib/javaws/miniSplash.jpg
bin/new_plugin**
bin/jureg*
bin/ssv*
bin/jqs*
bin/jp2*
lib/deploy/**/*
# if you do not need any RMI stuff
# wildcard to catch .exe files on Windows
# note rmi.dll is not excluded, which is needed by jconsole; add rmi.dll if you do not need jsonsole
bin/jbroker*
bin/java-rmi*
bin/rmid*
bin/rmiregistry*
bin/tnameserv*
bin/orbd*
bin/servertool*
# do not include QuickTime
# this will be in the jre dir for machines that have QT installed
lib/ext/QTJava.zip
Some update info: since java 8 there is an official Oracle tool called jrecreate for creating of small embedded JRE packages.
For my Java 8 Update 144 desktop application I exclude the 2 big Java FX files:
bin/jfxwebkit.dll // ~34 MB unpacked
lib/ext/jfxrt.jar // ~17 MB unpacked
The zipped jre is 49 MB instead of 66 MB.
For me this is an acceptable tradeoff between reduced size and added build complexity (and potential bugs).
You're trying to reduce a standard JRE's size? Don't do that. You can choose to bundle an alternative JRE which might be smaller. A list can be found on this Wikipedia page. As always, beware of compatibility issues and test your application thoroughly.
An other, and safer, way is to just require an installation of a JRE on the target machine.
You can use jlink tool to post-process JDK and create a smaller image by keeping only a specified set of JMPS modules and debug information. This is a common practice right now as we are using more containerized environments.
I'm using PMEase QuickBuild to perform automated builds of our Maven2 projects and a nightly sanity test to ensure nothing is broken.
The test needs to untar packages which are created by the automated Maven2 projects. The problem is that the package names change frequently due to project versions being incremented all the time.
Does anyone know how I can configure QuickBuild to pick up the version (ideally from the POM file of the individual components), if this is possible at all?
I don't know if this is an option for you but it looks like you can do it the other way around. Quoting Build with Maven:
Control build version
If you want to control the build
version from QuickBuild side, please
follow below steps:
Change the POM file and define the project version as
${buildVersion}. Do not forget to
commit the file into your SCM after
change.
Define a build property like below when define the Maven build
step:
buildVersion=${build.version}
There are maybe other options but I must admit that my knowledge (zero) of QuickBuild is very limited
I created a work around to this issue by having QuickBuild execute a shell script which did the untarring by using wildcards, similar to the following (to avoid computing the exact version):
tar xzf filename-*.tar.gz
I couldn't figure out how to do this in QuickBuild, so I offloaded the work to the shell script.