I am trying to find a way to enable incremental compilation with CMake through a toolchain upgrade. Here is the problematic scenario :
Branch main uses g++-9 (using CMAKE_CXX_COMPILER=g++-9)
A new branch uses g++-10 (using CMAKE_CXX_COMPILER=g++-10)
Commits are happening on both branches
Incremental builds on one branch work fine
Switching to the other branch and explicitly invoking CMake fails
My question is the following : I'm looking for the proper way to make the invocation of CMake succeed and rebuild all the project from scratch when a toolchain change happens.
Here is a script that will make it quick and easy to reproduce the problem. This script requires Docker. It will create folders Sources and Build at the location where it is executed to avoid littering your filesystem. It then creates Dockerfiles to build docker containers with both g++ and cmake. It then creates a dummy Hello World C++ CMake project. Finally, it creates a folder for build artifacts and then executes the build with g++-9 and then g++-10. The second build fails because CMake generates an error.
#!/bin/bash
set -e
mkdir -p Sources
mkdir -p Build
# Creates a script that will be executed inside the docker container to perform builds
cat << EOF > Sources/Compile.sh
cd /Build \
&& cmake /Sources \
&& make \
&& ./IncrementalBuild
EOF
# Creates a Dockerfile that will be used to have both gcc-9 and cmake
cat << EOF > Sources/Dockerfile-gcc9
FROM gcc:9
RUN apt-get update && apt-get install -y cmake
RUN ln -s /usr/local/bin/g++ /usr/local/bin/g++-9
ADD Compile.sh /Compile.sh
RUN chmod +x /Compile.sh
ENTRYPOINT /Compile.sh
EOF
# Creates a Dockerfile that will be used to have both gcc-10 and cmake
cat << EOF > Sources/Dockerfile-gcc10
FROM gcc:10
RUN apt-get update && apt-get install -y cmake
RUN ln -s /usr/local/bin/g++ /usr/local/bin/g++-10
ADD Compile.sh /Compile.sh
RUN chmod +x /Compile.sh
ENTRYPOINT /Compile.sh
EOF
# Creates a dummy C++ program that will be compiled
cat << EOF > Sources/main.cpp
#include <iostream>
int main()
{
std::cout << "Hello World!\n";
}
EOF
# Creates CMakeLists.txt that will be used to compile the dummy C++ program
cat << EOF > Sources/CMakeLists.txt
cmake_minimum_required(VERSION 3.9)
project(IncrementalBuild CXX)
add_executable(IncrementalBuild main.cpp)
set_target_properties(IncrementalBuild PROPERTIES CXX_STANDARD 17)
EOF
# Build the docker images with both Dockerfiles created earlier
docker build -t cmake-gcc:9 -f Sources/Dockerfile-gcc9 Sources
docker build -t cmake-gcc:10 -f Sources/Dockerfile-gcc10 Sources
# Run a build with g++-9
echo ""
echo "### Compiling with g++-9 and then running the result..."
docker run --rm --user $(id -u):$(id -g) -v $(pwd)/Sources:/Sources -v $(pwd)/Build:/Build -e CXX=g++-9 cmake-gcc:9
echo ""
# Run a build with g++-10
echo "### Compiling with g++-10 and then running the result..."
docker run --rm --user $(id -u):$(id -g) -v $(pwd)/Sources:/Sources -v $(pwd)/Build:/Build -e CXX=g++-10 cmake-gcc:10
echo ""
# Print success if we reach this point
echo "SUCCESS!"
I'm looking for the proper way to make the invocation of CMake succeed and rebuild all the project from scratch when a toolchain change happens.
The proper way is to use a fresh binary directory. Either remove the binary directory when changing and let it recreate or just use a separate different directory for each toolchain.
Use Build/gcc10 binary directory for gcc10 build and Build/gcc9 for gcc9 builds.
No need to cd Build and mkdir with nowadays cmake - use cmake -S. -BBuild. Also do not use make - prefer cmake --build Build to let you switch generator later.
"If you change the toolchain, you should start with a fresh build. There are too many things that assume the toolchain doesn’t change and while you may be able to find workarounds which appear to work, I recommend you always use a fresh build tree for a different toolchain. This same logic also applies if you update the existing toolchain in-place (e.g. you update to a newer version of GCC on Linux, a newer version of Xcode on macOS, etc.). CMake queries compiler capabilities and caches the results. If you change the toolchain in a way that CMake can’t catch, then you end up with stale cached capabilities being used for the new/updated toolchain. Please don’t do that." - Craig Scott
So essentially I don't think it's possible. You just need to blow away your build. The best thing you can do is alert users if CMake isn't doing it for you.
Perhaps reply on this also:
https://discourse.cmake.org/t/how-to-change-toolchain-without-breaking-developer-workflows/1166
Or start another discourse.
I want to setup the ct-ng for my gui application and now I want to use wxwidgets.
For setting up the crosstool, I have used:
# Install prerequisites:
apt-get -y install gcc gperf bison flex gawk libtool automake libncurses5-dev texinfo
# Setup toolchain
# instructions from https://github.com/crosstool-ng/crosstool-ng
cd toolchain/crosstool-ng
./bootstrap
./configure --prefix=$HOME/.local
make && make install
echo -ne "\n\nif [ -d \"$HOME/.local/bin\" ]; then\n PATH=\"$HOME/.local/bin:$PATH\"\nfi" >> ~/.profile
source ~/.profile
mkdir ../tc/
cd ../tc/
ct-ng list-samples
ct-ng x86_64-w64-mingw32
ct-ng build # lasts 30 minutes...
##################### WxWidgets ######################
cd ../wxWidgets/
sh autogen.sh
./configure --prefix="$HOME/prefix" --enable-static --disable-shared --build=x86_64-w64-mingw32 --enable-unicode --without-libtiff --without-libjpeg --with-expat=builtin --with-libpng=builtin
make
The only way I have found is to clone wxwidgets from github and compile it as above in the script. Then, I included as path -I
WXWIDGET=../toolchain/wxWidgets/include/
$(CXX) -I$(FLEX) -I$(WXWIDGET) $(WXWIDGETSFLAGS) $(CPPFLAGS) $(header) $(src) $(obj3) -o $(OUTPUT)/$(bin)
Hundreds of errors appearing while compiling:
In file included from ../toolchain/wxWidgets/include/wx/platform.h:485:0,
from ../toolchain/wxWidgets/include/wx/defs.h:20,
from ../toolchain/wxWidgets/include/wx/string.h:24,
from ../toolchain/wxWidgets/include/wx/artprov.h:14,
from parser/include/gui.h:17,
from parser/include/customdialogs.h:17:
../toolchain/wxWidgets/include/wx/chkconf.h:282:9: error: #error "wxUSE_SECRETSTORE must be defined, please read comment near the top of this file."
# error "wxUSE_SECRETSTORE must be defined, please read comment near the top of this file."
What should I do?
You need to try "--host" and "--target" configure options.
Just try "../configure --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --disable-shared --enable-unicode".
BTW, "--enable-unicode" should be turned on by default. So you can drop it.
Also, if you software required C++11, you should compile the library as:
CXXFLAGS="-std=c++11" ../configure --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --disable-shared --enable-unicode
I just created a docker container and tried to install SQL Relay inside it.
I've checked the prerequisites here and followed the installation documents here.
However, at the end of make install of sqlrelay, I saw an error like this:
update-rc.d: /etc/init.d/sqlrelay: file does not exist
update-rc.d: /etc/init.d/sqlrcachemanager: file does not exist
make[1]: *** [install] Error 1
make[1]: Leaving directory `/sqlrelay-0.66.0/init'
make: *** [install-init] Error 2
What might be wrong with my installation?
Here's the docker file I used to start my installation:
FROM ubuntu:trusty
RUN apt-get update && \
apt-get install libxml2-dev libpcre3 libpcre3-dev libmysqld-dev -y
RUN apt-get install mysql-server libmysqlclient-dev -y
# sql relay prerequisites
RUN apt-get install g++ make perl php5-dev python-dev ruby-dev \
tcl-dev openjdk-7-jdk erlang-dev nodejs-dev node-gyp mono-devel \
libmariadbclient-dev libpq-dev firebird-dev libfbclient2 libsqlite3-dev \
unixodbc-dev freetds-dev mdbtools-dev -y
COPY rudiments-0.56.0.tar.gz /
COPY sqlrelay-0.66.0.tar.gz /
EXPOSE 80
Here are the outputs of ./configure, make, and make install inside sqlrelay-0.66.0 folder:
configure_log
make_log
make_install_log
If you need more information of my installation process, just let me know. I can provide it.
I think you should use ADD instead of COPY in your lines such as
COPY rudiments-0.56.0.tar.gz /
Your COPY just copies the .tar.gz, but does not unpack them
as with ADD
If the <src> parameter of ADD is an archive in a recognised compression format, it will be unpacked
This is extracted from
What is the difference between the `COPY` and `ADD` commands in a Dockerfile?
I have recently hit the same issue. The issue I found was that the init Makefile was incorrectly detecting the use of systemctl on Ubuntu Trusty and putting the scripts there. Later on the script would try to find the scripts in init.d and fail.
The solution is to edit the Makefile: sqlrelay-X.X.X/init/Makefile
Replace:
install:
if ( test -d "/lib/systemd/system" ); \
With:
install:
if ( test -d "/lib/systemd/system_x" ); \
Make a similar change to the uninstall option later in the script and it will now correctly install on Ubuntu.
I wrote myself a litte script to install opencv under ubuntu14.04. Can I remove the directory 3party after the make install sorted the lib into system directories or are there dependencies? (Remove not only the MYBUILD but the complete 3party)
echo "\nInstall OpenCV?...<any key>\n"
read inp1; # $inp1
mkdir 3party;
cd 3party;
git clone https://github.com/Itseez/opencv.git
cd opencv;
mkdir MYBUILD;
cd MYBUILD;
#sudo mkdir -p /usr/local/lib/opencv;
cmake -L -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=/usr/local .. ;
echo"check if path is ok?...<any key> or abort";
read inp1; # $inp1
make;
#sudo mkdir -p /usr/local/lib/opencv;
make install;
cd ../../..;
chmod -R 777 3party;
echo "\nDone.\nPlease exit...<any key>";
EDIT: I did tag it cmake because the configuration step is performed with this build tool. Also the tutorial on the OpenCV website stated it. Please correct me if wrong.
Building OpenCV from Source Using CMake, Using the Command Line
Normally, after installation of any package its source and binary directories can be safetly removed. OpenCV follows this convention too.
I am trying to install Apache HTTP server locally in my box as a regular user (non-root).
I have downloaded Apache 2.4.1 version of Apache HTTP server [http://httpd.apache.org/download.cgi]. However when I am trying to build and install locally in my box I am getting below error:
httpd/httpd-2.4.1 1059> ./configure
checking for chosen layout... Apache
checking for working mkdir -p... yes
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
Configuring Apache Portable Runtime library ...
checking for APR... no
configure: error: APR not found. Please read the documentation.
I am not sure what dependency it is looking for - I mean the download package does not contains it? What I need to do to build / deploy Apache HTTP server?
When it tells you Please read documentation it means that you should go read Apache documentation ( http://httpd.apache.org/docs/2.4/install.html ) which tells you to
download the latest versions of both APR and APR-Util from Apache APR,
unpack them into ./srclib/apr and ./srclib/apr-util (be sure the
domain names do not have version numbers; for example, the APR
distribution must be under ./srclib/apr/)
then do
./configure --with-included-apr
For Ubuntu 11.10 this option seems to be working well:
# APR
wget http://mirrors.axint.net/apache//apr/apr-1.4.6.tar.gz
tar -xvzf apr-1.4.6.tar.gz
cd apr-1.4.6/
./configure
make
make install
cd ..
# APR Utils
wget http://mirrors.axint.net/apache//apr/apr-util-1.4.1.tar.gz
tar -xvzf apr-util-1.4.1.tar.gz
cd apr-util-1.4.1
./configure --with-apr=/usr/local/apr
make
make install
cd ..
# Apache
wget http://apache.petsads.us//httpd/httpd-2.4.1.tar.gz
tar -xvzf httpd-2.4.1.tar.gz
cd httpd-2.4.1
./configure --enable-file-cache --enable-cache --enable-disk-cache --enable-mem-cache --enable-deflate --enable-expires --enable-headers --enable-usertrack --enable-ssl --enable-cgi --enable-vhost-alias --enable-rewrite --enable-so --with-apr=/usr/local/apr/
make
make install
cd ..
You can find more about it below
Source: VaporCreations.com
If you have Debian/Ubuntu you can just:
apt-get install libapr1-dev libaprutil1-dev
Then ./configure
Done
Here are the steps of how I installed apache-httpd on a non-root users:
Download and extract apache-httpd-2.4.2 (but before ./configuring,
making and installing it, follow the steps below:)
Download and extract APR & APR-UTIL into "./srclib/apr" &
"./srclib/apr-util" folders. This requires ./configure
--with-apr=./apache/httpd-2.4.2/srclib/apr (and) --with-included-apr (options).
Download, extract, ./configure (with) --prefix=localURL, make and
make install PCRE into "./pcre" folder. This requires ./configure
--with-pcre=/home/username/apache/pcre (option).
Configure apache-httpd by entring following command (I like to
enable certain options as written in the command below):
./configure --enable-file-cache --enable-cache --enable-disk-cache
--enable-mem-cache --enable-deflate --enable-expires --enable-headers --enable-usertrack --enable-cgi --enable-vhost-alias --enable-rewrite --enable-so --with-apr=/home/username/apache/httpd-2.4.2/srclib/apr --prefix=/home/username/apache/httpd-2.4.2/ --with-included-apr --with-pcre=/home/username/apache/pcre
Note: When configuring apache-httpd, use option "--enable-ssl" ONLY if OpenSSL is installed otherwise DON'T enable it.
Now on the command-line, enter 'make' and 'make install' command.
Open and configure the 'httpd.conf' file e.g.:
"vi /home/eddie_kumar/apache/httpd-2.4.2/conf/httpd.conf"
IMPORTANT: don't forget to change the default port from 80 to something else e.g. 8080, this is especially important for non-root
user. (How to? open httpd.conf -> search "Listen 80" -> change it
to "Listen 8080".
And that's it, now open your browser enter "localhost:8080", it should display "It works!".
If you are using fedora, you can use yum to install APR, APR-Util and PCRE. You'll also need to download apr-devel, apr-util-devel and pcre-devel.
That being said, you can just run the following command on your terminal and no more "configure: error: APR not found.. ..APR-Util and PCRE" errors.
yum -y install arp apr-devel apr-util apr-util-devel pcre pcre-devel
I'm using fedora 17 and planning on using a shell script to setup apache 2.4.3. So yum works pretty slick instead of manually downloading apr, apr-util, and pcre.
1, You need APR (apache portable runtime), which is core component of apache web server
2, If you wnat to do make install, you may need root account
3, Even if not, apache can not start to listen on unprivileged port (lower then 1024) without root account
4, gain root or ask someone with root to install apache from official repo(I don;t know which distro you run) like using yum, apt-get, etc...
Apr or pcre related errors require the source to be downloaded and Apache HTTPD "configure" process needs to be made aware of these source locations on your file system. For instance: if you downloaded the source for APR at ./srclib (relative to apache httpd) then you would use
--with-included-apr
as the configure option.
On the other hand if you want to not build but install APR / APR-UTIL, then you need the following on CentOS / RedHat:
yum install apr-util-devel apr-devel
However it might so happen that the APR version provided by yum does not match what is expected by this version of Apache httpd. In that case you could download APR and APR-UTIL and use the --with-included-apr option.
You could also build PCRE utilizing the same "configure, make, make install" process and then continue where you left off building Apache httpd.
Or you could install pcre:
yum install pcre-devel
If while building PCRE: you see "compile: unrecognized option" then perhaps you would need other dependencies as well: Please see the details at:
http://khanna111.com/wordPressBlog/2012/09/11/94087-2/
It also covers "mod_deflate" and "zlib" as well.
Basic steps
tar -xvf httpd-2.4.1-customized.tar -C ../
#Balancer folder will be created
tar -xvzf openssl-1.0.1.tar.gz -C /balancer/
cd ->/balancer/openssl-1.0.1
./config --prefix=/usr/local/ssl/ shared zlib-dynamic enable-camellia
make depend
make
make install
tar -xvzf pcre-8.30.tar.gz -C ../balancer/
/balancer/pcre-8.30
./configure --prefix=/usr/local/pcre/
make
make install
remove pcre and openssl
Installing and compiling the Apache server on Ubuntu machine
step 1:Install the Java JDK
sudo apt install openjdk-11-jdk
sudo gedit /etc/environment
JAVA_HOME="/usr/lib/jvm/openjdk-11"
source /etc/environment
echo $JAVA_HOME
verify the java version
javac --version
**** install Other required packages:**
sudo apt-get install apache2-dev -y
sudo apt-get install libpcre3 libpcre3-dev
Step 2 :To Install Apache HTTP Server
#1 For ubuntu ,install development tools including the C compiler:
sudo apt-get install build-essential checkinstall
#2 Download and Extract the required files
assuming that all the files have been downloaded into the ~/Downloads directory
Download the Apache HTTP Server httpd-2.4.41.tar.gz from Apache download page (https://httpd.apache.org/download.cgi)
and following required libraries for compiling the apache HTTP server:
apr-1.7.0.tar.gz(http://apr.apache.org/download.cgi)
apr-util-1.6.1.tar.gz(http://apr.apache.org/download.cgi)
pcre2-10.34.tar.gz (ftp://ftp.pcre.org/pub/pcre/) http://pcre.org/
****Read Requirements Section(http://httpd.apache.org/docs/2.4/install.html)***
#3 Extract the tar files
tar -xvf httpd-2.4.41.tar.gz
tar -xvf apr-1.7.0.tar.gz
tar -xvf apr-util-1.6.1.tar.gz
tar -xvf pcre2-10.34.tar.gz
after extraction you should see following list of directories in ~/Downloads
httpd-2.4.41
apr-1.7.0
apr-util-1.6.1
pcre2-10.34
#4 Create a directory for the apache HTTP Server
*make sure to give all rights to this directory so that while compiling files can be read/written
sudo mkdir /home{your username here}/apache
To give all permissions to a apache directory :
sudo chmod -R 777 /home{your username here}/apache
#5 copy the arp and arp-util directory into the ~/Downloads/httpd-2.4.41/srclib/
cd ~/Downloads
mv apr-util-1.6.1 ~/Downloads/httpd-2.4.41/srclib/apr-util
mv apr-1.7.0 ~/Downloads/httpd-2.4.41/srclib/apr
#6 Configure the sources for compilation.
The --prefix option can be used to install the Web server in a location where you can write files.
cd ~/Downloads/httpd-2.4.41/
./configure --prefix=/home/{username here}/apache --with-pcre=~/Downloads/pcre2-10.34
NOTE:
If you have some problems while running above command,you can also try
insallting pcre in /usr/local/pcre, using
./configure --prefix=/usr/local/pcre
make
sudo make install
commands and then by running folllowing commands:
cd ~/Downloads/httpd-2.4.41/
./configure --prefix=/home/{username here}/apache --with-pcre=/usr/local/pcre
make
sudo make install
If you still face some problems ,make sure that ~/apache and its nested directory have read/write permissions. if not run **sudo chmod -R 777 /home{your username here}/apache** command again.
#7 Compile Apache HTTP Server.
cd ~/Downloads/httpd-2.2.25
sudo make
#8 Install Apache HTTP Server.
cd ~/Downloads/httpd-2.2.25
sudo make install
Optional
#9 To Prepare Your Hosts File
sudo gedit /etc/hosts
27.0.0.1 localhost www.example.com
sudo gedit /home/{your username here}/apache/conf/httpd.conf
and copy:
Listen 8000
ServerName www.example.com:8000
#10 Test the installation to ensure Apache HTTP Server is working.
/home/{your username here}//apache/bin/apachectl -k start