libvirt and VirtualBox / Getting Started - api

I'm trying to get started on libvirt with VirtualBox as a virtualization solution. I installed everything and VirtualBox itself is running when using their VBoxHeadless command.
However, libvirt fails to connect to VirtualBox:
# virsh -c vbox:///session
libvir: error : could not connect to vbox:///session
error: failed to connect to the hypervisor
I could not find any hints in the libvirt documentation that point to whether I have to make any domain specific configuration before using virsh.
Does anyone have a hint? Or even better, maybe a tutorial that works through the way of using libvirt, virsh or it's APIs (my later goal) from the ground up.

If you are doing this on Ubuntu, then the problem is their libvirt package is built without VirtualBox support.
You can rebuild the package with support very easily. Something like:
apt-get source -d libvirt
sudo apt-get build-dep libvirt
dpkg-source -x libvirt*dsc
Go into the libvirt directory and edit debian/rules so that instead of --without-vbox it says --with-vbox. You can add an entry to the top of debian/changelog so the package is compiled as a different version (e.g., append ~local1 to the version).
dpkg-buildpackage -us -uc -b -rfakeroot
You'll get new .debs built in the directory above. Use dpkg -i to install the relevant ones (libvirt0, libvirt0-bin, and whatever else you want).

Double-check whether or not you have write access to /var/run/libvirt/libvirt-sock.
The socket file should have permissions similar to:
$ sudo ls -la /var/run/libvirt/libvirt-sock
srwxrwx--- 1 root libvirtd 0 2010-08-24 14:54 /var/run/libvirt/libvirt-sock
I think it could be helpful also to increase the libvirt logging capabilities by running this in your shell:
export LIBVIRT_DEBUG=1

There is Ubuntu PPA for libvirt with VirtualBox support: https://launchpad.net/~cxl/+archive/ubuntu/libvirt

Related

Qemu 5.2 - nothing shows up after VNC running

i'm trying to use QEMU 5.x for research.
I got QEMU 5.2 source code from qemu.org and installed following instructions.
However, when i tried to run VM by this command:
qemu-system-x86_64 \
-monitor stdio \
--enable-kvm \
-m 4096 \
-cdrom ubuntu-20.04.iso \
-drive file=img.qcow,if=virtio \
-boot c
-rtc base=localtime \
-device virtio-keyboard-pci \
-vga virtio \
then the following texts are printed:
QEMU 5.2.0 monitor - type 'help' for more information
(qemu) VNC server running on 127.0.0.1:5900
then nothing shows up, while QEMU 4.x (used before) pops up a window showing guest ubuntu's GUI.
I'm using ubuntu 20.04. Hope anyone has breakthrough for this..
The message says that this QEMU is using the VNC protocol for graphics output. You can connect a VNC client to the 127.0.0.1:5900 port that it tells you about to see the graphics output.
If what you wanted was a native X11 window (GTK), then the problem is probably that you didn't have the necessary libraries installed to build the GTK support. QEMU's configure script's default behaviour is "build all the optional features that this host has the libraries installed for, and omit the features where the libraries aren't present". So if you don't have any of the GTK/SDL etc libraries when you build QEMU, the only thing you will get in the resulting QEMU binary is the lowest-common-denominator VNC support. If you want configure to report an error for a missing feature then you need to pass it the appropriate --enable-whatever option to force the feature to be enabled (in this case, --enable-gtk).
If you're running on Ubuntu and your apt sources.list file has deb-src lines in it, the easiest way to install all the dependencies that would get you the same feature list as the real Ubuntu QEMU package is to run "apt build-dep qemu". I recommend that you do that and then re-build QEMU, passing --enable-gtk to configure so you can confirm that the necessary dependencies were installed.

Multiple issues trying to install PyBOSSA

I am trying to set up PyBOSSA on an AWS EC2 instance running Ubuntu 18.04 LTS. I am following the official instructions and have encountered three errors so far.
sudo apt-get install -y git postgresql postgresql-all postgresql-server-dev-all libpq-dev python-psycopg2 libsasl2-dev libldap2-dev libssl-dev python-virtualenv python-dev build-essential libjpeg-dev libssl-dev libffi-dev dbus libdbus-1-dev libdbus-glib-1-dev libldap2-dev libsasl2-dev python-pip python3-pip redis-server
cd ~
git clone --recursive https://github.com/Scifabric/pybossa
cd pybossa
virtualenv -p python3 env (I'm using Python3 explicitly as my system also has Python 2.7 installed).
source env/bin/activate
pip install -U pip
pip install -r ~/pybossa/requirements.txt
At this point, I start getting error messages... I have copied the stdout and stderr into a file, which I have uploaded here.
I'm not sure if the errors there are what have caused my later errors, but I pushed on through the instructions anyway in hopes it'd work...
cp settings_local.py.tmpl settings_local.py
cp alembic.ini.template alembic.ini
redis-server contrib/sentinel.conf --sentinel
I noted that the Redis server version was 4.0.9 (the instructions say it needs to be v2.6 or greater).
The output from starting the Redis server was as follows:
30284:X 30 Mar 03:09:22.004 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
30284:X 30 Mar 03:09:22.004 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=30284, just started
30284:X 30 Mar 03:09:22.004 # Configuration loaded
...I gather that's ok...
rqscheduler --host 127.0.0.1
This command wasn't installed on my system. I tried to use apt to install it, but there was nothing there. I also tried apt install rq rqscheduler rq-scheduler - nothing found. I then Googled and found the website for rq-scheduler, and found that I could install it by running pip install rq-scheduler
That installed correctly. Nonetheless, running the command rqscheduler --host 127.0.0.1 in the terminal still failed: rqscheduler: command not found.
Knowing that it was a Python package, I wondered if maybe I needed to prepend python3 onto the start of the command: python3 rqscheduler --host 127.0.0.1. Response: python3: can't open file 'rqscheduler': [Errno 2] No such file or directory.
I also tried pip3 install rq-scheduler (which installed fine) and then running the command, but encountered the same error.
I would appreciate knowing how to get that running, but for the purpose of this test, I skipped setting up Regis and the scheduler, and continued with the PyBOSSA instructions:
sudo su postgres
createuser -d -P pybossa
(Password set)
createdb pybossa -O pybossa
exit
python3 cli.py db_create
...and then I got this error:
File "cli.py", line 162
'''SELECT id, created FROM task_run WHERE created LIKE ('\x%')''')
^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 54-55: truncated \xXX escape
I instead tried python cli.py db_create, just in case it'd work, and got a different error:
python cli.py db_create
ValueError: invalid \x escape
So I'm seeing three separate issues:
Installing the PyBOSSA-required Python packages.
The issue with the rqscheduler command.
The error when starting the PyBOSSA server.
What do these errors mean?
1 ) For the installation, try this:
virtualenv env
source env/bin/activate
sudo apt install python3-pip
pip3 install -r requirements.txt
Which ended with no error.
2) Try :
pip install rq-scheduler==0.9.1
or
pip3 install rq-scheduler==0.9.1
3) The \ char need to be escaped (like \\) in python.
So you may alter the cli.py line 162 (using text editor) from:
'''SELECT id, created FROM task_run WHERE created LIKE ('\x%')''')
To:
'''SELECT id, created FROM task_run WHERE created LIKE ('\\x%')''')
But it will be better to be fixed by dev on github ...
CONCLUSION
According to official documentation,
PYBOSSA for python 3 We’ve finally migrated PYBOSSA to python 3. We’re
not going to merge into master until we test it in production a bit
more, so please, help us by testing it. All you have to do is
basically, check out the python3 branch (migrate-python3) and run it.
Then, any bug, issue you find, you just report it and we will be happy
to help you.
The PYBOSSA python3 version is freshly migrated so finaly is not very stable ... I expect that it will be better to use the PYBOSSA python2.7 branch and follow exactly the documentation.
And according to official github account they try to make money with support (?...)
Get professional support You can hire us to help you with your PYBOSSA
project or server (specially for python 2.7). Go to our website, and
contact us.
The issue has now been fixed for the master branch (https://github.com/Scifabric/pybossa/pull/1986). You can fetch the new code and use it.

Unable to open X display when trying to run google-chrome on Centos (Rhel 7.5)

I need to run Google Chrome remotely on a virtual machine using SSH. I do not want xforwarding - I want to utilize the GPU available on the vm. When I try running google-chrome I get following error:
[19615:19615:0219/152933.751028:ERROR:browser_main_loop.cc(1512)] Unable to open X display.
I've tried to setting my DISPLAY env value to various values:
export DISPLAY=localhost:0.0
export DISPLAY=127.0.0.1:0.0
export DISPLAY=:0.0
I've also tried replacing 0.0 in abowe examples with different values.
I have ForwardX11 no in /etc/ssh/sshd_config
I tried setting up target like this:
systemctl isolate multi-user.target
When I try to run sudo lshw -C display i get folowing output:
*-display
description: VGA compatible controller
product: Hyper-V virtual VGA
vendor: Microsoft Corporation
physical id: 8
bus info: pci#0000:00:08.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: vga_controller bus_master rom
configuration: driver=hyperv_fb latency=0
resources: irq:11 memory:f8000000-fbffffff
*-display UNCLAIMED
description: VGA compatible controller
product: GM204GL [Tesla M60]
vendor: NVIDIA Corporation
physical id: 1
version: a1
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress vga_controller bus_master cap_list
configuration: latency=0
resources: iomemory:f0-ef iomemory:f0-ef memory:41000000-41ffffff memory:fe0000000-fefffffff memory:ff0000000-ff1ffffff
I've tried to update my gpu drivers by:
wget https://www.nvidia.com/content/DriverDownload-March2009/confirmation.php?url=/tesla/375.66/nvidia-diag-driver-local-repo-rhel7-375.66-1.x86_64.rpm
yum -y install nvidia-diag-driver-local-repo-rhel7-375.66-1.x86_64.rpm
But after that I still see UNCLIMED next to my NVIDIA gpu.
Aby ideas?
You can try with Xvfb. it does not require additional hardware.
Install Xvfb if you didn't install it yet and do the following steps.
sudo apt-get install -y xvfb
Dependencies to make "headless" chrome/selenium work:
sudo apt-get -y install xorg xvfb gtk2-engines-pixbuf
sudo apt-get -y install dbus-x11 xfonts-base xfonts-100dpi xfonts-75dpi xfonts-cyrillic xfonts-scalable
Optional but nifty: For capturing screenshots of Xvfb display:
sudo apt-get -y install imagemagick x11-apps
Make sure that Xvfb starts every time the box/vm is booted:
Xvfb -ac :99 -screen 0 1280x1024x16 &
export DISPLAY=:99
Run Google Chrome
google-chrome
Okay guys. I found my problem after 2 hours of going crazy. My box was configured correctly. What you can NOT do, is ssh from one box, to another box, to this box and expect X11 forwarding to play nicely. Without tearing apart the entire network, I found that if I shelled over from the MAIN box to this box ( no double or triple ssh'ing) chrome comes right up as a regular user using CLI. So it was a matter of multiple shells from multiple boxes that made the display say it was set to NOTHING! Setting the display manually only complicates the problems. Once I shelled directly over to this box from the main outside box, my display was set to 10:0, which is first instance in my configuration. Don't make this mistake, you will waste valuable time.
FWIW, I ran into this when using SSH to log into a Selenium chrome node in a Docker compose stack. Chrome would launch if I invoked it as root with sudo -u seluser google-chrome, but not if I logged in as seluser. The trick turned out to be that root had DISPLAY set to :99:0, and seluser didn't have it set at all. If I set it explicitly (either from a seluser shell or from the docker compose exec command line) it worked.
$ docker-compose exec -u seluser \
selenium-chrome \ # or whatever your service is called
/bin/bash
seluser#c02cda62b751:/$ export DISPLAY=:99:0
seluser#c02cda62b751:/$ google-chrome http://app.test:3000/home
or
$ docker-compose exec -u seluser -e DISPLAY=:99:0 \
selenium-chrome \
google-chrome http://app.test:3000/home
That :99.0 is undocumented, though, so if this isn't working, you might try checking root's DISPLAY value with:
docker-compose exec -u root selenium-chrome bash -c 'echo "${DISPLAY}"'
I faced the same issue with WSL and Ubuntu. I have unininstalled/Reset the ubuntu. After that, I have executed the below command
wsl --set-default-version 2
then I installed again the Ubuntu, I didn't get --no-sandbox issue or any issue.
Hope this will use for someone.

Found unknown Linux distribution on /dev/sdb2: grub configuration dual boot Arch Linux and NetBSD-7.0

I have Arch Linux on /dev/sdb1 and NetBSD-7.0 on /dev/sdb2.
On Arch Linux when I run sudo grub-mkconfig -o /boot/grub/grub.cfg I get a message like Found unknown Linux distribution on /dev/sdb2 but when I reboot, there is no grub option for that unknown Linux distribution which I know it is NetBSD-7.0.
How can I add NetBSD-7.0 to my grub menu option when rebooting.
There is a similar post, currently looking into it.
UPDATE: I mounted NetBSD partition with sudo mount -t ufs -o ro,ufstype=ufs2 /dev/sdb2 /mnt/ (ufstype=44bsd did not work) and then ran grub-mkconfig -o /boot/grub/grub.cfg but yet the issue persists.
UPDATE: Rebooted and pressed c to get the grub command line. Following commands booted the NetBSD-7.0:
ls
Ran ls to see the correct name of disks and partitions, /dev/sdb2 on Linux was (hd0,gpt2) on Grub. Then ran the following:
insmod ufs2
set root=(hd0,gpt2)
knetbsd /netbsd
boot
And NetBSD-7.0 booted.
To add NetBSD option to Grub menu, modified file /etc/grub/40_custom on Arch Linux like below:
menuentry "NetBSD-7.0"{
insmod ufs2
set root=(hd0,gpt2)
knetbsd /netbsd
}
However, after modifying 40_custom like above, NetBSD option does not appear on Grub menu. I don't know why.
Unless you have a typo, it looks like the 40_custom file is in the wrong directory. it should be located at /etc/grub.d/40_custom, notice the .d.
If your /boot is located on a separate partition, make sure that it is mounted with mount /boot before generating the grub.cfg. Otherwise your new grub.cfg won't be used.
Check which partition grub is loading the configuration from by running echo ${prefix} within the grub command line. It's possible that grub is loading the configuration from a partition that you don't expect.
Verify that netbsd was added to the config with grep -i netbsd /boot/grub/grub.cfg before rebooting to avoid some frustration after generating grub.cfg

How to install wkhtmltopdf on a linux based (shared hosting) web server

I have tried in all ways to get wkhtmltopdf installed on our web server but unfortunately it is not getting installed. I cannot access user/bin folder as stated in a tutorial on installation.
On the server in public_html folder there is a sub folder _vti_bin, I copied the file wkhtmltopdf-i386 from wkhtmltopdf-0.9.1-static-i386, but I am not able to execute it.
How to install wkhtmltopdf on (shared hosting) web server and get it working?
I've managed to successfully install wkhtmltopdf-amd64 on my shared hosting account without root access.
Here's what i did:
Downloaded the relevant static binary v0.10.0 from here: http://code.google.com/p/wkhtmltopdf/downloads/list
EDIT: The above has moved to here
via ssh on my shared host typed the following:
$ wget {relavant url to binary from link above}
$ tar -xvf {filename of above wget'd file}
you'll then have the binary on your host and will be able to run it regardless of if its in the /usr/bin/ folder or not. (or at least i was able to)
To test:
$ ./wkhtmltopdf-amd64 http://www.example.com example.pdf
Note remember that if you're in the folder in which the executable is, you should probably preface it with ./ just to be sure.
Worked for me anyway
If you have sudo access...
Ubuntu 14.04 / 15.04 / 18.04:
sudo apt-get install wkhtmltopdf
# or
sudo apt install wkhtmltopdf
Others
Look at the other answers.
If its ubuntu then go ahead with this, already tested.:--
first, installing dependencies
sudo aptitude install openssl build-essential xorg libssl-dev
for 64bits OS
wget http://wkhtmltopdf.googlecode.com/files/wkhtmltopdf-0.9.9-static-amd64.tar.bz2
tar xvjf wkhtmltopdf-0.9.9-static-amd64.tar.bz2
mv wkhtmltopdf-amd64 /usr/local/bin/wkhtmltopdf
chmod +x /usr/local/bin/wkhtmltopdf
for 32bits OS
wget http://wkhtmltopdf.googlecode.com/files/wkhtmltopdf-0.9.9-static-i386.tar.bz2
tar xvjf wkhtmltopdf-0.9.9-static-i386.tar.bz2
mv wkhtmltopdf-i386 /usr/local/bin/wkhtmltopdf
chmod +x /usr/local/bin/wkhtmltopdf
Debian 8 Jessie
This works
sudo apt-get install wkhtmltopdf
Chances are that without full access to this server (due to being a hosted account) you are going to have problems. I would go so far as to say that I think it is a fruitless endeavor--they have to lock servers down in hosted environments for good reason.
Call your hosting company and make the request to them to install it, but don't expect a good response--they typically won't install very custom items for single users unless there is a really good reason (bug fixes for example).
Lastly, depending on how familiar you are with server administration and what you are paying for server hosting now consider something like http://www.slicehost.com. $20 a month will get you a low grade web server (256 ram) and you can install anything you want. However, if you are running multiple sites or have heavy load the cost will go up as you need larger servers.
GL!
Latest update for CentOS:
sudo yum install -y libpng libjpeg openssl icu libX11 libXext libXrender xorg-x11-fonts-Type1 xorg-x11-fonts-75dpi
wget https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
tar -xvf wkhtmltox-0.12.4_linux-generic-amd64.tar
sudo mv wkhtmltox/bin/* /usr/local/bin/
check installation success: wkhtmltopdf -V
rm -rf wkhtmltox
rm -f wkhtmltox-0.12.4_linux-generic-amd64.tar
Place the wkhtmltopdf executable on the server and chmod it +x.
Create an executable shell script wrap.sh containing:
#!/bin/sh
export HOME="$PWD"
export LD_LIBRARY_PATH="$PWD/lib/"
exec $# 2>/dev/null
#exec $# 2>&1 # debug mode
Download needed shared objects for that architecture and place them an a folder named "lib":
lib/libfontconfig.so.1
lib/libfontconfig.so.1.3.0
lib/libfreetype.so.6
lib/libfreetype.so.6.3.18
lib/libX11.so.6 lib/libX11.so.6.2.0
lib/libXau.so.6 lib/libXau.so.6.0.0
lib/libxcb.so.1 lib/libxcb.so.1.0.0
lib/libxcb-xlib.so.0
lib/libxcb-xlib.so.0.0.0
lib/libXdmcp.so.6
lib/libXdmcp.so.6.0.0
lib/libXext.so.6 lib/libXext.so.6.4.0
(some of them are symlinks)
… and you're ready to go:
./wrap.sh ./wkhtmltopdf-amd64 --page-size A4 --disable-internal-links --disable-external-links "http://www.example.site/" out.pdf
If you experience font problems like squares for all the characters, define TrueType fonts explicitly:
#font-face {
font-family:Trebuchet MS;
font-style:normal;
font-weight:normal;
src:url("http://www.yourserver.tld/fonts/Trebuchet_MS.ttf");
format(TrueType);
}
List of stable versions wkhtmltopdf: http://wkhtmltopdf.org/downloads.html
Installing wkhtmltopdf on Debian 8.2 (jessie) x64:
sudo apt-get install xfonts-75dpi
sudo apt-get install xfonts-base
sudo wget http://download.gna.org/wkhtmltopdf/0.12/0.12.2.1/wkhtmltox-0.12.2.1_linux-jessie-amd64.deb
sudo dpkg -i wkhtmltox-0.12.2.1_linux-jessie-amd64.deb
Shared hosting no ssh or shell access?
Here is how i did it;
Visit https://wkhtmltopdf.org/downloads.html and download the appropriate stable release for Linux. For my case I chose 32-bit
which is wkhtmltox-0.12.4_linux-generic-i386.tar.xz
Unzip to a folder on your local drive.
Upload the folder to public_html (or whichever location fits your need) using an FTP program just like any other file(s)
Change the binary paths in snappy.php file to point the appropriate files in the folder you just uploaded.
Bingo! there you have it. You should be able to generate PDF files.
A few things have changed since the top answers were added. They used to work out for me, but not quite anymore, so I have been hacking around for a bit and came up with the following solution for Ubuntu 16.04. For Ubuntu 14.04, see the comment at the bottom of the answer. Apologies if this doesn't work for shared hosting, but it seems like this is the goto answer for wkhtmltopdf installation instructions in general.
# Install dependencies
apt-get install libfontconfig \
zlib1g \
libfreetype6 \
libxrender1 \
libxext6 \
libx11-6
# TEMPORARY FIX! SEE: https://github.com/wkhtmltopdf/wkhtmltopdf/issues/3001
apt-get install libssl1.0.0=1.0.2g-1ubuntu4.8
apt-get install libssl-dev=1.0.2g-1ubuntu4.8
# Download, extract and move binary in place
curl -L -o wkhtmltopdf.tar.xz https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
tar -xf wkhtmltopdf.tar.xz
mv wkhtmltox/bin/wkhtmltopdf /usr/local/bin/wkhtmltopdf
chmod +x /usr/local/bin/wkhtmltopdf
Test it out:
wkhtmltopdf http://www.google.com google.pdf
You should now have a file named google.pdf in the current working directory.
This approach downloads the binary from the website, meaning that you can use the latest version instead of relying on package managers to be updated.
Note that as of today, my solution includes a temporary fix to this bug. I realize that the solution is really not great, but hopefully it can be removed soon. Be sure to check the status of the linked GitHub issue to see if the fix is still necessary when you read this answer!
For Ubuntu 14.04, you will need to downgrade to a different version of libssl. You can find the versions here. Anyways, be sure to consider the implications of downgrading libssl before doing so on any production server.
I hope this helps someone!
After trying, below command work for me
cd ~
yum install -y xorg-x11-fonts-75dpi xorg-x11-fonts-Type1 openssl git-core fontconfig
wget https://downloads.wkhtmltopdf.org/0.12/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
tar xvf wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
mv wkhtmltox/bin/wkhtmlto* /usr/bin
Version 12.5 of wkhtmltopdf only lists DEB files on their download page now. Being a mac user and not knowing much linux or what DEB files were I couldn't use the solutions posted.
This page helped me get past the knew twist of downloading a DEB file: http://www.g-loaded.eu/2008/01/28/how-to-extract-rpm-or-deb-packages/
Basically what I did was:
Downloaded from https://wkhtmltopdf.org/downloads.html
Unzipped the DEB file.
Unzipped data.tar.xz
Uploaded the binary in the unzipped 'usr' folder from step 3 (usr/local/bin/wkhtmltopdf)
Then I found out that the 'exec' function was disabled on my host. So make sure you can specifically run 'exec' if you're using PHP to run this. "Can I run the wkhtmltopdf binary" isn't specific enough. My fault.