I am trying to set up an Apache server with the ModSecurity Rule set inside a Docker container. I followed a few tutorials (this, this and this) to build a secure Apache server. But I am unable to make the server work with the rule set.
I get this error:
AH00526: Syntax error on line 855 of /etc/httpd/modsecurity.d/crs-setup.conf:
ModSecurity: Found another rule with the same id
I searched for the error and according to the answers on this page the fault lies in including the same rules twice. But as far as I can see, I am not including the same rules twice and I wonder if the error lies elsewhere.
My project file structure is the following:
.
├── conf
│ └── httpd.conf
├── Dockerfile
├── index.html
├── modsecurity.d
│ ├── crs-setup.conf
│ ├── modsecurity.conf
│ └── rules
The httpd.conf file is the default config file used for an Apache server and the modsecurity configurations are inserted via commands in the Dockerfile.
The Dockerfile has the following configuration
FROM centos:7
RUN yum -y update && \
yum -y install less which tree httpd mod_security && \
yum clean all
COPY index.html /var/www/html/
#COPY conf/ /etc/httpd/conf/
COPY modsecurity.d/crs-setup.conf /etc/httpd/modsecurity.d/
COPY modsecurity.d/modsecurity.conf /etc/httpd/modsecurity.d/
COPY modsecurity.d/rules/* /etc/httpd/modsecurity.d/rules/
RUN echo "ServerName localhost" >> /etc/httpd/conf/httpd.conf
RUN echo "<IfModule security2_module>" >> /etc/httpd/conf/httpd.conf
RUN echo " Include modsecurity.d/crs-setup.conf" >> /etc/httpd/conf/httpd.conf
RUN echo " Include modsecurity.d/rules/*.conf" >> /etc/httpd/conf/httpd.conf
RUN echo " SecRuleEngine On" >> /etc/httpd/conf/httpd.conf
RUN echo "</IfModule>" >> /etc/httpd/conf/httpd.conf
EXPOSE 80
CMD ["/usr/sbin/httpd", "-D", "FOREGROUND"]
index.html is just a basic hello file:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" lang="en">
</head>
<body>
<h1>Hello there</h1>
</body>
</html>
crs-setup.conf has the following content (excluding all the comments)
SecRuleEngine On
SecDefaultAction "phase:1,log,auditlog,pass"
SecDefaultAction "phase:2,log,auditlog,pass"
SecCollectionTimeout 600
SecAction \
"id:900990,\
phase:1,\
nolog,\
pass,\
t:none,\
setvar:tx.crs_setup_version=310"
modsecurity.conf has only these two lines
SecRequestBodyAccess On
SecStatusEngine On
rules is a directory which contains the ModSecurity rule set.
I also placed the project files on github if anyone wants to have a look at the whole setup.
I found out why I got the error. The ModSecurity configuration file was misnamed and the rule files had been placed in the wrong directory.
The ModSecurity file was modsecurity.conf, when in fact it should have been mod_security.conf, notice the underscore (source). The rule files should have been placed in a folder called activated_rules(source).
In my working configuration I now have the following folder structure:
.
├── conf
│ └── httpd.conf
├── Dockerfile
├── index.html
└── modsecurity.d
├── crs-setup.conf
├── mod_security.conf
└── activated_rules
The Dockerfile is as follows
FROM centos:7
RUN yum -y update && \
yum -y install less which tree httpd mod_security && \
yum clean all
COPY index.html /var/www/html/
RUN echo "ServerName localhost" >> /etc/httpd/conf/httpd.conf
RUN echo "<IfModule security2_module>" >> /etc/httpd/conf/httpd.conf
RUN echo "Include modsecurity.d/crs-setup.conf" >> /etc/httpd/conf/httpd.conf
RUN echo "Include modsecurity.d/activated_rules/*.conf" >> /etc/httpd/conf/httpd.conf
RUN echo "</IfModule>" >> /etc/httpd/conf/httpd.conf
COPY modsecurity.d/crs-setup.conf /etc/httpd/modsecurity.d/
COPY modsecurity.d/mod_security.conf /etc/httpd/conf.d/
COPY modsecurity.d/rules/* /etc/httpd/modsecurity.d/activated_rules/
EXPOSE 80
CMD ["/usr/sbin/httpd", "-D", "FOREGROUND"]
Related
Trying to perform a fairly simple gzip command across my fastq files, but a strange error returns.
#!/usr/bin/env nextflow
nextflow.enable.dsl=2
params.gzip = "sequences/sequences_split/sequences_trimmed/trimmed*fastq"
workflow {
gzip_ch = Channel.fromPath(params.gzip)
GZIP(gzip_ch)
GZIP.out.view()
}
process GZIP {
input:
path read
output:
stdout
script:
"""
gzip ${read}
"""
}
Error:
Command error:
gzip: trimmed_SRR19573319_R2.fastq: Too many levels of symbolic links
Tried running a loop in the script instead or run gzip on individual files which works, but would rather use the nextflow syntax.
By default, Nextflow will try to stage process input files using symbolic links. The problem is that gzip actually ignores symbolic links. From the GZIP(1) man page:
The gzip command will only attempt to compress regular files. In particular, it will ignore symbolic links.
If the objective is to create a reproducible workflow, it's usually best to avoid modifying the workflow inputs directly anyway. Either use the stageInMode directive to change how the input files are staged in. For example:
process GZIP {
stageInMode 'copy'
input:
path fastq
output:
path "${fastq}.gz"
"""
gzip "${fastq}"
"""
}
Or, preferably, just modify the command to redirect stdout to a file:
process GZIP {
input:
path fastq
output:
path "${fastq}.gz"
"""
gzip -c "${fastq}" > "${fastq}.gz"
"""
}
Michael!
I can't reproduce your issue. I created the folders in my current directory like you described and created four files in it, as you can see below:
➜ ~ tree sequences/
sequences/
└── sequences_split
└── sequences_trimmed
├── trimmed_a_fastq
├── trimmed_b_fastq
└── trimmed_c_fastq
Then I copy-pasted your Nextflow script file (the only change I did was to use gzip -f ${read} instead of without the -f option. Then everything worked fine. The reason you need -f is because Nextflow has every task contained to a subfolder within work. This means your input files are symbolically linked and gunzip will complain they're not regular files (happened here, macOS Ventura) or something like that (It may depend on OS? Not sure). The -f solves for this issue.
N E X T F L O W ~ version 22.10.1
Launching `ex2.nf` [golden_goldstine] DSL2 - revision: 70559e4bcb
executor > local (3)
[ad/447348] process > GZIP (1) [100%] 3 of 3 ✔
➜ ~ tree work
work
├── 0c
│ └── ded66d5f2e56cfa38d85d9c86e4e87
│ └── trimmed_a_fastq.gz
├── 67
│ └── 949c28cce5ed578e9baae7be2d8cb7
│ └── trimmed_c_fastq.gz
└── ad
└── 44734845950f28f658226852ca4200
└── trimmed_b_fastq.gz
They're gzip compressed files (even though they may look just like text files, depending on the demo content). I decided to reply with an answer because it allows me to use markdown to show you how I did it. Feel free to comment this answer if you want to discuss this topic.
I have this Dockerfile that does extra configuration of the official Apache HTTP Server Docker image. Is there a cleaner way to do this inside a Dockerfile? I'm very unfamiliar with Apache HTTP Server configuration and was just about able to cobble this together. (The reason for enabling mod_rewrite is because I use it in a .htaccess file in the htdocs folder)
ARG BUILD_DIR=/usr/src/app
FROM node:10.13.0-alpine as build
ARG BUILD_DIR
WORKDIR $BUILD_DIR
COPY package.json .
RUN npm install
COPY src src
COPY public public
RUN npm run build
FROM httpd:2.4.37-alpine
ARG BUILD_DIR
ENV SERVER_CONTAINER_NAME=server
COPY --from=build $BUILD_DIR/build htdocs
RUN sed -i 's,#\(LoadModule rewrite_module modules/mod_rewrite.so\),\1,g' conf/httpd.conf \
&& sed -i -e '/<Directory "\/usr\/local\/apache2\/htdocs">/,/<\/Directory>/{s/AllowOverride None/AllowOverride All/}' conf/httpd.conf \
&& sed -i 's,#\(LoadModule proxy_module modules/mod_proxy.so\),\1,g' conf/httpd.conf \
&& sed -i 's,#\(LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so\),\1,g' conf/httpd.conf \
&& echo 'ProxyRequests off' >> conf/httpd.conf \
&& echo 'ProxyPass /ws ws://${SERVER_CONTAINER_NAME}:8080/ws interpolate' >> conf/httpd.conf
It might be cleaner to keep the modified versions of those config files outside of the image, and then copy them into the container during the build.
Alternatively, you could put all the shell commands in a script. During the build, COPY the script into the container. At the end of the script add something like this:
exec /usr/sbin/apache2ctl -f /etc/apache2/apache2.conf -DFOREGROUND
to start your server. Then have docker invoke the script as
CMD [ "/MyScript.sh" ]
I'm trying to create a docker image with openssl DLLs, but something goes wrong.
Before I build, I have following folder structure on my PC:
/docker-sandbox
├── deployment/
│ ├── libeay32.dll (OpenSSL)
│ ├── ssleay32.dll (OpenSSL)
│ └── deployment.ps1
└── Dockerfile
Dockerfile:
FROM microsoft/windowsservercore
COPY deployment/ C:/deployment
RUN "powershell C:\deployment\deployment.ps1"
And I have the powershell script:
#go to corrext location
Set-Location C:\deployment
#locate openssl libraries
New-Item -ItemType Directory -Path C:\openssllib
Move-Item -Path '.\libeay32.dll' -Destination 'C:\openssllib\libeay32.dll'
Move-Item -Path '.\ssleay32.dll' -Destination 'C:\openssllib\ssleay32.dll'
After I execute "docker build ." in the docker-sandbox folder, I start the container with following command:
docker run -i $IMAGE_ID powershell
But the C:\openssllib folder in the container is empty (and there are also no OpenSSL DLLs in the "deployment" folder). Why?
When I remove RUN from the dockerfile and execute "deployment.ps1" in the container by hand, openssl DLLs are located in the right place. What is the reason for such behavior?
When I execute my cp-sshkey.yml playbook (logged in as myself, not the vagrant user) from my top-level Vagrantfile directory...
ansible-playbook cp-sshkey.yml
I'm getting this error:
TASK: [authorized_key user=vagrant key="{{ lookup('file', './files/id_rsa_vagrant.pub') }}"] ***
fatal: [web1] => Failed to template user=vagrant key="{{ lookup('file', './files/id_rsa_vagrant.pub') }}": could not locate file in lookup: ./files/id_rsa_vagrant.pub
I don't understand why this error is occurring. It's a very simple playbook and the public key file is where I say it is:
.
├── .vagrant
│ └── machines
├── Vagrantfile
├── ansible.cfg
├── bootstrap-mgmt.sh
├── files
│ └── id_rsa_vagrant.pub
├── inventory.ini
├── secrets.yml
├── site.yml
├── website
└── cp-sshkey.yml
Here's my config and host files and the playbook:
# ansible.cfg
[defaults]
hostfile = inventory.ini
remote_user = vagrant
private_key_file = .vagrant/machines/default/virtualbox/private_key
host_key_checking = False
# inventory.ini
[local]
localhost ansible_connection=local
[web]
web1 ansible_ssh_host=127.0.0.1 ansible_ssh_port=2222
# cp-sshkey.yml
- name: Install vagrant's public key on VM
hosts: web1
sudo: True
tasks:
- authorized_key: user=vagrant key="{{ lookup('file', './files/id_rsa_vagrant.pub') }}"
What am I doing wrong here? Thanks.
Quick answer will refine - am playing with this myself but just learning:
I assume you are trying to add your public key (or other key on your ansible console that is no related to the vagrant keys ) to the vagrant machine to allow you to ssh into it without vagrant ssh
I assume that you have checked all the file permissions etc and that you aren't juggling multiple instances. Tried with 127.0.0.1 and localhost and that you've tried with the full file path instead of relative to working directory - my examples use files in subfolders with templates although not in the working snippet below.
Are you able to vagrant ssh
and perhaps check the .ssh/authorized_keys file ?
Are you able to confirm that ansible can connect doing something like ansible web -a df
are you able to ssh into the Vagrant machine using ssh -i .vagrant/machines/default/virtualbox/private_key vagrant#127.0.0.1 -p 2222
In my role task file I have this task.
- name: Copy origin public key to auth keys
authorized_key: user=vagrant key="{{ lookup('file', lookup('env','HOME') + '/.ssh/id_rsa.pub') }}"
Also my host definition has the user:
web1 ansible_ssh_host=127.0.0.1 ansible_ssh_port=2222 ansible_ssh_user=vagrant
but assume that the config you use should work.
This play worked is working for me although my directory structure is different - I expect you're comfortable that your is fine.
Some things to watch out for that caught me:
when you rebuild the Vagrant machine you will need to flush out your .ssh/known_hosts if you have already ssh'd - can remove from known_hosts with sh-keygen -R [localhost]:2222
makes me uneasy seeing localhost as a machine tag
My Setup:
ansible 2.2.0.0
vagrant Version: 1.9.1
Mac OSX
VBox 5.1.6
Vagrant Instance - ubuntu/trusty64
System: Ubuntu 14.04 LAMP running on Parallels VM set up with Vagrant
I'm writing my first non-trivial shell script to add new web projects to a dev VM on my Mac laptop.
Create a default folder structure in /var/www/
Add a .conf vhost file to /etc/apache2/sites-available with the new domain replacing placeholders via sed
Enable the new site and restart apache
I've got the folders and files copying over and sed seems happy customizing my index.html and .conf vhost file, but a2ensite doesn't seem to see the .conf file in /etc/apache2/sites-available/
I test for its existence and even print a debug listing: ls -al /etc/apache2/sites-available/ | grep $CONFFILE before attempting to enable the site.
I've read here and elsewhere about the importance of having the .conf extension since Ubuntu 13 (or 14) which seems to be a very common issue. My vhost file has the .conf extension so this seems like a different issue.
Can anyone point me in the right direction? I haven't been able to find other postings with this particular problem.
My feeling is that I've got an error in my $CONFFILE variable expansion in the a2ensite command because the error does not show the .conf extension even though the directory listing does:
ERROR: Site /etc/apache2/sites-available/example-com-80 does not exist!
Edit:
After running a2ensite from the command line per Micheal's suggestion below, it seemed to parse fine, but still doesn't show the extension:
$ sudo a2ensite example-com-80.conf
Enabling site example-com-80.
To activate the new configuration, you need to run:
service apache2 reload
End Edit
Edit: Found answer
After searching with broader terms, a2ensite instead of Ubuntu 14.04 Vagrant etc, I found a two year old question where #raina77ow points out that a2ensite just wants the site name, not the whole path. Changing sudo a2ensite /etc/apache2/sites-available/$CONFFILE to sudo a2ensite $CONFFILE
makes the script work as intended. This also explains why my previous attempts to run a2ensite from the command line failed; I was running it from inside /var/www/templates/ and passing in the whole path to the .conf file.
Now, a stackoverflow question, how best should I indicate this is the solution with the limited reputation that I have? And give credit properly?
See edit above for solution
Console output with example.com:
$ ./newvhost
New Server Name with Top Level Domain: example.com
Validating: example.com
New DocumentRoot created: /var/www/example
Copying template structure
Creating: example-com-80.conf
-rw-r--r-- 1 root root 811 Feb 17 15:11 example-com-80.conf
Enabling site
ERROR: Site /etc/apache2/sites-available/example-com-80 does not exist!
newvhost script:
OLDIFS=$IFS
IFS="."
printf "New Server Name with Top Level Domain: "
read NEW_SUBDOMAIN NEW_TLD
IFS=$OLDIFS
NEW_FULL_NAME="$NEW_SUBDOMAIN.$NEW_TLD"
echo "Validating: $NEW_FULL_NAME"
if [[ "$NEW_TLD" != "com" && "$NEW_TLD" != "dev" ]] ; then
echo -e "\E[31;1mTLD must be com or dev! \033[0m"
exit 1
fi
if [ -d "/var/www/$NEW_SUBDOMAIN" ]; then
echo -e "\E[31;1mRoot directory /var/www/$NEW_SUBDOMAIN already exists!\033[0m"
exit 1
fi
mkdir /var/www/$NEW_SUBDOMAIN
if [ -d "/var/www/$NEW_SUBDOMAIN" ]; then
echo "New DocumentRoot created: /var/www/$NEW_SUBDOMAIN"
else
echo -e "\E[31;1mUnable to make directory\033[0m"
exit 1
fi
echo "Copying template structure"
cp /var/www/templates/structure/. /var/www/$NEW_SUBDOMAIN/ -R
sed -i "s/TEMPLATE/$NEW_FULL_NAME/g" /var/www/$NEW_SUBDOMAIN/index.html
CONFFILE="$NEW_SUBDOMAIN-$NEW_TLD-80.conf"
echo "Creating: $CONFFILE"
sudo cp /var/www/templates/vhost_template.conf /etc/apache2/sites-available/$CONFFILE
sudo sed -i "s/FULLNAME/$NEW_FULL_NAME/g" /etc/apache2/sites-available/$CONFFILE
sudo sed -i "s/DOMAINNAME/$NEW_SUBDOMAIN/g" /etc/apache2/sites-available/$CONFFILE
if [ -e "/etc/apache2/sites-available/$CONFFILE" ]; then
ls -al /etc/apache2/sites-available/ | grep $CONFFILE # DEGBUG Listing to doubly confirm $CONFFILE exists
echo "Enabling site"
sudo a2ensite /etc/apache2/sites-available/$CONFFILE
sudo apache2ctl graceful
fi
Thanks,
Any other suggestions for improving the script are very welcome as long as that doesn't run afoul with the terms of StackOverflow.
The answer, in short, is that a2ensite just wants the name of the site.conf and not the whole path to the file.
So sudo a2ensite example-com-80.conf
I found this in an earlier answer by #raina77ow.