I have configured a systemd Unit for DataStax Enterprise 4.8.5:
### /etc/systemd/system/dse1.service
[Unit]
Description=DataStax Enterprise
[Service]
User=cassandra
ExecStart=/opt/dse/dse1/bin/dse cassandra -k
ExecStop=/opt/dse/dse1/bin/dse cassandra-stop
when I execute sudo systemctl start dse1, if i immediately do a status afterwards, i get:
● dse1.service - DataStax Enterprise 1
Loaded: loaded (/etc/systemd/system/dse1.service; static; vendor preset: disabled)
Active: active (running) since Wed 2016-03-23 13:47:57 EDT; 1s ago
Main PID: 31699 (cassandra)
CGroup: /system.slice/dse1.service
├─31699 /bin/sh /opt/dse/dse1/resources/cassandra/bin/cassandra -k -Djava.library.path=:/opt/dse/dse1/resources/hadoop/native...
├─31894 /bin/java -cp :/opt/dse/dse1/lib/dse-core-4.8.5.jar:/opt/dse/dse1/lib/dse-hadoop-4.8.5.jar:/opt/dse/dse1/lib/dse-hive...
└─31895 grep -q Error: Exception thrown by the agent : java.lang.NullPointerException
If I then wait a few seconds and try again, I get:
● dse1.service - DataStax Enterprise 1
Loaded: loaded (/etc/systemd/system/dse1.service; static; vendor preset: disabled)
Active: inactive (dead)
Mar 23 13:34:28 pspldsea01p.fleet.ad systemd[1]: Started DataStax Enterprise 1.
Mar 23 13:34:28 pspldsea01p.fleet.ad systemd[1]: Starting DataStax Enterprise 1...
Mar 23 13:38:33 pspldsea01p.fleet.ad systemd[1]: Started DataStax Enterprise 1.
Mar 23 13:38:33 pspldsea01p.fleet.ad systemd[1]: Starting DataStax Enterprise 1...
Mar 23 13:47:41 pspldsea01p.fleet.ad systemd[1]: Started DataStax Enterprise 1.
Mar 23 13:47:41 pspldsea01p.fleet.ad systemd[1]: Starting DataStax Enterprise 1...
Mar 23 13:47:44 pspldsea01p.fleet.ad dse[31267]: nodetool: Failed to connect to '127.0.0.1:7199' - ConnectException: 'Connection refused'.
Mar 23 13:47:57 pspldsea01p.fleet.ad systemd[1]: Started DataStax Enterprise 1.
Mar 23 13:47:57 pspldsea01p.fleet.ad systemd[1]: Starting DataStax Enterprise 1...
Mar 23 13:48:01 pspldsea01p.fleet.ad dse[32004]: nodetool: Failed to connect to '127.0.0.1:7199' - ConnectException: 'Connection refused'.
Hint: Some lines were ellipsized, use -l to show in full.
If I just execute /opt/dse/dse1/bin/dse cassandra -k as the cassandra user, it works fine.
I can't seem to find any additional logging in the normal logging locations or with sudo journalctl -u dse1
Any ideas? Thanks!
It is unfortunate that DataStax Enterprise doesn't come with a systemd service file to be able to use systemctl. However, it does come with an init script. Full documentation is available at the docs
Basically you have two options. The first one is to use the init.d directly, by starting the service:
sudo service dse start
I'm , however, too used to systemctl now to go back to that. So this is my systemd service file
[Unit]
Description=DataStax Enterprise
After=network.target
[Service]
PIDFile=/var/run/dse/dse.pid
ExecStart=/etc/init.d/dse start
ExecStop=/etc/init.d/dse stop
SuccessExitStatus=143
TimeoutSec=300
[Install]
WantedBy=multi-user.target
The init script has many configuration options. For the sake of simplicity, it may be wise to just use those directly in the script. For example, you specify the user in your systemd service file. That was giving me problems until I noticed that the user is already specified in the script. No need to duplicate options.
The SucessExitStatus=143 option is a common configuration for Java applications.
You may have to adapt the location of the script if you didn't install DSE with your package manager
Adding this in case post late but still if this can be useful.
Adding with adding additional parameter RemainAfterExit
/etc/systemd/system/dse1.service
[Unit]
Description=DataStax Enterprise
[Service]
User=cassandra
RemainAfterExit=yes
ExecStart=/opt/dse/dse1/bin/dse cassandra -k
ExecStop=/opt/dse/dse1/bin/dse cassandra-stop
[Install]
WantedBy=multi-user.target
Related
So I'm not sure how to even figure out what's happening. Running redis-server version 6.2.5 on RHEL 7.9. I can start REDIS with the user I want to run it as from the command line. But when I try to start it as a systemd service with the REDIS.service file:
[Unit]
Description=REDIS Service
After=syslog.target network.target remote-fs.target nss-lookup.target
[Service]
Type=simple
User=myuser
Group=myuser
WorkingDirectory=/app/redis
ExecStart=/bin/redis-server /app/redis/primary.conf
Restart=on-failure
[Install]
WantedBy=multi-user.target
It fails. The journalctl entry shows:
Aug 27 08:57:18 computerName redis-server[119338]: 119338:C 27 Aug 2021
08:57:18.981 # Fatal error, can't open config file
'/app/redis/primary.conf': Permission denied
The redis directory and all of it's files are owned by the user I'm telling to run the service. I even went to the extent of putting 777 on the directory and files. Thanks for any help you can give me!
Eric
This server was a mess. There was previous install of redis on the server, the rpm database was corrupt, there were old services that weren't running but were still there. I had to do a lot of cleanup and perform a clean install and it started working.
getting this error during startup time of scylla node as I am loading data
Nov 12 21:55:13 usw1-im-stage-scylladb1 scylla[53703]: [shard 0] database - Keyspace product_prod: Reading CF cleanup_transaction id=bb0a0640-058f-11ea-b8e4-00000000000c version=dde3ee6f-185b-37ba-80fb-6425cce4532f
Nov 12 22:10:02 usw1-im-stage-scylladb1 systemd[1]: scylla-server.service start operation timed out. Terminating.
running this on scylla enterprise 2019.1.2
Scylla's documentation includes KBs and FAQ. I think this KB is spot-on, what you are looking for:
https://docs.scylladb.com/troubleshooting/scylla_wont_start/#solution
Here is the solution suggested there:
Locate the directory with the systemd files where the scylla-server.service resides.
For Centos operating systems it is expected to be under /usr/lib/systemd/system/scylla-server.service
For Ubuntu operating systems it is expected to be under /etc/systemd/system/scylla-server.service.d
Create the following directory (if not exist)
Centos
sudo mkdir /usr/lib/systemd/system/scylla-server.service
Ubuntu
sudo mkdir /etc/systemd/system/scylla-server.service.d
Create a file inside that directory named 10-timeout.conf, with the following contents:
[Service]
TimeoutStartSec=9000
Reload the systemd Daemon for the new configurations to take in effect.
systemctl daemon-reload
I am trying to install a new galera cluster on 3 nodes with centos 7.
When I try to start the cluster on the master node with gcomm:// nothing in starts successfully but I cannot start Mariadb on the other nodes.
systemctl status mariadb.service -l
● mariadb.service - MariaDB database server
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/mariadb.service.d
└─migrated-from-my.cnf-settings.conf
Active: activating (auto-restart) (Result: signal) since Wed 2019-03-06 13:50:12 EET; 720ms ago
Process: 20749 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
Process: 54893 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=killed, signal=ABRT)
Process: 54813 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
Process: 54811 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
Main PID: 54893 (code=killed, signal=ABRT)
CGroup: /system.slice/mariadb.service
├─54902 /bin/sh -ue /usr//bin/wsrep_sst_rsync --role joiner --address 10.1.0.172" --datadir /var/lib/mysql/ --parent 54893
├─54976 rsync --daemon --no-detach --port 4444 --config /var/lib/mysql//rsync_sst.conf
└─55027 sleep 0.5
Mar 06 13:50:12 tms-galeracl2 systemd[1]: mariadb.service: main process exited, code=killed, status=6/ABRT
Mar 06 13:50:12 tms-galeracl2 systemd[1]: Failed to start MariaDB database server.
Mar 06 13:50:12 tms-galeracl2 systemd[1]: Unit mariadb.service entered failed state.
Mar 06 13:50:12 tms-galeracl2 systemd[1]: mariadb.service failed.
and the servers.cnf config
# Mandatory settings
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://10.x.x.x,10.x.x.x,10.x.x.x"
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
#Cluster name
wsrep_cluster_name="galeracl"
#
# Allow server to accept connections on all interfaces.
#
bind-address=0.0.0.0
wsrep_node_address=”10.x.x.x"
wsrep_node_name=”galeracl2"
wsrep_sst_method=rsync
# Optional setting
#wsrep_slave_threads=1
#innodb_flush_log_at_trx_commit=0
You have the "wrong type" of quotes in your config -- in servers.cnf note the opening quote of the following lines:
wsrep_node_address=”10.x.x.x"
wsrep_node_name=”galeracl2"
Edit and replace with the same, plain quote that is used for closing (copy paste from your config above):
wsrep_node_address="10.x.x.x"
wsrep_node_name="galeracl2"
And you should be golden!
I want to see if the running redis-server container status. So i want to execute info in a redis shell and see if the redis slave is in sync with the remote redis master.
My docker-compose file lists the redis as follows:
placements-store:
image: redis:3.0
command: redis-server ${REDIS_OPTIONS}
ports:
- "6379:6379"
Running docker-compose ps I can see it the container is up and running:
app_placements-store_1 docker-entrypoint.sh redis ... Up 0.0.0.0:6379->6379/tcp
I tried to execute:
docker-compose run --rm redis-cli
And got:
ERROR: No such service: redis-cli
I think this is understandable since there's no redis-cli container. I'm trying to tag along to the running redis-server instead but don't have any idea how to do that.
UPDATE
I can view the logs by executing:
docker-compose logs -f --tail 500 placements-store
And I do get some information like below, but I'm looking for more information and something that I can more easily control from the outside:
placements-store_1 | 1:S 06 Feb 19:16:35.427 * Connecting to MASTER mo-api.mydomain.com:6379
placements-store_1 | 1:S 06 Feb 19:16:35.589 * MASTER <-> SLAVE sync started
placements-store_1 | 1:S 06 Feb 19:16:35.889 * Non blocking connect for SYNC fired the event.
placements-store_1 | 1:S 06 Feb 19:16:36.189 * Master replied to PING, replication can continue...
placements-store_1 | 1:S 06 Feb 19:16:36.790 * Partial resynchronization not possible (no cached master)
placements-store_1 | 1:S 06 Feb 19:16:37.091 * Full resync from master: 5ada1d8c65fd49d67d931bea66530a169ce83a40:29442
placements-store_1 | 1:S 06 Feb 19:16:37.145 * MASTER <-> SLAVE sync: receiving 60 bytes from master
placements-store_1 | 1:S 06 Feb 19:16:37.145 * MASTER <-> SLAVE sync: Flushing old data
placements-store_1 | 1:S 06 Feb 19:16:37.145 * MASTER <-> SLAVE sync: Loading DB in memory
placements-store_1 | 1:S 06 Feb 19:16:37.145 * MASTER <-> SLAVE sync: Finished with success
You need to log on to the container using docker exec (as mentioned in another answer - not sure if the command is 100% correct as it may just run redis-cli then exit).
I would run the following command:
docker exec -it app_placements-store_1 sh
That will log you on to the container. You can then run redis-cli from the command prompt.
Use docker exec to execute commands inside a running container:
docker exec -it app_placements-store_1 redis-cli
Although, the answers are already given but I'm going to explain it a bit more with details for the newbies of the docker having such issues.
The command docker exec is used to run a command from the terminal of your machine which for a container.
You can also get the detailed help for the command by executing the command docker exec --help:
Usage: docker exec [OPTIONS] CONTAINER COMMAND [ARG...]
Run a command in a running container
Options:
-d, --detach Detached mode: run command in the background
--detach-keys string Override the key sequence for detaching a
container
-e, --env list Set environment variables
--help Print usage
**-i, --interactive Keep STDIN open even if not attached**
--privileged Give extended privileges to the command
**-t, --tty Allocate a pseudo-TTY**
-u, --user string Username or UID (format:
<name|uid>[:<group|gid>])
This docker exec [OPTIONS] CONTAINER COMMAND [ARG...] clearly shows that you need CONTAINER and the COMMAND along with the arguments. For example: docker exec -it app_placements-store_1 redis-cli
Note: I have displayed the '-iandtas bold in the output ofdocker exec --help` to so you can read it why we are using these options
I am running apache
$ httpd -v
Server version: Apache/2.2.15 (Unix)
Server built: Oct 16 2014 14:48:21
and it runs fine but when I do yum install mod_ssl, apache seg faults. It seg faults right away when I try to list modules using httpd -M.
$ httpd -M
Segmentation fault
If I try to restart apache, it seg faults again
$ sudo service httpd restart
Stopping httpd: [ OK ]
Starting httpd: /bin/bash: line 1: 1627 Segmentation fault /usr/sbin/httpd
[FAILED]
the problem is with CentOS it seems, because I found this resource:
https://chrisjean.com/adding-ssl-support-to-apache-on-centos/
so all I had to do was make sure Apache was loading the SSL module and everything is ok. Hopefully this helps someone else in the future.