I'm quite new with Docker and I'm trying to use docker-compose in order to set up Apache with PostgreSQL all together.
This is my docker-compose.yml:
version: '2'
services:
db:
image: postgres
apache2:
image: webdevops/apache:latest
ports:
- 8084:80
- 443:443
links:
- db
And every time I execute docker-compose up, it usually gets stuck in
AH00094: Command line: 'apache2 -D FOREGROUND -D APACHE_LOCK_DIR'
This is the output when I execute it.
First gets stuck with some LOG: autovacuum launcher started while installing postgres, and when I Ctrl+C it, and re-execute docker-compose, it get stuck in the apache2 -D FOREGROUND -D APACHE_LOCK_DIR line.
Here is the ouput
docker-compose up
Pulling db (postgres:latest)...
latest: Pulling from library/postgres
cd0a524342ef: Pull complete
9c784d04dcb0: Pull complete
d99dddf7e662: Pull complete
e5bff71e3ce6: Pull complete
cb3e0a865488: Pull complete
31295d654cd5: Pull complete
fc930a4e09f5: Pull complete
8650cce8ef01: Pull complete
61949acd8e52: Pull complete
527a203588c0: Pull complete
26dec14ac775: Pull complete
0efc0ed5a9e5: Pull complete
40cd26695b38: Pull complete
Digest: sha256:fd6c0e2a9d053bebb294bb13765b3e01be7817bf77b01d58c2377ff27a4a46dc
Status: Downloaded newer image for postgres:latest
Pulling apache2 (webdevops/apache:latest)...
latest: Pulling from webdevops/apache
aafe6b5e13de: Pull complete
0a2b43a72660: Pull complete
18bdd1e546d2: Pull complete
8198342c3e05: Pull complete
f56970a44fd4: Pull complete
7d255d8cc29c: Pull complete
6b76599eeb74: Pull complete
ac6a5bdae794: Pull complete
bfe65de9caf3: Pull complete
45f407574293: Pull complete
5f28b0c8228a: Pull complete
ceb4b3de72bc: Pull complete
Digest: sha256:acf3049ddc1cf7f615eb4751250881d31dccfc50752ac3a1261169b684f430cb
Status: Downloaded newer image for webdevops/apache:latest
Creating dockertest7_db_1
Creating dockertest7_apache2_1
Attaching to dockertest7_db_1, dockertest7_apache2_1
apache2_1 | 2017-05-09 01:36:54,332 CRIT Set uid to user 0
apache2_1 | 2017-05-09 01:36:54,332 WARN Included extra file "/opt/docker/etc/supervisor.d/apache.conf" during parsing
apache2_1 | 2017-05-09 01:36:54,332 WARN Included extra file "/opt/docker/etc/supervisor.d/cron.conf" during parsing
apache2_1 | 2017-05-09 01:36:54,332 WARN Included extra file "/opt/docker/etc/supervisor.d/dnsmasq.conf" during parsing
apache2_1 | 2017-05-09 01:36:54,332 WARN Included extra file "/opt/docker/etc/supervisor.d/postfix.conf" during parsing
apache2_1 | 2017-05-09 01:36:54,332 WARN Included extra file "/opt/docker/etc/supervisor.d/ssh.conf" during parsing
apache2_1 | 2017-05-09 01:36:54,332 WARN Included extra file "/opt/docker/etc/supervisor.d/syslog.conf" during parsing
apache2_1 | 2017-05-09 01:36:54,340 INFO RPC interface 'supervisor' initialized
apache2_1 | 2017-05-09 01:36:54,340 INFO supervisord started with pid 1
db_1 | The files belonging to this database system will be owned by user "postgres".
db_1 | This user must also own the server process.
db_1 |
db_1 | The database cluster will be initialized with locale "en_US.utf8".
db_1 | The default database encoding has accordingly been set to "UTF8".
db_1 | The default text search configuration will be set to "english".
db_1 |
db_1 | Data page checksums are disabled.
db_1 |
db_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1 | creating subdirectories ... ok
db_1 | selecting default max_connections ... 100
db_1 | selecting default shared_buffers ... 128MB
db_1 | selecting dynamic shared memory implementation ... posix
db_1 | creating configuration files ... ok
apache2_1 | 2017-05-09 01:36:55,345 INFO spawned: 'apached' with pid 17
apache2_1 | 2017-05-09 01:36:55,404 INFO success: apached entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
apache2_1 | [Tue May 09 01:36:55.414084 2017] [mpm_event:notice] [pid 17:tid 140039054796672] AH00489: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
apache2_1 | [Tue May 09 01:36:55.414134 2017] [core:notice] [pid 17:tid 140039054796672] AH00094: Command line: 'apache2 -D FOREGROUND -D APACHE_LOCK_DIR'
db_1 | running bootstrap script ... ok
db_1 | performing post-bootstrap initialization ... ok
db_1 | syncing data to disk ...
db_1 | WARNING: enabling "trust" authentication for local connections
db_1 | You can change this by editing pg_hba.conf or using the option -A, or
db_1 | --auth-local and --auth-host, the next time you run initdb.
db_1 | ok
db_1 |
db_1 | Success. You can now start the database server using:
db_1 |
db_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1 |
db_1 | ****************************************************
db_1 | WARNING: No password has been set for the database.
db_1 | This will allow anyone with access to the
db_1 | Postgres port to access your database. In
db_1 | Docker's default configuration, this is
db_1 | effectively any other container on the same
db_1 | system.
db_1 |
db_1 | Use "-e POSTGRES_PASSWORD=password" to set
db_1 | it in "docker run".
db_1 | ****************************************************
db_1 | waiting for server to start....LOG: database system was shut down at 2017-05-09 01:36:56 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
db_1 | done
db_1 | server started
db_1 | ALTER ROLE
db_1 |
db_1 |
db_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1 |
db_1 | LOG: received fast shutdown request
db_1 | LOG: aborting any active transactions
db_1 | waiting for server to shut down...LOG: autovacuum launcher shutting down
db_1 | .LOG: shutting down
db_1 | LOG: database system is shut down
db_1 | done
db_1 | server stopped
db_1 |
db_1 | PostgreSQL init process complete; ready for start up.
db_1 |
db_1 | LOG: database system was shut down at 2017-05-09 01:36:58 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
Here I cancel it, cause it got stuck, and execute docker-compose again:
docker-compose up
Starting dockertest7_db_1
Starting dockertest7_apache2_1
Attaching to dockertest7_db_1, dockertest7_apache2_1
db_1 | LOG: database system was interrupted; last known up at 2017-05-09 01:37:00 UTC
db_1 | LOG: database system was not properly shut down; automatic recovery in progress
db_1 | LOG: invalid record length at 0/14EE238: wanted 24, got 0
db_1 | LOG: redo is not required
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
apache2_1 | 2017-05-09 01:41:17,903 CRIT Set uid to user 0
apache2_1 | 2017-05-09 01:41:17,903 WARN Included extra file "/opt/docker/etc/supervisor.d/apache.conf" during parsing
apache2_1 | 2017-05-09 01:41:17,903 WARN Included extra file "/opt/docker/etc/supervisor.d/cron.conf" during parsing
apache2_1 | 2017-05-09 01:41:17,903 WARN Included extra file "/opt/docker/etc/supervisor.d/dnsmasq.conf" during parsing
apache2_1 | 2017-05-09 01:41:17,903 WARN Included extra file "/opt/docker/etc/supervisor.d/postfix.conf" during parsing
apache2_1 | 2017-05-09 01:41:17,903 WARN Included extra file "/opt/docker/etc/supervisor.d/ssh.conf" during parsing
apache2_1 | 2017-05-09 01:41:17,903 WARN Included extra file "/opt/docker/etc/supervisor.d/syslog.conf" during parsing
apache2_1 | Unlinking stale socket /.supervisor.sock
apache2_1 | 2017-05-09 01:41:18,226 INFO RPC interface 'supervisor' initialized
apache2_1 | 2017-05-09 01:41:18,226 INFO supervisord started with pid 1
apache2_1 | 2017-05-09 01:41:19,230 INFO spawned: 'apached' with pid 17
apache2_1 | 2017-05-09 01:41:19,307 INFO success: apached entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
apache2_1 | [Tue May 09 01:41:19.316042 2017] [mpm_event:notice] [pid 17:tid 139733962508160] AH00489: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g configured -- resuming normal operations
apache2_1 | [Tue May 09 01:41:19.316085 2017] [core:notice] [pid 17:tid 139733962508160] AH00094: Command line: 'apache2 -D FOREGROUND -D APACHE_LOCK_DIR'
Am I missing something? Maybe a basic stuff, configuration or something?
There are couple things with this installation.
One is obviously postgres that did not start up properly, and is now left in unstable state. Following article shows how to fix that.
For the rest, I cannot tell if you are having some incompatibility issues with systemd, or it is a consequence of the stalled db install, and then compose not starting up the dependencies (apache) properly.
Either way, I would remove all images, all temporary files, and try to rebuild and restart.
Related
I am trying to install and configure sonarqube to use it in small poroyect in my machine (Ubuntu 18.04.2 LTS) y have already download the zip Sonarqube 7.7 community edition from the oficial web and unzipped in /opt/. I tryed to do as is say in the guide from sonar and execute ./sonar.sh console in the path /opt/sonarqube/bin/linux-x86-64 and I get problems with the log permisions .
Running SonarQube...
wrapper | ERROR: Could not write pid file /opt/sonarqube/bin/linux-x86-64/./SonarQube.pid: Permission denied
Unable to open logfile ../../logs/sonar.log: Permission denied
So I try the same as superuser and I get
Running SonarQube...
wrapper | --> Wrapper Started as Console
wrapper | Launching a JVM...
jvm 1 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
jvm 1 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
jvm 1 |
jvm 1 | 2019.04.05 19:55:48 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
jvm 1 | 2019.04.05 19:55:48 INFO app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001
jvm 1 | 2019.04.05 19:55:48 INFO app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch
jvm 1 | 2019.04.05 19:55:48 INFO app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
jvm 1 | 2019.04.05 19:55:48 INFO app[][o.e.p.PluginsService] no modules loaded
jvm 1 | 2019.04.05 19:55:48 INFO app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
jvm 1 | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
jvm 1 | 2019.04.05 19:55:51 WARN app[][o.s.a.p.AbstractProcessMonitor] Process exited with exit value [es]: 1
jvm 1 | 2019.04.05 19:55:51 INFO app[][o.s.a.SchedulerImpl] Process [es] is stopped
jvm 1 | 2019.04.05 19:55:51 INFO app[][o.s.a.SchedulerImpl] SonarQube is stopped
jvm 1 | 2019.04.05 19:55:51 INFO app[][o.e.c.t.TransportClientNodesService] failed to get node info for {#transport#-1}{pr4ED0vsRgCYZjgn1jbLlQ}{127.0.0.1}{127.0.0.1:9001}, disconnecting...
jvm 1 | java.lang.IllegalStateException: Future got interrupted
jvm 1 | at org.elasticsearch.common.util.concurrent.FutureUtils.get(FutureUtils.java:60)
jvm 1 | at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:34)
jvm 1 | at org.elasticsearch.transport.ConnectionManager.internalOpenConnection(ConnectionManager.java:226)
jvm 1 | at org.elasticsearch.transport.ConnectionManager.openConnection(ConnectionManager.java:85)
jvm 1 | at org.elasticsearch.transport.TransportService.openConnection(TransportService.java:367)
jvm 1 | at org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler.doSample(TransportClientNodesService.java:410)
jvm 1 | at org.elasticsearch.client.transport.TransportClientNodesService$NodeSampler.sample(TransportClientNodesService.java:361)
jvm 1 | at org.elasticsearch.client.transport.TransportClientNodesService.addTransportAddresses(TransportClientNodesService.java:202)
jvm 1 | at org.elasticsearch.client.transport.TransportClient.addTransportAddress(TransportClient.java:342)
jvm 1 | at org.sonar.application.es.EsConnectorImpl$MinimalTransportClient.<init>(EsConnectorImpl.java:108)
jvm 1 | at org.sonar.application.es.EsConnectorImpl.buildTransportClient(EsConnectorImpl.java:89)
jvm 1 | at org.sonar.application.es.EsConnectorImpl.getTransportClient(EsConnectorImpl.java:74)
jvm 1 | at org.sonar.application.es.EsConnectorImpl.getClusterHealthStatus(EsConnectorImpl.java:61)
jvm 1 | at org.sonar.application.process.EsProcessMonitor.checkStatus(EsProcessMonitor.java:90)
jvm 1 | at org.sonar.application.process.EsProcessMonitor.checkOperational(EsProcessMonitor.java:75)
jvm 1 | at org.sonar.application.process.EsProcessMonitor.isOperational(EsProcessMonitor.java:60)
jvm 1 | at org.sonar.application.process.SQProcess.refreshState(SQProcess.java:161)
jvm 1 | at org.sonar.application.process.SQProcess$EventWatcher.run(SQProcess.java:220)
jvm 1 | Caused by: java.lang.InterruptedException: null
jvm 1 | at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1038)
jvm 1 | at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1343)
jvm 1 | at org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java:251)
jvm 1 | at org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:94)
jvm 1 | at org.elasticsearch.common.util.concurrent.FutureUtils.get(FutureUtils.java:57)
jvm 1 | ... 17 common frames omitted
wrapper | <-- Wrapper Stopped
I do not understand the error I get and why it does not start the sonarqube.
Don't try to use superuser. Just change the access rights on the complete SonarQube folder for the user you use.
I am trying to build a Docker image with Apache (httpd) and PHP 5.3.3. The image is based on a CentOS 6 && Supervisord that I have made. So far everything is working "fine" (and this means no issues during build process) but I am having some issues with Apache and Supervisord. Why? Because I think the image based on reynierpm/centos6-supervisord is already starting Apache and then Supervisord is trying to do the same thing. Here is a log of what I am getting when I start my stack using docker-compose up:
webserver_1 | 2018-02-03 19:29:52,882 CRIT Supervisor running as root (no user in config file)
webserver_1 | 2018-02-03 19:29:52,882 INFO Included extra file "/etc/supervisor.d/httpd.conf" during parsing
webserver_1 | 2018-02-03 19:29:52,913 INFO RPC interface 'supervisor' initialized
webserver_1 | 2018-02-03 19:29:52,920 INFO supervisord started with pid 8
webserver_1 | 2018-02-03 19:29:53,923 INFO spawned: 'httpd' with pid 11
webserver_1 | 2018-02-03 19:29:53,928 INFO spawned: 'httpd-reload' with pid 12
webserver_1 | 2018-02-03 19:29:53,964 DEBG 'httpd' stderr output:
webserver_1 | httpd: Could not reliably determine the server's fully qualified domain name, using 172.18.0.2 for ServerName
webserver_1 |
webserver_1 | 2018-02-03 19:29:53,965 DEBG fd 9 closed, stopped monitoring <POutputDispatcher at 41891168 for <Subprocess at 140421355793008 with name httpd in state STARTING> (stderr)>
webserver_1 | 2018-02-03 19:29:53,979 DEBG fd 7 closed, stopped monitoring <POutputDispatcher at 41890736 for <Subprocess at 140421355793008 with name httpd in state STARTING> (stdout)>
webserver_1 | 2018-02-03 19:29:53,980 INFO exited: httpd (exit status 0; not expected)
webserver_1 | 2018-02-03 19:29:53,981 DEBG received SIGCLD indicating a child quit
webserver_1 | 2018-02-03 19:29:54,983 INFO spawned: 'httpd' with pid 23
webserver_1 | 2018-02-03 19:29:54,985 INFO success: httpd-reload entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
I have created a gist having the Dockerfile and the httpd.conf file read by Supervisord, what I am doing wrong here?
I have a jenkins set up on a windows 2012 R2 server. We have added the plugin SeleniumHQ htmlsuite run plugin. We can execute the call manually on the server and everything runs fine. When we run it from Jenkins we get this error:
2017-07-19 10:41:46.603:INFO::main: Logging initialized #8197ms to org.seleniumhq.jetty9.util.log.StdErrLog
2017-07-19 10:41:46.666:INFO:osjs.Server:main: jetty-9.4.z-SNAPSHOT
2017-07-19 10:41:46.697:INFO:osjsh.ContextHandler:main: Started o.s.j.s.h.ContextHandler#fbd1f6{/tests,null,AVAILABLE}
2017-07-19 10:41:46.713:INFO:osjs.AbstractConnector:main: Started ServerConnector#1c0f706{HTTP/1.1,[http/1.1]}{0.0.0.0:39902}
2017-07-19 10:41:46.713:INFO:osjs.Server:main: Started #8313ms
Jul 19, 2017 10:41:48 AM org.openqa.selenium.server.htmlrunner.CoreTestCase run
INFO: |open | http://10.33.2.51/User/AccessSignin/Start | |
Jul 19, 2017 10:41:51 AM org.openqa.selenium.server.htmlrunner.CoreTestCase run
INFO: |type | id=UsernameField | badroutingtesting |
Jul 19, 2017 10:41:51 AM org.openqa.selenium.server.htmlrunner.CoreTestCase run
INFO: |clickAndWait | id=SubmitNext | |
Jul 19, 2017 10:41:52 AM org.openqa.selenium.server.htmlrunner.CoreTestCase run
INFO: |click | id=PasswordField | |
SmallTest.html
SmallTest</td></tr>
</tbody></table>
2017-07-19 10:41:53.057:INFO:osjs.AbstractConnector:main: Stopped ServerConnector#1c0f706{HTTP/1.1,[http/1.1]}{0.0.0.0:39902}
2017-07-19 10:41:53.057:INFO:osjsh.ContextHandler:main: Stopped o.s.j.s.h.ContextHandler#fbd1f6{/tests,null,UNAVAILABLE}
Jul 19, 2017 10:41:53 AM org.openqa.selenium.remote.ErrorCodes toStatus
INFO: HTTP Status: '404' -> incorrect JSON status mapping for 'unknown error' (500 expected)
Jul 19, 2017 10:41:54 AM org.openqa.selenium.server.htmlrunner.HTMLLauncher mainInt
WARNING: Test of browser failed: *firefox
org.openqa.selenium.WebDriverException: quit
It seems to me that it's not executing the click or it's not able to get to the next page. Same things happens for IE and Chrome. Could it be a permissions issue? Or is it a configuration with Jenkins? I'm a bit stumped
I am trying to run a oozie sqoop job to import from teradata to Hive.
Sqoop runs fine in CLI. But I am facing the issues in scheduling it with oozie.
Note: I am able to do shell actions in oozie and it works fine.
Find below the error logs and workflow
Error logs:
Log Type: stderr
Log Upload Time: Wed Feb 01 04:19:00 -0500 2017
Log Length: 513
log4j:ERROR Could not find value for key log4j.appender.CLA
log4j:ERROR Could not instantiate appender named "CLA".
log4j:WARN No appenders could be found for logger (org.apache.hadoop.yarn.client.RMProxy).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
No such sqoop tool: sqoop. See 'sqoop help'.
Intercepting System.exit(1)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]
Log Type: stdout
Log Upload Time: Wed Feb 01 04:19:00 -0500 2017
Log Length: 158473
Showing 4096 bytes of 158473 total. Click here for the full log.
curity.ShellBasedUnixGroupsMapping
dfs.client.domain.socket.data.traffic=false
dfs.client.read.shortcircuit.streams.cache.size=256
fs.s3a.connection.timeout=200000
dfs.datanode.block-pinning.enabled=false
mapreduce.job.end-notification.max.retry.interval=5000
yarn.acl.enable=true
yarn.nm.liveness-monitor.expiry-interval-ms=600000
mapreduce.application.classpath=$HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,$MR2_CLASSPATH
mapreduce.input.fileinputformat.list-status.num-threads=1
dfs.client.mmap.cache.size=256
mapreduce.tasktracker.map.tasks.maximum=2
yarn.scheduler.fair.user-as-default-queue=true
yarn.timeline-service.ttl-enable=true
yarn.nodemanager.linux-container-executor.resources-handler.class=org.apache.hadoop.yarn.server.nodemanager.util.DefaultLCEResourcesHandler
dfs.namenode.max.objects=0
dfs.namenode.service.handler.count=10
dfs.namenode.kerberos.principal.pattern=*
yarn.resourcemanager.state-store.max-completed-applications=${yarn.resourcemanager.max-completed-applications}
dfs.namenode.delegation.token.max-lifetime=604800000
mapreduce.job.classloader=false
yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size=10000
mapreduce.job.hdfs-servers=${fs.defaultFS}
yarn.application.classpath=$HADOOP_CLIENT_CONF_DIR,$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,$HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,$HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*
dfs.datanode.hdfs-blocks-metadata.enabled=true
mapreduce.tasktracker.dns.nameserver=default
dfs.datanode.readahead.bytes=4193404
mapreduce.job.ubertask.maxreduces=1
dfs.image.compress=false
mapreduce.shuffle.ssl.enabled=false
yarn.log-aggregation-enable=false
mapreduce.tasktracker.report.address=127.0.0.1:0
mapreduce.tasktracker.http.threads=40
dfs.stream-buffer-size=4096
tfile.fs.output.buffer.size=262144
fs.permissions.umask-mode=022
dfs.client.datanode-restart.timeout=30
dfs.namenode.resource.du.reserved=104857600
yarn.resourcemanager.am.max-attempts=2
yarn.nodemanager.resource.percentage-physical-cpu-limit=100
ha.failover-controller.graceful-fence.connection.retries=1
mapreduce.job.speculative.speculative-cap-running-tasks=0.1
hadoop.proxyuser.hdfs.groups=*
dfs.datanode.drop.cache.behind.writes=false
hadoop.proxyuser.HTTP.hosts=*
hadoop.common.configuration.version=0.23.0
mapreduce.job.ubertask.enable=false
yarn.app.mapreduce.am.resource.cpu-vcores=1
dfs.namenode.replication.work.multiplier.per.iteration=2
mapreduce.job.acl-modify-job=
io.seqfile.local.dir=${hadoop.tmp.dir}/io/local
yarn.resourcemanager.system-metrics-publisher.enabled=false
fs.s3.sleepTimeSeconds=10
mapreduce.client.output.filter=FAILED
------------------------
Sqoop command arguments :
sqoop
import
--connect
"jdbc:teradata://xx.xxx.xx:xxxx/DATABASE=Database_name"
--verbose
--username
xxx
-password
'xxx'
--table
BILL_DETL_EXTRC
--split-by
EXTRC_RUN_ID
--m
1
--fields-terminated-by
,
--hive-import
--hive-table
OPS_TEST.bill_detl_extr213
--target-dir
/hadoop/dev/TD_archive/bill_detl_extrc
Fetching child yarn jobs
tag id : oozie-56ea2084fcb1d55591f8919b405f0be0
Child yarn jobs are found -
=================================================================
Invoking Sqoop command line now >>>
3324 [uber-SubtaskRunner] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
Intercepting System.exit(1)
<<< Invocation of Main class completed <<<
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]
Oozie Launcher failed, finishing Hadoop job gracefully
Oozie Launcher, uploading action data to HDFS sequence file: hdfs://namenode:8020/user/hadoopadm/oozie-oozi/0000039-170123205203054-oozie-oozi-W/sqoop-action--sqoop/action-data.seq
Oozie Launcher ends
Log Type: syslog
Log Upload Time: Wed Feb 01 04:19:00 -0500 2017
Log Length: 16065
Showing 4096 bytes of 16065 total. Click here for the full log.
adoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Job jar is not present. Not adding any jar to the list of resources.
2017-02-01 04:18:51,990 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /user/hadoopadm/.staging/job_1485220715968_0219/job.xml
2017-02-01 04:18:52,074 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #5 tokens and #1 secret keys for NM use for launching container
2017-02-01 04:18:52,074 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 6
2017-02-01 04:18:52,074 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData
2017-02-01 04:18:52,174 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://svacld001.bcbsnc.com:8020]
2017-02-01 04:18:52,240 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapred.JobConf: Task java-opts do not specify heap size. Setting task attempt jvm max heap size to -Xmx820m
2017-02-01 04:18:52,243 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1485220715968_0219_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED
2017-02-01 04:18:52,243 INFO [uber-EventHandler] org.apache.hadoop.mapred.LocalContainerLauncher: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1485220715968_0219_01_000001 taskAttempt attempt_1485220715968_0219_m_000000_0
2017-02-01 04:18:52,245 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1485220715968_0219_m_000000_0] using containerId: [container_1485220715968_0219_01_000001 on NM: [svacld005.bcbsnc.com:8041]
2017-02-01 04:18:52,246 INFO [uber-SubtaskRunner] org.apache.hadoop.mapred.LocalContainerLauncher: mapreduce.cluster.local.dir for uber task: /disk1/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk10/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk11/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk12/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk2/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk3/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk4/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk5/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk6/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk7/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk8/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219,/disk9/yarn/nm/usercache/hadoopadm/appcache/application_1485220715968_0219
2017-02-01 04:18:52,247 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1485220715968_0219_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING
2017-02-01 04:18:52,247 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1485220715968_0219_m_000000 Task Transitioned from SCHEDULED to RUNNING
2017-02-01 04:18:52,249 INFO [uber-SubtaskRunner] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2017-02-01 04:18:52,258 INFO [uber-SubtaskRunner] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]
2017-02-01 04:18:52,324 INFO [uber-SubtaskRunner] org.apache.hadoop.mapred.MapTask: Processing split: org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit#9c73765
2017-02-01 04:18:52,329 INFO [uber-SubtaskRunner] org.apache.hadoop.mapred.MapTask: numReduceTasks: 0
2017-02-01 04:18:52,340 INFO [uber-SubtaskRunner] org.apache.hadoop.conf.Configuration.deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
WORKFLOW
<workflow-app xmlns="uri:oozie:workflow:0.5" name="oozie-wf">
<start to="sqoop-wf"/>
<action name="sqoop-wf">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>xx.xx.xx:8032</job-tracker>
<name-node>hdfs://xx.xxx.xx:8020</name-node>
<command>import --connect "jdbc:teradata://ip/DATABASE=EDW_EXTRC_TAB_HST" --connection-manager "com.cloudera.connector.teradata.TeradataManager" --verbose --username HADOOP -password 'xxxxx' --table BILL_DETL_EXTRC --split-by EXTRC_RUN_ID --m 1 --fields-terminated-by , --hive-import --hive-table OPS_TEST.bill_detl_extrc1 --target-dir /hadoop/dev/TD_archive/data/PDCRDATA_TEST/bill_detl_extrc </command>
</sqoop>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Failed, Error Message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
JOB PROPERTIES
oozie.wf.application.path=hdfs:///hadoop/dev/TD_archive/workflow1.xml
oozie.use.system.libpath=true
security_enabled=True
dryrun=False
jobtracker=xxx.xxx:8032
nameNode=hdfs://xx.xx:8020
NOTE:
We are using cloudera CDH5.5
All the necessary JARS (sqoop-connector-teradata-1.5c5.jar tdgssconfig.jar terajdbc4.jar) are placed in /var/lib/sqoop and as well as placed in HDFS too.
I try to use rabbit on ubuntu 12.04. After installation rabbitmq-server works fine. Than I stop it and add my configuration file.
root#rabbit1:~# tail /etc/rabbitmq/rabbitmq-env.conf
RABBITMQ_CONFIG_FILE=/etc/rabbitmq/myrabbitmq
root#rabbit1:~# tail /etc/rabbitmq/myrabbitmq.config
[{rabbit,
[{cluster_nodes, {['rabbit#rabbit1', 'rabbit#rabbit2'], disc}}]}].
With this files rabbitmq-server says on start:
root#rabbit1:~# rabbitmq-server
Activating RabbitMQ plugins ...
0 plugins activated:
+---+ +---+
| | | |
| | | |
| | | |
| +---+ +-------+
| |
| RabbitMQ +---+ |
| | | |
| v2.7.1 +---+ |
| |
+-------------------+
AMQP 0-9-1 / 0-9 / 0-8
Copyright (C) 2007-2011 VMware, Inc.
Licensed under the MPL. See http://www.rabbitmq.com/
node : rabbit#rabbit1
app descriptor : /usr/lib/rabbitmq/lib/rabbitmq_server-2.7.1/sbin/../ebin/rabbit.app
home dir : /var/lib/rabbitmq
config file(s) : /etc/rabbitmq/myrabbitmq.config
cookie hash : 31CaH3BCSDNL1hDIFQzH2Q==
log : /var/log/rabbitmq/rabbit#rabbit1.log
sasl log : /var/log/rabbitmq/rabbit#rabbit1-sasl.log
database dir : /var/lib/rabbitmq/mnesia/rabbit#rabbit1
erlang version : 5.8.5
-- rabbit boot start
starting file handle cache server ...done
starting worker pool ...done
starting database ...BOOT ERROR: FAILED
Reason: function_clause
Stacktrace: [{lists,usort,[{[rabbit#rabbit1,rabbit#rabbit2],disc}]},
{rabbit_mnesia,init_db,3},
{rabbit_mnesia,init,0},
{rabbit,'-run_boot_step/1-lc$^1/1-1-',1},
{rabbit,run_boot_step,1},
{rabbit,'-start/2-lc$^0/1-0-',1},
{rabbit,start,2},
{application_master,start_it_old,4}]
Erlang has closed
{"Kernel pid terminated",application_controller,"{application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}})
Have any suggestion what's wrong with my rabbit?
Configuration file should be like this:
root#rabbit1:~# cat /etc/rabbitmq/myrabbitmq.config
[{rabbit,
[{cluster_nodes, ['rabbit#rabbit1', 'rabbit#rabbit2'] }]}].
It seems, that disc or ram node configures in different way than in documentation to latest version of rabbit. In this configuration two disc nodes will be created. If somebody wants ram node this node should be ommitted in configuration of this node (not in other config files).