FIP Deployment Error - tooltwist

We are deploying on one of our server and we have below error.
ERROR: tooltwist.fip.FipException Unknown response from server: 500: Internal Server Error
Exception: tooltwist.fip.FipException: tooltwist.fip.FipException: Unknown response from server: 500: Internal Server Error
Looking at the FIP log, it shows:
Error installing batch: tooltwist.fip.FipException: Pre-commit command failed: protected/pre_commit.sh
tooltwist.fip.FipException: Pre-commit command failed: protected/pre_commit.sh
at tooltwist.fip.FipServer_updateExecuter.commitTransaction(FipServer_updateExecuter.java:309)
at tooltwist.fip.FipServer_updateExecuter.prepareUpdates_1_3(FipServer_updateExecuter.java:250)
at tooltwist.fip.FipServer_updateExecuter.executeUpdates(FipServer_updateExecuter.java:142)
at tooltwist.fip.FipServer.destination_installBatchOfFiles(FipServer.java:199)
at tooltwist.fip.jetty.InstallBatchServlet.doPost(InstallBatchServlet.java:134)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:530)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:426)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:931)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:361)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:867)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113)
at org.eclipse.jetty.server.Server.handle(Server.java:337)
at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:581)
at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1020)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:775)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:228)
at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:417)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:474)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:437)
at java.lang.Thread.run(Thread.java:662)
Any idea about the error?

Couple of suggestions:
1. Does the pre_commit.sh shell script exist on the server.
2. Does it have +x permissions?
If FIP was installed using the normal way, it should not cause an issue.

The fipserver initially saves files it receives on the destination server, but into temporary locations. Once all of the files have been received and saved it runs a three step process to complete the installation:
1. Run a script named protected/pre-commit.sh. The normal operation of this script is to shut down the web server.
2. For each new file:
a) move any existing file to .fip-rollback-xxxxxx/filename.
b) move the new file from it's temporary location to the correct location.
3. Run a script named protected/post-commit.sh. This most commonly restarts the server.
The pre and post commit scripts are user provided. They should should normally exit with a status of zero, as any other status indicates that an error has occurred.
As suggested in the previous answer, check that these scripts exist, and that they are executable. If this fails to solve your problem, insert debug into the scripts to determine where and why they are failing.

Related

working on dbt want to give the details of snowflake in project.yml file but getting error

Trying the give the details of snowfalke in dbt profiles.yml file . but as soon as when ran the command i,e
$atom /home/myname/.dbt/profiles.yml gives below error:
/usr/bin/atom: line 190: 1705 Trace/breakpoint trap (core dumped) nohup "$ATOM_PATH" --executed-from="$(pwd)" --pid=$$ "$#" > "$ATOM_HOME/nohup.out" 2>&1
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Permission denied
Following things i tried: Ran below commands but still no luck:
1)
$google-chrome --no-gpu --no-sandbox --disable-setuid-sandbox --headless --dump-dom http://www.chromestatus.com
Error:
[0627/161930.251811:ERROR:udev_watcher.cc(61)] Failed to enable receiving udev events.
[0627/161932.565713:ERROR:platform_shared_memory_region_posix.cc(46)] Descriptor access mode (0) differs from expected (2)
[0627/161932.566251:WARNING:crash_handler_host_linux.cc(366)] Could not translate tid - assuming crashing thread is thread group leader; syscall_supported=0
[0627/161932.769040:WARNING:crash_handler_host_linux.cc(366)] Could not translate tid - assuming crashing thread is thread group leader; syscall_supported=0
--2020-06-27 16:19:32-- https://clients2.google.com/cr/report
Resolving clients2.google.com (clients2.google.com)... 2404:6800:4009:805::200e, 172.217.174.238
Connecting to clients2.google.com (clients2.google.com)|2404:6800:4009:805::200e|:443... [0627/161933.036124:ERROR:headless_shell.cc(399)] Abnormal renderer termination.
connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
Saving to: ‘/dev/fd/4’
0K
Crash dump id: e870824b56e91b9f
$ google-chrome
Error:
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Permission denied
Trace/breakpoint trap (core dumped)
[1772:1772:0100/000000.825375:ERROR:zygote_linux.cc(653)] write: Broken pipe (32)
[0627/162152.831614:ERROR:nacl_helper_linux.cc(308)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
Could any one advise on the above issue.
This could be an atom error. Could be packages or something that they installed. I could be wrong, but I don’t see anything in that error message that indicates it’s a profiles.yml thing. I wonder if they can open other files in atom just fine?
Alternatively, use a text editor that isn’t based on chrome
Thanks #jake and #Christine from FishTown Analytics.
Here are a few helpful links.
Atom issue;
Blogpost!

Detected failed migration to version Flyway

I added a sql file to my project and now I am receiving the following error:
nested exception is org.flywaydb.core.api.FlywayException: Validate failed: Detected failed migration to version 1.0.9 (update)
Here is my sql file I'm adding:
ALTER TABLE `episodes`
ADD COLUMN `rating` TINYINT(1) NULL DEFAULT NULL;
I added the same query to MySQL Workbench and it's working fine, so I think the error here is beyond the sql file added
UPDATE: full stack trace below
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException: Validate failed: Detected failed migration to version 1.0.9 (update)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1708)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:581)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:503)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:304)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1089)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:859)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:395)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:327)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1255)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1243)
at com.nbcuni.cds.Application.main(Application.java:12)
Caused by: org.flywaydb.core.api.FlywayException: Validate failed: Detected failed migration to version 1.0.9 (update)
at org.flywaydb.core.Flyway.doValidate(Flyway.java:1286)
at org.flywaydb.core.Flyway.access$100(Flyway.java:71)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:1176)
at org.flywaydb.core.Flyway$1.execute(Flyway.java:1168)
at org.flywaydb.core.Flyway.execute(Flyway.java:1655)
at org.flywaydb.core.Flyway.migrate(Flyway.java:1168)
at org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:66)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1767)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1704)
... 18 common frames omitted
UPDATE: When I run mvn:flyway-validate, I receive the following error:
org.flywaydb.core.api.FlywayException: Unable to connect to the database. Configure the url, user and password!
I'm not sure where to configure it? It's already set in my applications.properties file? And w/o this version the spring application is working fine
I guess the problem is the error in migration history saved on your database.
Try deleting the line that failed and try again. If the error won't vanish, the problem is somehow in your statement.
I just had the same issue :)
below resolved the validate problem for me:
mvn flyway:validate -Dflyway.configFile=myFlywayConfig.properties
The error happen if there is any record is inserted into the 'flyway_schema_history' with failed status or part of the failed migration might already run. Just need to remove failed recored from the 'flyway_schema_history' then run "flyway migrate" again.
Query to flyway_schema_history table
select * from your_database_name.flyway_schema_history
You will get something like this:
Remove the record with success =0 and run again "flyway migrate".
Note, that this table might have a different name depending on the configuration

executeprocess error in nifi

i have hive in hdinsight cluster and nifi in my local machine.
i am trying to execute a hive script from executeprocess processor which has properties set as below:
command: hive
command argument: -f /home/name/firstq.hql
Redirect Error Stream: true
i have controller services to hiveconnection pool. when i start the processor, the error is thrown as shown below:
o.a.n.processors.standard.ExecuteProcess ExecuteProcess[id=d5db18b2-0159-1000-6569-c054490cbfa5] Failed to create process due to java.io.IOException: Cannot run program "hive": CreateProcess error=2, The system cannot find the file specified: java.io.IOException: Cannot run program "hive": CreateProcess error=2, The system cannot find the file specified
org.apache.nifi.util.ReflectionUtils Failed while invoking annotated method 'public void org.apache.nifi.processors.standard.ExecuteProcess.shutdownExecutor()' with arguments '[]'.
i have tried the command argument by giving the local machine path too. though same error is thrown.
in the script, i am trying to insert one row into the existing table.
please help me what am i doing wrong.
thanks

Cargo start up error

I am running 'mvn clean install -Dmaven.test.skip=true' command for a web application. but it gives following error. I set the catalina home as an environment variable. i am using maven 3.1.1 version. Please help me
FATAL ERROR in native method: JDWP No transports i
nitialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197)
[WARNING] [talledLocalContainer] ERROR: transport error 202: bind failed: Addres
s already in use
[ERROR] Failed to execute goal org.codehaus.cargo:cargo-maven2-plugin:1.2.4:star
t (start-container) on project RCMigrationWebApp: Execution start-container of g
oal org.codehaus.cargo:cargo-maven2-plugin:1.2.4:start failed: Failed to start t
he Tomcat 7.x container. Deployable [http://localhost:8080/cargocpc/index.html]
failed to finish deploying within the timeout period [120000]. The Deployable st
ate is thus unknown. -> [Help 1]
Probably you have to kill any process running at 8080, looks like the port is already in use..
ERROR: transport error 202: bind failed: Address already in use
Port in use. Run netstat -bv if you are using Windows. You will get to know which process is holding up the port. With your stackTrace, it looks very obvious.
Let me know if it helps.

Zookeeper error connection loss exception

I'm running a SeqWare VM on an amazon EC2 instance I'm trying to use the SeqWare query engine to query data from VCF files. When I first launch the instance and follow the instructions to import data, It works fine, and continues to work until I stop the instance. When I restart it. It won't let me import anything, nor create a new workspace. It always returns the error below. I looked at the processes and found that none of the required nodes were running, so I logged into root and went to the etc/init.d directory and start everything again, at which point, when T try to import data, I don't even get an error and I have to stop the process.
[seqware#master target]$ java -classpath seqware-distribution-0.13.6.7-qe-full.jar com.github.seqware.queryengine.system.importers.SOFeatureImporter -i ../../seqware-queryengine/src/test/resources/com/github/seqware/queryengine/system/FeatureImporter/consequences_annotated.vcf ALL.chr3.phase1_release_v3.20101123.snps_indels_svs.genotypes.3_100001-101000.vcf -o keyValueVCF.out -r hg_19 -s c111aea5-5e18-4c62-a8a7-ec82fe151301 -a ad_hoc -w VCFVariantImportWorker
[SeqWare Query Engine] 0 [main] ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - ZooKeeper exists failed after 3 retries
[SeqWare Query Engine] 1 [main] ERROR org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher - hconnection Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:154)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:226)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:82)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:580)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:569)
at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:186)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:100)
at com.github.seqware.queryengine.impl.HBaseStorage.<init>(HBaseStorage.java:89)
at com.github.seqware.queryengine.factory.SWQEFactory$Storage_Type$3.buildStorage(SWQEFactory.java:109)
at com.github.seqware.queryengine.factory.SWQEFactory.getStorage(SWQEFactory.java:174)
at com.github.seqware.queryengine.factory.SWQEFactory.getQueryInterface(SWQEFactory.java:199)
at com.github.seqware.queryengine.impl.SimpleModelManager.<init>(SimpleModelManager.java:49)
at com.github.seqware.queryengine.impl.HBaseModelManager.<init>(HBaseModelManager.java:36)
at com.github.seqware.queryengine.impl.MRHBaseModelManager.<init>(MRHBaseModelManager.java:32)
at com.github.seqware.queryengine.factory.SWQEFactory.getModelManager(SWQEFactory.java:211)
at com.github.seqware.queryengine.system.importers.FeatureImporter.performImport(FeatureImporter.java:66)
at com.github.seqware.queryengine.system.importers.SOFeatureImporter.runMain(SOFeatureImporter.java:141)
at com.github.seqware.queryengine.system.importers.SOFeatureImporter.main(SOFeatureImporter.java:60)
[SeqWare Query Engine] 3 [main] FATAL org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation - Unexpected exception during initialization, aborting
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:154)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:226)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:82)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:580)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:569)
at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:186)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:100)
at com.github.seqware.queryengine.impl.HBaseStorage.<init>(HBaseStorage.java:89)
at com.github.seqware.queryengine.factory.SWQEFactory$Storage_Type$3.buildStorage(SWQEFactory.java:109)
at com.github.seqware.queryengine.factory.SWQEFactory.getStorage(SWQEFactory.java:174)
at com.github.seqware.queryengine.factory.SWQEFactory.getQueryInterface(SWQEFactory.java:199)
at com.github.seqware.queryengine.impl.SimpleModelManager.<init>(SimpleModelManager.java:49)
at com.github.seqware.queryengine.impl.HBaseModelManager.<init>(HBaseModelManager.java:36)
at com.github.seqware.queryengine.impl.MRHBaseModelManager.<init>(MRHBaseModelManager.java:32)
at com.github.seqware.queryengine.factory.SWQEFactory.getModelManager(SWQEFactory.java:211)
at com.github.seqware.queryengine.system.importers.FeatureImporter.performImport(FeatureImporter.java:66)
at com.github.seqware.queryengine.system.importers.SOFeatureImporter.runMain(SOFeatureImporter.java:141)
at com.github.seqware.queryengine.system.importers.SOFeatureImporter.main(SOFeatureImporter.java:60)
I figured it out.The apache services were installed from the cloudera package. They weren't being restarted when the instance was being restarted and apparently, just running their script from the etc/init.d was the incorrect way to do it. I found the commands to restart them in the cloudera documentation.
I too faced this problem.I was able to solve this problem by providing jute.maxbuffer parameter while starting zookeeper.
For more info you can refer
https://issues.apache.org/jira/browse/SOLR-4793