egg file needs permission in scrapyd while deploying - scrapy

If I do this for another project means it shows.
$ scrapy deploy scrapyd
Packing version 1412325181
Deploying to project "project2" in http://localhost:6800/addversion.json
Server response (200):
{"status": "error", "message": "[Errno 13] Permission denied: 'eggs'"}

Related

Install Ambari, can't download hortonworks HDP from amazon S3

I'm tring to install apache ambari on my wsl(ubuntu 20.04) as the Ambari User Guides step by step. while install and packing the project to deb files use command:
mvn -B clean install jdeb:jdeb -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" .
got this error
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (default) on project ambari-metrics-timelineservice: An Ant BuildException has occured: Can't get https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz to /root/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz
[ERROR] around Ant part ...<get usetimestamp="true" src="https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz" dest="/root/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz"/>... # 5:273 in /root/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download HBase.xml
apache-ambari-2.7.5-src/ambari-metrics/pom.xml defined the nortonworks HDP sources:
<hbase.tar>http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hbase/hbase-2.0.0.3.0.0.0-1634-bin.tar.gz</hbase.tar>
<hbase.folder>hbase-2.0.0.3.0.0.0-1634</hbase.folder>
<hadoop.tar>http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hadoop/hadoop-3.1.0.3.0.0.0-1634.tar.gz</hadoop.tar>
<hadoop.folder>hadoop-3.1.0.3.0.0.0-1634</hadoop.folder>
<phoenix.tar>http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/phoenix/phoenix-5.0.0.3.0.0.0-1634.tar.gz</phoenix.tar>
<phoenix.folder>phoenix-5.0.0.3.0.0.0-1634</phoenix.folder>
I tried to download official Hbase-2.3.2, Hadoop-3.3.0,phoenix-5.0.0-HBase-2.0 to instead hotonworks HDP, but failed and got an other error.
I tried to download hortonworks HDP directly use wget and got:
Resolving dev.hortonworks.com.s3.amazonaws.com (dev.hortonworks.com.s3.amazonaws.com)... 52.217.40.204
Connecting to dev.hortonworks.com.s3.amazonaws.com
(dev.hortonworks.com.s3.amazonaws.com)|52.217.40.204|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
where/how can i download these hortonworks HDP files and continue to install ambari?
This solution from the Apache Ambari team fixed this and other similar issues with HDP urls: https://github.com/apache/ambari/pull/3283
You need to make the corresponding code changes: https://github.com/apache/ambari/pull/3283/commits/3dca705f831383274a78a8c981ac2b12e2ecce85
replace the downlink with
<hbase.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.1-1/tars/hbase/hbase-2.0.2.3.1.4.1-1-bin.tar.gz</hbase.tar>
<hbase.tar>https://private-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.1-1/tars/hbase/hbase-2.0.2.3.1.4.1-1-bin.tar.gz</hbase.tar>
<hbase.folder>hbase-2.0.2.3.1.4.1-1</hbase.folder>
<hadoop.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.1-1/tars/hadoop/hadoop-3.1.1.3.1.4.1-1.tar.gz</hadoop.tar>
<hadoop.tar>https://private-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.1-1/tars/hadoop/hadoop-3.1.1.3.1.4.1-1.tar.gz</hadoop.tar>
<hadoop.folder>hadoop-3.1.1.3.1.4.1-1</hadoop.folder>
<grafana.folder>grafana-6.4.2</grafana.folder>
<grafana.tar>https://dl.grafana.com/oss/release/grafana-6.4.2.linux-amd64.tar.gz</grafana.tar>
<phoenix.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.1-1/tars/phoenix/phoenix-5.0.0.3.1.4.1-1.tar.gz</phoenix.tar>
<phoenix.tar>https://private-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.1-1/tars/phoenix/phoenix-5.0.0.3.1.4.1-1.tar.gz</phoenix.tar>
<phoenix.folder>phoenix-5.0.0.3.1.4.1-1</phoenix.folder>
edit file ambari-metrics/pom.xml and replace download link, for example I changed
https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz
for
https://downloads.apache.org/hbase/2.3.5/hbase-2.3.5-src.tar.gz

I get “Error: Deploy failed: b''” when deploying to Scrapy Cloud

I am trying to deploy some spiders on Scrapy Cloud, I am getting this message at the end:
Error: Deploy failed: b''
The spiders appear in the dashboard but don't run (doesn't even start to run).
I have this in requirements.txt:
bs4==0.0.1
boto==2.49.0
And this is what scrapinghub.yml:
project: <my project ID>
stacks:
default: scrapy:1.7-py3
requirements:
file: requirements.txt

Web application could not be started by the Phusion Passenger application server - Redmine installation

I've installed a redmine web application on a centOs 7, with postgresql 9.1 and using passenger. When I access the redmine application, the following error appears:
We're sorry, but something went wrong.
The issue has been logged for investigation. Please try again later.
Error ID: f4adae6a Details:
Web application could not be started by the Phusion Passenger
application server.
Please read the Passenger log file (search for the Error ID) to find
the details of the error.
When I access the erro_log file at /var/log/httpd/ it is like this:
[ E 2018-05-25 16:28:01.7581 24316/Td age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /opt/redmine: An operating system error occurred while preparing to spawn an application process: Cannot create FIFO file /tmp/passenger.spawn.XXXXCTqsz4/response/finish: Permission denied (errno=13)
Error ID: f4adae6a
Error details saved to: /tmp/passenger-error-H1zmwo.html
[ E 2018-05-25 16:28:01.7609 24316/T6 age/Cor/Con/CheckoutSession.cpp:276 ]: [Client 1-2] Cannot checkout session because a spawning error occurred. The identifier of the error is f4adae6a. Please see earlier logs for details about the error.
It seems that, first of all, apache is not logging the error at /tmp/passenger-error-H1zmwo.html as it was supposed to do.
I've already tried to loossing the permissions to the /opt/redmine using the following guide HERE
I'm on redmine 3.4.3, ruby 2.3.1 and rails 4.2.8. Passenger 5.3.1
Do'nt know what to do anymore. Appreciate a little help. Any more info please ask me.
What you describe points very much to wrong filesystem permissions for /tmp
It should look like this:
user#host:/$ ls -ld /tmp
drwxrwxrwt 8 root root 4096 May 29 16:32 /tmp
If it does not try this (as root or via sudo):
chmod 1777 /tmp

Error installing DataStax OpsCenter while creating opscenter SSL files

I downloaded the setup from official site and executed ./setup.py but got following error
File "./setup.py", line 54
print "Error when creating opscenter SSL files. Error was:\n"
^
SyntaxError: Missing parentheses in call to 'print'
any alternatives?

After upgrade web server (sever 2012 R2) web deploy is now failing

We just upgraded our test web servers from Server 2008 to Server 2012 R2. We have have a batch script that deploy our site. When it goes to push the site to the server its now failing. Whats weird is that is i can publish through visual studio with the same publish profile and everything is fine. We were able to upgrade our dev web servers and ever ran into this issue.
Here is what happens when i publish through visual studio 2015 (which works):
>C:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe -v \ -p "C:\tfs\TRSApps\Test\Web\Secured Account Access\Member\obj\x86\Release\AspnetCompileMerge\Source" -u "C:\tfs\TRSApps\Test\Web\Secured Account Access\Member\obj\x86\Release\AspnetCompileMerge\TempBuildDir"
3>Auto ConnectionString Transformed obj\x86\Release\AspnetCompileMerge\TempBuildDir\Web.config into obj\x86\Release\CSAutoParameterize\transformed\Web.config.
3>Copying all files to temporary location below for package/publish:
3>obj\x86\Release\Package\PackageTmp.
3>Start Web Deploy Publish the Application/package to https://web-test3-new:8172/msdeploy.axd?site=TRS ...
3>Adding directory (TRS/subsections/SecureAccountAccess\App_Browsers).
3>Adding directory (TRS/subsections/SecureAccountAccess\bin).
3>Adding directory (TRS/subsections/SecureAccountAccess\Board).
3>Adding directory (TRS/subsections/SecureAccountAccess\Demographics).
3>Adding directory (TRS/subsections/SecureAccountAccess\Estimates).
3>Adding directory (TRS/subsections/SecureAccountAccess\General Information).
3>Adding directory (TRS/subsections/SecureAccountAccess\Includes).
3>Adding directory (TRS/subsections/SecureAccountAccess\Includes\Images).
3>Adding directory (TRS/subsections/SecureAccountAccess\Includes\Scripts).
Here is the batch statement to deploy the site:
"C:\Program Files (x86)\MSBuild\14.0\Bin\msbuild.exe" "c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj" /p:VisualStudioVersion=14.0;DeployOnBuild=True;Configuration=Release;Platform=x86;PublishProfile="c:\tfs\trsapps\test\Web\Secured Account Access\Member\My Project\PublishProfiles\Membertest.pubxml";AllowUntrustedCertificate=True;Username=bob;Password=MyPassword /t:Clean;Build /p:OutputPath="c:\tfs\trsapps\test\Web\Secured Account Access\Member\bin"
afd
PipelineCopyAllFilesToOneFolderForMsdeploy:
Publish Pipeline Deploy phase Stage PipelineCopyAllFilesToOneFolderForMsdeploy
GenerateMsdeployManifestFiles:
Generate source manifest file for Web Deploy package/publish ...
MSDeployPublish:
Start Web Deploy Publish the Application/package to https://web-test3-new:8172/msdeploy.axd?site=TRS ...
Starting Web deployment task from source: manifest(c:\tfs\trsapps\test\Web\Secured Account Access\Member\bin\_Publish
edWebsites\TRSSecureAccess_Package\TRSSecureAccess.SourceManifest.xml) to Destination: auto().
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : Web d
eployment task failed. (Could not complete the request to remote agent URL 'https://web-test3-new:8172/msdeploy.axd?sit
e=TRS'.) [c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : This
error indicates that you cannot connect to the server. Make sure the service URL is correct, firewall and network setti
ngs on this computer and on the server computer are configured properly, and the appropriate services have been started
on the server. [c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : Error
details:\r [c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : Could
not complete the request to remote agent URL 'https://web-test3-new:8172/msdeploy.axd?site=TRS'.\r [c:\tfs\trsapps\tes
t\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : The u
nderlying connection was closed: An unexpected error occurred on a send.\r [c:\tfs\trsapps\test\Web\Secured Account Acc
ess\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : Unabl
e to read data from the transport connection: An existing connection was forcibly closed by the remote host.\r [c:\tfs\
trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : An ex
isting connection was forcibly closed by the remote host [c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecu
reAccess.vbproj]
Publish failed to deploy.
Done Building Project "c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj" (Clean;Build targe
t(s)) -- FAILED.
Build FAILED.
"c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj" (Clean;Build target) (1) ->
(MSDeployPublish target) ->
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : Web
deployment task failed. (Could not complete the request to remote agent URL 'https://web-test3-new:8172/msdeploy.axd?s
ite=TRS'.) [c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : This
error indicates that you cannot connect to the server. Make sure the service URL is correct, firewall and network setti
ngs on this computer and on the server computer are configured properly, and the appropriate services have been started
on the server. [c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : Error
details:\r [c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : Could
not complete the request to remote agent URL 'https://web-test3-new:8172/msdeploy.axd?site=TRS'.\r [c:\tfs\trsapps\tes
t\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : The u
nderlying connection was closed: An unexpected error occurred on a send.\r [c:\tfs\trsapps\test\Web\Secured Account Acc
ess\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : Unabl
e to read data from the transport connection: An existing connection was forcibly closed by the remote host.\r [c:\tfs\
trsapps\test\Web\Secured Account Access\Member\TRSSecureAccess.vbproj]
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web.Publishing.targets(4276,5): error : An ex
isting connection was forcibly closed by the remote host [c:\tfs\trsapps\test\Web\Secured Account Access\Member\TRSSecu
reAccess.vbproj]
0 Warning(s)
1 Error(s)
Have you installed the server the server side pre-requisites.
This link may help.