Handling Fabric's stdout under mod_wsgi? - mod-wsgi

I'm working on an API that deploys code to different servers using Fabric. It works when I run it using gunicorn, but when I switch to Apache and mod_wsgi, the stdout restrictions kick in.
I tried sys.stdout = sys.stderr in my wsgi script, but instead of an IOError I get
AttributeError: 'mod_wsgi.Log' object has no attribute 'isatty'
What should I do to solve this?

Upgrade your mod_wsgi version. Looks like you are using very old version.

Related

Setting up S3 logging in Airflow

This is driving me nuts.
I'm setting up airflow in a cloud environment. I have one server running the scheduler and the webserver and one server as a celery worker, and I'm using airflow 1.8.0.
Running jobs works fine. What refuses to work is logging.
I've set up the correct path in airflow.cfg on both servers:
remote_base_log_folder = s3://my-bucket/airflow_logs/
remote_log_conn_id = s3_logging_conn
I've set up s3_logging_conn in the airflow UI, with the access key and the secret key as described here.
I checked the connection using
s3 = airflow.hooks.S3Hook('s3_logging_conn')
s3.load_string('test','test',bucket_name='my-bucket')
This works on both servers. So the connection is properly set up. Yet all I get whenever I run a task is
*** Log file isn't local.
*** Fetching here: http://*******
*** Failed to fetch log file from worker.
*** Reading remote logs...
Could not read logs from s3://my-bucket/airflow_logs/my-dag/my-task/2018-02-15T21:46:47.577537
I tried manually uploading the log following the expected conventions and the webserver still can't pick it up - so the problem is on both ends. I'm at a loss at what to do, everything I've read so far tells me this should be working. I'm close to just installing the 1.9.0 which I hear changes logging and see if I'm more lucky.
UPDATE: I made a clean install of Airflow 1.9 and followed the specific instructions here.
Webserver won't even start now with the following error:
airflow.exceptions.AirflowConfigException: section/key [core/remote_logging] not found in config
There is an explicit reference to this section in this config template.
So I tried removing it and just loading the S3 handler without checking first and I got the following error message instead:
Unable to load the config, contains a configuration error.
Traceback (most recent call last):
File "/usr/lib64/python3.6/logging/config.py", line 384, in resolve:
self.importer(used)
ModuleNotFoundError: No module named
'airflow.utils.log.logging_mixin.RedirectStdHandler';
'airflow.utils.log.logging_mixin' is not a package
I get the feeling that this shouldn't be this hard.
Any help would be much appreciated, cheers
Solved:
upgraded to 1.9
ran the steps described in this comment
added
[core]
remote_logging = True
to airflow.cfg
ran
pip install --upgrade airflow[log]
Everything's working fine now.

Tensorflow Serving Client Script Hangs At stub.predict.future()

This is my first time asking a question here so I will try to be descriptive. I am relatively new to python and tensorflow, and have been learning it specifically for a project.
I am currently working to deploy a tensorflow model using tensorflow serving and flask with wsgi. I have been following this architecture: https://github.com/Vetal1977/tf_serving_flask_app
I am running tensorflow_model_server on port=9000.
I know that tensorflow_model_server is working, because when I execute the tensorflow_serving_client.py from command line, I get the expected response. I have tested this command line execution on every user account.
Similarly, I know that Flask + WSGI is working, because I can see log.info points dropping into the apache error log as it works its way through the script. If I return something before it gets to the line in question, it works just fine.
However, when the application is executed with Flask + WSGI, it hangs at this line: result = stub.Predict.future(request, 5.0) # 5 seconds (https://github.com/Vetal1977/tf_serving_flask_app/blob/master/api/gan/logic/tf_serving_client.py#L70)
I can see that it hangs as I monitor top and tail -f error.log and see the same apache process sit there until it is killed or apache is restarted.
I am really stuck on the fact that it works when executed via command line, but not when Flask + WSGI runs it. Can anyone provide suggestions or point me in the right direction? Am I headed down the right path with this? Any assistance at all would be greatly appreciated.
EDIT: I have uploaded the minimal code to a github repo here: https://github.com/raymondtri/client-test along with a minimal setup that does require flask, wsgi, tensorflow, and tensorflow-serving.
Thanks in advance,
Ray
After much research and many hours, I think that this has something to do with how mod_wsgi forks processes, and how grpc has some known issues with that. I have a feeling that things are getting orphaned as the process is forked and that is what is causing the script to hang.
I could be wrong though, that is just my current hypothesis.

Adding LESS file to HTML [duplicate]

I'm trying to load a 3D model, stored locally on my computer, into Three.js with JSONLoader, and that 3D model is in the same directory as the entire website.
I'm getting the "Cross origin requests are only supported for HTTP." error, but I don't know what's causing it nor how to fix it.
My crystal ball says that you are loading the model using either file:// or C:/, which stays true to the error message as they are not http://
So you can either install a webserver in your local PC or upload the model somewhere else and use jsonp and change the url to http://example.com/path/to/model
Origin is defined in RFC-6454 as
...they have the same
scheme, host, and port. (See Section 4 for full details.)
So even though your file originates from the same host (localhost), but as long as the scheme is different (http / file), they are treated as different origin.
Just to be explicit - Yes, the error is saying you cannot point your browser directly at file://some/path/some.html
Here are some options to quickly spin up a local web server to let your browser render local files
Python 2
If you have Python installed...
Change directory into the folder where your file some.html or file(s) exist using the command cd /path/to/your/folder
Start up a Python web server using the command python -m SimpleHTTPServer
This will start a web server to host your entire directory listing at http://localhost:8000
You can use a custom port python -m SimpleHTTPServer 9000 giving you link: http://localhost:9000
This approach is built in to any Python installation.
Python 3
Do the same steps, but use the following command instead python3 -m http.server
VSCode
If you are using Visual Studio Code you can install the Live Server extension which provides a local web server enviroment.
Node.js
Alternatively, if you demand a more responsive setup and already use nodejs...
Install http-server by typing npm install -g http-server
Change into your working directory, where yoursome.html lives
Start your http server by issuing http-server -c-1
This spins up a Node.js httpd which serves the files in your directory as static files accessible from http://localhost:8080
Ruby
If your preferred language is Ruby ... the Ruby Gods say this works as well:
ruby -run -e httpd . -p 8080
PHP
Of course PHP also has its solution.
php -S localhost:8000
In Chrome you can use this flag:
--allow-file-access-from-files
Read more here.
Ran in to this today.
I wrote some code that looked like this:
app.controller('ctrlr', function($scope, $http){
$http.get('localhost:3000').success(function(data) {
$scope.stuff = data;
});
});
...but it should've looked like this:
app.controller('ctrlr', function($scope, $http){
$http.get('http://localhost:3000').success(function(data) {
$scope.stuff = data;
});
});
The only difference was the lack of http:// in the second snippet of code.
Just wanted to put that out there in case there are others with a similar issue.
Just change the url to http://localhost instead of localhost. If you open the html file from local, you should create a local server to serve that html file, the simplest way is using Web Server for Chrome. That will fix the issue.
I'm going to list 3 different approaches to solve this issue:
Using a very lightweight npm package: Install live-server using npm install -g live-server. Then, go to that directory open the terminal and type live-server and hit enter, page will be served at localhost:8080. BONUS: It also supports hot reloading by default.
Using a lightweight Google Chrome app developed by Google: Install the app, then go to the apps tab in Chrome and open the app. In the app point it to the right folder. Your page will be served!
Modifying Chrome shortcut in windows: Create a Chrome browser's shortcut. Right-click on the icon and open properties. In properties, edit target to "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --disable-web-security --user-data-dir="C:/ChromeDevSession" and save. Then using Chrome open the page using ctrl+o. NOTE: Do NOT use this shortcut for regular browsing.
Note: Use http:// like http://localhost:8080 in case you face error.
Use http:// or https:// to create url
error: localhost:8080
solution: http://localhost:8080
In an Android app — for example, to allow JavaScript to have access to assets via file:///android_asset/ — use setAllowFileAccessFromFileURLs(true) on the WebSettings that you get from calling getSettings() on the WebView.
fastest way for me was:
for windows users run your file on Firefox problem solved, or
if you want to use chrome easiest way for me was to install Python 3 then from command prompt run command python -m http.server then go to http://localhost:8000/ then navigate to your files
python -m http.server
Easy solution for whom using VS Code
I've been getting this error for a while. Most of the answers works. But I found a different solution. If you don't want to deal with node.js or any other solution in here and you are working with an HTML file (calling functions from another js file or fetch json api's) try to use Live Server extension.
It allows you to open a live server easily. And because of it creates localhost server, the problem is resolving. You can simply start the localhost by open a HTML file and right-click on the editor and click on Open with Live Server.
It basically load the files using http://localhost/index.html instead of using file://....
EDIT
It is not necessary to have a .html file. You can start the Live Server with shortcuts.
Hit (alt+L, alt+O) to Open the Server and (alt+L, alt+C) to Stop the server. [On MAC, cmd+L, cmd+O and cmd+L, cmd+C]
Hope it will help someone :)
If you use old version of Mozilla Firefox (pre-2019), it will work as expected without any issues;
P.S. Surprisingly, old versions of Internet Explorer & Edge work absolutely fine too.
For those on Windows without Python or Node.js, there is still a lightweight solution: Mongoose.
All you do is drag the executable to wherever the root of the server should be, and run it. An icon will appear in the taskbar and it'll navigate to the server in the default browser.
Also, Z-WAMP is a 100% portable WAMP that runs in a single folder, it's awesome. That's an option if you need a quick PHP and MySQL server. Though it hasn't been updated since 2013. A modern alternative would be Laragon or WinNMP. I haven't tested them, but they are portable and worth mentioning.
Also, if you only want the absolute basics (HTML+JS), here's a tiny PowerShell script that doesn't need anything to be installed or downloaded:
$Srv = New-Object Net.HttpListener;
$Srv.Prefixes.Add("http://localhost:8080/");
$Srv.Start();
Start-Process "http://localhost:8080/index.html";
While($Srv.IsListening) {
$Ctx = $Srv.GetContext();
$Buf = [System.IO.File]::OpenRead((Join-Path $Pwd($Ctx.Request.RawUrl)));
$Ctx.Response.ContentLength64 = $Buf.Length;
$Ctx.Response.Headers.Add("Content-Type", "text/html");
$Buf.CopyTo($Ctx.Response.OutputStream);
$Buf.Close();
$Ctx.Response.Close();
};
This method is very barebones, it cannot show directories or other fancy stuff. But it handles these CORS errors just fine.
Save the script as server.ps1 and run in the root of your project. It will launch index.html in the directory it is placed in.
I suspect it's already mentioned in some of the answers, but I'll slightly modify this to have complete working answer (easier to find and use).
Go to: https://nodejs.org/en/download/. Install nodejs.
Install http-server by running command from command prompt npm install -g http-server.
Change into your working directory, where index.html/yoursome.html resides.
Start your http server by running command http-server -c-1
Open web browser to http://localhost:8080
or http://localhost:8080/yoursome.html - depending on your html filename.
I was getting this exact error when loading an HTML file on the browser that was using a json file from the local directory. In my case, I was able to solve this by creating a simple node server that allowed to server static content. I left the code for this at this other answer.
It simply says that the application should be run on a web server. I had the same problem with chrome, I started tomcat and moved my application there, and it worked.
I suggest you use a mini-server to run these kind of applications on localhost (if you are not using some inbuilt server).
Here's one that is very simple to setup and run:
https://www.npmjs.com/package/tiny-server
Experienced this when I downloaded a page for offline view.
I just had to remove the integrity="*****" and crossorigin="anonymous" attributes from all <link> and <script> tags
If you insist on running the .html file locally and not serving it with a webserver, you can prevent those cross origin requests from happening in the first place by making the problematic resources available inline.
I had this problem when trying to to serve .js files through file://. My solution was to update my build script to replace <script src="..."> tags with <script>...</script>.
Here's a gulp approach for doing that:
1.
run npm install --save-dev to packages gulp, gulp-inline and del.
2.
After creating a gulpfile.js to the root directory, add the following code (just change the file paths for whatever suits you):
let gulp = require('gulp');
let inline = require('gulp-inline');
let del = require('del');
gulp.task('inline', function (done) {
gulp.src('dist/index.html')
.pipe(inline({
base: 'dist/',
disabledTypes: 'css, svg, img'
}))
.pipe(gulp.dest('dist/').on('finish', function(){
done()
}));
});
gulp.task('clean', function (done) {
del(['dist/*.js'])
done()
});
gulp.task('bundle-for-local', gulp.series('inline', 'clean'))
Either run gulp bundle-for-local or update your build script to run it automatically.
You can see the detailed problem and solution for my case here.
For all y'all on MacOS... setup a simple LaunchAgent to enable these glamorous capabilities in your own copy of Chrome...
Save a plist, named whatever (launch.chrome.dev.mode.plist, for example) in ~/Library/LaunchAgents with similar content to...
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>launch.chrome.dev.mode</string>
<key>ProgramArguments</key>
<array>
<string>/Applications/Google Chrome.app/Contents/MacOS/Google Chrome</string>
<string>-allow-file-access-from-files</string>
</array>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>
It should launch at startup.. but you can force it to do so at any time with the terminal command
launchctl load -w ~/Library/LaunchAgents/launch.chrome.dev.mode.plist
TADA! 😎 💁🏻 🙊 🙏🏾
Not possible to load static local files(eg:svg) without server. If you have NPM /YARN installed in your machine, you can setup simple http server using "http-server"
npm install http-server -g
http-server [path] [options]
Or open terminal in that project folder and type "hs". It will automaticaly start HTTP live server.
er. I just found some official words "Attempting to load unbuilt, remote AMD modules that use the dojo/text plugin will fail due to cross-origin security restrictions. (Built versions of AMD modules are unaffected because the calls to dojo/text are eliminated by the build system.)" https://dojotoolkit.org/documentation/tutorials/1.10/cdn/
One way it worked loading local files is using them with in the project folder instead of outside your project folder. Create one folder under your project example files similar to the way we create for images and replace the section where using complete local path other than project path and use relative url of file under project folder .
It worked for me
Install local webserver for java e.g Tomcat,for php you can use lamp etc
Drop the json file in the public accessible app server directory
Start the app server,and you should be able to access the file from localhost
For Linux Python users:
import webbrowser
browser = webbrowser.get('google-chrome --allow-file-access-from-files %s')
browser.open(url)
url should be like:
createUserURL = "http://www.localhost:3000/api/angular/users"
instead of:
createUserURL = "localhost:3000/api/angular/users"
Many problem for this, with my problem is missing '/' example:
jquery-1.10.2.js:8720 XMLHttpRequest cannot load http://localhost:xxxProduct/getList_tagLabels/
It's must be: http://localhost:xxx/Product/getList_tagLabels/
I hope this help for who meet this problem.
I have also been able to recreate this error message when using an anchor tag with the following href:
Example a tag
In my case an a tag was being used to get the 'Pointer Cursor' and the event was actually controlled by some jQuery on click event. I removed the href and added a class that applies:
cursor:pointer;
cordova achieve this. I still can not figure out how cordova did. It does not even go through shouldInterceptRequest.
Later I found out that the key to load any file from local is: myWebView.getSettings().setAllowUniversalAccessFromFileURLs(true);
And when you want to access any http resource, the webview will do checking with OPTIONS method, which you can grant the access through WebViewClient.shouldInterceptRequest by return a response, and for the following GET/POST method, you can just return null.
If you are searching for a solution for Firebase Hosting, you can run the
firebase serve --only hosting command from the Firebase CLI
That's what I came here for, so I thought I'd just leave it here to help like ones.
If your using VS code just trying loading a live server in there. fixed my problem immediately.

Serving lua pages in apache windows

I have been using php for CGI scripting for some time now and recently got interested in lua.
I installed the latest version of luarocks(2.1.2) and the bundled version of lua(5.1.4). I wanted to start from the basics and hence installed cgilua(5.1.4-2) and all its dependencies using "luarocks install cgilua".
I am able to run simple lua scripts using the shebang line to point to my lua interpreter but when i use it to point to the cgi launcher "cgilua.cgi.exe" to run .lp files it just won't work. I edited my httpd configuration file to allow cgi execution in my htdocs and cgi-bin directory and used the cgi-script handler for .lp pages. I am trying to run the login.lp example in the cgilua examples directory. I even added the line "Content-type:text/html" to no avail. Executing the cgilua.cgi.exe file from the command line without arguments just closes the application with the message "cgilua.cgi.exe" stopped working".
Could anyone tell me what am I missing? Maybe the launcher is supposed to be used in a different way?
I don't suppose permissions have a part to play in this as in windows all users have at least read and execute permissions.
The url I'm trying to access is http://localhost/login.lp. My apache error log shows "Premature end of script headers: login.lp" with a 500 internal server error and the same thing if I access http://localhost/cgilua.cgi.exe
I don't know what your requirements are, but perhaps it will be easier to simply use apache's mod_lua.
http://httpd.apache.org/docs/trunk/mod/mod_lua.html

Apache and Sqlite3 issue

I'm using ports only on this server. After restarting apache I got "segment fault (core dumped)" message. But apache is started at least... httpd-error.log shows this:
profiling:/usr/ports/databases/sqlite3/work/sqlite-3.7.3/.libs/sqlite3.gcda:Cannot open
profiling:/usr/ports/databases/sqlite3/work/sqlite-3.7.3/.libs/sqlite3.gcda:Cannot open
profiling:/usr/ports/databases/sqlite3/work/sqlite-3.7.3/.libs/sqlite3.gcda:Cannot open
profiling:/usr/ports/databases/sqlite3/work/sqlite-3.7.3/.libs/sqlite3.gcda:Cannot open
Listing of that dir:
baccy# ls /usr/ports/databases/sqlite3/work/sqlite-3.7.3/.libs/
libsqlite3.a libsqlite3.la libsqlite3.lai libsqlite3.so libsqlite3.so.8 sqlite3 sqlite3.gcno sqlite3.o
On this system I've got redmine installed, that's why sqlite is included in here.. now I installed django and I need sqlite3 so much..
It would be nice to know, WHY this error appears, and HOW to handle this =)
Just found an answer
it's GCONV param fault... just go to /usr/ports/databases/sqlite3 and then make config, and remove GCONV param.