Processing large images with Sharp on Lambda#Edge - serverless-framework

When using Sharp in a function deployed to Lambda#Edge I'm getting the following error:
vips__write: write failed unix error: No space left on device
Running on Lambda, this can be fixed using the VIPS_DISC_THRESHOLD env variable, but since env variable are not allowed for Lambda#Edge I'm looking for an alternative solution.

This solved it for me:
process.env.VIPS_DISC_THRESHOLD = '750m'

Related

java.awt.HeadlessException error running maxent in "dismo" through rJava

I am trying to run the maxent function in the dismo package on macOS through rJava and keep getting this error message.
Error in rJava::.jcall(mxe, "S", "fit", c("autorun", "-e", afn, "-o", : java.awt.HeadlessException
Has anyone seen it before and know how to fix it? Thank you so much!
I am working as a dev for EcoCommons (https://app.ecocommons.org.au)
I was having this issue with building out our coding cloud, which is a jupyterhub k8s deployment (https://analysis-tools.app.ecocommons.org.au/coding-cloud). We are using Rstudio Server, but I would imagine the issue is similar perhaps.
RStudio sets the DISPLAY env variable. What you will need to do is unset this high up in your script.
sys.unsetenv('DISPLAY')

How to receive tile data from ClusterBuster vector tile server in the sample example?

Summary
Seems the guys did a great work with the built-in filtering, clustering and caching on the ClusterBuster vector tile server.
I'm really excited about this project that looks very promising and i'm eager to try it!
I'm working on a project on plotting Deck.gl layers (MVTLayer) on top of Google Maps from a vector tile server through their integration.
I want to try to integrate and to serve from ClusterBuster server some clustered points saved in PostGIS.
Expected result:
I want to receive tile data from ClusterBuster vector tile server like in their provided sample example.
Actual result:
Provided sample example not working (points not showing on map, neither on Mapbox, nor on Google Maps with Deck.gl).
My attempts
I tried building and running the example provided but somehow, i'm not able to do this.
I ran the provided express server clusterbuster/example/express.ts and for frontend tried with the provided mapbox example from clusterbuster/example/mapbox.html (with my mapbox token) but nothing appears on mapbox map (nor on Google maps with my Deck.gl example).
As stated in the clusterbuster/example/readme.md i created an .env file with PostGIS connections settings.
After running the following commands:
yarn
yarn start
i get this output:
C:\zFVStuff\Google Maps - Deck.gl\_Servers\ClusterBuster\clusterbuster-master> yarn start
yarn run v1.22.4
$ yarn build && ts-node example/express.ts
$ rollup --config rollup.config.js
./lib/index.ts → dist/index.d.ts...
created dist/index.d.ts in 5.7s
./lib/index.ts → dist/index.js...
created dist/index.js in 5.1s
attempting to create supporting SQL functions
failure in creating First SQL function
failure in creating TileBBox SQL function
failure in creating TileDoubleBBox SQL function
Example app listening on port 3005!
send3663284b-0e58-4587-af82-b35604a99303: 4.964ms
3663284b-0e58-4587-af82-b35604a99303: 817.819ms
sendd0bdd671-1f47-4b40-84ba-2fed6adfd045: 3.451ms
d0bdd671-1f47-4b40-84ba-2fed6adfd045: 1276.588ms
sendcd36f672-4c4b-4a95-b772-7584e6eb8c55: 1.349ms
cd36f672-4c4b-4a95-b772-7584e6eb8c55: 1754.146ms
The requests seems to be ok, returning status 200 but tile data from ClusterBuster server is undefined nothing appears on the map (neither your Mapbox, or my Google Maps).
Also, it's curious that all tile responses seems to have the same size 216 B.
The data was imported ok in the PostGIS database.
Result is undefined after debugging the express.ts server file with VSCode Javascript Debug Terminal and starting it with the command ts-node .\example\express.ts.
In the Chrome network tab, in none of the sub-tabs Preview or Response i can't see any data.
I also tried logging the result tile in the express server in the success function,
server({
...//excluded for visibility
}).then(result => {
...//excluded for visibility
console.log(result)
res.status(200).send(result);
})
and below is the output, i get undefined:
Any help or advice is really appreciate! Thank you in advance!
After enabling debug in the TileServer configuration (with debug: true), i noticed 2 errors:
SELECT ST_AsMVT(tile, 'points', 4096, 'geom') AS mvt FROM tile
{
e: error: password authentication failed for user "userXYZ"
...
SELECT ST_AsMVT(tile, 'points', 4096, 'geom') AS mvt FROM tile
{
e: Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:205:27) {
...
Initially I ran from the root of clusterbuster with ts-node .\example\express.ts or with yarn start (that underneath uses the same and it tries to connect with my user instead of .env config file)
After running ts-node express.ts from the example folder it took into account the .env config file with postgres user instead of my user. Now the tile data from the server is returned ok and the example works at expected.
Many thanks to the contributor cosmin-petrescu that helped me.
Entire thread can be seen here on github.

Custom ejabberd authentication no longer working

I have created an authentication module which was build against
ejabberd 16.02 and runs fine when auth_method is set. It also works against
16.03.
However, from 16.04 onwards it gives me the error "[error] ignoring
option 'auth_method' with invalid value: [jwt]"
I checked the code diff between those releases and the only change
seems to be to the mod_pubsub.erl file, specifically adding the
following:
ServerHost = serverhost(Host),
+ ejabberd_hooks:run(pubsub_subscribe_node, ServerHost,
+ [ServerHost, Host, Node, Subscriber, SubId]),
https://github.com/processone/ejabberd/commit/639c2fb6401391663206c0e4c946d1a699689ac7
I have tried disabling this module and even deleting the beam file as
i don't use it, but i can't seem ti get round it.
Does anyone have any insight as to why these changes will have broken
my authentication module?
My source is at the link below, but as i say has worked fine for a year:
https://github.com/ParamountVentures/ejabberd-auth-jwt
The answer is that from 16.04 onwards you need to drop the .erl file into the ejabberd src folder and compile it with the source. Dropping in the .beam file to use alternative authentication modules no longer works.

ejabberd hook execution error while trying building a message receipts module

I am getting an error while trying to return a receipt message from server to acknowledge message received at server for which I am using a non official module (mod_stanza_ack) and ejabberd 15.07.
I have compiled and configured it successfully too, but it's giving the following error during ejabberd hook execution:
#ejabberd_hooks:run_fold1:371 {undef,[{mod_send_receipt,on_user_send_packet,[{xmlel,<<"message">>,[{<<"id">>,<<"6uAAO-218">>}.....
undef means the function mod_send_receipt:on_user_send_packet.
I see possible reason for this:
source file is not compiled (due to syntax error for example)
or the resulting .beam file is not in your Erlang VM code path
or the function is actually not defined in your module
You need to fix your module accordingly or put the .beam file in the right place.

EPIPE error uploading img to s3 using imagesquish

I've been testing imagesquish for a while and I can't seem to have it working.
It works with node.js, forever, express and knox and basically does image processing on the fly.
I'm trying to call an image on a server, it resizes, crops or whatever and then uploads to s3.
I've had permission issues but after creating a new s3 bucket, it seems I made some progress but then, the image doesn't get uploaded and I can't get any error.
I managed to go through it's code and go all the way to exports.uploadImage that would call img.toBuffer but apparently it doesn't get called.
At the end, I get the error:
events.js:72
throw er; // Unhandled 'error' event
^
Error: write EPIPE
at errnoException (net.js:904:11)
at Object.afterWrite (net.js:720:19)
error: Forever detected script exited with code: 8
error: Script restart attempt #1
I need some help on this. Does anybody have any idea?
Luis. Not sure if you're still interested in an answer to this. I'm the author of ImageSquish. From the information you provide, this sounds like img.toBuffer is failing which is usually because graphicsmagick is not installed. The latest version of ImageSquish checks for the existence of this library on startup (see https://github.com/dmcquay/imagesquish/blob/master/check.js).
You might also find it easier to try it out using docker (https://registry.hub.docker.com/u/dmcquay/imagesquish/) so you don't have to worry about installing anything.
If this doesn't resolve your problem, you can find me via irc on freenode #imagesquish.