ImageResizer FFmpeg plugin failing to produce an image for a frame - asp.net-mvc-4

Using a fairly simple installation of ImageResizer, integrated with our application. We're evaluating it, and so have not applied a license yet. Images are resizing as expected (with the red dot for no license), but we're trying to pull frames from videos using the FFmpeg plugin and having problems.
With a request like myvideo.mp4?ffmpeg.seconds=1 or myvideo.mp4?ffmpeg.percent=50, the browser shows The image ... cannot be displayed because it contains errors, and application log shows:
ImageResizer.ImageCorruptedException (0x80004005): File may be corrupted, empty, or may contain a PNG image with a single dimension greater than 65,535 pixels. ---> System.ArgumentException: Parameter is not valid.
at System.Drawing.Bitmap..ctor(Stream stream, Boolean useIcm)
at ImageResizer.ImageBuilder.DecodeStream(Stream s, ResizeSettings settings, String optionalPath)
at ImageResizer.ImageBuilder.LoadImage(Object source, ResizeSettings settings, Boolean restoreStreamPos)
at ImageResizer.ImageBuilder.LoadImage(Object source, ResizeSettings settings, Boolean restoreStreamPos)
at ImageResizer.ImageBuilder.BuildJob(ImageJob job)
at ImageResizer.ImageBuilder.BuildInternal(ImageJob job)
at ImageResizer.ImageBuilder.BuildInQueue(ImageJob job, Boolean useSemaphore, Int32 maxQueuingMilliseconds, CancellationToken cancel)
at ImageResizer.ImageBuilder.Build(ImageJob job)
at ImageResizer.InterceptModule.<>c__DisplayClass5_0.<handlerequest>b__1(Stream stream)
at ImageResizer.Plugins.DiskCache.CustomDiskCache.<>c__DisplayClass29_0.<trywritefile>b__0()
at ImageResizer.Plugins.DiskCache.LockProvider.TryExecute(String key, Int32 timeoutMs, LockCallback success)
at ImageResizer.Plugins.DiskCache.CustomDiskCache.GetCachedFile(String keyBasis, String extension, ResizeImageDelegate writeCallback, Int32 timeoutMs, Boolean asynchronous)
at ImageResizer.Plugins.DiskCache.DiskCache.Process(IResponseArgs e)
at ImageResizer.Plugins.DiskCache.DiskCache.Process(HttpContext context, IResponseArgs e)
at ImageResizer.InterceptModule.HandleRequest(HttpContext context, HttpModuleRequestAssistant ra, IVirtualFile vf)
at ImageResizer.InterceptModule.CheckRequest_PostAuthorizeRequest(Object sender, EventArgs e)
at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
1: Source:
ImageResizer
1: Stack Trace:
at ImageResizer.ImageBuilder.LoadImage(Object source, ResizeSettings settings, Boolean restoreStreamPos)
1: at ImageResizer.ImageBuilder.BuildJob(ImageJob job)
1: at ImageResizer.ImageBuilder.BuildInternal(ImageJob job)
1: at ImageResizer.ImageBuilder.BuildInQueue(ImageJob job, Boolean useSemaphore, Int32 maxQueuingMilliseconds, CancellationToken cancel)
1: at ImageResizer.ImageBuilder.Build(ImageJob job)
1: at ImageResizer.InterceptModule.<>c__DisplayClass5_0.<handlerequest>b__1(Stream stream)
1: at ImageResizer.Plugins.DiskCache.CustomDiskCache.<>c__DisplayClass29_0.<trywritefile>b__0()
1: at ImageResizer.Plugins.DiskCache.LockProvider.TryExecute(String key, Int32 timeoutMs, LockCallback success)
1: at ImageResizer.Plugins.DiskCache.CustomDiskCache.GetCachedFile(String keyBasis, String extension, ResizeImageDelegate writeCallback, Int32 timeoutMs, Boolean asynchronous)
1: at ImageResizer.Plugins.DiskCache.DiskCache.Process(IResponseArgs e)
1: at ImageResizer.Plugins.DiskCache.DiskCache.Process(HttpContext context, IResponseArgs e)
1: at ImageResizer.InterceptModule.HandleRequest(HttpContext context, HttpModuleRequestAssistant ra, IVirtualFile vf)
1: at ImageResizer.InterceptModule.CheckRequest_PostAuthorizeRequest(Object sender, EventArgs e)
1: at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
1: at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
1: at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
UPDATE
In order to make sure that there is no issue with the underlying ffmpeg and actually pulling an image from the video in question, i used the command line to pull a frame from it manually. Full output is below, and it produced the 'test.png' image as expected.
ffmpeg.exe -i P:\path\to\media\myvideo.mp4 -vframes 1 test.png
ffmpeg version N-69258-gf994000 Copyright (c) 2000-2015 the FFmpeg developers
built on Jan 25 2015 22:21:16 with gcc 4.9.2 (GCC)
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --e
v --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm
e-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable
bvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libw
ble-lzma --enable-decklink --enable-zlib
libavutil 54. 17.100 / 54. 17.100
libavcodec 56. 20.100 / 56. 20.100
libavformat 56. 19.100 / 56. 19.100
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 9.100 / 5. 9.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'P:\path\to\media\myvideo.mp4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: mp41mp42isom
creation_time : 2018-02-07 00:14:12
location : +nn.nnnn-nnn.nnnn/
Duration: 00:00:13.64, start: 0.000000, bitrate: 840 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 98 kb/s (default)
Metadata:
creation_time : 2018-02-07 00:14:12
handler_name : Core Media Audio
Stream #0:1(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 568x320, 735 kb/s, 29.98 fps, 29.97 tbr,
Metadata:
rotate : 90
creation_time : 2018-02-07 00:14:12
handler_name : Core Media Video
Side data:
displaymatrix: rotation of -90.00 degrees
Output #0, image2, to 'test.png':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: mp41mp42isom
location : +nn.nnnn-nnn.nnnn/
encoder : Lavf56.19.100
Stream #0:0(und): Video: png, rgb24, 568x320, q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
Metadata:
rotate : 90
creation_time : 2018-02-07 00:14:12
handler_name : Core Media Video
encoder : Lavc56.20.100 png
Stream mapping:
Stream #0:1 -> #0:0 (h264 (native) -> png (native))
Press [q] to stop, [?] for help
frame= 1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.03 bitrate=N/A
video:249kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
So why can ffmpeg.exe pull the frame manually, but when i try to do it through the browser and ImageResizer, i get the error above?

Hoo boy, don't i feel sheepish.. it's a good thing i like humility. :-|
Turns out the comment down at the bottom of the Ffmpeg documentation page is really important: "Make sure you use <pipeline vppusage="Always"/>, or ffmpeg will not be able to participate in the request.".
Seems like that should probably be up in the Installation section.

Related

StackExchange.Redis.RedisTimeoutException Timeout performing PSETEX

.NET 7.0 and StackExchange.Redis version=2.6.70.
I am facing this error when the cache is on the server but not when running locally with the cache in container on my machine.
StackExchange.Redis.RedisTimeoutException: Timeout performing PSETEX (5000ms),
inst: 0,
qu: 1,
qs: 0,
aw: False,
bw: SpinningDown,
rs: NotStarted,
ws: Idle,
in: 0,
serverEndpoint: 0.0.24.235:6379,
mc: 1/1/0,
mgr: 10 of 10 available,
clientName: 6fe450d7aad5(SE.Redis-v2.6.70.49541),
IOCP: (Busy=0,Free=1000,Min=1,Max=1000),
WORKER: (Busy=1,Free=32766,Min=32,Max=32767),
POOL: (Threads=5,QueuedItems=0,CompletedItems=279236),
v: 2.6.70.49541 (Please take a look at this article for some common client-side issues that can
cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)\n at
StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1
processor, ServerEndPoint server, T defaultValue) in
//src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1887\n at
StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor`1 processor,
ServerEndPoint server, T defaultValue) in //src/StackExchange.Redis/RedisBase.cs:line 62\n at
StackExchange.Redis.RedisDatabase.StringSet(RedisKey key, RedisValue value, Nullable`1 expiry,
Boolean keepTtl, When when, CommandFlags flags) in
/_/src/StackExchange.Redis/RedisDatabase.cs:line 3128\n at
ConfigurationManagementMicroService.Cache.CacheService.SetData[T](String key, T value,
DateTimeOffset expirationTime) in
/src/ConfigurationManagementMicroService/Cache/CacheService.cs:line 36\n
This is how I am connecting:
ConnectionHelper.lazyConnection = new Lazy<ConnectionMultiplexer>(() =>
{
var config = new ConfigurationOptions()
{
EndPoints = { { ip, Convert.ToInt32(port) } },
AbortOnConnectFail = false,
};
return ConnectionMultiplexer.Connect(config);
}
Sync handshake could be an issue or big sized payload? Give analysis on this.

Ignite 'Streamer is closed with error, Invalid request op code: 8000' when using Data Streaming

I am trying out the Data Streaming feature of Ignite (see https://ignite.apache.org/docs/latest/data-streaming) but even when I am running the very basic example shown on the documentation page I get the following exception:
ApacheIgniteExamplesDemo.DemoWorker[0]
Apache.Ignite.Core.Client.IgniteClientException: Streamer is closed with error, check inner exception for details.
---> System.AggregateException: One or more errors occurred. (One or more errors occurred. (Invalid request op code: 8000))
---> System.AggregateException: One or more errors occurred. (Invalid request op code: 8000)
---> Apache.Ignite.Core.Client.IgniteClientException: Invalid request op code: 8000
at Apache.Ignite.Core.Impl.Client.ClientSocket.DecodeResponse[T](BinaryHeapStream stream, Func`2 readFunc, Func`3 errorFunc)
at Apache.Ignite.Core.Impl.Client.ClientSocket.<>c__DisplayClass33_0`1.<DoOutInOpAsync>b__0(Task`1 responseTask)
at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
at System.Threading.Tasks.Task.<>c.<.cctor>b__271_0(Object obj)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) [StatusCode=InvalidOpCode]
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Apache.Ignite.Core.Impl.Client.Datastream.DataStreamerClient`2.ThrowIfClosed()
at Apache.Ignite.Core.Impl.Client.Datastream.DataStreamerClient`2.Add(DataStreamerClientEntry`2 entry)
at Apache.Ignite.Core.Impl.Client.Datastream.DataStreamerClient`2.Add(TK key, TV val)
at ApacheIgniteExamplesDemo.DemoWorker.DataStreamingExample() in C:\demos\ApacheIgniteDemo\DemoWorker.cs:line 262 [StatusCode=Fail]
I have not been able to find out anything about why the streamer is closed with an error and what "Invalid request op code: 8000" indicates. Any help or suggestion on what might be the problem would be highly appreciated.
I run the following code:
//setting up cache
var cache = _ignite.CreateCache<int, string>(new CacheClientConfiguration
{
Name = "cache",
QueryEntities = new[]
{
new QueryEntity(
typeof(int),
typeof(string))
{
TableName = "table",
}
},
SqlSchema = "PUBLIC",
CacheMode = CacheMode.Partitioned
});
//running streamer
using(var streamer = _ignite.GetDataStreamer<int, string>(cache.Name))
{
for (int i = 0; i < 10000; i++)
streamer.Add(i, i.ToString());
}
Ignite is running in a docker container with default configuration.
The error message indicates that the feature is not supported by the server node. Thin client data streamer is available since Ignite 2.11 - please make sure to use that or a later version.

Upload large video to GCS use signed url but video is broken

I have a application for Google App Engine(GAE), using Vue.js on the frontend and Flask on the backend.
My app allows users to upload large video and will analyze it. But since GAE's upload size limit is 32MB, so I allow users to upload directly to Google Cloud Storage(GCS) using signed url.
The problem I am facing is that the user can successfully upload the video to GCS, but on the backend(flask) when downloading the video for analysis, get the error below:
*** OSError: MoviePy error: failed to read the duration of file /tmp/source_video.mp4.
Here are the file infos returned by ffmpeg:
ffmpeg version 4.2.2-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
[mov,mp4,m4a,3gp,3g2,mj2 # 0x7444500] Format mov,mp4,m4a,3gp,3g2,mj2 detected only with low score of 1, misdetection possible!
[mov,mp4,m4a,3gp,3g2,mj2 # 0x7444500] moov atom not found
/tmp/source_video.mp4: Invalid data found when processing input
Flask code download:
class Analyser:
def __init__(self):
self.gcs_bucket = 'bucket_name'
self.gcs_blob_video = 'videos/'
def __storage_bucket(self):
client = storage.Client()
bucket = client.get_bucket(self.gcs_bucket)
bucket.cors = [
{
"origin": ["*"],
"responseHeader": [
"Access-Control-Allow-Origin"
],
"method": ['PUT', 'POST', 'GET'],
"maxAgeSeconds": 3600
}
]
bucket.patch()
return bucket
def __generate_upload_signed_url(self, bucket, blob):
blob = bucket.blob(blob)
return blob.generate_signed_url(
version='v4',
expiration=datetime.timedelta(minutes=15),
method='PUT',
)
def analyze_video(self, pid):
src_filepath = '/tmp/source_video.mp4'
bucket = self.__storage_bucket()
blob = bucket.blob(self.gcs_blob_video + 'filename.mp4')
blob.download_to_filename(src_filepath)
#error is here
video = VideoFileClip(src_filepath)
Vuejs code upload:
note: Uploaded successfully, on gcs there is a file to upload.
async uploadVideo(_: any, video: File): Promise<string> {
signed_url = "https://storage.googleapis.com/bucket_name...." #signed url is return from flask
const formData = new FormData();
formData.append('file', video);
const response_upload_file = await fetch(
signed_url,
{
method: 'PUT',
body: formData
}
)
return true
}
I've searched on google quite a bit and still haven't found a solution.
I really don't know if it's broken during the upload or when I download for analysis.
I would be very grateful if someone could suggest me a solution to solve this problem.
Thanks for reading.
You are using formData with the HTTP PUT method which corrupts the upload as a binary data stream is expected and not MIME data.
To use formdata:
Upload an object with HTML forms

RabbitMQ TLS Authentication

There is a task to configure the operation of some web services using certificate authorization.
There is:
Erlang 22.3.3
RabbitMQ 3.8.3
It makes no sense to describe their installation.
What has been done next:
1. In accordance with the article (https://www.rabbitmq.com/ssl.html) we perform the following actions:
git clone https://github.com/michaelklishin/tls-gen tls-gen
cd tls-gen / basic
CN = client PASSWORD = 123 make
make verify
make info
Copy the created certificates, change the owner
mv testca/ /etc/rabbitmq/
mv server/ /etc/rabbitmq/
mv client/ /etc/rabbitmq/
chown -R rabbitmq: /etc/rabbitmq/testca
chown -R rabbitmq: /etc/rabbitmq/server
chown -R rabbitmq: /etc/rabbitmq/client
We bring the configuration file to the form (/etc/rabbitmq/rabbitmq.config):
[
{ssl, [{versions, ['tlsv1.2', 'tlsv1.1', tlsv1]}]},
{rabbit, [
{ssl_listeners, [5671]},
{auth_mechanisms, ['PLAIN', 'AMQPLAIN', 'EXTERNAL']},
{ssl_cert_login_from, 'client'},
{ssl_options, [{cacertfile, "/ etc / rabbitmq / testca / cacert.pem"},
{certfile, "/ etc / rabbitmq / server / cert.pem"},
{keyfile, "/ etc / rabbitmq / server / key.pem"},
{verify, verify_peer},
{fail_if_no_peer_cert, true}]}]}}
].
We start the server, try to connect from the client. We get the error:
2020-05-18 17: 21: 57.166 +03: 00 [ERR] Failed to connect to broker 10.10.11.16, port 5671, vhost dmz
RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable
---> RabbitMQ.Client.Exceptions.PossibleAuthenticationFailureException: Possibly caused by authentication failure
---> RabbitMQ.Client.Exceptions.OperationInterruptedException: The AMQP operation was interrupted: AMQP close-reason, initiated by Library, code = 0, text = 'End of stream', classId = 0, methodId = 0, cause = System .IO.EndOfStreamException: Reached the end of the stream. Possible authentication failure.
at RabbitMQ.Client.Impl.InboundFrame.ReadFrom (Stream reader)
at RabbitMQ.Client.Impl.SocketFrameHandler.ReadFrame ()
at RabbitMQ.Client.Framing.Impl.Connection.MainLoopIteration ()
at RabbitMQ.Client.Framing.Impl.Connection.MainLoop ()
at RabbitMQ.Client.Impl.SimpleBlockingRpcContinuation.GetReply (TimeSpan timeout)
at RabbitMQ.Client.Impl.ModelBase.ConnectionStartOk (IDictionary`2 clientProperties, String mechanism, Byte [] response, String locale)
at RabbitMQ.Client.Framing.Impl.Connection.StartAndTune ()
--- End of inner exception stack trace ---
at RabbitMQ.Client.Framing.Impl.Connection.StartAndTune ()
at RabbitMQ.Client.Framing.Impl.Connection.Open (Boolean insist)
at RabbitMQ.Client.Framing.Impl.Connection..ctor (IConnectionFactory factory, Boolean insist, IFrameHandler frameHandler, String clientProvidedName)
at RabbitMQ.Client.Framing.Impl.ProtocolBase.CreateConnection (IConnectionFactory factory, Boolean insist, IFrameHandler frameHandler, String clientProvidedName)
at RabbitMQ.Client.ConnectionFactory.CreateConnection (IEndpointResolver endpointResolver, String clientProvidedName)
--- End of inner exception stack trace ---
at RabbitMQ.Client.ConnectionFactory.CreateConnection (IEndpointResolver endpointResolver, String clientProvidedName)
at RabbitMQ.Client.ConnectionFactory.CreateConnection (String clientProvidedName)
at EasyNetQ.ConnectionFactoryWrapper.CreateConnection ()
at EasyNetQ.PersistentConnection.TryToConnect ()
In the rabbitmq log:
2020-05-18 17: 24: 59.880 [info] <0.3442.0> accepting AMQP connection <0.3442.0> (10/10/15/14/1561 -> 10/10/11/166767)
2020-05-18 17: 25: 02.887 [error] <0.3442.0> closing AMQP connection <0.3442.0> (10/10/15/14/1561 -> 10/10/11/1667671):
{handshake_error, starting, 0, {error, function_clause, 'connection.start_ok', [{rabbit_ssl, peer_cert_auth_name, [client, << 48,130,3,42,48,130,2,18,160,3,2,1,2,2 , 1,2,48,13,6,9,42,134,72,134,247,13,1,1,11,5,0,48,4,49,49,32,48,30,6,3,85,4,3 12,23,84,76,83,71,101,110,83,101,108,102,83,105,103,110,101,100,116,82,111,111,116,67,65,49,13,48,11,6,3,85,4,7,12,4,36,36,36 , 36.48,30,23,13,50,48,48,53,49,56,49,52,48,49,53,53,90,23,13,51,48,48,53,49 , 54,49,52,48,49,53,53,90,48,34,49,15,48,13,6,3,85,4,3,12,6,99,108,105,101,110,116,49,15,48 , 13,6,3,85,4,10,12,6,99,108,105,101,110,116,48,130,1,34,48,13,6,9,42,134,72,134,247,13,1,1,1,5,0,3,130 1,15,0,48,130,1,10,2,130,1,1,0,183,198,116,156,3,177,131,5,148,11,154,34,99,210,88,115,60,228,180,245,80,212,113,57,181,249,20,5,164,49,72,95,153,116,103,49 , 58,119,15,48,147,107,112,243,105,122,189,44,0,193,114,138,169,250,165,97,188,158,188,95,163,37,30,75,143,21,103,11,131,223,124,96,244,111,210,30,8,175,72,206,162,14,86,63,146,215,179,226,239,48,76,122,150,200,183,82,114,1 73,116,32,224,202,196,129,131,96,34,237,34,144,177,92,200,105,212,0,133,141,118,146,229,140,246,229,137,0,9,27,180,163,233,134,0,187,110,9,126,92,172,105,96,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,1,118,11,11,118,11,118,11,118,11,118,11,118,11,118,11,118,11,118,11,118,11,118,11,118,1,118,11,11,118,11,11,11,11,1,1,1,1,1,1,1,1,1,1,1,1,1,111,1'''1,11,11,1'''1,1''''N''O'', '' 92,181,68,172,135,15,90,152,209,242,31,138,135,34,95,29,162,226,175,253,176,14
UPDATE
New rabbitmq.config:
[
{rabbit,[
{auth_backends, [rabbit_auth_backend_internal]},
{auth_mechanisms, ['PLAIN', 'AMQPLAIN', 'EXTERNAL']},
{ssl_listeners,[5671]},
{ssl_options,[
{versions,['tlsv1.2', 'tlsv1.1']},
{cacertfile, "/etc/rabbitmq/testca/cacert.pem"},
{certfile, "/etc/rabbitmq/server/cert.pem"},
{keyfile, "/etc/rabbitmq/server/key.pem"},
{verify,verify_peer},
{fail_if_no_peer_cert,true}]}
]}
].
New error:
2020-05-18 18:48:56.681 [info] <0.1410.0> Connection <0.1410.0> (10.10.15.14:52744 -> 10.10.11.16:5671) has a client-provided name: Viber.CallbackService.dll
2020-05-18 18:48:56.682 [error] <0.1410.0> Error on AMQP connection <0.1410.0> (10.10.15.14:52744 -> 10.10.11.16:5671, state: starting):
EXTERNAL login refused: user 'O=client,CN=client' - invalid credentials
Have you enabled the ssl plugin and restarted the broker?
sudo rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl
sudo systemctl restart rabbitmq-server
You may also try and set the following in rabbitmq.conf:
ssl_cert_login_from = common_name
ssl_options.password = 123
And create a user called client in the broker to match the CN name in your certificate.

Uart / GPS driver sample buffer overflow

I am trying the sample for the GPS driver with a raspberry pi 3 and the Ultimate GPS V3 breakout board.
Here is the full source code: https://github.com/androidthings/drivers-samples/tree/master/gps
The GPS board is connected following this schematics:
When launching the sample app, I get the following error:
com.example.androidthings.driversamples E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.androidthings.driversamples, PID: 1299
java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:508)
at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:142)
at com.google.android.things.contrib.driver.gps.NmeaGpsModule.processBuffer(NmeaGpsModule.java:178)
at com.google.android.things.contrib.driver.gps.NmeaGpsModule.readUartBuffer(NmeaGpsModule.java:160)
at com.google.android.things.contrib.driver.gps.NmeaGpsModule.access$000(NmeaGpsModule.java:35)
at com.google.android.things.contrib.driver.gps.NmeaGpsModule$1.onUartDeviceDataAvailable(NmeaGpsModule.java:139)
at com.google.android.things.pio.UartDevice$UartDeviceCallbackDispatch.dispatchInterruptEvent(UartDevice.java:507)
at com.google.android.things.pio.CallbackDispatch.onFileDescriptorEvents(CallbackDispatch.java:127)
at android.os.MessageQueue.dispatchEvents(MessageQueue.java:282)
at android.os.MessageQueue.nativePollOnce(Native Method)
at android.os.MessageQueue.next(MessageQueue.java:323)
at android.os.Looper.loop(Looper.java:136)
at android.app.ActivityThread.main(ActivityThread.java:6077)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755)
Update 1
After enabling the debug in the contrib-driver project, I see a new error: W/NmeaParser: Invalid checksum (62), expected 108
12-28 17:53:28.638 1378-1378/com.example.androidthings.driversamples D/XXX: Debug version used
12-28 17:53:29.451 1378-1378/com.example.androidthings.driversamples I/Choreographer: Skipped 34 frames! The application may be doing too much work on its main thread.
12-28 17:53:29.862 1378-1378/com.example.androidthings.driversamples W/NmeaParser: Invalid checksum (62), expected 108
12-28 17:53:29.862 1378-1378/com.example.androidthings.driversamples D/XXX: Buffer reset
12-28 17:53:30.427 1378-1378/com.example.androidthings.driversamples D/AndroidRuntime: Shutting down VM
12-28 17:53:30.428 1378-1378/com.example.androidthings.driversamples E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.androidthings.driversamples, PID: 1378
java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:508)
at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:142)
at com.google.android.things.contrib.driver.gps.NmeaGpsModule.processBuffer(NmeaGpsModule.java:179)
at com.google.android.things.contrib.driver.gps.NmeaGpsModule.readUartBuffer(NmeaGpsModule.java:161)
at com.google.android.things.contrib.driver.gps.NmeaGpsModule.access$000(NmeaGpsModule.java:35)
at com.google.android.things.contrib.driver.gps.NmeaGpsModule$1.onUartDeviceDataAvailable(NmeaGpsModule.java:140)
at com.google.android.things.pio.UartDevice$UartDeviceCallbackDispatch.dispatchInterruptEvent(UartDevice.java:507)
at com.google.android.things.pio.CallbackDispatch.onFileDescriptorEvents(CallbackDispatch.java:127)
at android.os.MessageQueue.dispatchEvents(MessageQueue.java:282)
at android.os.MessageQueue.nativePollOnce(Native Method)
at android.os.MessageQueue.next(MessageQueue.java:323)
at android.os.Looper.loop(Looper.java:136)
at android.app.ActivityThread.main(ActivityThread.java:6077)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755)
12-28 17:53:30.439 1378-1378/com.example.androidthings.driversamples I/Process: Sending signal. PID: 1378 SIG: 9
Update 2
After increasing the buffer size by 4, I was able to get few messages in. I see some messages have some junk that would explain the overflow:
12-28 23:38:17.393 2366-2366/? D/XXX: message: ��GPGGA,233817.000,3742.1931,N,12208.3976,W,1,04,1.96,164.3,M,-25.5,M,,*58
12-28 23:38:17.394 2366-2366/? D/XXX: Buffer reset
12-28 23:38:17.394 2366-2366/? D/XXX: message: GPGSA,A,3,23,03,26,22,,,,,,,,,2.20,1.96,1.00*0B
12-28 23:38:17.395 2366-2366/? D/XXX: Buffer reset
12-28 23:38:17.544 2366-2366/com.example.androidthings.driversamples D/XXX: message: GP��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������RMC,233817.000,A��������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������,3742.1931,N,12208.3976,W,0.42,205.67,281216,,,A*71
12-28 23:38:17.545 2366-2366/com.example.androidthings.driversamples D/XXX: Buffer reset
12-28 23:38:17.545 2366-2366/com.example.androidthings.driversamples D/XXX: message: GPVTG,205.67,T,,M,0.42,N,0.78,K,A*32
12-28 23:38:17.546 2366-2366/com.example.androidthings.driversamples D/XXX: Buffer reset
I have no idea where that junk could come from...
In some cases processBuffer() will process the whole buffer (512 bytes) regardless of how many bytes are read from the UART. If it happens to miss a start or end character you can end up with a lot of '0' in the message buffer. You can modify this to be like this:
int count;
while ((count = uart.read(buffer, buffer.length)) > 0) {
processBuffer(buffer, count);
}
then modify this to be more like this:
private void processBuffer(byte[] buffer, int length) {
for (int i = 0; i < length; i++) {
if (mParser.getFrameStart() == buffer[i]) {
handleFrameStart();
} else if (mParser.getFrameEnd() == buffer[i]) {
handleFrameEnd();
} else {
//Insert all other characters into the buffer
mMessageBuffer.put(buffer[i]);
}
}
}
Then you won't be filling the message buffer with garbage if an end character gets dropped. I noticed this happening in a project I'm working on that uses this source code. The readUartBuffer() method can read split messages, i.e., the start or end of a message. If it does, bad things happen.
Looking at the demo code https://github.com/androidthings/drivers-samples/blob/master/gps/src/main/java/com/example/androidthings/driversamples/GpsActivity.java#L35 the Baud rate is set correctly as per the data sheet: https://cdn-learn.adafruit.com/downloads/pdf/adafruit-ultimate-gps.pdf
And looking at the contrib-driver for NmeaGpsModule the message buffer length is twice the size of the buffer read from the driver https://github.com/androidthings/contrib-drivers/blob/master/gps/src/main/java/com/google/android/things/contrib/driver/gps/NmeaGpsModule.java#L169 so it can't overflow with just one read.
A theory for the issue could be that the message buffer is not being cleared after it receives data - therefore you get a BufferOverflowException after a few packets.
The buffer is reset in 2 places:
When a new frame starts:
https://github.com/androidthings/contrib-drivers/blob/master/gps/src/main/java/com/google/android/things/contrib/driver/gps/NmeaGpsModule.java#L187
Or when a frame ends:
https://github.com/androidthings/contrib-drivers/blob/master/gps/src/main/java/com/google/android/things/contrib/driver/gps/NmeaGpsModule.java#L209
I don't have the hardware to debug your issue, however I can recommend how you could do it:
This goes for anyone wanting to debug a contrib-driver
Fork / Make a copy of this android library: https://github.com/androidthings/contrib-drivers/tree/master/gps
Edit the build.gradle to add this dependency: https://github.com/novoda/bintray-release (follow the README for instructions).
Once you add that dependency the build.gradle will look like this:
apply plugin: 'com.android.library'
apply plugin: 'com.novoda.bintray-release' // new code
android {
compileSdkVersion 24
buildToolsVersion '24.0.3'
defaultConfig {
minSdkVersion 24
targetSdkVersion 24
versionCode 1
versionName "1.0"
}
}
buildscript {
repositories {
jcenter()
}
dependencies {
classpath 'com.novoda:bintray-release:0.4.0' // new code
}
}
dependencies {
compile 'com.android.support:support-annotations:24.2.0'
provided 'com.google.android.things:androidthings:0.1-devpreview'
}
publish { // new code
userOrg = 'google'
groupId = 'com.google.android.things.contrib'
artifactId = 'driver-gps'
publishVersion = '0.1-DEBUG'
}
Now you can release your own version of the GPS driver for debugging, but first:
Edit the java files, to add logging. In the NmeaGpsModule.java of your fork/copy, make two changes:
private void init(UartDevice device, int baudRate, Handler handler) throws IOException {
Log.d("XXX", "MY VERSION BEING USED"); // new code
mDevice = device;
mDevice.setBaudrate(baudRate);
mDevice.registerUartDeviceCallback(mCallback, handler);
mParser = new NmeaParser();
}
and
private void resetBuffer() {
Log.d("XXX", "BUFFER BEING RESET"); // new code
mMessageBuffer.clear();
mFrameFlag = false;
}
To release this version run this command in a terminal from the same folder as the build.gradle:
./gradlew clean build bintrayUpload -PdryRun=true
Now you have released a dependency!
Back in your application (in this case the driver-samples) https://github.com/androidthings/drivers-samples/blob/master/gps/build.gradle#L39:
Change the build.gradle to have a different dependency:
compile 'com.google.android.things.contrib:driver-gps:0.1'
becomes
compile 'com.google.android.things.contrib:driver-gps:0.1-DEBUG'
and tell it to look locally for this dependency:
repositories {
mavenLocal()
jcenter()
}
Now rerun the application and you should see your logs! First the sanity log to prove it worked "MY VERSION BEING USED" then the buffer being reset "BUFFER BEING RESET" if the buffer isn't being reset .. you need to check your hard wiring or add more logs to figure out why.