I am creating REST api with hapi and mongojs. I need to output large number of JSONs as API response. I am using following code. But getting error “Error: Stream must have a streams2 readable interface”.
handler: function(request, reply) {
reply(db.collection.find().pipe(JSONStream.stringify()));}
How do I convert mongojs object stream to readable stream?
I tried following code as well
var outputStream = db.collection.find().pipe(JSONStream.stringify());
reply(outputStream.pipe(new Readable().wrap(outputStream)));
but it doesn't show any data in output.
Related
I am writing code for Request and Response web compression in vb.net using GzipStream. I need to take analysis before and after compression of Response. I have written below code to capture the Response data length after de-compression.
Using testStream As New GZipStream(response.GetResponseStream(), CompressionMode.Decompress)
Dim decompressedBytes(800000) As Byte
compLength = zippedStream.Read(decompressedBytes, 0, 800000)
WriteFileStream(compLength, Path, Length, methodURL)
Return testStream
End Using
"Stream was not readable" error is coming after Return testStream. Without zippedStream.Read code everything is working fine. "WriteFileStream" is just a method which writes the data into a file.
Any suggessions here.
Regards
sangeetha
I am trying to send an audio file recorded using expo-av library to my server using web sockets.
Websocket will allow me to only send String, ArrayBuffer or Blob. I spent whole day trying to find out how to convert my .wav recording into a blob but without success. I tried to use expo-file-system method FileSystem.readAsStringAsync to read the filed as a string but I get an error that the file could not be read. How is that possible? I passed it the correct URI (using recording.getURI()).
I tried to re-engineer my approach to use fetch and FormData post request with the same URI and audio gets sent correctly. But I really would like to use WebSockets so that later I could try to make it stream the sound to the server in real time instead of recording it first and then sending it.
You can try ... But I can't find a way to read the blob itself
// this is from my code ...
let recording = Audio.Recording ;
const info = await FileSystem.getInfoAsync(recording.getURI() || "");
console.log(`FILE INFO: ${JSON.stringify(info)}`);
// get the file as a blob
const response = await fetch(info.uri);
const blob = await response.blob(); // slow - Takes a lot of time
When I try to transcribe some video's I do get the following error message:
java.util.concurrent.ExecutionException: com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Request contains an invalid argument.
at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:553) ~[guava-28.2-android.jar!/:na]
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:432) ~[guava-28.2-android.jar!/:na]
at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:93) ~[guava-28.2-android.jar!/:na]
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:68) ~[guava-28.2-android.jar!/:na]
at com.google.api.gax.longrunning.OperationFutureImpl.get(OperationFutureImpl.java:133) ~[gax-1.53.0.jar!/:1.53.0]
In the process MP4 video's are converted to WEBM so we are able to stream the video's correctly in our frontend. There seems to be a problem with the ByteArray of the WEBM file we pass to the request, so I tried to pass the ByteArray of the MP4 file directly. Unfortunately I did get the same error here with multiple MP4 files. There are also some MP4 files which are transcribed succesfully. So I don't know what is wrong with the ByteArray and how to fix this problem.
Our AnnotateVideoRequest is builded in this way:
private fun buildRequest(
videoBytes: ByteString,
transcriptionLanguage: String
): AnnotateVideoRequest {
val config: SpeechTranscriptionConfig = SpeechTranscriptionConfig
.newBuilder()
.setLanguageCode(transcriptionLanguage)
.setEnableAutomaticPunctuation(this.enableAutomaticPunctuation)
.setMaxAlternatives(this.maxAlternatives)
.build()
val context: VideoContext = VideoContext
.newBuilder()
.setSpeechTranscriptionConfig(config)
.build()
return AnnotateVideoRequest
.newBuilder()
.setInputContent(videoBytes)
.addFeatures(Feature.SPEECH_TRANSCRIPTION)
.setVideoContext(context)
.build()
}
We make use of the of the following version of the client library: com.google.cloud:google-cloud-video-intelligence:1.2.1
What are possible ways to determine what is actually going wrong when performing this request with some of our MP4 files?
I debugged it and I only get this error if i send file bytes as "inputContent". If i have file in google cloud and i send file url as "inputUri" then i do not get any error. (used gs://cloud-samples-data/video/cat.mp4" as sample for testing)
Original Question
I'm using an API to get a thumbnail image that I uploaded. Instead of providing me with a url to the image, the service returns me a garbles text string that I thought was Base64. However, all of my attempts to decode this string have failed. Does anyone have any ideas of what object types the service might be returning me? If it's not base64, what would it be?
screenshot on imgur of the API response
Answer to Original Question
#Andrew Tran: pointed out that the response I was getting looked like it was the raw binary data for the png file. This helped correct my lack of experience with Base64 and led me to some further research.
First Attempt at downloading the file
Dim path As String = "C:\Users\username\lpImages\img1.png"
Dim fs As FileStream = File.Create(path)
Dim info As Byte() = New UTF8Encoding(True).GetBytes(thumbnail)
fs.Write(info, 0, info.Length)
fs.Close()
This never worked... so I talked with a coworker and finally realized my mistake
I had been using an abstracted class to call the API and had glazed over the fact that under the hood the "request" method was actually reading the return stream into a string. The service returned the image png file as a stream, but the request method was converting this raw stream. Once I created a different request method I was able to get the png as a stream. From there it was relatively easy to use it as I intended: attaching the images as LinkedResources to an email that I then send out.
Original request method code
response = theRequest.GetResponse
Dim reader As StreamReader = New StreamReader(response.GetResponseStream)
lp_response.response = reader.ReadToEnd
New request code I wrote instead
lp_response.response = theRequest.GetResponse.GetResponseStream
VB.Net code to handle the stream (this is just a snippet where I'm building a List of LinkedResources to pass to my email function; just to give an idea of how I'm using it)
Dim document As Stream = LPApi.GetDocumentThumbnail(d("id").ToString)
Dim mediaType As String = Utils.GetContentType(Path.GetExtension(d("file_name")))
Dim lrDocument As New LinkedResource(document, mediaType)
lrDocument.ContentId = d("id").ToString
Thanks to everyone who commented. I'm pretty inexperienced when it comes to the deeper architecture of web requests/responses and data serialization. Any good learning resources would be helpful; otherwise I'll just keep Googling :)
It doesn't look like it's being encoded at all.
The IHDR chunk start is clearly visible and in plaintext at the start of the file.
It's probably not a good idea to use HttpWebRequest to download binary data, then convert the response to a string, just to try to convert that string back into binary data to save it to a file.
If you're getting an HttpWebResponse object, you should be able to use HttpWebResponse.GetResponseStream to get a stream for the image data. Then you should be able to copy the image data straight into your filestream using Stream.CopyTo.
I am writing a WCF client to a SOAP service that returns a mime multi-part result with binary data (actually, a PDF file). It uses a custom message encoder.
The service doesn't seem to mind if I make the request a single-part format, so I am able to get a result back. There are two problems with the result from what I can see:
It only seems to return the first part of the multi-part message.
The data I get back cannot be decoded by my custom encoder.
I have tried utilizing MTOM binding, but that messes up the request. It fails to add the "boundary" parameter in the content-type, so the server cannot understand the request.
I think what I want is a basic text SOAP request, but a response decoded MTOM-style. I have no idea how to set that up, however.
The closest solution I have found is this: http://blogs.msdn.com/b/carlosfigueira/archive/2011/02/16/using-mtom-in-a-wcf-custom-encoder.aspx
But it seems like a very invasive change to my project.
I figured this out. First of all, I was incorrect when I said that I was only getting the first part of the multi-part message using the MTOM encoder; I was getting the whole thing. I was looking at it in the debugger and the bottom must have gotten clipped in the debug viewer. Chalk it up to my inexperience manually looking at and deciphering multi-part messages.
To the second point, all I had to do was use the MTOM encoder when the Content-Type was multipart/related and everything worked just fine. If you read the referenced article above, it's all about dynamically detecting whether the message is multipart or regular text, and choosing the proper encoder based on that. Essentially, it's a custom encoder that has both a text encoder and MTOM encoder built into it, and switches back and forth based on the content-type of the incoming message.
Our project requires some post processing of the response message before it's handed off to the main program logic. So, we get the incoming SOAP content as an XML string, and do some XML manipulation on it.
This is a slight departure from the solution recommended in the article. All that's required in the article's solution is reading the message using the right encoder into a System.ServiceModel.Channels.Message, and returning that. In our solution, we need to interrupt this process and do the post-processing.
To do that, implement the following in your custom encoder:
public override Message ReadMessage(ArraySegment<byte> buffer, BufferManager bufferManager, string contentType)
{
//First, get the incoming message as a byte array
var messageData = new byte[buffer.Count];
Array.Copy(buffer.Array, buffer.Offset, messageData, 0, messageData.Length);
bufferManager.ReturnBuffer(buffer.Array);
//Now convert it into a string for post-processing. Look at the content-type to determine which encoder to use.
string stringResult;
if (contentType != null && contentType.Contains("multipart/related"))
{
Message unprocessedMessageResult = this.mtomEncoder.ReadMessage(buffer, bufferManager, contentType);
stringResult = unprocessedMessageResult.ToString();
}
else {
//If it's not a multi-part message, the byte array already has the complete content, and it simply needs to be converted to a string
stringResult = Encoding.UTF8.GetString(messageData);
}
Message processedMessageResult = functionToDoPostProccessing(stringResult);
return processedMessageResult;
}