We have a soap service that accepts a small file so we receive that in memory and we need to then sftp that off to another server. What would the configuration be for that? How to take our String(xml file) and send it to the server? ( i assume sftp connector is the best way to go here, but how to configure it as it looks like it takes one file as a parameter and I need it to be fed bytes to send instead with a filename that we specify).
thanks,
Dean
Related
Problem Description:
We are trying to transfer a zip file to a destination coming from an IBM MQ queue.
The Mulesoft flow has an IBM MQ 'On New Message' connector as a source.
Whenever there is a new message in the IBM MQ queue, the connector picks that up and the 'Write' connector is writing that into the local file system (in our original scenario we are using an SFTP connector to send the file to an SFTP server. However, for simplicity here we are just using the Write connector to write into our local file system to mimic the process of writing into the SFTP server file system).
The problem we are having is the zip file is getting corrupted when the Mule flow writes that in the file system.
I believe the reason behind that is the connector is trying to convert the character encoding, whereas we need a binary transfer of the data.
Please advise how we can configure the IBM MQ connector so that it transfers the data as binary without enforcing any character encoding.
The connector version we're currently leveraging is - 1.6.7 and we've tried upgrading its version as well but it didn't help either.
Any leads on the above problem statement would be very much appreciated.
As shown below, RabbitMQ management GUI can publish a message to the specific queue directly.
The consumer of this queue consumes the message with protobuf format, which means I should publish the byte array to the queue instead of string. I have tried to convert the protobuf bytes into a base64 string but failed, is possible to tell the RabbitMQ Management GUI to convert the base64 string into bytes or there is another way to publish byte array directly?
RabbitMQ Management GUI talk to RabbitMQ Server over HTTP (HTTP is a text transfer protocol), so it is impossible to send binary data directly by Management GUI.
RabbitMQ community provide a command line tool rabbitmq-perf-test wrapped RabbitMQ Java client, you can publish binary message with your Content-Type.
The server pays no attention to the Content-Type header; it just passes
it through. So make sure your client supports your Content-Type.
You can use any HTTP client like curl or Postman or anything else.
And just send HTTP POST request like this:
curl 'https://rabbitmq.host/api/exchanges/%2F/amq.default/publish'
--data-raw '{"payload_encoding":"base64","vhost":"/","name":"amq.default","properties":{"delivery_mode":2,"headers":{}},"routing_key":"YOUR QUEUE NAME HERE","delivery_mode":"2","payload":"THE BASE 64 PAYLOAD HERE","headers":{},"props":{}}'
I have some questions on concepts of Mulesoft. If you could answer that would be appreciated.
1.When mule connects to an FTP server and checks for new files,does Mule by default deletes the file once it is downloaded? And can FTP is available both as polling and as event subscription based?
When we connect to a RESTful service over HTTP, can Dynamic endpoints select whether HTTP or HTTPS is used or can be used to set the Host, Port and Path? (multiple choice for this question: a. Dynamic endpoints can select what transport to use. b. The Host, Port and Path can be set using dynamic endpoint. c. Dynamic endpoint can select whether HTTP or HTTPS is used. Which one is correct?)
in configuration, if
is used, does it mean my1.properties will take precedence?
Thanks!
1a) Community FTP connector will always delete the remote file. Enterprise FTP connector allows you to move it to another folder.
1b) Reading FTP connector is only available as a polling mechanism to read files from a FTP server.
2) The only part that can not be dynamic is the scheme, but you could place two connectors (one for HTTP and another for HTTPS) inside a choice and select which one to use dynamically.
3) I didn't get this, please elaborate a bit more.
HTH,
Marcos
Yes, it does. Mule EE give more features as you can see in the documentation FTP is always polling the directory.
You can setup HTTP or HTTPS. But, you can use a composite-source and tell that your flow is available for HTTP and HTTPS.
I know that it is possible, I found time ago the same requirement for FTP in mule forum. Hope this help
<!-- streaming to prevent deleting remote file -->
<ftp:connector name="ftpConnector" streaming="true" />
<flow name="ftpBridge">
<vm:inbound-endpoint path="fetchFtpFile" exchange-pattern="request-response"/>
<scripting:component>
<scripting:script engine="groovy">
def ftpFileUri = "ftp://${payload.userName}:${payload.password}#${payload.host}${payload.path}?fileNameFilter=${payload.fileName}"
muleContext.client.request(ftpFileUri, 30000L);
</scripting:script>
</scripting:component>
</flow>
If you want to load properties you can use spring to do that. Review this link.
What I'm trying to do is to implement a web server using Netty to store large file uploads to HDFS as HDFS files.
My basic work flow is as follows:
an end user send an HTTP PUT/POST request (with payload say 1GB) to my Netty server
Server accepts the HTTP connection and parses method/uri/headers
Server makes a call in DFSClient to HDFS to create a file and obtain a handle (DFSClient.create is a blocking call)
Server receives the rest of the upstream in the HTTP request and writes to the HDFS handle chunk by chunk (writing each chunk to the HDFS handle is a blocking call)
server closes the HDFS handle and acknowledge back to client (closing HDFS handle is blocking call)
I'm having problems in making the above steps work. Because I don't know what is the best way to efficiently make blocking calls in Netty (blocking the whole event loop as little as possible).
Can anybody show me a correct way of implementing the above logic? Many thanks in advance!
If you need to block you will need to put an ExecutionHandler in front of the handler that will perform the blocking operation to not affect the other channels on the same IO-Thread.
I need to automatically transfer an xml file from server A over internet to a server B. Ftp works fine, but should I be use a message queue instead?
It should be secure in the order that I won't lose messages and be able to log what is transferred.
You could use a message queue as well but not to transfer the files, just for keeping a queue of the files to be transferred. Then you can write a Service who uses sftp, https, ssh, or whatever other secure method to transfer the files. There are plenty of options. A common scenario to use is:
- Write a file to a given folder and a message to the message queue.
- The web service will be polling the message queue who will have a message with the filename to be transferred. If there is a file, use SECURE METHOD CHOSEN (see the links below), and do the transfer.
Well, you could simply avoid using message queue and use a secure client to connect to the server B from server A and do the transfer, here are some links that can help you:
How do I upload a file to an SFTP server in C# / .NET?
http://social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/bee2ae55-5558-4c5d-9b5c-fe3c17e3a190
http://social.msdn.microsoft.com/Forums/en-US/netfxnetcom/thread/f5d22700-552f-4214-81f5-fa43bfcc723d
Hope that helps
Use sftp whenever possible.
Use a POST over HTTPS - an implementation is available on every imaginable platform.
Of course, you need to check certificate validity, but this is also a part of the protocol itself; your part is to keep the certificates correct and secure.