I want to send Stream of data from VM to host machine and I am using method writeToSocket() as shown below:
joinedStreamEventDataStream.writeToSocket("192.168.1.10", 6998) ;
Here joinedStreamEventDataStream is of type DataStream<Integer,Integer>.
Can someone please tell me how should I pass serializer to above method.
Thanks in Advance
It depends a little bit on how you would like to read the data from the socket. If you expect it to be the String representation of the data, then you could do it via:
joinedStreamEventDataStream.map(new MapFunction<Type, String>() {
#Override
public String map(Type value) throws Exception {
return value.toString();
}
}).writeToSocket(hostname, port, new SimpleStringSchema());
If you want to keep Flink's serialization format, then you can do write:
joinedStreamEventDataStream.writeToSocket(
hostname,
port,
new TypeInformationSerializationSchema<>(
joinedStreamEventDataStream.getType(),
env.getConfig()));
If you want to output it in your own serialization format, then you have to implement your own SerializationSchema as pointed out by Alex.
The writeToSocket() method takes 3 arguments: a socket host and port and also an implementation of SerializationSchema interface which used to serialize your data. So your implementation maybe like this:
joinedStreamEventDataStream.writeToSocket(
"192.168.1.10", // host name
6998, // port
new SerializationSchema<Integer>() {
#Override
public byte[] serialize(Integer element) {
return ByteBuffer.allocate(4).putInt(element).array();
}
}
);
It's true if joinedStreamEventDataStream has DataStream<Integer> type.
Related
I'm looking for a way to copy some headers from the request message to the response message when I use RabbitMq in RPC mode.
so far I have tried with setBeforeSendReplyPostProcessors but I can only access the response and add headers to it. but I don't have access to the request to get the values I need.
I have also tried with the advice chain, but the returnObject is null after proceeding so I can't modify it (I admit I don't understand why it is null... I thought I could get the object to modify it):
#Bean
public SimpleRabbitListenerContainerFactory simpleRabbitListenerContainerFactory(SimpleRabbitListenerContainerFactoryConfigurer simpleRabbitListenerContainerFactoryConfigurer, ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory simpleRabbitListenerContainerFactory = new SimpleRabbitListenerContainerFactory();
simpleRabbitListenerContainerFactory.setAdviceChain(new MethodInterceptor() {
#Override
public Object invoke(MethodInvocation invocation) throws Throwable {
Object returnObject = invocation.proceed();
//returnObject is null here
return returnObject;
}
});
simpleRabbitListenerContainerFactoryConfigurer.configure(simpleRabbitListenerContainerFactory, connectionFactory);
return simpleRabbitListenerContainerFactory;
}
a working way is to change my method annotated with #RabbitListener so it returns a Message and there I can access both the requesting message (via arguments of the annotated method) and the response.
But I would like to do it automatically, since I need this feature at different places.
Basicaly I want to copy one header from the request message to the response.
this code do the job, but I want to do it through an aspect, or an interceptor.
#RabbitListener(queues = "myQueue"
, containerFactory = "simpleRabbitListenerContainerFactory")
public Message<MyResponseObject> execute(MyRequestObject myRequestObject, #Header("HEADER_TO_COPY") String headerToCopy) {
MyResponseObject myResponseObject = compute(myRequestObject);
return MessageBuilder.withPayload(myResponseObject)
.setHeader("HEADER_RESPONSE", headerToCopy)
.build();
}
The Message<?> return type support was added for this reason, but we could add an extension point to allow this, please open a GitHub issue.
Contributions are welcome.
I am working on a usecase where I am supposed to poll S3 -> read the stream for the content -> do some processing and upload it to another bucket rather than writing the file in my server.
I know I can achieve it using S3StreamingMessageSource in Spring aws integration but the problem I am facing is that I do not know on how to process the message stream received by polling
public class S3PollerConfigurationUsingStreaming {
#Value("${amazonProperties.bucketName}")
private String bucketName;
#Value("${amazonProperties.newBucket}")
private String newBucket;
#Autowired
private AmazonClientService amazonClient;
#Bean
#InboundChannelAdapter(value = "s3Channel", poller = #Poller(fixedDelay = "100"))
public MessageSource<InputStream> s3InboundStreamingMessageSource() {
S3StreamingMessageSource messageSource = new S3StreamingMessageSource(template());
messageSource.setRemoteDirectory(bucketName);
messageSource.setFilter(new S3PersistentAcceptOnceFileListFilter(new SimpleMetadataStore(),
"streaming"));
return messageSource;
}
#Bean
#Transformer(inputChannel = "s3Channel", outputChannel = "data")
public org.springframework.integration.transformer.Transformer transformer() {
return new StreamTransformer();
}
#Bean
public S3RemoteFileTemplate template() {
return new S3RemoteFileTemplate(new S3SessionFactory(amazonClient.getS3Client()));
}
#Bean
public PollableChannel s3Channel() {
return new QueueChannel();
}
#Bean
IntegrationFlow fileStreamingFlow() {
return IntegrationFlows
.from(s3InboundStreamingMessageSource(),
e -> e.poller(p -> p.fixedDelay(30, TimeUnit.SECONDS)))
.handle(streamFile())
.get();
}
}
Can someone please help me with the code to process the stream ?
Not sure what is your problem, but I see that you have a mix of concerns. If you use messaging annotations (see #InboundChannelAdapter in your config), what is the point to use the same s3InboundStreamingMessageSource in the IntegrationFlow definition?
Anyway it looks like you have already explored for yourself a StreamTransformer. This one has a charset property to convert your InputStreamfrom the remote S3 resource to the String. Otherwise it returns a byte[]. Everything else is up to you what and how to do with this converted content.
Also I don't see reason to have an s3Channel as a QueueChannel, since the start of your flow is pollable anyway by the #InboundChannelAdapter.
From big height I would say we have more questions to you, than vise versa...
UPDATE
Not clear what is your idea for InputStream processing, but that is really a fact that after S3StreamingMessageSource you are going to have exactly InputStream as a payload in the next handler.
Also not sure what is your streamFile(), but it must really expect InputStream as an input from the payload of the request message.
You also can use the mentioned StreamTransformer over there:
#Bean
IntegrationFlow fileStreamingFlow() {
return IntegrationFlows
.from(s3InboundStreamingMessageSource(),
e -> e.poller(p -> p.fixedDelay(30, TimeUnit.SECONDS)))
.transform(Transformers.fromStream("UTF-8"))
.get();
}
And the next .handle() will be ready for String as a payload.
UPD. Sorry, guys.
I have an application that acts as a SOAP server, how do I write a PHPUnit test to test it?
SOAP extension is reading data from PHP input stream. You just provide your own data there and create some integration/unit tests for your API.
Take a look at the signature of SoapServer::handle() method. It takes as an argument a string which is a request itself. This parameter is optional and if you don't pass anything in, PHP will just read the data itself. But you can simply override it.
I used streams to do it. First you wrap the SoapServer with your own class like this:
class MyServer
{
/** \SoapServer */
private $soapServer;
public function __construct(\SoapServer $soapServer)
{
$this->soapServer = $soapServer;
}
public function handle(Psr\Http\Message\StreamInterface $inputStream): void
{
$this->soapServer->handle($inputStream->getContent());
}
}
Now you are ready to mock the request.
In your test you can do:
class MyTest extends TestCase
{
public function testMyRequest(): void
{
$mySoapServer = $this->createMySoapServer();
$request = $this->createRequest();
$mySoapServer->handle($request);
}
private function createRequest(): StreamInterface
{
$requestString = '<soap:Envelope></soap:Envelope>';
$fh = fopen('php://temp', 'rw');
fwrite($fh, $requestString);
fseek($fh, SEEK_SET);
return new Psr\Http\Message\StreamInterface\Stream($fh);
}
private function createMySoapServer(): MyServer
{
return new MyServer(new \SoapServer());
}
}
One thing to keep in mind - this test will generate output. You may want to test this output or ignore it. Depends on your use case.
Another side note. What you are asking for has really nothing to do with PHPUnit. It just a matter of designing your SOAP server correctly.
If you are wondering how to set up the stream when you have a live request, this is really simple:
$server->handle(new Psr\Http\Message\StreamInterface\Stream(fopen('php://input', 'r+')));
Given this entry in application.properties:
server.port=0
which causes Spring Boot to chose a random available port, and testing a spring boot web application using spock, how can the spock code know which port to hit?
Normal injection like this:
#Value("${local.server.port}")
int port;
doesn't work with spock.
You can find the port using this code:
int port = context.embeddedServletContainer.port
Which for those interested in the java equivalent is:
int port = ((TomcatEmbeddedServletContainer)((AnnotationConfigEmbeddedWebApplicationContext)context).getEmbeddedServletContainer()).getPort();
Here's an abstract class that you can extends which wraps up this initialization of the spring boot application and determines the port:
abstract class SpringBootSpecification extends Specification {
#Shared
#AutoCleanup
ConfigurableApplicationContext context
int port = context.embeddedServletContainer.port
void launch(Class clazz) {
Future future = Executors.newSingleThreadExecutor().submit(
new Callable() {
#Override
public ConfigurableApplicationContext call() throws Exception {
return (ConfigurableApplicationContext) SpringApplication.run(clazz)
}
})
context = future.get(20, TimeUnit.SECONDS);
}
}
Which you can use like this:
class MySpecification extends SpringBootSpecification {
void setupSpec() {
launch(MyLauncher.class)
}
String getBody(someParam) {
ResponseEntity entity = new RestTemplate().getForEntity("http://localhost:${port}/somePath/${someParam}", String.class)
return entity.body;
}
}
The injection will work with Spock, as long as you've configured your spec class correctly and have spock-spring on the classpath. There's a limitation in Spock Spring which means it won't bootstrap your Boot application if you use #SpringApplicationConfiguration. You need to use #ContextConfiguration and configure it manually instead. See this answer for the details.
The second part of the problem is that you can't use a GString for the #Value. You could escape the $, but it's easier to use single quotes:
#Value('${local.server.port}')
private int port;
Putting this together, you get a spec that looks something like this:
#ContextConfiguration(loader = SpringApplicationContextLoader, classes = SampleSpockTestingApplication.class)
#WebAppConfiguration
#IntegrationTest("server.port=0")
class SampleSpockTestingApplicationSpec extends Specification {
#Value("\${local.server.port}")
private int port;
def "The index page has the expected body"() {
when: "the index page is accessed"
def response = new TestRestTemplate().getForEntity(
"http://localhost:$port", String.class);
then: "the response is OK and the body is welcome"
response.statusCode == HttpStatus.OK
response.body == 'welcome'
}
}
Also note the use of #IntegrationTest("server.port=0") to request a random port be used. It's a nice alternative to configuring it in application.properties.
You could do this too:
#Autowired
private org.springframework.core.env.Environment springEnv;
...
springEnv.getProperty("server.port");
I wanted to take advantage of built-in content negotiator and just get access to decision what formatter is going to be used. I don't want to use Request.Headers.Accept and check for whether it is json or xml content type because there lot of things are involved in that decision. Is there a way I can check at controller level or override any class that tells me what formatter going to be used OR what request content type is?
thanks in advance.
You can run conneg manually:
var conneg = Configuration.Services.GetContentNegotiator();
var connegResult = conneg.Negotiate(
typeof(YOUR_TYPE), Request, Configuration.Formatters
);
And use the output whichever way you want:
//the valid media type
var mediaType = connegResult.MediaType;
//do stuff
//the relevant formatter
var formatter = connegResult.Formatter;
//do stuff
If you want to see what is going on then install a TraceWriter and you will see what the conneg does.
A TraceWriter looks something like:
public class TraceWriter : ITraceWriter {
public bool IsEnabled(string category, TraceLevel level) {
return true;
}
public void Trace(HttpRequestMessage request, string category, TraceLevel level, Action<TraceRecord> traceAction) {
var rec = new TraceRecord(request, category, level);
traceAction(rec);
Log(rec);
}
private void Log(TraceRecord record) {
Console.WriteLine(record.Message);
}
}
and is installed like this,
config.Services.Replace(typeof(ITraceWriter), new TraceWriter());
If you want to manually invoke conneg then you can use,
config.Services.GetContentNegotiator().Negotiate(...)
Tugberk has a blog on this. Have a look.