HttpRequest does not have getters for the Request entity to allow me to manipulate it in an interceptor. Checking the type using instanceof is not working either. Would anybody have any ideas about how I can accomplish this?
public class FastinfosetRequestInterceptor implements HttpRequestInterceptor
{
#Override
public void process(HttpRequest request, HttpContext context)
throws HttpException, IOException
{
if(request instanceof HttpPost)
{
HttpEntity rqEntity = ((HttpPost)request).getEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
rqEntity.writeTo(baos);
byte[] encodedRq;
try
{
encodedRq = FastInfosetUtils.encodeToFastInfoSet(baos.toByteArray());
}
catch (ParserConfigurationException | SAXException
| TransformerException e)
{
throw new IOException("Error while encoding request to FastInfoSet", e);
}
((HttpPost) request).setEntity(new ByteArrayEntity(encodedRq));
}
}
}
This should fix the problem
if(request instanceof HttpEntityEnclosingRequest)
{
HttpEntity rqEntity = ((HttpEntityEnclosingRequest) request).getEntity();
Related
I use kafka send quote message, qps is up to 50w, the consumer can handle it, but just only deserialize message, the CPU utilization up to 70!
Anybody knows how to optimize it?
Here is the deserialize code:
public class QuoteMessageDeserializer implements Deserializer<RawQuote> {
private final ObjectMapper objectMapper;
public QuoteMessageDeserializer() {
objectMapper = new ObjectMapper();
objectMapper.configure(DeserializationFeature.FAIL_ON_IGNORED_PROPERTIES, false);
objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
}
#Override
public void configure(Map<String, ?> configs, boolean isKey) {
}
#Override
public RawQuote deserialize(String topic, byte[] data) {
try {
QuoteMessage msg = objectMapper.readValue(data, QuoteMessage.class);
return msg.getData();
} catch (Exception e) {
……
}
}
#Override
public void close() {
}
}
I try to use AfterBurner,but it doesn't work to lower cpu utilization
public QuoteMessageDeserializer() {
objectMapper = new ObjectMapper();
objectMapper.registerModule(new AfterburnerModule());
objectMapper.configure(DeserializationFeature.FAIL_ON_IGNORED_PROPERTIES, false);
objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
}
thanks.
Any idea how I can add a header for my calls using SoapCore?
what I have so far:
at startup.cs:
app.UseSoapEndpoint<IMyService>("/MyService.svc", new BasicHttpBinding(), SoapSerializer.DataContractSerializer);
in IMyService
[ServiceContract]
public interface IMyService
{
[OperationContract]
public List<SOADataGetService> GetService(string ServiceType, string ServiceName, string ServiceVersion);
}
then my soap ends up like that:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://tempuri.org/">
<soapenv:Header/>
<soapenv:Body>
<tem:GetService>
<tem:ServiceType>?</tem:ServiceType>
<tem:ServiceName>?</tem:ServiceName>
<tem:ServiceVersion>?</tem:ServiceVersion>
</tem:GetService>
</soapenv:Body>
</soapenv:Envelope>
I need to get in <soapenv:Header/> like user and password
You can access the header in SoapCore by implementing and registering a custom IServiceOperationTuner as described in the docs.
e.g.
public class MyServiceOperationTuner : IServiceOperationTuner
{
public void Tune(HttpContext httpContext, object serviceInstance, SoapCore.ServiceModel.OperationDescription operation)
{
if (operation.Name.Equals(nameof(MyService.SomeOperationName)))
{
MyService service = serviceInstance as MyService;
service.SetHttpRequest(httpContext.Request);
}
}
}
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
services.TryAddSingleton<IMyService, MyService>();
services.TryAddSingleton<IServiceOperationTuner>(provider => new MyServiceOperationTuner());
}
}
public class MyService : IMyService
{
private ThreadLocal<HttpRequest> _httpRequest = new ThreadLocal<HttpRequest>() { Value = null };
public void SetHttpRequest(HttpRequest request)
{
_httpRequest.Value = request;
}
public string SomeOperationName()
{
var soapHeader = GetHeaderFromRequest(_httpRequest.Value)
return $"SOAP Header: {soapHeader}";
}
private XmlNode GetHeaderFromRequest(HttpRequest request)
{
var bytes = (request.Body as MemoryStream)?.ToArray();
if (bytes == null)
{
// Body missing from request
return null;
}
var envelope = new XmlDocument();
envelope.LoadXml(Encoding.UTF8.GetString(bytes));
return envelope.DocumentElement?.ChildNodes.Cast<XmlNode>().FirstOrDefault(n => n.LocalName == "Header");
}
}
I hope this helps someone. I'm using SoapCore 1.1.0.28 with .Net Core 6. I tried the Tune method listed by #wolfyuk, but Core always returned bytes as null, so I was never able to get past the null check.
The most straightforward way I found is to use IMessageInspector2 from SoapCore to create middleware to intercept the SOAP request on the way in and intercept the SOAP response on the way out. Your class that implements IMessageInspector2 has access to the message so you can extract headers on the way in (that's what I needed), and add headers on the way out. I needed the request headers to be included in my response (a requirement of the system I'm communicating with).
public class AuthMessageFilter : IMessageInspector2
{
private const string WsNamespaceSecurityUri = "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd";
private const string WsUserNameTokenNodeName = "UsernameToken";
private const string WsSecurityNodeName = "Security";
private const string WsTimestampNodeName = "Timestamp";
private readonly IMyService _service;
private readonly IHttpContextAccessor _acc;
private readonly ILogger _logger;
private MessageHeaders _messageHeaders;
public AuthMessageFilter(IHttpContextAccessor acc, IMyService service, ILogger logger)
{
_acc = acc;
_service = service;
_logger = logger;
}
public object AfterReceiveRequest(ref Message message, ServiceDescription serviceDescription)
{
ValidateSoapAction();
var token = GetUserNameToken(message);
var userIsAuthenticated = _service.ValidateUser(token.Username, token.Password.Value).GetAwaiter().GetResult();
if (userIsAuthenticated)
{
_messageHeaders = message.Headers; // use in response.
return null;
}
const string msg = "The user credentials did not authenticate.";
_logger.LogEntry(msg);
throw new AuthenticationFailedException(msg);
}
private void ValidateSoapAction()
{
try
{
var soapAction = _acc.HttpContext?.Request.Headers["SOAPAction"].FirstOrDefault()?.Replace("\"", "");
if (soapAction == null)
{
throw new Exception(
"Error: Could not extract SoapAction from HttpContext.Request.Headers. Aborting SOAP operation.");
}
}
catch (Exception ex)
{
_logger.LogEntry("No SOAP Action found.", ex);
}
}
private WsUsernameToken GetUserNameToken(Message message)
{
WsUsernameToken wsUsernameToken = null;
for (var i = 0; i < _messageHeaders.Count; i++)
{
if (!_messageHeaders[i].Name.Equals(WsSecurityNodeName, StringComparison.OrdinalIgnoreCase))
continue;
using var reader = _messageHeaders.GetReaderAtHeader(i);
while (reader.Read())
{
if (reader.IsStartElement() &&
reader.NamespaceURI.Equals(WsNamespaceSecurityUri, StringComparison.OrdinalIgnoreCase) &&
reader.LocalName.Equals(WsUserNameTokenNodeName, StringComparison.OrdinalIgnoreCase))
{
var serializer = new XmlSerializer(typeof(WsUsernameToken));
wsUsernameToken = (WsUsernameToken)serializer.Deserialize(reader);
break;
}
}
break;
}
if (wsUsernameToken == null)
{
var ex = new SecurityException("An exception occurred when verifying security for the message.");
_logger.LogEntry(LoggingCategory.Service, LoggingLevel.Error, ex.Message, ex);
throw ex;
}
return wsUsernameToken;
}
public void BeforeSendReply(ref Message reply, ServiceDescription serviceDescription, object correlationState)
{
for (var i = 0; i < _messageHeaders.Count; i++)
{
if (!_messageHeaders[i].Name.Equals(WsSecurityNodeName, StringComparison.OrdinalIgnoreCase))
continue;
using var reader = _messageHeaders.GetReaderAtHeader(i);
while (reader.Read())
{
if (reader.IsStartElement() &&
reader.NamespaceURI.Equals(WsNamespaceSecurityUri, StringComparison.OrdinalIgnoreCase) &&
reader.LocalName.Equals(WsTimestampNodeName, StringComparison.OrdinalIgnoreCase))
{
reply.Headers.Add(_messageHeaders[i] as MessageHeader);
break;
}
}
break;
}
}
}
Do not use SoapCore it is outdated, try to use SmartSoap:
https://github.com/Raffa50/SmartSoap
it is also available as a nugetPackage:
https://www.nuget.org/packages/Aldrigos.SmartSoap.AspNet/
Have a look at it, try it and if you need further support I will be pleased to help you!
I have written the redis data enricher which will get the rules and timeout key from the redis based on the macid but it is working for sometime and after sometime it is not sending flowfile to next processor(it will be in running state but won't send flowfile to next processor). Nifi is working in cluster mode is there any thing wrong in the below processor(RedisDataEnricher).
In the below code I am taking redis connection only once and after that I'm using the same connection for fecthing data from redis.
public class RedisDataEnricher extends AbstractProcessor {
private volatile Jedis jedis;
public static final PropertyDescriptor ConnectionHost = new PropertyDescriptor
.Builder().name("ConnectionHost")
.displayName("ConnectionHost")
.description("ConnectionHost")
.required(true)
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.build();
public static final PropertyDescriptor ConnectionPort = new PropertyDescriptor
.Builder().name("ConnectionPort")
.displayName("ConnectionPort")
.description("ConnectionPort")
.required(true)
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.build();
public static final PropertyDescriptor JSONKEY = new PropertyDescriptor
.Builder().name("JSONKEY")
.displayName("JSONKEY")
.description("JSON key to be fetched from input")
.required(true)
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.build();
public static final PropertyDescriptor MACIDKEY = new PropertyDescriptor
.Builder().name("MACIDKEY")
.displayName("MACIDKEY")
.description("MACIDKEY to be fetched from input")
.required(true)
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.build();
public static final Relationship SUCCESS = new Relationship.Builder()
.name("SUCCESS")
.description("SUCCESS")
.build();
public static final Relationship FAILURE = new Relationship.Builder()
.name("FAILURE")
.description("FAILURE")
.build();
private List<PropertyDescriptor> descriptors;
private Set<Relationship> relationships;
#Override
protected void init(final ProcessorInitializationContext context) {
final List<PropertyDescriptor> descriptors = new ArrayList<PropertyDescriptor>();
descriptors.add(JSONKEY);
descriptors.add(MACIDKEY);
descriptors.add(ConnectionHost);
descriptors.add(ConnectionPort);
this.descriptors = Collections.unmodifiableList(descriptors);
final Set<Relationship> relationships = new HashSet<Relationship>();
relationships.add(SUCCESS);
relationships.add(FAILURE);
this.relationships = Collections.unmodifiableSet(relationships);
}
#Override
public Set<Relationship> getRelationships() {
return this.relationships;
}
#Override
public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
return descriptors;
}
#OnScheduled
public void onScheduled(final ProcessContext context) {
try {
jedis = new Jedis(context.getProperty("ConnectionHost").toString(),Integer.parseInt(context.getProperty("ConnectionPort").toString()));
} catch (Exception e) {
getLogger().error("Unable to establish Redis connection.");
}
}
#Override
public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
FlowFile flowFile = session.get();
if ( flowFile == null ) {
return;
}
else{
try{
InputStream inputStream = session.read(flowFile);
StringWriter writer = new StringWriter();
IOUtils.copy(inputStream, writer, "UTF-8");
Jedis jedis1=jedis;
JSONObject json=new JSONObject(writer.toString());
inputStream.close();
JSONObject json1=new JSONObject();
String rules=jedis1.hget(json.getJSONObject(context.getProperty("JSONKEY").toString()).getString(context.getProperty("MACIDKEY").toString()), "rules");
json1.put("data", json.getJSONObject(context.getProperty("JSONKEY").toString()));
json1.put("timeOut", jedis1.hget(json.getJSONObject(context.getProperty("JSONKEY").toString()).getString(context.getProperty("MACIDKEY").toString()),"timeOut"));
json1.put("rules", rules!=null?new ArrayList<String>(Arrays.asList(rules.split(" , "))):new ArrayList<>());
flowFile = session.write(flowFile, new OutputStreamCallback() {
#Override
public void process(OutputStream out) throws IOException {
out.write(json1.toString().getBytes());
}
});
flowFile = session.putAttribute(flowFile, "OutBound", jedis1.hget(json.getJSONObject(context.getProperty("JSONKEY").toString()).getString(context.getProperty("MACIDKEY").toString()),"OutBound"));
session.transfer(flowFile, SUCCESS);
}
catch(Exception e)
{
session.transfer(flowFile, FAILURE);
}
}
}
}
I have a REST handler servlet defined as follows (this works perfectly):
//REST handler context
ServletContextHandler restHandler = new ServletContextHandler(ServletContextHandler.SESSIONS);
restHandler.setContextPath("/");
// Jersey REST handling servlet
ServletHolder jerseyServlet = restHandler.addServlet(org.glassfish.jersey.servlet.ServletContainer.class, "/*");
jerseyServlet.setInitOrder(0);
// Tell Jersey which REST service class to load....
jerseyServlet.setInitParameter("jersey.config.server.provider.classnames", RestHandler.class.getCanonicalName());
I now want to add a authentication filter, which I do as:
FilterHolder authFilter = restHandler.addFilter(AuthFilter.class, "/",
EnumSet.of( DispatcherType.ASYNC,
DispatcherType.ERROR,
DispatcherType.FORWARD,
DispatcherType.INCLUDE,
DispatcherType.REQUEST));
if (authFilter == null) {
dlog.debug("Failed to load authentication filter");
};
All good so far, however, the filter does not fire on incoming REST. Calls still go through. The AuthFilter is straight from sample code:
public class AuthFilter implements javax.servlet.Filter {
private static final Logger dlog = Dlog.get();
public static final String AUTHENTICATION_HEADER = "Authorization";
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain filter)
throws IOException, ServletException {
dlog.entry(request, response, filter);
if (request instanceof HttpServletRequest) {
HttpServletRequest httpServletRequest = (HttpServletRequest) request;
String authCredentials = httpServletRequest.getHeader(AUTHENTICATION_HEADER);
AuthService authenticationService = new AuthService();
boolean authenticationStatus = authenticationService.authenticate(authCredentials);
if (authenticationStatus) {
filter.doFilter(request, response);
} else {
if (response instanceof HttpServletResponse) {
HttpServletResponse httpServletResponse = (HttpServletResponse) response;
httpServletResponse.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
}
}
}
dlog.exit();
}
#Override
public void destroy() {
}
#Override
public void init(FilterConfig arg0) throws ServletException {
}
}
I use handler collection as I also have a resource handler to serve static web pages besides the REST calls.
HandlerCollection handlerList = new HandlerCollection();
handlerList.setHandlers(new Handler[] { resourceHandler,
restHandler,
new DefaultHandler(),
requestLogHandler });
What else I need to do? I have scanned through number of related posts and come up empty. Thanks in advance.
In this official example for Apache HttpClient, there's no mention of releasing request or response objects. Are they released as part of httpclient.close() or releaseResources method needs to be overridden with something?
final CountDownLatch latch2 = new CountDownLatch(1);
final HttpGet request3 = new HttpGet("http://www.apache.org/");
HttpAsyncRequestProducer producer3 = HttpAsyncMethods.create(request3);
AsyncCharConsumer<HttpResponse> consumer3 = new AsyncCharConsumer<HttpResponse>() {
HttpResponse response;
#Override
protected void onResponseReceived(final HttpResponse response) {
this.response = response;
}
#Override
protected void onCharReceived(final CharBuffer buf, final IOControl ioctrl) throws IOException {
// Do something useful
}
#Override
protected void releaseResources() {
}
#Override
protected HttpResponse buildResult(final HttpContext context) {
return this.response;
}
};
httpclient.execute(producer3, consumer3, new FutureCallback<HttpResponse>() {
public void completed(final HttpResponse response3) {
latch2.countDown();
System.out.println(request2.getRequestLine() + "->" + response3.getStatusLine());
}
public void failed(final Exception ex) {
latch2.countDown();
System.out.println(request2.getRequestLine() + "->" + ex);
}
public void cancelled() {
latch2.countDown();
System.out.println(request2.getRequestLine() + " cancelled");
}
});
latch2.await();
} finally {
httpclient.close();
}
One needs to override #releaseResources() only if the consumer makes use of system resources such as files, pipes, etc. If response content is always held in memory it gets GCed the normal way.