I'm using JBoss AS 7 Hornetq. Our standalone java application interacts with a queue and sends messages. We had the entire environment setup and it was working pretty smoothly. Suddenly, one fine day, our standalone application failed with the below exception:
Caused by: java.io.NotSerializableException: org.hornetq.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy
Detailed exception stack trace is below
javax.naming.NamingException: Failed to lookup [Root exception is java.io.NotSerializableException: org.hornetq.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy]
at org.jboss.naming.remote.client.ClientUtil.namingException(ClientUtil.java:36)
at org.jboss.naming.remote.protocol.v1.Protocol$1.execute(Protocol.java:104)
at org.jboss.naming.remote.protocol.v1.RemoteNamingStoreV1.lookup(RemoteNamingStoreV1.java:79)
at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:79)
at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:83)
at javax.naming.InitialContext.lookup(Unknown Source)
at com.infosys.lbs.publishing.LocationProcessor.postMessageInQueue(LocationProcessor.java:377)
at com.infosys.lbs.publishing.LocationProcessor.process(LocationProcessor.java:69)
at com.infosys.lbs.publishing.main.Publisher.main(Publisher.java:34)
Caused by: java.io.NotSerializableException: org.hornetq.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:891)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62)
at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119)
at org.jboss.naming.remote.protocol.v1.Protocol$1$2.write(Protocol.java:138)
at org.jboss.naming.remote.protocol.v1.WriteUtil.write(WriteUtil.java:61)
at org.jboss.naming.remote.protocol.v1.Protocol$1.handleServerMessage(Protocol.java:128)
at org.jboss.naming.remote.protocol.v1.RemoteNamingServerV1$MessageReciever$1.run(RemoteNamingServerV1.java:73)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: an exception which occurred:
in field loadBalancingPolicy
in field serverLocator
in object org.hornetq.jms.client.HornetQJMSConnectionFactory#ea074d
Exception was happening when the app was trying to lookup the connection factory
QueueConnectionFactory qcf = (QueueConnectionFactory)context.lookup("jms/RemoteConnectionFactory");
Below are the steps on how we resolved the issue
There was almost zero help for resolving this issue. A web search on this exception returned next to nothing. However, this particular thread on JBoss AS Dev site spinned a thought in my head: RemoteConnectionFactory is not found when looking up in a remote client
The scenario mentioned in this thread was not same as ours. (In our app this is the first and the only lookup happening.) This thread got me thinking towards a possible connection factory initialization issue. While there is nothing I could do to debug or find the issue around it, I thought that if I could reinitialize it, that would help.
So I tried lookup with java:jboss/exported/jms/RemoteConnectionFactory As expected it failed with a NamingException. Hoping that this naming syntax (using java:/) would have resulted in a reinitialization, I tried lookup again with jms/RemoteConnectionFactory. And bingo!!! it worked!
Unfortunately, we still don't know why it happened, and if it is just a one-off case! Documenting it here just in case some mortal soul hits this issue.
Related
I am getting ClassCastException from AMQP internals while using Spring RabbitMQ. We found this in application logs on production and were not able to reproduce this, but it's a real pain for us.
Basically, it seems like the entire stack fails once this happens, because then we see invalid DeliveryTag being sent on that channel, which results in RabbitMQ closing this channel.
We are using AMQPClient 5.4.3 and SpringRabbit 2.1.2, RabbitMQ caching mode is Connection (because of HAProxy and failover), ACK mode is manual, Publisher returns are enabled and Publisher Confirms as well.
java.lang.ClassCastException: com.rabbitmq.client.impl.AMQImpl$Channel$OpenOk cannot be cast to com.rabbitmq.client.impl.AMQImpl$Confirm$SelectOk
at com.rabbitmq.client.impl.ChannelN.confirmSelect(ChannelN.java:1552)
at com.rabbitmq.client.impl.ChannelN.confirmSelect(ChannelN.java:52)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.doCreateBareChannel(CachingConnectionFactory.java:658)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createBareChannel(CachingConnectionFactory.java:649)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.getCachedChannelProxy(CachingConnectionFactory.java:608)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.getChannel(CachingConnectionFactory.java:499)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.access$1600(CachingConnectionFactory.java:100)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$ChannelCachingConnectionProxy.createChannel(CachingConnectionFactory.java:1331)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1967)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1935)
at org.springframework.amqp.rabbit.core.RabbitTemplate.send(RabbitTemplate.java:889)
at org.springframework.amqp.rabbit.core.RabbitTemplate.convertAndSend(RabbitTemplate.java:955)
at org.springframework.amqp.rabbit.core.RabbitTemplate.convertAndSend(RabbitTemplate.java:948)
at net.homecredit.lisa.messaging.amqp.SaveApplicationResponseProducer.sendResponse(SaveApplicationResponseProducer.kt:40)
at net.homecredit.lisa.messaging.amqp.SaveApplicationConsumer.onReceiveRequestMessage(SaveApplicationConsumer.kt:87)
at sun.reflect.GeneratedMethodAccessor372.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:170)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:49)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:190)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:127)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1521)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1444)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1431)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1410)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:848)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:832)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$700(SimpleMessageListenerContainer.java:78)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1073)
at java.lang.Thread.run(Thread.java:748)
This is complete mystery for me.
Does anyone know what's wrong?
The RabbitMQ client can be configured to check the response type for RPC via channelShouldCheckRpcResponseType, see also https://github.com/rabbitmq/rabbitmq-java-client/issues/708.
My consumer worked fine for about one week, then the consumer thread died and logged the below error:
It seems the Authentication failure is caused by
BufferedInputStream.read timeout, and I want to know if there is a way
to treat the Authentication failure as a non-fatal and do not kill the
consumer thread. and I think the timeout issue is just caused by
network issue not the Authentication failure, since this consumer
already worked for one week.
org.springframework.amqp.rabbit.listener.exception.FatalListenerStartupException: Authentication failure
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:460)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1171)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.PossibleAuthenticationFailureException: Possibly caused by authentication failure
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:61)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:296)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:524)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils$1.createConnection(ConnectionFactoryUtils.java:85)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.doGetTransactionalResourceHolder(ConnectionFactoryUtils.java:135)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.getTransactionalResourceHolder(ConnectionFactoryUtils.java:71)
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:456)
... 2 more
Caused by: com.rabbitmq.client.PossibleAuthenticationFailureException: Possibly caused by authentication failure
at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:341)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:824)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:736)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:283)
... 7 more
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66) at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36) at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:366) at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:233) at com.rabbitmq.client.impl.AMQChannel.rpc(AMQChannel.java:211) at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:326)
The target code looks like:
} catch (ShutdownSignalException e) {
...
throw new PossibleAuthenticationFailureException(e);
}
So, indeed, it isn't necessary that the issue with the connection is about authentication.
There is just ShutdownSignalException by the Connection timed out reason.
So, you should just try to increase connection timeout, which is 60000 by default.
But that is likely there is some problem with resources on the broker since we just can't connect.
We can't treat this kind of errors (ShutdownSignalException) as non-fatal, since your problem happens exactly on the start().
EDIT
In case of fatal failure the container emits ListenerContainerConsumerFailedEvent:http://docs.spring.io/spring-amqp/reference/html/_reference.html#consumer-events. You can handle it and restart the container on purpose.
I am getting the below error, while I am trying to install Spring XD on YARN.
Error executing a spring application; nested exception is org.springframework.yarn.YarnSystemException:
Call From c01dfobi43.vcac.dc1.dsghost.net/100.98.226.45 to c01dfobi41.vcac.dc1.dsghost.net:8032 failed on connection exception:
java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused;
nested exception is java.net.ConnectException:
Call From c01dfobi43.vcac.dc1.dsghost.net/100.98.226.45 to c01dfobi41.vcac.dc1.dsghost.net:8032 failed on connection exception:
java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
Not sure where I am committing a mistake. Also do we need to install Spring XD Yarn on all nodes?
It would be great if you can share any documentation mentioned explicitly for YARN.
I am going to assume that c01dfobi41.vcac.dc1.dsghost.net:8032 is a ResourceManager host, I am also going to assume that based on your comment stating that yarn applications do run, you have more then one. In that case what may be happening (and I see this all the time) is that your yarn client attempts to contact the resource manager by looking it up in yarn-site.xml, it picks the first one and gets ConnectionRefused as the standby resource manager does not listen on its RPC port, it moves on to the next one and succeeds. If this is the case this is not a fatal error and can be ignored.
I am using bean validation in my application. When there is no constraint validation errors everything works nicely. Every time there is a validation error, Glassfish throws the following exception:
Caused by: java.lang.ClassNotFoundException: javax.validation.groups.Default: java.net.MalformedURLException: Unknown protocol: osgi
at com.sun.corba.ee.impl.util.JDKBridge.loadClassM(JDKBridge.java:325)
at com.sun.corba.ee.impl.util.JDKBridge.loadClass(JDKBridge.java:228)
at com.sun.corba.ee.impl.javax.rmi.CORBA.Util.loadClass(Util.java:640)
at com.sun.corba.ee.impl.util.RepositoryId.getClassFromType(RepositoryId.java:628)
at com.sun.corba.ee.impl.orbutil.RepIdDelegator.getClassFromType(RepIdDelegator.java:169)
at com.sun.corba.ee.impl.encoding.CDRInputStream_1_0.readClass(CDRInputStream_1_0.java:1439)
The bean-validation.jar is present in glassfish/modules folder. The startup doesn't throw any exceptions regarding validation.
PS. Note that we are using remote beans with CORBA.
Reproducable on GlassFish 3.1.2.2 and 3.1.1.
I traced this problem to serialization of the ConstraintValidationException through CORBA. Somehow the bean-validation module is not loaded properly with osgi and the javax.validation.groups.Default class is missing. I made a quick workaround so that the ConstraintValidationException is intercepted and never sent through CORBA. Instead the validation error information is gathered in a custom Exception class that can be actually serialized through the services.
I am using a wsdl2java generated web service client for the axis2 web service (axis2 version is 1.6.1). So for http connection the old version of commons-httpclient 3.1 is used. While doing some load tests to my web service client. In some cases i came across this exception
org.apache.axis2.AxisFault: org.apache.axiom.ext.io.StreamCopyException: Error reading from source
at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
at com.ardic.arcsp.carbon.registry.ws.stub.WSStorageServiceStub.fromOM(WSStorageServiceStub.java:16706)
at com.ardic.arcsp.carbon.registry.ws.stub.WSStorageServiceStub.wSgetById(WSStorageServiceStub.java:6659)
at com.ardic.arcsp.update.wsclient.WSStorageServiceClient.getResourceById(WSStorageServiceClient.java:177)
at com.ardic.arcsp.update.wsclient.OTAUpdateTestServlet.downloadResource(OTAUpdateTestServlet.java:162)
at com.ardic.arcsp.update.wsclient.OTAUpdateTestServlet.doGet(OTAUpdateTestServlet.java:91)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:273)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:517)
at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:305)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:165)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:298)
at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:57)
at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:189)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.wso2.carbon.server.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:154)
at org.wso2.carbon.server.TomcatServer$1.invoke(TomcatServer.java:254)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:563)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:399)
at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:396)
at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:356)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1534)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:705)
Caused by: org.apache.axiom.om.OMException: org.apache.axiom.ext.io.StreamCopyException: Error reading from source
at org.apache.axiom.attachments.impl.PartFactory.createPart(PartFactory.java:172)
at org.apache.axiom.attachments.Attachments.getPart(Attachments.java:705)
at org.apache.axiom.attachments.Attachments.getNextPartDataHandler(Attachments.java:623)
at org.apache.axiom.attachments.Attachments.getDataHandler(Attachments.java:350)
at org.apache.axiom.om.impl.builder.OMAttachmentAccessorMimePartProvider.getDataHandler(OMAttachmentAccessorMimePartProvider.java:45)
at org.apache.axiom.util.stax.xop.XOPDecodingStreamReader$DataHandlerProviderImpl.getDataHandler(XOPDecodingStreamReader.java:81)
at org.apache.axiom.util.stax.xop.XOPDecodingStreamReader.getDataHandler(XOPDecodingStreamReader.java:472)
at org.apache.axiom.om.impl.SwitchingWrapper.getDataHandler(SwitchingWrapper.java:1360)
at org.apache.axiom.util.stax.XMLStreamReaderUtils.getDataHandlerFromElement(XMLStreamReaderUtils.java:230)
at com.ardic.arcsp.carbon.registry.ws.stub.xsd.WSResource$Factory.parse(WSResource.java:2087)
at com.ardic.arcsp.carbon.registry.ws.stub.WSgetByIdResponse$Factory.parse(WSgetByIdResponse.java:417)
at com.ardic.arcsp.carbon.registry.ws.stub.WSStorageServiceStub.fromOM(WSStorageServiceStub.java:16273)
... 37 more
Caused by: org.apache.axiom.ext.io.StreamCopyException: Error reading from source
at org.apache.axiom.attachments.utils.BAAOutputStream.readFrom(BAAOutputStream.java:114)
at org.apache.axiom.attachments.impl.BufferUtils.inputStream2OutputStream(BufferUtils.java:76)
at org.apache.axiom.attachments.impl.PartFactory.createPart(PartFactory.java:136)
... 48 more
Caused by: java.io.IOException: CRLF expected at end of chunk: 116/161
at org.apache.commons.httpclient.ChunkedInputStream.readCRLF(ChunkedInputStream.java:207)
at org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputStream.java:219)
at org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream.java:176)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:108)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at org.apache.axiom.om.util.DetachableInputStream.read(DetachableInputStream.java:147)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at java.io.PushbackInputStream.read(PushbackInputStream.java:169)
at org.apache.axiom.attachments.BoundaryPushbackInputStream.readFromStream(BoundaryPushbackInputStream.java:114)
at org.apache.axiom.attachments.BoundaryPushbackInputStream.read(BoundaryPushbackInputStream.java:248)
at org.apache.axiom.attachments.MIMEBodyPartInputStream.read(MIMEBodyPartInputStream.java:87)
at org.apache.axiom.attachments.utils.BAAOutputStream.readFrom(BAAOutputStream.java:112)
... 50 more
The web service client request files from the server and downloads the file. These exceptions occurs on the response while receiving files in chunks.
While searching i came across this fix mentioned here.
https://issues.apache.org/jira/browse/HTTPCORE-190
However the exceptions still occur. So what else can cause this exceptions?
UPDATE:
Ok, i got the real reason for exception.On the client side if the memory allocation for jvm is low, the response from the server cannot be processed due to SocketTimeOut. But the exception itself could not stress this. Increasing the heap on the client side can solve the problem temporary.
You definitely need enough memory to hold all the data that is chucked to the client. Java can give really weird results, especially the older JVMs, when there isn't enough memory to go around. It can be tricky sometimes to identify that as the root cause so bravo on that find.
There are a few helpful big on the Apache Axis2 User's Guide - Creating Clients page that might be helpful if you are new to wsdl2java.
Our next step is to learn what command line options are available. One thing you can try is using the -u option to generate separate classes for the data bindings instead of creating them as inner classes. (Take a look at this Stack Overflow question.)
Since this question is 10 years old if you run into this or another error please create a new question specific to that context.