I'm using AWS SNS SMS feature and send sms to the Phone. I want to read delivery status success|failed.
For that i'm following the below steps:
Create Cloudwatch logs group.
Create Kinesis subscription filter.
Now log will be available on Kinesis steam.
I'm able to read logs from Kinesis steam but not in required format, i want it in json format.
enter image description here
If i'm send direct data to Kinesis stream and reading back it work fine and properly readable format(json).
Delivery logs:
{
"notification": {
"messageId": "0aaabb6c-35ab-5a0e-b446-e1048f5623b9",
"timestamp": "2022-01-24 14:33:33.441"
},
"delivery": {
"destination": "<phone-number>",
"smsType": "Transactional",
"providerResponse": "Sandboxed account unable to send to number.",
"dwellTimeMs": 44
},
"status": "FAILURE"
}
Code:
DeliveryStatusProcessor#processRecords(ProcessRecordsInput processRecordsInput)
public void processRecords(ProcessRecordsInput processRecordsInput) {
try {
log.info("Processing {} record(s)", processRecordsInput.records().size());
processRecordsInput.records().forEach(incomingDeliveryStatus ->
{
try {
processRecord(incomingDeliveryStatus);
} catch (IOException e) {
log.info("Failed to process records.");
}
});
} catch (Throwable t) {
log.error("Caught throwable while processing records. Aborting.");
Runtime.getRuntime().halt(1);
} finally {
//
}
}
private void processRecord(KinesisClientRecord record) throws IOException {
byte[] messageStatus = new byte[record.data().remaining()];
record.data().get(messageStatus);
String string = new String(messageStatus);
System.out.println("================>>>"+string);
}
cloudwatch sends logs in gzip compressed format. you need to decompress them in consumer application to get in correct format
Related
I'm subscribing to a MQTT Topic(in my case it is app unique user id).I'm using AWS IOT core services for subscription.Whenever home screen opens up and I got connected callback from awsConnectClient,I make the call for subscription. Now what is happening if app gets open up three times It subscribed to the same topic 3 time.Now whenever any message publish to that topic.It received by app 3 times.
Now what I want to do that I want to know that if this userId is already subscribed from this device I would not make a call for subscription again from same device.
One approach could be If I save in my app that I had already subscribed to this topic and do not make the call for subscription again. but I doubt if this approach could be correct for all scenarios.Could it be possible that we could drive this logic from the server end only, if any aws iot api could give me that this is already subscribed.
fun connectClick() {
Log.d(TAG, "clientId = $clientId")
try {
mqttManager.connect(clientKeyStore) { status, throwable ->
Log.d(TAG, "Status = " + status.toString())
var formattedStatus = String.format(getString(R.string.status_msg),status.toString())
if (status == AWSIotMqttClientStatusCallback.AWSIotMqttClientStatus.Connected) {
Log.i(TAG, " subscribed to - " + VoiceXPreference(this).rosterName)
unsubscribe()
subscribeClick(VoiceXPreference(this).rosterName)
}
runOnUiThread {
tv_iot_status.text = formattedStatus
if (throwable != null) {
Log.e(TAG, "Connection error.", throwable)
}
}
}
} catch (e: Exception) {
Log.e(TAG, "Connection error.", e)
}
}
Above is my subscription code.Although I'm always unsubscribing before subscribing but this is not working for me.
Following is my initClient call which makes the connection request. I have added the if check if mqttManager is already initialised first disconnect and then make connect request. Although I have put initRequest inside onCreate() call back of app screen which calls only once when the app opens up. I have checked the logs it is being called only once.
AWSMobileClient.getInstance().initialize(this, object : Callback<UserStateDetails> {
override fun onResult(result: UserStateDetails) {
Log.i(TAG,"connect request called");
if(mqttManager != null){
mqttManager?.disconnect()
}
initIoTClient()
}
override fun onError(e: Exception) {
Log.e(TAG, "onError: ", e)
}
})
Following is my subscribe code snippet which is subscribing to unique userId
fun subscribeClick(topic: String) {
Log.d(TAG, "topic = $topic")
try {
mqttManager?.subscribeToTopic(topic, AWSIotMqttQos.QOS0,
{ topic, data ->
runOnUiThread {
try {
val message = String(data, Charsets.UTF_8)
Log.d(TAG, "Message arrived:")
Log.d(TAG, " Topic: $topic")
Log.d(TAG, " Message: $message")
val gson = Gson()
val notificationModel = gson.fromJson(message, NotificationModel::class.java)
var orderServiceMapperResponseModel = OrderServiceMapperResponseModel()
orderServiceMapperResponseModel.seatId = notificationModel.seatId
orderServiceMapperResponseModel.serviceName = notificationModel.service
orderServiceMapperResponseModel.id = notificationModel.id
orderServiceMapperResponseModel.createdDate = notificationModel.createdDate
serviceList.add(orderServiceMapperResponseModel)
if (isPictureInPictureMode) {
if (isShownNotification) {
updateNotificationCount()
} else {
updatePIPWindowContent()
}
} else {
updateAdapterDataSource()
}
} catch (e: UnsupportedEncodingException) {
Log.e(TAG, "Message encoding error.", e)
}
}
})
} catch (e: Exception) {
Log.e(TAG, "Subscription error.", e)
}
}
I'm also always making disconnect() request inside onDestroy() of my app screen
mqttManager?.disconnect()
But Still I'm getting 3 subscription messages instead of 1.
You receive 3 duplicated messages not because you subscribe 3 times but because you create 3 individual connections.
The MQTT specification clearly states that
If a Server receives a SUBSCRIBE Packet containing a Topic Filter that is identical to an existing Subscription’s Topic Filter then it MUST completely replace that existing Subscription with a new Subscription.
meaning duplicated subscriptions per connection never happen, unless the server has a broken implementation.
Your code looks like that it never send disconnect requests while a new connection is created whenever the code block is invoked.
You should keep a single MQTT session, or make sure you close the connection when the app is closed.
Our application is storing data the user enters in SalesForce. We are using Spring AMQP (RabbitMQ) to be able to store data and run the application even if SalesForce is not reachable (scheduled downtimes or else).
So once SalesForce is not available the queue should wait with retrying messages until it is up again. Is this possible using the CircuitBreakerRetryPolicy or do i have to implement a custom RetryPolicy?
Currently i just have a simply retry policy which should be replaced by a complex one that blocks the retries until the external system is reachable again.
#Bean
RetryOperationsInterceptor salesForceInterceptor() {
return RetryInterceptorBuilder.stateless()
.recoverer(new RejectAndDontRequeueRecoverer())
.retryPolicy(simpleRetryPolicy())
.build();
}
Message listener:
#Override
public void onMessage(Message message) {
try {
.....
} catch (ResourceAccessException e) {
//TODO probablyy wrong exception
throw new AmqpTimeoutException("Salesforce can not be accessed, retry later");
} catch (Exception e) {
throw new AmqpRejectAndDontRequeueException("Fatal error with message to salesforce, do not retry", e);
}
}
In RabbitMQ,I have a failure queue, in which I have all the failed messages from different Queues. Now I want to give the functionality of 'Retry', so that administrator can again move the failed messages to their respective queue. The idea is something like that:
Above diagram is structure of my failure queue. After click on Retry link, message should move into original queue i.e. queue1, queue2 etc.
If you are looking for a Java code to do this, then you have to simply consume the messages you want to move and publish those messages to the required queue. Just look up on the Tutorials page of rabbitmq if you are unfamiliar with basic consuming and publishing operations.
It's not straight forward consume and publish. RabbitMQ is not designed in that way. it takes into consideration that exchange and queue both could be temporary and can be deleted. This is embedded in the channel to close the connection after single publish.
Assumptions:
- You have a durable queue and exchange for destination ( to send to)
- You have a durable queue for target ( to take from )
Here is the code to do so:
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.QueueingConsumer;
import org.apache.commons.lang.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.rabbit.connection.CachingConnectionFactory;
public object shovelMessage(
String exchange,
String targetQueue,
String destinationQueue,
String host,
Integer port,
String user,
String pass,
int count) throws IOException, TimeoutException, InterruptedException {
if(StringUtils.isEmpty(exchange) || StringUtils.isEmpty(targetQueue) || StringUtils.isEmpty(destinationQueue)) {
return null;
}
CachingConnectionFactory factory = new CachingConnectionFactory();
factory.setHost(StringUtils.isEmpty(host)?internalHost.split(":")[0]:host);
factory.setPort(port>0 ? port: Integer.parseInt(internalPort.split(":")[1]));
factory.setUsername(StringUtils.isEmpty(user)? this.user: user);
factory.setPassword(StringUtils.isEmpty(pass)? this.pass: pass);
Channel tgtChannel = null;
try {
org.springframework.amqp.rabbit.connection.Connection connection = factory.createConnection();
tgtChannel = connection.createChannel(false);
tgtChannel.queueDeclarePassive(targetQueue);
QueueingConsumer consumer = new QueueingConsumer(tgtChannel);
tgtChannel.basicQos(1);
tgtChannel.basicConsume(targetQueue, false, consumer);
for (int i = 0; i < count; i++) {
QueueingConsumer.Delivery msg = consumer.nextDelivery(500);
if(msg == null) {
// if no message found, break from the loop.
break;
}
//Send it to destination Queue
// This repetition is required as channel looses the connection with
//queue after single publish and start throwing queue or exchange not
//found connection.
Channel destChannel = connection.createChannel(false);
try {
destChannel.queueDeclarePassive(destinationQueue);
SerializerMessageConverter serializerMessageConverter = new SerializerMessageConverter();
Message message = new Message(msg.getBody(), new MessageProperties());
Object o = serializerMessageConverter.fromMessage(message);
// for some reason msg.getBody() writes byte array which is read as a byte array // on the consumer end due to which this double conversion.
destChannel.basicPublish(exchange, destinationQueue, null, serializerMessageConverter.toMessage(o, new MessageProperties()).getBody());
tgtChannel.basicAck(msg.getEnvelope().getDeliveryTag(), false);
} catch (Exception ex) {
// Send Nack if not able to publish so that retry is attempted
tgtChannel.basicNack(msg.getEnvelope().getDeliveryTag(), true, true);
log.error("Exception while producing message ", ex);
} finally {
try {
destChannel.close();
} catch (Exception e) {
log.error("Exception while closing destination channel ", e);
}
}
}
} catch (Exception ex) {
log.error("Exception while creating consumer ", ex);
} finally {
try {
tgtChannel.close();
} catch (Exception e) {
log.error("Exception while closing destination channel ", e);
}
}
return null;
}
To requeue a message you can use the receiveAndReply method. The following code will move all messages from the dlq-queue to the queue-queue:
do {
val movedToQueue = rabbitTemplate.receiveAndReply<String, String>(dlq, { it }, "", queue)
} while (movedToQueue)
In the code example above, dlq is the source queue, { it } is the identity function (you could transform the message here), "" is the default exchange and queue is the destination queue.
I also have implemented something like that, so I can move messages from a dlq back to processing. Link: https://github.com/kestraa/rabbit-move-messages
Here is a more generic tool for some administrative/supporting tasks, the management-ui is not capable of.
Link: https://github.com/bkrieger1991/rabbitcli
It also allows you to fetch/move/dump messages from queues even with a filter on message-content or message-headers :)
Today i wanted to switch from GCM to FCM so i set up everything needed and wanted to implement the server side code. I used the gcm4j library and changed it so that the adress goes to https://fcm.googleapis.com/fcm/send.
So im doing the following:
FCM fcm = new FCMDefault(new FCMConfig().withKey(FCMGlobals.FCM_API_KEY));
FCMRequest request = new FCMRequest().withRegistrationId(android.getRegistration())
// .withCollapseKey(collapseKey)
.withDelayWhileIdle(true)
.withDataItem(FCMGlobals.FCM_PARAM_CODE, code)
.withDataItem(FCMGlobals.FCM_PARAM_USER_ID, "" + user.getId())
.withDataItem(FCMGlobals.FCM_PARAM_ADDITION, "" + addition);
ListenableFuture<FCMResponse> responseFuture = fcm.send(request);
Futures.addCallback(responseFuture, new FutureCallback<FCMResponse>() {
public void onFailure(Throwable t) {
log.error(t);
}
#Override
public void onSuccess(FCMResponse response) {
log.info(response.toString());
}
});
The implementation for that is:
protected FCMResponse executeRequest(FCMRequest request) throws IOException {
byte[] content = this.objectMapper.writeValueAsBytes(request);
HttpURLConnection conn = this.connectionFactory.open(this.fcmUrl);
conn.setRequestMethod("POST");
conn.addRequestProperty("Authorization", getAuthorization(request));
conn.addRequestProperty("Content-Type", "application/json");
conn.setDoOutput(true);
conn.setFixedLengthStreamingMode(content.length);
LoggerFactory.getLogger("FCMDefaultAbstract").info("Authorization: " + conn.getRequestProperty("Authorization"));
LoggerFactory.getLogger("FCMDefaultAbstract").info("Content-Type: " + conn.getRequestProperty("Content-Type"));
LoggerFactory.getLogger("FCMDefaultAbstract").info("send: " + new String(content));
try (OutputStream outputStream = conn.getOutputStream()) {
IOUtils.write(content, outputStream);
} catch (Exception e) {
throw new FCMNetworkException("Error sending HTTP request to FCM", e);
}
FCMResponse response;
try (InputStream inputStream = conn.getInputStream()) {
response = this.objectMapper.readValue(IOUtils.toByteArray(inputStream), FCMResponse.class);
} catch (IOException e) {
try (InputStream inputStreamError = conn.getErrorStream()) {
String str = inputStreamError != null ? IOUtils.toString(inputStreamError) : "No error details provided";
int responseCode = conn.getResponseCode();
if (responseCode < 500) {
throw new FCMNetworkException(conn.getResponseCode(), str.trim(), e);
} else {
throw new FCMNetworkException(conn.getResponseCode(), str.trim(), checkForRetryInResponse(conn), e);
}
}
}
response.setRequest(request);
response.setRetryAfter(checkForRetryInResponse(conn));
Iterator<String> iteratorId = request.getRegistrationIds().iterator();
Iterator<FCMResult> iteratorResponse = response.getResults().iterator();
while (iteratorId.hasNext() && iteratorResponse.hasNext()) {
iteratorResponse.next().setRequestedRegistrationId(iteratorId.next());
}
if (iteratorId.hasNext()) {
LOG.warn("Protocol error: Less results than requested registation IDs");
}
if (iteratorResponse.hasNext()) {
LOG.warn("Protocol error: More results than requested registation IDs");
}
return response;
}
Here the log output:
FCMDefaultAbstract Authorization: null
FCMDefaultAbstract Content-Type:application/json
FCMDefaultAbstract send: {"registration_ids":["dMpvzp*************************************2lRsSl_5lFET2"],"data":{"CODE":"201","USER_ID":"1","ADDITION":"1468083549493"},"delay_while_idle":true}
FCM FCMNetworkException: HTTP 401: No error details provided
The Authorization header is not null in fact. it is correctly set with my FCM API Key. Only the HTTPUrlConnection implementation says to return null if someone trys to access Authorization key.
As you can see i am not able to connect with FCM. The Code 401 means that authentication failed.
What could be the problem here?
Check that you are using a server type API-KEY, and not a client or browser API-KEY.
If you are using Firebase you can find the API-KEY in
Project Settings > Cloud Messaging
If you are using cloud console, or you are not sure which key you are using,
you can generate a new key through through https://console.cloud.google.com
Quoting the documentation
https://firebase.google.com/docs/cloud-messaging/concept-options#credentials
Server key: A server key that authorizes your app server for access to
Google services, including sending messages via Firebase Cloud
Messaging. [...]
Important: Do not include the server key anywhere in your client code.
Also, make sure to use only server keys to authorize your app server.
Android, iOS, and browser keys are rejected by FCM.
I am using Google Cloud Messaging to provide push notifications. I may need to send a broadcast notification to around 10.000 users. However, I read that a Multicast message can contain a list with 1000 registration ids, maximun.
So, do I need to send ten multicast messages? Is there any way to send a broadcast to all the clients without generating the lists with all the ids?
thanks in advace.
Since Play Services 7.5, it's now also possible to achieve this through topics:
https://developers.google.com/cloud-messaging/topic-messaging
After registering, you would have to send the GCM server a message over HTTP:
https://gcm-http.googleapis.com/gcm/send
Content-Type:application/json
Authorization:key=AIzaSyZ-1u...0GBYzPu7Udno5aA
{
"to": "/topics/foo-bar",
"data": {
"message": "This is a GCM Topic Message!",
}
}
E.g.:
JSONObject jGcmData = new JSONObject();
JSONObject jData = new JSONObject();
jData.put("message", "This is a GCM Topic Message!");
// Where to send GCM message.
jGcmData.put("to", "/topics/foo-bar");
// What to send in GCM message.
jGcmData.put("data", jData);
// Create connection to send GCM Message request.
URL url = new URL("https://android.googleapis.com/gcm/send");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Authorization", "key=" + API_KEY);
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestMethod("POST");
conn.setDoOutput(true);
// Send GCM message content.
OutputStream outputStream = conn.getOutputStream();
outputStream.write(jGcmData.toString().getBytes());
And your clients should subscribe to /topics/foo-bar :
public void subscribe() {
GcmPubSub pubSub = GcmPubSub.getInstance(this);
pubSub.subscribe(token, "/topics/foo-bar", null);
}
#Override
public void onMessageReceived(String from, Bundle data) {
String message = data.getString("message");
Log.d(TAG, "From: " + from);
Log.d(TAG, "Message: " + message);
// Handle received message here.
}
You have no choice but splitting your broadcast in chunks of maximum 1000 regIds.
You can then send your multicast messages in separate threads.
//regIdList max size is 1000
MulticastResult multicastResult;
try {
multicastResult = sender.send(message, regIdList, retryTimes);
} catch (IOException e) {
logger.error("Error posting messages", e);
return;
}