Google Cloud Functions - Realtime Database Trigger - how to deserialize data JSON to POJO? - kotlin

As described on the Google Cloud Functions docs, it is possible to trigger a Function based on Firebase Realtime Database events (write/create/update/delete).
The following docs sample explains how to get the delta snapshot.
public class FirebaseRtdb implements RawBackgroundFunction {
private static final Logger logger = Logger.getLogger(FirebaseRtdb.class.getName());
// Use GSON (https://github.com/google/gson) to parse JSON content.
private static final Gson gson = new Gson();
#Override
public void accept(String json, Context context) {
logger.info("Function triggered by change to: " + context.resource());
JsonObject body = gson.fromJson(json, JsonObject.class);
boolean isAdmin = false;
if (body != null && body.has("auth")) {
JsonObject authObj = body.getAsJsonObject("auth");
isAdmin = authObj.has("admin") && authObj.get("admin").getAsBoolean();
}
logger.info("Admin?: " + isAdmin);
if (body != null && body.has("delta")) {
logger.info("Delta:");
logger.info(body.get("delta").toString());
}
}
}
The sample works perfectly but the question is: How can I deserialize this delta to a POJO?
I tried:
val mObject = gson.fromJson(body.get("delta").toString(), MyCustomObject::class.java)
But I am getting:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_ARRAY but was BEGIN_OBJECT
As far as I know it is because MyObject class has a List<T> field, and Firebase Database always convert Lists to Maps with integer keys.
I preferably do not want to change every List<T> to Map<Int,T>, because I have a lot of classes :(
Thanks in advance!

So, here is what I ended up doing (maybe not the best solution!):
1) Create a custom Json Deserializer for Firebase-coming lists:
class ListFirebaseDeserializer<T> : JsonDeserializer<ArrayList<T>> {
override fun deserialize(json: JsonElement?, typeOfT: Type?, context: JsonDeserializationContext?): ArrayList<T> {
val result = ArrayList<T>()
val typeOfElement = (typeOfT as ParameterizedType).actualTypeArguments[0]
json?.let {
json.asJsonObject.entrySet().forEach {
entry->
result.add(Gson().fromJson(entry.value, typeOfElement))
}
}
return result
}
}
This takes the lists that Firebase turned into maps and convert it back to actual lists.
2) Annotate every list in my POJO with #JsonAdapter(ListFirebaseDeserializer::class), for instance:
class MyCustomObject {
#JsonAdapter(ListFirebaseDeserializer::class)
var myPaymentList = ArrayList<Payment>()
}
It could be a pain if you already have lots of lists to annotate, but it is better than having to use maps instead.
Hope it helps!

Related

WebFlux & formation DTO

Hello recently started studying Webflux.
And sometimes I encounter the tasks that you need to form a simple DTO and return it
Take for example the usual class dto
#Data
#Builder
public static class Dto {
private long id;
private String code1;
private String code2;
}
And a primitive service with two methods...
#Nullable Mono<String> getCode1(long id);
#Nullable String getCode2(long id);
And wrote a method that forms at the output of Mono
private Mono<Dto> fill(long id) {
var dto = Dto.builder()
.id(id)
.build();
//doOnNext
var dtoMono1 = service.getCode1(id)
.map(code -> {
dto.setCode1(code);
return dto;
})
.doOnNext(innerDto -> innerDto.setCode2(service.getCode2(id)));
//map
var dtoMono2 = service.getCode1(id)
.map(code -> {
dto.setCode1(code);
return dto;
})
.map(unused -> service.getCode2(id))
.map(code -> {
dto.setCode1(code);
return dto;
});
//just
var dtoMono3 = Mono.just(dto)
.flatMap(innerDto -> service.getCode1(innerDto.getId()));
//just
var dtoMono4 = Mono.fromCallable(() -> dto)
.subscribeOn(Schedulers.boundedElastic())
.flatMap(innerDto -> service.getCode1(innerDto.getId()));
}
QUESTION:
Is it possible to simply create DTO and use it in the Webflux call
chain ... Or I need to wrap it in mono.just or mono.fromcallable
(what are the pros and cons)
How best to fill in values via doOnNext
or through MAP. An extra line (Return DTO) appears in the case of
MAP and some people also told me if for example NULL comes, doOnNext will
miss it and go further to fill up current dto. But on the other, the MAP is used
to transform the object, and the doOnNext is more for debugging and
logging
Thanks you...
How about using zip operator in such a case?
I hope this example can help you:
private Mono<Dto> fill(long id) {
return Mono.zip(someService.getCode1(id), Mono.just(someService.getCode2(id)))
.map(tuple ->
Dto.builder()
.id(id)
.code1(tuple.getT1())
.code2(tuple.getT2())
.build()
);
}

spring-data-rest return multiple projections at once (compose projections)

Is there possibility to compose multiple projections? Is it best approach for my use case?
I have these projections for CarInBazar class for example:
SimpleCarInList
WidgetForHotSale
NumberOfItemViews
FullCarData
CarMainImage
CarMainImageIconSize
CarAdditionalImages
And frontend is now making some UI page, which requires some of these projections.
Should I do:
frontend will do multiple requests for same resource with different projections
Implement Projection for each screen of frontend (duplicating things like NumberOfItemViews calculation,...)
Use inheritance and make specific projections utilizing extends keyword. i.e.:
#Projection(name = "screen-dashboardHome", types = {CarInBazar.class})
public interface DashboardHomeProjectionForCarInBazar extends SimpleCarInList,
WidgetForHotSale, CarMainImageIconSize {
}
Is there any possibility to request more projections at once? It could be than rendered in UI using provided profiles perhaps.
EDIT: as requested, providing projection definition example:
import org.springframework.data.rest.core.config.Projection;
import java.awt.*;
import java.util.Date;
#Projection(name = "CarMainImage", types = {CarInBazar.class})
public interface CarMainImage {
Date getLastUpdateDate();
Image getMainImage();
default String getMainImageAdditionalInformation() {
final var updated = this.getLastUpdateDate().getTime();
final var created = this.getCreatedAtDate().getTime();
if (created >= (updated - 10 minutes)) {
return "some business logic on not published fields";
} else {
return "could happen not only in spel";
}
}
}
Many projections does not contain any business logic and are only filtering fields.
Using multiple projections seems to not be supported (or widely supported or easy). Using suggestion from #Aivaras comment I have used this approach:
Repository code with custom JPQL query:
#Repository
#RepositoryRestResource
#Transactional(readOnly = true)
public interface SomeRepository extends PagingAndSortingRepository<Some, Long> {
Page<Some> findByNameContaining(String namePart, Pageable pageable);
#Query("select new sk.qpp.documents.projections.SomeCustomViewByQuery(s.name, s.startDate, s.endDate, s.goLiveDate, 42) from Some s where s.id = :id")
Optional<SomeCustomViewByQuery> getByIdProjectedForSpecialScreen(Long id);
}
And class SomeCustomViewByQuery is just simple DTO like thing. Using lombok it can look like:
#Value
public class SomeCustomViewByQuery {
private String name;
private Date startDate;
private Date endDate;
private Date goLiveDate;
// TODO make SomeHealth to be enum and specific logic behind it.
String getSomeHealth() {
final var start = this.getStartDate().getTime();
final var end = this.getEndDate().getTime();
final var goLive = this.getGoLiveDate().getTime();
final var now = System.currentTimeMillis();
if (now < start) {
return "not started yet";
} else {
if (now < end) {
return "work in progress";
} else {
if (now < goLive) {
return "passed end, but before goLive";
} else {
return "something after goLive time";
}
}
}
}
private int unicornsCount;
}
This way, I can make hand-crafted query (JQPL) and also create custom DTO instance. It is handy, where I need to do some joins to other tables with aggregation (count, avg, max, min, ...) and other things, which are better done on database side.

How to get ArrayList of POJO's from Amazon Lambda (getting only LinkedTreeMap)

I try to call my AWS Lambda function (serverless backend) with my Android mobile app client. The AWS lambda function returns an ArrayList of POJO objects (as JSON).
The problem is that the android client AWS Lambda(JSON)DataBinder does not deserialize to my ArrayList of POJOs. I get an ArrayList of LinkedTreeMap (see code at onPostExecute() below).
At the android client side I'm using Android AWS SDK: com.amazonaws:aws-android-sdk-core:2.6
Here is some code:
public void readSurveyList(String strUuid, int intLanguageID) {
// Create an instance of CognitoCachingCredentialsProvider
// You have to configure at least an AWS identity pool to get access to your lambda function
CognitoCachingCredentialsProvider credentialsProvider = new CognitoCachingCredentialsProvider(
this.getApplicationContext(),
IDENTITY_POOL_ID,
Regions.EU_CENTRAL_1);
LambdaInvokerFactory factory = LambdaInvokerFactory.builder()
.context(this.getApplicationContext())
.region(Regions.EU_CENTRAL_1)
.credentialsProvider(credentialsProvider)
.build();
// Create the Lambda proxy object with default Json data binder.
myInterface = factory.build(MyInterface.class);
//create a request object (depends on your lambda function)
SurveyListRequest surveyListRequest = new SurveyListRequest(strUuid, intLanguageID);
// Lambda function in async task with definiton of
// request object (-> SurveyListRequest)
// response object (-> ArrayList<SurveyListItem>>)
new AsyncTask<SurveyListRequest, Void, ArrayList<SurveyListItem>>() {
#Override
protected ArrayList<SurveyListItem> doInBackground(SurveyListRequest... params) {
try {
return myInterface.ReadSurveyList(params[0]);
} catch (LambdaFunctionException lfe) {
Log.e("TAG", String.format("echo method failed: error [%s], details [%s].", lfe.getMessage(), lfe.getDetails()));
return null;
}
}
#Override
protected void onPostExecute(ArrayList<SurveyListItem> surveyList) {
// PROBLEM: here i get a ArrayList of LinkedTreeMap
}
}.execute(surveyListRequest);
}
Here is the code of my lambda function Interface:
public interface MyInterface {
#LambdaFunction
ArrayList<SurveyListItem> ReadSurveyList (SurveyListRequest surveyListRequest);
}
I would expect to get a list of my POJO objects. I found a lot of discussions about Gson and ArrayList type and solutions based on TypeToken (e.g. Gson TypeToken with dynamic ArrayList item type). Maybe same problem ...
I found a solution using a custom LambdaDataBinder. I have specified the type of my POJO-class "SurveyListItem" in deserialize function. The Gson uses the TypeToken definition and converts the JSON string correct to the list of POJOs (in my case "SurveyListItem" objects).
Here is the sourcecode of MyLambdaDataBinder:
public class MyLambdaDataBinder implements LambdaDataBinder {
private final Gson gson;
Type mType;
//CUSTOMIZATION: pass typetoken via class constructor
public MyLambdaDataBinder(Type type) {
this.gson = new Gson();
mType = type;
}
#Override
public <T> T deserialize(byte[] content, Class<T> clazz) {
if (content == null) {
return null;
}
Reader reader = new BufferedReader(new InputStreamReader(new ByteArrayInputStream(content)));
//CUSTOMIZATION: Original line of code: return gson.fromJson (reader, clazz);
return gson.fromJson(reader, mType);
}
#Override
public byte[] serialize(Object object) {
return gson.toJson(object).getBytes(StringUtils.UTF8);
}
}
Here is how to use the custom MyLambdaDataBinder. Use your POJO instead of "SurveyListItem":
myInterface = factory.build(LambdaInterface.class, new MyLambdaDataBinder(new TypeToken<ArrayList<SurveyListItem>>() {}.getType()));

Store and retrieve string arrays in HBase

I've read this answer (How to store complex objects into hadoop Hbase?) regarding the storing of string arrays with HBase.
There it is said to use the ArrayWritable Class to serialize the array. With WritableUtils.toByteArray(Writable ... writable) I'll get a byte[] which I can store in HBase.
When I now try to retrieve the rows again, I get a byte[] which I have somehow to transform back again into an ArrayWritable.
But I don't find a way to do this. Maybe you know an answer or am I doing fundamentally wrong serializing my String[]?
You may apply the following method to get back the ArrayWritable (taken from my earlier answer, see here) .
public static <T extends Writable> T asWritable(byte[] bytes, Class<T> clazz)
throws IOException {
T result = null;
DataInputStream dataIn = null;
try {
result = clazz.newInstance();
ByteArrayInputStream in = new ByteArrayInputStream(bytes);
dataIn = new DataInputStream(in);
result.readFields(dataIn);
}
catch (InstantiationException e) {
// should not happen
assert false;
}
catch (IllegalAccessException e) {
// should not happen
assert false;
}
finally {
IOUtils.closeQuietly(dataIn);
}
return result;
}
This method just deserializes the byte array to the correct object type, based on the provided class type token.
E.g:
Let's assume you have a custom ArrayWritable:
public class TextArrayWritable extends ArrayWritable {
public TextArrayWritable() {
super(Text.class);
}
}
Now you issue a single HBase get:
...
Get get = new Get(row);
Result result = htable.get(get);
byte[] value = result.getValue(family, qualifier);
TextArrayWritable tawReturned = asWritable(value, TextArrayWritable.class);
Text[] texts = (Text[]) tawReturned.toArray();
for (Text t : texts) {
System.out.print(t + " ");
}
...
Note:
You may have already found the readCompressedStringArray() and writeCompressedStringArray() methods in WritableUtils
which seem to be suitable if you have your own String array-backed Writable class.
Before using them, I'd warn you that these can cause serious performance hit due to
the overhead caused by the gzip compression/decompression.

Jackson vector serialization exception

I have the following code with a simple class and a method for writing and then reading:
ObjectMapper mapper = new ObjectMapper();
try{
DataStore testOut = new DataStore();
DataStore.Checklist ch1 = testOut.addChecklist();
ch1.SetTitle("Checklist1");
String output = mapper.writeValueAsString(testOut);
JsonNode rootNode = mapper.readValue(output, JsonNode.class);
Map<String,Object> userData = mapper.readValue(output, Map.class);
}
public class DataStore {
public static class Checklist
{
public Checklist()
{
}
private String _title;
public String GetTitle()
{
return _title;
}
public void SetTitle(String title)
{
_title = title;
}
}
//Checklists
private Vector<Checklist> _checklists = new Vector<Checklist>();
public Checklist addChecklist()
{
Checklist ch = new Checklist();
ch.SetTitle("New Checklist");
_checklists.add(ch);
return ch;
}
public Vector<Checklist> getChecklists()
{
return _checklists;
}
public void setChecklists(Vector<Checklist> checklists)
{
_checklists = checklists;
}
}
The line:
String output = mapper.writeValueAsString(testOut);
causes an exception that has had me baffled for hours and about to abandon using this at all.
Any hints are appreciated.
Here is the exception:
No serializer found for class DataStore$Checklist and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS) ) (through reference chain: DataStore["checklists"]->java.util.Vector[0])
There are multiple ways to do it, but I will start with what you are doing wrong: your naming of getter and setter method is wrong -- in Java one uses "camel-case", so you should be using "getTitle". Because of this, properties are not found.
Besides renaming methods to use Java-style names, there are alternatives:
You can use annotation JsonProperty("title") for GetTitle(), so that property is recognized
If you don't want the wrapper object, you could alternatively just add #JsonValue for GetTitle(), in which case value used for the whole object would be return value of that method.
The answer seems to be: You can't do that with Json. I've seen comments in the Gson tutorial as well, that state that some serialization just doesn't work. I downloaded XStream and spat it out with XML in a few minutes of work and a lot less construction around what I really wanted to persist. In the process, I was able to delete a lot of code.