I am using below code to map the json objects with AWS Dynamo DB attributes but I am not sure whether this will be having any performance issue or not? Please let me know if we have any other better way to achieve it.
I have list of maps and want store them in Dynamo. I am using AWS SDK V2 and Kotlin for it.
//items in model class
var items: List<Map<String, Any>>? = null
val itemValues = mutableMapOf<String, AttributeValue>()
itemValues.put("items", fromL(
order.items?.map { attributes ->
fromM(
attributes.mapValues { attribute ->
fromS(attribute.value.toString())
}
)
}
))
Above mentioned code is working but I want to know if there is any better way to do this.
Related
I'm writing some tests using rest-assured and its Kotlin extensions to test some simple Spring MVC endpoints. I'm trying to understand how to extract values.
One endpoint returns a BookDetailsView POJO, the other returns a Page<BookDetailsView> (where Page is an interface provided by Spring for doing paging).
BookDetailsView is a really simple Kotlin data class with a single field:
data class BookDetailsView(val id: UUID)
For the single object endpoint, I have:
#Test
fun `single object`() {
val details = BookDetailsView(UUID.randomUUID())
whenever(bookDetailsService.getBookDetails(details.id)).thenReturn(details)
val result: BookDetailsView = Given {
mockMvc(mockMvc)
} When {
get("/book_details/${details.id}")
} Then {
statusCode(HttpStatus.SC_OK)
} Extract {
`as`(BookDetailsView::class.java)
}
assertEquals(details.id, result.id)
}
This works as expected, but trying to apply the same technique for the Page<BookDetailsView> runs afoul of all sorts of parsing challenges since Page is an interface, and even trying to use PageImpl isn't entirely straightforward. In the end, I don't even really care about the Page object, I just care about the nested list of POJOs inside it.
I've tried various permutations like the code below to just grab the bit I care about:
#Test
fun `extract nested`() {
val page = PageImpl(listOf(
BookDetailsView(UUID.randomUUID())
))
whenever(bookDetailsService.getBookDetailsPaged(any())).thenReturn(page)
val response = Given {
mockMvc(mockMvc)
} When {
get("/book_details")
} Then {
statusCode(HttpStatus.SC_OK)
body("content.size()", `is`(1))
body("content[0].id", equalTo(page.first().id.toString()))
} Extract {
path<List<BookDetailsView>>("content")
}
println(response[0].javaClass)
}
The final println spits out class java.util.LinkedHashMap. If instead I try to actually use the object, I get class java.util.LinkedHashMap cannot be cast to class BookDetailsView. There are lots of questions and answers related to this, and I understand it's ultimately an issue of the underlying JSON parser not knowing what to do, but I'm not clear on:
Why does the "simple" case parse without issue?
Shouldn't the type param passed to the path() function tell it what type to use?
What needs configuring to make the second case work, OR
Is there some other approach for grabbing a nested object that would make more sense?
Digging a bit into the code, it appears that the two cases may actually be using different json parsers/configurations (the former seems to stick to rest-assured JSON parsing, while the latter ends up in JsonPath's?)
I don't know kotlin but here is the thing:
path() doesn't know the Element in your List, so it'll be LinkedHashMap by default instead of BookDetailsView.class
to overcome it, you can provide TypeReference for this.
java example
List<BookDetailsView> response = ....then()
.extract().jsonPath()
.getObject("content", new TypeRef<List<BookDetailsView>>() {});
kotlin example
#Test
fun `extract nested`() {
var response = RestAssured.given().get("http://localhost:8000/req1")
.then()
.extract()
.jsonPath()
.getObject("content", object : TypeRef<List<BookDetailsView?>?>() {});
println(response)
//[{id=1}, {id=2}]
}
I'd like to be able to use Micronaut's declarative client to hit an a different endpoint based on whether I'm in a local development environment vs a production environment.
I'm setting my client's base uri in application.dev.yml:
myserviceclient:
baseUri: http://localhost:1080/endpoint
Reading the docs from Micronaut, they have the developer jumping through quite a few hoops to get a dynamic value piped into the actual client. They're actually quite confusing. So I've created a configuration like this:
#ConfigurationProperties(PREFIX)
class MyServiceClientConfig {
companion object {
const val PREFIX = "myserviceclient"
const val BASE_URL = "http://localhost:1080/endpoint"
}
var baseUri: String? = null
fun toMap(): MutableMap<String, Any> {
val m = HashMap<String, Any>()
if (baseUri != null) {
m["baseUri"] = baseUri!!
}
return m
}
}
But as you can see, that's not actually reading any values from application.yml, it's simply setting a const value as a static on the class. I'd like that BASE_URL value to be dynamic based on which environment I'm in.
To use this class, I've created a declarative client like this:
#Client(MyServiceClientConfig.BASE_URL)
interface MyServiceClient {
#Post("/user/kfc")
#Produces("application/json")
fun sendUserKfc(transactionDto: TransactionDto)
}
The docs show an example where they're interpolating values from the config map that's built like this:
#Get("/api/\${bintray.apiversion}/repos/\${bintray.organization}/\${bintray.repository}/packages")
But how would I make this work in the #Client() annotation?
Nowhere in that example do they show how bintray is getting defined/injected/etc. This appears to be the same syntax that's used with the #Value() annotation. I've tried using that as well, but every value I try to use ends up being null.
This is very frustrating, but I'm sure I'm missing a key piece that will make this all work.
I'm setting my client's base uri in application.dev.yml
You probably want application-dev.yml.
But how would I make this work in the #Client() annotation?
You can put a config key in the #Client value using something like #Client("${myserviceclient.baseUri}").
If you want the url somewhere in your code use this:
#Value("${micronaut.http.services.occupancy.urls}")
private String occupancyUrl;
How can i get my json from web api to format only value or null for Option types and Discriminated Unions preferably using Newtonsoft.
I am currently using Newtonsoft and only have to add this to web api for it to work:
config.Formatters.JsonFormatter.SerializerSettings <- new JsonSerializerSettings()
When i consume the data on my side, i can easily convert it back to an F# item using: JsonConvert.DeserializeObject<'a>(json)
The api will be consumed by NON .NET clients as well so i would like a more standard formatted json result.
I would like to to fix my issue, w/o having to add code or decorators to all of my records/DU in order for it to work. I have lots of records with lots of properties, some are Option.
ex (this is how DU is serializing):
// When value
"animal": {
"case": "Dog"
}
// When no value
"animal": null
This is what I need:
// When value
"animal": "Dog"
// When no value
"animal": null
This is how an Option type is serializing:
"DocumentInfo": {
"case": "Some",
"fields": [
{
"docId": "77fb9dd0-bfbe-42e0-9d29-d5b1f5f0a9f7",
"docType": "Monkey Business",
"docName": "mb.doc",
"docContent": "why cant it just give me the values?"
}
]
}
This is what I need:
"DocumentInfo": {
"docId": "77fb9dd0-bfbe-42e0-9d29-d5b1f5f0a9f7",
"docType": "Monkey Business",
"docName": "mb.doc",
"docContent": "why cant it just give me the values?"
}
Thank you :-)
You could try using Chiron. I haven't used it myself so I can't give you an extensive example, but https://neoeinstein.github.io/blog/2015/12-13-chiron-json-ducks-monads/index.html has some bits of sample code. (And see https://neoeinstein.github.io/blog/2016/04-02-chiron-computation-expressions/index.html as well for some nicer syntax). Basically, Chiron knows how to serialize and deserialize the basic F# types (strings, numbers, options, etc.) already, and you can teach it to serialize any other type by providing that type with two static methods, ToJson and FromJson:
static member ToJson (x:DocumentInfo) = json {
do! Json.write "docId" x.docId
do! Json.write "docType" x.docType
do! Json.write "docName" x.docName
do! Json.write "docContent" x.docContent
}
static member FromJson (_:DocumentInfo) = json {
let! i = Json.read "docId"
let! t = Json.read "docType"
let! n = Json.read "docName"
let! c = Json.read "docContent"
return { docId = i; docType = t; docName = n; docContent = c }
}
By providing those two static methods on your DocumentInfo type, Chiron will automatically know how to serialize a DocumentInfo option. At least, that's my understanding -- but the Chiron documentation is sadly lacking (by which I mean literally lacking: it hasn't been written yet), so I haven't really used it myself. So this may or may not be the answer you need, but hopefully it'll be of some help to you even if you don't end up using it.
I have found the solution that allows me to use Newtonsoft (JSON.NET), apply custom converters for my types where needed and not require any changes to my DU's or Records.
The short answer is, create a custom converter for Json.Net and use the Read/Write Json overrides:
type CustomDuConverter() =
inherit JsonConverter() (...)
Unfortunately the ones I have found online that were already created doesn't work as is for my needs listed above, but will with slight modification. A great example is to look at: https://gist.github.com/isaacabraham/ba679f285bfd15d2f53e
To apply your custom serializer in Web Api for every call, use:
config.Formatters.JsonFormatter.SerializerSettings.Converters.Add(new CustomDuConverter())
To deserialize use (example that will deserialize to DU):
JsonConvert.DeserializeObject<Animal>("Dog", customConverter)
ex:
type Animal = Dog | Cat
json:
"animal": "Dog"
This will allow you to create a clean Api for consumers and allow you to consume 3rd party Json data into your types that use Option, etc.
I am using Google diff-match-patch JAVA plugin to create patch between two JSON strings and storing the patch to database.
diff_match_patch dmp = new diff_match_patch();
LinkedList<Patch> diffs = dmp.patch_make(latestString, originalString);
String patch = dmp.patch_toText(diffs); // Store patch to DB
Now is there any way to use this patch to re-create the originalString by passing the latestString?
I google about this and found this very old comment # Google diff-match-patch Wiki saying,
Unpatching can be done by just looping through the diff, swapping
DIFF_INSERT with DIFF_DELETE, then applying the patch.
But i did not find any useful code that demonstrates this. How could i achieve this with my existing code ? Any pointers or code reference would be appreciated.
Edit:
The problem i am facing is, in the front-end i am showing a revisions module that shows all the transactions of a particular fragment (take for example an employee details), like which user has updated what details etc. Now i am recreating the fragment JSON by reverse applying each patch to get the current transaction data and show it as a table (using http://marianoguerra.github.io/json.human.js/). But some JSON data are not valid JSON and I am getting JSON.parse error.
I was looking to do something similar (in C#) and what is working for me with a relatively simple object is the patch_apply method. This use case seems somewhat missing from the documentation, so I'm answering here. Code is C# but the API is cross language:
static void Main(string[] args)
{
var dmp = new diff_match_patch();
string v1 = "My Json Object;
string v2 = "My Mutated Json Object"
var v2ToV1Patch = dmp.patch_make(v2, v1);
var v2ToV1PatchText = dmp.patch_toText(v2ToV1Patch); // Persist text to db
string v3 = "Latest version of JSON object;
var v3ToV2Patch = dmp.patch_make(v3, v2);
var v3ToV2PatchTxt = dmp.patch_toText(v3ToV2Patch); // Persist text to db
// Time to re-hydrate the objects
var altV3ToV2Patch = dmp.patch_fromText(v3ToV2PatchTxt);
var altV2 = dmp.patch_apply(altV3ToV2Patch, v3)[0].ToString(); // .get(0) in Java I think
var altV2ToV1Patch = dmp.patch_fromText(v2ToV1PatchText);
var altV1 = dmp.patch_apply(altV2ToV1Patch, altV2)[0].ToString();
}
I am attempting to retrofit this as an audit log, where previously the entire JSON object was saved. As the audited objects have become more complex the storage requirements have increased dramatically. I haven't yet applied this to the complex large objects, but it is possible to check if the patch was successful by checking the second object in the array returned by the patch_apply method. This is an array of boolean values, all of which should be true if the patch worked correctly. You could write some code to check this, which would help check if the object can be successfully re-hydrated from the JSON rather than just getting a parsing error. My prototype C# method looks like this:
private static bool ValidatePatch(object[] patchResult, out string patchedString)
{
patchedString = patchResult[0] as string;
var successArray = patchResult[1] as bool[];
foreach (var b in successArray)
{
if (!b)
return false;
}
return true;
}
I am a beginner at Ignıte. I am doing a sample app in order to measure query times of it.
So the key in the cache is String, value is Map. One of the field in value Map is "order_item_subtotal" so the query is like:
select * from Map where order_item_subtotal>400
And the sample code is:
Ignite ignite= Ignition.ignite();
IgniteCache<String, Map<String, Object>> dummyCache= ignite.getOrCreateCache(cfg);
Map<String,Map<String, Object>> bufferMap=new HashMap<String,Map<String, Object>>();
int i=0;
for (String jsonStr : jsonStrs) {
if(i%1000==0){
dummyCache.putAll(bufferMap);
bufferMap.clear();
}
Map data=mapper.readValue(jsonStr, Map.class);
bufferMap.put(data.get("order_item_id").toString(), data);
i++;
}
SqlFieldsQuery asd=new SqlFieldsQuery("select * from Map where order_item_subtotal>400");
List<List<?>> result= dummyCache.query(asd).getAll();
But the result is always "[]", means empty. And there is no error or exceptions.
What am I missing here? any ideas?
PS: sample data below
{order_item_id=99, order_item_order_id=37, order_item_product_id=365, order_item_quantity=1, order_item_subtotal=59.9900016784668, order_item_product_price=59.9900016784668, product_id=365, product_category_id=17, product_name=Perfect Fitness Perfect Rip Deck, product_description=, product_price=59.9900016784668, product_image=http://images.acmesports.sports/Perfect+Fitness+Perfect+Rip+Deck}
This is not supported. You should use a simple POJO class instead of a map to make it work.
Note that Ignite will store data in binary format and will not deserialize objects when running queries. So you still don't need to deploy class definitions on server node. Please refer to this page for more details: https://apacheignite.readme.io/docs/binary-marshaller