I have the following class:
public class ChronicleMapIndex<K, V> implements Index<K, V> {
private ChronicleMap<K, V> index;
private Map<String, String> characteristicsMap;
public void buildIndex(String name, Map<String, String> characteristicsMap, Path indexPath, Class<?> keyType, Class<?> valueType){
this.characteristicsMap = characteristicsMap;
String filename = name + ".bin";
Path indexFilePath = Paths.get(indexPath + filename);
try {
index = (ChronicleMap<K, V>) ChronicleMap
.of(keyType, valueType)
.name(name)
.entries(Long.parseLong(characteristicsMap.get("entries")))
.averageValueSize(Double.parseDouble(characteristicsMap.get("averageValueSize")))
.averageKeySize(Double.parseDouble(characteristicsMap.get("averageKeySize")))
.createOrRecoverPersistedTo(indexFilePath.toFile(), true);
} catch (IOException e) {
e.printStackTrace();
}
}
#Override
public V get(K key) {
return index.get(key);
}
#Override
public void put(K key, V value) {
index.put(key, value);
}
}
I basically want to store a chronicle maps map in a wrapper function. Here, K and V are the same as the keyType and valueType values passed. So I would like the ChronicleMap map to have the same key and value type as K and V.
However, when I create it, I get the following error:
net.openhft.chronicle.hash.ChronicleHashRecoveryFailedException: java.lang.AssertionError: java.lang.NoSuchMethodException: sun.nio.ch.FileChannelImpl.map0(int,long,long)
at net.openhft.chronicle.map.ChronicleMapBuilder.openWithExistingFile(ChronicleMapBuilder.java:1877)
at net.openhft.chronicle.map.ChronicleMapBuilder.createWithFile(ChronicleMapBuilder.java:1701)
at net.openhft.chronicle.map.ChronicleMapBuilder.recoverPersistedTo(ChronicleMapBuilder.java:1655)
at net.openhft.chronicle.map.ChronicleMapBuilder.createOrRecoverPersistedTo(ChronicleMapBuilder.java:1638)
at net.openhft.chronicle.map.ChronicleMapBuilder.createOrRecoverPersistedTo(ChronicleMapBuilder.java:1629)
at edu.upf.taln.indexer.index.chroniclemap.ChronicleMapIndex.buildIndex(ChronicleMapIndex.java:27)
in this line:
.createOrRecoverPersistedTo(indexFilePath.toFile(), true);
I was wondering if this error is because I'm doing something wrong with generics.
I am using ChronicleMap 3.19.4 and JNA 5.5.0 in Windows 10.
Here is an individual test you can run easily:
Map<String, String> characteristicsMap = new HashMap<>();
characteristicsMap.put("entries", Long.toString(123));
characteristicsMap.put("averageKeySize", Integer.toString(5));
characteristicsMap.put("averageValueSize", Integer.toString(5));
String name = "test";
Path indexPath = Paths.get("D:/trabajo"); // substitute this as needed
ChronicleMapIndex<String, String> index = new ChronicleMapIndex<>();
index.buildIndex(name, characteristicsMap, indexPath ,String.class, String.class);
This has nothing to do with generics. You're using Java 12+, and it's not supported. The internal sun.nio.ch.FileChannelImpl has changed somewhere between version 11 to 14, and is not backwards-compatible.
Our support policy is to only support LTS versions of Java. The highest we support right now is Java 11. Next LTS version (TBC) is 17, and that what we plan to support next.
Related
I'm trying to serialize an object (Root), with some duplicated entries of MyObject. Just want store the whole objects one, I'm using #JsonIdentityReference, which works pretty well.
However, I realize that it will generate un-deserializable object, if there're equal objects with different reference. I wonder if there's a configuration in Jackson to change this behavior, thanks!
#Value
#AllArgsConstructor
#NoArgsConstructor(force = true)
class Root {
private List<MyObject> allObjects;
private Map<String, MyObject> objectMap;
}
#Value
#AllArgsConstructor
#NoArgsConstructor(force = true)
#JsonIdentityReference
#JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "id")
class MyObject {
private String id;
private int value;
}
public class Main {
public static void main() throws JsonProcessingException {
// Constructing equal objects
val obj1 = new MyObject("a", 1);
val obj2 = new MyObject("a", 1);
assert obj1.equals(obj2);
val root = new Root(
Lists.newArrayList(obj1),
ImmutableMap.of(
"lorem", obj2
)
);
val objectMapper = new ObjectMapper();
val json = objectMapper.writeValueAsString(root);
// {"allObjects":[{"id":"a","value":1}],"objectMap":{"lorem":{"id":"a","value":1}}}
// Note here both obj1 and obj2 are expanded.
// Exception: Already had POJO for id
val deserialized = objectMapper.readValue(json, Root.class);
assert root.equals(deserialized);
}
}
I'm using Jackson 2.10.
Full stacktrace:
Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Already had POJO for id (java.lang.String) [[ObjectId: key=a, type=com.fasterxml.jackson.databind.deser.impl.PropertyBasedObjectIdGenerator, scope=java.lang.Object]] (through reference chain: Root["objectMap"]->java.util.LinkedHashMap["lorem"]->MyObject["id"])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:394)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:353)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1714)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:371)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeWithObjectId(BeanDeserializerBase.java:1257)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:157)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4202)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3205)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3173)
at Main.main(Main.java:53)
Caused by: java.lang.IllegalStateException: Already had POJO for id (java.lang.String) [[ObjectId: key=a, type=com.fasterxml.jackson.databind.deser.impl.PropertyBasedObjectIdGenerator, scope=java.lang.Object]]
at com.fasterxml.jackson.annotation.SimpleObjectIdResolver.bindItem(SimpleObjectIdResolver.java:24)
at com.fasterxml.jackson.databind.deser.impl.ReadableObjectId.bindItem(ReadableObjectId.java:57)
at com.fasterxml.jackson.databind.deser.impl.ObjectIdValueProperty.deserializeSetAndReturn(ObjectIdValueProperty.java:101)
at com.fasterxml.jackson.databind.deser.impl.ObjectIdValueProperty.deserializeAndSet(ObjectIdValueProperty.java:83)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:369)
... 14 more
As I mentioned earlier, this setup only works if obj1 == obj2, as the two objects with same ID should be identity-equal. In that case, the second object would also net get expanded during serialization (alwaysAsId = false only expands the first object).
However, if you want to have this setup and are fine with the serialization, you could use a custom Resolver for deserialization that stores a single instance per key:
#JsonIdentityReference(alwaysAsId = false)
#JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "id", resolver = CustomScopeResolver.class)
static class MyObject {
private String id;
// ...
}
class CustomScopeResolver implements ObjectIdResolver {
Map<String, MyObject> data = new HashMap<>();
#Override
public void bindItem(final IdKey id, final Object pojo) {
data.put(id.key.toString(), (MyObject) pojo);
}
#Override
public Object resolveId(final IdKey id) {
return data.get(id.key);
}
#Override
public ObjectIdResolver newForDeserialization(final Object context) {
return new CustomScopeResolver();
}
#Override
public boolean canUseFor(final ObjectIdResolver resolverType) {
return false;
}
}
NEW EDIT: Apparently, its very easy: Just turn on objectMapper.configure(SerializationFeature.USE_EQUALITY_FOR_OBJECT_ID, true); so that the DefaultSerializerProvider uses a regular Hashmap instead of an IdentityHashMap to manage the serialized beans.
DEPRECATED: Update for Serialization: It is possible to achieve this by adding a custom SerializationProvider:
class CustomEqualObjectsSerializerProvider extends DefaultSerializerProvider {
private final Collection<MyObject> data = new HashSet<>();
private final SerializerProvider src;
private final SerializationConfig config;
private final SerializerFactory f;
public CustomEqualObjectsSerializerProvider(
final SerializerProvider src,
final SerializationConfig config,
final SerializerFactory f) {
super(src, config, f);
this.src = src;
this.config = config;
this.f = f;
}
#Override
public DefaultSerializerProvider createInstance(final SerializationConfig config, final SerializerFactory jsf) {
return new CustomEqualObjectsSerializerProvider(src, this.config, f);
}
#Override
public WritableObjectId findObjectId(final Object forPojo, final ObjectIdGenerator<?> generatorType) {
// check if there is an equivalent pojo, use it if exists
final Optional<MyObject> equivalentObject = data.stream()
.filter(forPojo::equals)
.findFirst();
if (equivalentObject.isPresent()) {
return super.findObjectId(equivalentObject.get(), generatorType);
} else {
if (forPojo instanceof MyObject) {
data.add((MyObject) forPojo);
}
return super.findObjectId(forPojo, generatorType);
}
}
}
#Test
public void main() throws IOException {
// Constructing equal objects
final MyObject obj1 = new MyObject();
obj1.setId("a");
final MyObject obj2 = new MyObject();
obj2.setId("a");
assert obj1.equals(obj2);
final Root root = new Root();
root.setAllObjects(Collections.singletonList(obj1));
root.setObjectMap(Collections.singletonMap(
"lorem", obj2));
final ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setSerializerProvider(
new CustomEqualObjectsSerializerProvider(
objectMapper.getSerializerProvider(),
objectMapper.getSerializationConfig(),
objectMapper.getSerializerFactory()));
final String json = objectMapper.writeValueAsString(root);
System.out.println(json); // second object is not expanded!
}
I try to call my AWS Lambda function (serverless backend) with my Android mobile app client. The AWS lambda function returns an ArrayList of POJO objects (as JSON).
The problem is that the android client AWS Lambda(JSON)DataBinder does not deserialize to my ArrayList of POJOs. I get an ArrayList of LinkedTreeMap (see code at onPostExecute() below).
At the android client side I'm using Android AWS SDK: com.amazonaws:aws-android-sdk-core:2.6
Here is some code:
public void readSurveyList(String strUuid, int intLanguageID) {
// Create an instance of CognitoCachingCredentialsProvider
// You have to configure at least an AWS identity pool to get access to your lambda function
CognitoCachingCredentialsProvider credentialsProvider = new CognitoCachingCredentialsProvider(
this.getApplicationContext(),
IDENTITY_POOL_ID,
Regions.EU_CENTRAL_1);
LambdaInvokerFactory factory = LambdaInvokerFactory.builder()
.context(this.getApplicationContext())
.region(Regions.EU_CENTRAL_1)
.credentialsProvider(credentialsProvider)
.build();
// Create the Lambda proxy object with default Json data binder.
myInterface = factory.build(MyInterface.class);
//create a request object (depends on your lambda function)
SurveyListRequest surveyListRequest = new SurveyListRequest(strUuid, intLanguageID);
// Lambda function in async task with definiton of
// request object (-> SurveyListRequest)
// response object (-> ArrayList<SurveyListItem>>)
new AsyncTask<SurveyListRequest, Void, ArrayList<SurveyListItem>>() {
#Override
protected ArrayList<SurveyListItem> doInBackground(SurveyListRequest... params) {
try {
return myInterface.ReadSurveyList(params[0]);
} catch (LambdaFunctionException lfe) {
Log.e("TAG", String.format("echo method failed: error [%s], details [%s].", lfe.getMessage(), lfe.getDetails()));
return null;
}
}
#Override
protected void onPostExecute(ArrayList<SurveyListItem> surveyList) {
// PROBLEM: here i get a ArrayList of LinkedTreeMap
}
}.execute(surveyListRequest);
}
Here is the code of my lambda function Interface:
public interface MyInterface {
#LambdaFunction
ArrayList<SurveyListItem> ReadSurveyList (SurveyListRequest surveyListRequest);
}
I would expect to get a list of my POJO objects. I found a lot of discussions about Gson and ArrayList type and solutions based on TypeToken (e.g. Gson TypeToken with dynamic ArrayList item type). Maybe same problem ...
I found a solution using a custom LambdaDataBinder. I have specified the type of my POJO-class "SurveyListItem" in deserialize function. The Gson uses the TypeToken definition and converts the JSON string correct to the list of POJOs (in my case "SurveyListItem" objects).
Here is the sourcecode of MyLambdaDataBinder:
public class MyLambdaDataBinder implements LambdaDataBinder {
private final Gson gson;
Type mType;
//CUSTOMIZATION: pass typetoken via class constructor
public MyLambdaDataBinder(Type type) {
this.gson = new Gson();
mType = type;
}
#Override
public <T> T deserialize(byte[] content, Class<T> clazz) {
if (content == null) {
return null;
}
Reader reader = new BufferedReader(new InputStreamReader(new ByteArrayInputStream(content)));
//CUSTOMIZATION: Original line of code: return gson.fromJson (reader, clazz);
return gson.fromJson(reader, mType);
}
#Override
public byte[] serialize(Object object) {
return gson.toJson(object).getBytes(StringUtils.UTF8);
}
}
Here is how to use the custom MyLambdaDataBinder. Use your POJO instead of "SurveyListItem":
myInterface = factory.build(LambdaInterface.class, new MyLambdaDataBinder(new TypeToken<ArrayList<SurveyListItem>>() {}.getType()));
A library is using Map to use some extra information. This map eventually is being converted a JSON object and I need to set request information to display for debugging purposes as this:
map.put("request", requestString);
I am considering to use Jackson specifically to create a JSON without quotes and want to set as requestString.
I am building necessary information regarding Request and building a Map including request headers, parameters, method etc.
Jackson is creating perfectly valid JSON with quotes but when I set this generated value inside map, It is displayed ugly because of having escaped quotes.
So Jackson is creating this:
{
method : "POST",
path : "/register"
}
When I set this in map, it turns to this:
{
method : \"POST\",
path : \"/register\"
}
Consider this as a huge map including all parameters and other information about request.
What I would like to want this:
{
method : POST,
path : /register
}
I know that this is not a valid JSON but I am using this as a String to a Map which is accepting String values.
public class UnQuotesSerializer extends NonTypedScalarSerializerBase<String>
{
public UnQuotesSerializer() { super(String.class); }
/**
* For Strings, both null and Empty String qualify for emptiness.
*/
#Override
public boolean isEmpty(String value) {
return (value == null) || (value.length() == 0);
}
#Override
public void serialize(String value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
jgen.writeRawValue(value);
}
#Override
public JsonNode getSchema(SerializerProvider provider, Type typeHint) {
return createSchemaNode("string", true);
}
#Override
public void acceptJsonFormatVisitor(JsonFormatVisitorWrapper visitor, JavaType typeHint) throws JsonMappingException {
if (visitor != null) visitor.expectStringFormat(typeHint);
}
}
and
ObjectMapper objectMapper = new ObjectMapper();
SimpleModule module = new SimpleModule("UnQuote");
module.addSerializer(new UnQuotesSerializer());
objectMapper.configure(JsonGenerator.Feature.QUOTE_FIELD_NAMES, false);
objectMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_FIELD_NAMES, true);
objectMapper.configure(JsonParser.Feature.ALLOW_UNQUOTED_CONTROL_CHARS, true);
objectMapper.registerModule(module);
This is generating without quotes strings.
The following test passes (Jackson 2.5.0)
#Test
public void test() throws Exception {
ObjectMapper mapper = new ObjectMapper();
Map map = new HashMap();
map.put("method", "POST");
map.put("request", "/register");
String s = mapper.writeValueAsString(map);
Map map2 = mapper.readValue(s, Map.class);
Assert.assertEquals(map, map2);
}
so your pseudo JSON without quotes does not seem the way to go
I am using Spring JDBC and some nice Java 8 lambda-syntax to execute queries with the JDBCTemplate.
The reason for choosing Springs JDBCTemplate, is the implicit resource-handling that Spring-jdbc offers (I do NOT want a ORM framework for my simple usecase's).
My problem is that I want to debug the whole SQL statements with their parameters. Spring prints the SQL by default but not the parameters. Therefor I have subclassed the JDBCTemplate and overridden a query-method.
An example usage of the JDBCTemplate:
public List<Product> getProductsByModel(String modelName) {
List<Product> productList = jdbcTemplate.query(
"select * from product p, productmodel m " +
"where p.modelId = m.id " +
"and m.name = ?",
(rs, rowNum) -> new Product(
rs.getInt("id"),
rs.getString("stc_number"),
rs.getString("version"),
getModelById(rs.getInt("modelId")), // method not shown
rs.getString("displayName"),
rs.getString("imageUrl")
),
modelName);
return productList;
}
To get hold of the parameters I have, as mentioned, overridden the JDBCTemplate class. By doing a cast and using reflection I get the Object[] field with the parameters from an instance of ArgumentPreparedStatementSetter.
I suspect this implementation could potentially be dangerous, as the actual implementation of the PreparedStatementSetter may not always be ArgumentPreparedStatementSetter (Yes I should do an instanceOf check). Also, the reflection code may not be as elegant, but that is besides the point now though :).
Here's my custom implementation:
public class CustomJdbcTemplate extends JdbcTemplate {
private static final Logger log = LoggerFactory.getLogger(CustomJdbcTemplate.class);
public CustomJdbcTemplate(DataSource dataSource) {
super(dataSource);
}
public <T> T query(PreparedStatementCreator psc, final PreparedStatementSetter pss, final ResultSetExtractor<T> rse)
throws DataAccessException {
if(log.isDebugEnabled()) {
ArgumentPreparedStatementSetter aps = (ArgumentPreparedStatementSetter) pss;
try {
Field args = aps.getClass().getDeclaredField("args");
args.setAccessible(true);
Object[] parameters = (Object[]) args.get(aps);
log.debug("Parameters for SQL query: " + Arrays.toString(parameters));
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new GenericException(e.toString(), e);
}
}
return super.query(psc, pss, rse);
}
}
So, when I execute the log.debug(...) statement I would also like to have the original SQL query logged (same line). Has anyone done something similar or are there any better suggestions as to how this can be achieved?
I do quite a few queries using this CustomJDBCTemplate and all my tests run, so I think it may be an acceptable solution of for most debug purposes.
Kind regards,
Thomas
I found a way to get the SQL-statement, so I will answer my own question :)
The PreparedStatementCreator has the following implementation:
private static class SimplePreparedStatementCreator implements PreparedStatementCreator, SqlProvider
So the SqlProvider has a getSql() method which does exactly what I need.
Posting the "improved" CustomJdbcTemplate class if anyone ever should need to do the same :)
public class CustomJdbcTemplate extends JdbcTemplate {
private static final Logger log = LoggerFactory.getLogger(CustomJdbcTemplate.class);
public CustomJdbcTemplate(DataSource dataSource) {
super(dataSource);
}
public <T> T query(PreparedStatementCreator psc, final PreparedStatementSetter pss, final ResultSetExtractor<T> rse)
throws DataAccessException {
if(log.isDebugEnabled()) {
if(pss instanceof ArgumentPreparedStatementSetter) {
ArgumentPreparedStatementSetter aps = (ArgumentPreparedStatementSetter) pss;
try {
Field args = aps.getClass().getDeclaredField("args");
args.setAccessible(true);
Object[] parameters = (Object[]) args.get(aps);
log.debug("SQL query: [{}]\tParams: {} ", getSql(psc), Arrays.toString(parameters));
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new GenericException(e.toString(), e);
}
}
}
return super.query(psc, pss, rse);
}
private static String getSql(Object sqlProvider) { // this code is also found in the JDBCTemplate class
if (sqlProvider instanceof SqlProvider) {
return ((SqlProvider) sqlProvider).getSql();
}
else {
return null;
}
}
}
I need to convert a pig formatted map string representation (e.g. "['k1'#'v1','k2'#'v2']") to Map object.
I'd prefer using pig implementation rather than implementing it myself.
I found a method for an opposite conversion: DataType.mapToString(Map m)
If you really stick to Pig, you may come up with the followings:
...
import org.apache.pig.builtin.Utf8StorageConverter;
...
public class Test {
public static void main(String[] args) throws IOException {
String mapStr = "[k1#v1,k2#v2]";
Utf8StorageConverter converter = new Utf8StorageConverter();
Map<String, Object> map = converter.bytesToMap(mapStr.getBytes());
for (Map.Entry<String, Object> entry : map.entrySet()) {
System.out.println(entry.getKey() + " :: " + entry.getValue());
}
}
}
I assume you have Pig v0.10.0