How to collect a stream into a CopyOnWriteArrayList - intellij-idea

I'm getting "Incompatible types, required: CopyOnWriteArrayList, found: Object" with the following. I'm using IntelliJ 2016.1.1.
CopyOnWriteArrayList<Foo> l = fields.stream()
.distinct()
.collect(toCollection(CopyOnWriteArrayList::new));

The problem is that fields has an inappropriate type, most likely, it has a raw type, which will turn the generic invocations of the Stream chain into unchecked operations returning their erased type, which is Object for the terminal collect call.
Using the right type, this works without problems, i.e.
List<String> fields=Arrays.asList("foo", "bar", "baz", "foo");
CopyOnWriteArrayList<String> l =
fields.stream()
.distinct()
.collect(Collectors.toCollection(CopyOnWriteArrayList::new));
works. But note that building a CopyOnWriteArrayList this way is rather expensive as the name “copy on write” already suggests. Copying the entire contents on each insertion leads to quadratic time complexity.
The solution is to collect into a temporary collection, better suited to incremental building, before converting to the desired target type. That copying step might look like overhead, but it’s linear overhead, compared to the quadratic complexity of collecting directly into the CopyOnWriteArrayList.
CopyOnWriteArrayList<String> l =
fields.stream()
.distinct()
.collect(Collectors.collectingAndThen(
Collectors.toList(), CopyOnWriteArrayList::new));
Note that in this specific case, distinct implicitly builds a Set behind the scenes, so we can improve the performance by building the Set explicitly in place of the temporary List and remove the distinct step:
CopyOnWriteArrayList<String> l =
fields.stream()
.collect(Collectors.collectingAndThen(
Collectors.toCollection(LinkedHashSet::new),
CopyOnWriteArrayList::new));
which leads to the conclusion that for this specific use case, we can have it all simpler and potentially even more efficient:
CopyOnWriteArrayList<String> l = new CopyOnWriteArrayList<>(new LinkedHashSet<>(fields));

It seams like your fields object is not of type Foo , otherwise it should work find below working code.
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.stream.Collectors;
public class Foo {
private String name;
Foo(String name){
this.name=name;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
#Override
public String toString() {
return "Foo [name=" + name + "]";
}
public static void main(String[] args) {
List<Foo> fields = new ArrayList<>();
fields.add(new Foo("aa"));
fields.add(new Foo("bb"));
CopyOnWriteArrayList<Foo> l = fields.stream().distinct().collect(Collectors.toCollection(CopyOnWriteArrayList::new));
System.out.println("l"+l);
}
}
PS:If your fields is non generic then also this will give error

Related

Object fingerprinting: serialization + untouchable legacy code + Getter-only auto-properties = cornered?

I have found myself cornered, so here we go.
Context
I need to produce a fingerprint hash code for object diffing. Comparing the hashes of two sets of objects will need to tell me if there are identical objects with the same hash.
The fingerprint hash must be platform-independent. So I went for MD5 hashing.
I am working with a large Object model code base that is out of my control. All types that I will be passed for this fingerprinting can not be modified by me. I cannot add attribute or constructors or modify anything. That does not exclude that the types will change in the future. So any approach must be programmatic -- I cannot just create a Surrogate class to avoid the problem; at least, not manually.
However, performance is not a concern, so reflection has complete green-light.
In addition, I will need to be able to control the exclusion of properties from the hashing. If I exclude a certain property, two object that have all the properties identical to each other except that one will still need to get the same hash.
Issue: serializing to Byte[] with hands tied on the legacy code
MD5 hashing requires the object to be Serialised in Byte[].
The serialisation requires the class to be marked as [Serializable]. Which I cannot add to the legacy code, and naturally it can not be added at runtime either.
So I went for protobuf-net.
Protobuf rightly fails when encountering types that implement an interface with Getter-only auto-properties:
public interface ISomeInterface
{
double Vpy { get; }
double Vy { get; }
double Vpz { get; }
...
}
Being this Interface implemented by many types, using Surrogates seems also a no-go (impractical, non maintainable).
I would just need to serialize, not to deserialize, so I don't see why the limitation of protobuf-net in this case. I understand protobuf-net would not be able to round-trip if needed, but I don't need to round-trip!
Question
Am I really cornered?
Is there any alternative?
My code
As I said, this works perfectly but only if the objects do not have any property (or nested property) that is a type with a Getter-only auto property.
public static byte[] ToByteArray(this object obj, List<PropertyInfo> exclusionsProps = null)
{
if (exclusionsProps == null)
exclusionsProps = new List<PropertyInfo>();
// Protobuf-net implementation
ProtoBuf.Meta.RuntimeTypeModel model = ProtoBuf.Meta.TypeModel.Create();
AddPropsToModel(model, obj.GetType(), exclusionsProps);
byte[] bytes;
using (var memoryStream = new MemoryStream())
{
model.Serialize(memoryStream, obj);
bytes = memoryStream.GetBuffer();
}
return bytes;
}
public static void AddPropsToModel(ProtoBuf.Meta.RuntimeTypeModel model, Type objType, List<PropertyInfo> exclusionsProps = null)
{
List<PropertyInfo> props = new List<PropertyInfo>();
if (exclusionsProps != null)
props.RemoveAll(pr => exclusionsProps.Exists(t => t.DeclaringType == pr.DeclaringType && t.Name == pr.Name));
props
.Where(prop => prop.PropertyType.IsClass || prop.PropertyType.IsInterface).ToList()
.ForEach(prop =>
{
AddPropsToModel(model, prop.PropertyType, exclusionsProps); //recursive call
}
);
var propsNames = props.Select(p => p.Name).OrderBy(name => name).ToList();
model.Add(objType, true).Add(propsNames.ToArray());
}
Which I will then use as such:
foreach (var obj in objs)
{
byte[] objByte = obj.ToByteArray(exclusionTypes);
using (MD5 md5Hash = MD5.Create())
{
string hash = GetMd5Hash(md5Hash, objByte);
Console.WriteLine(obj.GetType().Name + ": " + hash);
}
}
The simple solution here is to completely sidestep the root cause of your issue.
When you can't modify the existing classes, but you need some modifications for them, the easiest way to do that is to create a new and improved subclass, where the modifications you require are available.
Considering that the legacy codebase apparently will change outside of your control, the only way to deal with these changes is to generate these types at runtime. Luckily C# allows you to emit intermediate language which can solve exactly this problem.
You'd start with the DefineType method available from the ModuleBuilder class. Specifically you want to use the overload taking a String, TypeAttributes and a Type (representing the class you extend)
You pointed out that
If two objects have the same hash, you consider them exact copies of each other
Please realise that a hash has a finite entropy, while the source objects have infinite entropy. Hash collisions are bound to happen. Let's have a look at some examples:
public class Point
{
public int X;
public int Y;
}
public class Coordinate
{
public int X;
public int Y;
}
Let's say we calculate the hash as X ^ Y. Instances of both classes could have the same hash, even though they represent different classes. Even when taking just one of these classes, if we take one instance with X = 1, Y = 2 and the other X = 2, Y = 1, they have the same hash. Sure you could optimize the hash algorithm to mitigate the risk on collissions, but you cannot ensure that such collisions can be avoided at all time.
Instead, I would implement a DeepEquals method. This takes more effort (if writing it yourself). But when implemented correctly, it can ensure two objects to be copies.

Jackson deserialization of lombok enhanced class: Why it does work and why it doesn't work?

Background: I found "dysfunctional" code in spring-admin project: "Cannot construct instance of Registration (no Creators, like default construct, exist)". So I wrote custom deserializer and report the issue. But report was rejected, since it allegedly works. And after retest it seems to work now. Does not make sense. So I would like to know why that code work.
But here is the catch. When I wrote similar test class, it does not work in my project. Even when I literally take the code of "now-working" Registration class, and try it in own project, is simply does not deserialize. And then, with practically identical class, it works. It doesn't make any sense.
https://github.com/codecentric/spring-boot-admin/blob/master/spring-boot-admin-server/src/main/java/de/codecentric/boot/admin/server/domain/values/Registration.java
Following post explains how lombok-jackson combo works, but it does not work here. I'm totally confused, this is unbelievelably ridiculous situation, where (unnecessary) simplification creates superb complexity. But I'd like to understand it, since I can encounted this situation in future again.
Jackson Deserialization Fails because of non-default constructor created by lombok
So to have something easy to work with: here we have nice&working pure jackson:
public class TestTO_pureJackson {
private final String a;
private final String b;
#JsonCreator
private TestTO_pureJackson(#JsonProperty("a") String a, #JsonProperty("b") String b) {
this.a = a;
this.b = b;
}
}
and here we have not working lombok equivalent (even if I remove one field, so that it's "same" to latter example):
#lombok.Data
public class TestTO {
private final String a;
private final String b;
#lombok.Builder(builderClassName = "Builder")
private TestTO(String a, String b) {
this.a = a;
this.b = b;
}
public static TestTO.Builder create(String a) {
return builder().a(a);
}
}
and we are trying to deserialize:
{"a": "a", "b": "b"}
Can anyone understand the magic under the hood, and help me to understand what's wrong here?
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.2</version>
<scope>provided</scope>
</dependency>
And to make it even more ridiculous (do you actually see any significant difference with TestTO???), following code works:
#lombok.Data
public class Pair {
private final String left;
private final String right;
#lombok.Builder(builderClassName = "Builder")
private Pair(String pairId) {
left = pairId.substring(0, 3).toUpperCase(Locale.US);
right = pairId.substring(3).toUpperCase(Locale.US);
}
}
and main method:
public class PairTest {
public static final String DATA = "[\"btcusd\",\"ltcusd\",\"ltcbtc\"]";
public static void main(String[] args) throws IOException {
ObjectMapper objectMapper = new ObjectMapper();
Pair[] pairs = objectMapper.readValue(DATA, Pair[].class);
for (Pair pair : pairs) {
System.out.println(pair);
}
}
}
Can anyone see, why 2 almost same TO classes behave differently?
TestTO does not work because there is no constructor that Jackson can use. It cannot use the two-args constructor, because it does not know which JSON field should be used for which argument (because argument names are removed during compilation).
For lombok-generated constructors, you can work around that by advising Lombok to generate a #ConstructorProperties annotation. Just add
lombok.anyConstructor.addConstructorProperties=true
to your lombok.config. In your case of a manual constructor, you could also simply add the #JsonPropertys.
(Note that Jackson does not automatically use builders; you have to explicitly tell Jackson that with #JsonDeserialize and #JsonPOJOBuilder.)
TestTO_pureJackson works, because #JsonProperty is available at runtime and used by Jackson to determine the mapping.
Pair works, because there is a usable constructor: Jackson does not have to guess which parameter belongs to which field, because there is just one. Note that this only works for String, int, long or boolean one-arg constructors.
Lombok does not generate any additional constructor (here: the two-args constructor) if there is already one (see documentation of #Data), so this is the only constructor on the class.

Serialization in Hadoop - Writable

This is the class that implements Writable ..
public class Test implements Writable {
List<AtomicWritable> atoms = new ArrayList<AtomicWritable>();
public void write(DataOutput out) throws IOException {
IntWritable size = new IntWritable(atoms.size());
size.write(out);
for (AtomicWritable atom : atoms)
atom.write(out);
}
public void readFields(DataInput in) throws IOException {
atoms.clear();
IntWritable size = new IntWritable();
size.readFields(in);
int n = size.get();
while(n-- > 0) {
AtomicWritable atom = new AtomicWritable();
atom.readFields(in);
atoms.add(atom);
}
}
}
I will really appreciate if one can help me understand how to invoke write and readFields method.
Basically I m failing to understand how to construct Test object in this case. Once the object is written to DataOutput obj, how do we restore it in DataInput object. This may sound silly, but am a newbie to Hadoop and have been assigned a project that uses Hadoop. Please help.
Thanks!!!
Basically I m failing to understand how to construct Test object in this case.
Yup, you're missing the point. If you need to construct an instance of Test and populate atoms, then you need to add a constructor to Test:
public Test(ArrayList<AtomicWritable> atoms) {
this.atoms = atoms;
}
or you need to use the default constructor and add a method or a setter that lets you add items to atoms or set the value of atoms. The latter is actually pretty common in the Hadoop framework, to have a default constructor and a set method. cf., e.g., Text.set.
You don't call readFields and write; the Hadoop framework does that for you when it needs to serialize and deserialize inputs and outputs to and from map and reduce.

Hibernate/Spring taking out class mapping. About reflection

Im trying to write an aplication with uses hibernate to write to database, however in some actions i have to use JDBC on data in tables made by HB.
JDBS is requred to give administrator ability to create SQL queries with will return statistic info about data in database like number of processed document of specified type, numbers of success/failed log in attempts or total value of products in orders.
To do that i've done an from that allows to create class that has override toString() with return nice sql query string.
All works but now im trying to make administrator live easier by hiving him an ability to choose of table/column names. And here is an problem, because they are created by hibernate. some by #column annotation other by field name.
How can i check how field mapping?
I know its all about reflections but didnt do much of that in java yet.
example
#Entity
#Table(name= "my_table_name" )
public class TableOFSomething implements Serializable{
//This field isn't mapped into database and info about it is not requred.
//In fact, info about it may cause an error.
private static final long serialVersionUID = 7L;
#Id
#Column(name="id")
private String id;
private String fieldOne;
#Column(name="field_two")
private String fieldTwo;
#Column(name="renamed_just_for_fun")
private int Number;
//code with getters & setters
}
How to write methods that will have definition like
public <T> String tableName(Class<T> Target); //returns name of table in database
public <T> ArrayList<String> tabelFields(Class<T> Target); //returns name of fields in database
Hibernate has API - getClassMetadata that can explore the mapping. The API might change and is now located in another place , but i will use it and not in reflection for this.
look on this post for more details:
Get the table name from the model in Hibernate
if you want reflection , so use this link
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import javax.persistence.Column;
import javax.persistence.Table;
import odi.beans.statistic.QueryBean;
public class ReflectionTest {
public static void main(String[] args) {
ReflectionTest test=new ReflectionTest();
System.out.println("Table name of "+QueryBean.class.getName()+" is "+test.getTableName(QueryBean.class));
System.out.println("Column names in this table are:");
for(String n: test.getColumnNames(QueryBean.class)){
System.out.println("\t"+n);
}
System.out.println("Good bye ;)");
}
public <T> ArrayList<String> getColumnNames(Class<T> target) {
ArrayList<String> ret=new ArrayList<>();
Field[] fields = target.getDeclaredFields();
String fieldName =null;
for (Field f : fields) {
//jump to next if if field is static
if (Modifier.isStatic(f.getModifiers()))
continue;
if (f.isAnnotationPresent(Column.class)) {
Column a = f.getAnnotation(Column.class);
fieldName = a.name();
} else {
fieldName = f.getName();
}
ret.add(fieldName);
}
return ret;
}
public <T> String getTableName(Class<T> target){
String ret=target.getSimpleName();
if (target.isAnnotationPresent(Table.class))
{
Table t=target.getAnnotation(Table.class);
ret=t.name();
}
return ret;
}
}
Is it cover all possibilities?
I know now that Hibernate way would be easier, but this is also about learning of very useful reflection mechanism :)
EDIT:
Important question:
Will this work only on annotations or also on xml mapping?

EclipseLink - #ReadTransformer

I have this code:
#Column(name = "foo")
#ReadTransformer(transformerClass=transformer.class)
private Date foo;
public static class transformer implements AttributeTransformer {
#Override
public void initialize(AbstractTransformationMapping atm) {
}
#Override
public Object buildAttributeValue(Record record, Object o, Session sn) {
}
}
My question is, how do I get the value to transform (from column foo) inside of buildAttributeVaule? It is not inside the record array.
You need one or more #WriteTransformer to write the fields you want selected (and thus get them selected), #Column is not used with a transformation mapping.
However, if you just have a single column, then just use a converter instead, #Convert,
http://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Basic_JPA_Development/Mapping/Basic_Mappings/Default_Conversions_and_Converters
First check that the SQL generated is reading in the "foo" column by turning on logging. If it is, then check that the database is returning "foo" and not "FOO" - java is case sensitive on string looksups. It could be that "FOO" is in the record instead of "foo".