When I was trying to implement Delta Propagation in GemFire, I was getting Exception shown below...
Exception...
com.gemstone.gemfire.pdx.PdxSerializationException: Could not create an instance of a class DeltaTesting with root cause
java.lang.ClassNotFoundException: DeltaTesting
at org.apache.geode.internal.ClassPathLoader.forName(ClassPathLoader.java:437)
at org.apache.geode.internal.InternalDataSerializer.getCachedClass(InternalDataSerializer.java:4010)
at org.apache.geode.pdx.internal.PdxType.getPdxClass(PdxType.java:235)
at org.apache.geode.pdx.internal.PdxReaderImpl.basicGetObject(PdxReaderImpl.java:687)
at org.apache.geode.pdx.internal.PdxReaderImpl.getObject(PdxReaderImpl.java:682)
Code...
#Region("deltaTesting")
public class DeltaTesting implements Delta,Serializable {
private static final long serialVersionUID = 1L;
public DeltaTesting(){
}
public DeltaTesting(String id, String firstName, String lastName){
this.id = id;
this.firstName = firstName;
this.lastName = lastName;
}
#Id
private String id;
private String firstName;
private String lastName;
private transient boolean stringId = false;
private transient boolean stringFirstName = false;
private transient boolean stringLastName = false;
public String getId() {
return id;
}
public void setId(String id) {
this.stringId = true;
this.id = id;
}
public String getFirstName() {
return firstName;
}
public void setFirstName(String firstName) {
this.stringFirstName = true;
this.firstName = firstName;
}
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.stringLastName = true;
this.lastName = lastName;
}
#Override
public void fromDelta(DataInput in) throws IOException, InvalidDeltaException {
// TODO Auto-generated method stub
System.out.println("Applying delta to " + this.toString());
// For each field, read whether there is a change
if (in.readBoolean()) {
// Read the change and apply it to the object
this.id = in.readLine();
System.out.println(" Applied delta to field 'id' = "
+ this.id);
}
if (in.readBoolean()) {
// Read the change and apply it to the object
this.firstName = in.readLine();
System.out.println(" Applied delta to field 'firstname' = "
+ this.firstName);
}
if (in.readBoolean()) {
// Read the change and apply it to the object
this.lastName = in.readLine();
System.out.println(" Applied delta to field 'lastname' = "
+ this.lastName);
}
}
#Override
public boolean hasDelta() {
// TODO Auto-generated method stub
return this.stringId || this.stringFirstName || this.stringLastName;
}
#Override
public void toDelta(DataOutput out) throws IOException {
out.writeBoolean(stringId);
// TODO Auto-generated method stub
if (stringId) {
out.writeBytes(this.id);
this.stringId = false;
System.out.println(" Extracted delta from field 'id' = " + this.id);
}
out.writeBoolean(stringFirstName);
if (stringFirstName) {
out.writeBytes(this.firstName);
this.stringFirstName = false;
System.out.println(" Extracted delta from field 'firstName' = " + this.firstName);
}
out.writeBoolean(stringLastName);
if (stringLastName) {
out.writeBytes(this.lastName);
this.stringLastName = false;
System.out.println(" Extracted delta from field 'lastName' = " + this.lastName);
}
}
}
**cache.xml**
<pdx>
<pdx-serializer>
<class-name>com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer</class-
name>
<parameter name="classes">
<string>com\.xxx\..+</string>
</parameter>
</pdx-serializer>
</pdx>
#Vigneshwara-
Delta Propagation is not supported by PDX Serialization in GemFire. You must use the GemFire Data Serialization framework (i.e. DataSerializable and DataSerializers) with Deltas instead.
In short... if you use PDX, then you cannot use Deltas. If you use Deltas, then you cannot use PDX.
Sorry,
-j
Related
So i want to implement application which reads data from json format files. And I have created customer object for the data in json. And I want to send these object through kafka topic. So far i have successfully send String message to producer to consumer. But when i try to send object, in the consumer side, when I do .value().toString(). I got null value. The following is the code I have used:
This is producer:
public class MyProducer {
public static void main(String[] args) throws Exception {
Properties properties = new Properties();
properties.put("bootstrap.servers", "kafka.kafka-cluster-shared.non-prod-5-az-scus.prod.us.walmart.net:9092");
properties.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
properties.put("value.serializer", "xxxxxxxxx.KafkaJsonSerializer");
properties.put("acks", "1");
properties.put("retries", "2");
properties.put("batch.size", "16384");
properties.put("linger.ms", "1");
properties.put("buffer.memory", "33554432");
KafkaProducer<String, pharmacyData> kafkaProducer = new KafkaProducer<String, pharmacyData>(
properties);
String topic = "insights";
//try {
Gson gson = new Gson();
Reader reader = Files.newBufferedReader(Paths.get("......./part.json"));
List<pharmacyData> pdata = new Gson().fromJson(reader, new TypeToken<List<pharmacyData>>() {}.getType());
//pdata.forEach(System.out::println);
reader.close();
//} catch (Exception e) {
//e.printStackTrace();
//}
for (pharmacyData data : pdata) {
kafkaProducer.send(new ProducerRecord<String, pharmacyData>(topic, data), new Callback() {
#Override
public void onCompletion(RecordMetadata recordMetadata, Exception e) {
if (e == null) {
System.out.println(recordMetadata.partition() + "--" + recordMetadata.serializedValueSize());
} else {
e.printStackTrace();
}
}
});
}
kafkaProducer.close();
}
}
This is the customer object class:
public class pharmacyData {
private String load_date;
private String store_nbr;
private String state;
private String pmp_flag;
private String zero_flag;
private String submit_ts;
public pharmacyData(String load_date, String store_nbr, String state, String pmp_flag, String zero_flag, String submit_ts) {
this.load_date = load_date;
this.store_nbr = store_nbr;
this.state = state;
this.pmp_flag = pmp_flag;
this.zero_flag = zero_flag;
this.submit_ts = submit_ts;
}
public String getLoad_date() {
return load_date;
}
public void setLoad_date(String load_date) {
this.load_date = load_date;
}
public String getStore_nbr() {
return store_nbr;
}
public void setStore_nbr(String store_nbr) {
this.store_nbr = store_nbr;
}
public String getState() {
return state;
}
public void setState(String state) {
this.state = state;
}
public String getPmp_flag() {
return pmp_flag;
}
public void setPmp_flag(String pmp_flag) {
this.pmp_flag = pmp_flag;
}
public String getZero_flag() {
return zero_flag;
}
public void setZero_flag(String zero_flag) {
this.zero_flag = zero_flag;
}
public String getSubmit_ts() {
return submit_ts;
}
public void setSubmit_ts(String submit_ts) {
this.submit_ts = submit_ts;
}
#Override
public String toString() {
return "pharmacyData{" +
"load_date='" + load_date + '\'' +
", store_nbr='" + store_nbr + '\'' +
", state='" + state + '\'' +
", pmp_flag='" + pmp_flag + '\'' +
", zero_flag='" + zero_flag + '\'' +
", submit_ts='" + submit_ts + '\'' +
'}';
}
}
this is the customer Serializer:
public class KafkaJsonSerializer implements Serializer {
private Logger logger = LogManager.getLogger(this.getClass());
#Override
public void configure(Map map, boolean b) {
}
#Override
public byte[] serialize(String s, Object o) {
byte[] retVal = null;
ObjectMapper objectMapper = new ObjectMapper();
try {
retVal = objectMapper.writeValueAsBytes(o);
} catch (Exception e) {
logger.error(e.getMessage());
}
return retVal;
}
#Override
public void close() {
}
}
This is the customer Deserializer:
public class KafkaJsonDeserializer implements Deserializer {
#Override
public void configure(Map map, boolean b) {
}
#Override
public Object deserialize(String s, byte[] bytes) {
ObjectMapper mapper = new ObjectMapper();
pharmacyData pdata = null;
try {
pdata = mapper.readValue(bytes, pharmacyData.class);
} catch (Exception e) {
e.printStackTrace();
}
return pdata;
}
#Override
public void close() {
}
}
This is consumer:
public class MyConsumer {
public static void main(String[] args) {
Properties properties = new Properties();
properties.put("bootstrap.servers", "kafka.kafka-cluster-shared.non-prod-5-az-scus.prod.us.walmart.net:9092");
properties.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
properties.put("value.deserializer", "xxxxxxxx.KafkaJsonDeserializer");
properties.put("group.id", "consumer-group-1");
properties.put("enable.auto.commit", "true");
properties.put("auto.commit.interval.ms", "1000");
properties.put("auto.offset.reset", "earliest");
properties.put("session.timeout.ms", "30000");
KafkaConsumer<String, pharmacyData> consumer = new KafkaConsumer<>(properties);
String topic = "insights";
consumer.subscribe(Collections.singletonList(topic));
while (true) {
ConsumerRecords<String, pharmacyData> consumerRecords = consumer.poll(100);
for (ConsumerRecord<String, pharmacyData> consumerRecord : consumerRecords) {
System.out.println(consumerRecord.key() + "--" + consumerRecord.toString());
//System.out.println(consumerRecord.offset() + "--" + consumerRecord.partition());
}
}
}
}
Can someone please help me with the issues? Thank you very much!
Problem solved:
The solution of this issue just add a default constructor as below:
public pharmacyData() {
}
See this page for more details.
I want to write test where it checks if two objects are same. When the assert fails I want to know what fields are same and which are not. I can do assert on each of the field but was wondering if there is a way to compere the objects.
Truck Class:
import java.util.List;
public class Truck {
private String model;
private String make;
private int year;
private List<TruckPartsObject> parts;
public Truck(String model, String make, int year) {
super();
this.model = model;
this.make = make;
this.year = year;
}
public String getModel() {
return model;
}
public void setModel(String model) {
this.model = model;
}
public String getMake() {
return make;
}
public void setMake(String make) {
this.make = make;
}
public int getYear() {
return year;
}
public void setYear(int year) {
this.year = year;
}
public List<TruckPartsObject> getParts() {
return parts;
}
public void setParts(List<TruckPartsObject> parts) {
this.parts = parts;
}
#Override
public String toString() {
return "Truck{" + "model='" + model + '\'' + ", make='" + make + '\''
+ ", year=" + year + ", parts=" + parts + '}';
}
}
TruckPartsObject Class:
import java.util.HashMap;
public class TruckPartsObject {
private String name;
private String price;
private HashMap<String, String> partsHashMap;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getPrice() {
return price;
}
public void setPrice(String price) {
this.price = price;
}
public HashMap<String, String> getPartsHashMap() {
return partsHashMap;
}
public void setPartsHashMap(HashMap<String, String> partsHashMap) {
this.partsHashMap = partsHashMap;
}
#Override
public String toString() {
return "TruckPartsObject{" + "name='" + name + '\'' + ", price='"
+ price + '\'' + ", partsHashMap=" + partsHashMap + '}';
}
}
Test:
#Test
public void truckTest () {
Truck pickupTruck1 = new Truck("Big 10", "Chevy", 1976);
Truck pickupTruck2 = new Truck("Big 10", "Chevy", 1976);
List<TruckPartsObject> parts1 = new ArrayList<TruckPartsObject>();
TruckPartsObject truckPartsObject1 = new TruckPartsObject();
truckPartsObject1.setName("part1");
parts1.add(truckPartsObject1);
pickupTruck1.setParts(parts1);
List<TruckPartsObject> parts2 = new ArrayList<TruckPartsObject>();
TruckPartsObject truckPartsObject2 = new TruckPartsObject();
truckPartsObject2.setName("part1");
parts2.add(truckPartsObject2);
pickupTruck2.setParts(parts2);
pickupTruck2.setParts(parts2);
System.out.println(pickupTruck1);
System.out.println(pickupTruck2);
//how to assert pickupTruck1 and pickupTruck2 have the same
}
how to assert pickupTruck1 and pickupTruck2 have the same?
Thanks to a co-worker for the bellow solution:
Use Shazamcrest (https://github.com/shazam/shazamcrest)
assertThat(pickupTruck1, sameBeanAs(pickupTruck2));
I am getting the below exception:
Could not find PortableFactory for factory-id: 1
com.hazelcast.nio.serialization.HazelcastSerializationException: Could
not find PortableFactory for factory-id: 1
On the client side I have the following code:
public class ClientTest {
public static void main(String[] args) {
List<String> nodes = new ArrayList<String>();
nodes.add("localhost:5701");
ClientConfig clientConfig = new ClientConfig();
ClientNetworkConfig networkConfig = new ClientNetworkConfig();
networkConfig.setAddresses(nodes);
clientConfig.setNetworkConfig(networkConfig);
SerializationConfig serCong = clientConfig.getSerializationConfig();
serCong.addPortableFactory(1, new UserFactoryImpl());
serCong.setPortableVersion(1);
HazelcastInstance hzClient1 = HazelcastClient.newHazelcastClient(clientConfig);
IMap<String, User> map = hzClient1.getMap("user");
System.out.println(map.size() + "hiten");
User user1 = new User();
user1.setFirstName("hiten");
user1.setLastName("singh");
map.put("1", user1);
//hz1.getLifecycleService().terminate();
System.out.println(map.size() + "after");
User user2 = new User();
user2.setFirstName("hiten1");
user2.setLastName("singh1");
map.put("2", user2);
UserEntryProcessor entryProc = new UserEntryProcessor();
User userRes = (User)map.executeOnKey("1", entryProc);
}
static class UserEntryProcessor implements EntryProcessor<String, User>, HazelcastInstanceAware {
private transient HazelcastInstance hazelcastInstance;
#Override
public Object process(Entry<String, User> entry) {
User user = entry.getValue();
if(user != null) {
System.out.println(user.getFirstName());
}
return user;
}
#Override
public EntryBackupProcessor<String, User> getBackupProcessor() {
return null;
}
#Override
public void setHazelcastInstance(HazelcastInstance hazelcastInstance) {
this.hazelcastInstance = hazelcastInstance;
}
}
static class UserFactoryImpl implements PortableFactory{
public final static int USER_PORTABLE_ID = 1;
public final static int FACTORY_ID = 1;
public Portable create(int classId) {
switch (classId) {
case USER_PORTABLE_ID:
return new User();
}
return null;
}
}
static class User implements Portable {
private String firstName;
private String lastName;
public String getLastName() {
return lastName;
}
public void setLastName(String lastName) {
this.lastName = lastName;
}
public String getFirstName() {
return firstName;
}
public void setFirstName(String firstName) {
this.firstName = firstName;
}
#Override
public int getFactoryId() {
return UserFactoryImpl.FACTORY_ID;
}
#Override
public int getClassId() {
return UserFactoryImpl.USER_PORTABLE_ID;
}
#Override
public void writePortable(PortableWriter writer) throws IOException {
writer.writeUTF("first_name", firstName);
writer.writeUTF("last_name", lastName);
}
#Override
public void readPortable(PortableReader reader) throws IOException {
firstName = reader.readUTF("first_name");
lastName = reader.readUTF("last_name");
}
}
}
Yes it does, just as you figured out the factory and the classes need to be available. Currently there is no built-in solution to not share classes for more sophisticated use cases than simple gets / puts. I have JSON support and some other ideas cooking but nothing really done yet.
I'm trying to delete node with fields in node_auto_indexing.
When I try to delete node using repository.delete(id).
Right after that I'm trying to get deleted Node by its id and I get following exception:
java.lang.IllegalStateException: This index (Index[__rel_types__,Relationship]) has been marked as deleted in this transaction
at org.neo4j.index.impl.lucene.LuceneTransaction$DeletedTxDataBoth.illegalStateException(LuceneTransaction.java:475)
at org.neo4j.index.impl.lucene.LuceneTransaction$DeletedTxDataBoth.removed(LuceneTransaction.java:470)
at org.neo4j.index.impl.lucene.LuceneTransaction.remove(LuceneTransaction.java:112)
at org.neo4j.index.impl.lucene.LuceneXaConnection.remove(LuceneXaConnection.java:116)
at org.neo4j.index.impl.lucene.LuceneIndex.remove(LuceneIndex.java:215)
at org.springframework.data.neo4j.support.typerepresentation.AbstractIndexBasedTypeRepresentationStrategy.remove(AbstractIndexBasedTypeRepresentationStrategy.java:113)
at org.springframework.data.neo4j.support.typerepresentation.AbstractIndexBasedTypeRepresentationStrategy.preEntityRemoval(AbstractIndexBasedTypeRepresentationStrategy.java:100)
at org.springframework.data.neo4j.support.mapping.EntityRemover.removeRelationship(EntityRemover.java:63)
at org.springframework.data.neo4j.support.mapping.EntityRemover.removeNode(EntityRemover.java:51)
at org.springframework.data.neo4j.support.mapping.EntityRemover.removeNodeEntity(EntityRemover.java:45)
at org.springframework.data.neo4j.support.mapping.EntityRemover.remove(EntityRemover.java:85)
at org.springframework.data.neo4j.support.Neo4jTemplate.delete(Neo4jTemplate.java:267)
at org.springframework.data.neo4j.repository.AbstractGraphRepository.delete(AbstractGraphRepository.java:276)
at org.springframework.data.neo4j.repository.AbstractGraphRepository.delete(AbstractGraphRepository.java:282)
Also, when I'm trying to delete node via Cypher query
#Query("MATCH ()-[r]-(p:Product) WHERE id(p) = {productId} DELETE r, p")
void deleteProduct(#Param("productId") Long productId);
I'm getting another exception after looking this deleted Node by its Id:
java.lang.IllegalStateException: No primary SDN label exists .. (i.e one starting with _)
at org.springframework.data.neo4j.support.typerepresentation.LabelBasedNodeTypeRepresentationStrategy.readAliasFrom(LabelBasedNodeTypeRepresentationStrategy.java:126)
at org.springframework.data.neo4j.support.typerepresentation.LabelBasedNodeTypeRepresentationStrategy.readAliasFrom(LabelBasedNodeTypeRepresentationStrategy.java:39)
at org.springframework.data.neo4j.support.mapping.TRSTypeAliasAccessor.readAliasFrom(TRSTypeAliasAccessor.java:36)
at org.springframework.data.neo4j.support.mapping.TRSTypeAliasAccessor.readAliasFrom(TRSTypeAliasAccessor.java:26)
at org.springframework.data.convert.DefaultTypeMapper.readType(DefaultTypeMapper.java:102)
at org.springframework.data.convert.DefaultTypeMapper.getDefaultedTypeToBeUsed(DefaultTypeMapper.java:165)
at org.springframework.data.convert.DefaultTypeMapper.readType(DefaultTypeMapper.java:142)
at org.springframework.data.neo4j.support.mapping.Neo4jEntityConverterImpl.read(Neo4jEntityConverterImpl.java:78)
at org.springframework.data.neo4j.support.mapping.Neo4jEntityPersister$CachedConverter.read(Neo4jEntityPersister.java:170)
at org.springframework.data.neo4j.support.mapping.Neo4jEntityPersister.createEntityFromState(Neo4jEntityPersister.java:189)
at org.springframework.data.neo4j.support.Neo4jTemplate.createEntityFromState(Neo4jTemplate.java:224)
at org.springframework.data.neo4j.repository.AbstractGraphRepository.createEntity(AbstractGraphRepository.java:62)
at org.springframework.data.neo4j.repository.AbstractGraphRepository.findOne(AbstractGraphRepository.java:127)
at org.springframework.data.neo4j.repository.AbstractGraphRepository.delete(AbstractGraphRepository.java:282)
How to correctly delete node that participates in Lucene Legacy Indexing node_auto_indexing ? How to remove this Node from Lucene index ?
UPDATED:
This is my Neo4jConfig:
#Configuration
#EnableNeo4jRepositories(basePackages = "com.example")
#EnableTransactionManagement
public class Neo4jConfig extends Neo4jConfiguration implements BeanFactoryAware {
#Resource
private Environment environment;
private BeanFactory beanFactory;
public Neo4jConfig() {
setBasePackage("com.example");
}
#Bean(destroyMethod = "shutdown")
public GraphDatabaseService graphDatabaseService() {
GraphDatabaseService graphDb = new GraphDatabaseFactory()
.newEmbeddedDatabaseBuilder("target/example-test-db")
.setConfig(GraphDatabaseSettings.node_keys_indexable, "name,description")
.setConfig(GraphDatabaseSettings.node_auto_indexing, "true")
.newGraphDatabase();
return graphDb;
}
/**
* Hook into the application lifecycle and register listeners that perform
* behaviour across types of entities during this life cycle
*
*/
#Bean
protected ApplicationListener<BeforeSaveEvent<BaseEntity>> beforeSaveEventApplicationListener() {
return new ApplicationListener<BeforeSaveEvent<BaseEntity>>() {
#Override
public void onApplicationEvent(BeforeSaveEvent<BaseEntity> event) {
BaseEntity entity = event.getEntity();
if (entity.getCreateDate() == null) {
entity.setCreateDate(new Date());
} else {
entity.setUpdateDate(new Date());
}
}
};
}
#Override
public void setBeanFactory(BeanFactory beanFactory) throws BeansException {
this.beanFactory = beanFactory;
}
public BeanFactory getBeanFactory() {
return beanFactory;
}
}
Base entity for entities in the project:
public class BaseEntity {
private Date createDate;
private Date updateDate;
public BaseEntity() {
}
public Date getCreateDate() {
return createDate;
}
public void setCreateDate(Date createDate) {
this.createDate = createDate;
}
public Date getUpdateDate() {
return updateDate;
}
public void setUpdateDate(Date updateDate) {
this.updateDate = updateDate;
}
}
and the Vote entity that I tried to delete:
#NodeEntity
public class Vote extends BaseEntity {
private static final String VOTED_ON = "VOTED_ON";
private final static String VOTED_FOR = "VOTED_FOR";
private static final String CREATED_BY = "CREATED_BY";
#GraphId
private Long id;
#RelatedTo(type = VOTED_FOR, direction = Direction.OUTGOING)
private Decision decision;
#RelatedTo(type = VOTED_ON, direction = Direction.OUTGOING)
private Criterion criterion;
#RelatedTo(type = CREATED_BY, direction = Direction.OUTGOING)
private User author;
private double weight;
private String description;
public Vote() {
}
public Vote(Decision decision, Criterion criterion, User author, double weight, String description) {
this.decision = decision;
this.criterion = criterion;
this.author = author;
this.weight = weight;
this.description = description;
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public Decision getDecision() {
return decision;
}
public void setDecision(Decision decision) {
this.decision = decision;
}
public Criterion getCriterion() {
return criterion;
}
public void setCriterion(Criterion criterion) {
this.criterion = criterion;
}
public User getAuthor() {
return author;
}
public void setAuthor(User author) {
this.author = author;
}
public double getWeight() {
return weight;
}
public void setWeight(double weight) {
this.weight = weight;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
#Override
public boolean equals(Object o) {
if (this == o)
return true;
if (o == null || getClass() != o.getClass())
return false;
Vote vote = (Vote) o;
if (id == null)
return super.equals(o);
return id.equals(vote.id);
}
#Override
public int hashCode() {
return id != null ? id.hashCode() : super.hashCode();
}
#Override
public String toString() {
return ToStringBuilder.reflectionToString(this, ToStringStyle.SHORT_PREFIX_STYLE);
}
}
Thanks to #MichaelHunger and Neo4j this issue has been fixed in Neo4j 2.2.2 and SDN 3.4.0.M1
I'm trying to deserialize JSON from Reddit that you can obtain by appending .json to the url. An example would be:
http://www.reddit.com/r/pics/comments/1wvx52/.json?sort=top
However, I am getting the error message:
Exception in thread "main" com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 9765
At line 1 column 9765 in the json there is the following code: "replies": "", whereas normally this would contain an object like this: replies: {
kind: "Listing",
data: {}
},
Does this mean that the json is a String when there is no data, but an object otherwise? How can I deserialize with gson properly if this is the case? I've included my classes below. I still need to figure out how to handle the json starting off with an array of basically two different objects (the first listing in the json is describing the link, while the second listing is describing the comments), but I'll cross that bridge when I get there. Thanks in advance if anyone can shed some light on this issue.
Main Class
public static void main(String[] args)
{
ArrayList<CommentsResults> commentsResults = new ArrayList<CommentsResults>();
String commentsURL = "http://www.reddit.com/r/pics/comments/1wvx52/.json?sort=top";
URL url = null;
try
{
url = new URL(commentsURL);
} catch (MalformedURLException ex)
{
System.out.println(ex.getMessage());
}
try
{
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(url.openStream()));
String jsonText = readAll(bufferedReader);
Gson gson = new GsonBuilder().create();
commentsResults = gson.fromJson(jsonText, new TypeToken<ArrayList<CommentsResults>>(){}.getType());
} catch (IOException ex)
{
System.out.println(ex.getMessage());
}
}
private static String readAll(Reader reader) throws IOException
{
StringBuilder stringBuilder = new StringBuilder();
int cp;
while ((cp = reader.read()) != -1)
{
stringBuilder.append((char) cp);
}
return stringBuilder.toString();
}
CommentsResults Class
public class CommentsResults {
private String kind;
private CommentsData data;
public CommentsResults()
{
}
public CommentsResults(String kind, CommentsData data)
{
this.kind = kind;
this.data = data;
}
public String getKind()
{
return kind;
}
public CommentsData getData()
{
return data;
}
public void setKind(String kind)
{
this.kind = kind;
}
public void setData(CommentsData data)
{
this.data = data;
}
}
CommentsData Class
private String modhash;
private List <CommentsChild> children;
public CommentsData()
{
}
public CommentsData(String modhash, List<CommentsChild> children)
{
this.modhash = modhash;
this.children = children;
}
public String getModhash()
{
return modhash;
}
public List<CommentsChild> getChildren()
{
return children;
}
public void setModhash(String modhash)
{
this.modhash = modhash;
}
public void setChildren(List<CommentsChild> children)
{
this.children = children;
}
CommentsChild Class
private String kind;
private Comment data;
public CommentsChild()
{
}
public CommentsChild(String kind, Comment comment)
{
this.kind = kind;
this.data = comment;
}
public String getKind()
{
return kind;
}
public Comment getComment()
{
return data;
}
public void setKind(String kind)
{
this.kind = kind;
}
public void setComment(Comment comment)
{
this.data = comment;
}
Comment Class
public class Comment {
private CommentsResults replies;
private String id;
private int gilded;
private String author;
private String parent_id;
private String body;
private int downs;
private String link_id;
private boolean score_hidden;
private int created_utc;
private String distinguished;
public Comment()
{
}
public Comment(CommentsResults replies, String id, int gilded, String author, String parent_id, String body, int downs, String link_id, boolean score_hidden, int created_utc, String distinguished)
{
this.replies = replies;
this.id = id;
this.gilded = gilded;
this.author = author;
this.parent_id = parent_id;
this.body = body;
this.downs = downs;
this.link_id = link_id;
this.score_hidden = score_hidden;
this.created_utc = created_utc;
this.distinguished = distinguished;
}
public CommentsResults getReplies()
{
return replies;
}
public String getId()
{
return id;
}
public int getGilded()
{
return gilded;
}
public String getAuthor()
{
return author;
}
public String getParent_id()
{
return parent_id;
}
public String getBody()
{
return body;
}
public int getDowns()
{
return downs;
}
public String getLink_id()
{
return link_id;
}
public boolean isScore_hidden()
{
return score_hidden;
}
public int getCreated_utc()
{
return created_utc;
}
public String getDistinguished()
{
return distinguished;
}
public void setReplies(CommentsResults replies)
{
this.replies = replies;
}
public void setId(String id)
{
this.id = id;
}
public void setGilded(int gilded)
{
this.gilded = gilded;
}
public void setAuthor(String author)
{
this.author = author;
}
public void setParent_id(String parent_id)
{
this.parent_id = parent_id;
}
public void setBody(String body)
{
this.body = body;
}
public void setDowns(int downs)
{
this.downs = downs;
}
public void setLink_id(String link_id)
{
this.link_id = link_id;
}
public void setScore_hidden(boolean score_hidden)
{
this.score_hidden = score_hidden;
}
public void setCreated_utc(int created_utc)
{
this.created_utc = created_utc;
}
public void setDistinguished(String distinguished)
{
this.distinguished = distinguished;
}
}
So in the off chance this helps somebody (which seems dubious at this point) I decided to parse the Json manually using recursion. Here's how I did it:
public static void getCommentsOnLink()
{
String commentsURL= "http://www.reddit.com/r/pics/comments/1wvx52/.json?sort=top";
URL url = null;
try
{
url = new URL(commentsURL);
} catch (MalformedURLException ex)
{
System.out.println(ex.getMessage());
}
String JsonText = readCommentJsonFromURL(url);
RedditCommentResults redditCommentResults = getCommentResults(JsonText);
}
private static String readCommentJsonFromURL(URL url)
{
String JSONText = null;
try
{
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(url.openStream()));
JSONText = readAll(bufferedReader);
} catch (IOException ex)
{
System.out.println(ex.getMessage());
}
return JSONText;
}
private static String readAll(Reader reader) throws IOException
{
StringBuilder stringBuilder = new StringBuilder();
int cp;
while ((cp = reader.read()) != -1)
{
stringBuilder.append((char) cp);
}
return stringBuilder.toString();
}
private static RedditCommentResults getCommentResults(String JsonText)
{
JsonParser parser = new JsonParser();
JsonArray completeJson = (JsonArray) parser.parse(JsonText);
//get link and comment object from the array containing an object for each
JsonObject linkParentJson = (JsonObject) completeJson.get(0);
JsonObject commentParentJson = (JsonObject) completeJson.get(1);
//use automatic deserializer for redditLink
JsonObject linkListingDataJson = linkParentJson.getAsJsonObject("data");
JsonObject linkChildrenJson = linkListingDataJson.getAsJsonArray("children").get(0).getAsJsonObject();
JsonObject linkDataJson = linkChildrenJson.getAsJsonObject("data");
Link commentLink = gson.fromJson(linkDataJson, Link.class);
RedditLink redditCommentLink = new RedditLink(commentLink);
//parse comments manually
JsonObject commentDataJson = commentParentJson.getAsJsonObject("data");
JsonArray commentChildrenJson = commentDataJson.getAsJsonArray("children");
//get all of the comments from the JsonArray
ArrayList<RedditComment> redditComments = getNestedComments(commentChildrenJson);
RedditCommentResults redditCommentResults = new RedditCommentResults(redditComments, redditCommentLink);
return redditCommentResults;
}
private static ArrayList<RedditComment> getNestedComments(JsonArray commentWrapperJsonArray)
{
ArrayList<RedditComment> redditComments = new ArrayList();
for (JsonElement commentWrapperJson : commentWrapperJsonArray)
{
//cast Element to Object so we can search for the primitive "kind". Finally we get it as a String
String kind = commentWrapperJson.getAsJsonObject().getAsJsonPrimitive("kind").getAsString();
//if the comment is of type t1 meaning it is a comment and not a "more" (a "more" is a comment which
//hasn't been loaded yet because it does not have a great deal of upvotes relative to other comments)
if (kind.equals("t1"))
{
JsonObject commentJson = commentWrapperJson.getAsJsonObject().getAsJsonObject("data");
Comment comment = gson.fromJson(commentJson, Comment.class);
RedditComment redditComment = new RedditComment(comment);
JsonElement repliesJson = commentJson.get("replies");
//if the reply is not equal to an empty String (i.e. if there is at least one reply)
if (!repliesJson.isJsonPrimitive())
{
JsonObject dataJson = repliesJson.getAsJsonObject().getAsJsonObject("data");
JsonArray childrenJson = dataJson.getAsJsonArray("children");
ArrayList<RedditComment> nestedComments = getNestedComments(childrenJson);
redditComment.setReplies(nestedComments);
}
redditComments.add(redditComment);
}
}
return redditComments;
}
You have to remove the private CommentsResults replies; from Comment Class and compose replies in CommentsChild class. According to Json fomate you model is
CommentResult -
CommentsData --
List <CommentsChild> children---
CommentsResults replies
recursion/ repeation of comment result
public class CommentsChild {
private String kind;
private Comment data;
//
private CommentsResults replies;
}