How to output elements of an ArrayList based on the frequency? - arraylist

My Arraylist looks like [A, B, D, E, C, A, A, B]
and i want to print to console
A,3
B,2
C,1
D,1
E,1
How can I do this in Java?

Create a hashmap, in which there will be one entry per symbol. The symbol itself will be the 'key' and the # of times that symbol shows up will be the value within the hashmap.
Iterate through your list, each time you find a new symbol, add a matching entry to the map, and each time you find an existing symbol, add one to its value in the hashmap
Add the hashmaps' entryset to a list, sort that list using a comparator then print this sorted list out.
public static void main(String[] args) {
String mySymbols = "A,B,D,E,C,A,A,B";
ArrayList<String> myList = new ArrayList<String>();
Collections.addAll(myList, mySymbols.split(","));
HashMap<String, Integer> countingMap = new HashMap<String, Integer>();
for (String s : myList){
if(countingMap.containsKey(s)){
Integer newValue = countingMap.get(s) + 1;
countingMap.put(s, newValue);
}
else{
countingMap.put(s, 1);
}
}
List<Entry> entries = new ArrayList<Entry>();
entries.addAll(countingMap.entrySet());
entries.sort(new Comparator<Entry>() {
#Override
public int compare(Entry o1, Entry o2) {
return (Integer)o2.getValue() - (Integer)o1.getValue();
}
});
Iterator iter = entries.iterator();
while(iter.hasNext()){
Entry thisEntry = (Entry) iter.next();
Object key = thisEntry.getKey();
Object value = thisEntry.getValue();
System.out.println(key+", "+value);
}
}
Test the code here

You could accomplish this with a Stream.
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.stream.Collectors;
class MyClass {
public static void main(String...args) {
printListElementFrequencies(Arrays.asList("A", "B", "D", "E", "C", "A", "A", "B"));
}
static void printListElementFrequencies(List<String> list) {
Map<String, Long> frequencies = list.stream().collect(Collectors.groupingBy(s -> s, TreeMap::new, Collectors.counting()));
Comparator<String> sortByFrequencyThenLexicographically = Comparator.<String, Long>comparing(frequencies::get).reversed().thenComparing(Comparator.comparing(s -> s));
list.stream().sorted(sortByFrequencyThenLexicographically).forEach(s -> System.out.println(s + ", " + frequencies.get(s)));
}
}
Edit: I missed the sorting part.

If the contents of your array are of type char this can be a approached using another array
char[] foo = new char['Z' - 'A'];
then you will loop through your original array incrementing the position of that char in foo. if you use a fast sorting algorithm afterwards afterwards you can just print them out but if i+1 = i then compare them by char value

It could be done by converting the list into frequency map and sorting the entry set by values in descending order and then by key:
List<String> list = Arrays.asList("A", "B", "D", "E", "C", "A", "A", "B", "Z", "D");
list.stream()
.collect(Collectors.groupingBy(x -> x, Collectors.counting())) // Map<String, Long>
.entrySet()
.stream()
.sorted(Map.Entry.<String, Long>comparingByValue().reversed()
.thenComparing(Map.Entry.comparingByKey())
) // Stream<Map.Entry<String, Long>>
.forEach(e -> System.out.println(e.getKey() + ", " + e.getValue()));
Output:
A, 3
B, 2
D, 2
C, 1
E, 1
Z, 1
Also it is possible to use Collectors::toMap with a merge function (here the frequency is calculated as Integer):
list.stream()
.collect(Collectors.toMap(x -> x, x -> 1, Integer::sum)) // Map<String, Integer>
.entrySet()
.stream()
.sorted(Map.Entry.<String, Integer>comparingByValue().reversed()
.thenComparing(Map.Entry.comparingByKey())
)
.forEach(e -> System.out.println(e.getKey() + " -> " + e.getValue()));

Related

How to add columns subsequently in a dataset using Spark within a for loop ( where for loop contains the column name)

Here trying to add subsequently column to dataset Row, the issue coming up is last column is only visible. The columns added earlier do not persist
private static void populate(Dataset<Row> res, String[] args)
{
String[] propArr = args[0].split(","); // Eg: [abc, def, ghi]
// Dataset<Row> addColToMergedData = null;
/** Here each element is the name of the column to be inserted */
for(int i = 0; i < propArr.length; i++){
// addColToMergedData = res.withColumn(propArr[i], lit(null));
}
}
the logic in the for loop is flawed hence the issue.
you can modify the program as follows :
private static void populate(Dataset<Row> res, String[] args)
{
String[] propArr = args[0].split(","); // Eg: [abc, def, ghi]
Dataset<Row> addColToMergedData = null;
/** Here each element is the name of the column to be inserted */
for(int i = 0; i < propArr.length; i++)
{
res = res.withColumn(propArr[i], lit(null));
}
addColToMergedData = res
}
Sol:
// addColToMergedData = res.withColumn(colMap.get(propArr[i]), lit(null));
should be written as :
res = res.withColumn(colMap.get(propArr[i]), lit(null));
Here's the clean and conscious way to do this by using Scala:
val columnsList = Seq("one", "two", "three", "four", "five")
val res = columnsList.foldLeft(addColToMergedData) { case (df, colName) =>
df.withColumn(colu, lit(null))
}

how can we iterate in ArrayList inside HashMap?

I want to print each value in ArrayList separately
i.e {1=[A, B,C, D], 2=[E, F, G, H]}
HashMap> hash=new HashMap>();
// Now i wnat to travesr inside array list of specific key in HashMap
if user enter 1 (i.e Key) then output should be
A
B
C
D
if user enter 2 (i.e Key) then output should be
E
F
G
H
I'm not quite sure, I get your question, but if you want to traverse an array of keys and look up all the values corresponding to these keys in a hashmap, you could do something like this (assuming the question is in java):
import java.util.*;
class Main {
public static HashMap<Integer, String[]> hmap = new HashMap<Integer, String[]>();
public static int[] arrToTraverse = {1,2};
public static void main(String[] args) {
String[] s1 = {"A", "B", "C", "D"};
String[] s2 = {"E", "F", "G", "H"};
hmap.put(1, s1);
hmap.put(2, s2);
for(int no : arrToTraverse) {
System.out.println(Arrays.toString(getValue(no)));
}
}
public static String[] getValue(int key) {
return hmap.get(key);
}
}
Which will output:
[A, B, C, D]
[E, F, G, H]

Comparator in binary search

I am not sure how to write comparator for Collectionos.binarySearch(). Can anyone help ? sample code:
List<Object> list1 = new ArrayList<>();
List<List<Object>> list2 = new ArrayList<>();
//loop starts
// adds elements into list1
list1.add(values);//values is an object containing elements like [3, John, Smith]
if (list2.size() == 0) {
list2.add(list1);//first element
} else {
if (index >= 0) {
int index = Collections.binarySearch(list2, list1, comparator);
list2.add(index, list1);//I want to add these elements in ascending order ?
}
}
//loop ends
How do I write comparator, so that elements in list 2 are added in ascending or descending order.
You can use an anonymous class which implements a Comparator<List<Object>>:
int index = Collections.binarySearch(list2, list1, new Comparator<List<Object>>() {
#Override
public int compare(List<Object> o1, List<Object> o2) {
// Your implementation here
return 0;
}
});
You could implement an IComparer<List<Object>> class, or use a lambda expression.
You just need to create a class that implements the Comparator interface.
For example, you can do this inline with an anonymous class:
Comparator<List<Object>> comparator = new Comparator<List<Object>>() {
#Override
public int compare(List<Object> x, List<Object> y) {
// custom logic to compare x and y here. Return a negative number
// if x < y, a positive number if x > y, and 0 otherwise
}
};
Collections.binarySearch(list, comparator);

How to write this Pig query?

I have a many-to-many mapping table between two collections. Each row in the mapping table represents a possible mapping with a weight score.
mapping(id1, id2, weight)
Query: Generate one to one mapping between id1 and id2. Use lowest weight to remove duplicate mappings. If there is tie, output any arbitrary one.
Example input:
(1, X, 1)
(1, Y, 2)
(2, X, 3)
(2, Y, 1)
(3, Z, 2)
Output
(1, X)
(2, Y)
(3, Z)
1 and 2 are both mapped to X and Y. We pick mapping (1, X) and (2, Y) because they have the lowest weight.
I will assume that you are only interested in mappings where the weight is the lowest of any mapping involving id1, and also the lowest of any mapping involving id2. For example, if you additionally had the mapping (2, Y, 4), it would not conflict with (1, X, 1). I will exclude such mappings because the weight is smaller than (1, Y, 2) and (2, X, 3), which were disqualified.
My solution proceeds as follows: find the minimum mapping weight for each id1, and then join that into the mapping relation for future reference. Use a nested foreach to go through each id2: use ORDER and LIMIT to select the record with the smallest weight for that id2, and then only keep it if the weight is also the minimum for that id1.
Here is the full script, tested on your input:
mapping = LOAD 'input' AS (id1:chararray, id2:chararray, weight:double);
id1_weights =
FOREACH (GROUP mapping BY id1)
GENERATE group AS id1, MIN(mapping.weight) AS id1_min_weight;
mapping_with_id1_mins =
FOREACH (JOIN mapping BY id1, id1_weights BY id1)
GENERATE mapping::id1, id2, weight, id1_min_weight;
accepted_mappings =
FOREACH (GROUP mapping_with_id1_mins BY id2)
{
ordered = ORDER mapping_with_id1_mins BY weight;
selected = LIMIT ordered 1;
acceptable = FILTER selected BY weight == id1_min_weight;
GENERATE FLATTEN(acceptable);
};
DUMP accepted_mappings;
Solved it by using Java UDF. it's not perfect in a sense that it won't maximize the number of one-to-one mappings but it's good enough.
Pig:
d = load 'test' as (fid, iid, priority:double);
g = group d by fid;
o = foreach g generate FLATTEN(com.propeld.pig.DEDUP(d)) as (fid, iid, priority);
store o into 'output';
g2 = group o by iid;
o2 = foreach g2 generate FLATTEN(com.propeld.pig.DEDUP(o)) as (fid, iid, priority);
store o2 into 'output2';
Java UDF:
package com.propeld.pig;
import java.io.IOException;
import java.util.Iterator;
import org.apache.pig.Algebraic;
import org.apache.pig.EvalFunc;
import org.apache.pig.PigException;
import org.apache.pig.backend.executionengine.ExecException;
import org.apache.pig.data.DataBag;
import org.apache.pig.data.Tuple;
import org.apache.pig.data.TupleFactory;
public class DEDUP extends EvalFunc<Tuple> implements Algebraic{
public String getInitial() {return Initial.class.getName();}
public String getIntermed() {return Intermed.class.getName();}
public String getFinal() {return Final.class.getName();}
static public class Initial extends EvalFunc<Tuple> {
private static TupleFactory tfact = TupleFactory.getInstance();
public Tuple exec(Tuple input) throws IOException {
// Initial is called in the map.
// we just send the tuple down
try {
// input is a bag with one tuple containing
// the column we are trying to operate on
DataBag bg = (DataBag) input.get(0);
if (bg.iterator().hasNext()) {
Tuple dba = (Tuple) bg.iterator().next();
return dba;
} else {
// make sure that we call the object constructor, not the list constructor
return tfact.newTuple((Object) null);
}
} catch (ExecException e) {
throw e;
} catch (Exception e) {
int errCode = 2106;
throw new ExecException("Error executing an algebraic function", errCode, PigException.BUG, e);
}
}
}
static public class Intermed extends EvalFunc<Tuple> {
public Tuple exec(Tuple input) throws IOException {
return dedup(input);
}
}
static public class Final extends EvalFunc<Tuple> {
public Tuple exec(Tuple input) throws IOException {return dedup(input);}
}
static protected Tuple dedup(Tuple input) throws ExecException, NumberFormatException {
DataBag values = (DataBag)input.get(0);
Double min = Double.MAX_VALUE;
Tuple result = null;
for (Iterator<Tuple> it = values.iterator(); it.hasNext();) {
Tuple t = (Tuple) it.next();
if ((Double)t.get(2) < min){
min = (Double)t.get(2);
result = t;
}
}
return result;
}
#Override
public Tuple exec(Tuple input) throws IOException {
return dedup(input);
}
}

On Java 7's equals() and deepEquals()

Method description says:
Returns true if the arguments are deeply equal to each other and false
otherwise... Equality is determined by using the equals method
of the first argument.
Which (to me) suggests that Objects are deeply equal if every object they maintain references to are also equal using the equals() method. And every objects they have a reference to are also equal. And ..
So .. equality is determined by using the equals method of the first argument.
How is this different from .equals()? Assuming that we describe equals appropriately where, objects is equal to another object is every field of the object is equal to it as well.
Can you please provide an example illustrating the difference between Objects.deepEquals() and Objects.equals()?
String[] firstArray = {"a", "b", "c"};
String[] secondArray = {"a", "b", "c"};
System.out.println("Are they equal 1 ? " + firstArray.equals(secondArray) );
System.out.println("Are they equal 2 ? " + Objects.equals(firstArray, secondArray) );
System.out.println("Are they deepEqual 1? " + Arrays.deepEquals(firstArray, secondArray) );
System.out.println("Are they deepEqual 2? " + Objects.deepEquals(firstArray, secondArray) );
will return
Are they equal 1 ? false
Are they equal 2 ? false
Are they deepEqual 1? true
Are they deepEqual 2? true
How come the "shallow" equals methods return false? This is because in Java, for arrays, equality is determined by object identity. In this example, firstArray and secondArray are distinct objects.
Doing String[] secondArray = firstArray instead will therefore return true for all four tests.
If at least one of the arguments of deepEquals method is not an array, then Objects.deepEquals and Objects.equals are same.
Example:
import java.util.Arrays;
import java.util.Objects;
public class Main {
public static void main(String[] args) {
Integer[] x = { 1, 2 };
Integer[] y = { 1, 2 };
System.out.println(Objects.equals(x, y)); // false
System.out.println(Objects.deepEquals(x, y)); // true
System.out.println(Arrays.equals(x, y)); // true
System.out.println(Arrays.deepEquals(x, y)); // true
System.out.println();
int[][] a = { { 1, 2 }, { 3, 4 } };
int[][] b = { { 1, 2 }, { 3, 4 } };
System.out.println(Objects.equals(a, b)); // false
System.out.println(Objects.deepEquals(a, b)); // true
System.out.println(Arrays.equals(a, b)); // false
System.out.println(Arrays.deepEquals(a, b)); // true
}
}
Documentation and decompiled code:
Objects#equals(Object a, Object b): Returns true if the arguments are equal to each other and false otherwise. Consequently, if both arguments are null, true is returned and if exactly one argument is null, false is returned. Otherwise, equality is determined by using the equals method of the first argument.
public static boolean equals(Object a, Object b) {
return (a == b) || (a != null && a.equals(b));
}
Objects#deepEquals(Object a, Object b): Returns true if the arguments are deeply equal to each other and false otherwise. Two null values are deeply equal. If both arguments are arrays, the algorithm in Arrays.deepEquals is used to determine equality. Otherwise, equality is determined by using the equals method of the first argument.
public static boolean deepEquals(Object a, Object b) {
if (a == b)
return true;
else if (a == null || b == null)
return false;
else
return Arrays.deepEquals0(a, b);
}
Arrays#equals(Object[] a, Object[] a2): Returns true if the two specified arrays of Objects are equal to one another. The two arrays are considered equal if both arrays contain the same number of elements, and all corresponding pairs of elements in the two arrays are equal.
public static boolean equals(Object[] a, Object[] a2) {
if (a==a2)
return true;
if (a==null || a2==null)
return false;
int length = a.length;
if (a2.length != length)
return false;
for (int i=0; i<length; i++) {
if (!Objects.equals(a[i], a2[i]))
return false;
}
return true;
}
Arrays#deepEquals(Object[] a1, Object[] a2): Returns true if the two specified arrays are deeply equal to one another. Unlike the equals(Object[],Object[]) method, this method is appropriate for use with nested arrays of arbitrary depth.
public static boolean deepEquals(Object[] a1, Object[] a2) {
if (a1 == a2)
return true;
if (a1 == null || a2==null)
return false;
int length = a1.length;
if (a2.length != length)
return false;
for (int i = 0; i < length; i++) {
Object e1 = a1[i];
Object e2 = a2[i];
if (e1 == e2)
continue;
if (e1 == null)
return false;
// Figure out whether the two elements are equal
boolean eq = deepEquals0(e1, e2);
if (!eq)
return false;
}
return true;
}
static boolean deepEquals0(Object e1, Object e2) {
assert e1 != null;
boolean eq;
if (e1 instanceof Object[] && e2 instanceof Object[])
eq = deepEquals ((Object[]) e1, (Object[]) e2);
else if (e1 instanceof byte[] && e2 instanceof byte[])
eq = equals((byte[]) e1, (byte[]) e2);
else if (e1 instanceof short[] && e2 instanceof short[])
eq = equals((short[]) e1, (short[]) e2);
else if (e1 instanceof int[] && e2 instanceof int[])
eq = equals((int[]) e1, (int[]) e2);
else if (e1 instanceof long[] && e2 instanceof long[])
eq = equals((long[]) e1, (long[]) e2);
else if (e1 instanceof char[] && e2 instanceof char[])
eq = equals((char[]) e1, (char[]) e2);
else if (e1 instanceof float[] && e2 instanceof float[])
eq = equals((float[]) e1, (float[]) e2);
else if (e1 instanceof double[] && e2 instanceof double[])
eq = equals((double[]) e1, (double[]) e2);
else if (e1 instanceof boolean[] && e2 instanceof boolean[])
eq = equals((boolean[]) e1, (boolean[]) e2);
else
eq = e1.equals(e2);
return eq;
}
deepEquals() is used with nested arrays of arbitrary depth.
equals() is used with simple primitive data types.
For ex:
public class TwoDArray {
public static void main(String args[]) {
int a[][] = new int[2][2];
int b[][] = new int[2][2];
for(int i=0;i<2;i++)
for(int j=0;j<2;j++) {
a[i][j] = i+j;
b[i][j] = i+j;
}
System.out.println(Arrays.deepEquals(a,b));//return true
System.out.println(Arrays.equals(a, b));//return false
}
}
Attaching a very good example i found on javarevisited.blogspot.in
public class ArrayCompareTest {
public static void main(String args[]) {
//comparing primitive int arrays in Java
int[] i1 = new int[] {1,2,3,4};
int[] i2 = new int[] {1,2,3,4};
int[] i3 = new int[] {0,2,3,4};
//Arrays.equals() compare Array and return true if both array are equal
//i..e either both of them are null or they are identical in length, and each pair
//match each other e.g. i[0]=i2[0], i[1]=i2[1] and so on
//i1 and i2 should be equal as both contains same elements
boolean result = Arrays.equals(i1, i2);
System.out.println("Comparing int array i1: " + Arrays.toString(i1)
+ " and i1: " + Arrays.toString(i2));
System.out.println("Does array i1 and i2 are equal : " + result);
//array ii2 and i3 are not equals as only length is same, first pair is not same
result = Arrays.equals(i2, i3);
System.out.println("Comparing int array i2: " + Arrays.toString(i2)
+ " and i3: " + Arrays.toString(i3));
System.out.println("Does array i2 and i3 are equal : " + result);
//comparing floating point or double arrays in Java
double[] d1 = new double[] {1.5, 2.4, 3.2, 4,1};
double[] d2 = new double[] {1.5, 2.4, 3.2, 4,1};
double[] d3 = new double[] {0.0, 2.4, 3.2, 4,1};
//Comparing two floating-point arrays using Arrays.equals() in Java
//double array d1 and d2 should be equal - length same, each index matches
result = Arrays.equals(d1, d2);
System.out.println("Comparing double array d1: " + Arrays.toString(d1)
+ " and d2: " + Arrays.toString(d2));
System.out.println("Does double array d1 and d2 are equal : " + result);
//double array d2 and d3 is not equal - length same, first pair does not match
result = Arrays.equals(d2, d3);
System.out.println("Comparing double array d2: " + Arrays.toString(d2)
+ " and d3: " + Arrays.toString(d3));
System.out.println("Does double array d2 and d3 are same : " + result);
//comparing Object array, here we will use String array
String[] s1 = new String[]{"One", "Two", "Three"};
String[] s2 = new String[]{"One", "Two", "Three"};
String[] s3 = new String[]{"zero", "Two", "Three"};
//String array s1 and s2 is equal - length same, each pair matches
result = Arrays.equals(s1, s2);
System.out.println("Comparing two String array s1: " + Arrays.toString(s1)
+ " and s2: " + Arrays.toString(s2));
System.out.println("Are both String array s1 and s2 are equal : " + result);
//String array s2 and s3 is not equal - length same, first pair different
result = Arrays.equals(d2, d3);
System.out.println("Comparing two String array s2: " + Arrays.toString(s2)
+ " and s3: " + Arrays.toString(s3));
System.out.println("Are both String array s2 and s3 are equal : " + result);
//Comparing nested arrays with equals and deepEquals method
//Arrays.equals() method does not compare recursively,
//while deepEquals() compare recursively
//if any element inside Array is type of Array itself,
//as here second element is String array
Object[] o1 = new Object[]{"one", new String[]{"two"}};
Object[] o2 = new Object[]{"one", new String[]{"two"}};
System.out.println("Object array o1: " + Arrays.toString(o1) + " and o2: "
+ Arrays.toString(o2));
System.out.println("Comparing Object Array o1 and o2 with Arrays.equals : "
+ Arrays.equals(o1, o2));
System.out.println("Comparing Object Array o1 and o2 with Arrays.deepEquals : "
+ Arrays.deepEquals(o1, o2));
}
}
Output:
Comparing int array i1: [1, 2, 3, 4] and i1: [1, 2, 3, 4]
Does array i1 and i2 are equal : true
Comparing int array i2: [1, 2, 3, 4] and i3: [0, 2, 3, 4]
Does array i2 and i3 are equal : false
Comparing double array d1: [1.5, 2.4, 3.2, 4.0, 1.0] and d2: [1.5, 2.4, 3.2, 4.0, 1.0]
Does double array d1 and d2 are equal : true
Comparing double array d2: [1.5, 2.4, 3.2, 4.0, 1.0] and d3: [0.0, 2.4, 3.2, 4.0, 1.0]
Does double array d2 and d3 are same : false
Comparing two String array s1: [One, Two, Three] and s2: [One, Two, Three]
Are both String array s1 and s2 are equal : true
Comparing two String array s2: [One, Two, Three] and s3: [zero, Two, Three]
Are both String array s2 and s3 are equal : false
Object array o1: [one, [Ljava.lang.String;#19821f] and o2: [one, [Ljava.lang.String;#addbf1]
Comparing Object Array o1 and o2 with Arrays.equals : false
Comparing Object Array o1 and o2 with Arrays.deepEquals : true