Hey I am working on tree data structure. I want to know can we get index of each node in level wise. I below diagram represent how I want the value + index. Level A or B represent node value and index value represent index value
Node
| | |
Level A -> 1 2 3
index value-> 0 1 2
| | | | | |
| | | | | |
Leve B-> 4 5 6 7 8 9
index value-> 0 1 2 3 4 5
....// more level
How can we achieved index in each level wise. I am adding my logic how I am adding value in each level wise. Could you someone suggest how can I achieve this?
var baseNode: LevelIndex = LevelIndex()
var defaultId = "1234"
fun main() {
val list = getUnSortedDataListForLevel()
val tempHashMap: MutableMap<String, LevelIndex> = mutableMapOf()
list.forEach { levelClass ->
levelClass.levelA?.let { levelA ->
val levelOneTempHashMapNode = tempHashMap["level_a${levelA}"]
if (levelOneTempHashMapNode != null) {
if (defaultId == levelClass.id && levelOneTempHashMapNode is LevelOne) {
levelOneTempHashMapNode.defaultValue = true
}
return#let
}
val tempNode = LevelOne().apply {
value = levelA
if (defaultId == levelClass.id) {
defaultValue = true
}
}
baseNode.children.add(tempNode)
tempHashMap["level_a${levelA}"] = tempNode
}
levelClass.levelB?.let { levelB ->
val levelTwoTempHashMapNode = tempHashMap["level_a${levelClass.levelA}_level_b${levelB}"]
if (levelTwoTempHashMapNode != null) {
if (defaultId == levelClass.id && levelOneTempHashMapNode is LevelTwo) {
levelTwoTempHashMapNode.defaultValue = true
}
return#let
}
val tempNode = LevelTwo().apply {
value = levelB
if (defaultId == levelClass.id) {
defaultValue = true
}
}
val parent =
tempHashMap["level_a${levelClass.levelA}"] ?: baseNode
parent.children.add(tempNode)
tempHashMap["level_a${levelClass.levelA}_level_b${levelB}"] =
tempNode
}
levelClass.levelC?.let { levelC ->
val tempNode = LevelThree().apply {
value = levelC
if (defaultId == levelClass.id) {
defaultValue = true
}
}
val parent =
tempHashMap["level_a${levelClass.levelA}_level_b${levelClass.levelB}"]
?: baseNode
parent.children.add(tempNode)
}
}
}
open class LevelIndex(
var value: String? = null,
var children: MutableList<LevelIndex> = arrayListOf()
)
class LevelOne : LevelIndex() {
var defaultValue: Boolean? = false
}
class LevelTwo : LevelIndex() {
var defaultValue: Boolean? = false
}
class LevelThree : LevelIndex() {
var defaultValue: Boolean = false
}
UPDATE
I want index value by root level because, I have one id, I want to match that combination with that id, if that value is present then I am storing that value b true, and need to find that index value.
Node
| | |
Level A -> 1 2 3
index value-> 0 1 2
default value-> false true false
| | | | | |
| | | | | |
Leve B-> 4 5 6 7 8 9
index value-> 0 1 2 3 4 5
default value->false false true false false false
....// more level
So, Level A I'll get index 1.
For Level B I'll get index 2
I'd create a list to put the nodes at each level in order. You can recursively collect them from your tree.
val nodesByLevel = List(3) { mutableListOf<LevelIndex>() }
fun collectNodes(parent: LevelIndex) {
for (child in parent.children) {
val listIndex = when (child) {
is LevelOne -> 0
is LevelTwo -> 1
is LevelThree -> 2
// I made LevelIndex a sealed class. Otherwise you would need an else branch here.
}
nodesByLevel[listIndex] += child
collectNodes(child)
}
}
collectNodes(baseNode)
Now nodesByLevel contains three lists containing all the nodes in each layer in order.
If you just need the String values, you could change that mutableList to use a String type and use += child.value ?: "" instead, although I would make value non-nullable (so you don't need ?: ""), because what use is a node with no value?
Edit
I would move defaultValue up into the parent class so you don't have to cast the nodes to be able to read it. And I'm going to treat is as non-nullable.
sealed class LevelIndex(
var value: String = "",
val children: MutableList<LevelIndex> = arrayListOf()
var isDefault: Boolean = false
)
Then if you want to do something with the items based on their indices:
for ((layerNumber, layerList) in nodesByLevel.withIndex()) {
for((nodeIndexInLayer, node) in layerList) {
val selectedIndexForThisLayer = TODO() //with layerNumber
node.isDefault = nodeIndexInLayer == selectedIndexForThisLayer
}
}
Given the data set below:
a | b | c | d
1 | 3 | 7 | 11
1 | 5 | 7 | 11
1 | 3 | 8 | 11
1 | 5 | 8 | 11
1 | 6 | 8 | 11
Perform a reverse Cartesian product to get:
a | b | c | d
1 | 3,5 | 7,8 | 11
1 | 6 | 8 | 11
I am currently working with scala, and my input/output data type is currently:
ListBuffer[Array[Array[Int]]]
I have come up with a solution (seen below), but feel it could be optimized. I am open to optimizations of my approach, and completely new approaches. Solutions in scala and c# are preferred.
I am also curious if this could be done in MS SQL.
My current solution:
def main(args: Array[String]): Unit = {
// Input
val data = ListBuffer(Array(Array(1), Array(3), Array(7), Array(11)),
Array(Array(1), Array(5), Array(7), Array(11)),
Array(Array(1), Array(3), Array(8), Array(11)),
Array(Array(1), Array(5), Array(8), Array(11)),
Array(Array(1), Array(6), Array(8), Array(11)))
reverseCartesianProduct(data)
}
def reverseCartesianProduct(input: ListBuffer[Array[Array[Int]]]): ListBuffer[Array[Array[Int]]] = {
val startIndex = input(0).size - 1
var results:ListBuffer[Array[Array[Int]]] = input
for (i <- startIndex to 0 by -1) {
results = groupForward(results, i, startIndex)
}
results
}
def groupForward(input: ListBuffer[Array[Array[Int]]], groupingIndex: Int, startIndex: Int): ListBuffer[Array[Array[Int]]] = {
if (startIndex < 0) {
val reduced = input.reduce((a, b) => {
mergeRows(a, b)
})
return ListBuffer(reduced)
}
val grouped = if (startIndex == groupingIndex) {
Map(0 -> input)
}
else {
groupOnIndex(input, startIndex)
}
val results = grouped.flatMap{
case (index, values: ListBuffer[Array[Array[Int]]]) =>
groupForward(values, groupingIndex, startIndex - 1)
}
results.to[ListBuffer]
}
def groupOnIndex(list: ListBuffer[Array[Array[Int]]], index: Int): Map[Int, ListBuffer[Array[Array[Int]]]] = {
var results = Map[Int, ListBuffer[Array[Array[Int]]]]()
list.foreach(a => {
val key = a(index).toList.hashCode()
if (!results.contains(key)) {
results += (key -> ListBuffer[Array[Array[Int]]]())
}
results(key) += a
})
results
}
def mergeRows(a: Array[Array[Int]], b: Array[Array[Int]]): Array[Array[Int]] = {
val zipped = a.zip(b)
val merged = zipped.map{ case (array1: Array[Int], array2: Array[Int]) =>
val m = array1 ++ array2
quickSort(m)
m.distinct
.array
}
merged
}
The way this works is:
Loop over columns, from right to left (the groupingIndex specifies which column to run on. This column is the only one which does not have to have values equal to each other in order to merge the rows.)
Recursively group the data on all other columns (not groupingIndex).
After grouping all columns, it is assumed that the data in each group have equivalent values in every column except for the grouping column.
Merge the rows with the matching columns. Take the distinct values for each column and sort each one.
I apologize if some of this does not make sense, my brain is not functioning today.
Here is my take on this. Code is in Java but could easily be converted into Scala or C#.
I run groupingBy on all combinations of n-1 and go with the one that has the lowest count, meaning largest merge depth, so this is kind of a greedy approach. However it is not guaranteed that you will find the optimal solution, meaning minimize the number k which is np-hard to do, see link here for an explanation, but you will find a solution that is valid and do it rather fast.
Full example here: https://github.com/jbilander/ReverseCartesianProduct/tree/master/src
Main.java
import java.util.*;
import java.util.stream.Collectors;
public class Main {
public static void main(String[] args) {
List<List<Integer>> data = List.of(List.of(1, 3, 7, 11), List.of(1, 5, 7, 11), List.of(1, 3, 8, 11), List.of(1, 5, 8, 11), List.of(1, 6, 8, 11));
boolean done = false;
int rowLength = data.get(0).size(); //4
List<Table> tables = new ArrayList<>();
// load data into table
for (List<Integer> integerList : data) {
Table table = new Table(rowLength);
tables.add(table);
for (int i = 0; i < integerList.size(); i++) {
table.getMap().get(i + 1).add(integerList.get(i));
}
}
// keep track of count, needed so we know when to stop iterating
int numberOfRecords = tables.size();
// start algorithm
while (!done) {
Collection<List<Table>> result = getMinimumGroupByResult(tables, rowLength);
if (result.size() < numberOfRecords) {
tables.clear();
for (List<Table> tableList : result) {
Table t = new Table(rowLength);
tables.add(t);
for (Table table : tableList) {
for (int i = 1; i <= rowLength; i++) {
t.getMap().get(i).addAll(table.getMap().get(i));
}
}
}
numberOfRecords = tables.size();
} else {
done = true;
}
}
tables.forEach(System.out::println);
}
private static Collection<List<Table>> getMinimumGroupByResult(List<Table> tables, int rowLength) {
Collection<List<Table>> result = null;
int min = Integer.MAX_VALUE;
for (List<Integer> keyCombination : getKeyCombinations(rowLength)) {
switch (rowLength) {
case 4: {
Map<Tuple3<TreeSet<Integer>, TreeSet<Integer>, TreeSet<Integer>>, List<Table>> map =
tables.stream().collect(Collectors.groupingBy(t -> new Tuple3<>(
t.getMap().get(keyCombination.get(0)),
t.getMap().get(keyCombination.get(1)),
t.getMap().get(keyCombination.get(2))
)));
if (map.size() < min) {
min = map.size();
result = map.values();
}
}
break;
case 5: {
//TODO: Handle n = 5
}
break;
case 6: {
//TODO: Handle n = 6
}
break;
}
}
return result;
}
private static List<List<Integer>> getKeyCombinations(int rowLength) {
switch (rowLength) {
case 4:
return List.of(List.of(1, 2, 3), List.of(1, 2, 4), List.of(2, 3, 4), List.of(1, 3, 4));
//TODO: handle n = 5, n = 6, etc...
}
return List.of(List.of());
}
}
Output of tables.forEach(System.out::println)
Table{1=[1], 2=[3, 5, 6], 3=[8], 4=[11]}
Table{1=[1], 2=[3, 5], 3=[7], 4=[11]}
or rewritten for readability:
a | b | c | d
--|-------|---|---
1 | 3,5,6 | 8 | 11
1 | 3,5 | 7 | 11
If you were to do all this in sql (mysql) you could possibly use group_concat(), I think MS SQL has something similar here: simulating-group-concat or STRING_AGG if SQL Server 2017, but I think you would have to work with text columns which is a bit nasty in this case:
e.g.
create table my_table (A varchar(50) not null, B varchar(50) not null,
C varchar(50) not null, D varchar(50) not null);
insert into my_table values ('1','3,5','4,15','11'), ('1','3,5','3,10','11');
select A, B, group_concat(C order by C) as C, D from my_table group by A, B, D;
Would give the result below, so you would have to parse and sort and update the comma separated result for any next merge iteration (group by) to be correct.
['1', '3,5', '3,10,4,15', '11']
I have the following data sample :
file name: sample.txt
| TRANSACTION_ID | ITEM_ID | AUC_END_DT | BD_ID | BD_SITE |
+----------------+--------------+------------+-----------+---------+
| 320562466 | 7322548247 | 5/22/2005 | 32148826 | 77 |
| 569643695009 | 190558793670 | 7/31/2011 | 112644812 | 0 |
Here is the query that I'm running :
select * from table_name where item_id = '$item_id';
I need to convert this sample.txt file into sequence file and then need to create DataFrame for that sequence file for further analysis.
case class db_col( transaction_id:Double,
item_id:Long,
auc_end_dt:String,
bd_id:Long,
bd_site:Int)
object V_bd {
def main(args: Array[String]) {
val item_id_args = args(0)
val conf = new SparkConf().setAppName("POC_Naren").setMaster("local")
val sc = new SparkContext(conf)
val ssc = new SQLContext(sc)
import ssc.implicits._
val dw_bid_base_rdd = sc.textFile("C:/Users/Downloads/sqlscript/reference/data/sample.txt")
val bd_trans_rdd = dw_bid_base_rdd.map(row => row.split("\\|"))
val bd_col_rdd = bd_trans_rdd.map(p => db_col(p(0).trim.toDouble,p(1),p(2),p(.3).trim.tolong,p(4).trim.toInt))
val bd_df_rdd = bd_col_rdd.toDF()
bd_df_rdd.registerTempTable("bd_table")
val bd_table_query = ssc.sql("select * from table_name where item_id = '$item_id_args';")
bd_table_query.show()
}
}
You'll need to convert your DataFrame into a RDD[(K,V)]. Example
import org.apache.spark.rdd.RDD
import org.apache.spark.sql.{Row, DataFrame}
val bd_table_query : DataFrame = ???
val rdd : RDD[(Int,String)] = df.rdd.map {
case r : Row => (r.getAs[Int](0),r.getAs[String](1)) // I'll let you choose your keys and convert into the right format
}
Then you can save the RDD :
rdd.saveAsSequenceFile("output.seq")
In a simple example, I'm confused about how to turn this grammar into a LL one by removing the left recursion. Any hints are welcome.
G = {
A -> A a | A B | a
B -> b
}
I get the following by applying this algorithm:
G = {
A -> a X
X -> e | A | B X
B -> b
}
This seem to work to generate a C pseudo-code for the parser:
void A() {
switch (token) {
case 'a' : next(); X(); break;
}
}
void X() {
switch (token) {
case 'e' : finish(); break;
case 'a' : A(); break;
case 'b' : B(); X(); break;
}
}
void B() {
next();
}
And to generate a parsing tree for the word: aabab:
A ---+
| |
a X
|
A ---+
| |
a X ---+
| |
B X
| |
b A ---+
| |
a X ---+
| |
B X
| |
b e
Well, I'm just not sure if it's right...
First, your grammar seems to accept sequences of as separated by single bs, so that no 2 b come together. (aaa...abaaaaa...abaaa...a) This should be equivalent to something like:
Q -> P | P b P
P -> a | a P
I have the following datatables
table1:
+-----------+------+------+
| catalogid | name | snum |
+-----------+------+------+
| 353 | xx | 4 |
| 364 | yy | 3 |
| 882 | zz | 3 |
| 224 | mm | 71 |
| 999 | kk | 321 |
| 74 | kk | 4 |
| 54 | ii | 5 |
| 11 | u | 6 |
| 23 | yy | 6 |
+-----------+------+------+
table2:
+-----------+----------+--------------+
| catalogid | numitems | ignoreditems |
+-----------+----------+--------------+
| 353 | 4 | 0 |
| 364 | 10 | 0 |
| 882 | 2 | 0 |
| 224 | 0 | 7 |
+-----------+----------+--------------+
Using LINQ I want to join them and copy the result to a new datatable. They both share catalogid, and in the result it should only display the records that their catalogid exist in table2
result:
+-----------+------+------+-----------+---------------+
| catalogid | name | snum | numitems | ignoreditems |
+-----------+------+------+-----------+---------------+
| 353 | xx | 4 | 4 | 0 |
| 364 | yy | 3 | 10 | 0 |
| 882 | zz | 3 | 2 | 0 |
| 224 | mm | 71 | 0 | 7 |
+-----------+------+------+-----------+---------------+
Here is my attempt but it's not working:
Dim query = From a In oresult.AsEnumerable
Group Join b In products.AsEnumerable
On a.Field(Of Integer)("catalogid") Equals b.Field(Of Integer)("catalogid")
Into Group
query.copytodatatable
CopyToDatatable is not working, and I can't figure out why
CopyToDataTable() only works when your query returns an IEnumerable<'DataRow>. In your query, you are returning an anonymous type. Anonymous types don't carry the extension method for CopyToDataTable().
You can create a table using the ConvertToDataTable extension listed below. You'll have to convert it to VB.NET (there are converters out there if you google).
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using Common;
namespace TestConsole
{
public class Linq_join_2_datatables_that_share_a_column_and_put_result_in_new_datatable
{
public class Table1
{
public int CatalogId { get; set; }
public string Name { get; set; }
public int SNum { get; set; }
}
public class Table2
{
public int CatalogId { get; set; }
public int NumItems { get; set; }
public int IgnoredItems { get; set; }
}
public static void Start()
{
DataTable table1 = new DataTable();
table1.Columns.Add("catalogid", typeof(int));
table1.Columns.Add("name", typeof(string));
table1.Columns.Add("snum", typeof(int));
DataRow row = table1.Rows.Add(353, "xx", 4);
DataTable table2 = new DataTable();
table2.Columns.Add("catalogid", typeof(int));
table2.Columns.Add("numitems", typeof(int));
table2.Columns.Add("ignoreditems", typeof(int));
table2.Rows.Add(353, 4, 0);
var query = (from t1 in table1.AsEnumerable()
join t2 in table2.AsEnumerable() on t1.Field<int>("catalogid") equals t2.Field<int>("catalogid")
select new
{
catalogid = t1.Field<int>("catalogid"),
name = t1.Field<string>("name"),
snum = t1.Field<int>("snum"),
numitems = t2.Field<int>("numitems"),
ignoreditems = t2.Field<int>("ignoreditems")
}).ToList();
DataTable table3 = query.ConvertToDataTable();
}
}
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using System.ComponentModel;
using System.Reflection;
namespace Common
{
public static class DataTableExtensions
{
public static DataTable ConvertToDataTable<T>(this IList<T> data)
{
PropertyDescriptorCollection properties =
TypeDescriptor.GetProperties(typeof(T));
DataTable table = new DataTable();
foreach (PropertyDescriptor prop in properties)
table.Columns.Add(prop.Name, Nullable.GetUnderlyingType(prop.PropertyType) ?? prop.PropertyType);
foreach (T item in data)
{
DataRow row = table.NewRow();
foreach (PropertyDescriptor prop in properties)
row[prop.Name] = prop.GetValue(item) ?? DBNull.Value;
table.Rows.Add(row);
}
table.AcceptChanges();
return table;
}
}
}
CopyToDataRow only works on IEnumerables of DataRow. See this article at MSDN for an implementation of CopyToDataRow for arbitrary IEnumerables.