How to get result using better algorithm? - sql
We have the list like below
List:
(("herry","0,1,2"),("herry","1,3"),("herry","3,6"),("herry","4"),("John","5"))
As the number in the string may be referred by different elements, the expected result is:
("herry","0,1,2,3,6"), ("herry","4"), ("John","5")
I worked out solution using scala, but it looks complicated, is there a more clean and easy way to work out the result? Thanks in advance!
Here is my solution in scala,
val foo=List(("herry","0,1,2"),("herry","1,3"),("herry","3,6"),("herry","4"),("John","5"))
println(GetValue)
def GetValue()={
foo.zipWithIndex.map((tuple: ((String, String), Int)) =>{
val tuples = getrelated(tuple._1, foo)
(tuple._2, tuples)
}).map((tuple: (Int, List[(String, String)])) => tuple._2)
.map((tuples: List[(String, String)]) => (tuples.head._1,tuples.map((tuple: (String, String)) => tuple._2)))
.map((tuple: (String, List[String])) => (tuple._1, tuple._2.mkString(",").split(",").distinct.sorted.mkString(",")))
.distinct
}
def getrelated(start:(String,String),fooList:List[(String,String)]):List[(String,String)]={
val fooListWithout = fooList.filter((tuple: (String, String)) => tuple != start)
val result=fooListWithout
.filter((tuple: (String, String)) => findmatching(tuple._2,start._2))
.flatMap((tuple: (String, String)) => start :: getrelated(tuple,fooListWithout))
if (result.isEmpty)
List(start)
else
result
}
def findmatching(key1:String,key2:String)={
(key1.split(",")++key2.split(","))
.groupBy(identity)
.mapValues((strings: Array[String]) => strings.size)
.exists((tuple: (String, Int)) => tuple._2>1)
}
Let me clarify the algorithm
if the number list has the overlapping number, then group these number as one element
if the number list has no overlapping number, then consider it as independent element
for example,
input: List(("herry","0,1,2"),("herry","1,3"),("herry","7,4"),("herry","4"),("John","5"))
expected output: List(("herry","0,1,2,3" ), ("herry","4,7"), ("John","5"))
input: List(("herry","0,1,2"),("herry","1,3"),("herry","3,6"),("herry","4"),("John","5"))
expected: List("herry","0,1,2,3,6"), ("herry","4"), ("John","5")
input: List(("herry","0,1"),("herry","2,3"),("herry","3,6"),("herry","4"),("John","5"))
expected: List("herry","0,1"),("herry","2,3,6"), ("herry","4"), ("John","5")
My assumption is that tuples for the same key that contain multiple values should be aggregated. It is unclear what should happen in the same value appears singly and as part of a list, e.g. ("herry", "4"), ("herry, "1,4") however it should be simple enough to remove any such cases
val data = List(("herry","0,1,2"),("herry","1,3"),("herry","3,6"),("herry","4"),("John","5"))// Start writing your ScalaFiddle code here
val (singles, multiples) = data.partition{case (name, list) => !list.contains(",")}
val multiplesAggregated = multiples
.groupBy{case (key, _) => key)
.map{
case (key, values) =>
key -> values.flatMap{case (_, numbers) => numbers.split(",")}.distinct.sorted.mkString(",")}.toList
println(multiplesAggregated ++ singles)
Output:
List((herry,0,1,2,3,6), (herry,4), (John,5))
Related
Constructor can not be instantiated Slick Scala
I was trying to convert a query from SQL into Scala code with Slick, but I have got a compiler error in filter clause: constructor cannot be instantiated to expected type. My code in Slick: val subquery = (for { pit <- PassInTripTable.table t <- TripTable.table if pit.tripNoFk === t.tripNo } yield (pit, t)) .map{ case (pit, t) => ( pit, Case.If(t.townFrom <= t.townTo).Then(t.townFrom ++ t.townTo).Else(t.townFrom ++ t.townTo) )} .groupBy(_._1.idPsgFk) .filter{ case ((pit, count), group) => ( group.map(_._2).countDistinct === 1)} .map(_._1) val query = PassengerTable.table.filter(_.idPsg in subquery).map(_.name) db.run(query.result) The query in SQL itself: select name from passenger where id_psg in ( select id_psg from trip t,pass_in_trip pit where t.trip_no=pit.trip_no group by id_psg having count(distinct case when town_from<=town_to then town_from+town_to else town_to+town_from end)=1 ) I would be very grateful if someone helped me to find an error.
From looking at your code, it looks like the type you are matching on is not supposed to be "((pit, count), group)". groupBy in Slick only returns a collection of Tuple2s. http://slick.lightbend.com/doc/3.0.0/queries.html So, the filter might look something like... .filter{ case (pit, count) => ( count.map(_._2).countDistinct === 1)}
The problem is that Slick .groupBy requires a .map call with aggregating functions afterwards. You can find detailed information here. So, try this: .groupBy(_._1.idPsgFk) .map{ case (key, group) => (key, group.map(_._2).countDistinct)} .filter{ case (_, count) => count === 1} .map(_._1) P.S. I've also found "bad smells" in your code. You get pairs as a result of for-comrehension, but it looks like standard join would be more appropriate here (and more efficient), something like: PassInTripTable.table.join(TripTable.table).on(_.tripNoFk === _.tripNo) .map{ case (pit, t) => ...} And why would you use such condition: Case.If(t.townFrom <= t.townTo).Then(t.townFrom ++ t.townTo).Else(t.townFrom ++ t.townTo)? Its branches are the same, so equals to t.townFrom ++ t.townTo.
How correctly built an object graph based on multi level join in Slick?
I have a model structure as following: Group -> Many Parties -> Many Participants In on of the API calls I need to get single groups with parties and it's participants attached. This whole structure is built on 4 tables: group party party_participant participant Naturally, with SQL it's a pretty straight forward join that combines all of them. And this is exactly what I am trying to do with slick. Mu method is dao class looks something like this: def findOneByKeyAndAccountIdWithPartiesAndParticipants(key: UUID, accountId: Int): Future[Option[JourneyGroup]] = { val joins = JourneyGroups.groups join Parties.parties on (_.id === _.journeyGroupId) joinLeft PartiesParticipants.relations on (_._2.id === _.partyId) joinLeft Participants.participants on (_._2.map(_.participantId) === _.id) val query = joins.filter(_._1._1._1.accountId === accountId).filter(_._1._1._1.key === key) val q = for { (((journeyGroup, party), partyParticipant), participant) <- query } yield (journeyGroup, party, participant) val result = db.run(q.result) result ???? } The problem here, is that the result is type of Future[Seq[(JourneyGroup, Party, Participant)]] However, what I really need is Future[Option[JourneyGroup]] Note: case classes of JourneyGroup and Party have sequences for there children defined: case class Party(id: Option[Int] = None, partyType: Parties.Type.Value, journeyGroupId: Int, accountId: Int, participants: Seq[Participant] = Seq.empty[Participant]) and case class JourneyGroup(id: Option[Int] = None, key: UUID, name: String, data: Option[JsValue], accountId: Int, parties: Seq[Party] = Seq.empty[Party]) So they both can hold the descendants. What is the correct way to convert to the result I need? Or am I completely in a wrong direction? Also, is this statement is correct: Participants.participants on (_._2.map(_.participantId) === _.id) ?
I ended up doing something like this: journeyGroupDao.findOneByKeyAndAccountIdWithPartiesAndParticipants(key, account.id.get) map { data => val groupedByJourneyGroup = data.groupBy(_._1) groupedByJourneyGroup.map { case (group, rows) => val parties = rows.map(_._2).distinct map { party => val participants = rows.filter(r => r._2.id == party.id).flatMap(_._3) party.copy(participants = participants) } group.copy(parties = parties) }.headOption } where DAO method's signature is: def findOneByKeyAndAccountIdWithPartiesAndParticipants(key: UUID, accountId: Int): Future[Seq[(JourneyGroup, Party, Option[Participant])]]
Is there a way in Scalding Type Safe API to mapWithValue with two or more ValuePipes?
Using Scalding, Type Safe API, this code works, where dictForKeys and dictForValues are both ValuePipe[Map[String,String]]: SomeKeyValueTypedPipe .mapWithValue(dictForKeys) { case ((key, value), dictForKeys) => (dictForKeys.get.getOrElse(key, key), value) } .mapWithValue(dictForValues) { case ((key, value), dictForValues) => (key, dictForValues.get.getOrElse(value, value)) } I was just wondering whether there's a more compact way of writing this, i.e. use only 1 mapWithValue step with 2 separate ValuePipes.
You could create a ValuePipe of a tuple of Maps, like ValuePipe[(Map[String, String], Map[String, String])], and then use it like so: SomeKeyValueTypedPipe .mapWithValue(dict) { case ((key, value), (dictForKeys, dictForValues)) => (dictForKeys.get.getOrElse(key, key), dictForValues.get.getOrElse(value, value)) }
Slick query for one to optional one (zero or one) relationship
Given tables of: case class Person(id: Int, name: String) case class Dead(personId: Int) and populated with: Person(1, "George") Person(2, "Barack") Dead(1) is it possible to have a single query that would produce a list of (Person, Option[Dead]) like so? (Person(1, "George"), Some(Dead(1))) (Person(2, "Barack"), None)
For slick 3.0 it should be something like this: val query = for { (p, d) <- persons joinLeft deads on (_.id === _.personId) } yield (p, d) val results: Future[Seq[(Person, Option[Dead])]] = db.run(query.result) In slick, outer joins are automatically wrapped in an Option type. You can read more about joining here: http://slick.typesafe.com/doc/3.0.0/queries.html#joining-and-zipping
Slick: Read nullable values as option when left join
Problem when using Slick to join: I have 2 tables User and UserInfo and I want to leftJoin them to get user's info. I've tried this: val q = for{ (user,info) <- User leftJoin UserInfo on (_.id === _.userid) } yield(user, info) But the UserInfo table has some nullable field, so when I try to execute the query: q.map(user_info => (user_info._1,user_info._2)).list It makes error because user_info._2 has some null values. I know a solution that yield each field in UserInfo and add getOrElse(None) for nullable fields. However, UserInfo has many field so I don't want to use this. Can anyone help me?
What you CAN do, is this define a function that does the conversion, and then use it in your map: def nullToOption[A](input: A): Option[A] = input match { case null => None case x => Some(x) } And then you just use it in your map. I made a simple example using a simple list: val lst = List("Hello", null, "hi", null) val newlst = map lst nullToOption newList is now the following: List(Some("Hello"), None, Some("hi"), None) Of course you can modify nullToOption to fit your needs; here's a version that takes tuples: def nullToOption[A, B](input: (A,B)): (Option[A], Option[B]) = input match { case (x, y) => (Some(x), Some(y)) case (x, null) => (Some(x), None) case (null, y) => (None, Some(y)) case (null, null) => (None, None) }