NHibernate: how to select a sorted parent/child and retrieve only specific row_numbers - nhibernate

I have 2 tables: Parent and Child which have the following relation: Parent has many Childs.
public class Parent
{
public DateTime Timestamp;
public IList<Child> Child;
}
public Child
{
public string Name;
}
I want to select both Parent and Child, Sorted by Timestamp and get only rows between index x to y.
public IList<Parent> Get(DateTime from, DateTime to, int startRow, int count)
{
QueryOver<Parent>().Where(row => row.Timestamp >= from)
.And(row => row.Timestamp <= to).OrderBy(row => row.Timestamp).Asc.List();
}
I don't know how to get only the required rows.
Should I do it with QueryOver? or its better doing it in HQL?
Thanks

I changed the relation and instead of having Parent and Child I use only one table:
public class Info
{
public DateTime Timestamp;
public string Name;
}
In order to get all records between dates, sorted and get them from index startRow to startRow + count I used the following:
public IList<Info> GetInfo (DateTime fromDate, DateTime toDate, int startRow, int count)
{
IList<Info> result =
QueryOver<Info>()
.Where(row => row.Timestamp >= fromDate)
.And(row => row.Timestamp <= toDate)
.OrderBy(row => row.Timestamp).Asc
.Skip(startRow).Take(count).List();
return result;
}
The resulted SQL is:
SELECT * FROM Info WHERE timestamp >= :fromDate AND timestamp <= :toDate
ORDER BY timestamp ASC OFFSET :startRow ROWS FETCH NEXT :count ROWS ONLY

Related

how to use spark sql udaf to implement window counting with condition?

I have a table with columns: timestamp and id and condition, and I want to count the number of each id per interval such as 10 seconds.
If condition is true, the count++, otherwise return the previous value.
the udaf code like:
public class MyCount extends UserDefinedAggregateFunction {
#Override
public StructType inputSchema() {
return DataTypes.createStructType(
Arrays.asList(
DataTypes.createStructField("condition", DataTypes.BooleanType, true),
DataTypes.createStructField("timestamp", DataTypes.LongType, true),
DataTypes.createStructField("interval", DataTypes.IntegerType, true)
)
);
}
#Override
public StructType bufferSchema() {
return DataTypes.createStructType(
Arrays.asList(
DataTypes.createStructField("timestamp", DataTypes.LongType, true),
DataTypes.createStructField("count", DataTypes.LongType, true)
)
);
}
#Override
public DataType dataType() {
return DataTypes.LongType;
}
#Override
public boolean deterministic() {
return true;
}
#Override
public void initialize(MutableAggregationBuffer mutableAggregationBuffer) {
mutableAggregationBuffer.update(0, 0L);
mutableAggregationBuffer.update(1, 0L);
}
public void update(MutableAggregationBuffer mutableAggregationBuffer, Row row) {
long timestamp = mutableAggregationBuffer.getLong(0);
long count = mutableAggregationBuffer.getLong(1);
long event_time = row.getLong(1);
int interval = row.getInt(2);
if (event_time > timestamp + interval) {
timestamp = event_time - event_time % interval;
count = 0;
}
if (row.getBoolean(0)) {
count++;
}
mutableAggregationBuffer.update(0, timestamp);
mutableAggregationBuffer.update(1, count);
}
#Override
public void merge(MutableAggregationBuffer mutableAggregationBuffer, Row row) {
}
#Override
public Object evaluate(Row row) {
return row.getLong(1);
}
}
Then I sumbit a sql like:
select timestamp, id, MyCount(true, timestamp, 10) over(PARTITION BY id ORDER BY timestamp) as count from xxx.xxx
the result is:
timestamp id count
1642760594 0 1
1642760596 0 2
1642760599 0 3
1642760610 0 2 --duplicate
1642760610 0 2
1642760613 0 3
1642760594 1 1
1642760597 1 2
1642760600 1 1
1642760603 1 2
1642760606 1 4 --duplicate
1642760606 1 4
1642760608 1 5
When the timestamp is repeated, I get 1,2,4,4,5 instead of 1,2,3,4,5
How to fix it?
And another requestion is that when to execute the merge method of udaf? I empty implement it but it runs normally. I try to add the log in the method but I haven't seen this log. Is it really necessary?
There is a similar question: Apache Spark SQL UDAF over window showing odd behaviour with duplicate input
However, row_number() does not have such a problem. row_number() is a hive udaf, then I try to create a hive udaf. But I also have the problem...Why hive udaf row_number() terminate() returns 'ArrayList'? I create my udaf row_number2() by copying its code then I got list return?
Finally I solved it by spark aggregateWindowFunction:
case class Count(condition: Expression) extends AggregateWindowFunction with Logging {
override def prettyName: String = "myCount"
override def dataType: DataType = LongType
override def children: Seq[Expression] = Seq(condition)
private val zero = Literal(0L)
private val one = Literal(1L)
private val count = AttributeReference("count", LongType, nullable = false)()
private val increaseCount = If(condition, Add(count, one), count)
override val initialValues: Seq[Expression] = zero :: Nil
override val updateExpressions: Seq[Expression] = increaseCount :: Nil
override val evaluateExpression: Expression = count
override val aggBufferAttributes: Seq[AttributeReference] = count :: Nil
Then use spark_session.functionRegistry.registerFunction to register it.
"select myCount(true) over(partition by window(timestamp, '10 seconds'), id order by timestamp) as count from xxx"

Calculate volumes based on date

I have this MariaDB table which I would like to use for bar chart:
CREATE TABLE `payment_transaction_daily_facts` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`date` date DEFAULT NULL,
`year` int(11) DEFAULT NULL,
`month` int(11) DEFAULT NULL,
`week` int(11) DEFAULT NULL,
`day` int(11) DEFAULT NULL,
`volume` int(11) DEFAULT NULL,
`count` int(11) DEFAULT NULL,
'created_at' date DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
In my example SQL query I have single column for Date. How I can calculate the volumes per day for last 10 days when I have split date, year, month, week and day into different columns?
The final result should be for example:
Date | Amount| Number of transactions per day |
11-11-2018 | 30 | 3 |
11-12-2018 | 230 | 13 |
I tried this:
SELECT SUM(amount) AS sum_volume, COUNT(*) AS sum_Transactions
WHERE (created_at BETWEEN '2018-11-07' AND '2018-11-08')
GROUP BY DATE(created_at)
I want to return the generated data using DTO:
public class DashboardDTO {
private Date date;
private int sum_volume;
private int sum_Transactions;
... getters and setters
}
Rest controller:
#RestController
#RequestMapping("/dashboard")
public class DashboardController {
private static final Logger LOG = LoggerFactory.getLogger(DashboardController.class);
#Autowired
private DashboardRepository dashboardRepository;
#Autowired
private PaymentTransactionsDailyFactsMapper mapper;
#GetMapping("/volumes")
public ResponseEntity<List<DashboardDTO>> getProcessingVolumes(#PathVariable String start_date, #PathVariable String end_date) {
List<DashboardDTO> list = StreamSupport.stream(dashboardRepository.findPaymentTransactionsDailyFacts(start_date, end_date).spliterator(), false)
.map(mapper::toDTO)
.collect(Collectors.toList());
return ResponseEntity.ok(list);
}
}
JPA query:
public List<PaymentTransactionsDailyFacts> findPaymentTransactionsDailyFacts(LocalDateTime start_date, LocalDateTime end_date) {
String hql = "SELECT SUM(amount) AS sum_volume, COUNT(*) AS sum_Transactions " +
" WHERE (created_at BETWEEN :start_date AND :end_date )" +
" GROUP BY DATE(created_at)";
TypedQuery<PaymentTransactionsDailyFacts> query = entityManager.createQuery(hql,
PaymentTransactionsDailyFacts.class).setParameter("start_date", start_date).setParameter("end_date", end_date);
List<PaymentTransactionsDailyFacts> data = query.getResultList();
return data;
}
How should I implement the query properly?
When I receive start_date and end_date as String from Angular how should I convert it into LocaDateTime?
Well, as I commented, time is a dimension in a data warehouse star schema, and I guess period is as well. So you should have two dimension tables, a TimeDim for LocalDate, and a PeriodDim for Period. Then you should have a Fact with the an embeddedId made up of the various dimensions in your schema. Then you would have facts for 1 day periods and facts for 10 day periods. If you insisted on summing facts you have the issue that JPA cannot do a <= or >= comparison against composite keys. Since you are only summing 10 days you could use a in clause to select 10 keys, but again, you should have facts for the periods you need.
#Entity
public class TimeDim {
#Id
private LocalDate localDate;
#Entity
public class PeriodDim {
#Id
private Period period;
// need this too
#Converter(autoApply = true)
public class LocalDateAttributeConverter implements AttributeConverter<LocalDate, Date> {
#Override
public Date convertToDatabaseColumn(LocalDate locDate) {
return (locDate == null ? null : Date.valueOf(locDate));
}
#Override
public LocalDate convertToEntityAttribute(Date sqlDate) {
return (sqlDate == null ? null : sqlDate.toLocalDate());
}
}
#SuppressWarnings("serial")
#Embeddable
public class DimKey implements Serializable {
private LocalDate localDate;
private Period period;
#Entity
public class Fact {
#EmbeddedId
private DimKey dimKey = new DimKey();
private long amount;
And for example:
tx.begin();
TimeDim td10 = new TimeDim();
td10.setLocalDate(LocalDate.now().minusDays(5));
em.persist(td10);
TimeDim td5 = new TimeDim();
td5.setLocalDate(LocalDate.now().minusDays(10));
em.persist(td5);
PeriodDim pd5 = new PeriodDim();
pd5.setPeriod(Period.ofDays(5));
em.persist(pd5);
PeriodDim pd10 = new PeriodDim();
pd10.setPeriod(Period.ofDays(10));
em.persist(pd10);
Fact f10 = new Fact();
f10.getDimKey().setLocalDate(td10.getLocalDate());
f10.getDimKey().setPeriod(pd10.getPeriod());
f10.setAmount(100);
em.persist(f10);
Fact f51 = new Fact();
f51.getDimKey().setLocalDate(td10.getLocalDate());
f51.getDimKey().setPeriod(pd5.getPeriod());
f51.setAmount(50);
em.persist(f51);
Fact f52 = new Fact();
f52.getDimKey().setLocalDate(td5.getLocalDate());
f52.getDimKey().setPeriod(pd5.getPeriod());
f52.setAmount(50);
em.persist(f52);
tx.commit();
em.clear();
DimKey dk = new DimKey();
dk.setLocalDate(td10.getLocalDate());
dk.setPeriod(pd10.getPeriod());
Fact f = em.createQuery("select f from Fact f where f.dimKey = :dimKey", Fact.class)
.setParameter("dimKey", dk)
.getSingleResult();
System.out.println("From 10 day period: " + f.getAmount());
DimKey dk1 = new DimKey();
dk1.setLocalDate(td10.getLocalDate());
dk1.setPeriod(pd5.getPeriod());
DimKey dk2 = new DimKey();
dk2.setLocalDate(td5.getLocalDate());
dk2.setPeriod(pd5.getPeriod());
Long sum = em.createQuery("select sum(f.amount) from Fact f where f.dimKey in (:dimKey1 , :dimKey2)", Long.class)
.setParameter("dimKey1", dk1)
.setParameter("dimKey2", dk2)
.getSingleResult();
System.out.println("From 2*5 day period: " + sum);

How to convert this sql part to linq

select sum(DATEDIFF(day,LeaveBreakup.StartDate,LeaveBreakup.EndDate)+1)
what I want is to convert the statement to linq select statement
class LeaveBreakup
{
public DateTime StartDate { get; set; }
public DateTime EndDate { get; set; }
}
List<LeaveBreakup> Dates = new List<LeaveBreakup>();
Dates.Add(new LeaveBreakup(){StartDate = DateTime.Now.AddDays(-3), EndDate = DateTime.Now });
Dates.Add(new LeaveBreakup(){StartDate = DateTime.Now.AddDays(-2), EndDate = DateTime.Now });
LINQ BIT
var Result = (from D in Dates
select (D.EndDate - D.StartDate).TotalDays + 1)
.Sum();
If you want to know the diff without having to worry about having a negative value then wrap the calculation in Math.Abs
var Result = (from D in Dates
select Math.Abs((D.StartDate - D.EndDate).TotalDays) + 1)
.Sum();
In this example your Result is 7

Nhibernate Join 2 QueryOver

I can't find how to join two different QueryOver, group by and perform a substraction in the select.
Say you have :
public class EntityA
{
public virtual int Id;
public virtual string Reference;
public virtual int Quantity;
[Some properties]
}
public class EntityB
{
public virtual int Id;
public virtual int EntityAId;
[Some properties]
}
If i translate my query in pseudo-SQL, i would like to have :
SELECT A.Id, A.Reference, A.Quantity - COALESCE(DERIV_B.TOTAL, 0)
FROM EntityA A
LEFT JOIN (
SELECT B.EntityAId, COUNT(B.Id) AS TOTAL
FROM EntityB B
GROUP BY B.EntityAId) DERIV_B
ON A.Id = DERIV_B.EntityAId
WHERE (A.Quantity - COALESCE(DERIV_B.TOTAL, 0)) >= 0
I can have the subquery on EntityB via QueryOver, but i can't join on EntityA :
var entitiesB = GetCurrentSession().QueryOver<EntityB>().SelectList(select => select.SelectGroup(x => x.EntityAId).SelectCount(x => x.Id));
var entitiesA = GetCurrentSession().QueryOver<EntityA>(). ???
I tried to store the entitiesB in an alias and the perform a JoinAlias on it but i have an exception because it can't retrieve my alias.
Do you have any solution ?
I don't want to create a reference between these two entites.
Short answer is not , you can't do QueryOver if your entities are not connected through model.
One solution would be to use NHibernate.Linq and subqueries
var session = GetCurrentSession();
var entityBQuery = session.Query<EntityB>();
var entityAQuery = session.Query<EntityA>()
.Select(eA=>new { Id = eA.Id,
Description = eA.Description,
Quantity = eA.Quantity - entityBQuery.Where(eb=>eb.EntityAId = eA.Id).Count()
});

LINQ to SQL retrieve multiple columns from single datatable including grouping and sum

I've got a single datatable that contains the fields:
TransactionID (int),
TransactionDate (DateTime)
AdjustedValue (decimal)
I'd like to achieve in LINQ the equivalent of the following SQL:
SELECT TransactionID, TransactionDate, Sum(AdjustedValue), COUNT(*) AS ItemCount
FROM DATATABLE
WHERE TransactionDate >= BeginDate
and TransacationDate < EndDate
GROUP BY TransactionID
ORDER BY TransactionID DESCENDING
I've tried the following:
var query = (from lui in DTbatches.AsEnumerable()
where lui.TransactionDate >= BeginDate &&
lui.TransactionDate < EndDate
group lui.TransactionID by lui.TransactionID into g
order by g.Key descending
select new TransactionIDListItem
{
TransactionID = g.Key,
ItemCount = g.Count()
}).ToList();
This works, but only returns the TransactionID and the count of records in the specified transaction. How should I modify my LINQ to also include the TransactionDate and the sum(AdjustedValue) fields?
My TransactionIDListItem class is defined as:
public int BatchID {get; set;}
public DateTime TransactionDate {get; set;}
public int ItemCount {get; set;}
public decimal TotalValue {get; set;}
You need to create an anonymous type for your group to get multiple values.
group lui by new
{
lui.TransactionID,
lui.TransactionDate
}
into g
Now in your select, you just need to add the new values. g.Key is your anonymous type, so it has the properties you grouped by.
select new TransactionIDListItem
{
TransactionID = g.Key.TransactionID,
TransactionDate = g.Key.TransactionDate,
TotalValue = g.Sum(x => x.AdjustedValue)
ItemCount = g.Count()
}).ToList();