GroupBy generating wrong OrderBy query - asp.net-core

Question

I have the following query:
var trains = queryableTrains.Select(t => new TrainDto
{
Id = t.Id,
Prefix = t.Prefix,
Color = t.Color,
PlannedMovements = t.TrainMovementPlans
.Where(p => trackCircuitIds.Contains(p.TrackCircuitId) && p.TimeStamp >= start && p.TimeStamp <= end)
.Select(p => new TrainMovementDto
{
SgfId = null,
Prefix = t.Prefix,
BlockSectionId = p.TrackCircuit.TrackCircuitGroupId ?? p.TrackCircuitId,
Timestamp = p.TimeStamp,
IsExit = false
})
.GroupBy(o => o.BlockSectionId)
.Select(o => o.First())
//.OrderBy(d => d.Timestamp)
.ToList(),
// ...
It's a lot to code to share everything so I will share the concept hoping it helps.
Basically my data model is as follows:
Train has several PlannedMovements
I want to use GroupBy to remove "duplicates", i.e., movements that has the same BlockSectionId (after the projection).
When I run the above code, even with the OrderBy commented, I get the following exception:
[17:00:55 ERR] Failed executing DbCommand (9ms) [Parameters=[#_outer_Prefix='?' (Size = 4000), #__start_2='?' (DbType = DateTime2), #__end_3='?' (DbType = DateTime2), #_outer_Id2='?' (DbType = Int64)], CommandType='"Text"', CommandTimeout='30']
SELECT #_outer_Prefix AS [Prefix], COALESCE([p.TrackCircuit3].[YRCTCGID], [p5].[YSBTRCID]) AS [BlockSectionId], [p5].[YSBTIMES], CAST(0 AS bit) AS [IsExit], [p.TrackCircuit3].[YRCTCGID], [p5].[YSBTRCID]
FROM [YSBPLTMT] AS [p5]
INNER JOIN [YRCTRCIT] AS [p.TrackCircuit3] ON [p5].[YSBTRCID] = [p.TrackCircuit3].[YRCTRCID]
WHERE (([p5].[YSBTRCID] IN ( _HUGE_GROUP_I_HAVE_TO_CHANGE_FOR_A_JOIN_ ) AND ([p5].[YSBTIMES] >= #__start_2)) AND ([p5].[YSBTIMES] <= #__end_3)) AND (#_outer_Id2 = [p5].[YSBTRAID])
ORDER BY COALESCE([p.TrackCircuit3].[YRCTCGID], [p5].[YSBTRCID]), [BlockSectionId]
System.Data.SqlClient.SqlException (0x80131904): A column has been specified more than once in the order by list. Columns in the order by list must be unique.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.ExecuteReader()
at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.Execute(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues)
ClientConnectionId:f68441e8-baa8-4c16-aa66-0cae875ba173
Error Number:169,State:1,Class:15
[17:00:55 ERR] An exception occurred while iterating over the results of a query for context type 'GCO.Data.GcoContext'.
System.Data.SqlClient.SqlException (0x80131904): A column has been specified more than once in the order by list. Columns in the order by list must be unique.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.ExecuteReader()
at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.Execute(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues)
at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.ExecuteReader(IRelationalConnection connection, IReadOnlyDictionary`2 parameterValues)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.BufferlessMoveNext(DbContext _, Boolean buffer)
at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
at Microsoft.EntityFrameworkCore.Query.QueryMethodProvider._InjectParameters[TElement](QueryContext queryContext, IEnumerable`1 source, String[] parameterNames, Object[] parameterValues)+MoveNext()
at Microsoft.EntityFrameworkCore.Query.QueryMethodProvider._GroupBy[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)+MoveNext()
at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source, Int32& length)
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.OrderedEnumerable`1.GetEnumerator()+MoveNext()
at System.Linq.Enumerable.SelectIPartitionIterator`2.MoveNext()
at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at lambda_method(Closure , TransparentIdentifier`2 )
at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext()
ClientConnectionId:f68441e8-baa8-4c16-aa66-0cae875ba173
Error Number:169,State:1,Class:15
System.Data.SqlClient.SqlException (0x80131904): A column has been specified more than once in the order by list. Columns in the order by list must be unique.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.ExecuteReader()
at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.Execute(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues)
at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.ExecuteReader(IRelationalConnection connection, IReadOnlyDictionary`2 parameterValues)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.BufferlessMoveNext(DbContext _, Boolean buffer)
at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
at Microsoft.EntityFrameworkCore.Query.QueryMethodProvider._InjectParameters[TElement](QueryContext queryContext, IEnumerable`1 source, String[] parameterNames, Object[] parameterValues)+MoveNext()
at Microsoft.EntityFrameworkCore.Query.QueryMethodProvider._GroupBy[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)+MoveNext()
at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source, Int32& length)
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.OrderedEnumerable`1.GetEnumerator()+MoveNext()
at System.Linq.Enumerable.SelectIPartitionIterator`2.MoveNext()
at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at lambda_method(Closure , TransparentIdentifier`2 )
at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext()
ClientConnectionId:f68441e8-baa8-4c16-aa66-0cae875ba173
Error Number:169,State:1,Class:15
Exception thrown: 'System.Data.SqlClient.SqlException' in Microsoft.EntityFrameworkCore.dll
[17:00:55 INF] Executed action GCO.App.Controllers.MareysChart.MareysChartController.GetInitialData (GCO.App) in 331.5571ms
[17:00:55 INF] Executed endpoint 'GCO.App.Controllers.MareysChart.MareysChartController.GetInitialData (GCO.App)'
Exception thrown: 'System.Data.SqlClient.SqlException' in System.Private.CoreLib.dll
[17:00:55 ERR] An unhandled exception has occurred while executing the request.
System.Data.SqlClient.SqlException (0x80131904): A column has been specified more than once in the order by list. Columns in the order by list must be unique.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.ExecuteReader()
at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.Execute(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues)
at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.ExecuteReader(IRelationalConnection connection, IReadOnlyDictionary`2 parameterValues)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.BufferlessMoveNext(DbContext _, Boolean buffer)
at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
at Microsoft.EntityFrameworkCore.Query.QueryMethodProvider._InjectParameters[TElement](QueryContext queryContext, IEnumerable`1 source, String[] parameterNames, Object[] parameterValues)+MoveNext()
at Microsoft.EntityFrameworkCore.Query.QueryMethodProvider._GroupBy[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)+MoveNext()
at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source, Int32& length)
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.OrderedEnumerable`1.GetEnumerator()+MoveNext()
at System.Linq.Enumerable.SelectIPartitionIterator`2.MoveNext()
at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at lambda_method(Closure , TransparentIdentifier`2 )
at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext()
at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at GCO.App.MareysChart.Services.TrainService.GetTrains(DateTime start, DateTime end, List`1 trackCircuitIds) in C:\Projects\VSTS\GCO_server-dotnet\GCO\GCO\MareysChart\Services\TrainService.cs:line 66
at GCO.App.Controllers.MareysChart.MareysChartController.GetInitialData(Int64 tabId, String dateUtcString) in C:\Projects\VSTS\GCO_server-dotnet\GCO\GCO\Controllers\MareysChart\MareysChartController.cs:line 269
at Microsoft.AspNetCore.Mvc.Internal.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
at System.Threading.Tasks.ValueTask`1.get_Result()
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeActionMethodAsync()
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeNextActionFilterAsync()
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow(ActionExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(HttpContext httpContext)
at GCO.App.Middlewares.UserInfoMiddleware.InvokeAsync(HttpContext httpContext, GcoContext dbContext, ILuwakUserInfoAccessor userInfoAccessor) in C:\Projects\VSTS\GCO_server-dotnet\GCO\GCO\Middlewares\UserInfoMiddleware.cs:line 94
at Microsoft.AspNetCore.Cors.Infrastructure.CorsMiddleware.InvokeCore(HttpContext context)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
ClientConnectionId:f68441e8-baa8-4c16-aa66-0cae875ba173
Error Number:169,State:1,Class:15
[17:00:55 INF] Request finished in 556.6788ms 500 text/html; charset=utf-8
If I remove the GroupBy altogether it works (but I have duplicated data).
Am I doing something wrong? Why is it generating duplicate order by's? Why is it generating Order By at all? Does Group By translates to an order by?
More strange, this happened to a collegue. In my database this was working just fine. I asked her to make a dump for me to debug. In both cases we have no data on the TrainMovementPlanned table...