How do I execute RAW SQL query and dump it to JSON in ASP.NET MVC 5 (EF 6)? - sql

I want to execute a raw SQL query as shown below:
select material, method, count(*)
from treatment
group by material, method
and return it as a JSON object.
Because EF 6 allows the execution of raw query (i.e., Database.SQLQuery()), I use it to create a JSON Object.
My code looks like this.
// GET: Dummy
public string Dummy()
{
string query = "select material, method, count(*) as cnt from Treatment "
+ " group by material, method";
var result = db.Database.SqlQuery<List<String>>(query);
var jsonResult = JsonConvert.SerializeObject(result);
return jsonResult;
}
However, instead of getting a JSON object with the material and methods, I get an empty json object instead.
[[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]]
Is it possible to return the correct JSON object without having the model for the the raw query? If so, how to update my code to return the correct result?
Thanks in advance

You can't resolve that query to a list of strings, you need to resolve it to a (list of) class(es), like this :
public static string Dummy()
{
using (var db = new TestDbEntities())
{
string query = "select 'material' as material , 'method' as method, 1 as cnt ";
var result = db.Database.SqlQuery<MyClass>(query);
var res = result.ToList();
var jsonResult = JsonConvert.SerializeObject(res);
return jsonResult;
}
}
where your class will be something like this :
public class MyClass
{
public string Material { get; set; }
public string Method { get; set; }
public int Cnt { get; set; }
}

You need to capture query result sets with multiple columns using a class with matching property names (make sure they have exactly same name and proper casing) and parameterless constructor:
public string Dummy()
{
string query = "select material as Material, method as Method, count(*) as Cnt from Treatment group by material, method";
var result = db.Database.SqlQuery<ResultSet>(query).ToList();
var jsonResult = JsonConvert.SerializeObject(result);
return jsonResult;
}
public class ResultSet
{
public string Material { get; set; }
public string Method { get; set; }
public int Cnt { get; set; }
}
Note that Database.SqlQuery<T> will return an IEnumerable<T> with type System.Data.Entity.Internal.InternalSqlQuery<T>, so that a List<string> is not fit in this context. If you just return single result set, Database.SqlQuery<string>(...).ToList() should be used instead of Database.SqlQuery<List<string>>.
Similar issue:
Entity framework raw SQL Query

Use FOR JSON AUTO:
string query = "select material, method, count(*) as cnt from Treatment "
+ " group by material, method FOR JSON AUTO";

Related

Using FromSqlRaw or Linq query to Group and Sum data

I am trying to use a query with GroupBy and Sum. First I tried it with SQL:
string query = $"SELECT Year(Datum) AS y, Month(Datum) AS m, SUM(Bedrag) AS Total FROM Facturens GROUP BY Year(Datum), Month(Datum) ORDER BY y, m";
Grafiek = await _db.Facturens.FromSqlRaw(query).ToListAsync();
I get this error:
"InvalidOperationException: The required column 'FacturenID' was not present in the results of a 'FromSql' operation." "FacturenID" is the first column in the Facturens table.
The SQL query works fine when used directly.
I then tried Linq:
Grafiek = (IEnumerable<Factuur>)await _db.Facturens
.GroupBy(a => new { a.Datum.Value.Year, a.Datum.Value.Month }, (key, group) => new
{
jaar = key.Year,
maand = key.Month,
Total = group.Sum(b => b.Bedrag)
})
.Select(c => new { c.jaar, c.maand, c.Total })
.ToListAsync();
This results in error: "InvalidOperationException: Nullable object must have a value."
Factuur:
using System.ComponentModel.DataAnnotations;
namespace StallingRazor.Model
{
public class Factuur
{
[Key]
public int FacturenID { get; set; }
public int EigenarenID { get; set; }
[Display(Name = "Factuurdatum")]
[DataType(DataType.Date)]
[DisplayFormat(NullDisplayText = "")]
public DateTime? Datum { get; set; }
public decimal? Bedrag { get; set; }
public decimal? BTW { get; set; }
[DataType(DataType.Date)]
[DisplayFormat(NullDisplayText = "")]
public DateTime? Betaaldatum { get; set; }
[Display(Name = "Betaald bedrag")]
public decimal? Betaald_bedrag { get; set; }
[Display(Name = "Totaal bedrag")]
public decimal? Totaal_bedrag { get; set; }
public int ObjectenID { get; set; }
[DataType(DataType.Date)]
public DateTime? Verzonden { get; set; }
public string? Mededeling { get; set; }
[Display(Name = "Begindatum")]
[DataType(DataType.Date)]
[DisplayFormat(DataFormatString = "{dd-MM-yyyy}", NullDisplayText = "")]
public DateTime? Begin_datum { get; set; }
[Display(Name = "Einddatum")]
[DataType(DataType.Date)]
[DisplayFormat(NullDisplayText = "")]
public DateTime? Eind_datum { get; set; }
}
}
When performing aggregate queries against a model using SQL, the result will not and in general cannot easily be the same structural form as the original model, the Set<T>.FromSqlRaw() method that you are using requires the SQL to resolve ALL of the properties for the specified type of T
FromSqlRaw Limitations
The SQL query must return data for all properties of the entity type.
The column names in the result set must match the column names that properties are mapped to. Note this behavior is different from EF6. EF6 ignored property to column mapping for raw SQL queries and result set column names had to match the property names.
The SQL query can't contain related data. However, in many cases you can compose on top of the query using the Include operator to return related data (see Including related data).
For aggregate queries, we would generally define a new type to hold the response from the SQL aggregate. In C# LINQ GroupBy behaves very differently to SQL, in SQL the detail rows are excluded and only the aggregate set is returned. In LINQ all of the rows are retained, but they are projected into groups by the key, there is no specific aggregation at all, after a LINQ groupby you would them perform any aggregate analysis you may require.
The first thing we need to do is define the structure of the response, something like this:
public class FactuurSamenvatting
{
public int? Jaar { get; set; }
public int? Maand { get; set; }
public int? Total { get; set; }
}
Then if this type is registered with the DBContext as a new DbSet:
/// <summary>Summary of Invoice Totals by Month</summary>
public Set<FactuurSamenvatting> FacturenOmmen { get;set; }
You can then use this raw SQL query:
string query = $"SELECT Year(Datum) AS Jaar, Month(Datum) AS Maand, SUM(Bedrag) AS Total FROM Facturens GROUP BY Year(Datum), Month(Datum) ORDER BY Jaar, Maand";
var grafiek = await _db.FacturenOmmen.FromSqlRaw(query).ToListAsync();
Ad-Hoc Generic Solution
Though the above solution is encouraged, it is possible to achieve the same thing without formally adding your aggregate type directly to your DbContext. Following this advice from #ErikEj and his updated reference on Github we can create a dynamic context that explicitly contains the setup for any generic type
public static class SqlQueryExtensions
{
public static IList<T> SqlQuery<T>(this DbContext db, string sql, params object[] parameters) where T : class
{
using (var db2 = new ContextForQueryType<T>(db.Database.GetDbConnection()))
{
return db2.Set<T>().FromSqlRaw(sql, parameters).ToList();
}
}
private class ContextForQueryType<T> : DbContext where T : class
{
private readonly DbConnection connection;
public ContextForQueryType(DbConnection connection)
{
this.connection = connection;
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer(connection, options => options.EnableRetryOnFailure());
base.OnConfiguring(optionsBuilder);
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<T>().HasNoKey();
base.OnModelCreating(modelBuilder);
}
}
}
Now we do not need to pre-register the aggregate types at all, you can simply use this syntax to execute you query:
You can then use this raw SQL query:
string query = $"SELECT Year(Datum) AS Jaar, Month(Datum) AS Maand, SUM(Bedrag) AS Total FROM Facturens GROUP BY Year(Datum), Month(Datum) ORDER BY Jaar, Maand";
var grafiek = _db.SqlQuery<FactuurSamenvatting>(query).ToList();
Original Response
Updated after Factuur model posted
Below is a general walk through responding to the original post and the specific exceptions that were raised. I had originally assumed that OP was using an aggregate type definition, I had forgotten that to do so is itself an advanced technique, the following response is still helpful if you define your aggregate type correctly but still observe the same exceptions.
LINQ expressions in general that project into a known type will throw two common errors:
InvalidOperationException: The required column 'FacturenID' was not present...
This error is reasonably obvious, the model Factuur that you are projecting into has a required column called FacturenID, which your output does not provide. Your projection in the first attempt is expecting these columns in Factuur:
public int y { get;set; }
public int m { get;set; }
public int? Total { get;set; }
If you change the first query to use the matching property names of those existing in Factuur then you will most likekly still encounter the next issue...
The error InvalidOperationException: Nullable object must have a value. is experienced in two situations:
When your LINQ expression is operating in memory and tries to access a property on an object that is null, most likely in the case of the second query this can occur if any values of Datum are null, that would invalidate Datum.Value.
this syntax is allowed even if the field is null if the expression is being evaluated in SQL, the result will simply be null.
When a SQL result is projected into a c# type, when a value in one of the columns in the result set is null but the corresponding property of the type you are projecting into does not allow for nulls.
In this case one of the jaar,maand,Total columns needs to be null, usually it will be the result of the SUM aggregate but in this case that can only happen if Bedrag is nullable in your dataset.
Test your data by inspecting this recordset, notice that I am NOT casting the results to a specific type, we will leave them in the anonymous type form for this analysis, also we will exclude null datums. for this test.
var data = await _db.Facturens
.Where (f => f.Datum != null)
.GroupBy(a => new { a.Datum.Value.Year, a.Datum.Value.Month }, (key, group) => new
{
jaar = key.Year,
maand = key.Month,
Total = group.Sum(b => b.Bedrag)
})
.Select(c => new { c.jaar, c.maand, c.Total })
.ToListAsync();
In your original query, to account for the nulls and return zero for the Total instead of altering your model to accept nulls, then you could use this:
string query = $"SELECT Year(Datum) AS jaar, Month(Datum) AS maand, SUM(ISNULL(Bedrag,0)) AS Total FROM Facturens GROUP BY Year(Datum), Month(Datum) ORDER BY jaar, maand";
Grafiek = await _db.Facturens.FromSqlRaw(query).ToListAsync();
In this SQL we didn't need to exclude the null datums, these will be returned with respctive values of null for both of jaar and maand
Given that the only case where jaar and maand might be null is if the column Datum has a null value so you could use this SQL to return the same columns as the expected type without modifying the model, as long as these were all the columns in the model. In this case I would recommend excluding those records from the results with a simple WHERE clause
SELECT
Year(Datum) AS jaar
, Month(Datum) AS maand
, SUM(ISNULL(Bedrag,0)) AS Total
FROM Facturens
WHERE Datum IS NOT NULL
GROUP BY Year(Datum), Month(Datum) ORDER BY jaar, maand

How to filter data in model in ActionResult of controller in ASP.net core MVC?

I have an index.chtml set up with about 10 ActionLinks. Those actionLinks trigger different ActionResult functions within the controller since each of them essentially perform unique queries on a data model.
I also have an entities object named db which has all the data. Instead of just displaying all the data, I want to perform complex filtering on the entities object to find where certain properties of records are null or where a property is greater than some input then returns a view of all columns on only those records that were filtered.
Find nulls:
public class printJobsController : Controller {
private PrintJobsEntities db = new PrintJobsEntities
public ActionResult IncompleteJobs {
//get jobs where processDate is null
...
}
}
Find where count is greater than 10:
public class printJobsController : Controller {
private PrintJobsEntities db = new PrintJobsEntities
public ActionResult JobsGreaterThan(int limit) {
//group by printerName and find counts greater than limit
...
}
}
How do I go about doing this?
Seems you are trying to populate the View with filtered data as per your request parameter in controller action.
You could follow the below steps to achieve what you are trying to:
Your imaginary Data Model
public class PrinterJob
{
[Key]
public int PrinterId { get; set; }
public string PrinterName { get; set; }
public int PrintedBy { get; set; }
public int TotalPrint { get; set; }
}
Sample Data In Database:
Controller Action:
public ActionResult <PrinterJob> JobsGreaterThan(int limit) {
var printCountByGroup =
(from objPrint in _context.PrinterJobs group objPrint by new {
objPrint.PrinterName, objPrint.PrintedBy, objPrint.TotalPrint
}
into grp where grp.Sum(p => p.TotalPrint) > limit
select new {
PrinterName = grp.Key.PrinterName, PrintedBy = grp.Key.PrintedBy,
TotalPrint = grp.Key.TotalPrint
});
return View(printCountByGroup);
}
Output After Applying Filter:
Note: Here I am trying to filter printer information which printed more then 30 printing jobs.
Hope it would help you to achieve your goal. If you still have any problem feel free to let me know.

How i can get a single property insie my async code

I have the following model class-
public partial class Settings
{
public int Id { get; set; }
public string Value { get; set; }
public string Name { get; set; }
}
Inside my asp.net core MVC 3.1, i want to only get the Value of a single item, now i can NOT do this:-
var result = await _context.Settings.SingleOrDefaultAsync(a => a.Name.ToLower() == "noofvisits").Value;
and the only way is to first get the whole object from the database and then get its Value, as follow:-
var result = await _context.Settings.SingleOrDefaultAsync(a => a.Name.ToLower() == "noofvisits");
var val = result.Value;
but to make my code more efficient how i can directly get the Value property from the database asynchronously ?
You should use Raw Sql query.
Try this :
var result = await _context.Settings.FromSql("SELECT Top(1) NAME FROM dbo.Settings ").SingleOrDefaultAsync();
Try the below statement:
var result = await _context.Settings.Where(a => a.Name.ToLower() == "noofvisits").Select(o => o.Value).FirstOrDefaultAsync();

JSON Arrays for multiple values in ASP.NET API

I've built a simple API for querying a local SQL database (for a database course). I'm using Postman to test my endpoints, but the results I'm getting aren't formatted the way I'd like. For example, in this query I ask my database for data about 1 person but it's returning all the unique sets.
What I would like is something like this (multivalues in a list):
My API is calling a stored procedure that's pretty lengthy, but this is the select statement at the end (a lengthy inner join):
SELECT DISTINCT specializationType, memberSince, teamName, E.meetingsAttended, lifetimeScore
FROM participants AS A
INNER JOIN
ONTEAM AS B
ON A.participantKey = B.participantKey
INNER JOIN
MEMBERSHIP AS C
ON A.participantKey = C.participantKey
INNER JOIN
SPECIALIZATIONPART AS D
ON A.participantKey = D.participantKey
INNER JOIN
MEETINGCOUNT as E
ON A.participantKey = E.participantKey;
The endpoint that's calling this is:
// GET api/InfoSecDB/adminSelectsParticipant
[HttpGet]
[Route("adminSelectsParticipant")]
public ActionResult<IEnumerable<string>> adminSelectsParticipant([FromBody] JObject data)
{
string pName = (string)data["pName"];
List<string> distinctSpecializations = new List<string>();
List<string> allTeams = new List<string>();
List<string> myP = new List<string>();
DatabaseModel dbm = new DatabaseModel();
DataTable dt = dbm.adminSelectsParticipant(pName);
return Ok(dt);
}
I'm not sure if this something that's supposed to be done by the stored procedure or the API endpoint. Any suggestions would be greatly appreciated!
Something you may want to do for this is set up a Model with each of the fields that you want to display. You can then loop through your DataTable and assign it to your model, then return your model in the way you want with your second image above.
Here is something that I found, it should be similar to what you want:
How do I bind a datatable to model and then to a dropdownlist in mvc?
Your model would look something like this:
public class Model
{
public List<string> SpecializationType { get; set; }
public DateTime MemberSince { get; set; }
public List<string> TeamName { get; set; }
public int MeetingsAttended { get; set; }
public int LifetimeScore { get; set; }
}
Then you could do var model = new List(); and assign your DataTable values to it.

Can I dynamically reference multiple databases in a LINQPad script?

I have a database with a table describing multiple entities, with one column being the name of another database holding data for that entity. All entity databases are on the same SQL Server as the one listing them and all have an identical schema.
I know that I can use Ctrl-drag to add the additional databases to my script but what I actually want is to do this dynamically from the database name. Something like this.
var entities = ParentDatabase.EntityList
.Where(e => ??)
.Select(e => new { e.Id, e.DatabaseName });
var results = new List<ResultCarrier>();
foreach (var entity in entities)
{
results.AddRange(
GetDataContextFor(entity.DatabaseName).SomeTable
.Select(t => new ResultCarrier()
{
EntityId = e.Id,
Column1 = t.Column1,
Column2 = t.Column2,
...
}));
}
// further process combined results
Is this possible?
I see that the type of one of these databases is LINQPad.User.DatabaseNameTypes.TypedDataContext and wondered whether, as each database has the same schema, there might be a base class that I could use in some way to achieve this.
TypedDataContext is your base class, and you can just create a new instance of this and pass it the sql connection string.
You can find your current connection string using
this.Connection.ConnectionString.Dump();
For example, I use Integrated Security and I have a little routine that goes through all the database in my server and dumps out a table, so I use the following routine.
var databases = ExecuteQuery<String>("SELECT name FROM sys.databases").ToList();
foreach(var r in databases)
{
switch (r)
{
case "master" :
case "tempdb" :
case "model" :
case "msdb" :
break;
default:
try
{
string newConnectionString = String.Format("Data Source={0};Integrated Security=SSPI;Initial Catalog={1};app=LINQPad", this.Connection.DataSource, r);
var dc = new TypedDataContext(newConnectionString);
dc.Table.Dump(r);
}
catch (Exception ex)
{
ex.Message.Dump(r);
}
break;
}
}
#sgmoore's answer got me on the right track. I hadn't come across ExecuteQuery in LINQPad and I was able to use it to achieve what I wanted. Below is the code I ended up with. I will now extend it to further retrieve data from a service and join it to databaseLocations to give the final result I'm after.
void Main()
{
var organisations = ExecuteQuery<OrganisationCarrier>(#"
SELECT do.GroupId [Id], o.sOrganisationName [Name], o.sConnectDatabase [Database]
FROM dbo.Organisation o
INNER JOIN dynamix.Organisations do ON o.liOrgID = do.OrganisationID
INNER JOIN dynamix.OrganisationFeatures oft ON do.OrganisationKey = oft.OrganisationKey
INNER JOIN dynamix.Features ft ON oft.FeatureKey = ft.FeatureKey
WHERE ft.FeatureName = 'LightningLocations'").ToList();
var databaseLocations = new List<LocationExtract>();
foreach (var organisation in organisations)
{
this.Connection.ConnectionString = $"Data Source={this.Connection.DataSource};Integrated Security=SSPI;Initial Catalog={organisation.Database};app=LINQPad";
databaseLocations.AddRange(ExecuteQuery<LocationCarrier>(#"
SELECT dml.DmxLocationId [Id], ml.sLocationName [Name], ml.bDeleted [IsDeleted]
FROM dynamix.MapLocations dml
INNER JOIN dbo.MapLocations ml ON dml.FmLocationId = ml.liLocationID")
.Select(l => new LocationExtract(organisation.Id, l.Id, l.Name, l.IsDeleted)));
}
databaseLocations.Dump();
}
class OrganisationCarrier
{
public long Id { get; set; }
public string Name { get; set; }
public string Database { get; set; }
}
class LocationCarrier
{
public long Id { get; set; }
public string Name { get; set; }
public bool IsDeleted { get; set; }
}
class LocationExtract
{
public long OrganisationId { get; }
public long LocationId { get; }
public string Name { get; }
public bool IsDeleted { get; }
public LocationExtract(long organisationId, long locationId, string name, bool isDeleted)
{
OrganisationId = organisationId;
LocationId = locationId;
Name = name;
IsDeleted = isDeleted;
}
}