DTO design considerations for nested collections - api

Given that I have a model denoting a city which holds a collection of streets.
public class City {
public int Id { get; set; }
public string Name { get; set; }
public IEnumerable<Street> Streets { get; }
}
public class Street {
public int Id { get; }
public string Name { get; }
public IEnumerable<Building> Buildings { get; }
}
If a client is interested in all cities (api/cities/all) and I'd return him the full collection, this would lead to a massive response, depending on the size of the dataset. So I thought first about returning only street ids inside the streets collection. This however feels awkward because while the ids might be useful to further fetch the streets, they hold no meaningful value for a client (it doesn't make sense to populate a list of ids on a view in order to show what streets are in a city, doesn't it?).
My next idea was to ditch the streets collection completely and instead offer an API endpoint to fetch the streets of a city:
api/cities/3737/streets
That way I can fetch a complete list of streets, however the returned data then doesn't contain any information to where the streets belong to. If I a client now wants to show both the streets and the city, he'd have to make 2 API calls to get the information he needs.
What is a common way to return such data?

I would create 2 city objects, a city object containing only the basic data(like ID & Name) and a cityDetail object containing all the data (including the full street plan).
Depending on your situation you might repeat this pattern for the buildings.
You can then return the city object in your list call and only if a get by id is performed you would return the cityDetail object.
The bad => your api is sorta "inconsistent" as the city objects you get via a list are not the same as the ones you get via the get. You could make 2 resources: city & cityDetail with cityDetail not having a List function and city not having a getByID function but still it's not as clean & predictable as a single resource.
The good => Performance wise & usage wise this usually is a perfect match. You will never show a list of cities with streets so the city object would be sufficient. When you are viewing a single city it is very likely you want to show all data including streets(or at least use this data on that page) so the cityDetail object would fit here as well.
Performance wise it's kinda obvious that fetching 1 milion streets for a getAllMyCities call is overkill ;)

Related

Need help deserializing JSON data from an external Web API and storing the data in a SQL database

So, I'm working with JSON for the first time within ASP.net.
Apologies, should have specified, language being worked on is c# within ASP.Net.
I currently have the following:
private static async void UpdateStreetWebApiProperties()
{
var client = new HttpClient();
var request = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri("https://inventorymanchestertest.co.uk/api/property-feed/sales/search"),
Headers =
{
{ "ContentType", "application/json" },
{ "Authorization", "Bearer API_Key" },
},
};
using (var response = await client.SendAsync(request))
{
response.EnsureSuccessStatusCode();
var body = await response.Content.ReadAsStringAsync();
if (response.IsSuccessStatusCode)
{
System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
StreetWebApi_GetProperties streetwebapi_getproperties = new StreetWebApi_GetProperties();
}
else
{
Console.WriteLine("Could not get properties");
}
Console.WriteLine();
}
}
The variable body successfully draws down the JSON data in it's complete form. However, I'm looking to seperate each part into it's respective columns in a SQL DB.
The JSON looks like this:
{"data":[{"type":"property","id":"a91ab45e-5db8-4486-9bdf-f38dcb63c400","attributes":{"branch_uuid":"3e7a4a68-ab41-46c3-9a48-e3d1635cd056","inline_address":"101 London Road, Peterborough","public_address":"London Road, Peterborough, PE2","postcode":"PE2 9DD","bedrooms":5,"bathrooms":2,"receptions":2,"floor_area":null,"plot_area":null,"land_area":null,"property_type":"Detached House","property_age_bracket":null,"construction_year":null,"status":"For Sale","sale_status":"For Sale","lettings_status":null,"owner_label":"Vendor","tenure":null,"tenure_notes":null,"lease_expiry_year":null,"lease_expiry_date":null,"public_url":"https:\/\/inventorymanchester.co.uk\/platform\/properties\/a91ab45e-5db8-4486-9bdf-f38dcb63c400","created_at":"2022-06-17T15:18:53+01:00","updated_at":"2022-07-12T11:23:11+01:00","custom_meta_data":[],"property_urls":[],"viewing_booking_url":"https:\/\/inventorymanchester.co.uk\/platform\/properties\/a91ab45e-5db8-4486-9bdf-f38dcb63c400\/book-viewing"},"relationships":{"address":{"data":{"type":"address","id":"433518e4-d544-42ce-aba4-7d1137465af1"}},"details":{"data":{"type":"details","id":"1bf2b0fc-36c1-40f1-9e04-5b5cf72ffd0c"}},"salesListing":{"data":{"type":"sales_listing","id":"992114a6-3fcf-48b1-af1d-f5f3976a23da"}},"lettingsListing":{"data":null},"primaryImage":{"data":{"type":"media","id":"9ed40865-0873-4159-808b-5941faa520c9"}}}},{"type":"property","id":"4fd57964-71ea-4a77-b773-b4079a0f95dc","attributes":{"branch_uuid":"3e7a4a68-ab41-46c3-9a48-e3d1635cd056","inline_address":"4 Riverside Mead, Peterborough","public_address":"Riverside Mead, Peterborough, PE2","postcode":"PE2 8JN","bedrooms":4,"bathrooms":3,"receptions":2,"floor_area":null,"plot_area":null,"land_area":null,"property_type":"Detached House","property_age_bracket":null,"construction_year":null,"status":"Sold STC","sale_status":"Sold STC","lettings_status":null,"owner_label":"Vendor","tenure":null,"tenure_notes":null,"lease_expiry_year":null,"lease_expiry_date":null,"public_url":"https:\/\/inventorymanchester.co.uk\/platform\/properties\/4fd57964-71ea-4a77-b773-b4079a0f95dc","created_at":"2022-06-17T16:39:19+01:00","updated_at":"2022-07-19T11:39:26+01:00","custom_meta_data":[],"property_urls":[],"viewing_booking_url":"https:\/\/inventorymanchester.co.uk\/platform\/properties\/4fd57964-71ea-4a77-b773-b4079a0f95dc\/book-viewing"},"relationships":{"address":{"data":{"type":"address","id":"03d1a68a-6f4a-42ff-bf65-5b9768d6ce81"}},"details":{"data":{"type":"details","id":"f2b1a173-0611-4014-a980-894257b0bab0"}},"salesListing":{"data":{"type":"sales_listing","id":"be1cec3a-cf2f-40c4-a627-427cf3fbdfa7"}},"lettingsListing":{"data":null},"primaryImage":{"data":{"type":"media","id":"125542ce-27f1-4852-8fb6-b71daaaa70d1"}}}}],"included":[{"type":"address","id":"433518e4-d544-42ce-aba4-7d1137465af1","attributes":{"anon_address":"London Road, Peterborough, PE2","line_1":"101 London Road","line_2":"Peterborough","line_3":null,"town":"Peterborough","postcode":"PE2 9DD","inline":"101 London Road, Peterborough, PE2 9DD","longitude":-0.2465764,"latitude":52.560172}},{"type":"details","id":"1bf2b0fc-36c1-40f1-9e04-5b5cf72ffd0c","attributes":{"display_property_style":null,"work_required":null,"heating_system":null,"council_tax_band":null,"council_tax_cost":null,"local_authority":null,"service_charge":null,"service_charge_period":"month","service_charge_notes":null,"ground_rent":null,"ground_rent_period":"month","ground_rent_review_period_years":null,"ground_rent_uplift":null,"ground_rent_expiry":null,"full_description":"<p>Tortoise Property are pleased to offer this five bed detached house that is situated in the popular location of London Road, Fletton.<br><br>**Please call for either a viewing or virtual tour of this property.**<br><br>The property has a hallway, ground floor bathroom, bedroom, kitchen, dining room, lounge and converted garage on the ground floor. There are four bedrooms and the family bathroom on the first floor.<br><br>Outside the property has a front garden, a back garden and off-road parking for four cars.<br><\/p>","short_description":null,"location_summary":"London Road is a great location that is situated within walking distance of the city centre and local amenities. The Queensgate shopping centre is an 16 minute walk. The train station is a 21 minute walk or 5 minutes by car. The Kings secondary school is a 6 minute drive.\r\n\r\nPeterborough City Centre can be reached by car in 5 minutes and by bus in 10 minutes.\r\n\r\nThe A1 Junction can be reached by car in 11 minutes and the surrounding parkways give access to the A47 both east and west. \r\n\r\nWe love Fletton because of the lifestyle you can enjoy here. Great homes, close to nature and superb facilities make this one of our favourite places to live and work.","has_parking":null,"has_outdoor_space":null,"virtual_tour":null,"shared_ownership":false,"shared_ownership_notes":null,"shared_ownership_rent":null,"shared_ownership_rent_frequency":null,"shared_ownership_percentage_sold":null,"created_at":"2022-06-17T15:18:54+01:00","updated_at":"2022-06-17T15:28:08+01:00"}},{"type":"sales_listing","id":"992114a6-3fcf-48b1-af1d-f5f3976a23da","attributes":{"status":"For Sale","price":300000,"price_qualifier":"In Excess of","display_price":true,"archived":false,"is_low_profile":false,"occupancy_status":1,"new_home":false,"created_at":"2022-06-17T15:29:16+01:00","updated_at":"2022-06-17T15:29:57+01:00"}},{"type":"media","id":"9ed40865-0873-4159-808b-5941faa520c9","attributes":{"name":"136511_31517777_IMG_17_0000","order":0,"is_featured":true,"feature_index":1,"title":null,"is_image":true,"url":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844548\/136511_31517777_IMG_17_0000.jpeg","urls":{"thumbnail":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844548\/136511_31517777_IMG_17_0000.jpeg?tr=pr-true,n-property_thumb","small":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844548\/136511_31517777_IMG_17_0000.jpeg?tr=pr-true,n-property_small_fill_crop","medium":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844548\/136511_31517777_IMG_17_0000.jpeg?tr=pr-true,n-property_medium_fill_crop","large":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844548\/136511_31517777_IMG_17_0000.jpeg?tr=pr-true,n-property_large_fill_crop","hero":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844548\/136511_31517777_IMG_17_0000.jpeg?tr=pr-true,n-property_hero","full":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844548\/136511_31517777_IMG_17_0000.jpeg"}}},{"type":"address","id":"03d1a68a-6f4a-42ff-bf65-5b9768d6ce81","attributes":{"anon_address":"Riverside Mead, Peterborough, PE2","line_1":"4 Riverside Mead","line_2":"Peterborough","line_3":null,"town":"Peterborough","postcode":"PE2 8JN","inline":"4 Riverside Mead, Peterborough, PE2 8JN","longitude":-0.2305068,"latitude":52.5631968}},{"type":"details","id":"f2b1a173-0611-4014-a980-894257b0bab0","attributes":{"display_property_style":null,"work_required":null,"heating_system":null,"council_tax_band":null,"council_tax_cost":null,"local_authority":null,"service_charge":null,"service_charge_period":"month","service_charge_notes":null,"ground_rent":null,"ground_rent_period":"month","ground_rent_review_period_years":null,"ground_rent_uplift":null,"ground_rent_expiry":null,"full_description":"<p>Here at Tortoise Property, we pride ourselves on doing things differently, by offering a complete partnership and consistent approach to construct a comprehensive marketing package tailored for the single purpose of selling your property as agreed at the initial valuation.<br><br>\"Tortoise provided me with a comprehensive property management service over a four year period, offering a friendly, transparent and consistent relationship.<br><br>When I decided to sell my property I immediately engaged with Tortoise to undertake the action. They actively advertised my property, were proactive with local sale opportunities and provided timely updates on progress. The sale on my property was agreed, exchanged and completed within five weeks. I would highly recommend Tortoise Property for their professional and friendly approach.\" - Tracey Matthews - Testimonial <br><br>Valuations<br><br>We concentrate on the maximum price your house is likely to sell for then agree a sensible timeframe for which the property should be sold whilst clearly explaining how the fee you are charged, is invested in enabling us to find your buyer from across the country.<br><br>\"I recently used Tortoise to sell my house, Chris came round and went through everything there price was better than all others I had received and they seemed a lot more genuine.\" - James Richards - Testimonial <br><br>Relationship management<br><br>Our relationship managers are here to personally look after you offering complete transparency and guidance throughout the sales process, following a 12-week programme that provides regular viewings with prompt feedback.<br><br>\"They say selling your home can be very stressful, not with this team, there was never a time you could not get in touch with these guys. You will be in safe hands all the way from start to finish.\" - Maxine Ambrose - Testimonial <br><br>Facebook<br><br>The growth of our sales portfolio into the wider Peterborough area we believe is the result of our unique strategy to capture maximum exposure. Facebook provides us with the opportunity to target our property marketing and expand our reach beyond the property portals.<br><br>Facebook live<br><br>The potential reach of a digital tour is limitless. Our live feed property tours on facebook are great for potential buyers to not only view the property but to ask relevant questions and get instant replies from wherever they are based.<br><br>Online and traditional auctions<br><br>Our property auction service gives you the ability to sell your property at auction either online or at a live auction. The buyer pays a commission so your house is sold at no cost to you. The buyer must complete within 28 or 56 days meaning your property is sold fast.<br><br>Performance-related fees<br><br>Here at Tortoise we do offer traditional fee structures based on a standard percentage of the purchase price or a fixed fee. However, we are so good at what we do that we are confident enough to offer you performance related fees we believe we should win together.<br><br>24\/7 services<br><br>Property sales can be daunting, especially if it is your first time. Here at Tortoise, we have real people available to talk to 24 hours a day 7 days a week as well as a live web chat so that you can chat to someone at your convenience.<br><br><br><br><br><br>Negotiator awards<br><br>In 2017 Tortoise Property was shortlisted for website of the year in the negotiator awards competing with large national estate agency chains illustrating the quality and presentation of our brand and level of service.<br><br>Our micro-site offers plenty of information so please choose from one of the tabs on the left that is applicable to your requirements and we look forward to seeing you in the near future or to find out more about us and our services visit www.tortoise property.co.uk<br><\/p>","short_description":null,"location_summary":null,"has_parking":null,"has_outdoor_space":null,"virtual_tour":null,"shared_ownership":false,"shared_ownership_notes":null,"shared_ownership_rent":null,"shared_ownership_rent_frequency":null,"shared_ownership_percentage_sold":null,"created_at":"2022-06-17T16:39:20+01:00","updated_at":"2022-06-17T16:45:16+01:00"}},{"type":"sales_listing","id":"be1cec3a-cf2f-40c4-a627-427cf3fbdfa7","attributes":{"status":"Sold STC","price":350000,"price_qualifier":"Fixed Price","display_price":true,"archived":false,"is_low_profile":false,"occupancy_status":1,"new_home":false,"created_at":"2022-07-19T11:38:08+01:00","updated_at":"2022-07-19T11:39:26+01:00"}},{"type":"media","id":"125542ce-27f1-4852-8fb6-b71daaaa70d1","attributes":{"name":"136511_31519016_IMG_00_0000","order":0,"is_featured":true,"feature_index":1,"title":null,"is_image":true,"url":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844609\/136511_31519016_IMG_00_0000.jpeg","urls":{"thumbnail":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844609\/136511_31519016_IMG_00_0000.jpeg?tr=pr-true,n-property_thumb","small":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844609\/136511_31519016_IMG_00_0000.jpeg?tr=pr-true,n-property_small_fill_crop","medium":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844609\/136511_31519016_IMG_00_0000.jpeg?tr=pr-true,n-property_medium_fill_crop","large":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844609\/136511_31519016_IMG_00_0000.jpeg?tr=pr-true,n-property_large_fill_crop","hero":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844609\/136511_31519016_IMG_00_0000.jpeg?tr=pr-true,n-property_hero","full":"https:\/\/ik.imagekit.io\/street\/street-mobile\/properties\/general\/844609\/136511_31519016_IMG_00_0000.jpeg"}}}],"meta":{"pagination":{"total":2,"count":2,"per_page":250,"current_page":1,"total_pages":1}},"links":{"self":"https:\/\/inventorymanchester.co.uk\/api\/property-feed\/sales\/search?page%5Bnumber%5D=1","first":"https:\/\/inventorymanchester.co.uk\/api\/property-feed\/sales\/search?page%5Bnumber%5D=1","last":"https:\/\/inventorymanchester.co.uk\/api\/property-feed\/sales\/search?page%5Bnumber%5D=1"}}
I've used XML before and used XML readers to do the job but my understanding is that JSON is a very different way of working with data.
Could I please have some examples of ways I could deserialise the information and then some stored procedures to store them correctly as currently I've looked up several ways to acheive this but none have made too much sense.
Many thanks
-- Micro update --
I have setup the classes using paste special for JSON and am then running the following:
private static async void UpdateStreetWebApiProperties()
{
var client = new HttpClient();
var request = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri("https://inventorymanchestertest.co.uk/api/property-feed/sales/search"),
Headers =
{
{ "ContentType", "application/json" },
{ "Authorization", "Bearer auth_foo" },
},
};
using (var response = await client.SendAsync(request))
{
response.EnsureSuccessStatusCode();
var properties = await response.Content.ReadAsStringAsync();
if (response.IsSuccessStatusCode)
{
tortoise_common.JSONModel.Datum streetResponse = JsonConvert.DeserializeObject<tortoise_common.JSONModel.Datum>(properties);
{
Console.WriteLine(streetResponse);
}
}
else
{
Console.WriteLine("Could not get branch");
}
Console.WriteLine(properties);
}
}
However, upon running the code through the deserializer, the values assigned are coming through as "null". I would assume this would mean that the classes are not getting and setting any values.
Not quite sure on why the serializer is returning no values despite the "properties" variable having the full JSON string within.
Ok, so you could do this one of two ways:
use netonsoft - parse the jason.
(just like we do with xml - really a VERY simular approach).
Or, build a class that represents the XML, or json data.
So, in some cases, its easier to pluck out the values of the json string (using parsers and attribute selectors). A few loops.
So, MORE important, what is that json data?
Is it
Simple one reocrd, say FirstName, LastName, City?
or, is the data REPEATING data - a complex master customer invoice form with say one record at the top (customer information), and then repeating data of invoice details?
So, lets take a quick look at the data, see what we get.
Add a new blank class to y our project - ctrl-A - del key (clear out the class).
Now, we grab your sample json (copy).
Now, back to the empty class in Visual Studio, and then this:
when we do this, the base (bottom root) class tends to get a default name of RootObject, and you no doubt been code and slicing and dicing json data all day.
So, we either change that name, but better yet, lets just toss around the whole mess a namespace.
Well, VS now cranks out this:
Namespace MyDataTest
Public Class Rootobject
Public Property data() As Datum
Public Property included() As Included
Public Property meta As Meta
Public Property links As Links
End Class
Public Class Meta
Public Property pagination As Pagination
End Class
Public Class Pagination
Public Property total As Integer
Public Property count As Integer
Public Property per_page As Integer
Public Property current_page As Integer
Public Property total_pages As Integer
End Class
Public Class Links
Public Property self As String
Public Property first As String
Public Property last As String
End Class
Public Class Attributes
Public Property branch_uuid As String
Public Property inline_address As String
Public Property public_address As String
Public Property postcode As String
Public Property bedrooms As Integer
Public Property bathrooms As Integer
Public Property receptions As Integer
Public Property floor_area As Object
Public Property plot_area As Object
Public Property land_area As Object
Public Property property_type As String
Public Property property_age_bracket As Object
Public Property construction_year As Object
Public Property status As String
Public Property sale_status As String
Public Property lettings_status As Object
Public Property owner_label As String
Public Property tenure As Object
Public Property tenure_notes As Object
Public Property lease_expiry_year As Object
Public Property lease_expiry_date As Object
Public Property public_url As String
Public Property created_at As Date
Public Property updated_at As Date
Public Property custom_meta_data() As Object
Public Property property_urls() As Object
Public Property viewing_booking_url As String
End Class
(and more - it too rude to post more code - stop here)
End Class
End Namespace
Hum, not a super simple structure, and it does show/have some "repeating" data.
(and with repeating data, we care because then that SIGNIFICALY increases the difficult in taking that data to sql server.
One of the interview type of questions that google, Microsoft ask you?
They are questions of how high, how far, how big.
In other words, they want developers that have a sense of "scope".
I mean, you can walk with ease to a store down the block. But, if it is 5 miles away, then you really starting to solve that issue by adopting some form of motorized transportation - its too far to walk (so, how far, how big, how high type of question).
Ok, so we generated the class. lets look at it.
Right click -> view class diagram.
We get this:
Hum, ok, lets expand a few of the objects now:
Ok, I'm going for coffee. So, at least we asked the how high, how far, and how big question then right?
You not just adding a few simple rows to some database.
You asking me to come to your house, clear out the garage, then cook you dinner, and then clean up the kitchen afterwards.
But, as least you can see how Visual Studio has some built in tools, and some things that let you get a quick and easy feel for what you up against.
Looking at above? I think you probably take the class road, and let Visual Studio create the class for you. You can then send that string to newtonsoft, and it will then peal it out like layers of a onion to the above class. (and its only 4 lines of code to do this).
At that point, then you can start to take that data, and send it to SQL server. but, it not just a simple User name and address here, but a grouping of multiple tables and data - all of which would have to be added to a database, and no doubt highly relational database that has all that information.
If you have that existing database schema? Then hey, this is not too bad then.
but, do the fields and columns and existing database you have match the json data?
So, then we ask another how high how far, how big question:
If the columns and database structure you have now does NOT match the json data, then we not really adding rows to a database, but are in fact doing a type of data migration - and now we have to introduce "mapping" for what amounts to 100+ columns or more. So, that's what I have that 2nd question:
Do you have a matching database structure (schema) now that follows the field names used in this json data ? (are field names 100% exact???).
Since, if the columns don't match, and that database you already have does not match the incoming data? Then you just increased the workload here by significant amounts - we now not adding rows, but having to translate from one data schema to another - and that going to cost you even more time and efforts here.
Edit2: parse out using test data
Ok, so I don't have the web service, so for this I'll just paste the sample data into a notepad.txt and read it. No problem.
So, we take the above data - use the paste speical->json.
I changed the name space around the class - due to having other test json objects. But, no changes - just a extra name space.
So, the start of the class looks like this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
namespace jdat
{
public class Rootobject
{
public Datum[] data { get; set; }
public Included[] included { get; set; }
public Meta meta { get; set; }
public Links links { get; set; }
}
public class Meta
{
public Pagination pagination { get; set; }
}
public class Pagination
{
public int total { get; set; }
public int count { get; set; }
public int per_page { get; set; }
etc. etc. etc.
Ok, so now our code to test this:
drop a button on a web form, and this code:
protected void cmdPARSE_Click(object sender, EventArgs e)
{
// don't have web service - paste test data into notepage
string strBufJSON = File.ReadAllText(#"c:\test7\jdata.txt");
// pretend above is web service results
jdat.Rootobject jData;
jData = JsonConvert.DeserializeObject<jdat.Rootobject>(strBufJSON);
// now display some values
Debug.Print(jData.data[0].attributes.public_url);
Debug.Print(jData.data[0].attributes.property_type);
Debug.Print(jData.data[0].attributes.sale_status);
}
So a few things:
Try tagging the language - I am fluent in vb/c#, but will post as vb.net if lanaguage not speicfed.
So, since the data is a "array", then you have to either loop over all data tiems, or just pull out [0] as above.
Intel sense still works shows all items/properties of the class - VERY nice and helpful.

EF Core generates too many queries for nested data

I have a simple class to represent a tree structure, defined like this:
public class LicenceCategory
{
[Key]
[Column("LicenceCategoryID")]
public Guid ID { get; set; }
public string Name { get; set; }
public Guid? ParentLicenceCategoryID { get; set; }
[ForeignKey("ParentLicenceCategoryID")]
public virtual List<LicenceCategory> Categories { get; set; }
}
Then, from an ASP.NET Core controller, I simply return myContext.LicenceCategory, which is about as simple as it gets.
Right now, there are five records on the database: the parent (null ParentLicenceCategoryID), and four children for that one parent. So no massive volumes and no very deep nesting. This is the SQL that gets generated, and is as I expect:
SELECT [obj].[LicenceCategoryID], [obj].[Name], [obj].[ParentLicenceCategoryID]
FROM [LicenceCategory] AS [obj]
However, it also generates this, five times:
SELECT [e].[LicenceCategoryID], [e].[Name], [e].[ParentLicenceCategoryID]
FROM [LicenceCategory] AS [e]
WHERE [e].[ParentLicenceCategoryID] = #__get_Item_0
Notice how the first statement already contains every field you need to build the tree structure client-side. Why on earth even do the extra select statements?
I noticed that if I Include navigation properties, things get much worse: For three navigation properties, I wound up with 21 select statements! Most of which are just the same statement executed again and again and again. It may do so with different parameters perhaps, but there is hardly any way to make a program any less efficient. And these are five records - what will EF do when I throw our millions of transaction records its way?
Is there a way to prevent this kind of code generation, or is EF Core simply a non-starter?

nhibernate - sproutcore : How to only retrieve reference ID's and not load the reference/relation?

I use as a front-end sproutcore, and as back-end an nhibernate driven openrasta REST solution.
In sproutcore, references are actualy ID's / guid's. So an Address entity in the Sproutcore model could be:
// sproutcore code
App.Address = App.Base.extend(
street: SC.Record.attr(String, { defaultValue: "" }),
houseNumber: SC.Record.attr(String),
city: SC.Record.toOne('Funda.City')
);
with test data:
Funda.Address.FIXTURES = [
{ guid: "1",
street: "MyHomeStreet",
houseNumber: "34",
city: "6"
}
]
Here you see that the reference city has a value of 6. When, at some point in your program, you want to use that reference, it is done by:
myAddress.Get("city").MyCityName
So, Sproutcore automatically uses the supplied ID in a REST Get, and retrieves the needed record. If the record is available in de local memory of the client (previously loaded), then no round trip is made to the server, otherwise a http get is done for that ID : "http://servername/city/6". Very nice.
Nhibernate (mapped using fluent-nhibernate):
public AddressMap()
{
Schema(Config.ConfigElement("nh_default_schema", "Funda"));
Not.LazyLoad();
//Cache.ReadWrite();
Id(x => x.guid).Unique().GeneratedBy.Identity();
Table("Address");
Map(x => x.street);
Map(x => x.houseNumber);
References(x => x.city,
"cityID").LazyLoad().ForeignKey("fk_Address_cityID_City_guid");
}
Here i specified the foreign key, and to map "cityID" on the database table. It works ok.
BUT (and these are my questions for the guru's):
You can specify to lazy load / eager load a reference (city). Off course you do not want to eager load all your references. SO generally your tied to lazy loading.
But when Openrast (or WCF or ...) serializes such an object, it iterates the properties, which causes all the get's of the properties to be fired, which causes all of the references to be lazy loaded.
SO if your entity has 5 references, 1 query for the base object, and 5 for the references will be done. You might better be off with eager loading then ....
This sucks... Or am i wrong?
As i showed how the model inside sproutcore works, i only want the ID's of the references. So i Don't want eagerloading, and also not lazy loading.
just a "Get * from Address where ID = %" and get that mapped to my Address entity.
THen i also have the ID's of the references which pleases Sproutcore and me (no loading of unneeded references). But.... can NHibernate map the ID's of the references only?
And can i later indicate nHibernate to fully load the reference?
One approach could be (but is not a nice one) to load all reference EAGER (with join) (what a waste of resources.. i know) and in my Sever-side Address entity:
// Note: NOT mapped as Datamember, is NOT serialized!
public virtual City city { get; set; }
Int32 _cityID;
[Datamember]
public virtual Int32 cityID
{
get
{
if (city != null)
return city .guid;
else
return _cityID;
}
set
{
if (city!= null && city.guid != value)
{
city= null;
_cityID = value;
}
else if (city == null)
{
_cityID = value;
}
}
}
So i get my ID property for Sproutcore, but on the downside all references are loaded.
A better idea for me???
nHibernate-to-linq
3a. I want to get my address without their references (but preferably with their id's)
Dao myDao = new Dao();
from p in myDao.All()
select p;
If cities are lazy loading in my mapping, how can i specify in the linq query that i want it also to include my city id only?
3b.
I want to get addresses with my cities loaded in 1 query: (which are mapped as lazyloaded)
Dao myDao = new Dao();
from p in myDao.All()
join p.city ???????
select p;
My Main Question:
As argued earlier, with lazy loading, all references are lazy loaded when serializing entities. How can I prevent this, and only get ID's of references in a more efficient way?
Thank you very much for reading, and hopefully you can help me and others with the same questions. Kind regards.
as a note you wrote you do this
myAddress.Get("city").MyCityName
when it should be
myAddress.get("city").get("MyCityName")
or
myAddress.getPath("city.MyCityName")
With that out of the way, I think your question is "How do I not load the city object until I want to?".
Assuming you are using datasources, you need to manage in your datasource when you request the city object. So in retrieveRecord in your datasource simply don't fire the request, and call dataSourceDidComplete with the appropriate arguments (look in the datasource.js file) so the city record is not in the BUSY state. You are basically telling the store the record was loaded, but you pass an empty hash, so the record has no data.
Of course the problem with this is at some point you will need to retrieve the record. You could define a global like App.WANTS_CITY and in retrieveRecords only do the retrieve when you want the city. You need to manage the value of that trigger; statecharts are a good place to do this.
Another part of your question was "How do I load a bunch of records at once, instead of one request for each record?"
Note on the datasource there is a method retrieveRecords. You can define your own implementation to this method, which would allow you to fetch any records you want -- that avoids N requests for N child records -- you can do them all in one request.
Finally, personally, I tend to write an API layer with methods like
getAddress
and
getCity
and invoke my API appropriately, when I actually want the objects. Part of this approach is I have a very light datasource -- I basically bail out of all the create/update/fetch methods depending on what my API layer handles. I use the pushRetrieve and related methods to update the store.
I do this because the store uses in datasources in a very rigid way. I like more flexibility; not all server APIs work in the same way.

nhibernate - disable automatic\lazy loading of child records for one to many relationsihps

I would like to know if there is a way to disable automatic loading of child records in nHibernate ( for one:many relationships ).
We can easily switch off lazy loading on properties but what I want is to disable any kind of automatic loading ( lazy and non lazy both ). I only want to load data via query ( i.e. HQL or Criteria )
I would still like to define the relationship between parent child records in the mapping file to facilitate HQL and be able to join parent child entities, but I do not want the child records to be loaded as part of the parent record unless a query on the parent record
explicitly states that ( via eager fetch, etc ).
Example:
Fetching Department record from the database should not fetch all employee records from the database because it may never be needed.
One option here is to set the Employees collection on Department as lazy load. The problem with this approach is that once the object is given to the calling API it can 'touch' the lazy load property and that will fetch the entire list from the db.
I tried to use 'evict' - to disconnect the object but it does not seem to be working at all times and does not do a deep evict on the object.
Plus it abstracts the lazy loaded property type with a proxy class that plays havoc later in the code where we are trying to operate on the object via reflection and it encounters unexpended type on the object.
I am a beginner to nHibernate, any pointers or help would be of great help.
Given your request, you could simply not map from Department to Employees, nor have an Employees property on your department. This would mean you always have to make a database hit to find the employees of a database.
Aplogies if these code examples don't work out of the box, I'm not near a compiler at the moment
So, your department class might look like:
public class Department
{
public int Id { get; protected set; }
public string Name { get; set; }
/* Equality and GetHashCode here */
}
and your Employee would look like:
public class Employee
{
public int Id { get; protected set; }
public Name Name { get; set; }
public Department Department { get; set; }
/* Equality and GetHashCode here */
}
Any time you wanted to find Employees for a department, you've have to call:
/*...*/
session.CreateCriteria(typeof(Employee))
.Add(Restrictions.Eq("Department", department)
.List<Employee>();
Simply because your spec says "Departments have many Employees", doesn't mean you have to map it as a bi-directional association. If you can keep your associated uni-directional, you can really get your data-access to fly too.
Google "Domain Driven Design" Aggregate, or see Page 125 of Eric Evan's book on Domain Driven Design for more information
You can have the lazy attribute on the collection. In your example, Department has n employees, if lazy is enabled, the employees will not be loaded by default when you load a department : http://www.nhforge.org/doc/nh/en/#collections-lazy
You can have queries that explicitly load department AND employees together. It's the "fetch" option : http://www.nhforge.org/doc/nh/en/#performance-fetching-lazy

Fluent NHibernate Architecture Question

I have a question that I may be over thinking at this point but here goes...
I have 2 classes Users and Groups. Users and groups have a many to many relationship and I was thinking that the join table group_users I wanted to have an IsAuthorized property (because some groups are private -- users will need authorization).
Would you recommend creating a class for the join table as well as the User and Groups table? Currently my classes look like this.
public class Groups
{
public Groups()
{
members = new List<Person>();
}
...
public virtual IList<Person> members { get; set; }
}
public class User
{
public User()
{
groups = new Groups()
}
...
public virtual IList<Groups> groups{ get; set; }
}
My mapping is like the following in both classes (I'm only showing the one in the users mapping but they are very similar):
HasManyToMany<Groups>(x => x.Groups)
.WithTableName("GroupMembers")
.WithParentKeyColumn("UserID")
.WithChildKeyColumn("GroupID")
.Cascade.SaveUpdate();
Should I write a class for the join table that looks like this?
public class GroupMembers
{
public virtual string GroupID { get; set; }
public virtual string PersonID { get; set; }
public virtual bool WaitingForAccept { get; set; }
}
I would really like to be able to adjust the group membership status and I guess I'm trying to think of the best way to go about this.
I generally only like to create classes that represent actual business entities. In this case I don't think 'groupmembers' represents anything of value in your code. To me the ORM should map the database to your business objects. This means that your classes don't have to exactly mirror the database layout.
Also I suspect that by implementing GroupMembers, you will end up with some nasty collections in both your user and group classes. I.E. the group class will have the list of users and also a list of groupmembers which references a user and vice versa for the user class. To me this isn't that clean and will make it harder to maintain and propagate changes to the tables.
I would suggest keeping the join table in the database as you have suggested, and add a List of groups called waitingtoaccept in users and (if it makes sense too) add List of users called waitingtoaccept in groups.
These would then pull their values from your join-table in the database based on the waitingtoaccept flag.
Yes, sure you need another class like UserGroupBridge. Another good side-effect is that you can modify user membership and group members without loading potentially heavy User/Group objects to NHibernate session.
Cheers.