I have so category and this categories have unlimited sub category.
In Database Table, Fields are ID, UpperID and Title.
If I call a category and its subcategory in DataTable with recursive method in program(ASP.NET project)
performance is very bad.
And many user will use this application so everything goes bad.
Maybe All categories Fill to A Cache object and then we musnt go to Database.
But category count is 15000 or 20000.
So I think isn't a good method.
What can I do for fast performance?
Are you give me any suggestion?
caching or other in-memory-persistance is by far better than doing this on a relational system :) ... hey... it's oop!
just my 2 cents!
eg.
var categories = /* method for domain-objects*/.ToDictionary(category => category.ID);
foreach (var category in categories.Values)
{
if (!category.ParentCategoryID.HasValue)
{
continue;
}
Category parentCategory;
if (categories.TryGetValue(category.ParentCategoryID.Value, out parentCategory))
{
parentCategory.AddSubCategory(category);
}
}
et voila ... your tree is ready to go!
edit:
do you exactly know where your performance bottle-neck is?...
to give you some ideas, eg:
loading from database
building up the structure
querying the structure
loading from database:
then you should load it once and be sure to have some changetracking/notifying to get changes (if made) or optimize your query!
building up the structure:
the way i create the tree (traversal part) is the wastest you can do with a Dictionary<TKey, TValue>
querying the structure:
the structure i've used in my example is faster than List<T>. Dictionary<TKey, TValue> uses an index over the keys - so you may use int for the keys (IDs)
edit:
So you use DataTable to fix the
problem. Now you've got 2 problems: me
and DataTable
what do you have right now? where are you starting from? can you determine where your mudhole is? give us code!
Thanks All,
I find my solution with Common Table Expressions(CTE) fifty- fifty.
Its allow fast recursive queries.
WITH CatCTE(OID, Name, ParentID)
AS
(
SELECT OID, Name, ParentID FROM Work.dbo.eaCategory
WHERE OID = 0
UNION ALL
SELECT C.OID, C.Name, C.ParentID FROM Work.dbo.eaCategory C JOIN CatCTE as CTE ON C.ParentID= CTE.OID
)
SELECT * FROM CatCTE
Related
how to write the sql bellow to django model sql language ?
select id,author_id from forum_node as answer_node where node_type='answer' and
'android' in (select tagnames from forum_node as question_node where
id=answer_node.parent_id );
could any one give help
Hard to say exactly without seeing you model classes, but selecting the relevant Forum_Node objects will look something like this:
Forum_Node.objects.filter(
node_type="answer",
parent__tagnames__name__in=['android']
)
Or, to get just the id and author_id:
Forum_Node.objects.filter(
node_type="answer",
parent__tagnames__name__in=['android']
).values_list('id', 'author_id')
I have a database table called Event which in CakePHP has its relationships coded to like so:
var $belongsTo = array('Sport');
var $hasOne = array('Result', 'Point', 'Writeup', 'Timetable', 'Photo');
Now am doing a query and only want to pull out Sport, Point, and Timetable
Which would result in me retrieving Sports, Events, Points, and Timetable.
Reason for not pulling everything is due the results having 17000+ rows.
Is there a way to only select those tables using:
$this->Event->find('all');
I have had a look at the API but can't see how its done.
You should set recursive to -1 in your app_model and only pull the things you require. never use recursive of 2 and http://book.cakephp.org/view/1323/Containable is awesome.
just $this->Event->find('all', array('contain' => array()));
if you do the trick of recursive as -1 in app_model, this is not needed, if would just be find('all') like you have
I have a simple test object model in which there are schools, and a school has a collection of students.
I would like to retrieve a school and all its students who are above a certain age.
I carry out the following query, which obtains a given school and the children which are above a certain age:
public School GetSchoolAndStudentsWithDOBAbove(int schoolid, DateTime dob)
{
var school = this.Session.CreateCriteria(typeof(School))
.CreateAlias("Students", "students")
.Add(Expression.And(Expression.Eq("SchoolId", schoolid), Expression.Gt("students.DOB", dob)))
.UniqueResult<School>();
return school;
}
This all works fine and I can see the query going to the database and returning the expected number of rows.
However, when I carry out either of the following, it gives me the total number of students in the given school (regardless of the preceding request) by running another query:
foreach (Student st in s.Students)
{
Console.WriteLine(st.FirstName);
}
Assert.AreEqual(s.Students.Count, 3);
Can anyone explain why?
You made your query on the School class and you restricted your results on it, not on the mapped related objects.
Now there are many ways to do this.
You can make a static filter as IanL said, however its not really flexible.
You can just iterate the collection like mxmissile but that is ugly and slow (especially considering lazy loading considerations)
I would provide 2 different solutions:
In the first you maintain the query you have and you fire a dynamic filter on the collection (maintaining a lazy-loaded collection) and doing a round-trip to the database:
var school = GetSchoolAndStudentsWithDOBAbove(5, dob);
IQuery qDob = nhSession.CreateFilter(school.Students, "where DOB > :dob").SetDateTime("dob", dob);
IList<Student> dobedSchoolStudents = qDob.List<Student>();
In the second solution just fetch both the school and the students in one shot:
object result = nhSession.CreateQuery(
"select ss, st from School ss, Student st
where ss.Id = st.School.Id and ss.Id = :schId and st.DOB > :dob")
.SetInt32("schId", 5).SetDateTime("dob", dob).List();
ss is a School object and st is a Student collection.
And this can definitely be done using the criteria query you use now (using Projections)
Unfortunately s.Students will not contain your "queried" results. You will have to create a separate query for Students to reach your goal.
foreach(var st in s.Students.Where(x => x.DOB > dob))
Console.WriteLine(st.FirstName);
Warning: That will still make second trip to the db depending on your mapping, and it will still retrieve all students.
I'm not sure but you could possibly use Projections to do all this in one query, but I am by no means an expert on that.
You do have the option of filtering data. If it there is a single instance of the query mxmissle option would be the better choice.
Nhibernate Filter Documentation
Filters do have there uses, but depending on the version you are using there can be issues where filtered collections are not cached correctly.
When paging data, I want to not only return 10 results, but I also want to get the total number of items in all the pages.
How can I get the total count AND the results for the page in a single call?
My paged method is:
public IList GetByCategoryId(int categoryId, int firstResult, int maxResults)
{
IList<Article> articles = Session.CreateQuery(
"select a from Article as a join a.Categories c where c.ID = :ID")
.SetInt32("ID", categoryId)
.SetFirstResult(firstResult)
.SetMaxResults(maxResults)
.List<Article>();
return articles;
}
The truth is that you make two calls. But a count(*) call is very, very cheap in most databases and when you do it after the main call sometimes the query cache helps out.
Your counter call will often be a little different too, it doesn't actually need to use the inner joins to make sense. There are a few other little performance tweaks too but most of the time you don't need them.
I believe you actually can do what you ask. You can retieve the count and the page in one go in code but not in one SQL statement. Actually two queries are send to the database but in one round trip and the results are retrieved as an array of 2 elements. One of the elements is the total count as an integer and the second is an IList of your retrieved entities.
There are 2 ways to do that:
MultyQuery
MultiCriteria
Here is a sample taken from the links below:
IList results = s.CreateMultiQuery()
.Add("from Item i where i.Id > :id")
.Add("select count(*) from Item i where i.Id > :id")
.SetInt32("id", 50)
.List();
IList items = (IList)results[0];
long count = (long)((IList)results[1])[0];
Here is more info on how you can do that. It is really straight forward.
http://ayende.com/Blog/archive/2006/12/05/NHibernateMutliQuerySupport.aspx
http://ayende.com/Blog/archive/2007/05/20/NHibernate-Multi-Criteria.aspx
If you read the 2 articles above you will see that there is a performance gain of using this approach that adds value to the anyway more transparent and clear approach of doing paging with MultiQuery and MultiCriteria against the conventional way.
Please note that recent versions of NHibernate support the idea of futures, so you can do.
var items = s.CreateQuery("from Item i where i.Id > :id")
.SetInt32("id", 50)
.Future<Item>();
var count = s.CreateQuery("select count(*) from Item i where i.Id > :id")
.SetInt32("id", 50)
.FutureValue<long>();
This is a much more natural syntax, and it will still result in a single DB query.
You can read more about it here:
http://ayende.com/Blog/archive/2009/04/27/nhibernate-futures.aspx
ActiveRecord objects of the class 'Location' (representing the db-table Locations) have the attributes 'url', 'lat' (latitude) and 'lng' (longitude).
Lat-lng-combinations on this model should be unique. The problem is, that there are a lot of Location-objects in the database having duplicate lat-lng-combinations.
I need help in doing the following
Find objects that share the same
lat-lng-combination.
If the 'url' attribute of the object
isn't empty, keep this object and delete the
other duplicates. Otherwise just choose the
oldest object (by checking the attribute
'created_at') and delete the other duplicates.
As this is a one-time-operation, solutions in SQL (MySQL 5.1 compatible) are welcome too.
If it's a one time thing then I'd just do it in Ruby and not worry too much about efficiency. I haven't tested this thoroughly, check the sorting and such to make sure it'll do exactly what you want before running this on your db :)
keep = []
locations = Location.find(:all)
locations.each do |loc|
# get all Locations's with the same coords as this one
same_coords = locations.select { |l| l.lat == loc.lat and \
l.lng == loc.lng }
with_urls = same_coords.select { |l| !l.url.empty? }
# decide which list to use depending if there were any urls
same_coords = with_urls.any? ? with_urls : same_coords
# pick the best one
keep << same_coords.sort { |a,b| b.created_at <=> a.created_at }.first.id
end
# only keep unique ids
keep.uniq!
# now we just delete all the rows we didn't decide to keep
locations.each do |loc|
loc.destroy unless keep.include?( loc.id )
end
Now like I said, this is definitely poor, poor code. But sometimes just hacking out the thing that works is worth the time saved in thinking up something 'better', especially if it's just a one-off.
If you have 2 MySQL columns, you can use the CONCAT function.
SELECT * FROM table1 GROUP BY CONCAT(column_lat, column_lng)
If you need to know the total
SELECT COUNT(*) AS total FROM table1 GROUP BY CONCAT(column_lat, column_lng)
Or, you can combine both
SELECT COUNT(*) AS total, table1.* FROM table1
GROUP BY CONCAT(column_lat, column_lng)
But if you can explain more on your question, perhaps we can have more relevant answers.