I am using TaxonomyManager gettree(path) method to get a particular tree hierarchy in my c# code but it is taking more than 3 min to get the result, due to this the website is taking long time to load. How to reduce the time to load the website, is there any other way i can use to get the hierarchy from Ektron.
We had this exact same issue and actually got on with Ektron support to help resolve it.
Now, whenever we work with taxonomies we cache them on the server-side to avoid the performance hit. Something like
string cacheKey = "Something unique for your situation";
TaxonomyData taxonomyData;
if (Ektron.Cms.Context.HttpContext.Cache[cacheKey] == null)
{
// Pull taxonomy data and store in cache.
Ektron.Cms.Context.HttpContext.Cache.Insert(cacheKey, taxonomyData);
}
else
{
taxonomyData = (TaxonomyData)Ektron.Cms.Context.HttpContext.Cache[cacheKey];
}
Since you already know how to pull the TaxonomyData I left that out. We don't store the taxonomy data, instead we store the object we create with the taxonomy data, so just cache whatever you need to and then you can avoid the performance hit 'most' of the time.
I don't remember where the ektron cache time is set, whether it's in the web.config or within the WorkArea. Ektron support said to use the Ektron cache, not sure how much of a difference it would make to use the regular cache instead.
Related
This seems like a simple thing and should be part of the base code for Yii, but I can't find a solution anywhere. Here is my scenario.
1) User updates their record (use beforesave to set a cache value, changes with each new save, php unique())
public function beforeSave()
{
Yii::app()->cache->set('userupdate'.$this->id,uniqid());
return parent::beforeSave();
}
2) User data is cached using the cache value in step one as a dependency in the loadModel function of the model.
$model=Users::model()->cache(1800, $dependency)->findByPk($id);
3) User views a page that calls to retrieve their data. Yii evaluates the request to see if the cached valued from step 1 has changed, if it has not pull from cache, if it has pull from db.
While reading this page (http://www.yiiframework.com/doc/guide/1.1/en/caching.data) it has that function if a file date changes, but not one for it a variable changes. Any help in this matter would be great as I am at a loss of how to implement this.
NOTE: I need to use cache to hold the variable as I'm running multiple instances of my application and they need shared over each server and all users (thus session won't work).
After fighting with this I found the solution, don't feel it's completely pretty, but it does work. Any feedback on a cleaner way is much appreciated.
$cache = Yii::app()->cache;
$key1 = 'userupdate'.$id; //main cache value
$key2 = '2userupdate'.$id; //will equal main cache when query is cached
$cache1 = $cache['userupdate'.$id];
$cache2 = $cache['2userupdate'.$id];
$dependency = new CExpressionDependency("Yii::app()->cache->get('$key1') == Yii::app()->cache->get('$key2')");
$model=Users::model()->cache(1800,$dependency)->findByPk($id);
if($cache1 != $cache2)
$cache['2userupdate'.$id] = $cache['userupdate'.$id];
One of the dependency options is CExpressionDependency. You could compare the currently cached beforeSave value to the value you get from the loadModel call.
I am trying get a record updated from database with QueryOver.
My code initially creates an entity and saves in database, then the same record is updated on database externally( from other program, manually or the same program running in other machine), and when I call queryOver filtering by the field changed, the query gets the record but without latest changes.
This is my code:
//create the entity and save in database
MyEntity myEntity = CreateDummyEntity();
myEntity.Name = "new_name";
MyService.SaveEntity(myEntity);
// now the entity is updated externally changing the name property with the
// "modified_name" value (for example manually in TOAD, SQL Server,etc..)
//get the entity with QueryOver
var result = NhibernateHelper.Session
.QueryOver<MyEntity>()
.Where(param => param.Name == "modified_name")
.List<T>();
The previous statement gets a collection with only one record(good), BUT with the name property established with the old value instead of "modified_name".
How I can fix this behaviour? First Level cache is disturbing me? The same problem occurs with
CreateCriteria<T>();
The session in my NhibernateHelper is not being closed in any moment due application framework requirements, only are created transactions for each commit associated to a session.Save().
If I open a new session to execute the query evidently I get the latest changes from database, but this approach is not allowed by design requirement.
Also I have checked in the NHibernate SQL output that a select with a WHERE clause is being executed (therefore Nhibernate hits the database) but don´t updates the returned object!!!!
UPDATE
Here's the code in SaveEntity after to call session.Save: A call to Commit method is done
public virtual void Commit()
{
try
{
this.session.Flush();
this.transaction.Commit();
}
catch
{
this.transaction.Rollback();
throw;
}
finally
{
this.transaction = this.session.BeginTransaction();
}
}
The SQL generated by NHibernate for SaveEntity:
NHibernate: INSERT INTO MYCOMPANY.MYENTITY (NAME) VALUES (:p0);:p0 = 'new_name'.
The SQL generated by NHibernate for QueryOver:
NHibernate: SELECT this_.NAME as NAME26_0_
FROM MYCOMPANY.MYENTITY this_
WHERE this_.NAME = :p0;:p0 = 'modified_name' [Type: String (0)].
Queries has been modified due to company confidential policies.
Help very appreciated.
As far as I know, you have several options :
have your Session as a IStatelessSession, by calling sessionFactory.OpenStatelesSession() instead of sessionFactory.OpenSession()
perform Session.Evict(myEntity) after persisting an entity in DB
perform Session.Clear() before your QueryOver
set the CacheMode of your Session to Ignore, Put or Refresh before your QueryOver (never tested that)
I guess the choice will depend on the usage you have of your long running sessions ( which, IMHO, seem to bring more problems than solutions )
Calling session.Save(myEntity) does not cause the changes to be persisted to the DB immediately*. These changes are persisted when session.Flush() is called either by the framework itself or by yourself. More information about flushing and when it is invoked can be found on this question and the nhibernate documentation about flushing.
Also performing a query will not cause the first level cache to be hit. This is because the first level cache only works with Get and Load, i.e. session.Get<MyEntity>(1) would hit the first level cache if MyEntity with an id of 1 had already been previously loaded, whereas session.QueryOver<MyEntity>().Where(x => x.id == 1) would not.
Further information about NHibernate's caching functionality can be found in this post by Ayende Rahien.
In summary you have two options:
Use a transaction within the SaveEntity method, i.e.
using (var transaction = Helper.Session.BeginTransaction())
{
Helper.Session.Save(myEntity);
transaction.Commit();
}
Call session.Flush() within the SaveEntity method, i.e.
Helper.Session.Save(myEntity);
Helper.Session.Flush();
The first option is the best in pretty much all scenarios.
*The only exception I know to this rule is when using Identity as the id generator type.
try changing your last query to:
var result = NhibernateHelper.Session
.QueryOver<MyEntity>()
.CacheMode(CacheMode.Refresh)
.Where(param => param.Name == "modified_name")
if that still doesn't work, try add this after the query:
NhibernateHelper.Session.Refresh(result);
After search and search and think and think.... I´ve found the solution.
The fix: It consist in open a new session, call QueryOver<T>() in this session and the data is succesfully refreshed. If you get child collections not initialized you can call HibernateUtil.Initialize(entity) or sets lazy="false" in your mappings. Take special care about lazy="false" in large collections, because you can get a poor performance. To fix this problem(performance problem loading large collections), set lazy="true" in your collection mappings and call the mentioned method HibernateUtil.Initialize(entity) of the affected collection to get child records from database; for example, you can get all records from a table, and if you need access to all child records of a specific entity, call HibernateUtil.Initialize(collection) only for the interested objects.
Note: as #martin ernst says, the update problem can be a bug in hibernate and my solution is only a temporal fix, and must be solved in hibernate.
People here do not want to call Session.Clear() since it is too strong.
On the other hand, Session.Evict() may seem un-applicable when the objects are not known beforehand.
Actually it is still usable.
You need to first retrieve the cached objects using the query, then call Evict() on them. And then again retrieve fresh objects calling the same query again.
This approach is slightly inefficient in case the object was not cached to begin with - since then there would be actually two "fresh" queries - but there seems to be not much to do about that shortcoming...
By the way, Evict() accepts null argument too without exceptions - this is useful in case the queried object is actually not present in the DB.
var cachedObjects = NhibernateHelper.Session
.QueryOver<MyEntity>()
.Where(param => param.Name == "modified_name")
.List<T>();
foreach (var obj in cachedObjects)
NhibernateHelper.Session.Evict(obj);
var freshObjects = NhibernateHelper.Session
.QueryOver<MyEntity>()
.Where(param => param.Name == "modified_name")
.List<T>()
I'm getting something very similar, and have tried debugging NHibernate.
In my scenario, the session creates an object with a couple children in a related collection (cascade:all), and then calls ISession.Flush().
The records are written into the DB, and the session needs to continue without closing. Meanwhile, another two child records are written into the DB and committed.
Once the original session then attempts to re-load the graph using QueryOver with JoinAlias, the SQL statement generated looks perfectly fine, and the rows are being returned correctly, however the collection that should receive these new children is found to have already been initialized within the session (as it should be), and based on that NH decides for some reason to completely ignore the respective rows.
I think NH makes an invalid assumption here that if the collection is already marked "Initialized" it does not need to be re-loaded from the query.
It would be great if someone more familiar with NHibernate internals could chime in on this.
My SQL and Entity Framework knowledge is a somewhat limited. In one Entity Framework (4) application, I notice it takes forever (about 2 minutes) to complete one of my method calls. The first queries do not take much time, but when I loop through the Entity Framework objects returned by the queries, even though I am only reading (not modifying) the data I supposedly got, it takes forever to complete the nested loops, even though there are only dozens of entries in each list and a few levels of looping.
I expect the example below could be re-written with a fancier query that could probably include all of the filtering I am doing in my loops with some SQL words I don't really know how to use, so if someone could show me what the equivalent SQL expression would be, that would be extremely educational to me and probably solve my current performance problem.
Moreover, since other parts of this and other applications I develop often want to do more complex computations on SQL data, I would also like to know a good way to retrieve data from Entity Framework to local memory objects that do not have huge delays in reading them. In my LINQ-to-SQL project there was a similar performance problem, and I solved it by refactoring the whole application to load all SQL data into parallel objects in RAM, which I had to write myself, and I wonder if there isn't a better way to either tell Entity Framework to not keep doing whatever high-latency communication it is doing, or to load into local RAM objects.
In the example below, the code gets a list of food menu items for a member (i.e. a person) on a certain date via a SQL query, and then I use other queries and loops to filter out the menu items on two criteria: 1) If the member has a rating of zero for any group id which the recipe is a member of (a many-to-many relationship) and 2) If the member has a rating of zero for the recipe itself.
Example:
List<PFW_Member_MenuItem> MemberMenuForCookDate =
(from item in _myPfwEntities.PFW_Member_MenuItem
where item.MemberID == forMemberId
where item.CookDate == onCookDate
select item).ToList();
// Now filter out recipes in recipe groups rated zero by the member:
List<PFW_Member_Rating_RecipeGroup> ExcludedGroups =
(from grpRating in _myPfwEntities.PFW_Member_Rating_RecipeGroup
where grpRating.MemberID == forMemberId
where grpRating.Rating == 0
select grpRating).ToList();
foreach (PFW_Member_Rating_RecipeGroup grpToExclude in ExcludedGroups)
{
List<PFW_Member_MenuItem> rcpsToRemove = new List<PFW_Member_MenuItem>();
foreach (PFW_Member_MenuItem rcpOnMenu in MemberMenuForCookDate)
{
PFW_Recipe rcp = GetRecipeById(rcpOnMenu.RecipeID);
foreach (PFW_RecipeGroup group in rcp.PFW_RecipeGroup)
{
if (group.RecipeGroupID == grpToExclude.RecipeGroupID)
{
rcpsToRemove.Add(rcpOnMenu);
break;
}
}
}
foreach (PFW_Member_MenuItem rcpToRemove in rcpsToRemove)
MemberMenuForCookDate.Remove(rcpToRemove);
}
// Now filter out recipes rated zero by the member:
List<PFW_Member_Rating_Recipe> ExcludedRecipes =
(from rcpRating in _myPfwEntities.PFW_Member_Rating_Recipe
where rcpRating.MemberID == forMemberId
where rcpRating.Rating == 0
select rcpRating).ToList();
foreach (PFW_Member_Rating_Recipe rcpToExclude in ExcludedRecipes)
{
List<PFW_Member_MenuItem> rcpsToRemove = new List<PFW_Member_MenuItem>();
foreach (PFW_Member_MenuItem rcpOnMenu in MemberMenuForCookDate)
{
if (rcpOnMenu.RecipeID == rcpToExclude.RecipeID)
rcpsToRemove.Add(rcpOnMenu);
}
foreach (PFW_Member_MenuItem rcpToRemove in rcpsToRemove)
MemberMenuForCookDate.Remove(rcpToRemove);
}
You can use EFProf http://www.hibernatingrhinos.com/products/EFProf to track see exactly what EF is sending to SQL. It can also show you how many queries you are sending and how many unique queries. It also provides you some analysis of each query (e.g. is it unbound etc). Entity Framework with its navigation properties, it is quite easy to not realize you are making a db request. When you are in a loop, and have a navigation property, you get in to the N + 1 problem.
You could use the Keyword Virtual on your List parts of your model if you are using code first to enable proxying, that way you will not have to get all the data back at once, only as you need it.
Also consider NoTracking for read only data
context.bigTable.MergeOption = MergeOption.NoTracking;
I have saved certain MDX query and I run them using ADOMD.NET. I get CellSet back which I convert into dataset. All this is working fine. Now the DB team has changed the cube structure. They have updated the DimesnionName, Attribute Name etc. Some dimensions got renamed and some got deleted. Becuase of this I am unable to run my saved queries. I want to create a console application, which will take list of keys ([DimensionName].[AttributeName] or [DimensionName].[AttributeName].[MemeberName] format) and it will tell me following keys does not exists.
Please let me know if this is possible programatically. I dont want to check it manually.
Kindly share a link or code which will help me acheive this.
Thank you.
If you're using ADOMD already this should be no problem, just use the metadata queries:
http://msdn.microsoft.com/en-us/library/ms123485.aspx
Alternatively, AMO is nice http://msdn.microsoft.com/en-us/library/microsoft.analysisservices.aspx
I use it in SSIS for processing, you could easily use it in .Net to test existence of elements:
using Microsoft.AnalysisServices;
...
Server server = new Server();
server.Connect(cubeConnectionString);
Database database = server.Databases.FindByName(databaseName);
Cube cube = database.Cubes.FindByName(cubeName);
foreach (MeasureGroup measureGroup in cube.MeasureGroups)
{
foreach (Partition partition in measureGroup.Partitions)
{
...
}
}
foreach (CubeDimension cubeDimension in cube.Dimensions)
{
Dimension dimension = cubeDimension.Dimension;
var dimName = dimension.Name;
...
}
Finding the names in advance for all the elements you need is probably the hard part (And keeping it all up-to-date).
Would it not be easier to fire all the queries at the cube and try to trap the "no such thing" response?
I'm using the Magento API to get product data for products from a certain category from another domain. I have made the API call etc... The code I'm currently using to get the product data looks like this:
$productList = $client->call($session, 'catalog_category.assignedProducts', 7);
foreach ($productList as $product){
$theProduct = array();
$theProduct['info'] = $client->call($session, 'catalog_product.info', $product['sku']);
$allProducts[] = $theProduct;
}
The code works fine, but it goes extremely slow. When I add the image call to the loop it takes about 50 seconds for the page to load, and that's for a site with only 5 products. What I want to know is the following:
Is the code above correct and it's just Magento's API script is very slow?
Is the code above not the best way of doing what I need?
Could there be any other factors making this go so slow?
Any help would be much appreciated. At least if I know I'm using the code right I can look at other avenues.
Thanks in advance!
================= EDIT =================
Using multicall suggested by Matthias Zeis, the data arrives much quicker. Here's the code I used:
$apicalls = array();
$i = 0;
$productList = $client->call($session, 'catalog_category.assignedProducts', 7);
foreach ($productList as $product){
$apicalls[$i] = array('catalog_product.info', $product['product_id']);
$i++;
}
$list = $client->multiCall($session, $apicalls);
This now works much quicker than before! The next issue I've found is that the catalog_product_attribute_media.list call doesn't seem to work in the same way, even though the products all have images set.
The error I'm getting in the var_dump is:
Requested image not exists in product images' gallery.
Anybody know why this may now be happening? Thanks again in advance.
1. Is the code above correct and it's just Magento's API script is very slow?
Your code is correct, but the script is slow because (a) the SOAP API is not blazingly fast and (b) you are doing seperate calls for every single product.
2. Is the code above not the best way of doing what I need?
If you use the SOAP v1 API or XML-RPC, you can test multiCall. At first, call catalog_category.assignedProducts to fetch the product ids. Collect the product ids and execute a multiCall call. That should cut the waiting time down quite a bit.
Unfortunately, Magento doesn't provide a nice solution out of the box to deliver the data like you need it. I recommend that you implement your own custom API call.
Use a product collection model:
$collection = Mage::getModel('catalog/product')->getCollection();
This will get you a Mage_Catalog_Model_Resource_Product_Collection object which can be used to filter, sort, paginate, ... your product list. Iterate over the collection and build an array containing the data you need. You also can generate thumbnails for your products directly while building the data array:
foreach ($products as $product) {
$data[$product->getSku()] = array(
/* the attributes no need ... */
'small_image' => Mage::helper('catalog/image')->init($product, 'image')
->constrainOnly(true)
->keepAspectRatio(true)
->keepFrame(false)
->resize(100,150)
->__toString(),
/* some more attributes ... */
);
}
This should give you quite a performance improvement.
But of course this only is the tip of the iceberg. If this solution is not fast enough for you, avoid SOAP and bypass a part of the Magento stack by building your own API. This doesn't have to be a complex solution: it could be a simple PHP script with HTTP Basic Authentication which parses the URL for filter criteria etc., includes app/Mage.php and calls Mage::app() to initialise the Magento framework. The benefit is that you have the comfort of using Magento classes but you don't have to go through the whole routing process.
Not to forget, you may cache the results because I could imagine that you will show the same products to quite a few visitors on the other domain. Even caching for a few minutes may help your server.
3. Could there be any other factors making this go so slow?
There may be some reasons why the calls are that slow on your server - but without knowing the volume of your data, your server hardware and the customisations you have done, even a best guess won't be that good.