413 Error FULL HEAD Query URL too large Rally - rally

I am using app SDK 2.0.I have the following code to retrieve the tasks for first 100 Team Members of the project for a particular Iteration
//getting the first 100 elements of the owners Array
owners = Ext.Array.splice(owners,0,100);
//Initial configuration of the filter object
var ownerFilter = Ext.create('Rally.data.QueryFilter', {
property: 'Owner.DisplayName',
operator:'=',
value: owners[0]
});
/*Creating the filter object to get all the tasks for 100 members in that project*/
Ext.Array.each(owners,function(member){
ownerFilter = ownerFilter.or(
Ext.create('Rally.data.QueryFilter',{
property: 'Owner.DisplayName',
operator:'=',
value: member
}));
});
//Iteration Filter for the Object
var iterationFilter = Ext.create('Rally.data.QueryFilter', {
property: 'Iteration.Name',
operator:'=',
value: 'Iteration 4.2'
});
var filter = ownerFilter.and(iterationFilter);
Rally.data.ModelFactory.getModel({
type: 'Task',
success: function(model) {
var taskStore = Ext.create('Rally.data.WsapiDataStore', {
model: model,
fetch:true,
limit:Infinity,
pageSize:200,
filters:[filter],
autoLoad:true,
projectScopeDown:true,
listeners:{
load:function(store,data,success) {
Ext.Array.each(data, function(record){
console.log(record.data);
});
}
}
});
}
});
This code is giving me a 413 error since the URL for the query is too large.The request URL has names of all 100 members of the project.How can I solve this problem ? Are there any efficient filtering options available?

Team Membership is a tough attribute to filter Artifacts on, since it resides as an attribute on Users and is a collection that can't be used in Queries.
In this case you might be better off doing more of the filtering client-side. Since Team Membership "approximates" or associates to Projects, you may wish to start by filtering your Tasks on Project, i.e. (Project.Name = "My Project"). Then narrow things down client side by looping through and ensuring each Task is owned by a member of the Team in your owners collection.
Alternatively, you might look to do something like use Tags on your Tasks that mirror the Team Names. You could then filter your Tasks server side by doing a query such as:
(Tags.Name contains "My Team")
I know these aren't ideal options and probably not the answer you're looking for, but, they are some ideas to avoid having to build a massive username-based query.

Related

Only a node can be linked! Not "undefined"!

I want to put nodes in Gun set.
const Gun = require('gun');
const _ = require('lodash');
require( "gun/lib/path" );
const gun = new Gun({peers:['http://localhost:8080/gun']});
const watchers = [
{
_id: '123',
_type: 'skeleton',
_source: {
trigger: {
schedule: {
later: 'every 1 sec'
}
}
}
},
{
_id: '456',
_type: 'snowman',
_source: {
trigger: {
schedule: {
later: 'every 1 sec'
}
}
}
}
];
const tasks = gun.get('tasks');
_.forEach(watchers, function (watcher) {
let task = gun.get(`watcher/${watcher._id}`).put(watcher);
tasks.set(task);
});
In the end, I receive only the following message. And script stuck in the terminal.
Only a node can be linked! Not "undefined"!
There is nothing on the listener side:
const tasks = gun.get('tasks');
tasks.map().val(function (task) {
console.log('task', task);
});
What is wrong?
The result is received on the listener side only if I change the watchers objects to simpler ones, for example:
_.forEach(watchers, function (watcher) {
let task = gun.get(`watcher/${watcher._id}`).put({id: '123'});
tasks.set(task);
});
Results:
task { _: { '#': 'watcher/123', '>': { id: 1506953120419 } },
id: '123' }
task { _: { '#': 'watcher/456', '>': { id: 1506953120437 } },
id: '123' }
#trex you correctly reported this as a bug, and we got this fixed here: https://github.com/amark/gun/issues/427 .
When a node is referenced, it should not act as if it is undefined. This was a bug.
However, in the future, some people may attempt to link non-node references. As such, I would like to answer the title of your subject (but note, your actual issue has been fixed, and your code should now work correctly in v0.8.8+).
Why do I get "Only a node can be linked!" error?
Say you have a reference to a thing in gun:
var thing = gun.get('alice').get('age');
You may want to add it to a set (otherwise called a table, or list, or collection) like so:
gun.get('list').set(thing);
You will get a "Only a node can be linked!" error. This is annoying! But here is why:
Because age (or any other example data) is a primitive value, adding it to a table would cause it to lose its context. Instead, we can achieve the exact same end result using the following approach:
var person = gun.get('alice').get('age');
gun.get('list').set(thing);
gun.get('list').map().get('age').on(callback);
Now we get back a list of ages, but those ages will always reflect their latest/current realtime context. Had we just added the age to the table, it would no longer have a realtime context.
This is why only nodes can be linked, because any of the data on that node that you were trying to link can just be linked to by traversing via the node. In this case, it was by doing .get('age') after the list. There are a couple really cool things about this:
Bandwidth is saved. GUN will only load the age property from each item in the list, it will not load the rest of the item. It syncs the data you ask about.
Everything is traversable. No matter where your data is in a graph, whether it is a document, a key/value pair, a table, relational data, or anything else, it will always be traversable from its node in the graph. This is possible because it is always the node that is linked, not the primitive data.
Note: What can be frustrating is that you may not know in advance whether a certain gun reference is a node or a primitive, as you could always allow your users to dynamically change the data on that reference. This would require you to handle the error gracefully and do whatever you can best guess the user intended. You can avoid this problem by enforcing a schema on the data in your app. If your app is deployed, we strongly recommend using a schema.
But what if I want the raw data linked, not a realtime context?
Then all you have to do is pass the actual value of the data, not the reference to it. Like so:
gun.get('list').set(thing);
As always, the chatroom is super friendly so come say hi. Please use StackOverflow for asking questions, but notify us in the chatroom. The chatroom is for quick help, and SO is for long standing questions that others would benefit from.
Thanks for asking this question! I hope this answered it, give us a shout if you have any further questions or concerns.

sequelize: how to do equivalent of findAndCountAll for associations

One of my biggest pains with sequelize is to basically "paginate and count" for associations.
Using findAndCountAll is fairly straightforward for one-off models, a bit painful for has-many, but utterly impossible to work with with many-to-many.
As an example, I have this many-to-many association established, where user can belong to many groups:
const UserGroup = db.define('user_groups', {
})
User.belongsToMany(Group, {through: UserGroup})
Group.belongsToMany(User, {through: UserGroup})
getting all users from a group would be straightforward:
user.getGroups().then....
but porting this to findAndCountAll just doesn't seem to work the same way:
User.findAndCountAll({
include: [{model: Group,
through: UserGroup,
where: { groupId : group.id,
userId : {$ne: user.id}}
}],
limit: limit,
offset: offset, ...
this doesn't work, as it associates keys from the where clause to the Group model.
I also tried:
User.findAndCountAll({
include: [{model: Group,
include: {
model: UserGroup,
where: { groupId : group.id,
userId : {$ne: user.id}}}
}],
limit: limit,
offset: offset, ...
but it fails as well, with Error: user_groups is not associated to groups!, which is not really true.
Is there a clean way to do this, preferably with the helper methods?
I had this problem earlier, and I decided to change functionality in this way...
const include = [
{
model: models.UserGroup,
attributes: [],
as: 'groups'
}
];
const [total, data] = await Promise.all([
User.count({ distinct: true, include }),
User.findAll({
order: [sequelize.literal(`MAX("groups"."createdAt") desc`)],
group: ['User.id'],
include
})
]);
I'm afraid there is no possibility to use findAndCountAll method when eager loading associations (or at least when you also want to add some conditions to associations). There is no option to get information about total count of associated items using, in your case, user.getGroups().
Although you can use Sequelize's shorthand method for counting association's instances, in your case, user.countGroups() before returning user.getGroups(). If you want to fetch related groups with some conditions, simply pass them as getGroups() method parameter (it takes same options parameter as standard findAll)

JSON API design - express

I want to write a JSON API.
My problem is, that sometimes I want to query for an ID, sometimes for a String.
One option would be to add a querystring, for example:
example.com/user/RandomName
example.com/user/1234556778898?id=true
and use it like:
api.get('user/:input', function(req, res) {
if(req.query.id) {
User.find({ '_id': req.params.input }, cb);
} else {
User.find({ 'name': req.params.input }, cb);
}
};
But this seems like bad practice to me, since it leads to a bunch of conditional expressions.
Are there more elegant ways?
I would suggest handling two endpoints. One for getting ALL the users and one for getting a SPECIFC user by ID.
example.com/users
example.com/users/:id
The second endpoint can be used to find a specific user by id.
The first endpoint can be used to find all users, but filters can be applied to this endpoint.
For example: example.com/users?name=RandomName
By doing this, you can very easily create a query in your Node service based on the parameters that are in the URL.
api.get('/users', function(req, res) {
// generate the query object based on URL parameters
var queryObject = {};
for (var key in req.query) {
queryObject[key] = req.query[key];
}
// find the users with the filter applied.
User.find(queryObject, cb);
};
By constructing your endpoints this way, you are following a RESTful API standard which will make it very easy for others to understand your code and your API. In addition, you are constructing an adaptable API as you can now filter your users by any field by adding the field as a parameter to the URL.
See this response for more information on when to use path parameters vs URL parameters.

Unwrapping breeze Entity properties

I'm very new to breeze/knockout, but I'm 99% of the way to doing what I need to do. I'm using the Hot Towel template, and I'm successfully retrieving a list of items via breeze. The entity (ITBAL) is a database first Entity Framework entity. When I look at the JSON coming back in Fiddler, I see the correct data. The problem is that all of the properties of data.results are dependentobservables, and not the raw values themselves.
We have a custom grid control that is trying to display the data.results array. Because it is not expecting observables, it is simply displaying "function dependentobservable" instead of the value.
I tried to unwrap the object, but keep getting the circular reference error. I don't know why that is, since ITBAL isn't associated with anything.
The data as Fiddler reports it:
[{"$id":"1","$type":"WebUIHtml5HotTowel.Models.ITBAL, WebUIHtml5HotTowel","IBITNO":"A100 ","IBWHID":"1 ","IBITCL":"50","IBITSC":"3 ","IBSUSP":" ","IBVNNO":"100 ","IBPRLC":" ","IBSCLC":" ","IBCCCD":"P","IBPICD":" ","IBSAFL":"Y","IBSTCS":399.99000,"IBUSCS":0.00000,"IBAVCS":414.95214,"IBLCST":7.00000,"IBLCCC":20.0,"IBLCDT":110923.0,"IBLSCC":20.0,"IBLSDT":130111.0,"IBLXCC":19.0,"IBLXDT":990102.0,"IBMXO1":2100.000,"IBMXO2":0.000,"IBMXO3":0.000,"IBMNO1":5.000,"IBMNO2":0.000,"IBMNO3":0.000,"IBFOQ1":0.000,"IBFOQ2":0.000,"IBFOQ3":0.000,"IBOHQ1":327.000,"IBOHQ2":0.000,"IBOHQ3":0.000,"IBAQT1":1576.000,"IBAQT2":0.000,"IBAQT3":0.000,"IBBOQ1":50.000,"IBBOQ2":0.000,"IBBOQ3":0.000,"IBPOQ1":448.000,"IBPOQ2":0.000,"IBPOQ3":0.000,"IBIQT1":1446.000,"IBIQT2":0.000,"IBIQT3":0.000,"IBRMD1":10.000,"IBRMD2":0.000,"IBRMD3":0.000,"IBRYD1":10.000,"IBRYD2":0.000,"IBRYD3":0.000,"IBISM1":0.000,"IBISM2":0.000,"IBISM3":0.000,"IBISY1":0.000,"IBISY2":0.000,"IBISY3":0.000,"IBAMD1":0.000,"IBAMD2":0.000,"IBAMD3":0.000,"IBAYD1":0.000,"IBAYD2":0.000,"IBAYD3":0.000,"IBMMD1":0.000,"IBMMD2":0.000,"IBMMD3":0.000,"IBMYD1":0.000,"IBMYD2":0.000,"IBMYD3":0.000,"IBSMD1":1.0,"IBSMD2":0.0,"IBSMD3":0.0,"IBSYD1":1.0,"IBSYD2":0.0,"IBSYD3":0.0,"IBBLME":335.000,"IBBLYO":2680.000,"IBBLLY":1441.000,"IBNMTY":8.0,"IBNMLY":11.0,"IBQSMD":21.000,"IBQSYD":21.000,"IBQSLY":20.000,"IBISMD":16318.19,"IBISYD":16318.19,"IBISLY":45714.87,"IBCSMD":373.46,"IBCSYD":373.46,"IBCSLY":67.00,"IBDQMD":0.000,"IBDQYD":0.000,"IBDQLY":0.000,"IBDSMD":0.00,"IBDSYD":0.00,"IBDSLY":0.00,"IBDCMD":0.00,"IBDCYD":0.00,"IBDCLY":0.00,"IBNOMD":18.0,"IBNOYD":18.0,"IBNOLY":18.0,"IBPKMD":15.0,"IBPKYD":15.0,"IBPKLY":14.0,"IBINUS":" ","IBIAID":0.0,"IBSAID":0.0,"IBCQT1":1527.000,"IBCQT2":0.000,"IBCQT3":0.000,"IBFCST":"Y","IBDRSH":" ","IBWMIU":"JP","IBFL15":" ","IBUS20":" ","IBLPR1":0.00000,"IBLPR2":0.00000,"IBLPR3":0.00000,"IBLPR4":0.00000,"IBLPR5":0.00000,"IBLPCD":" ","IBABCC":"B","IBPRCL":0.0,"IBQBCL":" ","IBACDC":"Y","IBTDCD":" ","IBDOUM":" ","IBTP01":0.0,"IBTP02":0.0,"IBTP03":0.0,"IBTP04":0.0,"IBLMCC":20.0,"IBLMDT":130513.0,"IBTMPH":"Y","IBCOMC":" ","IBCOMF":0.00000,"IBITCT":" ","IBEOQT":0.000,"IBITCM":0.0,"IBBRVW":" ","IBPTID":" ","IBQTLT":0.0000,"IBCTY1":"AUS","IBCTY2":"AUS","IBTXCD":"1","IBREVS":"Y","IBITXC":" ","IBMNOQ":0.000,"IBSTUS":0.000,"IBUS30":" ","IBPSLN":" ","IBPLIN":"N","IBUPDP":"Y","IBDFII":"2011-08-11T00:00:00.000","IBLHRK":"A","IBPLNC":" "}]
My Controller:
[BreezeController]
public class ItemInquiryController : ApiController
{
readonly EFContextProvider<AplusEntities> _contextProvider = new EFContextProvider<AplusEntities>();
[System.Web.Http.HttpGet]
public string Metadata()
{
return _contextProvider.Metadata();
}
[HttpGet]
public IQueryable<ITBAL> ItemBalances(string itemNumber, string warehouse)
{
return _contextProvider.Context.ITBALs.Where(i => i.IBITNO == itemNumber && i.IBWHID == warehouse)
.OrderBy(i => i.IBWHID)
.ThenBy(i => i.IBITNO);
}
}
The relevant portion from the viewmodel:
var manager = new breeze.EntityManager("api/ItemInquiry");
var store = manager.metadataStore;
var itbalInitializer = function (itbal) {
itbal.CompositeKey = ko.computed(function () {
return itbal.IBITNO() + itbal.IBWHID();
});
};
store.registerEntityTypeCtor("ITBAL", null, itbalInitializer);
var index = "0" + (args.pageNum * args.pageSize);
var query = new breeze.EntityQuery("ItemBalances")
.withParameters({ itemNumber: "A100", warehouse: "1" })
.take(args.pageSize);
if (index > 0) {
query = query.skip(index);
}
manager.executeQuery(query).then(function (data) {
vm.itbals.removeAll();
var itbals = data.results;//[0].Data;
itbals.forEach(function (itbal) {
vm.itbals.push(itbal);
});
vm.totalRecords(1);
itemBalancesGrid.mergeData(vm.itbals(), args.pageNum, parseInt(vm.totalRecords()));
}).fail(function (e) {
logger.log(e, null, loggerSource, true, 'error');
});
I figure I must be missing something fairly simple, but it is escaping me.
UPDATE: I removed the BreezeController attribute from the ApiController, and it works correctly.
Jon, removing the [Breeze] attribute effectively disables breeze for your application so I don't think that is the long term answer to your problem.
If you don't actually want entities for this scenario - you just want data - than a Breeze projection that mentions just the data to display in the grid would seem to be the best choice. Projections return raw data that are not wrapped in KO observables and are not held in the Breeze EntityManager cache.
If you want the data as cached entities and you also want to display them in a grid that doesn't like KO observable properties ... read on.
You can unwrap a KO'd object with ko.toJS. However, the grid is likely to complain about circular references (or throw an "out of memory" exception as some grids do) ... even if the entity has no circular navigation paths. The difficulty stems from the fact that every Breeze entity has an inherent circularity by way of its entityAspect property:
something.entityAspect.entity //'entity' points back to 'something'
Because you are using Knockout for your model library and because you say ITBAL has no navigation properties ("is not related to anything"), I think the following will work for you:
manager.executeQuery(query).then(success) ...
function success(data) {
var unwrapped = ko.toJS(data.results).map(
function(entity) {
delete entity.entityAspect;
return entity;
});
vm.itbals(unwrapped);
vm.totalRecords(1); // huh? What is that "parseInt ..." stuff?
itemBalancesGrid.mergeData(vm.itbals(), args.pageNum, parseInt(vm.totalRecords()));
})
ko.toJS is a Knockout function that recursively unwraps an object or collection of objects, returning copies of values. Then we iterate over the copied object graphs, deleting their entityAspect properties. The array of results is stuffed into the vm.itbals observable and handed along.
You can imagine how to generalize this to remove anything that is giving you trouble.
Extra
What the heck is vm.totalRecords? I sense that this is supposed to be the total number of matching records before paging. You can get that from Breeze by adding .inlineCount() to the breeze query definition. You get the value after the query returns from the data.inlineCount property.
Do you really need vm.itbals()? If all you do here is pass values to the grid, why not do that and cut out the middle man?
The following success callback combines these thoughts
function success(data) {
var unwrapped = ko.toJS(data.results).map(
function(entity) {
delete entity.entityAspect;
return entity;
});
itemBalancesGrid.mergeData(unwrapped, args.pageNum, data.inlineCount);
})

Sproutcore datasources and model relationships

I currently have a Sproutcore app setup with the following relationships on my models:
App.Client = SC.Record.extend({
name: SC.Record.attr(String),
brands: SC.Record.toMany('App.Brand', {isMaster: YES, inverse: 'client'})
});
App.Brand = SC.Record.extend({
name: SC.Record.attr(String),
client: SC.Record.toOne('App.Client, {isMaster: NO, inverse: 'brands'})
});
When I was working with fixtures my fixture for a client looked like this:
{
guid: 1,
name: 'My client',
brands: [1, 2]
}
And my fixture for a brand looked like this:
{
guid: 1,
name: 'My brand',
client: 1
}
Which all worked fine for me getting a clients brands and getting a brands client.
My question is in regards to how Datasources then fit into this and how the server response should be formatted.
Should the data returned from the server mirror exactly the format of the fixtures file? So clients should always contain a brands property containing an array of brand ids? And vice versa.
If I have a source list view which displays Clients with brands below them grouped. How would I go about loading that data for the source view with my datasource? Should I make a call to the server to get all the Clients and then follow that up with a call to fetch all the brands?
Thanks
Mark
The json you return will mostly mirror the fixtures. I recently had pretty much the same question as you, so I built a backend in Grails and a front end in SC, just to explore the store and datasources. My models are:
Scds.Project = SC.Record.extend(
/** #scope Scds.Project.prototype */ {
primaryKey: 'id',
name: SC.Record.attr(String),
tasks: SC.Record.toMany("Scds.Task", {
isMaster: YES,
inverse: 'project'
})
});
Scds.Task = SC.Record.extend(
/** #scope Scds.Task.prototype */ {
name: SC.Record.attr(String),
project: SC.Record.toOne("Scds.Project", {
isMaster: NO
})
});
The json returned for Projects is
[{"id":1,"name":"Project 1","tasks":[1,2,3,4,5]},{"id":2,"name":"Project 2","tasks":[6,7,8]}]
and the json returned for tasks, when I select a Project, is
{"id":1,"name":"task 1"}
obviously, this is the json for 1 task only. If you look in the projects json, you see that i put a "tasks" array with ids in it -- thats how the internals know which tasks to get. so to answer your first question, you dont need the id from child to parent, you need the parent to load with all the children, so the json does not match the fixtures exactly.
Now, it gets a bit tricky. When I load the app, I do a query to get all the Projects. The store calls the fetch method on the datasource. Here is my implementation.
Scds.PROJECTS_QUERY = SC.Query.local(Scds.Project);
var projects = Scds.store.find(Scds.PROJECTS_QUERY);
...
fetch: function(store, query) {
console.log('fetch called');
if (query === Scds.PROJECTS_QUERY) {
console.log('fetch projects');
SC.Request.getUrl('scds/project/list').json().
notify(this, '_projectsLoaded', store, query).
send();
} else if (query === Scds.TASKS_QUERY) {
console.log('tasks query');
}
return YES; // return YES if you handled the query
},
_projectsLoaded: function(response, store, query) {
console.log('projects loaded....');
if (SC.ok(response)) {
var recordType = query.get('recordType'),
records = response.get('body');
store.loadRecords(recordType, records);
store.dataSourceDidFetchQuery(query);
Scds.Statechart.sendEvent('projectsLoaded')
} else {
console.log('oops...error loading projects');
// Tell the store that your server returned an error
store.dataSourceDidErrorQuery(query, response);
}
}
This will get the Projects, but not the tasks. Sproutcore knows that as soon as I access the tasks array on a Project, it needs to get them. What it does is call retrieveRecords in the datasource. That method in turn calls retrieveRecord for every id in the tasks array. My retrieveRecord method looks like
retrieveRecord: function(store, storeKey) {
var id = Scds.store.idFor(storeKey);
console.log('retrieveRecord called with [storeKey, id] [%#, %#]'.fmt(storeKey, id));
SC.Request.getUrl('scds/task/get/%#'.fmt(id)).json().
notify(this, "_didRetrieveRecord", store, storeKey).
send();
return YES;
},
_didRetrieveRecord: function(response, store, storeKey) {
if (SC.ok(response)) {
console.log('succesfully loaded task %#'.fmt(response.get('body')));
var dataHash = response.get('body');
store.dataSourceDidComplete(storeKey, dataHash);
} ...
},
Note that you should use sc-gen to generate your datasource, because it provides a fairly well flushed out stub that guides you towards the methods you need to implement. It does not provide a retrieveMethods implementation, but you can provide your own if you don't want to do a single request for each child record you are loading.
Note that you always have options. If I wanted to, I could have created a Tasks query and loaded all the tasks data up front, that way I wouldn't need to go to my server when I clicked a project. So in answer to your second question, it depends. You can either load the brands when you click on the client, or you can load all the data up front, which is probably a good idea if there isn't that much data.