How to fetch isolation source and sequence for subsequence in C3.ai COVID-19 Data Lake? - c3ai-datalake

I'm running API calls to the C3.ai COVID-19 Data Lake in Postman. I'm using Fetch on the Subsequence type, and want to get the isolationSource for the BiologicalAsset that is related to the Subsequence. But I can't figure out how to access this field properly. I'm running POST on https://api.c3.ai/covid/api/1/subsequence/fetch. Here's the request body:
{
spec: {
include: "startIndex, endIndex, sequence, sequence.isolationSource",
limit: 10
}
}
But then each object in the resulting XML looks like this:
<k>0</k>
<v>
<startIndex>1</startIndex>
<endIndex>182</endIndex>
<id>LC522350_1-182</id>
<meta>
<fetchInclude>[startIndex,endIndex,sequence,id,version]</fetchInclude>
<fetchType>Subsequence</fetchType>
</meta>
<version>1</version>
</v>
This doesn't properly show the sequence or isolation source. How do I find these?

Based on the API docs, you need to use "parent" to refer to the Sequence associated with a Subsequence, and then use that parent's parent to refer to a BiologicalAsset. That would make your request:
{
spec: {
include: "startIndex, endIndex, parent.sequence, parent.parent.isolationSource",
limit: 10
}
}
I just tried this and noticed that some Subsequences don't have isolation sources. If you want to only include results that have an isolation source, you would add filter: "exists(parent.parent.isolationSource)" to your spec.

Related

How to avoid rails as_json creating flood of database requests

I have a model questions that has many tags through another model question_tags. I am trying to write an api endpoint that returns a list of questions and for each question a list of it's related tag's name and id.
I'm currently using;
render json: #questions.as_json(include: {
tags: { only: %i[id name] }
})
But this runs a separate db request for every question in questions is there an alternative that will result in a smaller number of database requests?
The standard N+1 queries fixes should work here, which in Rails is eager loading.
In this case you can use include to preload the associated tags:
render json: #questions.includes(:tags).as_json(
include: {
tags: { only: %i[id name] }
}
)
You can read more about this in the dedicated rails guide section: Eager Loading Associations

Merging data from different graphql resolvers in vue.js client side for simple outputting

I do query cars from an api with a single query but two resolvers (listing and listings)(hopefully resolver is the right name for it). One car I get by the id via listing and the other cars I get without filters by listings. The resolvers output the data i a little different structure on the server-side but I do get the same fields just at different „places“. I want to merge the structure in order to get a single array I can simply loop over in vue.js. For the apicalls I do use vue-apollo.
Couldn't find any information to merge data client-side inside graphqlqueries. All I found is about handling it serverside with resolvers but it's an api I do not own.
Is it possible with graphql or do I have to merge it inside my vuecomponent and if so what would be the best way to do so?
The output will be a grid of cars where I show the car of the week (requested by id) together with the newest cars of the regarding cardealer.
Full screenshot including response: https://i.imgur.com/gkCZczY.png
Stripped down example with just the id to show the problem:
query CarTeaser ($guid: String! $withVehicleDetails: Boolean!) {
search {
listing(guid: $guid){
details{
identifier{
id #for example: here I get the id under details->identifier
}
}
}
listings( metadata: { size: 2 sort:{ field: Age order: Asc}}) {
listings{
id #here it's right under listings
details{
…
}
}
}
}
}
}
Ideally you're right, it should be handled server-side, but if it's not your API the only solution is to manipulate the data on the client side, meaning in your component.
It's probably a lot simpler to leave the listings array untouched and to just merge the listing element with it, like this for instance:
// assuming 'search' holds the entire data queried from the api
const fullListing = [
// car of the week, data reformatted to have an identical structure as
// the 'other' cars
{
id: search.listing.details.identifier.id,
details: {
vehicle: search.listing.details.vehicle,
},
},
...search.listings.listings, // the 'other' cars
]

Backendless returns no paging data

According to API documentation here https://backendless.com/documentation/data/rest/data_search_and_query.htm , backendless shall provide paging info in response body like this
{
"nextPage":null,
"data":[
{
"updated":null,
"created":"02/05/2014 18:13:40 GMT+0000",
"ownerId":null,
"objectId":"6FAF3CE5-6F55-1B32-FF83-D333252D0300",
"name":"Bob",
"age":20
},
{
"updated":null,
"created":"02/04/2014 19:40:10 GMT+0000",
"ownerId":null,
"objectId":"28325E9F-2DED-D3CA-FFC6-C76911AFBB00",
"name":"Frank",
"age":26
}],
"offset":0,
"totalObjects":2
}
However, when I send request to get data from table like this:
https://api.backendless.com/<version>/data/<table-name>
it returns only collection of objects (which should be in "data") and now paging info.
Adding page requests:
https://api.backendless.com/<version>/data/<table-name>?pageSize=10&offset=10
returns correct data, but still without paging info.
What I'm doing wrong? How can I access paging info?
The docs link you mention is for Backendless version which is being deprecated (there's a warning on the top of the page). Here is a link to the docs for the current version: https://backendless.com/docs/rest/doc.html#data_basic_search
Indeed, in version 4 we got rid of that additional information in the response to simplify the response object in non-REST clients. But still you can easily determine current params since you're specifying them in the request (or they're zeros in case you're not).

Unable to use Ember data with JSONAPI and fragments to support nested JSON data

Overview
I'm using Ember data and have a JSONAPI. Everything works fine until I have a more complex object (let's say an invoice for a generic concept) with an array of items called lineEntries. The line entries are not mapped directly to a table so need to be stored as raw JSON object data. The line entry model also contains default and computed values. I wish to store the list data as a JSON object and then when loaded back from the store that I can manipulate it as normal in Ember as an array of my model.
What I've tried
I've looked at and tried several approaches, the best appear to be (open to suggestions here!):
Fragments
Replace problem models with fragments
I've tried making the line entry model a fragment and then referencing the fragment on the invoice model as a fragmentArray. Line entries add to the array as normal but default values don't work (should they?). It creates the object and I can store it in the backend but when I return it, it fails with either a normalisation issue or a serialiser issue. Can anyone state the format the data be returned in? It's confusing as normalising the data seems to require JSONAPI but the fragment requires JSON serialiser. I've tried several combinations but no luck so far. My line entries don't have actual ids as the data is saved and loaded as a block. Is this an issue?
DS.EmbeddedRecordsMixin
Although not supported in JSONAPI, it sounds possible to use JSONAPI and then switch to JSONSerializer or RESTSerializer for the problem models. If this is possible could someone give me a working example and the JSON format that should be returned by the API? I have header authorisation and other such data so would I still be able to set this at the application level for all request not using my JSONAPI?
Ember-data-save-relationships
I found an add on here that provides an add on to do this. It seems more involved than the other approaches but when I've tried this I can send the data up by setting a the data as embedded. Great! But although it saves it doesn't unwrap it correct and I'm back with the same issues.
Custom serialiser
Replace the models serialiser with something that takes the data and sends it as plain JSON data and then deserialises back into something Ember can use. This sounds similar to the above but I do the heavy lifting. The only reason to do this is because all examples for the above solutions are quite light and don't really show how to set this up with an actual JSONAPI set up that would need it.
Where I am and what I need
Basically all approaches lead to saving the JSON fine but the return JSON from the server not being the correct format or the deserialisation failing but it's unclear what it should be or what needs to change without breaking the existing JSONAPI models that work fine.
If anyone know the format for return API data it may resolve this. I've tried JSONAPI with lineEntries returning the same format as it saved. I've tried placing relationship sections like the add on suggested and I've also tried placing relationship only data against the entries and an include section with all the references. Any help on this would be great as I've learned a lot through this but deadlines a looming and I can't see a viable solution that doesn't break as much as it fixes.
If you are looking for return format for relational data from the API server you need to make sure of the following:
Make sure the relationship is defined in the ember model
Return all successes with a status code of 200
From there you need to make sure you return relational data correctly. If you've set the ember model for the relationship to {async: true} you need only return the id of the relational model - which should also be defined in ember. If you do not set {async: true}, ember expects all relational data to be included.
return data with relationships in JSON API specification
Example:
models\unicorn.js in ember:
import DS from 'ember-data';
export default DS.Model.extend({
user: DS.belongsTo('user', {async: true}),
staticrace: DS.belongsTo('staticrace',{async: true}),
unicornName: DS.attr('string'),
unicornLevel: DS.attr('number'),
experience: DS.attr('number'),
hatchesAt: DS.attr('number'),
isHatched: DS.attr('boolean'),
raceEndsAt: DS.attr('number'),
isRacing: DS.attr('boolean'),
});
in routes\unicorns.js on the api server on GET/:id:
var jsonObject = {
"data": {
"type": "unicorn",
"id": unicorn.dataValues.id,
"attributes": {
"unicorn-name" : unicorn.dataValues.unicornName,
"unicorn-level" : unicorn.dataValues.unicornLevel,
"experience" : unicorn.dataValues.experience,
"hatches-at" : unicorn.dataValues.hatchesAt,
"is-hatched" : unicorn.dataValues.isHatched,
"raceEndsAt" : unicorn.dataValues.raceEndsAt,
"isRacing" : unicorn.dataValues.isRacing
},
"relationships": {
"staticrace": {
"data": {"type": "staticrace", "id" : unicorn.dataValues.staticRaceId}
},
"user":{
"data": {"type": "user", "id" : unicorn.dataValues.userId}
}
}
}
}
res.status(200).json(jsonObject);
In ember, you can call this by chaining model functions. For example when this unicorn goes to race in controllers\unicornracer.js:
raceUnicorn() {
if (this.get('unicornId') === '') {return false}
else {
return this.store.findRecord('unicorn', this.get('unicornId', { backgroundReload: false})).then(unicorn => {
return this.store.findRecord('staticrace', this.get('raceId')).then(staticrace => {
if (unicorn.getProperties('unicornLevel').unicornLevel >= staticrace.getProperties('raceMinimumLevel').raceMinimumLevel) {
unicorn.set('isRacing', true);
unicorn.set('staticrace', staticrace);
unicorn.set('raceEndsAt', Math.floor(Date.now()/1000) + staticrace.get('duration'))
this.set('unicornId', '');
return unicorn.save();
}
else {return false;}
});
});
}
}
The above code sends a PATCH to the api server route unicorns/:id
Final note about GET,POST,DELETE,PATCH:
GET assumes you are getting ALL of the information associated with a model (the example above shows a GET response). This is associated with model.findRecord (GET/:id)(expects one record), model.findAll(GET/)(expects an array of records), model.query(GET/?query=&string=)(expects an array of records), model.queryRecord(GET/?query=&string=)(expects one record)
POST assumes you at least return at least what you POST to the api server from ember , but can also return additional information you created on the apiServer side such as createdAt dates. If the data returned is different from what you used to create the model, it'll update the created model with the returned information. This is associated with model.createRecord(POST/)(expects one record).
DELETE assumes you return the type, and the id of the deleted object, not data or relationships. This is associated with model.deleteRecord(DELETE/:id)(expects one record).
PATCH assumes you return at least what information was changed. If you only change one field, for instance in my unicorn model, the unicornName, it would only PATCH the following:
{
data: {
"type":"unicorn",
"id": req.params.id,
"attributes": {
"unicorn-name" : "This is a new name!"
}
}
}
So it only expects a returned response of at least that, but like POST, you can return other changed items!
I hope this answers your questions about the JSON API adapter. Most of this information was originally gleamed by reading over the specification at http://jsonapi.org/format/ and the ember implementation documentation at https://emberjs.com/api/data/classes/DS.JSONAPIAdapter.html

GitHub API (v3): Order tags by creation date

I ran into a problem / question while using the GitHub API.
I need a list of all tags created after a single tag. The only way to do this, is to compare the tags by date. However, the results from the API aren't ordered by date:
Result from the API (rails repository example):
Results from the webinterface:
What i did expect is a list ordered by date. However, as you can see in the pictures: the API is returning v4.0.0rc1 & v4.0.0rc2 before the release of v4.0.0, while 4.0.0 is released after the release candidates. There isn't even a creation / commit date to order at server side.
The releases API isn't a solution either. This API is only returning releases created by Github, not the releases created by tags.
Is there any way to order the tags by date?
Thanks in advance!
Ruben
The Repositories API currently returns tags in the order they would be returned by the "git tag" command, which means they are alphabetically sorted.
The problem with sorting tags chronologically in Git is that there are two types of tags, lightweight and annotated), and for the lightweight type Git doesn't store the creation date.
The Releases/Tags UI currently sorts tags chronologically by the date of the commit to which the tag points to. This again isn't the date on which the tag itself was created, but it does establish a chronological order of things.
Adding this alternative sorting option to the API is on our feature request list.
With GraphQL API v4, we can now filter tags by commit date with field: TAG_COMMIT_DATE inside orderBy. The following will perform ascending sort of tags by commit date :
{
repository(owner: "rails", name: "rails") {
refs(refPrefix: "refs/tags/", last: 100, orderBy: {field: TAG_COMMIT_DATE, direction: ASC}) {
edges {
node {
name
target {
oid
... on Tag {
message
commitUrl
tagger {
name
email
date
}
}
}
}
}
}
}
}
Test it in the explorer
Here, the tagger field inside target will only be filled for annotated tag & will be empty for lightweight tags.
As date property in tagger gives the creation date of the tag (for annotated tag only), it's possible to filter by creation date on the client side easily (without having to retrieve all the tags 1 by 1)
Note that available options for orderBy.field at this time are TAG_COMMIT_DATE & ALPHABETICAL (no TAG_CREATION_DATE)
Edit: This is now possible using the GitHub GraphQL API.
As workaround, there is a node module for this,
which basically fetches the commit details of each tag:
github-api-tags-full
> npm install github-api-tags-full github moment
var GitHubApi = require('github'),
moment = require('moment'),
githubTags = require('github-api-tags-full');
var github = new GitHubApi({
version: '3.0.0'
});
githubTags({ user: 'golang', repo: 'go' }, github)
.then(function(tags) {
var tagsSorted = tags.sort(byAuthorDateAsc).reverse(); // descending
console.log(tagsSorted); // prints the array of tags sorted by their creation date
});
var byAuthorDateAsc = function(tagA, tagB) {
return githubCompareDates(
tagA.commit.author.date,
tagB.commit.author.date
);
};
var githubCompareDates = function(dateStrA, dateStrB) {
return moment(dateStrA).diff(dateStrB);
};
With best regards
You can use the Git References API.
This can return also all the tags matching a certain prefix.
In you case, you probably want something like:
https://api.github.com/repos/rails/rails/git/matching-refs/tags/v
Or in the case of a monorepo:
https://api.github.com/repos/grafana/loki/git/matching-refs/tags/helm-loki-
Downsides:
sorting: the results are sorted in increasing semver order and you will get the oldest first.
you don't get much info about each tag and you might have to parse the versions out of the ref name/path
Upside
you get all the ref/tags that match (i.e. no pagination, until GitHub decides to remove/optimise this :) )
you can use it to filter out tags in a monorepo (that most probably tag release components with prefixed tags)