Selection Transform as Text/Title - data-visualization

How can I display a transform over selected points as text within a title or subtitle?
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"data": {"url": "data/stocks.csv"},
"transform": [{"filter": "datum.symbol==='GOOG'"}],
"width": 800,
"title": {
"text": "Google's stock price over time.",
"subtitle": "selected average: ???"
},
"selection": {
"interval": {
"type": "interval",
"encodings": ["x"]
}
},
"mark": "point",
"encoding": {
"x": {"field": "date", "type": "temporal"},
"y": {"field": "price", "type": "quantitative"}
}
}
A sum transformation does not lend itself to any graphical/visual representations. An idea is to have it as a selection tooltip, but that is not possible. So I settle for a subtitle.

I don't think there is an easy way to insert a computed value within a subtitle, but you can achieve the same effect using a layer chart with a text mark that contains a computed value.
For example (open in editor):
{
"data": {"url": "data/stocks.csv"},
"transform": [{"filter": "datum.symbol==='GOOG'"}],
"width": 800,
"title": {"text": "Google's stock price over time."},
"layer": [
{
"transform": [
{"filter": {"selection": "interval"}},
{"aggregate": [{"op": "sum", "field": "price", "as": "total_price"}]}
],
"mark": "text",
"encoding": {
"x": {"value": 400},
"y": {"value": -10},
"text": {"field": "total_price", "type": "quantitative"}
}
},
{
"mark": "point",
"encoding": {
"x": {"field": "date", "type": "temporal"},
"y": {"field": "price", "type": "quantitative"}
},
"selection": {"interval": {"type": "interval", "encodings": ["x"]}}
}
]
}
Note that the text layer has x and y encoding values specified in pixels from the upper left of the chart.

I was going to say there are two possible approaches, but then #jakevdp post his answer, so I guess there are now three.
Use an ExprRef in title.subtitle. While that isn't explicitly supported in the documentation it doesn't make sense that only title.text would support an ExprRef. So I gave it a shot, and it works. Though the Vega editor will still raise schema validation warnings. I have a more involved example below using binned data.
Filter and aggregate the selection. This will modify the current data stream and Vega-Lite doesn't support more than one data stream (unless using layers), so you'll have to patch the generated Vega.
Access the text mark of the title using title.encode.subtitle.update.text. Once again, necessary to patch the generated Vega. (This was on the assumption that a subtitle couldn't be an ExprRef).
The neat thing about (2) is that it allows adding a text mark to the selection mark, so the text could be made to follow the selection. That said that's possible with #jakevdp's answer as well, and his answer is much simpler.
JavaScript loader ensuring that nothing runs out-of-order:
%%javascript
function loadjs(ls, o) {
let i = 0;
let b = JSON.parse(document.querySelector('#jupyter-config-data').text)["baseUrl"];
ls = ls.reduce((a,l) => {
if (!l.path)
return a;
let p = l.path;
if (l.local)
p = "/" + [b,"/files",p].map(s => s.replace(/^\/+|\/+$/, "")).filter(i => i).join("/");
if (document.querySelector("script[src='" + p + "']"))
return a;
return [...a, p];
}, [])
function load() {
if (i >= ls.length)
return o();
let t = document.createElement("script");
[t.type, t.src, t.onload] = ["text/javascript", ls[i], (i+1>=ls.length) ? o : load];
i = i+1;
document.head.appendChild(t);
}
return load();
}
window.loadjs = loadjs
JavaScript loader for the JupyterLab cell as an IPython magic:
import IPython
import IPython.core.magic as ipymagic
#ipymagic.magics_class
class LoadJSMagics(ipymagic.Magics):
#ipymagic.cell_magic
def loadjs(self, line, cell):
js = f"loadjs({line}, () => {{\n{cell}\n}});"
return IPython.display.Javascript(js)
IPython.get_ipython().register_magics(LoadJSMagics)
Jinja templating for JupyterLab cells as an IPython magic:
import jinja2
import IPython
import IPython.core.magic as ipymagic
#ipymagic.magics_class
class JinjaMagics(ipymagic.Magics):
#ipymagic.cell_magic
def jinja(self, line, cell):
t = jinja2.Template(cell)
r = t.render({k:v for k,v in self.shell.user_ns.items() if k not in self.shell.user_ns_hidden})
IPython.get_ipython().run_cell(r)
#d = getattr(IPython.display, line.strip(), IPython.display.display)
#return d(r)
IPython.get_ipython().register_magics(JinjaMagics)
Generate some example temporal data in Pandas:
import pandas as pd
import numpy as np
c1 = np.random.randint(1,6, size=15)
c2 = pd.date_range(start="2021-01-01",end="2021-01-15")
df = pd.DataFrame({"day": c2, "value": c1})
df = df.drop([2, 5,6,7,13])
df
The necessary imports:
# Convert the Pandas dataframe to a format suitable for Vega-Lite.
import altair
# Tag Vega-Embed div's with UUIDs ensuring the correct div is targeted.
import uuid
import json
vega_libs =\
[ {"path": "https://cdn.jsdelivr.net/npm/vega#5"}
, {"path": "/libs/vega-lite#4-fix.js", "local": True}
, {"path": "https://cdn.jsdelivr.net/npm/vega-embed#6"}
]
The Vega-Lite:
s =\
{ "title":
{ "text": "Daily Counts"
, "subtitle": {"expr": "selectionSum(data('interval_store'), data('data_0'))"}
, "subtitleFont": "monospace"
}
, "mark": "bar"
, "encoding":
{ "x":
{ "type": "temporal"
, "bin": "binned"
, "field": "start"
, "axis": { "tickCount": "day" }
}
, "x2": {"field": "end"}
, "y": {"type": "quantitative", "field": "value"}
}
, "selection":
{ "interval":
{ "type": "interval"
, "encodings": ["x"]
}
}
, "transform":
[ # Convert 'day' from 'string' to timestamp ('number')
{"calculate": "toDate(datum.day)", "as": "day"}
# Provide "start" and "end" as Date objects to match the
# type of temporal domain objects
, {"calculate": "timeOffset('hours', datum.day, -12)", "as": "start"}
, {"calculate": "timeOffset('hours', datum.day, 12)", "as": "end"}
]
, "height": 250
, "width": "container"
, "$schema": "https://vega.github.io/schema/vega-lite/v4.json"
, "config": {"customFormatTypes": "True"}
, "data": altair.utils.data.to_values(df)
}
And finally running the Vega-Lite:
%%jinja
%%loadjs {{json.dumps(vega_libs)}}
{% set visid = uuid.uuid4() %}
element.innerHTML = `
<style>.vega-embed.has-actions {width:90%}</style>
<div id="vis-{{visid}}"></div>
`
var spec = {{json.dumps(s)}}
vega.expressionFunction("selectionSum", function(selection, data) {
var view = this.context.dataflow;
function intersects(i1, i2) {
return (i1[1] >= i2[0] && i1[0] <= i2[1]);
}
function cmp_interval_pt(i0, i1, p) {
if (i1 < p)
return -1;
if (i0 > p)
return 1;
return 0;
}
function cmp_primitive(a,b) {
if (a < b)
return -1
if (a > b)
return 1;
return 0;
}
function bisect_left(l, v, fc=cmp_primitive) {
return _bisect_left(l, 0, l.length, v, fc);
}
function _bisect_left(l, l0, l1, v, fc) {
if (l1 <= l0)
return l0;
var i = Math.floor((l0+l1)/2);
var c = fc(l[i], v);
if (c < 0)
l0 = i + 1;
else
l1 = i;
return _bisect_left(l, l0, l1, v, fc);
}
function bisect_right(l, v, fc=cmp_primitive) {
return _bisect_right(l, 0, l.length, v, fc);
}
function _bisect_right(l, l0, l1, v, fc) {
if (l1 <= l0)
return l0;
var i = Math.floor((l0+l1)/2);
var c = fc(l[i], v);
if (c <= 0)
l0 = i + 1;
else
l1 = i;
return _bisect_right(l, l0, l1, v, fc);
}
function cmp_data(lv, v) {
return cmp_interval_pt(lv.start, lv.end, v);
}
function constant_len_digits(s,l) {
return " ".repeat(Math.max(0, l-s.toString().length)) + s
}
if (selection.length) {
var r = selection[0]["values"][0];
var d0 = bisect_left(data, r[0], cmp_data);
var d1 = bisect_right(data, r[1], cmp_data);
var s = data.slice(d0,d1).reduce((a,v)=>a+v.value, 0);
}
else
var s = 0
return `selected: ${constant_len_digits(s,3)}`;
});
vegaEmbed('#vis-{{visid}}', spec).then(function(result) {
}).catch(console.error);
And the result:
Note that when it comes to coloring and selecting binned regions, you'll have to calculate the intersections yourself.

Related

Karate - filter a specific json key from response based on another static array

I have the following JSON response (reference name: "list") and
[
{
"key": "101",
"val": {
"portCall": {
"id": 12664978
},
"status": "in-port"
}
},
{
"key": "102",
"val": {
"portCall": {
"id": 12415798
},
"status": "in-port"
}
},
{
"key": "103",
"val": {
"status": "on-voyage",
"voyage": {
"id": "7kynv-7lq85"
}
}
},
{
"key": "104",
"val": {
"status": "on-voyage",
"voyage": {
"id": "7kynv-2385"
}
}
}
]
also, I have an array list of few key values, evImos = [101,102,104]
In that, I have to identify the first key in the "list" response that has status as "on-voyage". So, the result should be "104".
I have tried the following and I need some help to make it work. Any help would be appreciated.
* def function getFirst = function(evImos) { for (let num of evImos) { let results = list.filter(d => d["key"] === num && d["val"]["status"] === "on-voyage"); if(results.length === 1) { karate.log(num); return num; } } }
* list.forEach(getFirst(evImos))
I'll just give you one hint. This one line will convert the whole thing in to a form that is much easier for you to validate:
* def temp = {}
* list.forEach(x => temp[x.key] = x.val.status)
Which gives you:
{
"101": "in-port",
"102": "in-port",
"103": "on-voyage",
"104": "on-voyage"
}
Now you can do:
* def isOnVoyage = function(key){ return temp[key] == 'on-voyage' }
Also read this: https://stackoverflow.com/a/59162760/143475
Thanks, to #Peter.
Based on his hint, I just tweaked it a little bit to match my requirement and it worked for me.
Here is the working copy for anyone to refer in the future.
* def temp = {}
* list.forEach(x => temp[x.key] = x.val.status)
* def isOnVoyage = function(keys){ for (let key of keys) { if(temp[key] == 'on-voyage'){ karate.log(key); karate.set('num', key); break; }}}
* isOnVoyage(evImos)

Getting the maximum value from an array in a JSON response in Karate

I have the following Json as a response from a API call
{
"location": {
"name": "London",
"region": "City of London, Greater London",
"country": "United Kingdom",
"lat": 51.52,
"lon": -0.11,
"tz_id": "Europe/London",
"localtime_epoch": 1583594426,
"localtime": "2020-03-07 15:20"
},
"forecast": {
"forecastday": [
{
"date": "2020-03-03",
"day": {
"maxtemp_c": 9,
"mintemp_c": 4
}
},
{
"date": "2020-03-04",
"day": {
"maxtemp_c": 8,
"mintemp_c": 4.1
}
},
{
"date": "2020-03-05",
"day": {
"maxtemp_c": 7,
"mintemp_c": 5.6
}
}
]
}
}
I want to find out which date had the highest temperature amongst the 3 days.
The way I am currently doing feels inefficient as I am checking for the temperature element within my js function and it is as follows
* def hottest =
"""
function(array) {
var greatest;
var indexOfGreatest;
for (var i = 0; i < array.length; i++) {
if (!greatest || array[i].day.maxtemp_c > greatest) {
greatest = array[i].day.maxtemp_c;
indexOfGreatest = i;
}
}
return indexOfGreatest;
}
"""
* def index = call hottest response.forecast.forecastday
* def hottestdate = response.forecast.forecastday[index].date
* print hottestdate
With this I am getting the correct result but can someone kindly suggest a better way of doing this?
Best practice in Karate is to NOT use JS for loops at all. It results in cleaner, more readable code:
* def fun = function(x){ return { max: x.day.maxtemp_c, date: x.date } }
* def list = karate.map(response.forecast.forecastday, fun)
* def max = 0
* def index = 0
* def finder =
"""
function(x, i) {
var max = karate.get('max');
if (x.max > max) {
karate.set('max', x.max);
karate.set('index', i);
}
}
"""
* karate.forEach(list, finder)
* print 'found at index', index
* print 'item:', list[index]
Note how easy it is to re-shape a given JSON, the result of list here would be:
[
{
"max": 9,
"date": "2020-03-03"
},
{
"max": 8,
"date": "2020-03-04"
},
{
"max": 7,
"date": "2020-03-05"
}
]

How could I render a ExtWebComponent Area Chart using Alpha Vantage stock data?

I would like to use Alpha Vanatage stock data in my ExtWebComponent chart. How could I fetch the data and render it in a Cartesian Area chart?
If you've generated an ExtWebComponents project, you could add these 2 files and declare the web component html element tag.
Usage
In the html file like index.html declare my-chart-area which is defined in the web component below.
AreaChartComponent.html - HTML Template
<ext-cartesian
width="1000px"
height="600px"
downloadServerUrl="http://svg.sencha.io"
shadow="true"
insetPadding="25 35 0 10"
axes='[{
"type": "numeric" ,
"position": "left" ,
"fields": [ "1. open" ],
"label": { "rotate": { "degrees": "-30" } },
"grid": { "odd": { "fill": "#e8e8e8" } },
"title": { "text": "Alphabet Inc Stock Data" , "fontSize": "20" }
},
{
"type": "category",
"position": "bottom",
"fields": "time",
"grid": "true",
"title": { "text": "Monthly", "fontSize": "20" }
}]'
legend='{
"type": "sprite",
"position": "bottom"
}'
series='[{
"type": "area" ,
"xField": "time",
"yField": [ "1. open", "2. high", "3. low", "4. close" ],
"title": [ "open", "high", "low", "close" ],
"style": { "stroke": "black" , "lineWidth": "2", "fillOpacity": "0.8" },
"colors": ["#003f5c", "#58508d", "#bc5090", "#ff6361", "#ffa600"]
}]'
platformConfig='{
"phone": { "insetPadding": "15 5 0 0" }
}'>
</ext-cartesian>
AreaChartComponent.js - Web Component
import template from './AreaChartComponent.html'
Ext.require([
'Ext.chart.theme.Midnight',
'Ext.chart.theme.Green',
'Ext.chart.theme.Muted',
'Ext.chart.theme.Purple',
'Ext.chart.theme.Sky',
'Ext.chart.series.Area',
'Ext.chart.axis.Numeric',
'Ext.chart.axis.Category'
]);
class AreaChartComponent extends HTMLElement {
constructor() {
super()
}
connectedCallback() {
this.innerHTML = template;
this._fetchChartData();
}
disconnectedCallback() {
}
attributeChangedCallback(attrName, oldVal, newVal) {
}
/**
* Fetch the chart data from https://www.alphavantage.co/ using an API Key.
*
* TODO Fetch your api key here: https://www.alphavantage.co/support/#api-key
*/
_fetchChartData() {
let me = this;
let apiKey = 'demo';
let stockSymbol = 'GOOGL';
let url = `https://www.alphavantage.co/query?function=TIME_SERIES_MONTHLY&symbol=${stockSymbol}&apikey=${apiKey}`;
fetch(url)
.then(response => {
return response.json();
})
.then(json => {
return me._flattenData(json);
})
.then(jsonflatRows => {
me._renderChart(jsonflatRows);
})
.catch(err => {
console.log("error", err);
})
}
/**
* The goal is to flatten the nested json data, so it's easy to consume in the charts.
* #param json data
* #returns {*[]} array of json data
* #private
*/
_flattenData(json) {
console.log("json=", json);
let jsonTimes = json['Monthly Time Series']
let flatRows = [];
for (let jsonTime in jsonTimes) {
let row = {
"time": jsonTime
};
let jsonNestedTime = jsonTimes[jsonTime];
for (let nestedKey in jsonNestedTime) {
row[nestedKey] = jsonNestedTime[nestedKey];
}
flatRows.push(row);
}
return flatRows.reverse();
}
_renderChart(jsonflatRows) {
console.log('_renderChart jsonflatRows=', jsonflatRows);
let store = Ext.create('Ext.data.Store', {
fields: ["time", "1. open", "2. high", "3. low", "4. close", "5. volume"]
});
store.loadData(jsonflatRows);
let areaChartEl = this.querySelector('ext-cartesian');
areaChartEl.ext.bindStore(store);
}
}
window.customElements.define('my-chart-area', AreaChartComponent);
Source
https://gist.github.com/branflake2267/4652a5d7188dfe0b33d3d02a808d8d74

data.tables is empty - community visualization in DataStudio

I'm trying to draw custom graph with Google DataStudio community visualization and BigQuery source.
But even if data is exist and valid(check with other basic chart), input data of my drawViz function is empty.
See below Javascript code:
function drawViz(data) {
let rowData = data.tables.DEFAULT;
var metricName = data.fields['barMetric'][0].name;
var dimensionName = data.fields['barDimension'][0].name;
title = metricName + ' by ' + dimensionName + '2';
console.log(rowData , title )
}
Console output:
> {DEFAULT : Array(0)} "my metrics by my dimension"
Is there any restriction using community visualization functionality with bigquery?
Or need any additional setting except in codelab (https://codelabs.developers.google.com/codelabs/community-visualization/#0) ?
** update
manifest.json : https://storage.googleapis.com/vd-qoe-bucket/test/manifest.json
myViz.json : https://storage.googleapis.com/vd-qoe-bucket/test/myViz.json
From your links:
The "data" part of your config appears to be invalid:
"data": [
{
"id": "concepts",
"label": "Concepts",
"elements": [ // setting metric and dimension counts
{
"id": "barDimension",
"label": "Dimension",
"type": "DIMENSION",
"options": {
"min": 1,
"max": 2
}
},
{
"id": "barMetric",
"label": "Metric",
"type": "METRIC",
"options": {
"min": 1,
"max": 2
}
}
]
}
]
Removing the comment // setting dimensions... should work.

Extract data from json array in Karate

In the below JSON response, I need to extract the 'cid' for the record that has the 'nationalityDecription' as 'USA'. By using this query as a reference, I used the below loc in the karate feature file, but 1st line itself fails with syntax error(tried different combinations). For now, I'm using the custom javascript as a workaround which is working fine. I need help to check if i'm missing anything in syntax. Thanks
Response:
{
"header": {
"Id": "12345678",
"timeStamp": "2018-09-17T10:09:812.000"
},
"dataRecords": [
{
"cid": "31H678",
"cidMeta": "00",
"nationalityDecription": "CHINA"
},
{
"cid": "31S421",
"cidMeta": "01",
"nationalityDecription": "USA"
}
]
}
Feature file:
* def record= $response.dataRecords[?(#.nationalityDecription=='USA')]
* def cid = record.cid
* def response = { "header": { "Id": "12345678", "timeStamp": "2018-09-17T10:09:812.000" }, "dataRecords": [ { "cid": "31H678", "cidMeta": "00", "nationalityDecription": "CHINA" }, { "cid": "31S421", "cidMeta": "01", "nationalityDecription": "USA" } ] }
* def cid = get[0] response.dataRecords[?(#.nationalityDecription=='USA')].cid
* match cid == '31S421'