I would like to use Bacon to combine time series with irregular timestamps in a single EventStream. Each event would contain the last value of each time series at the given time.
Here is a an example
var ts1 = new Bacon.fromArray([
[1,2], //Each array is an event,i.e here timestamp is 1 and value is 2
[2,3],
[5,9]
])
var ts2 = new Bacon.fromArray([
[4,2],
[9,3],
[12,9]
])
What I would like to have is something like this
var ts12 =[
[1,2,undefined], //At time 1, only ts1 was defined
[2,3,undefined],
[4,3,2], //at time 4, we take the last value of ts1 (3) and ts2 (2)
[5,9,2],
[9,9,3],
[12,9,9],
]
I tried to implement it using Bacon.update but I didn't go very far. How would you approach the problem?
I make the assumption that your time series values arrive as values in the order of the time parameter. Hence the change to Bacon.later in my code.
var ts1 = new Bacon.mergeAll([
Bacon.later(100, [1,2]),
Bacon.later(200, [2,3]),
Bacon.later(500, [5,9])
])
var ts2 = new Bacon.mergeAll([
Bacon.later(400, [4,2]),
Bacon.later(900, [9,3]),
Bacon.later(1200, [12,9])
])
var ts12 = ts1.toProperty(null).combine(ts2.toProperty(null), function(v1, v2) {
if(v1 && v2) {
return [Math.max(v1[0], v2[0]), v1[1], v2[1]]
} else if(v1) {
return [v1[0], v1[1], undefined]
} else if(v2) {
return [v2[0], undefined, v2[1]]
}
}).changes()
ts12.log()
You can play around with the solution in this JSfiddle.
Related
I struggle accessing the row-elements of a Frame.
One idea I have is to filter the dataframe down to a row, convert it to a vec or something similar and access the elements this way ?!
In Panadas I used to just use ".at / .loc / .iloc / etc."; with Polars in Rust I have no clue.
Any suggestions on what the proper way to do this is ?
Thanks to #isaactfa ... he got me onto the right track. I ended up getting the row not with "get_row" but rather with "get" ... this is probably due to my little RUST understanding (my 2nd week).
Here is a working code sample:
use polars::export::arrow::temporal_conversions::date32_to_date;
use polars::prelude::*;
fn main() -> Result<()> {
let days = df!(
"date_string" => &["1900-01-01", "1900-01-02", "1900-01-03", "1900-01-04", "1900-01-05",
"1900-01-06", "1900-01-07", "1900-01-09", "1900-01-10"])?;
let options = StrpTimeOptions {
date_dtype: DataType::Date, // the result column-datatype
fmt: Some("%Y-%m-%d".into()), // the source format of the date-string
strict: false,
exact: true,
};
// convert date_string into dtype(date) and put into new column "date_type"
// we convert the days DataFrame to a LazyFrame ...
// because in my real-world example I am getting a LazyFrame
let mut new_days_lf = days.lazy().with_column(
col("date_string")
.alias("date_type")
.str()
.strptime(options),
);
// Getting the weekday as a number:
// This is what I wanted to do ... but I get a string result .. need u32
// let o = GetOutput::from_type(DataType::Date);
// new_days_lf = new_days_lf.with_column(
// col("date_type")
// .alias("weekday_number")
// .map(|x| Ok(x.strftime("%w").unwrap()), o.clone()),
// );
// This is the convoluted workaround for getting the weekday as a number
let o = GetOutput::from_type(DataType::Date);
new_days_lf = new_days_lf.with_column(col("date_type").alias("weekday_number").map(
|x| {
Ok(x.date()
.unwrap()
.clone()
.into_iter()
.map(|opt_name: Option<i32>| {
opt_name.map(|datum: i32| {
// println!("{:?}", datum);
date32_to_date(datum)
.format("%w")
.to_string()
.parse::<u32>()
.unwrap()
})
})
.collect::<UInt32Chunked>()
.into_series())
},
o,
));
new_days_lf = new_days_lf.with_column(
col("weekday_number")
.shift_and_fill(-1, 9999)
.alias("next_weekday_number"),
);
// now we convert the LazyFrame into a normal DataFrame for further processing:
let mut new_days_df = new_days_lf.collect()?;
// convert the column to a series
// to get a column by name we need to collect the LazyFrame into a normal DataFrame
let col1 = new_days_df.column("weekday_number")?;
// convert the column to a series
let col2 = new_days_df.column("next_weekday_number")?;
// now I can use series-arithmetics
let diff = col2 - col1;
// create a bool column based on "element == 2"
// add bool column to DataFrame
new_days_df.replace_or_add("weekday diff eq(2)", diff.equal(2)?.into_series());
// could not figure out how to filter the eager frame ...
let result = new_days_df
.lazy()
.filter(col("weekday diff eq(2)").eq(true))
.collect()
.unwrap();
// could not figure out how to access ROW elements
// thus I used "get" instead af of "get_row"
// getting the date where diff is == 2 (true)
let filtered_row = result.get(0).unwrap();
// within the filtered_row get element with an index
let date = filtered_row.get(0).unwrap();
println!("\n{:?}", date);
Ok(())
}
What I need is to sort data I get from an API into different arrays but add '0' value where there is no value at 1 type but is at the other type/s. Now is this possible with array.filter since its faster then a bunch of for and if loops ?
So let's say I get following data from SQL to the API:
Day Type Amount
-----------------------
12.1.2022 1 11
12.1.2022 2 4
13.1.2022 1 5
14.1.2022 2 9
16.1.2022 2 30
If I run this code :
this.data = result.Data;
let date = [];
const data = { 'dataType1': [], 'dataType2': [], 'dataType3': [], 'dataType4': [] }
/*only writing example for 2 types since for 4 it would be too long but i desire
answer that works for any amount of types or for 4 types */
this.data.forEach(x => {
var lastAddress = date[date.length - 1]
if (x.type == 1) {dataType1.push(x.Amount) }
if (x.type == 2) {dataType2.push(x.Amount) }}
lastAddress != x.Day ? date.push(x.Day) : '';
The array I get for type1 is [11,5]
and for type2 I get [4,9,30].
And for dates i get all the unique dates.
But the data I would like is: [11,5,0,0] and [4,0,9,30]
The size of array also has to match the size of Day array at the end.
which would be unique dates.. in this case:
[12.1.2022, 13.1.2022, 14.1.2022, 16.1.2022]
I have already tried to solve this with some for, if and while loops but it gets way too messy, so I'm looking for an alternative.
Also i have 4 types but for reference i only wrote sample for 2.
you can
first get the uniq values
loop over the data to create an array of object with
{date:string,values:number[]}
//create a function:
matrix(data:any[])
{
const uniqTypes=data.reduce((a,b)=>a.indexOf(b.type)>=0?a:
[...a,b.type],[])
const result=[]
this.data.forEach((x:any)=>{
let index=result.findIndex(r=>r.date==x.date)
if (index<0)
{
result.push({date:x.date,values:[...uniqTypes].fill(0)})
index=result.length-1;
}
result[index].values[uniqTypes.indexOf(x.type)]=x.amount
})
return result
}
//and use like
result=this.matrix(this.data);
NOTE: You can create the uniqType outside the function as variable and pass as argument to the function
stackblitz
const type1 = [];
const type2 = [];
data.forEach(item=>{
if(item.type==1){
type1.push(item.amount)
type2.push(0)
}else{
type1.push(0)
type2.push(item.amount)
}
})
console.log(type1)
console.log(type2)
I'm working on an algorithm type challenge, and i am debugging via print statements and i can't seem to figure out why the the values for keys are not what i am expecting
var mapNums = mutableMapOf<Int, Int>()
//imaginary array
//var nums = [34,28,11,21,3,34,8,7,34,7,31,7,3,28,18]
var count = 0
for (n in nums) {
if (mapNums.containsKey(n)) {
count ++
mapNums[n] = count
} else if (!mapNums.containsKey(n)) {
count = 1
mapNums[n] = count
}
}
println(mapNums)
//prints {34=2, 28=4, 11=1, 21=1, 3=3, 8=1, 7=2, 31=1, 18=1}
as you can see the key and values aren't what theyre supposed to be and i am not sure why.
You can use the following code to generate the desired map:
val nums = intArrayOf(34, 28, 11, 21, 3, 34, 8, 7, 34, 7, 31, 7, 3, 28, 18).toList()
println(nums.groupingBy { it }.eachCount())
try it yourself
Here groupingBy creates a Grouping source using the same element as the key selector. Then eachCount groups elements from the Grouping source by key and counts elements in each group.
You can also refer the documentation for more info about groupingBy and eachCount.
It's because you reuse the same count variable outside of the loop so it keeps incrementing from different keys.
Instead you should get the current count from the map, then put it back one higher:
val nums = intArrayOf(34,28,11,21,3,34,8,7,34,7,31,7,3,28,18)
val mapNums = mutableMapOf<Int, Int>()
for (n in nums) {
val count = mapNums[n] ?: 0
mapNums[n] = count + 1
}
println(mapNums) // {34=3, 28=2, 11=1, 21=1, 3=2, 8=1, 7=3, 31=1, 18=1}
Firstly check n number is contain this map as key, if found then increment 1 its value using plus method. If not found any value from the map, it will null and check if null and set 1.
var mapNums = mutableMapOf<Int, Int>()
//imaginary array
var nums = arrayOf(34,28,11,21,3,34,8,7,34,7,31,7,3,28,18)
for (n in nums) {
mapNums[n] = mapNums[n]?.plus(1) ?: 1
}
println(mapNums)
In my app i use ios-charts library (swift alternative of MPAndroidChart).
All i need is to display line chart with dates and values.
Right now i use this function to display chart
func setChart(dataPoints: [String], values: [Double]) {
var dataEntries: [ChartDataEntry] = []
for i in 0..<dataPoints.count {
let dataEntry = ChartDataEntry(value: values[i], xIndex: i)
dataEntries.append(dataEntry)
}
let lineChartDataSet = LineChartDataSet(yVals: dataEntries, label: "Items count")
let lineChartData = LineChartData(xVals: dataPoints, dataSet: lineChartDataSet)
dateChartView.data = lineChartData
}
And this is my data:
xItems = ["27.05", "03.06", "17.07", "19.09", "20.09"] //String
let unitsSold = [25.0, 30.0, 45.0, 60.0, 20.0] //Double
But as you can see - xItems are dates in "dd.mm" format. As they are strings they have same paddings between each other. I want them to be more accurate with real dates. For example 19.09 and 20.09 should be very close. I know that i should match each day with some number in order to accomplish it. But i don't know what to do next - how i can adjust x labels margins?
UPDATE
After small research where i found out that many developers had asked about this feature but nothing happened - for my case i found very interesting alternative to this library in Swift - PNChart. It is easy to use, it solves my problem.
The easiest solution will be to loop through your data and add a ChartDataEntry with a value of 0 and a corresponding label for each missing date.
In response to the question in the comments here is a screenshot from one of my applications where I am filling in date gaps with 0 values:
In my case I wanted the 0 values rather than an averaged line from data point to data point as it clearly indicates there is no data on the days skipped (8/11 for instance).
From #Philipp Jahoda's comments it sounds like you could skip the 0 value entries and just index the data you have to the correct labels.
I modified the MPAndroidChart example program to skip a few data points and this is the result:
As #Philipp Jahoda mentioned in the comments the chart handles missing Entry by just connecting to the next data point. From the code below you can see that I am generating x values (labels) for the entire data set but skipping y values (data points) for index 11 - 29 which is what you want. The only thing remaining would be to handle the x labels as it sounds like you don't want 15, 20, and 25 in my example to show up.
ArrayList<String> xVals = new ArrayList<String>();
for (int i = 0; i < count; i++) {
xVals.add((i) + "");
}
ArrayList<Entry> yVals = new ArrayList<Entry>();
for (int i = 0; i < count; i++) {
if (i > 10 && i < 30) {
continue;
}
float mult = (range + 1);
float val = (float) (Math.random() * mult) + 3;// + (float)
// ((mult *
// 0.1) / 10);
yVals.add(new Entry(val, i));
}
What I did is fully feed the dates for x data even no y data for it, and just not add the data entry for the specific xIndex, then it will not draw the y value for the xIndex to achieve what you want, this is the easiest way since you just write a for loop and continue if you detect no y value there.
I don't suggest use 0 or nan, since if it is a line chart, it will connect the 0 data or bad things will happen for nan. You might want to break the lines, but again ios-charts does not support it yet (I also asked a feature for this), you need to write your own code to break the line, or you can live with connecting the 0 data or just connect to the next valid data.
The down side is it may has performance drop since many xIndex there, but I tried ~1000 and it is acceptable. I already asked for such feature a long time ago, but it took lot of time to think about it.
Here's a function I wrote based on Wingzero's answer (I pass NaNs for the entries in the values array that are empty) :
func populateLineChartView(lineChartView: LineChartView, labels: [String], values: [Float]) {
var dataEntries: [ChartDataEntry] = []
for i in 0..<labels.count {
if !values[i].isNaN {
let dataEntry = ChartDataEntry(value: Double(values[i]), xIndex: i)
dataEntries.append(dataEntry)
}
}
let lineChartDataSet = LineChartDataSet(yVals: dataEntries, label: "Label")
let lineChartData = LineChartData(xVals: labels, dataSet: lineChartDataSet)
lineChartView.data = lineChartData
}
The solution which worked for me is splitting Linedataset into 2 Linedatasets. First would hold yvals till empty space and second after emptyspace.
//create 2 LineDataSets. set1- till empty space set2 after empty space
set1 = new LineDataSet(yVals1, "DataSet 1");
set2= new LineDataSet(yVals2,"DataSet 1");
//load datasets into datasets array
ArrayList<ILineDataSet> dataSets = new ArrayList<ILineDataSet>();
dataSets.add(set1);
dataSets.add(set2);
//create a data object with the datasets
LineData data = new LineData(xVals, dataSets);
// set data
mChart.setData(data);
I'm creating a Dojo line chart from a dojo.data.ItemFileReadStore using a dojox.charting.DataSeries. I'm using the third parameter (value) of the constructor of DataSeries to specify a method which will generate the points on the chart. e.g.
function formatLineGraphItem(store,item)
{
var o = {
x: graphIndex++,
y: store.getValue(item, "fileSize"),
};
return o;
}
The graphIndex is an integer which is incremented for every fileSize value. This gives me a line chart with the fileSize shown against a numeric count. This works fine.
What I'd like is to be able to specify the x axis label to use instead of the value of graphIndex i.e. the under lying data will still be 1,2,3,4 but the label will show text (in this case the time at which the file size was captured).
I can do this by passing in an array of labels into the x asis when I call chart.addAxis() but this requires me to know the the values before I iterate through the data. e.g.
var dataSeriesConfig = {query: {id: "*"}};
var xAxisLabels = [{text:"2011-11-20",value:1},{text:"2011-11-21",value:2},{text:"2011-11-22",value:3}];
var chart1 = new dojox.charting.Chart("chart1");
chart1.addPlot("default", {type: "Lines", tension: "4"});
chart1.addAxis("x", {labels: xAxisLabels});
chart1.addAxis("y", {vertical: true});
chart1.addSeries("Values", new dojox.charting.DataSeries(dataStore, dataSeriesConfig, formatLineGraphItem));
chart1.render();
The xAxisLabels array can be created by preparsing the dataSeries but it's not a very nice work around.
Does anyone have any ideas how the formatLineGraphItem method could be extended to provide the x axis labels. Or does anyone have any documentation on what values the object o can contain?
Thanks in advance!
This will take a unix timestamp, multiply the value by 1000 (so that it has microseconds for JavaScript, and then pass the value to dojo date to format it).
You shouldn't have any problems editing this to the format you need.
You provided examples that your dates are like "1", "2", "3", which is clearly wrong. Those aren't dates.. so this is the best you can do unless you edit your question.
chart1.addAxis("x",{
labelFunc: function(n){
if(isNaN(dojo.number.parse(n)) || dojo.number.parse(n) % 1 != 0){
return " ";
}
else {
// I am assuming that your timestamp needs to be multiplied by 1000.
var date = new Date(dojo.number.parse(n) * 1000);
return dojo.date.locale.format(date, {
selector: "date",
datePattern: "dd MMMM",
locale: "en"
});
}
},
maxLabelSize: 100
}