It works fine with a pair of array. I don't need to use Ramda in this example.
const addChild1 = (x , y) => `${x}.addChild(${y}); `
const result1 = addChild(["containerA"], ["a1","a2", "a3"])
console.log(result1) //containerA.addChild(a1,a2,a3)
I couldn't make it work with 2d matrix of strings. I used R.map, but I get 'undefined' in the second argument.
const addChild2 = R.map ((x , y) => `${x}.addChild(${y}); `)
const result2 = addChild2(["containerA", "containerB", "containerC"], [["a1","a2", "a3"], ["b1","b2", "b3"], ["c1","c2", "c3"]])
console.log(result2) //["containerA.addChild(undefined); ","containerB.addChild(undefined); ","containerC.addChild(undefined); "]
How can I avoid the 'undefined'? Desirable output is the below :
["containerA.addChild("a1","a2", "a3"); ","containerB.addChild("b1","b2", "b3"); ","containerC.addChild("c1","c2", "c3");"]
map takes string as second parameters, so in this case, only ["containerA", "containerB", "containerC"] got in the loop
You should use zipWith in this case instead of map
const addChild2 = R.zipWith((x, y) => `${x}.addChild(${y}); `)
const result2 = addChild2(
["containerA", "containerB", "containerC"],
[
["a1", "a2", "a3"],
["b1", "b2", "b3"],
["c1", "c2", "c3"],
]
)
console.log(result2)
<script src="//cdnjs.cloudflare.com/ajax/libs/ramda/0.25.0/ramda.min.js"></script>
Related
I struggle accessing the row-elements of a Frame.
One idea I have is to filter the dataframe down to a row, convert it to a vec or something similar and access the elements this way ?!
In Panadas I used to just use ".at / .loc / .iloc / etc."; with Polars in Rust I have no clue.
Any suggestions on what the proper way to do this is ?
Thanks to #isaactfa ... he got me onto the right track. I ended up getting the row not with "get_row" but rather with "get" ... this is probably due to my little RUST understanding (my 2nd week).
Here is a working code sample:
use polars::export::arrow::temporal_conversions::date32_to_date;
use polars::prelude::*;
fn main() -> Result<()> {
let days = df!(
"date_string" => &["1900-01-01", "1900-01-02", "1900-01-03", "1900-01-04", "1900-01-05",
"1900-01-06", "1900-01-07", "1900-01-09", "1900-01-10"])?;
let options = StrpTimeOptions {
date_dtype: DataType::Date, // the result column-datatype
fmt: Some("%Y-%m-%d".into()), // the source format of the date-string
strict: false,
exact: true,
};
// convert date_string into dtype(date) and put into new column "date_type"
// we convert the days DataFrame to a LazyFrame ...
// because in my real-world example I am getting a LazyFrame
let mut new_days_lf = days.lazy().with_column(
col("date_string")
.alias("date_type")
.str()
.strptime(options),
);
// Getting the weekday as a number:
// This is what I wanted to do ... but I get a string result .. need u32
// let o = GetOutput::from_type(DataType::Date);
// new_days_lf = new_days_lf.with_column(
// col("date_type")
// .alias("weekday_number")
// .map(|x| Ok(x.strftime("%w").unwrap()), o.clone()),
// );
// This is the convoluted workaround for getting the weekday as a number
let o = GetOutput::from_type(DataType::Date);
new_days_lf = new_days_lf.with_column(col("date_type").alias("weekday_number").map(
|x| {
Ok(x.date()
.unwrap()
.clone()
.into_iter()
.map(|opt_name: Option<i32>| {
opt_name.map(|datum: i32| {
// println!("{:?}", datum);
date32_to_date(datum)
.format("%w")
.to_string()
.parse::<u32>()
.unwrap()
})
})
.collect::<UInt32Chunked>()
.into_series())
},
o,
));
new_days_lf = new_days_lf.with_column(
col("weekday_number")
.shift_and_fill(-1, 9999)
.alias("next_weekday_number"),
);
// now we convert the LazyFrame into a normal DataFrame for further processing:
let mut new_days_df = new_days_lf.collect()?;
// convert the column to a series
// to get a column by name we need to collect the LazyFrame into a normal DataFrame
let col1 = new_days_df.column("weekday_number")?;
// convert the column to a series
let col2 = new_days_df.column("next_weekday_number")?;
// now I can use series-arithmetics
let diff = col2 - col1;
// create a bool column based on "element == 2"
// add bool column to DataFrame
new_days_df.replace_or_add("weekday diff eq(2)", diff.equal(2)?.into_series());
// could not figure out how to filter the eager frame ...
let result = new_days_df
.lazy()
.filter(col("weekday diff eq(2)").eq(true))
.collect()
.unwrap();
// could not figure out how to access ROW elements
// thus I used "get" instead af of "get_row"
// getting the date where diff is == 2 (true)
let filtered_row = result.get(0).unwrap();
// within the filtered_row get element with an index
let date = filtered_row.get(0).unwrap();
println!("\n{:?}", date);
Ok(())
}
I have 2 Tensors of same length, data and groupIds. I want to split data into several groups by the corresponding values in groupId. For example,
const data = tf.tensor([1,2,3,4,5]);
const groupIds = tf.tensor([0,1,1,0,0]);
// expected result: [tf.tensor([1,4,5]), tf.tensor([2,3])]
In Tensorflow there is tf.dynamic_partition which does exactly that. Tensorflow.js doesn't seem to have a similar method. I also looked into mask or filtering as work-arounds, but they don't exist either. Does anyone have an idea how to implement this?
To partition your tensor, you can first iterate over your ids tensor to get the number of subtensor to create and the index of the elements it should contain. This information can be stored in an object where the key is the number of the partition in the ids array and the value is an array of indexes.
const data = tf.tensor([6,2,8,4,5]);
const ids = tf.tensor([0,1,1,0,2]);
const data2 = tf.tensor([[6,2],[8,4], [5, 4], [6, 5]]);
const ids2 = tf.tensor([0,1,1,0]);
const filterT = (t, p) => {
t.print()
p.print()
const l = p.unstack().reduce((a, b, i) => {
const v = b.dataSync()[0]
if (Object.keys(a).includes(v.toString())) {
a[v].push(i)
} else {
a[v] = [i]
}
return a
}, {})
const r = Object.keys(l).map(k => t.gather(tf.tensor1d(l[k], 'int32')))
r.forEach(e => e.print())
}
filterT(data, ids)
filterT(data2, ids2)
<html>
<head>
<!-- Load TensorFlow.js -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/tensorflow/0.12.4/tf.js"> </script>
</head>
<body>
</body>
</html>
Here is my problem, I have a map of Map[Array[String],String], and I want to pass that into a UDF.
Here is my UDF:
def lookup(lookupMap:Map[Array[String],String]) =
udf((input:Array[String]) => lookupMap.lift(input))
And here is my Map variable:
val srdd = df.rdd.map { row => (
Array(row.getString(1),row.getString(5),row.getString(8)).map(_.toString),
row.getString(7)
)}
Here is how I call the function:
val combinedDF = dftemp.withColumn("a",lookup(lookupMap))(Array($"b",$"c","d"))
I first got an error about immutable array, so I changed my array into immutable type, then I got an error about type mismatch. I googled a bit, apparently I can't pass in non-column type directly into a UDF. Can somebody help? Kudos.
Update: So I did convert everything to a wrapped array. Here is what I did:
val srdd = df.rdd.map{row => (WrappedArray.make[String](Array(row.getString(1),row.getString(5),row.getString(8))),row.getString(7))}
val lookupMap = srdd.collectAsMap()
def lookup(lookupMap:Map[collection.mutable.WrappedArray[String],String]) = udf((input:collection.mutable.WrappedArray[String]) => lookupMap.lift(input))
val combinedDF = dftemp.withColumn("a",lookup(lookupMap))(Array($"b",$"c",$"d"))
Now I am having an error like this:
required: Map[scala.collection.mutable.WrappedArray[String],String]
-ksh: Map[scala.collection.mutable.WrappedArray[String],String]: not found [No such file or directory]
I tried to do something like this:
val m = collection.immutable.Map(1->"one",2->"Two")
val n = collection.mutable.Map(m.toSeq: _*)
but then I just got back to the error of column type.
First, you have to pass a Column as an argument of the UDF; Since you want this argument to be an array, you should use the array function in org.apache.spark.sql.functions, which creates an array Column from a series of other Columns. So the UDF call would be:
lookup(lookupMap)(array($"b",$"c",$"d"))
Now, since array columns are deserialized into mutable.WrappedArray, in order for the map lookup to succeed you'd best make sure that's the type used by your UDF:
def lookup(lookupMap: Map[mutable.WrappedArray[String],String]) =
udf((input: mutable.WrappedArray[String]) => lookupMap.lift(input))
So altogether:
import spark.implicits._
import org.apache.spark.sql.functions._
// Create an RDD[(mutable.WrappedArray[String], String)]:
val srdd = df.rdd.map { row: Row => (
mutable.WrappedArray.make[String](Array(row.getString(1), row.getString(5), row.getString(8))),
row.getString(7)
)}
// collect it into a map (I assume this is what you're doing with srdd...)
val lookupMap: Map[mutable.WrappedArray[String], String] = srdd.collectAsMap()
def lookup(lookupMap: Map[mutable.WrappedArray[String],String]) =
udf((input: mutable.WrappedArray[String]) => lookupMap.lift(input))
val combinedDF = dftemp.withColumn("a",lookup(lookupMap)(array($"b",$"c",$"d")))
Anna your code for srdd/lookupmap is of type org.apache.spark.rdd.RDD[(Array[String], String)]
val srdd = df.rdd.map { row => (
Array(row.getString(1),row.getString(5),row.getString(8)).map(_.toString),
row.getString(7)
)}
Where as in lookup method you are expecting a Map as a parameter
def lookup(lookupMap:Map[Array[String],String]) =
udf((input:Array[String]) => lookupMap.lift(input))
That is the reason why you are getting type mismatch error.
First make srdd from RDD[tuple] to a RDD[Map] and then try converting the RDD to Map to resolve this error.
val srdd = df.rdd.map { row => Map(
Array(row.getString(1),row.getString(5),row.getString(8)).map(_.toString) ->
row.getString(7)
)}
I love lodash and I use it in many projects.
I always have this problem and I can't find a solution for it.
I would like to do something with lodash, save a temporary state, and continue.
For example lets assume I have moviesList which is a list of movies with their id and profit, and I want to
index movies by their id and keep the result in moviesById
filter profitable movies and keep the result in overMillionProfit
And I want to do so without breaking the chain.
Something like (wishful thinking):
var moviesList = [{id : 1}, {id:2}, ...];
var moviesById = {};
var overMillionProfit = [];
_(moviesList)
.keyBy('id')
.do((unwrappedValue)=> moviesById = unwrappedValue)
.values()
.filter( (m) => m.profit > 1000000)
.do((unwrappedValue) => overMillionProfit = unwrappedValue)
Currently, I find it necessary to break the chain like so:
var moviesList = [{id : 1}, {id:2}, ...];
var moviesById = _.keyBy(moviesList,'id');
var overMillionProfit = _.filter(moviesList, ...);
Perhaps for this scenario it is better, but I want to be able to do it like the above in some cases.
Is there a way to do this?
You can use _.tap() for that:
var moviesList = [{id : 1}, {id:2}, {id:3}];
var moviesById;
var overMillionProfit;
var result = _(moviesList)
.keyBy('id')
.tap((unwrappedValue)=> moviesById = unwrappedValue)
.values()
.filter( (m) => m.id > 1)
.tap((unwrappedValue) => overMillionProfit = unwrappedValue)
.value();
console.log('moviesById\n', moviesById);
console.log('overMillionProfit\n', overMillionProfit);
console.log('result\n', result);
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.4/lodash.min.js"></script>
I want to write a MODULE where the client function returns a taglist with 2 textInputs. The user can enter a value in either textInput 1 or textInput 2 but not both. In other words, one textInput excludes the other, emulating a set of radio buttons.
Hence, the server function should observe the input in such a way that if the user enters a value in textInput 1, then textInput 2 is made blank and vice versa.
Also, the server function returns a dataframe with the values in the textInputs, i.e. either data.frame (one = enteredValue, two = NA) or data.frame (one = NA, two = enteredValue)
As we are planning on using this two-textInput widget in many of our shiny apps, I really want to make it a module. It seems like a simple thing to implement, but so far I have not been successful. (My experience is that observe, observeEvent, and eventReactive work differently in modules than in regular apps)
Any ideas to point me in the right direction are welcome.
I believe eventReactive and observeEvent work inside of a Shiny module. I've created a small module that basically does what you describe.
ui.R
library(shiny)
library(shinydashboard)
source("doubleField.R")
shinyUI(dashboardPage(
dashboardHeader(title = "Test"),
dashboardSidebar(disable = T),
dashboardBody(
doubleFieldUI("fields"),
fluidRow(
dataTableOutput("outputTable")
)
)
))
server.R
library(shiny)
source("doubleField.R")
shinyServer(function(input, output) {
fields <- callModule(doubleField, "fields")
output$outputTable <- renderDataTable(fields())
})
doubleField.R
library(stringr)
doubleFieldUI <- function(id) {
ns <- NS(id)
return(
tagList(
fluidRow(
column(width = 6, textInput(ns("fieldA"), "Field A")),
column(width = 6, textInput(ns("fieldB"), "Field B"))
),
fluidRow(
column(width = 2, "Output: "),
column(width = 4, textOutput(ns("outputValue")))
)
)
)
}
is_empty_string <- function(s) {
return(str_length(s) == 0)
}
doubleField <- function(input, output, session) {
valueA <- eventReactive(input$fieldA, {
if(!is_empty_string(input$fieldA)) {
ns <- session$ns
updateTextInput(session, "fieldB", value = "")
return(input$fieldA)
}
return("")
})
valueB <- eventReactive(input$fieldB, {
if(!is_empty_string(input$fieldB)) {
ns <- session$ns
updateTextInput(session, "fieldA", value = "")
return(input$fieldB)
}
return("")
})
value <- reactive({
values <- c(input$fieldA, input$fieldB)
return(values[which(!is_empty_string(values))])
})
output$outputValue <- renderText({
value()
})
result_df <- reactive({
v_A <- valueA()
v_B <- valueB()
df <- data.frame(
list(
"valueA" = ifelse(is_empty_string(v_A), NULL, v_A),
"valueB" = ifelse(is_empty_string(v_B), NULL, v_B)
)
)
return(df)
})
return(result_df);
}
I hope this helps getting you started.