I have the following pandas DataFrame:
account_num = [
1726905620833, 1727875510892, 1727925550921, 1727925575731, 1727345507414,
1713565531401, 1725735509119, 1727925546516, 1727925523656, 1727875509665,
1727875504742, 1727345504314, 1725475539855, 1791725523833, 1727925583805,
1727925544791, 1727925518810, 1727925606986, 1727925618602, 1727605517337,
1727605517354, 1727925583101, 1727925583201, 1727925583335, 1727025517810,
1727935718602]
total_due = [
1662.87, 3233.73, 3992.05, 10469.28, 799.01, 2292.98, 297.07, 5699.06, 1309.82,
1109.67, 4830.57, 3170.12, 45329.73, 46.71, 11981.58, 3246.31, 3214.25, 2056.82,
1611.73, 5386.16, 2622.02, 5011.02, 6222.10, 16340.90, 1239.23, 1198.98]
net_returned = [
0.0, 0.0, 0.0, 2762.64, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 12008.27,
0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2762.69, 0.0, 0.0, 0.0, 9254.66, 0.0, 0.0]
total_fees = [
0.0, 0.0, 0.0, 607.78, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2161.49, 0.0,
0.0, 0.0, 0.0, 0.0, 0.0, 536.51, 0.0, 0.0, 0.0, 1712.11, 0.0, 0.0]
year = [2021, 2022, 2022, 2021, 2021, 2020, 2020, 2022, 2019, 2019, 2020, 2022, 2019,
2018, 2018, 2022, 2021, 2022, 2022, 2020, 2019, 2019, 2022, 2019, 2021, 2022]
flipped = [1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0]
proba = [
0.960085, 0.022535, 0.013746, 0.025833, 0.076159, 0.788912, 0.052489, 0.035279,
0.019701, 0.552127, 0.063949, 0.061279, 0.024398, 0.902681, 0.009441, 0.015342,
0.006832, 0.032988, 0.031879, 0.026412, 0.025159, 0.023195, 0.022104, 0.021285,
0.026480, 0.025837]
d = {
"account_num" : account_num,
"total_due" : total_due,
"net_returned" : net_returned,
"total_fees" : total_fees,
"year" : year,
"flipped" : flipped,
"proba" : proba
}
df = pd.DataFrame(data=d)
I want to sample the DataFrame by the "year" column according to a specific ratio for each year, which I have successfully done with the following code:
df_fractions = pd.DataFrame({"2018": [0.5], "2019": [0.5], "2020": [1.0], "2021": [0.8],
"2022": [0.7]})
df.year = df.year.astype(str)
grouped = df.groupby("year")
df_training = grouped.apply(lambda x: x.sample(frac=df_fractions[x.name]))
df_training = df_training.reset_index(drop=True)
However, when I invoke sample(), I also want to ensure the samples from each year are stratified according to the number of flipped accounts in that year. So, I want to stratify the per-year samples based on the flipped column. With this small, toy DataFrame, after sampling per year, the ratio of flipped per year are pretty good with respect to the original proportions. But this is not true for a really large DataFrame with close to 300K accounts.
So, that's really my question to all you Python experts: is there a better way to solve this problem than the solution I came up with?
Related
i want to implement a bezier curve animation which is provided by easing in react native but the docs are not very clear about how to implement it. please need your suggestion
Here on this repository you can see some examples of the use of react-native-easing:
react-native-easing
Here's the file on the repository:
import { Easing } from 'react-native';
export default {
step0: Easing.step0,
step1: Easing.step1,
linear: Easing.linear,
ease: Easing.ease,
quad: Easing.quad,
cubic: Easing.cubic,
poly: Easing.poly,
sin: Easing.sin,
circle: Easing.circle,
exp: Easing.exp,
elastic: Easing.elastic,
back: Easing.back,
bounce: Easing.bounce,
bezier: Easing.bezier,
in: Easing.in,
out: Easing.out,
inOut: Easing.inOut,
easeIn: Easing.bezier(0.42, 0, 1, 1),
easeOut: Easing.bezier(0, 0, 0.58, 1),
easeInOut: Easing.bezier(0.42, 0, 0.58, 1),
easeInCubic: Easing.bezier(0.55, 0.055, 0.675, 0.19),
easeOutCubic: Easing.bezier(0.215, 0.61, 0.355, 1.0),
easeInOutCubic: Easing.bezier(0.645, 0.045, 0.355, 1.0),
easeInCirc: Easing.bezier(0.6, 0.04, 0.98, 0.335),
easeOutCirc: Easing.bezier(0.075, 0.82, 0.165, 1.0),
easeInOutCirc: Easing.bezier(0.785, 0.135, 0.15, 0.86),
easeInExpo: Easing.bezier(0.95, 0.05, 0.795, 0.035),
easeOutExpo: Easing.bezier(0.19, 1.0, 0.22, 1.0),
easeInOutExpo: Easing.bezier(1.0, 0.0, 0.0, 1.0),
easeInQuad: Easing.bezier(0.55, 0.085, 0.68, 0.53),
easeOutQuad: Easing.bezier(0.25, 0.46, 0.45, 0.94),
easeInOutQuad: Easing.bezier(0.455, 0.03, 0.515, 0.955),
easeInQuart: Easing.bezier(0.895, 0.03, 0.685, 0.22),
easeOutQuart: Easing.bezier(0.165, 0.84, 0.44, 1.0),
easeInOutQuart: Easing.bezier(0.77, 0.0, 0.175, 1.0),
easeInQuint: Easing.bezier(0.755, 0.05, 0.855, 0.06),
easeOutQuint: Easing.bezier(0.23, 1.0, 0.32, 1.0),
easeInOutQuint: Easing.bezier(0.86, 0.0, 0.07, 1.0),
easeInSine: Easing.bezier(0.47, 0.0, 0.745, 0.715),
easeOutSine: Easing.bezier(0.39, 0.575, 0.565, 1.0),
easeInOutSine: Easing.bezier(0.445, 0.05, 0.55, 0.95),
easeInBack: Easing.bezier(0.6, -0.28, 0.735, 0.045),
easeOutBack: Easing.bezier(0.175, 0.885, 0.32, 1.275),
easeInOutBack: Easing.bezier(0.68, -0.55, 0.265, 1.55),
easeInElastic: Easing.out(Easing.elastic(2)),
easeInElasticCustom: (bounciness = 2) => Easing.out(Easing.elastic(bounciness)),
easeOutElastic: Easing.in(Easing.elastic(2)),
easeOutElasticCustom: (bounciness = 2) => Easing.in(Easing.elastic(bounciness)),
easeInOutElastic: Easing.inOut(Easing.out(Easing.elastic(2))),
easeInOutElasticCustom: (bounciness = 2) => Easing.inOut(Easing.out(Easing.elastic(bounciness))),
easeInBounce: Easing.out(Easing.bounce),
easeOutBounce: Easing.in(Easing.bounce),
easeInOutBounce: Easing.inOut(Easing.out(Easing.bounce)),
};
And here's what each function generates:
I'm doing a homework on WebGL2 and am provided with a projection and view matrix I have to use to form a camera. It says "the matrices have to be send to the shaders and the shaders have to be extended by new uniforms".
It's part two of a multipart assignment where part one was to send the vertices of a cube to the vertex shader.
I get to the part where it shows a rectangle, as all over parts of the cube are behind that one.
I looked at some examples on webgl2fundamentals but wasn't able to adapt the code to the code we were provided with. I've tried several positionings, especially with looking up the uniforms during init() and then binding them either in createGeometry() or render(), where all questionable lines of code currently sit for better overview.
I think at least the lookup shouldn't happen at render time.
vertex shader:
#version 300 es
precision mediump float;
layout(location = 0) in vec3 aPos;
layout(location = 1) in vec3 aColor;
uniform mat4 u_pmatrix;
uniform mat4 u_vmatrix;
out vec3 color;
void main() {
color = aColor;
gl_Position = u_pmatrix * u_vmatrix * vec4(aPos, 1.0);
}
"use strict"
var gl;
var viewMatrix;
var projectionMatrix;
var program;
var vao;
function render(timestamp, previousTimestamp)
{
var light = getLightPosition(); // vec3
var rotation = getRotation(); // vec3
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
gl.useProgram(program);
gl.bindVertexArray(vao);
var pMatLocation = gl.getUniformLocation(program, "u_pmatrix");
var vMatLocation = gl.getUniformLocation(program, "u_vmatrix");
gl.uniformMatrix4fv(pMatLocation, false, projectionMatrix);
gl.uniformMatrix4fv(vMatLocation, false, viewMatrix);
gl.drawArrays(gl.TRIANGLES, 0, 6 * 6);
window.requestAnimFrame(function (time) {
render(time, timestamp);
});
}
function createGeometry()
{
var positions = [];
positions.push(vec3(-0.5, -0.5, -0.5));
positions.push(vec3(-0.5, 0.5, -0.5));
positions.push(vec3(0.5, -0.5, -0.5));
positions.push(vec3(-0.5, 0.5, -0.5));
positions.push(vec3(0.5, 0.5, -0.5));
positions.push(vec3(0.5, -0.5, -0.5));
positions.push(vec3(-0.5, -0.5, 0.5));
positions.push(vec3(0.5, -0.5, 0.5));
positions.push(vec3(-0.5, 0.5, 0.5));
positions.push(vec3(-0.5, 0.5, 0.5));
positions.push(vec3(0.5, -0.5, 0.5));
positions.push(vec3(0.5, 0.5, 0.5));
positions.push(vec3(-0.5, 0.5, -0.5));
positions.push(vec3(-0.5, 0.5, 0.5));
positions.push(vec3(0.5, 0.5, -0.5));
positions.push(vec3(-0.5, 0.5, 0.5));
positions.push(vec3(0.5, 0.5, 0.5));
positions.push(vec3(0.5, 0.5, -0.5));
positions.push(vec3(-0.5, -0.5, -0.5));
positions.push(vec3(0.5, -0.5, -0.5));
positions.push(vec3(-0.5, -0.5, 0.5));
positions.push(vec3(-0.5, -0.5, 0.5));
positions.push(vec3(0.5, -0.5, -0.5));
positions.push(vec3(0.5, -0.5, 0.5));
positions.push(vec3(-0.5, -0.5, -0.5));
positions.push(vec3(-0.5, -0.5, 0.5));
positions.push(vec3(-0.5, 0.5, -0.5));
positions.push(vec3(-0.5, 0.5, -0.5));
positions.push(vec3(-0.5, 0.5, 0.5));
positions.push(vec3(-0.5, 0.5, -0.5));
positions.push(vec3(0.5, -0.5, -0.5));
positions.push(vec3(0.5, 0.5, -0.5));
positions.push(vec3(0.5, -0.5, 0.5));
positions.push(vec3(0.5, -0.5, 0.5));
positions.push(vec3(0.5, 0.5, -0.5));
positions.push(vec3(0.5, 0.5, 0.5));
vao = gl.createVertexArray();
gl.bindVertexArray(vao);
var vertexBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer)
gl.bufferData(gl.ARRAY_BUFFER, flatten(positions), gl.STATIC_DRAW);
gl.vertexAttribPointer(0, 3, gl.FLOAT, gl.FALSE, 0, 0);
gl.enableVertexAttribArray(0);
var colors = [];
colors.push(vec3(0.0, 1.0, 0.0));
colors.push(vec3(0.0, 1.0, 0.0));
colors.push(vec3(0.0, 1.0, 0.0));
colors.push(vec3(0.0, 1.0, 0.0));
colors.push(vec3(0.0, 1.0, 0.0));
colors.push(vec3(0.0, 1.0, 0.0));
colors.push(vec3(1.0, 0.0, 0.0));
colors.push(vec3(1.0, 0.0, 0.0));
colors.push(vec3(1.0, 0.0, 0.0));
colors.push(vec3(1.0, 0.0, 0.0));
colors.push(vec3(1.0, 0.0, 0.0));
colors.push(vec3(1.0, 0.0, 0.0));
colors.push(vec3(0.0, 0.0, 1.0));
colors.push(vec3(0.0, 0.0, 1.0));
colors.push(vec3(0.0, 0.0, 1.0));
colors.push(vec3(0.0, 0.0, 1.0));
colors.push(vec3(0.0, 0.0, 1.0));
colors.push(vec3(0.0, 0.0, 1.0));
colors.push(vec3(1.0, 1.0, 0.0));
colors.push(vec3(1.0, 1.0, 0.0));
colors.push(vec3(1.0, 1.0, 0.0));
colors.push(vec3(1.0, 1.0, 0.0));
colors.push(vec3(1.0, 1.0, 0.0));
colors.push(vec3(1.0, 1.0, 0.0));
colors.push(vec3(1.0, 1.0, 0.5));
colors.push(vec3(1.0, 1.0, 0.5));
colors.push(vec3(1.0, 1.0, 0.5));
colors.push(vec3(1.0, 1.0, 0.5));
colors.push(vec3(1.0, 1.0, 0.5));
colors.push(vec3(1.0, 1.0, 0.5));
colors.push(vec3(1.0, 0.0, 1.0));
colors.push(vec3(1.0, 0.0, 1.0));
colors.push(vec3(1.0, 0.0, 1.0));
colors.push(vec3(1.0, 0.0, 1.0));
colors.push(vec3(1.0, 0.0, 1.0));
colors.push(vec3(1.0, 0.0, 1.0));
var vboColor = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vboColor);
gl.bufferData(gl.ARRAY_BUFFER, flatten(colors), gl.STATIC_DRAW);
gl.vertexAttribPointer(1, 3, gl.FLOAT, gl.FALSE, 0, 0);
gl.enableVertexAttribArray(1);
}
function loadModel()
{
var meshData = loadMeshData();
var positions = meshData.positions;
var normals = meshData.normals;
var colors = meshData.colors;
var vertexCount = meshData.vertexCount;
}
window.onload = function init() {
var canvas = document.getElementById('rendering-surface');
gl = WebGLUtils.setupWebGL( canvas );
gl.viewport(0, 0, canvas.width, canvas.height);
gl.enable(gl.DEPTH_TEST);
gl.clearColor(0.0, 0.0, 0.0, 0.0);
program = initShaders(gl, "vertex-shader","fragment-shader");
gl.useProgram(program);
createGeometry();
loadModel();
var projectionMatrix = mat4(1.0);
projectionMatrix = perspective(90, canvas.width / canvas.height, 0.1, 100);
var eyePos = vec3(0, 1.0, 2.0);
var lookAtPos = vec3(0.0, 0.0, 0.0);
var upVector = vec3(0.0, 1.0, 0.0);
viewMatrix = lookAt(eyePos, lookAtPos, upVector);
render(0,0);
}
There should be a cube, but all that's to be seen is blank space. Either the positioning or transformation is wrong, or the program is crashing.
In your init function you're shadowing your global projectionMatrix thus your projection matrix used in render always remains undefined.
var projectionMatrix = mat4(1.0);// << shadowing your global with the same name
projectionMatrix = perspective(90, canvas.width / canvas.height, 0.1, 100);
You might want to take a look at this article on how to use developer tools for debugging.
I am trying to do sentiment analysis on tweets using sentimentIntensityAnalyzer() from nltk.sentiment.vader
sid = SentimentIntensityAnalyzer()
listy = []
for index, row in data.iterrows():
ss = sid.polarity_scores(row["Tweets"])
listy.append(ss)
se = pd.Series(listy)
data['polarity'] = se.values
display(data.head(100))
This is the resulting dataFramee :
Tweets polarity
0 RT #spectatorindex: Facebook controls:\n\n- Wh... {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...
1 RT #YAATeamWest: Today we're at #BradfordUniSU... {'neg': 0.0, 'neu': 0.902, 'pos': 0.098, 'comp...
2 #SachinTendulkar launches India’s first Multip... {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...
3 How To Create a 360 Render (And How to Improv... {'neg': 0.0, 'neu': 0.722, 'pos': 0.278, 'comp...
4 The Most Disturbing Virtual Reality You Will E... {'neg': 0.174, 'neu': 0.826, 'pos': 0.0, 'comp...
5 VR Training for Troops 🎮\n\n... {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...
6 RT #DefenceHQ: The #BritishArmy has awarded a ... {'neg': 0.0, 'neu': 0.847, 'pos': 0.153, 'comp...
7 RT #UofGHumanities: #UofGCSPE Humanities Lectu... {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...
8 RT #OyezServices: Ever wanted a tour of Machu ... {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...
9 RT #ProjectDastaan: We are an Oxford Universit... {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound...
10 RT #Paula_Piccard: Virtual reality will change... {'neg': 0.0, 'neu': 0.878, 'pos': 0.122, 'comp...
In order to do statistical analysis on the 'neg','pos','neu' and 'compound' entities in the polarity column I wanted to split the data into four different columns. To achieve this I used :
list_pos= []
list_neg = []
list_comp = []
list_neu = []
for index, row in data.iterrows():
list_pos.append(row['polarity']['pos'])
list_neg.append(row['polarity']['neg'])
list_comp.append(row['polarity']['compound'])
list_neu.append(row['polarity']['neu'])
se_pos = pd.Series(list_pos)
se_neg = pd.Series(list_neg)
se_comp = pd.Series(list_comp)
se_neu = pd.Series(list_neu)
data['positive'] = se_pos.values
data['negative'] = se_neg.values
data['compound'] = se_comp.values
data['neutral'] = se_neu.values
The resulting dataFrame:
Tweets polarity positive negative compound neutral
0 RT #spectatorindex: Facebook controls:\n\n- Wh... {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound... 0.000 0.000 0.0000 1.000
1 RT #YAATeamWest: Today we're at #BradfordUniSU... {'neg': 0.0, 'neu': 0.902, 'pos': 0.098, 'comp... 0.098 0.000 0.3612 0.902
2 #SachinTendulkar launches India’s first Multip... {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound... 0.000 0.000 0.0000 1.000
Is there a more concise way of achieving a similar dataFrame? Using the lambda function perhaps? Thanks for the help!
I want plot histograms from tensorboard on my own, to publish it. I wrote this extraction function to get the histogram values:
def _load_hist_from_tfboard(path):
event_acc = event_accumulator.EventAccumulator(path)
event_acc.Reload()
vec_dict = {}
for tag in sorted(event_acc.Tags()["distributions"]):
hist_dict = {}
for hist_event in event_acc.Histograms(tag):
hist_dict.update({hist_event.step: (hist_event.histogram_value.bucket_limit,
hist_event.histogram_value.bucket)})
vec_dict[tag] = hist_dict
return vec_dict
The function collects all histograms of a event file. The output of one bucket_limit and bucket is as follows:
[0.0, 1e-12, 0.0005418219168477906, 0.0005960041085325697, 0.0020575678275470133, 0.0022633246103017147, 0.004009617609950718, 0.00441057937094579, 0.005336801038844407, 0.005870481142728848, 0.007813610400972098, 0.008594971441069308, 0.022293142370048362, 0.0245224566070532, 0.026974702267758523, 0.035903328718386605, 0.03949366159022527, 0.043443027749247805, 0.04778733052417259, 0.052566063576589855, 0.057822669934248845, 0.06360493692767373, 0.06996543062044111, 0.07696197368248522, 0.24153964213356663, 0.2656936063469233, 0.29226296698161564, 0.3214892636797772, 0.35363819004775493, 0.38900200905253046, 0.42790220995778355, 0.47069243095356195, 0.5177616740489182, 0.56953784145381, 0.6264916255991911, 0.6891407881591103, 0.7580548669750213, 0.8338603536725235, 0.917246389039776, 1.0089710279437536]
[0.0, 3999936.0, 0.0, 4.0, 0.0, 4.0, 0.0, 4.0, 0.0, 4.0, 0.0, 4.0, 0.0, 4.0, 4.0, 0.0, 8.0, 8.0, 0.0, 4.0, 4.0, 0.0, 8.0, 4.0, 0.0, 9.0, 45.0, 50.0, 48.0, 85.0, 100.0, 109.0, 114.0, 15908.0, 74.0, 15856.0, 11908.0, 3973.0, 42.0, 7951679.0]
Can someone help me to interpret these numbers to a histogram.
I have an input array from a camera (greyscale image) that looks like:
[
[0.5, 0.75, 0.1, 0.6],
[0.3, 0.75, 1.0, 0.9]
]
actual size = 434x512
I need an output which is a list of XYZ coordinates:
i.e. [[x,y,z],[x,y,z],...]
[[0,0,0.5],[1,0,0.75],[2,0,0.1],[3,0,0.6],[0,1,0.3],[1,1,0.75],[2,1,1.0],[3,1,0.9]]
Are there any efficient ways to do this using Numpy?
Here's an approach -
m,n = a.shape
R,C = np.mgrid[:m,:n]
out = np.column_stack((C.ravel(),R.ravel(), a.ravel()))
Sample run -
In [45]: a
Out[45]:
array([[ 0.5 , 0.75, 0.1 , 0.6 ],
[ 0.3 , 0.75, 1. , 0.9 ]])
In [46]: m,n = a.shape
...: R,C = np.mgrid[:m,:n]
...: out = np.column_stack((C.ravel(),R.ravel(), a.ravel()))
...:
In [47]: out
Out[47]:
array([[ 0. , 0. , 0.5 ],
[ 1. , 0. , 0.75],
[ 2. , 0. , 0.1 ],
[ 3. , 0. , 0.6 ],
[ 0. , 1. , 0.3 ],
[ 1. , 1. , 0.75],
[ 2. , 1. , 1. ],
[ 3. , 1. , 0.9 ]])
In [48]: out.tolist() # Convert to list of lists if needed
Out[48]:
[[0.0, 0.0, 0.5],
[1.0, 0.0, 0.75],
[2.0, 0.0, 0.1],
[3.0, 0.0, 0.6],
[0.0, 1.0, 0.3],
[1.0, 1.0, 0.75],
[2.0, 1.0, 1.0],
[3.0, 1.0, 0.9]]