I wanna stemming my dataset. Before stemming, I did tokenize use nltk tokenize
You can see the output on the pic
Dataset
Col Values
But when i do stemming, it return error :
[Error][3]
TypeError Traceback (most recent call
last)
<ipython-input-102-7700a8e3235b> in <module>()
----> 1 df['Message'] = df['Message'].apply(stemmer.stem)
2 df = df[['Message', 'Category']]
3 df.head()
5 frames
/usr/local/lib/python3.7/dist-
packages/Sastrawi/Stemmer/Filter/TextNormalizer.py in
normalize_text(text)
2
3 def normalize_text(text):
----> 4 result = str.lower(text)
5 result = re.sub(r'[^a-z0-9 -]', ' ', result, flags =
re.IGNORECASE|re.MULTILINE)
6 result = re.sub(r'( +)', ' ', result, flags =
re.IGNORECASE|re.MULTILINE)
TypeError: descriptor 'lower' requires a 'str' object but received a
'list'
Hope all you guys can help me
Related
I have to import a processed h5ad file, but it seems that X has been passed as a numpy array instead of a numpy matrix. See below:
# Read the data
data_path = "/home/bbb5130/snOMICS/maria/msrna.h5ad"
adata = sn.pp.read_h5ad(data_path, pr_process="Yes")
adata
But the output was:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In [15], line 3
1 # Read the data
2 data_path = "/home/bbb5130/snOMICS/maria/msrna.h5ad"
----> 3 adata = sn.pp.read_h5ad(data_path, pr_process="Yes")
4 adata
File ~/miniconda3/envs/snOMICS/lib/python3.9/site-packages/scanet/preprocessing.py:54, in Preprocessing.read_h5ad(cls, filename, pr_process)
51 return sc.read_h5ad(filename)
52 else:
53 # initial preprocessing as it is required later
---> 54 return cls._intial(adata)
File ~/miniconda3/envs/snOMICS/lib/python3.9/site-packages/scanet/preprocessing.py:35, in Preprocessing._intial(adata)
33 adata.var['mt'] = adata.var_names.str.startswith('MT-')
34 mito_genes = adata.var_names.str.startswith('MT-')
---> 35 adata.obs['percent_mito'] = np.sum(adata[:, mito_genes].X, axis=1).A1 / np.sum(adata.X, axis=1).A1
36 sc.pp.calculate_qc_metrics(adata, qc_vars=['mt'], percent_top=None, inplace=True)
37 sc.pp.filter_cells(adata, min_genes=0)
AttributeError: 'ArrayView' object has no attribute 'A1'
Is there anyway I can change the format, so the file can be read?
Thanks in advance.
I need to find people with greater or equal to threshold gain =>
dataframe contains column 'capitalGain' with different values 10,20,30,50,1000,5000,10000 ...etc
I try :
Function:
def get_num_people_with_higher_gain(dataframe, threshold_gain)
threshold_gain = dataframe["capitalGain"][dataframe["capitalGain"] >= threshold_gain].count()
return threshold_gain
Call function
df = get_num_people_with_higher_gain(dataframe, threshold_gain)
But I get the following error message:
NameError Traceback (most recent call last)
<ipython-input-50-5485c90412c8> in <module>
----> 1 df = get_num_people_with_higher_gain(dataframe, threshold_gain)
2 threshold = get_num_people_with_higher_gain(dataframe, threshold_gain)
NameError: name 'dataframe' is not defined
Since there are 2 arguments in the function (dataframe, threshold_gain), does it mean that both should be somehow defined within the function ?
Thanks
Finally,
Here is the solution
def get_num_people_with_higher_gain(dataframe, threshold_gain):
result = len(dataframe[dataframe["capitalGain"] >= threshold_gain])
return result
result = get_num_people_with_higher_gain(dataframe,60000)
result
I am running Wilconox signed-rank test on the dataset which looks like :
df = {'Year': ['2019','2018','2017', ....], 'Name':{jon, tim, luca,...}, 'SelfPromotion': [1,0,1,...]}
the script is as follows:
import pandas
from scipy.stats import mannwhitneyu
data1 = df['SelfPromotion']=1
data2 = df['SelfPromotion']=0
print(mannwhitneyu(data1, data2))
this gives me the following error:
TypeError: len() of unsized object
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-30-e49d9838e5ac> in <module>
3 data1 = data['SelfPromotion']=1
4 data2 = data['SelfPromotion']=0
----> 5 print(mannwhitneyu(data11, data22))
~/opt/anaconda3/envs/shityaar/lib/python3.7/site-packages/scipy/stats/stats.py in mannwhitneyu(x, y, use_continuity, alternative)
6391 x = np.asarray(x)
6392 y = np.asarray(y)
-> 6393 n1 = len(x)
6394 n2 = len(y)
6395 ranked = rankdata(np.concatenate((x, y)))
TypeError: len() of unsized object
I have tried every possible solution for this error by looking at similar questions but unfortunately, no solution could get it to work. I would appreciate some help.
mannwhitneyu expects array like parameters and you are passing integers as args, hence the failure.
Do something like this:
In [26]: data1 = df['SelfPromotion'] == 1
In [28]: data2 = df['SelfPromotion'] == 0
In [31]: mannwhitneyu(data1, data2)
Out[31]: MannwhitneyuResult(statistic=3.0, pvalue=0.30962837708843105)
I am using an amazon dataset to do sentiment analysis. Dataset content is
https://i.stack.imgur.com/qcKZp.png
dataset con be found on:
https://www.kaggle.com/PromptCloudHQ/amazon-reviews-unlocked-mobile-phones
I am trying to remove html from Review column.
This is what I am doing. Note: dataset is assigned to df.
df_removedNoise = []
def removingHTML(text):
soup = BeautifulSoup(text, 'lxml').get_text()
return soup
def removingNoise(text):
html_removed = removingHTML(text)
return html_removed
for i in df["Reviews"]:
text = removingNoise(i)
df_removedNoise.append(text)
Even though Reviews column has object as a datatype, I am still getting an error like.
TypeError Traceback (most recent call last)
<ipython-input-83-3591f5d7a54f> in <module>
9
10 for i in df["Reviews"]:
---> 11 df_removedNoise.append(removingNoise(i))
<ipython-input-83-3591f5d7a54f> in removingNoise(text)
5
6 def removingNoise(text):
----> 7 html_removed = removingHTML(text)
8 return html_removed
9
<ipython-input-83-3591f5d7a54f> in removingHTML(text)
1 df_removedNoise = []
2 def removingHTML(text):
----> 3 soup = BeautifulSoup(text, 'lxml').get_text()
4 return soup
5
~/anaconda3/lib/python3.7/site-packages/bs4/__init__.py in __init__(self, markup, features, builder, parse_only, from_encoding, exclude_encodings, **kwargs)
244 if hasattr(markup, 'read'): # It's a file-type object.
245 markup = markup.read()
--> 246 elif len(markup) <= 256 and (
247 (isinstance(markup, bytes) and not b'<' in markup)
248 or (isinstance(markup, str) and not '<' in markup)
TypeError: object of type 'float' has no len()
Any help will be appreciated!
Check for NaN with df[df['Reviews'].isnull()], if you find any try to dropna first
im working on football data set and this is following error im getting. please help,
#what is the win rate of HomeTeam?
n_matches = df.shape[0]
n_features = df.shape[1] -1
n_homewin = len(df(df.FTR == 'H'))
win_rate = (float(n_homewin) / (n_matches)) * 100
print ("Total number of matches,{}".format(n_matches))
print ("Number of features,{}".format(n_features))
print ("Number of maches won by hom team,{}".format (n_homewin))
print ("win rate of home team,{:.2f}%" .format(win_rate))
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-122-7e4d81fc684e> in <module>
5 n_features = df.shape[1] -1
6
----> 7 n_homewin = len(df(df.FTR == 'H'))
8
9 win_rate = (float(n_homewin) / (n_matches)) * 100
TypeError: 'DataFrame' object is not
expected result should print the team winning ratio
I think problem is with (), need [] for filter by boolean indexing:
n_homewin = len(df[df.FTR == 'H'])
Or simplier count Trues values by sum:
n_homewin = (df.FTR == 'H').sum()
you should modify it to df[df.FTR == 'H']. The parentheses imply a function call