pd.Categorical.from_codes with missing values - pandas

Question

Assume I have:
df = pd.DataFrame({'gender': np.random.choice([1, 2], 10), 'height': np.random.randint(150, 210, 10)})
I'd like to make the gender column categorical. If I try:
df['gender'] = pd.Categorical.from_codes(df['gender'], ['female', 'male'])
it'll fail.
I can pad the categories
df['gender'] = pd.Categorical.from_codes(df['gender'], ['N/A', 'female', 'male'])
But then 'N/A' is returned in some methods:
In [67]: df['gender'].value_counts()
Out[67]:
female 5
male 5
N/A 0
Name: gender, dtype: int64
I thought about using None as the padding value. It works as intended in the value_counts however I get a warning:
opt/anaconda3/bin/ipython:1: FutureWarning:
Setting NaNs in `categories` is deprecated and will be removed in a future version of pandas.
#!/opt/anaconda3/bin/python
Any better way to do this? Also is there a way to give a mapping from code to category explicitly?

Answer 1

you can use rename_categories() method:
Demo:
In [33]: df
Out[33]:
gender height
0 1 203
1 2 169
2 2 181
3 1 172
4 2 174
5 1 166
6 2 187
7 2 200
8 1 208
9 1 201
In [34]: df['gender'] = df['gender'].astype('category').cat.rename_categories(['male','feemale'])
In [35]: df
Out[35]:
gender height
0 male 203
1 feemale 169
2 feemale 181
3 male 172
4 feemale 174
5 male 166
6 feemale 187
7 feemale 200
8 male 208
9 male 201
In [36]: df.dtypes
Out[36]:
gender category
height int32
dtype: object

Answer 2

Assign the new categories directly to it's .categories attribute and it would then be renamed to these values:
df['gender'] = df['gender'].astype('category')
df['gender'].cat.categories = ['female', 'male']
df['gender'].value_counts()
Out[23]:
female 7
male 3
Name: gender, dtype: int64
df.dtypes
Out[24]:
gender category
height int32
dtype: object
If you want a mapper dict of code and it's respective category, then:
old = df['gender'].cat.categories
new = ['female', 'male']
dict(zip(old, new))
Out[28]:
{1: 'female', 2: 'male'}

Answer 3

The error you get from pd.Categorical.from_codes(df['gender'], ['female', 'male']) should alert you that your codes need to be 0 indexed.
So you can simply make it so with your DataFrame declaration.
df = pd.DataFrame({'gender': np.random.choice([0, 1], 10), 'height': np.random.randint(150, 210, 10)})