Replace other values with np.nan

86
March 28, 2022, at 2:00 PM

I have a pandas data frame:

import pandas as pd
X = pd.DataFrame({'col1': [1,2],
                  'col2': [4,5]})

I have a replacement dictionary:

dict_replace = {
    'col1': {1:'a', 2:'b'},
    'col2': {4:'c', 5:'d'}
}

I can easily replace the values in X using:

X = X.replace(dict_replace)

Resulting in:

X = pd.DataFrame({'col1': ['a','b'],
                  'col2': ['c','d']})

However, if a new value appears in X which is not in dict_replace (for the respective column) I want it replaced with np.nan.

For example, a data frame:

X = pd.DataFrame({'col1': [1,2,3],
                  'col2': [4,5,7]})

Should look like:

X = pd.DataFrame({'col1': ['a','b',np.nan],
                  'col2': ['c','d',np.nan]})

What are some ways I can do this without having to iterate?

Answer 1

You are looking for pandas.Series.map, which, though only available on columns, can be used on the whole dataframe with apply:

X = X.apply(lambda col: col.map(dict_replace[col.name]))

Output:

>>> X
  col1 col2
0    a    c
1    b    d
2  NaN  NaN
Answer 2

Try with mask

out = X.replace(dict_replace).mask(lambda x : x==X)
Out[215]: 
  col1 col2
0    a    c
1    b    d
2  NaN  NaN
READ ALSO
Python, calculator class definition

Python, calculator class definition

I'm struggling with a practice problem below where I'm confused as to what the definitions would look like

98
python pydantic How to choose class based on another value

python pydantic How to choose class based on another value

some of example of schemaI have different constraints based on type

106
How to fit UMAP object with an embedding_ property of shape (200,2)

How to fit UMAP object with an embedding_ property of shape (200,2)

I'm trying to take the first 200 rows of my dataframe and pass it through a UMAP fit method but I'm not able to get it to shape the way I want at (200,2)Should I add an embedding parameter?

77
Django - passing one model to another while creating new object

Django - passing one model to another while creating new object

My site has multiple tests with multiple questions eachI'd like to make question creation form which will have preset test object depended on url

78