Compare column contents of CSV file using dataframe in python

418
February 02, 2017, at 02:46 AM

I have a CSV file as such

ID  OLD_A  NEW_A  OLD_B     NEW_B     OLD_C  NEW_C
1   0      0      1/1/2017  1/1/2017  ABC    BCD
2   0      0      1/1/2017  2/1/2017  ABC    ABC
3   1      2      1/1/2017  1/1/2017  ABC    BCD

I want to compare the old and new columns for A, B and C and in case there's a difference in the values between OLD and NEW (there's 10k+ rows) , I want return an output as such (from above example):

ID Field_Changed OLD_Value   NEW_Value
1  C             ABC         BCD
2  B             1/1/2017    2/1/2017
3  A             1           2
3  C             ABC         BCD

I'm using the following code:

df = pd.read_csv('test.csv')
df = df.set_index('ID')
df.columns = df.columns.str.split('_', expand=True)
df1 = df.stack()[['OLD','NEW']].reset_index().rename(columns={'level_1':'Field_Changed'})
df2 = df1[df1['OLD'].ne(df1['NEW']).values]

Just wondering, is there a better way to approach this? Maybe a for loop? Any thoughts would be appreciated!

READ ALSO
Broadcasting a 1d numpy array with a 2d numpy array

Broadcasting a 1d numpy array with a 2d numpy array

This is probably a really simple question, but I am not figuring this out

501
SQLAlchemy populate object relationships without persisting to database

SQLAlchemy populate object relationships without persisting to database

I have a problem while using SQLAlchemy where I would like to dig into a managed object's relstionships before pushing it to the databaseI have a schema like this:

434
Django Ajax Post does not work properly

Django Ajax Post does not work properly

Hello guys I'm trying to pass a variable from jquery to the views trying it returns in the DOM the value but it does not work for me

442
pyopengl - finding the edges of a shape during rotation

pyopengl - finding the edges of a shape during rotation

The idea behind this is to create a detection area for a security cameraCurrently, I know how to find and use the modelview matrix data as shown below in the function "matrixTransformation"

400