how to iterate each row of one dataframe and compare with rows in another dataframe in Python?

48
October 22, 2019, at 7:20 PM

I have two dataframes:

DF1:

ID     v1           v2         v3
289  1455.0        2.0        0.62239  
289  1460.0        0.0        0.46037  
289  1465.0        4.0        0.41280 
290  1470.0        0.0        0.39540 
290  1475.0        2.0        0.61809 
290  1475.0        2.0        0.61809

DF2:

ID     v1           v2         v3
289  1423.0        2.0        0.62239  
289  142Q.0        0.0        0.46037  
289  14FW.0        4.0        0.41280  
290  14Q3.0        0.0        0.39540  
290  1453.0        2.0        0.61809 
290  1454.0        2.0        0.61809

I want to iterate each row in DF1 with every row in DF2 and see if it is in DF2, something like:

for row in results_01.iterrows():
    diff = []
    if row not in results_02:
        add different one to 'diff'
        print(diff)

I know the logic but not sure how to do this, new to Python, can anyone help me? Many thanks.

Answer 1

You can do it easily with ‘inner’ merge.

intersect = pd.merge(df1, df2, how='inner')

Edit:

It turns out that the rows that are in df1 and not in df2 are wanted and not the intersection. In this case one should use the isin pandas method. Here is SO link that deals with it.

Answer 2

One way to do it (maybe not the most efficient) would be to append the dataframes together and then drop duplicates, like so:

full_df = df1.append(df2)
full_df = full_df.drop_duplicates(keep=False)
Answer 3

The code block you have looks pretty close to what you do in python. Take a row from one dataframe and iterate through the other dataframe looking for matches.

for index, row in results_01.iterrows():
    diff = []
    compare_item = row['col_name'] 
    for index, row in results_02.iterrows():
       if compare_item == row['compare_col_name']:
           diff.append(compare_item, row['col_name']
    return diff 

Here I am taking a specific column value from a row from one dataframe and comparing it to another value from the other dataframe

READ ALSO
Local Storage using JavaScript/jQuery to store the favorites from the list and show the favorites list when the user clicks the favorites tab

Local Storage using JavaScript/jQuery to store the favorites from the list and show the favorites list when the user clicks the favorites tab

I am having a PHP page where the list loads from the databaseI am having a star icon near each item on clicking the icon changes color and that will be stored in the local storage

51
show and hide spinner with jquery : does not work on Iphone, safary etc

show and hide spinner with jquery : does not work on Iphone, safary etc

I am trying to show / hide a spinner with JQueryBasicaly I have this code :

50
Using beforeSend and complete in every instance of ajaxForm globally

Using beforeSend and complete in every instance of ajaxForm globally

I am trying to use the beforeSend and complete functions on all ajaxForms in a projectI would like to do this without adding the exact bit a code all throughout the project

25
Convert a SQL query to Laravel Eloquent

Convert a SQL query to Laravel Eloquent

I'm trying to achieve something like the following:

42