Looking for differences in two CSV files with Python

199
August 07, 2021, at 02:00 AM

I have a script that scrapes a site and puts specific site names into a csv. Some days it has 0 site names and some days it has more the 4. I have another script that takes the csv from today and the csv from yesterday and compares the two. If today's csv has site names that were also on yesterday's csv I want to outfile those site names to a different txt file. I have:

with open(filepath + today + filename, 'r') as t1, open(filepath + yesterday + filename, 'r') as t2:
    fileone = t1.readlines()
    filetwo = t2.readlines()
with open(checklistFile, 'w') as outfile:
    for line in fileone:
        if line in file:
            outfile.write(line)
            print("bad")
        else:
            outfile.write("good")
            print("good")

this only works if the csvs have the same number of lines and only works if they are in the same order. For instance, if today had "site1, site2, site3" and yesterday had "site4, site1, site5", this script would miss it. Any help would be appreciated. I'm running Python 2.7 so I cant use csv-diff.

Answer 1

You can achieve this using pandas:

import pandas as pd
df_today = pd.read_csv(filepath + today + filename)
df_yesterday = pd.read_csv(filepath + yesterday + filename)
df_common = pd.concat([df_today,df_yesterday])
duplicates_df = df_common[df_common.duplicated()]
duplicates_df.to_csv(checklistFile, index=False)
Rent Charter Buses Company
READ ALSO
Check for the status of insufficient balance from Stripe during create subscription

Check for the status of insufficient balance from Stripe during create subscription

Currently I am facing an issue of not able to retrieve the status of insufficient balance during subscription creationI am using the below link to retrieve status as a reference, but not able to do this

247
Css selector nth sibling and stops if it encounters another parent class

Css selector nth sibling and stops if it encounters another parent class

Is it possible to have an nth sibling selector for the code below?

190
When use nested transaction, will the nested transaction see changes maded by the outside transaction?

When use nested transaction, will the nested transaction see changes maded by the outside transaction?

If the transaction type of inner transaction is REQUIRES_NEW, will it see the changes made by outside transaction? What about other transaction type?

172
CSS: how to make a div extend beyond a scrollable container

CSS: how to make a div extend beyond a scrollable container

I have a scrollable container - a div with overflow-y: scroll and position: relative

190