how to interpolate grouped time seris (pandas DateFrame)

468
January 05, 2017, at 08:33 AM

I have data in type dataframe, looks like the following

type  date  sum
A     Jan-1 1
A     Jan-3 2
B     Feb-1 1
B     Feb-2 3
B     Feb-5 6

the task is to build a continuous time series for each type (the missing date should be filled with 0)

the expected result is

type  date  sum
A     Jan-1 1
A     Jan-2 0
A     Jan-3 2
B     Feb-1 1
B     Feb-2 3
B     Feb-3 0
B     Feb-4 0
B     Feb-5 6

is that possible to do that with pandas or other python tools?

the real dataset is of course millions of rows

Answer 1

You first must change your date to a datetime and put that column in the index to take advantage of resampling and then you can convert the date back to its original format

# change to datetime
df['date'] =pd.to_datetime(df.date, format="%b-%d")
df = df.set_index('date')
# resample to fill in missing dates
df1 = df.groupby('type').resample('d')['sum'].asfreq().fillna(0)
df1 = df1.reset_index()
# change back to original date format
df1['date'] = df1.date.dt.strftime('%b-%d')

output

  type    date  sum
0    A  Jan-01  1.0
1    A  Jan-02  0.0
2    A  Jan-03  2.0
3    B  Feb-01  1.0
4    B  Feb-02  3.0
5    B  Feb-03  0.0
6    B  Feb-04  0.0
7    B  Feb-05  6.0
Rent Charter Buses Company
READ ALSO
Iteration over an array is behaving as if the array is being modified during iteration, when this is not happening

Iteration over an array is behaving as if the array is being modified during iteration, when this is not happening

I’m writing some software for a weather station in pythonEvery minute, on the minute, new values are taken from sensors (the values are actually taken from a file that updates each minute) and added to an array

281
How to have radio buttons with materializecss and wtforms?

How to have radio buttons with materializecss and wtforms?

In materializecss you define the radio group of a radio button using the attribute name, but Flask-Wtforms binds the input with a atribute name

452
Float indexer to numpy array

Float indexer to numpy array

I want to make a numpy array that takes a float as a indexes (like, for example, a Pandas Series would if it had a float-type index)So let me explain

671
cannot import name Shrinkage

cannot import name Shrinkage

I'm running a python script which uses Shrinkage class from pyRiemann packageEven though that I've installed this package, I got this error:

343