pythhonデータフレームのフィルタ条件：任意のより高速な方法

Question

parts_list = imp_parts_df['Parts'].tolist() 
sub_week_list = ['2016-12-11', '2016-12-04', '2016-11-27', '2016-11-20', '2016-11-13'] 
i = 0 
start = DT.datetime.now() 
for p in parts_list: 
     for thisdate in sub_week_list: 
      thisweek_start = pd.to_datetime(thisdate, format='%Y-%m-%d') #'2016/12/11' 
      thisweek_end = thisweek_start + DT.timedelta(days=7) # add 7 days to the week date 

      val_shipped = len(shipment_df[(shipment_df['loc'] == 'USW1') & (shipment_df['part'] == str(p)) & (shipment_df['shipped_date'] >= thisweek_start) & (shipment_df['shipped_date'] < thisweek_end)]) 

print (DT.datetime.now() - start).total_seconds() 

shipment_dfは

パーツリストは、436の部品

sub_week_listは、それが全体的な438.13秒を取ったことで5日付

を持っていた周りの35000レコードを持っていますこのコードを実行してください

それを行うより速い方法があります

Answer 1

parts_list = imp_parts_df['Parts'].astype(str).tolist() 
i = 0 
start = DT.datetime.now() 
for p in parts_list: 

    q = 'loc == "xxx" & part == @p & "2016-11-20" <= shipped_date < "2016-11-27"' 
    val_shipped = len(shipment_df.query(q)) 

print (DT.datetime.now() - start).total_seconds()