2017-03-28 3 views
0

6個のxlsxファイルをそれぞれ数MBずつマージしています。彼らは一般的に同じデータを持っていますが、いくつかの余分な変数(私が保持したい)といくつかは同じ変数のためのユニークな名前を持っています。以下は私が試してみたPythonコードですが、私はMemoryErrorを取得しました(私はタスクマネージャーで、その時に〜2GBのRAMを占有していました)。このプログラムでは、ファイルサイズを考慮して、数百MBのRAMを私に費やすことは意味がありません。だから、誰かが私のエラーがどこにあるのかを指摘できると思っています。比較的小さいエクセルファイル上に連結されたPandas MemoryError

import pandas as pd 
import os.path 

fpath = 'K:\\DATA\\python\\Mergingtest\\tess\\' 

#Specify file locations and sheet names 
data_headers = pd.read_excel(os.path.join(fpath, 'TESS_Headers_2006-2015.xlsx'), sheetname='Sheet1', header=1) 
data_07 = pd.read_excel(os.path.join(fpath, 'TESS 2007.xlsx'), sheetname='Errors', header=0) 
data_08 = pd.read_excel(os.path.join(fpath, 'TESS 2008.xlsx'), sheetname='Errors', header=0) 
data_09 = pd.read_excel(os.path.join(fpath, 'TESS 2009.xlsx'), sheetname='Errors', header=0) 
data_10 = pd.read_excel(os.path.join(fpath, 'TESS 2010.xlsx'), sheetname='Sheet1', header=0) 
data_11 = pd.read_excel(os.path.join(fpath, 'TESS 2011.xlsx'), sheetname='Sheet1', header=0) 
data_12_15 = pd.read_excel(os.path.join(fpath, 'COPY_TESS 2012-2015 Master File.xlsx'), sheetname='Sheet1', header=0) 

#Standardizing naming for equivalent variables from each file 
data_headers.columns = ["fldErrorID", "fdSiteCaseID", "fdSiteID", "fdYear", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC", "fdReviewedWithPersonInvolved", "fdBloodSupplierManufacurerNotifi", "cptr_delay2", "clin_lab_err", "sitegraded", "cptr_delay", "gen_code1", "disc_cnsq1", "cnsqnc_gen", "capacity1", "capacity", "severity1", "SECTION A - DISCOVERY INFORMATION: Describe the event", "SECTION A - DISCOVERY INFORMATION: Describe how the event was discovered", "SECTION A - DISCOVERY INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Did the actual event harm the patient?", "SECTION B - OCCURRENCE INFORMATION: If event caused harm to patient, describe", "SECTION B - OCCURRENCE INFORMATION: Was the near-miss event discovered because of a planned/unplanned detection procedure/process?", "SECTION B - OCCURRENCE INFORMATION: Discovery/Consequence code: Specify", "SECTION B - OCCURRENCE INFORMATION: Actual Severity", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 4", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 8", "SECTION B - OCCURRENCE INFORMATION: What was the outcome of the event to the patient up to 30 days after it was discovered?", "SECTION B - OCCURRENCE INFORMATION: If patient died, did the consequence of the event play a role in death?", "Date Created", "Case State", "Site ID grading", "dvcq"] 
data_07.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC"] 
data_08.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "fdAccessionNum", "sitegraded", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "gen_code1", "event2", "fdDiscoveryConsequenceEvent", "cnsqnc_gen", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "Site ID grading", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC"] 
data_09.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "sitegraded", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "gen_code1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC"] 
data_10.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "fdAccessionNum", "sitegraded", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "cptr_delay", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "gen_code1", "dvcq", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "capacity1", "capacity", "severity1", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdNoAction", "fdClosed", "fdDiscoveryTime", "fdReviewedWithPersonInvolved", "fdTimeEventOccurred", "cptr_delay2", "clin_lab_err"] 
data_11.columns = ["fldErrorID", "fdSiteCaseID", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "fdYear", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSiteID", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC"] 
data_12_15.columns = ["fldErrorID", "fdSiteCaseID", "fdSiteID", "fdYear", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC", "fdReviewedWithPersonInvolved", "fdBloodSupplierManufacurerNotifi", "cptr_delay2", "clin_lab_err", "sitegraded", "cptr_delay", "gen_code1", "disc_cnsq1", "cnsqnc_gen", "capacity1", "capacity", "severity1", "SECTION A - DISCOVERY INFORMATION: Describe the event", "SECTION A - DISCOVERY INFORMATION: Describe how the event was discovered", "SECTION A - DISCOVERY INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Did the actual event harm the patient?", "SECTION B - OCCURRENCE INFORMATION: If event caused harm to patient, describe", "SECTION B - OCCURRENCE INFORMATION: Was the near-miss event discovered because of a planned/unplanned detection procedure/process?", "SECTION B - OCCURRENCE INFORMATION: Discovery/Consequence code: Specify", "SECTION B - OCCURRENCE INFORMATION: Actual Severity", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 4", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 8", "SECTION B - OCCURRENCE INFORMATION: What was the outcome of the event to the patient up to 30 days after it was discovered?", "SECTION B - OCCURRENCE INFORMATION: If patient died, did the consequence of the event play a role in death?", "Date Created", "Case State"] 

concat = (pd.concat([data_headers,data_07,data_08,data_09,data_10,data_11,data_12_15])) 
#specify column order to prevent default alphabetical-ordering 
concat = concat[["fldErrorID", "fdSiteCaseID", "fdSiteID", "fdYear", "fdProvince", "fdAccessionNum", "fdDateEventOccurred", "fdDateStamp", "fdDiscoveredDuringSpecify", "fdDiscoveryDate", "fdPersonInvolvedSpecify", "fdProductRecordActionSpecify", "event1", "event2", "fdDiscoveryConsequenceEvent", "fdLocationCodeDiscovery", "fdLocationCodeOccurence", "fdDayOfWeekOccurred", "fdDiscoveredDuring", "fdDiscoveryTime", "fdPersonInvolved", "fdTimeEventOccurred", "fdDayOfWeekOccurredHoliday", "fdAdditionalTesting", "fdFloorClinicNotified", "fdOther", "fdProvincialID", "fdPatientSampleRecollected", "fdProductDestroyed", "fdProductRetrieved", "fdRecordCorrected", "fdSeverity", "fdNumberOfUnits", "fdProductType", "fdDiscovery", "fdProductDenied", "fdDiscoverersJob", "fdDiscoverersJobSpecify", "fdProductRecordAction", "fdNoAction", "fdClosed", "fldShareWithPHAC", "fdReviewedWithPersonInvolved", "fdBloodSupplierManufacurerNotifi", "cptr_delay2", "clin_lab_err", "sitegraded", "cptr_delay", "gen_code1", "disc_cnsq1", "cnsqnc_gen", "capacity1", "capacity", "severity1", "SECTION A - DISCOVERY INFORMATION: Describe the event", "SECTION A - DISCOVERY INFORMATION: Describe how the event was discovered", "SECTION A - DISCOVERY INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Sub-location", "SECTION B - OCCURRENCE INFORMATION: Did the actual event harm the patient?", "SECTION B - OCCURRENCE INFORMATION: If event caused harm to patient, describe", "SECTION B - OCCURRENCE INFORMATION: Was the near-miss event discovered because of a planned/unplanned detection procedure/process?", "SECTION B - OCCURRENCE INFORMATION: Discovery/Consequence code: Specify", "SECTION B - OCCURRENCE INFORMATION: Actual Severity", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 1", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 2", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 3", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Product type - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Name of product: Specify - 4", "SECTION B - OCCURRENCE INFORMATION: If product destroyed, provide products: Number of units - 4", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 5", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 6", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 7", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Product type - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Name of product: Specify - 8", "SECTION B - OCCURRENCE INFORMATION: If product denied, provide products: Number of units - 8", "SECTION B - OCCURRENCE INFORMATION: What was the outcome of the event to the patient up to 30 days after it was discovered?", "SECTION B - OCCURRENCE INFORMATION: If patient died, did the consequence of the event play a role in death?", "Date Created", "Case State", "Site ID grading", "dvcq"]] 

concat.to_excel(os.path.join(fpath, 'Merge_Output.xlsx')) 

答えて

0

ではなく

data_07 = pd.read_excel(os.path.join(fpath, 'TESS 2007.xlsx'), sheetname='Errors') 
data_08 = pd.read_excel(os.path.join(fpath, 'TESS 2008.xlsx'), sheetname='Errors') 
#keep headers for all xls files 

concat =pd.concat([data_07,data_08], axis = 0) 
#remove data headers and add axis = 0 

#get rid of individual files 
del data_07 
del data_08 
+0

を試してみてくださいだから、これは成功しましたが、それはまだRAMの使用量の数百MBに上りました。それは普通ですか?ファイルはそれぞれわずか数MBですが、〜10k行が含まれています。 – melonfacedoom

+0

その後、単一のデータフレームを削除する必要があります。あなたがそれで元気になれば編集コード –

+0

を見てください。 –

関連する問題