有効なjsonでは、.txtを解析してCSVでツイートを保存する方法は？

Question

私はこのようなjsonつぶやきを持つtxtファイルを持っています。すべてのツイートは["text"：& "is_quote_status"：]の間にあり、日付は行末にあり、つぶやきと日付だけをcsvファイルに抽出する方法？

私は最終的に2つのコードでこれを分解することができました。 @markのおかげで、それを理解するのに時間がかかりましたが、私はそれをしました。私は

{ 
    "_id": "582f4fbd44b65941a0a81213", 
    "contributors": null, 
    "truncated": false, 
    "text": "Tonight at 10 PM ET, 7 PM PT, on @FoxNews, a one hour special on me and my life by @HarveyLevinTMZ. Enjoy!", 
    "is_quote_status": false, 
    "in_reply_to_status_id": null, 
    "id": "799660246788612100", 
    "favorite_count": 15765, 
    "source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>", 
    "retweeted": false, 
    "coordinates": null, 
    "entities": { 
     "symbols": [], 
     "user_mentions": [{ 
      "id": 1367531, 
      "indices": [33, 41], 
      "id_str": "1367531", 
      "screen_name": "FoxNews", 
      "name": "Fox News" 
     }, { 
      "id": 36098990, 
      "indices": [83, 98], 
      "id_str": "36098990", 
      "screen_name": "HarveyLevinTMZ", 
      "name": "Harvey Levin" 
     }], 
     "hashtags": [], 
     "urls": [] 
     }, 
     "in_reply_to_screen_name": null, 
     "in_reply_to_user_id": null, 
     "retweet_count": 5251, 
     "id_str": "799660246788612100", 
     "favorited": false, 
    "user": { 
     "id": 25073877, 
     "id_str": "25073877" 
     }, 
     "geo": null, 
     "in_reply_to_user_id_str": null, 
     "lang": "en", 
     "created_at": "Fri Nov 18 17:07:14 +0000 2016", 
     "in_reply_to_status_id_str": null, 
     "place": null, 
     "created_at_date": "2016-11-18T17:07:14Z" 
    } 

Answer 1

をhad-ツイートを抽出するにはJSONファイル：

fin = open("sim.txt") 
fout = open("output.txt", "w+") 
delete_list = ['ObjectId(', 'NumberLong(','ISODate(', ')'] 
for line in fin: 
    for word in delete_list: 
     line = line.replace(word, "") 
    fout.write(line)  
fin.close() 
fout.close() 

をきれいにする

と.CSV

import json 
import csv 
infile = open("output1.txt","r") 
outfile=open("output4.csv","w") 
json_s=infile.read() 
writer=csv.writer(outfile) 
for data in(json.loads(json_s)): 
    x=data['text'].encode("utf-8") 
    y=data['created_at_date'].encode("utf-8") 
    writer.writerow([x,y]) 
infile.close() 
outfile.close() 
print 'DONE' 

に格納する日付サンプル.json文字列jsonのパスをメモし、テキストファイルに有効なjsonが必要です。

/path/to/json/file.json

[{ 
     "_id": "dummyid1", 
     "contributors": null, 
     "truncated": false, 
     "text": "Dummy tweet 1", 
     "is_quote_status": false, 
     "in_reply_to_status_id": null, 
     "id": "799660246788612100", 
     "favorite_count": 15765, 
     "source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>", 
     "retweeted": false, 
     "coordinates": null, 
     "entities": { 
      "symbols": [], 
      "user_mentions": [{ 
       "id": 1367531, 
       "indices": [33, 41], 
       "id_str": "1367531", 
       "screen_name": "FoxNews", 
       "name": "Fox News" 
      }, { 
       "id": 36098990, 
       "indices": [83, 98], 
       "id_str": "36098990", 
       "screen_name": "HarveyLevinTMZ", 
       "name": "Harvey Levin" 
      }], 
      "hashtags": [], 
      "urls": [] 
     }, 
     "in_reply_to_screen_name": null, 
     "in_reply_to_user_id": null, 
     "retweet_count": 5251, 
     "id_str": "799660246788612100", 
     "favorited": false, 
     "user": { 
      "id": 25073877, 
      "id_str": "25073877" 
     }, 
     "geo": null, 
     "in_reply_to_user_id_str": null, 
     "lang": "en", 
     "created_at": "Fri Nov 18 17:07:14 +0000 2016", 
     "in_reply_to_status_id_str": null, 
     "place": null, 
     "created_at_date": "2016-11-18T17:07:14Z" 
    }, 
    { 
     "_id": "dummyid2", 
     "contributors": null, 
     "truncated": false, 
     "text": "Dummy tweet 2", 
     "is_quote_status": false, 
     "in_reply_to_status_id": null, 
     "id": "799660246788612100", 
     "favorite_count": 15765, 
     "source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>", 
     "retweeted": false, 
     "coordinates": null, 
     "entities": { 
      "symbols": [], 
      "user_mentions": [{ 
       "id": 1367531, 
       "indices": [33, 41], 
       "id_str": "1367531", 
       "screen_name": "FoxNews", 
       "name": "Fox News" 
      }, { 
       "id": 36098990, 
       "indices": [83, 98], 
       "id_str": "36098990", 
       "screen_name": "HarveyLevinTMZ", 
       "name": "Harvey Levin" 
      }], 
      "hashtags": [], 
      "urls": [] 
     }, 
     "in_reply_to_screen_name": null, 
     "in_reply_to_user_id": null, 
     "retweet_count": 5251, 
     "id_str": "799660246788612100", 
     "favorited": false, 
     "user": { 
      "id": 25073877, 
      "id_str": "25073877" 
     }, 
     "geo": null, 
     "in_reply_to_user_id_str": null, 
     "lang": "en", 
     "created_at": "Fri Nov 18 17:07:14 +0000 2016", 
     "in_reply_to_status_id_str": null, 
     "place": null, 
     "created_at_date": "2016-11-18T17:07:14Z" 
    } 
] 

script.py

import json 

with open('/path/to/json/file.json', 'r') as f: 
    json_string = f.read() 

datas_from_json = json.loads(json_string) # json string now a iterable list 

for data in datas_from_json: 
    print(data['text']) 

# outputs 
# Dummy tweet 1 
# Dummy tweet 2 

Answer 2

このプロセスはPandasを使用することによって簡単にすることができます。

/path/to/input.jsonまたは/path/to/input.txtに有効なjsonファイルがあるとします。ファイル拡張子は有効なjsonが存在する限り重要ではありません。

import pandas as pd 

df = pd.read_json("path/to/input.txt") 
df[["text", "created_at_date"]].to_csv("output.csv", index=False)