Python Web Scrape - Forループの問題

Question

私はこのプログラムを習得しようとしているので、私はあまりにも私と一緒に裸のpythonに慣れています。私はこの小さなプロジェクトでウェブを傷つけます。私は競争相手のために別のウェブサイトを完成させましたが、私は現在のものと苦労しています。

コードが現在行っているのは、（私が欲しい）csvファイルを作成することです.csvファイルでは、ヘッダは表示されていますが、その下にはデータはありません。

私のforループスクリプトで助けてもらえる人がいますか？私はそれがCSVファイルに書き込むためにデータをキャプチャしていないと信じています。

ありがとうございました。以下は

from urllib.request import urlopen as uReq 
from bs4 import BeautifulSoup as soup 

# setting my_url to the wesite 
urls = ['https://www.extraspace.com/Storage/Facilities/US/North_Carolina/Charlotte/1000000398/Facility.aspx' 
    , 'https://www.extraspace.com/Storage/Facilities/US/North_Carolina/Charlotte/1000000404/Facility.aspx'] 

#https://www.extraspace.com/Storage/Facilities/US/North_Carolina/Charlotte/1000000398/Facility.aspx?cid=org::maps&utm_source=google&utm_medium=organic&utm_campaign=org::maps 



filename = "extraspace.csv" 
open(filename, 'w').close() 
f = open(filename, "a") 
num = 0 

headers = "unit_size, size_dim1, unit_type, online_price, reg_price, street_address, store_city, store_postalcode\n" 

f.write(headers) 

for my_url in urls: 
    # Opening up connection, grabbing the page 
    uClient = uReq(my_url) 

    # naming uClient to page_html 
    page_html = uClient.read() 

    # closing uClient 
    uClient.close() 

    # this does my html parsing 
    page_soup = soup(page_html, "html.parser") 

    # setting container to capture where the actual info is using inspect element 

    #----- <div class="right-col-unit-listings" class="unit-listings"> ==$0 -------this is body of each unit container 
    #----- <div class="results"> ==$0 -------this is body of each unit container 
    #grabs each product 
    containers = page_soup.findAll("div", {"itemprop": "makesOffer"}) 

    #----- <div itemprop="address" itemscope itemtype="http://schema.org/PostalAddress"> ==$0  -----this is body of address 
    #grabs address 
    store_locator = page_soup.findAll("div", {"itemprop": "address"}) 

    f.write("website " + str(num) + ": \n") 
    for container in containers: 
     for store_location in store_locator: 
      street_address = store_location.findAll("span", {"itemprop": "streetAddress"}) 
      store_city = store_location.findAll("span", {"itemprop": "addressLocality"}) 
      store_postalcode = store_location.findAll("spand", {"itemprop": "postalCode"}) 
      title_container = container.findAll("div", {"class": "size RamaGothicSemiBold"}) 
      size_dim = container.findAll("div", {"itemprop": "description"}) 
      #unit_type = container.findAll("ul", {"itemprop": "description"}) 
      unit_container = container.ul.li 
      unit_type = container.text 
      online_price = container.findAll("div", {"itemprop": "price"}) 
      reg_price = container.findAll("div", {"class": "rate strikeout"}) 

     for item in zip(title_container, size_dim, unit_type, online_price, reg_price, street_address, store_city, store_postalcode): 
      csv = item[0].text + "," + item[1].text + "," + item[2] + "," + item[3].text + "," + item[4].text + "," + item[5].text + "," + item[6].text + "," + item[7].text + "\n" 
      f.write(csv) 
    num += 1 

は、コンテナのHTMLです：：

<div itemprop="makesOffer" itemscope="" itemtype="http://schema.org/Offer"> 
    <div itemprop="itemOffered" itemscope="" itemtype="http://schema.org/Product"> 
     <div class="guide"> 
      <div class="size-help-lnk size-guide hidden" data-locker="False" data-square-feet="25">Size Help</div> 
      <div alt="5x5" class="video-btn-5x5 video-link" onclick="trackSC('UnitListingVideo');"></div> 
     </div> 
     <div class="size RamaGothicSemiBold"> 
      <div itemprop="description">5' x 5'</div> 
      <div>SMALL</div> 
     </div> 
     <div class="features"> 
      <ul itemprop="description"> 
       <li><i class="check-icon"></i>Enclosed Storage</li> 
       <li><i class="check-icon"></i>Indoor</li> 
       <li><i class="check-icon"></i>1st Floor Access</li> 
      </ul> 
     </div> 
    </div> 
    <div> 
     <div class="rate strikeout"> 
      <div><span style="width:100%;"></span>$57</div><span class="StreetRate">IN-STORE</span></div> 
     <div class="rate"> 
      <div content="35.00" itemprop="price">$35 
       <meta content="USD" itemprop="priceCurrency" /> 
      </div><span class="WebRate">WEB RATE</span></div> 
     <div class="promo"><span style="color:#000;">Act fast:<br/>Limited units</span></div> 
    </div> 
    <a class="btn btn-orange cta-test is-vehicle" href="https://www.extraspace.com/Storage/ReserveOrHold.aspx?uid=a0GC000000tUNupMAG" id="ctl00_mContent_UnitListPopular_ctrl0_hlReserveLink" onclick="upDown('unitRows|8506|1;05X05|NDN|57|35| | ; | | | | | ; | | | | | ;05X05|CDN|71|48| | ;05X07|CDN|74|50| | ');">RESERVE</a> 
    <div class="clear"></div> 
    <link href="http://schema.org/OnlineOnly" itemprop="availability"> 
    </link> 
</div> 

、最終的には下記のアドレスのためのHTMLです：

< div itemprop = "address" 
    itemscope = "" 
    itemtype = "http://schema.org/PostalAddress" > 
    <span id = "ctl00_mContent_lbAddress" 
    itemprop = "streetAddress" > 3304 Eastway Dr < br/> Ste D < /span><br/ > 
    <span id = "ctl00_mContent_lbCity" 
    itemprop = "addressLocality" > Charlotte < /span>, < 
    span id = "ctl00_mContent_lbState" 
    itemprop = "addressRegion" > NC < /span> < 
    span id = "ctl00_mContent_lbPostalCode" 
    itemprop = "postalCode" > 28205 < /span> < 
    /div>] 

Answer 1

があります以下

はPythonスクリプトですあなたはそれを一度だけ取得する必要があるので、Webページ上の1つのアドレス。だから、forループの1つを取り除くことができます。

store_postalcode = store_location.findAll("spand", {"itemprop": "postalCode"}) 

この場合、「なし」の値になります。

最短入力反復可能が使い果たされるときにジップドキュメントH ttps://docs.python.org/3/library/functions.html#zip

からイテレータが停止します。

したがって、[なし]の値で停止したため、出力が得られませんでした。それ以外の場合は、アドレスの詳細が使い果たされたときに1回の反復後に停止してしまいます。これらのあなたのコードを修正

作品：

from urllib.request import urlopen as uReq 
from bs4 import BeautifulSoup as soup 
import re 

urls = ['https://www.extraspace.com/Storage/Facilities/US/North_Carolina/Charlotte/1000000398/Facility.aspx' 
    , 'https://www.extraspace.com/Storage/Facilities/US/North_Carolina/Charlotte/1000000404/Facility.aspx'] 

filename = "extraspace.csv" 
open(filename, 'w').close() 
f = open(filename, "a") 
num = 0 
headers = "unit_size_0, unit_size_1, size_dim1, unit_type, online_price, reg_price, street_address, store_city, store_postalcode\n" 
f.write(headers) 

for my_url in urls: 
    uClient = uReq(my_url) 
    page_html = uClient.read() 
    uClient.close() 
    page_soup = soup(page_html, "html.parser") 

    street_address = page_soup.find("span", {"itemprop": "streetAddress"}).text 
    store_city = page_soup.find("span", {"itemprop": "addressLocality"}).text 
    store_postalcode = page_soup.find("span", {"itemprop": "postalCode"}).text 

    containers = page_soup.findAll("div", {"itemprop": "makesOffer"}) 
    for container in containers: 
     title_container = container.findAll("div", {"class": "size RamaGothicSemiBold"}) 
     size_dim = container.findAll("div", {"itemprop": "description"}) 
     unit_type = container.findAll("ul", {"itemprop": "description"}) 
     online_price = container.findAll("div", {"itemprop": "price"}) 
     reg_price = container.findAll("div", {"class": "rate strikeout"}) 

     for item in zip(title_container, size_dim, unit_type, online_price, reg_price): 
      i= re.match(r"([^A-Z]*)([A-Z]*)", item[0].text.replace('\n', '').strip("\"")) 
      csv = i.group(1) + "," + i.group(2) + "," + item[1].text + "," + item[2].text + "," + item[3].text + "," + item[4].text + "," \ 
        + street_address + "," + store_city + "," + store_postalcode + "\n" 
      f.write(csv) 
    num += 1 

f.close() 

あなたはPythonのcsvモジュールに終わりhttps://docs.python.org/3/library/csv.html