TornadoはStreamClosedErrorを発生させます：ストリームは閉じます

Question

私は、非同期APIサービスを構築するためにtornadoとasyncmcを使用します。数週間は正常に動作しますが、今日問題が発生しました。 asyncmcによってキャッシュされたほとんどすべてのサービスは、StreamClosedErrorを発生させます。

竜巻ログから、それは言った：

ile "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/web.py", line 1348, in _execute 
    result = yield result 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run 
    value = future.result() 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result 
    raise_exc_info(self._exc_info) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run 
    yielded = self.gen.throw(*sys.exc_info()) 
    File "/opt/app/python/tv-appstore/handler/base.py", line 100, in wrapper 
    result = yield mc.get(self.cache_key) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run 
    value = future.result() 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result 
    raise_exc_info(self._exc_info) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run 
    yielded = self.gen.throw(*sys.exc_info()) 
    File "build/bdist.linux-x86_64/egg/asyncmc/client.py", line 63, in wrapper 
    res = yield func(self, conn, *args, **kwargs) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run 
    value = future.result() 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result 
    raise_exc_info(self._exc_info) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run 
    yielded = self.gen.throw(*sys.exc_info()) 
    File "build/bdist.linux-x86_64/egg/asyncmc/client.py", line 238, in get 
    result = yield self._multi_get(conn, self._key_type(key=key)) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run 
    value = future.result() 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result 
    raise_exc_info(self._exc_info) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run 
    yielded = self.gen.throw(*sys.exc_info()) 
    File "build/bdist.linux-x86_64/egg/asyncmc/client.py", line 278, in _multi_get 
    servers_resp = yield conn.send_cmd_all(cmd, stream=True) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run 
    value = future.result() 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result 
    raise_exc_info(self._exc_info) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 810, in run 
    yielded = self.gen.throw(*sys.exc_info()) 
    File "build/bdist.linux-x86_64/egg/asyncmc/pool.py", line 91, in send_cmd_all 
    server_resp = yield host.send_cmd(cmd, *arg, **kw) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 807, in run 
    value = future.result() 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/concurrent.py", line 209, in result 
    raise_exc_info(self._exc_info) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/gen.py", line 212, in wrapper 
    yielded = next(result) 
    File "build/bdist.linux-x86_64/egg/asyncmc/host.py", line 77, in send_cmd 
    yield self.stream.write(cmd) 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/iostream.py", line 356, in write 
    self._check_closed() 
    File "/opt/python2.7/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/iostream.py", line 864, in _check_closed 
    raise StreamClosedError("Stream is closed") 

Answer 1

私は同じ問題がありました。これは、minsize（asyncmc.Clientの場合）を1より大きく設定し、ConnectionPoolへの接続が、使用されているライブラリ（tornado、asyncmc）を超えて壊れた場合に発生します。この問題を解決するために、私は自分の方法でキャッシュから値を取得：

self.mc_get(key) 

代わり

self.mc.get(key) 

の

と呼んself.mc.pool.clear（）は、接続のプールをきれいにすると、初期の「完全再接続」。

Pythonコード：

import tornado 
from tornado.iostream import StreamClosedError 

class SomeBaseHandler(tornado.web.RequestHandler): 

    def mc_get(self, key, default=None): 
     """ Additional interlayer for debug and logging requested keys from memcached 

     Args: 
      key (str): name of key in memcached 
      default: default value, returns if key not exists 

     Returns: 
      obj 
     """ 
     try: 
      return yield self.mc.get(key, default) 
     except StreamClosedError as e: 
      self.mc.pool.clear() 
      # Here you can raise own exception 
      return default 

    # … 

私はメソッドに追加のコードを追加することなく、無限ループを避けるために、再帰呼び出しmc_getを使用しないでください。