我有code写入文件到S3。在code是工作的罚款

I have code that writes files to s3. The code was working fine

    conn = S3Connection(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
    bucket = conn.get_bucket(BUCKET, validate=False)
    k = Key(bucket)
    k.key = self.filekey 
    k.set_metadata('Content-Type', 'text/javascript')
    k.set_contents_from_string(json.dumps(self.output))
    k.set_acl(FILE_ACL)

这是工作得很好。然后我发现我没有收我的连接，所以我说这行的末尾：

This was working just fine. Then I noticed I wasn't closing my connection so I added this line at the end:

    conn.close()

现在，该文件写道：作为之前的但的，我看到这个错误在我的日志现在

Now, the file writes as before but, I'm seeing this error in my logs now

    S3Connection instance has no attribute '_cache', unable to write file 

任何人都看到我在做什么错在这里，或知道是什么引起的？我注意到，没有对博托示人，关闭连接，但我知道你应该关闭连接的IO操作作为一般规则的教程...

Anyone see what I'm doing wrong here or know what's causing this? I noticed that none of the tutorials on boto show people closing connections but I know you should close your connections for IO operations as a general rule...

修改 关于这个的注意事项，当我注释掉 conn.close（）错误消失

EDIT A note about this, when I comment out conn.close() the error disappears

推荐答案

我无法找到最新的博托源$ C ​​$ C是错误信息，所以很遗憾我不能告诉你什么原因造成的。最近，我们有问题的时候，我们并没有叫 conn.close（），所以肯定是至少有一个情况下，你必须关闭连接。这是我的什么事情的理解：

I can't find that error message in the latest boto source code, so unfortunately I can't tell you what caused it. Recently, we had problems when we were NOT calling conn.close(), so there definitely is at least one case where you must close the connection. Here's my understanding of what's going on:

S3Connection（当然，它的父类）的处理几乎所有的连接细节透明的，你不应该去想关闭资源，重新连接，等等。这就是为什么大多数的教程和文档不提关闭资源。事实上，我只知道一个情况下，你应该明确关闭的资源，这是我描述的底部。阅读！

S3Connection (well, its parent class) handles almost all connectivity details transparently, and you shouldn't have to think about closing resource, reconnecting, etc.. This is why most tutorials and docs don't mention closing resources. In fact, I only know of one situation where you should close resources explicitly, which I describe at the bottom. Read on!

在幕后，博托使用 httplib的。此客户端库支持HTTP 1.1保持活动，因此它可以与应保持插座打开，以便它可以通过相同的连接执行多个请求。

Under the covers, boto uses httplib. This client library supports HTTP 1.1 Keep-Alive, so it can and should keep the socket open so that it can perform multiple requests over the same connection.

AWS将被关闭的连接（插座）有两个原因：

AWS will close your connection (socket) for two reasons:

按照博托源$ C ​​$ C，AWS开始后三分钟时间的事情了。 presumably东西的意思是空闲连接。 根据最佳实践使用Amazon S3 ，S3将接受多达100个请求关闭连接，然后（导致'连接复位'）。 According to the boto source code, "AWS starts timing things out after three minutes." Presumably "things" means "idle connections." According to Best Practices for Using Amazon S3, "S3 will accept up to 100 requests before it closes a connection (resulting in 'connection reset')."

幸运的是，博托通过回收过期连接三个著名前几分钟均达到能解决第一种情况。不幸的是，博托不处理第二种情况这么透明的：

Fortunately, boto works around the first case by recycling stale connections well before three minutes are up. Unfortunately, boto doesn't handle the second case quite so transparently:

当AWS关闭连接，您的连接的端部进入CLOSE_WAIT，这意味着该插座正在等待该应用程序以执行关闭（）。 S3Connection处理连接细节，以便透明，你不能真正直接做到这一点！这是最好的prevent它发生在首位。

When AWS closes a connection, your end of the connection goes into CLOSE_WAIT, which means that the socket is waiting for the application to execute close(). S3Connection handles connectivity details so transparently that you cannot actually do this directly! It's best to prevent it from happening in the first place.

所以，盘旋回，当您需要关闭明确的，如果你的应用程序运行很长一段时间的原来的问题，有一个参考，（再利用），很长一段时间一个博托连接，并通过使许多博托S3请求该连接（由此而引发的插座通过AWS一个连接重置），那么你会发现，越来越多的插座都在CLOSE_WAIT。您可以通过调用检查在Linux上这个条件的netstat |的grep CLOSE_WAIT 。为prevent这一点，做出博托的的Connection.close 您已经作出前100请求的显式调用。我们做了长期运行的进程数十万S3的请求，我们称之为的Connection.close 以后每隔，比如说，80请求。

So, circling back to the original question of when you need to close explicitly, if your application runs for a long time, keeps a reference to (reuses) a boto connection for a long time, and makes many boto S3 requests over that connection (thus triggering a "connection reset" on the socket by AWS), then you may find that more and more sockets are in CLOSE_WAIT. You can check for this condition on linux by calling netstat | grep CLOSE_WAIT. To prevent this, make an explicit call to boto's connection.close before you've made 100 requests. We make hundreds of thousands of S3 requests in a long running process, and we call connection.close after every, say, 80 requests.