S3文件使用路易吉当地引起了统一codeDE codeError路易、当地、文件、codeDE

2023-09-11 09:33:04 作者:别再让我心动了

我复制 PDF 文件到本地,使用下面这段code:

I am copying a pdf file to local, using the following piece of code:

with self.input_target().open('r') as r:
    with self.output_target().open('w') as w:
       for line in r: 
           w.write(line) 

(种)

这是基于这个问题

但是,当我执行了code,我得到了以下内容:

But when I execute that code I get the following:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 11:
      invalid continuation byte

这个我试过其他的方法,没有了良好的效果:

I tried this other approach, without good results:

   with self.input_target().open('r') as r, self.output_target().open('w') as w:                                                                                                                        
       w.write(r.read())                         

什么是做这件事的正确方法是什么?

What is the correct way of doing it?

推荐答案

我最终使用 luigi.s3.S3Client 和方法 GET ,它复制一个二进制文件从/到 S3

I ended using luigi.s3.S3Clientand the method get which copies a binary file from/to the s3.

摘录:

进口路易吉

class ATask(luigi.Task):
    s3_path = luigi.Parameter()
    local_path = luigi.Parameter()
    ...

    def run(self):
        client = luigi.s3.S3Client()
        client.get(s3_path, local_path)  ## This gets the file
        ...

我认为,根本原因在于路易吉使用博托获取/把文件从/到 S3 。 (你可以在源$ C ​​$ C见)

I think that the underlying reason is that luigi uses boto for getting/putting files from/to the s3. (As you can see in the source code)

 
精彩推荐
图片推荐