在Amazon EC2上提高数据传输速率数据传输、速率、Amazon

2023-09-11 08:38:48 作者:雨落成殇

我有一个1千兆EBS卷装到一个EC2实例, 我复制从本地硬盘驱动器的二进制数据600MB(通过RDS连接) 和复制过程的窗口中显示的 10小时剩余。 虽然我有一个高速连接(100 + Mbps)的

I've got a 1-Gig EBS Volume mounted to an EC2 instance, I am copying 600MB of binary data from a Local Hard Drive (via RDS Connection) and the copying process windows is showing 10 Hours remaining. Though I have a High Speed connection (100+Mbps)

无论数据量,数据传输速率为1分钟/ MB(即16Kbs /秒) 我读白鲸在我的工作站前,或只是每天服用过之间犹豫。

Whatever the data volume, data transfer rate is 1 Min / MB (i.e 16Kbs / Sec) I am hesitating between reading Moby Dick in front of my workstation or just taking a day-off.

是否有任何合理的方案,以加快这一传输速率? (理想情况下512 Kbps的/秒,最小) 我对任何解决方案非常开放,缩短了上传/下载的时间/距离和EC2实例。

Are there any reasonable options to speed up this transfer rate ? (Ideally 512 Kbps/Sec at the minimum) I am very open to ANY solution to shorten up the uploading/downloading time to/from and EC2 instance.

在此先感谢。

编辑: 我只是偶然发现[亚马逊导入/导出服务] [1] 的 AWS导入/导出加速转移,你邮寄给我们大量的AWS云计算和移动存储设备之间的数据的通过发邮件给我们,他们从字面上的意思是你重大航运存储设备亚马逊。不要说这是石器时代,这是全新的技术,好家伙! : - )

EDIT : I just stumbled upon [Amazon Export/Import Service][1] "AWS Import/Export accelerates transferring large amounts of data between the AWS cloud and portable storage devices that you mail to us" By "mail to us", they literally mean you "materially" shipping your storage device to Amazon. Don't say this is Stoneage, this is BRAND new TECHNOLOGY, Dude ! :-)

EDIT2: 这听起来伟大:[Aspera的用于AWS] [2]但不幸的是太昂贵; 专为世界500强与大的需求和巨额现金。

EDIT2 : This sounded great: [Aspera for AWs][2] But unfortunately way too expensive; Tailored for Fortune 500 with big needs and big cash.

推荐答案

有时候,如果你想要的东西做得最好还是自己做:-) 我没有发现任何令人满意的过网,所以我花了晚上做了相当的完成台架试验我自己的。 我已经测试了几个备选方案/场景和这里的结果:

Sometimes, if you want something done it's better to do it yourself :-) I did not find anything satisfying over the net so I spent the evening doing a rather Complete Bench Test of my own. I have tested a few alternatives /scenarios and here are the results :

FTP服务器是Filezilla的服务器(正确的配置是敏感的) 用于此台架试验FTP客户端是WinSPC(Filezilla的客户端没有工作。看到this其他职位 图注:[慧聪]代表家庭连接(100MBps的)的 上传带宽 RDS上传: 15 kb / s的 =>更糟糕曾经的 FTP上传[安装在EC2上的FTP服务器]: 100 kb / s的 从AWS管理控制台从HC上传到S3: 60 kb / s的 使用AWS Console口从EC2上传到S3: 145 kb / s的 使用 S3浏览器上传到S3 的来自慧聪网: 120 kb / s的 使用 S3浏览器上传到S3 的从EC2: 2000 KB / S

FTP server installed on EC2 was Filezilla Server (correct configuration is touchy) FTP Client used for this Bench test was WinSPC (Filezilla Client didn't work. see this other post Legend : [HC] stands for Home Connection (100MBps) Upload Bandwidth RDS Upload : 15 Kb/S => Worse Ever FTP Upload [FTP Server Installed on EC2]: 100 Kb/S Upload to S3 from AWS Management Console from HC: 60 Kb/S Upload to S3 using AWS Console interface from EC2 : 145 Kb/S Upload to S3 using S3 Browser from HC: 120 Kb/S Upload to S3 using S3 Browser from EC2 : 2000 Kb/S

下载带宽 RDS下载和上传: 15 kb / s的 =>更糟糕曾经的 FTP下载[安装在EC2上的FTP服务器]: 360 kb / s的 下载从EC2 S3 A​​WS Console界面: 350 kb / s的 下载从S3使用 S3的浏览器:由慧聪网: 380〜620 kb / s的 下载从S3使用 S3的浏览器:从EC2: 3000 KB / S

Download Bandwidth RDS Download and Upload: 15 Kb/S => Worse Ever FTP Download [FTP Server Installed on EC2]: 360 Kb/S Download from S3 AWS Console interface from EC2 : 350 Kb/S Download from S3 using S3 Browser: from HC: 380 to 620 Kb/S Download from S3 using S3 Browser: from EC2 : 3000 Kb/S

结论:

所以,截至目前, Amazon S3的结合S3浏览器提供最好的结果。 (S3浏览器只是一个层在S3我不明白为什么上传速度比较好)的 然而,人们应该记住,在一个EC2实例的FTP服务器具有映射的巨大优势,直接本地EC2目录到EC2。 与S3,只有一个出让涉及。事实上,S3要求2接送服务:形式的地方资源,以S3 /从S3到EC2和圆形的其他方式,而FTP访问通过缩短传输周期给予即时性。此外,它免去S3铲斗的成本。版 这也是有趣的一提的是EC2实例的带宽是真正的强者。所以这是更有趣-needless说,使用的协议,真正利用它的优势,例如S3或FTP,而不是RDS。

So, as of now, Amazon S3 combined with S3 Browser give the best results. (S3 Browser is just a layer over S3 I don't get it why the upload rate is better) However, one should keep in mind that an FTP Server on an EC2 instance has the great advantage of Mapping directly a local EC2 Directory into EC2. Unlike S3, there is only one transfer involved. Indeed, S3 requires 2 Transfers : Form Local Resource to S3/and from S3 to EC2 and the other way round, while FTP Access grants immediacy by shortening transfer cycles. Besides, it spares the cost of S3 Buckets. It is also interesting to mention that EC2 Instance's bandwidth is really strong. It is hence far more interesting -needless to say, to use protocols that really take advantage of it such as S3 or FTP, rather than RDS.

我希望这将是有益的面临着同样的问题,他们免受precious时间的其他人。

I hope this will be useful to other people facing the same issue and spare them precious time.