以避免单点故障与elasticsearch集群和Web服务器集群的最佳方式集群、单点、故障、方式

2023-09-11 23:46:40 作者:风雨都一个人扛

我们有以下的建筑在AWS上运行的Web应用程序:

We have a web application running on AWS with the following architecture:

有2个数据节点在1 elasticseach集群 Web服务器 1自动缩放负载平衡群集 1 elasticseach cluster with 2 data nodes 1 auto-scaling load-balanced cluster of web servers

由于elasticsearch做了一些巧妙的内部负载均衡,我们可以只在数据节点中的一个点所有Web服务器。但是,这会造成单点故障的 - 如果一个节点出现故障,然后我不会得到任何的查询结果。

As elasticsearch does some clever internal load balancing we could just point all the web servers at one of the data nodes. But this would create a single point of failure - if that node goes down then I'm not going to get any query results.

我的解决方案,迄今为止已经有elasticsearch每个Web服务器作为非数据节点上运行。每个Web服务器查询其本地elasticsearch节点,这反过来农场截止到数据节点中的一个请求。这似乎是在 elasticsearch网站建议的做法

My solution thus far has been to have elasticsearch running on each web server as non-data nodes. Each web server queries its local elasticsearch node, which in turn farms the request off to one of the data nodes. This seems to be the suggested approach on the elasticsearch website

这是伟大的,如果数据节点中的一个出现故障在某些方面,我们不会失去服务的搜索查询的能力。然而,这并不意味着elasticsearch使用每个Web服务器上的资源,如果我们迁移到使用弹性魔豆(这我渴望这样做),那么我们就需要一些如何获得elasticsearch安装在我们的网站实例。编辑:我已经成功与此了,但还没有弄清楚如何指定一个不同的配置为每个环境

This is great in that if one of the data nodes fails in some way we don't lose the ability to serve search queries. However, it does mean elasticsearch is using resources on each web server, and if we migrate to using elastic beanstalk (which I'm keen to do) then we'll need to some how get elasticsearch installed on our web instances. I've succeeded with this now, but have yet to figure out how to specify a different config for each environment.

有另一种方式,以避免单点故障,而无需elasticsearch每个Web服务器上运行?

Is there another way to avoid a single point of failure without having elasticsearch running on each web server?

我想到了在数据节点满足来自Web服务器的查询前面使用负载平衡器,但是这也意味着打开集群多达公众访问,而不设置VPC来限制访问。

I thought about using a load balancer in front of the data nodes to serve queries from the web servers, but that would also mean opening the cluster up to public access without setting up VPC to restrict access.

有没有简单的解决方案,我失踪?

Is there a simpler solution I'm missing?

推荐答案

我不认为这直接回答你的问题,但是如果你仍然确定与Web服务器上的节点上运行ES,你可以自定义软件使用 .ebextensions机制,这使得安装你要运行脚本和/或安装软件包时,新的弹性魔豆实例启动。如果这还不够,你可以使用自定义AMI启动弹性青苗实例。

I don't think this directly answers your question, but if you are still ok with running ES on your web server nodes, you can customize the software that is installed using the .ebextensions mechanism, which allows you to run scripts and/or install packages when new Elastic Beanstalk instances are started up. If this isn't sufficient you can start your Elastic Beanstalk instances using a custom AMI.

此外,你可能不知道,你可以在一个VPC运行弹性魔豆。

Also, you may not be aware that you can run Elastic Beanstalk in a VPC.