服务来处理队列中长时间运行的进程 - 从哪里开始?队列、进程、时间、从哪里

2023-09-04 04:55:05 作者:无情血影

什么是建立一个服务来处理排队的长期运行的进程的最佳方法是什么?例如,这就是我们正在努力做的事情。

What's the best way to build a service to handle queued long-running processes? For example, this is what we're trying to do

在用户上传的401K数据网络 在数据进入处理队列(数据库表) 401K被处理(可能需要每个客户端几分钟) 在用户通过电子邮件通知,401K被接受

我们当然可以写一个服务来处理这个问题,但如果服务失败?怎么会有人通知?如果有什么异常?此外,应将此服务处理其他非用户启动的进程(如发送​​电子邮件等等)。

We could certainly write a service to handle this, but what if the service fails? How would someone be notified? What if there is an exception? Also, should this service handle other non-user-initiated processes (like sending e-mails and such).

我们拥有一支技术精湛的.NET开发人员,但我们所有的经验,在网页/客户端应用程序或任务调度程序启动控制台应用程序,所以任何方向,你可以提供将大大AP preciated。

We have a team of skilled .NET developers, but all of our experience is in web/client apps or console apps launched by the task scheduler, so any direction you can provide would be much appreciated.

推荐答案

考虑使用MSMQ(的 System.Messaging )完成这个任务。或许,这样的流程:

Consider using MSMQ (System.Messaging) for this task. Perhaps a flow like this:

在用户上传数据到网站。 在数据被写入到一个MSMQ。 在区间的 N 的,一个Windows服务的偷窥/从队列中读取下一个等待消息。 的工作是由服务完成的,数据被写入到一个数据块,并把它发送另一个消息到另一个队列用于通知客户。 在另一个服务的偷窥/读取第二个队列。发送电​​子邮件/通知客户的要求。表明,此第二队列是需要处理SMTP停电等,并且可以从所述第一队列独立管理。 User uploads data to a website. data is written to an MSMQ. at interval n, a Windows Service peeks/reads from the queue for the next waiting message. the work is done by the service, data is written to a DB, and it sends another message to another queue for notifying the customer. another service peeks/reads from the 2nd queue. Sends email/notification to the customer as required. Suggest that this 2nd queue is needed to handle SMTP outages, etc., and can be managed independently from the 1st queue.

您周围其他潜在缺陷的关注可能是又一个服务,或者在你的网站轮询机制。阅读DB最后处理消息的日期时间。阅读项目等待的数量。如果数字不理想,请发送电子邮件至管理员/客户/等。是必要的。

Your other concern around potential failings could be yet another service, or perhaps a polling mechanism in your website. Read the DB for the datetime of the last processed message. Read the number of items 'waiting'. If the numbers aren't satisfactory, send email to admins/customers/etc. as necessary.

使用MSMQ意味着您可以使用一个拉系统,而不会有负担你的数据库和放大器;网络通过查询每一个的 N 的对数据库。消息发送都是事务性的,所以你不必担心在所有关于丢失/ unAck'd mssages。

Using MSMQ means you get to use a pull-system, and won't be having to burden your database & network by polling every n against the database. The message send are all transactional, so you won't have to worry at all about lost/unAck'd mssages.

@Jess :明白了你试图掩盖基地。通过适当的队列将有助于它没有锤击与请求你的数据库。您的应用程序/项目/客户的规模将决定这是否是一个问题。事实上,你可能会抛出到所有这些在Page_Load中单个的.aspx,但你绝对知道更好。

@Jess: Understood you're trying to cover bases. A proper queue would help in that it's not hammering your database with requests. The scale of your app/project/customers will dictate on whether this is an issue. Indeed, you could have thrown this all into a single .aspx in Page_Load, but you definitely know better.

回复:矫枉过正。我读到的问题,有关于停工的一些问题和这样一个应用程序可能会如何处理这个问题。一个Web服务,开箱即用,并不:

Re: overkill. I 'read' into the question that there were some concerns about downtimes and how such an app could handle this. A web service, out of the box, doesn't:

手柄DB停电好,因为它是依赖于数据库保存其工作的输出。这是捕手中,工人,和resulter一切在一杆。这是否真的是在DMZ属于?注意项目1:网,不是企业内部网

handle DB outages well, as it's reliant on the DB to save the output of its work. It's the 'catcher', the 'worker', and the 'resulter' all in one shot. Does this really belong in the DMZ? Note Item 1: "Web", not intranet.

无需添加更多的Web +工作节点的规模。

scale without adding more web+worker nodes.

此建议的解决方案包括:

This suggested solution involves:

MSMQ有它自己的存储实现,所以它不依赖于你的应用程序数据库进行排队和订单的交付。这是Windows的一部分,并运行一个service.If MSMQ下跌,那么Windows是下降,或安全配置不正确。

MSMQ has its own storage implementation, so it's not reliant on your app database for queueing and order-of-delivery. It's part of Windows, and runs as a service.If MSMQ was down, then Windows is down, or security is not configured correctly.

异步处理。您的Web层可以简单地抓的要求,把'队列。而已。没有起转线程,对应用程序数据库不依赖。它可以回到接收其他用户的请求的工作。用户将不会有感觉的繁忙工作日的滞后。

asynchronous processing. Your web tier can simply 'catch' the request, and 'put' to the queue. That's it. No spinning up threads, no reliance on application databases. It can get back to the job of receiving requests from other users. Users won't have to 'feel' the lag of a busy workday.

可扩展性。你可以配置你的Windows服务1+机器工作。

scalability. You can deploy your Windows Service to 1+ machines to work.

工作的分离:401K处理,收发电子邮件,请求处理+认证都在不同的模块

separation of work: 401k processing, emailing, request-handling + authentication all in distinct modules.

安全 - 这不是100%清楚这是否在内部网或互联网的解决方案。如果网络曝光,然后再考虑要如何从DMZ发送邮件到你的内部应用程序。你能证明具有开放的互联网读写访问你的应用程序数据库?考虑某种形式的门面。队列提供了这一点。

security - it's not 100% clear whether this in an intranet or internet solution. If internet-exposed, then consider how you want to send messages from DMZ to your internal application. Can you justify having read-write access to your application DB from the open internet? Consider a facade of some kind. The queue provides this.