From 40067c1dcb67e4e12053774f28698156d0118777 Mon Sep 17 00:00:00 2001 From: Savorboard Date: Mon, 15 Jul 2019 10:55:09 +0800 Subject: [PATCH] Translate docs to english. --- docs/content/about/contributing.md | 3 - docs/content/user-guide/en/cap/idempotence.md | 128 ++++++++++-------- .../content/user-guide/en/cap/transactions.md | 74 ++++++++-- 3 files changed, 132 insertions(+), 73 deletions(-) delete mode 100644 docs/content/about/contributing.md diff --git a/docs/content/about/contributing.md b/docs/content/about/contributing.md deleted file mode 100644 index 3a44bc9..0000000 --- a/docs/content/about/contributing.md +++ /dev/null @@ -1,3 +0,0 @@ -# Contributing - -Pull requests, issues and commentary welcome! No special process just create a request and get in touch either via gitter or create an issue. \ No newline at end of file diff --git a/docs/content/user-guide/en/cap/idempotence.md b/docs/content/user-guide/en/cap/idempotence.md index 7ba46fd..5356ad8 100644 --- a/docs/content/user-guide/en/cap/idempotence.md +++ b/docs/content/user-guide/en/cap/idempotence.md @@ -1,114 +1,126 @@ # Idempotence -幂等性（你可以在[Wikipedia](https://en.wikipedia.org/wiki/Idempotence)读到关于幂等性的定义），当我们谈论幂等时，一般是指可以重复处理传毒的消息，而不是产生意外的结果。 +Imdempotence (which you may read a formal definition of on [Wikipedia](https://en.wikipedia.org/wiki/Idempotence), when we are talking about messaging, is when a message redelivery can be handled without ending up in an unintended state. -## 交付保证 +## Delivery guarantees[^1] -在说幂等性之前，我们先来说下关于消费端的消息交付。 +[^1]: The chapter refers to the [Delivery guarantees](https://github.com/rebus-org/Rebus/wiki/Delivery-guarantees) of rebus, which I think is described very good. -由于CAP不是使用的 MS DTC 或其他类型的2PC分布式事务机制，所以存在至少消息严格交付一次的问题，具体的说在基于消息的系统中，存在一下三种可能： +Before we talk about idempotency, let's talk about the delivery of messages on the consumer side. -* Exactly Once(*) （仅有一次） -* At Most Once （最多一次） -* At Least Once （最少一次） +Since CAP is not a used MS DTC or other type of 2PC distributed transaction mechanism, there is a problem that at least the message is strictly delivered once. Specifically, in a message-based system, there are three possibilities: -带 * 号表示在实际场景中，很难达到。 +* Exactly Once(*) +* At Most Once +* At Least Once + +Exactly once has a (*) next to it, because in the general case, it is simply not possible. ### At Most Once -最多一次交付保证，涵盖了保证一次或根本不接收所有消息的情况。 +The At Most Once delivery guarantee covers the case when you are guaranteed to receive all messages either once, or maybe not at all. + +This type of delivery guarantee can arise from your messaging system and your code performing its actions in the following order: -这种类型的传递保证可能来自你的消息系统，你的代码按以下顺序执行其操作： ``` -1. 从队列移除消息 -2. 开始一个工作事务 -3. 处理消息 ( 你的代码 ) -4. 是否成功 ? +1. Remove message from queue +2. Start work transaction +3. Handle message (your code) +4. Success? Yes: - 1. 提交工作事务 + 1. Commit work transaction No: - 1. 回滚工作事务 - 2. 将消息发回到队列。 + 1. Roll back work transaction + 2. Put message back into the queue ``` -正常情况下，他们工作的很好，工作事务将被提交。 +In the sunshine scenario, this is all well and good – your messages will be received, and work transactions will be committed, and you will be happy. -然而，有些时候并不能总是成功，比如在 1 之后出现异常，或者是你在将消息放回到队列中出现网络问题由或者宕机重启等情况。 +However, the sun does not always shine, and stuff tends to fail – especially if you do enough stuff. Consider e.g. what would happen if anything fails after having performed step (1), and then – when you try to execute step (4)/(2) (i.e. put the message back into the queue) – the network was temporarily unavailable, or the message broker restarted, or the host machine decided to reboot because it had installed an update. -使用这个协议，你将冒着丢失消息的风险，如果可以接受，那就没有关系。 +This can be OK if it's what you want, but most things in CAP revolve around the concept of DURABLE messages, i.e. messages whose contents is just as important as the data in your database. ### At Least Once -这个交付保证包含你收到至少一次的消息，当出现故障时，可能会收到多次消息。 +This delivery guarantee covers the case when you are guaranteed to receive all messages either once, or maybe more times if something has failed. -它需要稍微改变我们执行步骤的顺序，它要求消息队列系统支持事务或ACK机制，比如传统的 begin-commit-rollback 协议（MSMQ是这样），或者是 receive-ack-nack 协议（RabbitMQ，Azure Service Bus等是这样的）。 +It requires a slight change to the order we are executing our steps in, and it requires that the message queue system supports transactions, either in the form of the traditional begin-commit-rollback protocol (MSMQ does this), or in the form of a receive-ack-nack protocol (RabbitMQ, Azure Service Bus, etc. do this). -大致步骤如下: +Check this out – if we do this: ``` -1. 抢占队列中的消息。 -2. 开始一个工作事务 -3. 处理消息 ( 你的代码 ) -4. 是否成功 ? +1. Grab lease on message in queue +2. Start work transaction +3. Handle message (your code) +4. Success? Yes: - 1. 提交工作事务 - 2. 从队列删除消息 + 1. Commit work transaction + 2. Delete message from queue No: - 1. 回滚工作事务 - 2. 从队列释放抢占的消息 + 1. Roll back work transaction + 2. Release lease on message ``` -当出现失败或者抢占消息超时的时候，我们总是能够再次接收到消息以保证我们工作事务提交成功。 +and the "lease" we grabbed on the message in step (1) is associated with an appropriate timeout, then we are guaranteed that no matter how wrong things go, we will only actually remove the message from the queue (i.e. execute step (4)/(2)) if we have successfully committed our "work transaction". -### 什么是 “工作事务” ? +### What is a "work transaction"? -上面所说的“工作事务”并不是特指关系型数据库中的事务，这里的工作事务是一个概念，也就是说执行代码的原子性。 +It depends on what you're doing 😄 maybe it's a transaction in a relational database (which traditionally have pretty good support in this regard), maybe it's a transaction in a document database that happens to support transaction (like RavenDB or Postgres), or maybe it's a conceptual transaction in the form of whichever work you happen to carry out as a consequence of handling a message, e.g. update a bunch of documents in MongoDB, move some files around in the file system, or mutate some obscure in-mem data structure. -比如它可以是传统的RDMS事务，也或者是 MongoDB 事务或者是一个交易等。 +The fact that the "work transaction" is just a conceptual thing is what makes it impossible to support the aforementioned Exactly Once delivery guarantee – it's just not generally possible to commit or roll back a "work transaction" and a "queue transaction" (which is what we could call the protocol carried out with the message queue systems) atomically and consistently. -在这里它代表一个执行单元，这个执行单元是一个概念性的事实以支持前面提到的仅交付一次的这种问题。 +## Idempotence at CAP -通常，不可能做到消息的事务和工作事务来形成原子性进行提交或者回滚。 +In the CAP, the delivery guarantees we use is **At Least Once**. -## CAP 中的幂等性 +Since we have a temporary storage medium (database table), we may be able to do At Most Once, but in order to strictly guarantee that the message will not be lost, we do not provide related functions or configurations. -在CAP中，我们采用的交付保证为 At Least Once。 +### Why are we not providing(achieving) idempotency ? -由于我们具有临时存储介质（数据库表），也许可以做到 At Most Once, 但是为了严格保证消息不会丢失，我们没有提供相关功能或配置。 +1. The message was successfully written, but the execution of the Consumer method failed. -### 为什么没有实现幂等？ + There are a lot of reasons why the Consumer method fails. I don't know if the specific scene is blindly retrying or not retrying is an incorrect choice. + For example, if the consumer is debiting service, if the execution of the debit is successful, but fails to write the debit log, the CAP will judge that the consumer failed to execute and try again. If the client does not guarantee idempotency, the framework will retry it, which will inevitably lead to serious consequences for multiple debits. -1、消息写入成功了，但是此时执行Consumer方法失败了 +2. The implementation of the Consumer method succeeded, but received the same message. -执行Consumer方法失败的原因有非常多，我如果不知道具体的场景盲目进行重试或者不进行重试都是不正确的选择。 -举个例子：假如消费者为扣款服务，如果是执行扣款成功了，但是在写扣款日志的时候失败了，此时CAP会判断为消费者执行失败，进行重试。如果客户端自己没有保证幂等性，框架对其进行重试，这里势必会造成多次扣款出现严重后果。 + The scenario is also possible here. If the Consumer has been successfully executed at the beginning, but for some reason, such as the Broker recovery, and received the same message, the CAP will consider this a new after receiving the Broker message. The message will be executed again by the Consumer. Because it is a new message, the CAP cannot be idempotent at this time. -2、执行Consumer方法成功了，但是又收到了同样的消息 +3. The current data storage mode can not be idempotent. -此处场景也是可能存在的，假如开始的时候Consumer已经执行成功了，但是由于某种原因如 Broker 宕机恢复等，又收到了相同的消息，CAP 在收到Broker消息后会认为这个是一个新的消息，会对 Consumer再次执行，由于是新消息，此时 CAP 也是无法做到幂等的。 + Since the table of the CAP message is deleted after 1 hour for the successfully consumed message, if the historical message cannot be idempotent. Historically, if the broker has maintained or manually processed some messages for some reason. -3、目前的数据存储模式无法做到幂等 +4. Industry practices. -由于CAP存消息的表对于成功消费的消息会于1个小时后删除，所以如果对于一些历史性消息无法做到幂等操作。 历史性指的是，假如 Broker由于某种原因维护了或者是人工处理的一些消息。 + Many event-driven frameworks require users to ensure idempotent operations, such as ENode, RocketMQ, etc... -4、业界做法 +From an implementation point of view, CAP can do some less stringent idempotence, but strict idempotent cannot. -许多基于事件驱动的框架都是要求 用户 来保证幂等性操作的，比如 ENode, RocketMQ 等等... +### Naturally idempotent message processing -从实现的角度来说，CAP可以做一些比较不严格的幂等，但是严格的幂等无法做到的。 +Generally, the best way to deal with message redeliveries is to make the processing of each message naturally idempotent. -### 以自然的方式处理幂等消息 +Natural idempotence arises when the processing of a message consists of calling an idempotent method on a domain object, like -通常情况下，保证消息被执行多次而不会产生意外结果是很自然的一种方式是采用操作对象自带的一些幂等功能。比如： +``` +obj.MarkAsDeleted(); + +``` + +or + +``` +obj.UpdatePeriod(message.NewPeriod); +``` -数据库提供的 `INSERT ON DUPLICATE KEY UPDATE` 或者是采取类型的程序判断行为。 +You can use the `INSERT ON DUPLICATE KEY UPDATE` provided by the database to easily done. -### 显式处理幂等消息 +### Explicitly handling redeliveries -另外一种处理幂等性的方式就是在消息传递的过程中传递ID，然后由单独的消息跟踪器来处理。 +Another way of making message processing idempotent, is to simply track IDs of processed messages explicitly, and then make your code handle a redelivery. -比如你使用具有事务数据存储的 IMessageTracker 来跟踪消息ID，你的代码可能看起来像这样： +Assuming that you are keeping track of message IDs by using an `IMessageTracker` that uses the same transactional data store as the rest of your work, your code might look somewhat like this: ```c# readonly IMessageTracker _messageTracker; @@ -134,4 +146,4 @@ public async Task Handle(SomeMessage message) } ``` -至于 `IMessageTracker` 的实现，可以使用诸如Redis或者数据库等存储消息Id和对应的处理状态。 \ No newline at end of file +As for the implementation of `IMessageTracker`, you can use a storage message Id such as Redis or a database and the corresponding processing state. \ No newline at end of file diff --git a/docs/content/user-guide/en/cap/transactions.md b/docs/content/user-guide/en/cap/transactions.md index 7ab5b40..dcbbc6b 100644 --- a/docs/content/user-guide/en/cap/transactions.md +++ b/docs/content/user-guide/en/cap/transactions.md @@ -1,21 +1,71 @@ -# 事务 +# Transaction -## 分布式事务? +## Distributed transactions? -CAP 不直接提供开箱即用的基于 DTC 或者 2PC 的分布式事务，相反我们提供一种可以用于解决在分布式事务遇到的问题的一种解决方案。 +CAP does not directly provide out-of-the-box MS DTC or 2PC-based distributed transactions, instead we provide a solution that can be used to solve problems encountered in distributed transactions. -在分布式环境中，由于涉及通讯的开销，使用基于2PC或DTC的分布式事务将非常昂贵，在性能方面也同样如此。另外由于基于2PC或DTC的分布式事务同样受**CAP定理**的约束，当发生网络分区时它将不得不放弃可用性(CAP中的A)。 +In a distributed environment, using 2PC or DTC-based distributed transactions can be very expensive due to the overhead involved in communication, as is performance. In addition, since distributed transactions based on 2PC or DTC are also subject to the **CAP theorem**, it will have to give up availability (A in CAP) when network partitioning occurs. -针对于分布式事务的处理，CAP 采用的是“异步确保”这种方案。 +> A distributed transaction is a very complex process with a lot of moving parts that can fail. Also, if these parts run on different machines or even in different data centers, the process of committing a transaction could become very long and unreliable. -### 异步确保 +> This could seriously affect the user experience and overall system bandwidth. So **one of the best ways to solve the problem of distributed transactions is to avoid them completely**.[^1] + +For the processing of distributed transactions, CAP uses the "Eventual Consistency and Compensation" scheme. -异步确保这种方案又叫做本地消息表，这是一种经典的方案，方案最初来源于 eBay，参考资料见段末链接。这种方案目前也是企业中使用最多的方案之一。 +### Eventual Consistency and Compensation [^1] -相对于 TCC 或者 2PC/3PC 来说，这个方案对于分布式事务来说是最简单的，而且它是去中心化的。在TCC 或者 2PC 的方案中，必须具有事务协调器来处理每个不同服务之间的状态，而此种方案不需要事务协调器。 -另外 2PC/TCC 这种方案如果服务依赖过多，会带来管理复杂性增加和稳定性风险增大的问题。试想如果我们强依赖 10 个服务，9 个都执行成功了，最后一个执行失败了，那么是不是前面 9 个都要回滚掉？这个成本还是非常高的。 +[^1]: This chapter is quoted from: https://www.baeldung.com/transactions-across-microservices -但是，并不是说 2PC 或者 TCC 这种方案不好，因为每一种方案都有其相对优势的使用场景和优缺点，这里就不做过多介绍了。 +By far, one of the most feasible models of handling consistency across microservices is [eventual consistency](https://en.wikipedia.org/wiki/Eventual_consistency). -> 中文：[http://www.cnblogs.com/savorboard/p/base-an-acid-alternative.html](http://www.cnblogs.com/savorboard/p/base-an-acid-alternative.html) -> 英文：[http://queue.acm.org/detail.cfm?id=1394128](http://queue.acm.org/detail.cfm?id=1394128) +This model doesn’t enforce distributed ACID transactions across microservices. Instead, it proposes to use some mechanisms of ensuring that the system would be eventually consistent at some point in the future. + +#### A Case for Eventual Consistency + +For example, suppose we need to solve the following task: + +* register a user profile +* do some automated background check that the user can actually access the system + +The second task is to ensure, for example, that this user wasn’t banned from our servers for some reason. + +But it could take time, and we’d like to extract it to a separate microservice. It wouldn’t be reasonable to keep the user waiting for so long just to know that she was registered successfully. + +**One way to solve it would be with a message-driven approach including compensation**. Let’s consider the following architecture: + +* the user microservice tasked with registering a user profile +* the validation microservice tasked with doing a background check +* the messaging platform that supports persistent queues + +The messaging platform could ensure that the messages sent by the microservices are persisted. Then they would be delivered at a later time if the receiver weren’t currently available + +#### Happy Scenario + +In this architecture, a happy scenario would be: + +* the user microservice registers a user, saving information about her in its local database +* the user microservice marks this user with a flag. It could signify that this user hasn’t yet been validated and doesn’t have access to full system functionality +* a confirmation of registration is sent to the user with a warning that not all functionality of the system is accessible right away +* the user microservice sends a message to the validation microservice to do the background check of a user +* the validation microservice runs the background check and sends a message to the user microservice with the results of the check +* if the results are positive, the user microservice unblocks the user +* if the results are negative, the user microservice deletes the user account + +After we’ve gone through all these steps, the system should be in a consistent state. However, for some period of time, the user entity appeared to be in an incomplete state. + +The last step, when the user microservice removes the invalid account, is a compensation phase. + +#### Failure Scenarios + +Now let’s consider some failure scenarios: + +* if the validation microservice is not accessible, then the messaging platform with its persistent queue functionality ensures that the validation microservice would receive this message at some later time +* suppose the messaging platform fails, then the user microservice tries to send the message again at some later time, for example, by scheduled batch-processing of all users that were not yet validated +* if the validation microservice receives the message, validates the user but can’t send the answer back due to the messaging platform failure, the validation microservice also retries sending the message at some later time +* if one of the messages got lost, or some other failure happened, the user microservice finds all non-validated users by scheduled batch-processing and sends requests for validation again + +Even if some of the messages were issued multiple times, this wouldn’t affect the consistency of the data in the microservices’ databases. + +**By carefully considering all possible failure scenarios, we can ensure that our system would satisfy the conditions of eventual consistency. At the same time, we wouldn’t need to deal with the costly distributed transactions.** + +But we have to be aware that ensuring eventual consistency is a complex task. It doesn’t have a single solution for all cases. \ No newline at end of file