-
Notifications
You must be signed in to change notification settings - Fork 452
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
guides/features: add document for how to gracefully shutdown server #1388
base: main
Are you sure you want to change the base?
Changes from 1 commit
d2aa9ab
4f1bf98
37f05fc
2ed94db
07ec527
53a97a5
a144c96
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
--- | ||
title: GracefulStop | ||
description: >- | ||
Explains how to gracefully shut down a gRPC server. | ||
--- | ||
|
||
### Overview | ||
|
||
gRPC servers often need to shut down gracefully, ensuring that in-flight RPCs | ||
are completed within a reasonable timeframe and new RPCs are no longer | ||
accepted. `GracefulStop()` facilitates this process, allowing the server to | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I know you are using "GracefulStop()" here as a placeholder for the language specific functions, but I still think it might still lead readers to look for a function with this exact name. How about just referring it to as the "graceful stop function"? It's more wordy but can avoid a potential confusion. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Changed to "Graceful Shutdown Function" |
||
transition smoothly without abruptly terminating active connections. | ||
|
||
When `GracefulStop()` is called, the server immediately stops accepting new | ||
RPCs. In-flight RPCs are allowed to continue until they complete or a specified | ||
deadline is reached. Once all active RPCs finish or the deadline expires, the | ||
server shuts down completely. | ||
|
||
For detailed semantics see [this][grpc doc]. | ||
|
||
### How to use Wait-for-Ready | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Need to replace "Wait-for-Ready" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed |
||
|
||
The exact implementation of `GracefulStop()` varies depending on the | ||
programming language you are using. However, the general pattern | ||
involves: | ||
|
||
- Initiating the graceful shutdown process by calling `GracefulStop()` on your | ||
gRPC server object. This function blocks until all currently running RPCs | ||
completed. This ensures that in-flight requests are allowed to finish | ||
processing. | ||
- Specify a timeout period to limit the time allowed for in-progress RPCs to | ||
finish. It's crucial to call `Stop()` on server object with a timeout before | ||
calling `GracefulStop()`. This acts as a safety net, ensuring that the server | ||
eventually shuts down even if some in-flight RPCs don't complete within a | ||
reasonable timeframe. This prevents indefinite blocking. | ||
|
||
The following shows the sequence of events that occur, when a server graceful | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. s/server/server's/ There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||
shutdown is invoked while active RPCs are being process and client is sending | ||
new RPCs. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you also mention here the two alternative scenarios the diagram covers? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added |
||
```mermaid | ||
sequenceDiagram | ||
Client->>Server: New RPC Request 1 | ||
Client->>Server: New RPC Request 2 | ||
Server->>Client: Processing RPC 1 | ||
Server->>Client: Processing RPC 2 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I found these lines to be confusing when initially reading. I thought they indicated the RPCs had completed. Can the same meaning be conveyed if these lines are just deleted? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yeah i removed. The fact that New RPC Request 1 and New RPC Request 2 is Client->Server, it means request is received by server for processing. |
||
Server-->>Server: GracefulStop() Invoked | ||
Server->>Client: Continues Processing In-Flight RPCs | ||
Client->>Server: New RPC Request 3 (Rejected) | ||
alt RPCs complete within timeout | ||
Server->>Client: Completes RPC 1 | ||
Server->>Client: Completes RPC 2 | ||
Server-->>Server: Shutdown Complete | ||
Client->>Server: New RPC Request 4 (Rejected) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Once a connection has been closed, RPCs aren't technically "rejected" anymore. They literally can't even be attempted. Also should we discuss that the client will detect the server shutting down and go find other servers to send its traffic to, if at all possible? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed the rejected part and added the client will find another server if applicable |
||
else Timeout reached | ||
Server->>Client: RPC 2 and other pending RPCs forcefully terminated | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should clearly show the forceful shutdown function being called by the application. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated |
||
Server-->>Server: Shutdown Complete | ||
Client->>Server: New RPC Request 4 (Rejected) | ||
end | ||
``` | ||
The following is a state based view | ||
```mermaid | ||
stateDiagram-v2 | ||
[*] --> RUNNING : Server Started | ||
RUNNING --> GRACEFUL_STOP_INITIATED : GracefulStop() Called (with Timeout) | ||
GRACEFUL_STOP_INITIATED --> GRACEFUL_STOPPING : Reject New RPCs | ||
dfawley marked this conversation as resolved.
Show resolved
Hide resolved
|
||
GRACEFUL_STOPPING --> TERMINATED : Complete In-Flight RPCs (Before Timeout) | ||
GRACEFUL_STOPPING --> FORCE_STOP : Timeout Reached | ||
FORCE_STOP --> TERMINATED : Force Terminate Pending RPCs and Shutdown | ||
dfawley marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
|
||
### Alternatives | ||
|
||
- Forceful Shutdown: Immediately terminates the server using `Stop()` on server | ||
object, potentially interrupting active RPCs and leading to errors on the | ||
client-side. Use only as a last resort. | ||
|
||
### Language Support | ||
|
||
| Language | Example | | ||
|----------|-------------------| | ||
| Java | | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can point to an existing example for Java: https://github.com/grpc/grpc-java/tree/master/examples/example-hostname/src/main/java/io/grpc/examples/hostname There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added |
||
| Go | [Go example] | | ||
| Python | | | ||
dfawley marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
[Go example]: https://github.com/grpc/grpc-go/tree/master/examples/features/gracefulstop | ||
[grpc doc]: https://github.com/grpc/grpc/blob/master/doc/server-graceful-stop.md | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This link takes me to a 404. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. i think the link will work only after merge? That's what i see in other guides. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. FWIW, you can use the "deploy preview" at the bottom to view the page as it would appear on grpc.io There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have removed the self link |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We would want the title to be "Graceful Stop".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest "Gracefully stopping a server"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change to "Gracefully stopping a server"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Graceful Shutdown
?If you look at the website preview, this becomes the second guide that word wraps. I'd rather avoid that if at all possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to Graceful Shutdown