Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

guides/features: add document for how to gracefully shutdown server #1388

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions content/en/docs/guides/server-graceful-stop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
title: GracefulStop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would want the title to be "Graceful Stop".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest "Gracefully stopping a server"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to "Gracefully stopping a server"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graceful Shutdown?

If you look at the website preview, this becomes the second guide that word wraps. I'd rather avoid that if at all possible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to Graceful Shutdown

description: >-
Explains how to gracefully shut down a gRPC server.
---

### Overview

gRPC servers often need to shut down gracefully, ensuring that in-flight RPCs
are completed within a reasonable timeframe and new RPCs are no longer
accepted. `GracefulStop()` facilitates this process, allowing the server to
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you are using "GracefulStop()" here as a placeholder for the language specific functions, but I still think it might still lead readers to look for a function with this exact name. How about just referring it to as the "graceful stop function"? It's more wordy but can avoid a potential confusion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to "Graceful Shutdown Function"

transition smoothly without abruptly terminating active connections.

When `GracefulStop()` is called, the server immediately stops accepting new
RPCs. In-flight RPCs are allowed to continue until they complete or a specified
deadline is reached. Once all active RPCs finish or the deadline expires, the
server shuts down completely.

For detailed semantics see [this][grpc doc].

### How to use Wait-for-Ready
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to replace "Wait-for-Ready"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


The exact implementation of `GracefulStop()` varies depending on the
programming language you are using. However, the general pattern
involves:

- Initiating the graceful shutdown process by calling `GracefulStop()` on your
gRPC server object. This function blocks until all currently running RPCs
completed. This ensures that in-flight requests are allowed to finish
processing.
- Specify a timeout period to limit the time allowed for in-progress RPCs to
finish. It's crucial to call `Stop()` on server object with a timeout before
calling `GracefulStop()`. This acts as a safety net, ensuring that the server
eventually shuts down even if some in-flight RPCs don't complete within a
reasonable timeframe. This prevents indefinite blocking.

The following shows the sequence of events that occur, when a server graceful
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/server/server's/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

shutdown is invoked while active RPCs are being process and client is sending
new RPCs.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also mention here the two alternative scenarios the diagram covers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

```mermaid
sequenceDiagram
Client->>Server: New RPC Request 1
Client->>Server: New RPC Request 2
Server->>Client: Processing RPC 1
Server->>Client: Processing RPC 2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found these lines to be confusing when initially reading. I thought they indicated the RPCs had completed. Can the same meaning be conveyed if these lines are just deleted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i removed. The fact that New RPC Request 1 and New RPC Request 2 is Client->Server, it means request is received by server for processing.

Server-->>Server: GracefulStop() Invoked
Server->>Client: Continues Processing In-Flight RPCs
Client->>Server: New RPC Request 3 (Rejected)
alt RPCs complete within timeout
Server->>Client: Completes RPC 1
Server->>Client: Completes RPC 2
Server-->>Server: Shutdown Complete
Client->>Server: New RPC Request 4 (Rejected)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once a connection has been closed, RPCs aren't technically "rejected" anymore. They literally can't even be attempted.

Also should we discuss that the client will detect the server shutting down and go find other servers to send its traffic to, if at all possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the rejected part and added the client will find another server if applicable

else Timeout reached
Server->>Client: RPC 2 and other pending RPCs forcefully terminated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should clearly show the forceful shutdown function being called by the application.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Server-->>Server: Shutdown Complete
Client->>Server: New RPC Request 4 (Rejected)
end
```
The following is a state based view
```mermaid
stateDiagram-v2
[*] --> RUNNING : Server Started
RUNNING --> GRACEFUL_STOP_INITIATED : GracefulStop() Called (with Timeout)
GRACEFUL_STOP_INITIATED --> GRACEFUL_STOPPING : Reject New RPCs
dfawley marked this conversation as resolved.
Show resolved Hide resolved
GRACEFUL_STOPPING --> TERMINATED : Complete In-Flight RPCs (Before Timeout)
GRACEFUL_STOPPING --> FORCE_STOP : Timeout Reached
FORCE_STOP --> TERMINATED : Force Terminate Pending RPCs and Shutdown
dfawley marked this conversation as resolved.
Show resolved Hide resolved
```

### Alternatives

- Forceful Shutdown: Immediately terminates the server using `Stop()` on server
object, potentially interrupting active RPCs and leading to errors on the
client-side. Use only as a last resort.

### Language Support

| Language | Example |
|----------|-------------------|
| Java | |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

| Go | [Go example] |
| Python | |
dfawley marked this conversation as resolved.
Show resolved Hide resolved

[Go example]: https://github.com/grpc/grpc-go/tree/master/examples/features/gracefulstop
[grpc doc]: https://github.com/grpc/grpc/blob/master/doc/server-graceful-stop.md
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link takes me to a 404.

Copy link
Contributor Author

@purnesh42H purnesh42H Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the link will work only after merge? That's what i see in other guides.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, you can use the "deploy preview" at the bottom to view the page as it would appear on grpc.io

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed the self link