Support Client Migration (Client Side) #4218
Conversation
|
@microsoft-github-policy-service agree |
| @@ -2523,7 +2523,6 @@ CxPlatRecvDataReturn( | |||
| DATAPATH_RX_IO_BLOCK* IoBlock = | |||
| CXPLAT_CONTAINING_RECORD(Datagram, DATAPATH_RX_PACKET, Data)->IoBlock; | |||
|
|
|||
| CXPLAT_DBG_ASSERT(Binding == NULL || Binding == IoBlock->Binding); | |||
There was a problem hiding this comment.
When a connection binds more than one bindings (i.e. after adding a new local address on the client side), we cannot assure that the Datagrams in RecvDataChain come from the same binding.
| QUIC_PARAM_CONN_STATISTICS_V2_PLAT, | ||
| &Size, | ||
| &Stats)); | ||
| TEST_EQUAL(Stats.RecvDroppedPackets, 0); |
There was a problem hiding this comment.
How does this line help? Why would the client drop a packet? Also, are there other test conditions we can add here to determine if the new path worked?
There was a problem hiding this comment.
The intention is that I would like to check whether the client succeeded in adding the source connection IDs to a newly opened bindings (if it failed, it will drop a packet whose destination is a newly added local address).
And I think that an event notification about the validation of a path is helpful for both testing and real application.
| @@ -746,8 +768,24 @@ QuicLookupAddLocalCid( | |||
| Hash); | |||
|
|
|||
| if (ExistingConnection == NULL) { | |||
| Result = | |||
| QuicLookupInsertLocalCid(Lookup, Hash, SourceCid, TRUE); | |||
| QUIC_CID_HASH_ENTRY* CID = | |||
There was a problem hiding this comment.
Just a thought, but we might need to start using pools for allocating and freeing all these entries so they don't hurt perf.
src/core/connection.c
Outdated
|
|
||
| BOOLEAN Collision = FALSE; | ||
|
|
||
| CxPlatDispatchLockAcquire(&MsQuicLib.DatapathLock); |
There was a problem hiding this comment.
Why do we have to grab a global lock here? That's not great, and definitely could cause perf issues.
There was a problem hiding this comment.
I agree with you. I've tried to remove a global lock in masa-koz#5.
In this change, we will replace all the existing source connection IDs if we find the collision in adding a source connection ID to a newly opened bindings.
There was a problem hiding this comment.
I`ve committed it in 5da50d7
src/core/connection.c
Outdated
| if (!Connection->State.Started) { | ||
| Status = QUIC_STATUS_INVALID_STATE; | ||
| break; | ||
| } |
There was a problem hiding this comment.
Why not? It seems kind of intuitive to me that a client could create a connection, queue all paths it wants to use and then start it. No?
There was a problem hiding this comment.
I agree with you. So, I've also implemented this. masa-koz#4
There was a problem hiding this comment.
Now, we can add local address before connection start. And I've added tests for adding/removing address.
2b3c6fd
src/core/connection.c
Outdated
|
|
||
| QUIC_CID_LIST_ENTRY* NewDestCid = QuicConnGetUnusedDestCid(Connection); | ||
| if (NewDestCid == NULL) { | ||
| Status = QUIC_STATUS_INVALID_STATE; |
There was a problem hiding this comment.
I wonder if we can support a model where we always queue the path (somehow) even if we don't have a CID to use for it, yet...
There was a problem hiding this comment.
I've implemented this.
https://github.com/masa-koz/msquic/pull/3/files
There was a problem hiding this comment.
Thanks. The hardest part with this change overall will likely be that we need to make sure we have lots of tests for all the different scenarios.
There was a problem hiding this comment.
Now, we defer sending a path challange if there is no unused ConnID. So, I've removed QUIC_PARAM_CONN_LOCAL_UNUSED_DEST_CID_COUNT.
And I've modified Basic/WithProbePathArgs.ProbePath for testing wheter a path challenge will be sent when the peer sends a new ConnID.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4218 +/- ##
==========================================
- Coverage 87.20% 86.33% -0.87%
==========================================
Files 56 56
Lines 17363 17696 +333
==========================================
+ Hits 15141 15278 +137
- Misses 2222 2418 +196 ☔ View full report in Codecov by Sentry. |
src/core/connection.c
Outdated
|
|
||
| QUIC_BINDING* Binding = | ||
| CXPLAT_CONTAINING_RECORD(Link, QUIC_BINDING, Link); | ||
| QUIC_CONNECTION* Connection1 = |
There was a problem hiding this comment.
Can we use a more descriptive name instead of Connection1? Like SearchConnection or CollisionConnection
There was a problem hiding this comment.
I agree. I will change this.
| _IRQL_requires_max_(PASSIVE_LEVEL) | ||
| static | ||
| QUIC_STATUS | ||
| QuicConnRemoveLocalAddress( |
There was a problem hiding this comment.
There doesn't appear to be any test coverage for the Remove address function here. Could you add a test for that?
| @@ -221,6 +243,7 @@ QuicConnGetPathForPacket( | |||
| && QuicAddrGetFamily(&Packet->Route->RemoteAddress) == QuicAddrGetFamily(&Connection->Paths[i].Route.RemoteAddress) | |||
| && QuicAddrCompareIp(&Packet->Route->RemoteAddress, &Connection->Paths[i].Route.RemoteAddress) | |||
| && QuicAddrCompare(&Packet->Route->LocalAddress, &Connection->Paths[i].Route.LocalAddress)) { | |||
| QuicLibraryReleaseBinding(Connection->Paths[i].Binding); | |||
There was a problem hiding this comment.
Where is this reference added?
There was a problem hiding this comment.
In the case of server, QuicLibraryTryAddRefBinding() was called in QuicConnGetPathForPacket(). In the case of client, QuicBindingGetLocalAddress() was called in QuicConnAddLocalAddress().
By the way, the non-null check should be added.
There was a problem hiding this comment.
First of all, your changes look great, thanks for doing this!
I'd like to see some more tests. Since you're adding new APIs, could you add some negative test cases for SetParam(QUIC_CONN_ADD_LOCAL_ADDRESS)/SetParam(QUIC_CONN_REMOVE_LOCAL_ADDRESS). For example, tests that add/remove an address that is not valid, add/remove an address before the connection is started, add the same address twice/remove the same address twice, a test that adds more addresses than the maximum path count, and one that adds an address that isn't on the system, but is valid, e.g. 100.64.10.20.
In the Path tests, I'd appreciate a test that adds an address and removes the old one and verifies no traffic drops. If you have time, also a test that removes all addresses from a connection.
Thanks again!
|
@masa-koz Are you going to have time to add the requested extra tests? If not, do you mind if we add them later? |
…quic into add_active_migration
…on in adding a source connection ID to a newly opened bindings
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| QuicAddr ClientLocalAddr; | ||
| TEST_QUIC_SUCCEEDED(Connection.GetLocalAddr(ClientLocalAddr)); | ||
| TEST_QUIC_STATUS( | ||
| QUIC_STATUS_INVALID_STATE, |
There was a problem hiding this comment.
Can/should we really block removal of current active path? In the future, removal can/should be in response to the address being removed from the system itself (i.e. Wi-Fi gets disconnected). You can't say 'no, you're not allowed because I'm using it'. So, really, I think we need to allow removal, but if that's the last one, the connection just dies.
There was a problem hiding this comment.
I agree with you. How should we handle the challenge? Separate PR?
There was a problem hiding this comment.
Ideally I think we need to have it in this PR.
|
This PR is big, so would it be possible to add more detailed description of what exactly is changed and why? |
There was a problem hiding this comment.
Copilot reviewed 17 out of 37 changed files in this pull request and generated no comments.
Files not reviewed (20)
- src/core/binding.c: Language not supported
- src/core/binding.h: Language not supported
- src/core/cid.h: Language not supported
- src/core/configuration.c: Language not supported
- src/core/connection.h: Language not supported
- src/core/crypto.c: Language not supported
- src/core/inline.c: Language not supported
- src/core/library.h: Language not supported
- src/core/lookup.c: Language not supported
- src/core/lookup.h: Language not supported
- src/core/loss_detection.c: Language not supported
- src/core/packet_builder.c: Language not supported
- src/core/packet_builder.h: Language not supported
- src/core/path.c: Language not supported
- src/core/path.h: Language not supported
- src/core/quicdef.h: Language not supported
- src/core/send.c: Language not supported
- src/core/settings.c: Language not supported
- src/core/settings.h: Language not supported
- src/inc/msquic.h: Language not supported
|
@nibanks I've updated the description. If any missing, please notify me. |
| QUIC_CID_SLIST_ENTRY, | ||
| Link); | ||
|
|
||
| CXPLAT_SLIST_ENTRY** Link1 = &Entry->HashEntries.Next; |
There was a problem hiding this comment.
| CXPLAT_SLIST_ENTRY** Link1 = &Entry->HashEntries.Next; | |
| CXPLAT_SLIST_ENTRY** HashLink = &Entry->HashEntries.Next; |
Let's use HashLink here instead.
|
|
||
| CXPLAT_SLIST_ENTRY** Link1 = &Entry->HashEntries.Next; | ||
| while (*Link1 != NULL) { | ||
| QUIC_CID_HASH_ENTRY* Entry1 = |
There was a problem hiding this comment.
| QUIC_CID_HASH_ENTRY* Entry1 = | |
| QUIC_CID_HASH_ENTRY* HashEntry = |
| typedef struct QUIC_CID_SLIST_ENTRY { | ||
|
|
||
| CXPLAT_SLIST_ENTRY Link; | ||
| QUIC_CONNECTION* Connection; |
There was a problem hiding this comment.
Do we really need the Connection pointer since all the hash entries already have it?
| if (Entry1->Binding == Binding) { | ||
| QuicBindingRemoveSourceConnectionID(Binding, Entry1); | ||
| *Link1 = (*Link1)->Next; | ||
| CxPlatListPushEntry(&EntriesToFree, &Entry1->Link); |
There was a problem hiding this comment.
Why have a separate list that you have to iterate again later to free? Just free here directly.
|
|
||
| QUIC_CID_LIST_ENTRY* NewDestCid = QuicConnGetUnusedDestCid(Connection); |
There was a problem hiding this comment.
| QUIC_CID_LIST_ENTRY* NewDestCid = QuicConnGetUnusedDestCid(Connection); | |
| QUIC_CID_LIST_ENTRY* NewDestCid = QuicConnGetUnusedDestCid(Connection); |
|
|
||
| case QUIC_PARAM_CONN_REMOVE_LOCAL_ADDRESS: { | ||
|
|
||
| if (BufferLength != sizeof(QUIC_ADDR)) { |
There was a problem hiding this comment.
Let's move QuicAddrIsValid(LocalAddress) to here, instead of in QuicConnRemoveLocalAddress.
| CxPlatHashSimple(CID->CID.Length, CID->CID.Data), | ||
| CID, | ||
| FALSE); | ||
| CXPLAT_SLIST_ENTRY* Entry1 = CID->HashEntries.Next; |
There was a problem hiding this comment.
| CXPLAT_SLIST_ENTRY* Entry1 = CID->HashEntries.Next; | |
| CXPLAT_SLIST_ENTRY* HashEntry = CID->HashEntries.Next; |
| CXPLAT_CONTAINING_RECORD( | ||
| Entry, | ||
| QUIC_CID_HASH_ENTRY, | ||
| Entry); | ||
| (void)QuicLookupInsertLocalCid( | ||
| Lookup, | ||
| CxPlatHashSimple(CID->CID.Length, CID->CID.Data), | ||
| CxPlatHashSimple(CID->CID->CID.Length, CID->CID->CID.Data), |
There was a problem hiding this comment.
CID->CID->CID is a bit ridiculous. 😄 We should figure out how to get some more descriptive names.
| Link1 != NULL; | ||
| Link1 = Link1->Next) { | ||
|
|
||
| const QUIC_CID_HASH_ENTRY* const Entry1 = |
There was a problem hiding this comment.
| const QUIC_CID_HASH_ENTRY* const Entry1 = | |
| const QUIC_CID_HASH_ENTRY* const HashEntry = |
| QuicAddr ClientLocalAddr; | ||
| TEST_QUIC_SUCCEEDED(Connection.GetLocalAddr(ClientLocalAddr)); | ||
| TEST_QUIC_STATUS( | ||
| QUIC_STATUS_INVALID_STATE, |
There was a problem hiding this comment.
Ideally I think we need to have it in this PR.
nibanks
added
Area: Protocol Updates
Description
This PR adds the support of Client Migration (Client SIde). Fix #1946.
This pull request includes significant changes to the QUIC connection ID management and introduces a new feature for manual connection ID generation for testing purposes. The most important changes include the replacement of
QUIC_CID_HASH_ENTRYwithQUIC_CID_SLIST_ENTRYto register connection IDs into more than one bindings, the addition of new APIs to add/remove local address in client, and the introduction of a conditional compilation feature for manual connection ID generation.Added
QuicConnOpenNewPathandQuicConnOpenNewPathsare to manage opening new paths for a connection. These functions will be called when Destination Connection ID is available for sendingPATH_CHALLENGE.Testing
I've added two test.
Documentation
In this PR, the following two Connection level Parameters will be added.