Fix use of precedence in endpoint routing DFA#20801
Merged
Merged
Conversation
rynowak
requested review from
Tratcher,
analogrelay and
jkotalik
as code owners
April 14, 2020 00:07
JamesNK
reviewed
Apr 14, 2020
pranavkm
reviewed
Apr 14, 2020
| work.Add((endpoint, precedenceDigit, new List<DfaNode>() { root, })); | ||
|
|
||
| work.Add((endpoint, new List<DfaNode>() { root, })); | ||
| maxDepth = Math.Max(maxDepth, endpoint.RoutePattern.PathSegments.Count); |
Contributor
There was a problem hiding this comment.
Is there any significance to changing where this is computed?
Member
Author
There was a problem hiding this comment.
No, it just seemed to make sense to me to place after the other stuff.
pranavkm
approved these changes
Apr 14, 2020
once Fixes: #18677 Fixes: #16579 This is a change to how sorting is use when building endpoint routing's graph of nodes that is eventually transformed into the route table. There were bugs in how this was done that made it incompatible in some niche scenarios both with previous implementations and how we describe the features in the abstract. There are a wide array of cases that might have been impacted by this bug because routing is a pattern language. Generally the bugs will involve a catch-all, and some something that changes ordering of templates. Issue #18677 has the simplest repro for this, the following templates would not behave as expected: ``` a/{*b} {a}/{b} ``` One would expect any URL Path starting with `/a` to match the first route, but that's not what happens. --- The root cause of this bug was an issue in how the algorithm used to be build the DFA was designed. Specifically that it uses a BFS to build the graph, and it uses an up-front one-time sort of endpoints in order to drive that BFS. The building of the graph has the expectation that at each level, we will process **all** literal segments (`/a`) and then **all** parameter segments (`/{a}`) and then **all** catch-all segments (`/{*a}`). Routing defines a concept called *precedence* that defines the *conceptual* order in while segments types are ordered. So there are two problems: - We sort based on criteria other than precedence (#16579) - We can't rely on a one-time sort, it needs to be done at each level (#18677) --- The fix is to repeat the sort operation at each level and use precedence as the only key for sorting (as dictated by the graph building algo). We do a sort of the matches of each node *after* building the precedence-based part of the DFA, based on the full sorting criteria, to maintain compatibility.
rynowak
force-pushed
the
rynowak/routing-fixes
branch
from
April 22, 2020 01:35
730e1ad to
c5fe7c0
Compare
Member
Author
|
Updated with a brand new test 🥂 |
JamesNK
approved these changes
Apr 22, 2020
rynowak
added a commit
that referenced
this pull request
Apr 23, 2020
* Fix use of precedence in endpoint routing DFA Fixes: #18677 Fixes: #16579 This is a change to how sorting is use when building endpoint routing's graph of nodes that is eventually transformed into the route table. There were bugs in how this was done that made it incompatible in some niche scenarios both with previous implementations and how we describe the features in the abstract. There are a wide array of cases that might have been impacted by this bug because routing is a pattern language. Generally the bugs will involve a catch-all, and some something that changes ordering of templates. Issue #18677 has the simplest repro for this, the following templates would not behave as expected: ``` a/{*b} {a}/{b} ``` One would expect any URL Path starting with `/a` to match the first route, but that's not what happens. --- The root cause of this bug was an issue in how the algorithm used to be build the DFA was designed. Specifically that it uses a BFS to build the graph, and it uses an up-front one-time sort of endpoints in order to drive that BFS. The building of the graph has the expectation that at each level, we will process **all** literal segments (`/a`) and then **all** parameter segments (`/{a}`) and then **all** catch-all segments (`/{*a}`). Routing defines a concept called *precedence* that defines the *conceptual* order in while segments types are ordered. So there are two problems: - We sort based on criteria other than precedence (#16579) - We can't rely on a one-time sort, it needs to be done at each level (#18677) --- The fix is to repeat the sort operation at each level and use precedence as the only key for sorting (as dictated by the graph building algo). We do a sort of the matches of each node *after* building the precedence-based part of the DFA, based on the full sorting criteria, to maintain compatibility. * Add test
rynowak
added a commit
that referenced
this pull request
Apr 24, 2020
* Fix use of precedence in endpoint routing DFA Fixes: #18677 Fixes: #16579 This is a change to how sorting is use when building endpoint routing's graph of nodes that is eventually transformed into the route table. There were bugs in how this was done that made it incompatible in some niche scenarios both with previous implementations and how we describe the features in the abstract. There are a wide array of cases that might have been impacted by this bug because routing is a pattern language. Generally the bugs will involve a catch-all, and some something that changes ordering of templates. Issue #18677 has the simplest repro for this, the following templates would not behave as expected: ``` a/{*b} {a}/{b} ``` One would expect any URL Path starting with `/a` to match the first route, but that's not what happens. --- The change supports an opt-out via the following AppContext switch: ``` Microsoft.AspNetCore.Routing.Use30CatchAllBehavior ``` Set to true to enable the buggy behavior for compatibility. --- The root cause of this bug was an issue in how the algorithm used to be build the DFA was designed. Specifically that it uses a BFS to build the graph, and it uses an up-front one-time sort of endpoints in order to drive that BFS. The building of the graph has the expectation that at each level, we will process **all** literal segments (`/a`) and then **all** parameter segments (`/{a}`) and then **all** catch-all segments (`/{*a}`). Routing defines a concept called *precedence* that defines the *conceptual* order in while segments types are ordered. So there are two problems: - We sort based on criteria other than precedence (#16579) - We can't rely on a one-time sort, it needs to be done at each level (#18677) --- The fix is to repeat the sort operation at each level and use precedence as the only key for sorting (as dictated by the graph building algo). We do a sort of the matches of each node *after* building the precedence-based part of the DFA, based on the full sorting criteria, to maintain compatibility. * Add test
rynowak
added a commit
that referenced
this pull request
Apr 29, 2020
* Fix use of precedence in endpoint routing DFA Fixes: #18677 Fixes: #16579 This is a change to how sorting is use when building endpoint routing's graph of nodes that is eventually transformed into the route table. There were bugs in how this was done that made it incompatible in some niche scenarios both with previous implementations and how we describe the features in the abstract. There are a wide array of cases that might have been impacted by this bug because routing is a pattern language. Generally the bugs will involve a catch-all, and some something that changes ordering of templates. Issue #18677 has the simplest repro for this, the following templates would not behave as expected: ``` a/{*b} {a}/{b} ``` One would expect any URL Path starting with `/a` to match the first route, but that's not what happens. --- The change supports an opt-in via the following AppContext switch: ``` Microsoft.AspNetCore.Routing.UseCorrectCatchAllBehavior ``` Set to true to enable the correct behavior. --- The root cause of this bug was an issue in how the algorithm used to be build the DFA was designed. Specifically that it uses a BFS to build the graph, and it uses an up-front one-time sort of endpoints in order to drive that BFS. The building of the graph has the expectation that at each level, we will process **all** literal segments (`/a`) and then **all** parameter segments (`/{a}`) and then **all** catch-all segments (`/{*a}`). Routing defines a concept called *precedence* that defines the *conceptual* order in while segments types are ordered. So there are two problems: - We sort based on criteria other than precedence (#16579) - We can't rely on a one-time sort, it needs to be done at each level (#18677) --- The fix is to repeat the sort operation at each level and use precedence as the only key for sorting (as dictated by the graph building algo). We do a sort of the matches of each node *after* building the precedence-based part of the DFA, based on the full sorting criteria, to maintain compatibility. * Add test
rynowak
added a commit
that referenced
this pull request
Apr 29, 2020
* Fix use of precedence in endpoint routing DFA Fixes: #18677 Fixes: #16579 This is a change to how sorting is use when building endpoint routing's graph of nodes that is eventually transformed into the route table. There were bugs in how this was done that made it incompatible in some niche scenarios both with previous implementations and how we describe the features in the abstract. There are a wide array of cases that might have been impacted by this bug because routing is a pattern language. Generally the bugs will involve a catch-all, and some something that changes ordering of templates. Issue #18677 has the simplest repro for this, the following templates would not behave as expected: ``` a/{*b} {a}/{b} ``` One would expect any URL Path starting with `/a` to match the first route, but that's not what happens. --- The change supports an opt-in via the following AppContext switch: ``` Microsoft.AspNetCore.Routing.UseCorrectCatchAllBehavior ``` Set to true to enable the correct behavior. --- The root cause of this bug was an issue in how the algorithm used to be build the DFA was designed. Specifically that it uses a BFS to build the graph, and it uses an up-front one-time sort of endpoints in order to drive that BFS. The building of the graph has the expectation that at each level, we will process **all** literal segments (`/a`) and then **all** parameter segments (`/{a}`) and then **all** catch-all segments (`/{*a}`). Routing defines a concept called *precedence* that defines the *conceptual* order in while segments types are ordered. So there are two problems: - We sort based on criteria other than precedence (#16579) - We can't rely on a one-time sort, it needs to be done at each level (#18677) --- The fix is to repeat the sort operation at each level and use precedence as the only key for sorting (as dictated by the graph building algo). We do a sort of the matches of each node *after* building the precedence-based part of the DFA, based on the full sorting criteria, to maintain compatibility. * Add test
rynowak
added a commit
that referenced
this pull request
Apr 29, 2020
* Fix use of precedence in endpoint routing DFA Fixes: #18677 Fixes: #16579 This is a change to how sorting is use when building endpoint routing's graph of nodes that is eventually transformed into the route table. There were bugs in how this was done that made it incompatible in some niche scenarios both with previous implementations and how we describe the features in the abstract. There are a wide array of cases that might have been impacted by this bug because routing is a pattern language. Generally the bugs will involve a catch-all, and some something that changes ordering of templates. Issue #18677 has the simplest repro for this, the following templates would not behave as expected: ``` a/{*b} {a}/{b} ``` One would expect any URL Path starting with `/a` to match the first route, but that's not what happens. --- The change supports an opt-in via the following AppContext switch: ``` Microsoft.AspNetCore.Routing.UseCorrectCatchAllBehavior ``` Set to true to enable the correct behavior. --- The root cause of this bug was an issue in how the algorithm used to be build the DFA was designed. Specifically that it uses a BFS to build the graph, and it uses an up-front one-time sort of endpoints in order to drive that BFS. The building of the graph has the expectation that at each level, we will process **all** literal segments (`/a`) and then **all** parameter segments (`/{a}`) and then **all** catch-all segments (`/{*a}`). Routing defines a concept called *precedence* that defines the *conceptual* order in while segments types are ordered. So there are two problems: - We sort based on criteria other than precedence (#16579) - We can't rely on a one-time sort, it needs to be done at each level (#18677) --- The fix is to repeat the sort operation at each level and use precedence as the only key for sorting (as dictated by the graph building algo). We do a sort of the matches of each node *after* building the precedence-based part of the DFA, based on the full sorting criteria, to maintain compatibility. * Add test (cherry picked from commit e317089)
wtgodbe
pushed a commit
that referenced
this pull request
May 13, 2020
* Fix use of precedence in endpoint routing DFA Fixes: #18677 Fixes: #16579 This is a change to how sorting is use when building endpoint routing's graph of nodes that is eventually transformed into the route table. There were bugs in how this was done that made it incompatible in some niche scenarios both with previous implementations and how we describe the features in the abstract. There are a wide array of cases that might have been impacted by this bug because routing is a pattern language. Generally the bugs will involve a catch-all, and some something that changes ordering of templates. Issue #18677 has the simplest repro for this, the following templates would not behave as expected: ``` a/{*b} {a}/{b} ``` One would expect any URL Path starting with `/a` to match the first route, but that's not what happens. --- The change supports an opt-in via the following AppContext switch: ``` Microsoft.AspNetCore.Routing.UseCorrectCatchAllBehavior ``` Set to true to enable the correct behavior. --- The root cause of this bug was an issue in how the algorithm used to be build the DFA was designed. Specifically that it uses a BFS to build the graph, and it uses an up-front one-time sort of endpoints in order to drive that BFS. The building of the graph has the expectation that at each level, we will process **all** literal segments (`/a`) and then **all** parameter segments (`/{a}`) and then **all** catch-all segments (`/{*a}`). Routing defines a concept called *precedence* that defines the *conceptual* order in while segments types are ordered. So there are two problems: - We sort based on criteria other than precedence (#16579) - We can't rely on a one-time sort, it needs to be done at each level (#18677) --- The fix is to repeat the sort operation at each level and use precedence as the only key for sorting (as dictated by the graph building algo). We do a sort of the matches of each node *after* building the precedence-based part of the DFA, based on the full sorting criteria, to maintain compatibility. * Add test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: #18677
Fixes: #16579
This is a change to how sorting is use when building endpoint routing's graph of
nodes that is eventually transformed into the route table. There were
bugs in how this was done that made it incompatible in some niche
scenarios both with previous implementations and how we describe the
features in the abstract.
There are a wide array of cases that might have been impacted by this
bug because routing is a pattern language. Generally the bugs will involve a
catch-all, and some something that changes ordering of templates.
Issue #18677 has the simplest repro for this, the following templates
would not behave as expected:
One would expect any URL Path starting with
/ato match the firstroute, but that's not what happens.
The root cause of this bug was an issue in how the algorithm used to be
build the DFA was designed. Specifically that it uses a BFS to build the
graph, and it uses an up-front one-time sort of endpoints in order to
drive that BFS.
The building of the graph has the expectation that at each level, we
will process all literal segments (
/a) and then all parametersegments (
/{a}) and then all catch-all segments (/{*a}). Routingdefines a concept called precedence that defines the conceptual
order in while segments types are ordered.
So there are two problems:
(Odd results mixing fully-tokenized attribute-route and wildcard-map within endpoint-routing #18677)
The fix is to repeat the sort operation at each level and use precedence
as the only key for sorting (as dictated by the graph building algo).
We do a sort of the matches of each node after building the
precedence-based part of the DFA, based on the full sorting criteria, to
maintain compatibility.
Summary of the changes (Less than 80 chars)
Addresses #bugnumber (in this specific format)