Reduce TypeForm recognition slowdown (Take 3)

[davidfstr]

References #21262. Replaces #21585 and obsoletes #21596.

Summary

Enabling TypeForm by default (referenced #21262) made
SemanticAnalyzer.try_parse_as_type_expression run eagerly on every expression
in certain syntactic positions. The cost is concentrated in the expensive
full-parse block (expr_to_analyzed_type + isolated_error_analysis), which
fails ~87% of the time - pure wasted work.

This branch adds early-reject filters that eliminate 74% of full parses
(2570 → 666) on mypy's self-check, recovering ~46% of the regression:
+1.57% → +0.84% CPU time.

No new regexes - per review feedback on replaced #21585. Every filter here
is plain string/isinstance work, and the two shape tests that were regexes
are now helper functions.

Why not do a type-context check?

Review of replaced #21585 suggested skipping the call to
SemanticAnalyzer.try_parse_as_type_expression entirely when the type context
cannot be a TypeForm. That optimization already exists, but in the other
type checker pass at ExpressionChecker.try_parse_as_type_expression.
The same skip cannot be used in the semantic analyzer pass's function
because the type context is not yet known.

So cheaply filtering the inputs to SA.try_parse_as_type_expression is the
only remaining (obvious) lever to reduce its runtime contribution.

Optimization Results

CPU time, single worker, paired per-round deltas, n=300:

python misc/perf_compare.py --warmup-runs 3 --num-runs 300 -j 3 \
    --metric cpu --workers1 \
    <TypeForm-disabled-commit> 5bb72b788 <tip-of-this-pr-branch>

Commit	Mean	Median	Δ vs baseline (paired median ±95% CI)
baseline, <TypeForm-disabled-commit>	2.679 s	2.676 s	-
master, 5bb72b788	2.720 s	2.720 s	+41.9 ms ±2.9 (+1.57%)
all filters, <tip-of-this-pr-branch>	2.703 s	2.699 s	+22.4 ms ±2.9 (+0.84%)

The feature branch recovers
19.5 ms of the 41.9 ms regression (~46% by paired median) -
leaving +22.4 ms (~54%). Derivation:

• +41.9 ms - +22.4 ms == 19.5 ms recovered
• 19.5 ms / +41.9 ms == 46.5% (~46%) recovered
• +22.4 ms / +41.9 ms == 53.5% (~54%) left

A separate 2-way run of master vs <tip-of-this-pr-branch> measured
−21.1 ms ±3.2, consistent with the 19.5 ms recovered that was derived above.

Notes on the measurement

The baseline (<TypeForm-disabled-commit>) is current master (5bb72b788)
with referenced #21262 (SHA: dd851f559) reverted, so all three arms share
today's code and differ only in TypeForm. Measuring against the original
pre-#21262 master commit instead of today's master would have conflated
optimizations made during the following ~80 commits, including notably c0cced35c,
which optimised SA.try_parse_as_type_expression specifically.

Thus runtime regression measured here (+41.9 ms) is smaller than the
+50.2 ms reported in replaced #21585: part of the original regression has
already been absorbed upstream.

Full parses per self-check, identical corpus:

	master	branch
full parses	2570	666	−74.1%
- succeeded (produced a type)	345	345	±0 ✅
- failed (wasted work)	2225	321	−85.6%

The successful-parse count is unchanged at every commit on the branch,
as expected: No expression that previously parsed as a type stopped doing so.

Overview of changes

• Most changes are made to the SemanticAnalyzer.try_parse_as_type_expression
  function. All other changes occur within the same file.
• 5 commits, each individually profiled:
  • 4 commits add a filter
  • 1 commit reorders existing filters
• Any added filter can be dropped (if needed) without disturbing the other filters

The filter commits

Bare-identifier strings ("Foo"):

1. Reject a Var whose declared type is a concrete Instance - a value, not a type.
2. Reject FuncDef / OverloadedFuncDef / MypyFile - functions and modules are never types.
3. Reorder the mutually-exclusive checks by measured rejection frequency.

Other strings:

4. Reject strings containing a character or boundary pattern that never appears
   in a type expression - leading/trailing ., or one of !:/<>@%$^?;&~`\,
   or a - that is not a Literal[...] unary minus. Catches "utf-8",
   ".pyi", "error:", "pkg/mod.py".
5. Dotted-name strings ("builtins.tuple", "typing.Mapping"): look up the
   leftmost component and reject when it does not resolve, or resolves to a
   placeholder or a value Var.

Filters 4 and 5 replace _NONTYPE_PATTERN_RE and _DOTTED_IDENTIFIER_RE from
replaced #21585 with the helpers has_nontype_char() and dotted_identifier_leftmost().
Each was verified to agree with the regex it replaces on all 1171 distinct
strings the full-parse profiler observes during a self-check.

Two specific hazards, and how they are handled

var_is_typing_special_form was extended to recognize typing.Self /
typing_extensions.Self, so filter 1 does not reject a stringified 'Self'
annotation (otherwise testSelfRecognizedInOtherSyntacticLocations regresses).

In filter 4, - is treated as a unary minus wherever the preceding non-space
character is [ or ,, so "Literal[-1, -2]" and "Literal[1, -2]" are still
recognized. (_NONTYPE_PATTERN_RE in replaced #21585 used (?<!\[)-, which rejected
those.) On the strings observed during a self-check the two rules reject
identical sets, so the (improved) soundness costs nothing.

Notes

• I don't think it's worth trying to recover the remaining +22.4 ms:

  • The 321 surviving failed parses are spread across four categories with
    no common cheap/obvious shape left
  • I experimented with adding some fancy OpExpr filters that actually
    gave a net slowdown of 1.9ms.

• The profiling instrumentation and the misc/perf_compare.py improvements used to produce these numbers are in a separate PR: Enhance/extend general & TypeForm-specific performance instrumentation #21832. Happy to fold them in here instead if that is easier to review.

[ilevkivskyi]

This looks better, thanks! I have couple comments/questions, plus also

The same skip cannot be used in the semantic analyzer pass's function
because the type context is not yet known.

This is not true. Some type context is known very early. And I think in reality such situations are quite common, since many types are declared, not inferred.

[ilevkivskyi]

@davidfstr Not sure what is the situation here. Are you expecting me/us to just merge the code the logic of which no one (not even you, the author) understand? Or are you going to do some more work here?

[davidfstr]

Let's consider the comment threads that are open:

Why only Instance? What about other situations where we know it cannot be a type form

This comment appears to be asking for further optimization work. In particular I don't see a correctness concern. My high level response is: "Isn't this PR providing enough of a speedup?"

FMI why do we need the not self.var_is_typing_special_form(node) here? Can an attribute of a special form be a type form?

I will spend some additional time to take the empirical measurements needed to answer this question, in the next few days.

[davidfstr]

Sorry I've been curt. Many things this week (unrelated to this thread/PR) have been frustrating me and I see it's starting to bleed into my communications elsewhere.

[ilevkivskyi]

My high level response is: "Isn't this PR providing enough of a speedup?"

Speed up is OK, I am worried about merging code logic/motivation behind which I don't understand. I know there is some widespread opinion that nobody really knows (or even cares) how anything works. And although it may be true for 99% of all software, it is not true for mypy, there are people who know exactly how it works. And I want it to stay this way :-)

So, if possible, please spend some time investigating my questions/comments, and either adjust the code, or add some clear (but concise) comments explaining the motivation.

[davidfstr]

OK. I've reloaded this code into my head and made some improvements at the 2 sites that had comments.

I am worried about merging code logic/motivation behind which I don't understand.

FWIW, I agree that this is a completely valid concern, and that you were right to push back.

[davidfstr]

Also, I can rerun the profiling numbers in the PR description, now that additional changes have been made on this branch. Let me know.

[ilevkivskyi]

Thanks! This looks much better. I have one more minor comment. Also could you please refactor most of the block under elif isinstance(maybe_type_expr, StrExpr) into a separate method like string_can_be_type_expression()?

[davidfstr]

refactor most of the block under elif isinstance(maybe_type_expr, StrExpr) into a separate method like string_can_be_type_expression()?

Will do

[davidfstr]

All outstanding feedback applied.

I think extracting the now-more-complex string logic to its own function is a notable improvement. The left-over try_parse_as_type_expression function fits in about 1-2 vertical screens, which makes its structure easy to understand. 👍

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[ilevkivskyi]

LG, thanks!

[davidfstr]

@ilevkivskyi Thank you for the several rounds of feedback 👍🏻

I’m glad to close out this optimization thread. It took longer to pin down the details than I expected. But with persistent effort, we managed it.