[clang-format] Add `VariableTemplates` option #121318

owenca · 2024-12-30T04:19:13Z

Closes #120148.

llvmbot · 2024-12-30T04:19:44Z

@llvm/pr-subscribers-clang-format

@llvm/pr-subscribers-clang

Author: Owen Pan (owenca)

Changes

Closes #120148.

Full diff: https://github.com/llvm/llvm-project/pull/121318.diff

9 Files Affected:

(modified) clang/docs/ClangFormatStyleOptions.rst (+11-2)
(modified) clang/docs/ReleaseNotes.rst (+1)
(modified) clang/include/clang/Format/Format.h (+12-3)
(modified) clang/lib/Format/Format.cpp (+1)
(modified) clang/lib/Format/FormatToken.h (+1)
(modified) clang/lib/Format/FormatTokenLexer.cpp (+4)
(modified) clang/lib/Format/FormatTokenLexer.h (+2-1)
(modified) clang/lib/Format/TokenAnnotator.cpp (+15-4)
(modified) clang/unittests/Format/TokenAnnotatorTest.cpp (+13)

diff --git a/clang/docs/ClangFormatStyleOptions.rst b/clang/docs/ClangFormatStyleOptions.rst
index 4be448171699ca..d7456fb701563d 100644
--- a/clang/docs/ClangFormatStyleOptions.rst
+++ b/clang/docs/ClangFormatStyleOptions.rst
@@ -6725,8 +6725,8 @@ the configuration (without a prefix: ``Auto``).
 .. _TemplateNames:
 
 **TemplateNames** (``List of Strings``) :versionbadge:`clang-format 20` :ref:`¶ <TemplateNames>`
-  A vector of non-keyword identifiers that should be interpreted as
-  template names.
+  A vector of non-keyword identifiers that should be interpreted as template
+  names.
 
   A ``<`` after a template name is annotated as a template opener instead of
   a binary operator.
@@ -6793,6 +6793,15 @@ the configuration (without a prefix: ``Auto``).
 
 
 
+.. _VariableTemplates:
+
+**VariableTemplates** (``List of Strings``) :versionbadge:`clang-format 20` :ref:`¶ <VariableTemplates>`
+  A vector of non-keyword identifiers that should be interpreted as variable
+  template names.
+
+  A ``)`` after a non-variable template instantiation may be annotated as
+  the closing parenthesis of the C-style cast operator.
+
 .. _VerilogBreakBetweenInstancePorts:
 
 **VerilogBreakBetweenInstancePorts** (``Boolean``) :versionbadge:`clang-format 17` :ref:`¶ <VerilogBreakBetweenInstancePorts>`
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index d9b0cb815a15db..f05a95e31dc255 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -1118,6 +1118,7 @@ clang-format
   ``Never``, and ``true`` to ``Always``.
 - Adds ``RemoveEmptyLinesInUnwrappedLines`` option.
 - Adds ``KeepFormFeed`` option and set it to ``true`` for ``GNU`` style.
+- Adds ``VariableTemplates`` option.
 
 libclang
 --------
diff --git a/clang/include/clang/Format/Format.h b/clang/include/clang/Format/Format.h
index 6383934afa2c40..b245b753624980 100644
--- a/clang/include/clang/Format/Format.h
+++ b/clang/include/clang/Format/Format.h
@@ -5038,8 +5038,8 @@ struct FormatStyle {
   /// \version 3.7
   unsigned TabWidth;
 
-  /// A vector of non-keyword identifiers that should be interpreted as
-  /// template names.
+  /// A vector of non-keyword identifiers that should be interpreted as template
+  /// names.
   ///
   /// A ``<`` after a template name is annotated as a template opener instead of
   /// a binary operator.
@@ -5099,6 +5099,15 @@ struct FormatStyle {
   /// \version 3.7
   UseTabStyle UseTab;
 
+  /// A vector of non-keyword identifiers that should be interpreted as variable
+  /// template names.
+  ///
+  /// A ``)`` after a non-variable template instantiation may be annotated as
+  /// the closing parenthesis of the C-style cast operator.
+  ///
+  /// \version 20
+  std::vector<std::string> VariableTemplates;
+
   /// For Verilog, put each port on its own line in module instantiations.
   /// \code
   ///    true:
@@ -5308,7 +5317,7 @@ struct FormatStyle {
            TableGenBreakInsideDAGArg == R.TableGenBreakInsideDAGArg &&
            TabWidth == R.TabWidth && TemplateNames == R.TemplateNames &&
            TypeNames == R.TypeNames && TypenameMacros == R.TypenameMacros &&
-           UseTab == R.UseTab &&
+           UseTab == R.UseTab && VariableTemplates == R.VariableTemplates &&
            VerilogBreakBetweenInstancePorts ==
                R.VerilogBreakBetweenInstancePorts &&
            WhitespaceSensitiveMacros == R.WhitespaceSensitiveMacros;
diff --git a/clang/lib/Format/Format.cpp b/clang/lib/Format/Format.cpp
index 95129a8fe9240c..fe5465ce7b09de 100644
--- a/clang/lib/Format/Format.cpp
+++ b/clang/lib/Format/Format.cpp
@@ -1164,6 +1164,7 @@ template <> struct MappingTraits<FormatStyle> {
     IO.mapOptional("TypeNames", Style.TypeNames);
     IO.mapOptional("TypenameMacros", Style.TypenameMacros);
     IO.mapOptional("UseTab", Style.UseTab);
+    IO.mapOptional("VariableTemplates", Style.VariableTemplates);
     IO.mapOptional("VerilogBreakBetweenInstancePorts",
                    Style.VerilogBreakBetweenInstancePorts);
     IO.mapOptional("WhitespaceSensitiveMacros",
diff --git a/clang/lib/Format/FormatToken.h b/clang/lib/Format/FormatToken.h
index f6bb860a1fea31..8917049cefb865 100644
--- a/clang/lib/Format/FormatToken.h
+++ b/clang/lib/Format/FormatToken.h
@@ -186,6 +186,7 @@ namespace format {
   TYPE(UnionLBrace)                                                            \
   TYPE(UnionRBrace)                                                            \
   TYPE(UntouchableMacroFunc)                                                   \
+  TYPE(VariableTemplate)                                                       \
   /* Like in 'assign x = 0, y = 1;' . */                                       \
   TYPE(VerilogAssignComma)                                                     \
   /* like in begin : block */                                                  \
diff --git a/clang/lib/Format/FormatTokenLexer.cpp b/clang/lib/Format/FormatTokenLexer.cpp
index 7a264bddcdfe19..0f8d4940d4369a 100644
--- a/clang/lib/Format/FormatTokenLexer.cpp
+++ b/clang/lib/Format/FormatTokenLexer.cpp
@@ -76,6 +76,8 @@ FormatTokenLexer::FormatTokenLexer(
     TemplateNames.insert(&IdentTable.get(TemplateName));
   for (const auto &TypeName : Style.TypeNames)
     TypeNames.insert(&IdentTable.get(TypeName));
+  for (const auto &VariableTemplate : Style.VariableTemplates)
+    VariableTemplates.insert(&IdentTable.get(VariableTemplate));
 }
 
 ArrayRef<FormatToken *> FormatTokenLexer::lex() {
@@ -1382,6 +1384,8 @@ FormatToken *FormatTokenLexer::getNextToken() {
         FormatTok->setFinalizedType(TT_TemplateName);
       else if (TypeNames.contains(Identifier))
         FormatTok->setFinalizedType(TT_TypeName);
+      else if (VariableTemplates.contains(Identifier))
+        FormatTok->setFinalizedType(TT_VariableTemplate);
     }
   }
 
diff --git a/clang/lib/Format/FormatTokenLexer.h b/clang/lib/Format/FormatTokenLexer.h
index 71389d2ade2b73..61474a3f9ada8c 100644
--- a/clang/lib/Format/FormatTokenLexer.h
+++ b/clang/lib/Format/FormatTokenLexer.h
@@ -129,7 +129,8 @@ class FormatTokenLexer {
 
   llvm::SmallMapVector<IdentifierInfo *, TokenType, 8> Macros;
 
-  llvm::SmallPtrSet<IdentifierInfo *, 8> TemplateNames, TypeNames;
+  llvm::SmallPtrSet<IdentifierInfo *, 8> TemplateNames, TypeNames,
+      VariableTemplates;
 
   bool FormattingDisabled;
 
diff --git a/clang/lib/Format/TokenAnnotator.cpp b/clang/lib/Format/TokenAnnotator.cpp
index f2cfa7f49f62f9..b0f570966a63f3 100644
--- a/clang/lib/Format/TokenAnnotator.cpp
+++ b/clang/lib/Format/TokenAnnotator.cpp
@@ -2792,6 +2792,16 @@ class AnnotatingParser {
       return true;
     }
 
+    auto IsNonVariableTemplate = [](const FormatToken &Tok) {
+      if (Tok.isNot(TT_TemplateCloser))
+        return false;
+      const auto *Less = Tok.MatchingParen;
+      if (!Less)
+        return false;
+      const auto *BeforeLess = Less->getPreviousNonComment();
+      return BeforeLess && BeforeLess->isNot(TT_VariableTemplate);
+    };
+
     // Heuristically try to determine whether the parentheses contain a type.
     auto IsQualifiedPointerOrReference = [](const FormatToken *T,
                                             const LangOptions &LangOpts) {
@@ -2825,10 +2835,11 @@ class AnnotatingParser {
       }
       return T && T->is(TT_PointerOrReference);
     };
-    bool ParensAreType =
-        BeforeRParen->isOneOf(TT_TemplateCloser, TT_TypeDeclarationParen) ||
-        BeforeRParen->isTypeName(LangOpts) ||
-        IsQualifiedPointerOrReference(BeforeRParen, LangOpts);
+
+    bool ParensAreType = IsNonVariableTemplate(*BeforeRParen) ||
+                         BeforeRParen->is(TT_TypeDeclarationParen) ||
+                         BeforeRParen->isTypeName(LangOpts) ||
+                         IsQualifiedPointerOrReference(BeforeRParen, LangOpts);
     bool ParensCouldEndDecl =
         AfterRParen->isOneOf(tok::equal, tok::semi, tok::l_brace, tok::greater);
     if (ParensAreType && !ParensCouldEndDecl)
diff --git a/clang/unittests/Format/TokenAnnotatorTest.cpp b/clang/unittests/Format/TokenAnnotatorTest.cpp
index b2fb5227993c3f..d61b9adf4f58c6 100644
--- a/clang/unittests/Format/TokenAnnotatorTest.cpp
+++ b/clang/unittests/Format/TokenAnnotatorTest.cpp
@@ -3615,6 +3615,19 @@ TEST_F(TokenAnnotatorTest, TemplateInstantiation) {
   EXPECT_TOKEN(Tokens[18], tok::greater, TT_TemplateCloser);
 }
 
+TEST_F(TokenAnnotatorTest, VariableTemplate) {
+  auto Style = getLLVMStyle();
+  Style.VariableTemplates.push_back("a");
+
+  auto Tokens = annotate("auto t3 = (a<int>) + b;", Style);
+  ASSERT_EQ(Tokens.size(), 13u) << Tokens;
+  EXPECT_TOKEN(Tokens[4], tok::identifier, TT_VariableTemplate);
+  EXPECT_TOKEN(Tokens[5], tok::less, TT_TemplateOpener);
+  EXPECT_TOKEN(Tokens[7], tok::greater, TT_TemplateCloser);
+  EXPECT_TOKEN(Tokens[8], tok::r_paren, TT_Unknown); // Not TT_CastRParen
+  EXPECT_TOKEN(Tokens[9], tok::plus, TT_BinaryOperator);
+}
+
 TEST_F(TokenAnnotatorTest, SwitchInMacroArgument) {
   auto Tokens = annotate("FOOBAR(switch);\n"
                          "void f() {}");

Closes llvm#120148.

mydeveloperday

Should there be a parse test?

owenca · 2025-01-02T02:22:30Z

Should there be a parse test?

About half of the List of Strings options don't have a CHECK_PARSE test. I'll fix that in a separate patch.

owenca · 2025-01-02T04:20:50Z

Should there be a parse test?

About half of the List of Strings options don't have a CHECK_PARSE test. I'll fix that in a separate patch.

See #121451.

Closes llvm#120148.

llvmbot added clang Clang issues not falling into any other category clang-format labels Dec 30, 2024

owenca changed the title ~~[clang-format] Add VariableTemplate option~~ [clang-format] Add VariableTemplates option Dec 30, 2024

owenca changed the title ~~[clang-format] Add VariableTemplates option~~ [clang-format] Add VariableTemplates option Dec 30, 2024

owenca removed the clang Clang issues not falling into any other category label Dec 30, 2024

llvmbot added the clang Clang issues not falling into any other category label Dec 30, 2024

owenca removed the clang Clang issues not falling into any other category label Dec 30, 2024

owenca force-pushed the VariableTemplates branch from 81975a0 to 8f61998 Compare December 31, 2024 01:46

llvmbot added the clang Clang issues not falling into any other category label Dec 31, 2024

owenca force-pushed the VariableTemplates branch from 23067d0 to abbe00e Compare December 31, 2024 01:53

owenca removed the clang Clang issues not falling into any other category label Dec 31, 2024

[clang-format] Add VariableTemplate option

7e865d6

Closes llvm#120148.

owenca force-pushed the VariableTemplates branch from 24d9f2e to 7e865d6 Compare December 31, 2024 02:22

llvmbot added the clang Clang issues not falling into any other category label Dec 31, 2024

Merge branch 'main' into VariableTemplates

b4244b9

owenca removed the clang Clang issues not falling into any other category label Dec 31, 2024

owenca requested review from mydeveloperday, HazardyKnusperkeks and rymiel January 1, 2025 04:11

mydeveloperday approved these changes Jan 1, 2025

View reviewed changes

Merge branch 'main' into VariableTemplates

49a65e8

llvmbot added the clang Clang issues not falling into any other category label Jan 1, 2025

owenca removed the clang Clang issues not falling into any other category label Jan 1, 2025

owenca merged commit 1a0d0ae into llvm:main Jan 2, 2025
10 checks passed

owenca deleted the VariableTemplates branch January 2, 2025 02:24

owenca mentioned this pull request Jan 2, 2025

[Clang] Repair the function "rParenEndsCast" to make incorrect judgments in template variable cases #120904

Open

meltq pushed a commit to meltq/llvm-project that referenced this pull request Jan 2, 2025

[clang-format] Add VariableTemplates option (llvm#121318)

1fb63c9

Closes llvm#120148.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[clang-format] Add `VariableTemplates` option #121318

[clang-format] Add `VariableTemplates` option #121318

owenca commented Dec 30, 2024

llvmbot commented Dec 30, 2024 •

edited

Loading

mydeveloperday left a comment

owenca commented Jan 2, 2025

owenca commented Jan 2, 2025

[clang-format] Add VariableTemplates option #121318

[clang-format] Add VariableTemplates option #121318

Conversation

owenca commented Dec 30, 2024

llvmbot commented Dec 30, 2024 • edited Loading

mydeveloperday left a comment

Choose a reason for hiding this comment

owenca commented Jan 2, 2025

owenca commented Jan 2, 2025

[clang-format] Add `VariableTemplates` option #121318

[clang-format] Add `VariableTemplates` option #121318

llvmbot commented Dec 30, 2024 •

edited

Loading