From 03be803ab1bd8d361c9c5e11a8b3fed2c9c30562 Mon Sep 17 00:00:00 2001 From: Dave Peck Date: Thu, 16 Jan 2025 14:41:37 -0800 Subject: [PATCH] PEP 750: Updates after community feedback (#4124) Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --- peps/pep-0750.rst | 541 +++++++++++++++++++++++++++------------------- 1 file changed, 315 insertions(+), 226 deletions(-) diff --git a/peps/pep-0750.rst b/peps/pep-0750.rst index 3291da787ea..8a1634a6315 100644 --- a/peps/pep-0750.rst +++ b/peps/pep-0750.rst @@ -6,15 +6,15 @@ Author: Jim Baker , Koudai Aono , Lysandros Nikolaou , Dave Peck -Discussions-To: https://discuss.python.org/t/pep-750-tag-strings-for-writing-domain-specific-languages/60408 +Discussions-To: https://discuss.python.org/t/71594 Status: Draft Type: Standards Track Created: 08-Jul-2024 Python-Version: 3.14 Post-History: `09-Aug-2024 `__, `17-Oct-2024 `__, - `21-Oct-2024 `__ - + `21-Oct-2024 `__, + `18-Nov-2024 `__, Abstract ======== @@ -108,14 +108,14 @@ Template String Literals ------------------------ This PEP introduces a new string prefix, ``t``, to define template string literals. -These literals resolve to a new type, ``Template``, found in a new top-level -standard library module, ``templatelib``. +These literals resolve to a new type, ``Template``, found in the standard library +module ``<>``. The following code creates a ``Template`` instance: .. code-block:: python - from templatelib import Template + from TBD import Template template = t"This is a template string." assert isinstance(template, Template) @@ -124,49 +124,75 @@ the ability to nest template strings within interpolations, as well as the abili to use all valid quote marks (``'``, ``"``, ``'''``, and ``"""``). Like other string prefixes, the ``t`` prefix must immediately precede the quote. Like f-strings, both lowercase ``t`` and uppercase ``T`` prefixes are supported. Like -f-strings, t-strings may not be combined with the ``b`` or ``u`` prefixes. +f-strings, t-strings may not be combined with ``u`` or the ``b`` prefix. + Additionally, f-strings and t-strings cannot be combined, so the ``ft`` -prefix is invalid as well. t-strings *may* be combined with the ``r`` prefix; +prefix is invalid. t-strings *may* be combined with the ``r`` prefix; see the `Raw Template Strings`_ section below for more information. The ``Template`` Type --------------------- -Template strings evaluate to an instance of a new type, ``templatelib.Template``: +Template strings evaluate to an instance of a new immutable type, ``<>.Template``: .. code-block:: python class Template: - args: Sequence[str | Interpolation] - - def __init__(self, *args: str | Interpolation): + strings: tuple[str, ...] + """ + A non-empty tuple of the string parts of the template, + with N+1 items, where N is the number of interpolations + in the template. + """ + + interpolations: tuple[Interpolation, ...] + """ + A tuple of the interpolation parts of the template. + This will be an empty tuple if there are no interpolations. + """ + + def __new__(cls, *args: str | Interpolation): + """ + Create a new Template instance. + + Arguments can be provided in any order. + """ + ... + + @property + def values(self) -> tuple[object, ...]: + """ + Return a tuple of the `value` attributes of each Interpolation + in the template. + This will be an empty tuple if there are no interpolations. + """ ... -The ``args`` attribute provides access to the string parts and -any interpolations in the literal: + def __iter__(self) -> Iterator[str | Interpolation]: + """ + Iterate over the string parts and interpolations in the template. + + These may appear in any order. Empty strings will not be included. + """ + ... + +The ``strings`` and ``interpolations`` attributes provide access to the string +parts and any interpolations in the literal: .. code-block:: python name = "World" template = t"Hello {name}" - assert isinstance(template.args[0], str) - assert isinstance(template.args[1], Interpolation) - assert template.args[0] == "Hello " - assert template.args[1].value == "World" - -See `Interleaving of Template.args`_ below for more information on how the -``args`` attribute is structured. - -The ``Template`` type is immutable. ``Template.args`` cannot be reassigned -or mutated. + assert template.strings[0] == "Hello " + assert template.interpolations[0].value == "World" The ``Interpolation`` Type -------------------------- The ``Interpolation`` type represents an expression inside a template string. -Like ``Template``, it is a new concrete type found in the ``templatelib`` module: +Like ``Template``, it is a new class found in the ``<>`` module: .. code-block:: python @@ -178,8 +204,8 @@ Like ``Template``, it is a new concrete type found in the ``templatelib`` module __match_args__ = ("value", "expr", "conv", "format_spec") - def __init__( - self, + def __new__( + cls, value: object, expr: str, conv: Literal["a", "r", "s"] | None = None, @@ -187,7 +213,7 @@ Like ``Template``, it is a new concrete type found in the ``templatelib`` module ): ... -Like ``Template``, ``Interpolation`` is shallow immutable. Its attributes +The ``Interpolation`` type is shallow immutable. Its attributes cannot be reassigned. The ``value`` attribute is the evaluated result of the interpolation: @@ -196,7 +222,7 @@ The ``value`` attribute is the evaluated result of the interpolation: name = "World" template = t"Hello {name}" - assert template.args[1].value == "World" + assert template.interpolations[0].value == "World" The ``expr`` attribute is the *original text* of the interpolation: @@ -204,7 +230,7 @@ The ``expr`` attribute is the *original text* of the interpolation: name = "World" template = t"Hello {name}" - assert template.args[1].expr == "name" + assert template.interpolations[0].expr == "name" We expect that the ``expr`` attribute will not be used in most template processing code. It is provided for completeness and for use in debugging and introspection. @@ -220,7 +246,7 @@ are supported: name = "World" template = t"Hello {name!r}" - assert template.args[1].conv == "r" + assert template.interpolations[0].conv == "r" If no conversion is provided, ``conv`` is ``None``. @@ -231,7 +257,7 @@ As with f-strings, this is an arbitrary string that defines how to present the v value = 42 template = t"Value: {value:.2f}" - assert template.args[1].format_spec == ".2f" + assert template.interpolations[0].format_spec == ".2f" Format specifications in f-strings can themselves contain interpolations. This is permitted in template strings as well; ``format_spec`` is set to the eagerly @@ -242,7 +268,7 @@ evaluated result: value = 42 precision = 2 template = t"Value: {value:.{precision}f}" - assert template.args[1].format_spec == ".2f" + assert template.interpolations[0].format_spec == ".2f" If no format specification is provided, ``format_spec`` defaults to an empty string (``""``). This matches the ``format_spec`` parameter of Python's @@ -256,6 +282,30 @@ It would be surprising if, for example, a template string that uses ``{value:.2f did not round the value to two decimal places when processed. +Convenience Accessors in ``Template`` +------------------------------------- + +The ``Template.values`` property is equivalent to: + +.. code-block:: python + + @property + def values(self) -> tuple[object, ...]: + return tuple(i.value for i in self.interpolations) + + +The ``Template.__iter__()`` method is equivalent to: + +.. code-block:: python + + def __iter__(self) -> Iterator[str | Interpolation]: + for s, i in zip_longest(self.strings, self.interpolations): + if s: + yield s + if i: + yield i + + Processing Template Strings --------------------------- @@ -265,16 +315,16 @@ interpolations in uppercase: .. code-block:: python - from templatelib import Template, Interpolation + from TBD import Template, Interpolation def lower_upper(template: Template) -> str: """Render static parts lowercased and interpolations uppercased.""" parts: list[str] = [] - for arg in template.args: - if isinstance(arg, Interpolation): - parts.append(str(arg.value).upper()) + for item in template: + if isinstance(item, Interpolation): + parts.append(str(item.value).upper()) else: - parts.append(arg.lower()) + parts.append(item.lower()) return "".join(parts) name = "world" @@ -299,11 +349,15 @@ and a ``str``: .. code-block:: python name = "World" - template1 = t"Hello " - template2 = t"{name}" - assert template1 + template2 == t"Hello {name}" - assert template1 + "!" == t"Hello !" - assert "Hello " + template2 == t"Hello {name}" + template = t"{name}" + + assert isinstance(t"Hello " + template, Template) + assert (t"Hello " + template).strings == ("Hello ", "") + assert (t"Hello " + template).interpolations[0].value == "World" + + assert isinstance("Hello " + template, Template) + assert ("Hello " + template).strings == ("Hello ", "") + assert ("Hello " + template).interpolations[0].value == "World" Concatenation of templates is "viral": the concatenation of a ``Template`` and a ``str`` always results in a ``Template`` instance. @@ -314,72 +368,23 @@ will work as expected: .. code-block:: python name = "World" - template = t"Hello " "World" - assert template == t"Hello World" - template2 = t"Hello " t"World" - assert template2 == t"Hello World" - - -The ``Template`` type implements the ``__add__()`` and ``__radd__()`` methods -roughly as follows: - -.. code-block:: python - - class Template: - def __add__(self, other: object) -> Template: - if isinstance(other, str): - return Template(*self.args[:-1], self.args[-1] + other) - if not isinstance(other, Template): - return NotImplemented - return Template(*self.args[:-1], self.args[-1] + other.args[0], *other.args[1:]) + assert (t"Hello " t"World").strings == ("Hello World",) + assert ("Hello " t"World").strings == ("Hello World",) - def __radd__(self, other: object) -> Template: - if not isinstance(other, str): - return NotImplemented - return Template(other + self.args[0], *self.args[1:]) - -Special care is taken to ensure that the interleaving of ``str`` and ``Interpolation`` -instances is maintained when concatenating. (See the -`Interleaving of Template.args`_ section for more information.) +The ``Template`` type supports the ``__add__()`` and ``__radd__()`` methods +between two ``Template`` instances and between a ``Template`` instance and a +``str``. Template and Interpolation Equality ----------------------------------- -Two instances of ``Template`` are defined to be equal if their ``args`` attributes -contain the same strings and interpolations in the same order: - -.. code-block:: python - - assert t"I love {stilton}" == t"I love {stilton}" - assert t"I love {stilton}" != t"I love {roquefort}" - assert t"I " + t"love {stilton}" == t"I love {stilton}" - -The implementation of ``Template.__eq__()`` is roughly as follows: - -.. code-block:: python - - class Template: - def __eq__(self, other: object) -> bool: - if not isinstance(other, Template): - return NotImplemented - return self.args == other.args +``Template`` and ``Interpolation`` instances compare with object identity +(``is``). -Two instances of ``Interpolation`` are defined to be equal if their ``value``, -``expr``, ``conv``, and ``format_spec`` attributes are equal: - -.. code-block:: python - - class Interpolation: - def __eq__(self, other: object) -> bool: - if not isinstance(other, Interpolation): - return NotImplemented - return ( - self.value == other.value - and self.expr == other.expr - and self.conv == other.conv - and self.format_spec == other.format_spec - ) +``Template`` instances are intended to be used by template processing code, +which may return a string or any other type. Those types can provide their +own equality semantics as needed. No Support for Ordering @@ -399,14 +404,21 @@ The debug specifier, ``=``, is supported in template strings and behaves similar to how it behaves in f-strings, though due to limitations of the implementation there is a slight difference. -In particular, ``t'{expr=}'`` is treated as ``t'expr={expr}'``: +In particular, ``t'{expr=}'`` is treated as ``t'expr={expr!r}'``: .. code-block:: python name = "World" template = t"Hello {name=}" - assert template.args[0] == "Hello name=" - assert template.args[1].value == "World" + assert template.strings[0] == "Hello name=" + assert template.interpolations[0].value == "World" + assert template.interpolations[0].conv == "r" + +If a separate format string is also provided, ``t'{expr=:fmt}`` is treated instead as +``t'expr={expr!s:fmt}'``. + +Whitespace is preserved in the debug specifier, so ``t'{expr = }'`` is treated as +``t'expr = {expr!r}'``. Raw Template Strings @@ -417,9 +429,9 @@ Raw template strings are supported using the ``rt`` (or ``tr``) prefix: .. code-block:: python trade = 'shrubberies' - t = rt'Did you say "{trade}"?\n' - assert t.args[0] == r'Did you say "' - assert t.args[2] == r'"?\n' + template = rt'Did you say "{trade}"?\n' + assert template.strings[0] == r'Did you say "' + assert template.strings[1] == r'"?\n' In this example, the ``\n`` is treated as two separate characters (a backslash followed by 'n') rather than a newline character. This is @@ -452,58 +464,17 @@ Exceptions raised in t-string literals are the same as those raised in f-string literals. -Interleaving of ``Template.args`` ---------------------------------- - -In the ``Template`` type, the ``args`` attribute is a sequence that will always -alternate between string literals and ``Interpolation`` instances. Specifically: - -- Even-indexed elements (0, 2, 4, ...) are always of type ``str``, representing - the literal parts of the template. -- Odd-indexed elements (1, 3, 5, ...) are always ``Interpolation`` instances, - representing the interpolated expressions. - -For example, the following assertions hold: - -.. code-block:: python - - name = "World" - template = t"Hello {name}" - assert len(template.args) == 3 - assert template.args[0] == "Hello " - assert template.args[1].value == "World" - assert template.args[2] == "" - -These rules imply that the ``args`` attribute will always have an odd length. -As a consequence, empty strings are added to the sequence when the template -begins or ends with an interpolation, or when two interpolations are adjacent: - -.. code-block:: python - - a, b = "a", "b" - template = t"{a}{b}" - assert len(template.args) == 5 - assert template.args[0] == "" - assert template.args[1].value == "a" - assert template.args[2] == "" - assert template.args[3].value == "b" - assert template.args[4] == "" +No ``Template.__str__()`` Implementation +---------------------------------------- -Most template processing code will not care about this detail and will use -either structural pattern matching or ``isinstance()`` checks to distinguish -between the two types of elements in the sequence. +The ``Template`` type does not provide a specialized ``__str__()`` implementation. -The detail exists because it allows for performance optimizations in template -processing code. For example, a template processor could cache the static parts -of the template and only reprocess the dynamic parts when the template is -evaluated with different values. Access to the static parts can be done with -``template.args[::2]``. +This is because ``Template`` instances are intended to be used by template processing +code, which may return a string or any other type. There is no canonical way to +convert a Template to a string. -Interleaving is an invariant maintained by the ``Template`` class. Developers can -take advantage of it but they are not required to themselves maintain it. -Specifically, ``Template.__init__()`` can be called with ``str`` and -``Interpolation`` instances in *any* order; the constructor will "interleave" them -as necessary before assigning them to ``args``. +The ``Template`` and ``Interpolation`` types both provide useful ``__repr__()`` +implementations. Examples @@ -535,7 +506,7 @@ specifiers like ``:.2f``. The full code is fairly simple: .. code-block:: python - from templatelib import Template, Interpolation + from TBD import Template, Interpolation def convert(value: object, conv: Literal["a", "r", "s"] | None) -> object: if conv == "a": @@ -546,11 +517,10 @@ specifiers like ``:.2f``. The full code is fairly simple: return str(value) return value - def f(template: Template) -> str: parts = [] - for arg in template.args: - match arg: + for item in template: + match item: case str() as s: parts.append(s) case Interpolation(value, _, conv, format_spec): @@ -571,10 +541,9 @@ specifiers like ``:.2f``. The full code is fairly simple: Example: Structured Logging --------------------------- -Structured logging allows developers to log data in both a human-readable format -*and* a structured format (like JSON) using only a single logging call. This is -useful for log aggregation systems that process the structured format while -still allowing developers to easily read their logs. +Structured logging allows developers to log data in machine-readable +formats like JSON. With t-strings, developers can easily log structured data +alongside human-readable messages using just a single log statement. We present two different approaches to implementing structured logging with template strings. @@ -610,7 +579,7 @@ We can implement an improved version of ``StructuredMessage`` using template str .. code-block:: python import json - from templatelib import Interpolation, Template + from TBD import Interpolation, Template from typing import Mapping class TemplateMessage: @@ -625,9 +594,9 @@ We can implement an improved version of ``StructuredMessage`` using template str @property def values(self) -> Mapping[str, object]: return { - arg.expr: arg.value - for arg in self.template.args - if isinstance(arg, Interpolation) + item.expr: item.value + for item in self.template + if isinstance(item, Interpolation) } def __str__(self) -> str: @@ -667,7 +636,7 @@ and a ``ValuesFormatter`` for JSON output: import json from logging import Formatter, LogRecord - from templatelib import Interpolation, Template + from TBD import Interpolation, Template from typing import Any, Mapping @@ -686,9 +655,9 @@ and a ``ValuesFormatter`` for JSON output: class ValuesFormatter(Formatter): def values(self, template: Template) -> Mapping[str, Any]: return { - arg.expr: arg.value - for arg in template.args - if isinstance(arg, Interpolation) + item.expr: item.value + for item in template + if isinstance(item, Interpolation) } def format(self, record: LogRecord) -> str: @@ -793,6 +762,15 @@ developers call a processing function that they get the result they want: typically, a string, although processing code can of course return any arbitrary type. +Developers will also want to understand how template strings relate to other +string formatting methods like f-strings and :meth:`str.format`. They will need +to decide when to use each method. If a simple string is all that is needed, and +there are no security implications, f-strings are likely the best choice. For +most cases where a format string is used, it can be replaced with a function +wrapping the creation of a template string. In cases where the format string is +obtained from user input, the filesystem, or databases, it is possible to write +code to convert it into a ``Template`` instance if desired. + Because developers will learn that t-strings are nearly always used in tandem with processing functions, they don't necessarily need to understand the details of the ``Template`` type. As with descriptors and decorators, we expect many more @@ -800,13 +778,13 @@ developers will use t-strings than write t-string processing functions. Over time, a small number of more advanced developers *will* wish to author their own template processing code. Writing processing code often requires thinking -in terms of formal grammars. Developers will need to learn how to parse the -``args`` attribute of a ``Template`` instance and how to process interpolations -in a context-sensitive fashion. More sophisticated grammars will likely require -parsing to intermediate representations like an AST. Great template processing -code will handle format specifiers and conversions when appropriate. Writing -production-grade template processing code -- for instance, to support HTML -templates -- can be a large undertaking. +in terms of formal grammars. Developers will need to learn how to work with the +``strings`` and ``interpolation`` attributes of a ``Template`` instance and how +to process interpolations in a context-sensitive fashion. More sophisticated +grammars will likely require parsing to intermediate representations like an +AST. Great template processing code will handle format specifiers and conversions +when appropriate. Writing production-grade template processing code -- for +instance, to support HTML templates -- can be a large undertaking. We expect that template strings will provide framework authors with a powerful new tool in their toolbox. While the functionality of template strings overlaps @@ -815,22 +793,38 @@ the language itself. Bringing the full power and generality of Python to bear on string processing tasks opens new possibilities for framework authors. +Why another templating approach? +================================ + +The world of Python already has mature templating languages with wide adoption, +such as Jinja. Why build support for creating new templating systems? + +Projects such as Jinja are still needed in cases where the template is less part +of the software by the developers, and more part of customization by designers +or even content created by users, for example in a CMS. + +The trends in frontend development have treated templating as part of the +software and written by developers. They want modern language features and a +good tooling experience. PEP 750 envisions DSLs where the non-static parts are +Python: same scope rules, typing, expression syntax, and the like. + + Common Patterns Seen in Processing Templates ============================================ Structural Pattern Matching --------------------------- -Iterating over the ``Template.args`` with structural pattern matching is the expected +Iterating over the ``Template`` with structural pattern matching is the expected best practice for many template function implementations: .. code-block:: python - from templatelib import Template, Interpolation + from TBD import Template, Interpolation def process(template: Template) -> Any: - for arg in template.args: - match arg: + for item in template: + match item: case str() as s: ... # handle each string part case Interpolation() as interpolation: @@ -857,12 +851,12 @@ The structure of ``Template`` objects allows for effective memoization: .. code-block:: python - source = template.args[::2] # Static string parts - values = [i.value for i in template.args[1::2]] # Dynamic interpolated values + strings = template.strings # Static string parts + values = template.values # Dynamic interpolated values -This separation enables caching of processed static parts, while dynamic parts can be -inserted as needed. Authors of template processing code can use the static -``source`` as cache keys, leading to significant performance improvements when +This separation enables caching of processed static parts while dynamic parts +can be inserted as needed. Authors of template processing code can use the static +``strings`` as cache keys, leading to significant performance improvements when similar templates are used repeatedly. @@ -963,8 +957,8 @@ in a ``lambda`` in the template string literal: name = "World" template = t"Hello {(lambda: name)}" - assert callable(template.args[1].value) - assert template.args[1].value() == "World" + assert callable(template.interpolations[0].value) + assert template.interpolations[0].value() == "World" This assumes, of course, that template processing code anticipates and handles callable interpolation values. (One could imagine also supporting iterators, @@ -1008,17 +1002,17 @@ and await it before processing the template string: return "Sleepy" template = t"Hello {get_name}" - assert await aformat(template) == "Hello Sleepy" + assert await async_f(template) == "Hello Sleepy" -This assumes that the template processing code in ``aformat()`` is asynchronous +This assumes that the template processing code in ``async_f()`` is asynchronous and is able to ``await`` an interpolation's value. .. note:: Example code - See `aformat.py`__ and `test_aformat.py`__. + See `afstring.py`__ and `test_afstring.py`__. - __ https://github.com/davepeck/pep750-examples/blob/main/pep/aformat.py - __ https://github.com/davepeck/pep750-examples/blob/main/pep/test_aformat.py + __ https://github.com/davepeck/pep750-examples/blob/main/pep/afstring.py + __ https://github.com/davepeck/pep750-examples/blob/main/pep/test_afstring.py Approaches to Template Reuse @@ -1038,11 +1032,69 @@ values, they can write a function to return a ``Template`` instance: This is, of course, no different from how f-strings can be reused. +Relation to Format Strings +-------------------------- + +The venerable :meth:`str.format` method accepts format strings that can later +be used to format values: + +.. code-block:: python + + alas_fmt = "We're all out of {cheese}." + assert alas_fmt.format(cheese="Red Leicester") == "We're all out of Red Leicester." + +If one squints, one can think of format strings as a kind of function definition. +The *call* to :meth:`str.format` can be seen as a kind of function call. The +t-string equivalent is to simply define a standard Python function that returns +a ``Template`` instance: + +.. code-block:: python + + def make_template(*, cheese: str) -> Template: + return t"We're all out of {cheese}." + + template = make_template(cheese="Red Leicester") + # Using the f() function from the f-string example, above + assert f(template) == "We're all out of Red Leicester." + +The ``make_template()`` function itself can be thought of as analogous to the +format string. The call to ``make_template()`` is analogous to the call to +:meth:`str.format`. + +Of course, it is common to load format strings from external sources like a +filesystem or database. Thankfully, because ``Template`` and ``Interpolation`` +are simple Python types, it is possible to write a function that takes an +old-style format string and returns an equivalent ``Template`` instance: + +.. code-block:: python + + def from_format(fmt: str, /, *args: object, **kwargs: object) -> Template: + """Parse `fmt` and return a `Template` instance.""" + ... + + # Load this from a file, database, etc. + fmt = "We're all out of {cheese}." + template = from_format(fmt, cheese="Red Leicester") + # Using the f() function from the f-string example, above + assert f(template) == "We're all out of Red Leicester." + +This is a powerful pattern that allows developers to use template strings in +places where they might have previously used format strings. A full implementation +of ``from_format()`` is available in the examples repository, +which supports the full grammar of format strings. + +.. note:: Example code + + See `format.py`__ and `test_format.py`__. + + __ https://github.com/davepeck/pep750-examples/blob/main/pep/format.py + __ https://github.com/davepeck/pep750-examples/blob/main/pep/test_format.py + + Reference Implementation ======================== -At the time of this PEP's announcement, a fully-working implementation is -`available `_. +A CPython implementation of PEP 750 is `available `_. There is also a public repository of `examples and tests `_ built around the reference implementation. If you're interested in playing with @@ -1117,17 +1169,48 @@ This was rejected for several reasons: static analysis. Most importantly, there are viable (if imperfect) alternatives to implicit -lambda wrapping when lazy evaluation is desired. See the section on -`Approaches to Lazy Evaluation`_, above, for more information. +lambda wrapping in many cases where lazy evaluation is desired. See the section +on `Approaches to Lazy Evaluation`_, above, for more information. + +While delayed evaluation was rejected for *this* PEP, we hope that the community +continues to explore the idea. Making ``Template`` and ``Interpolation`` Into Protocols -------------------------------------------------------- An early version of this PEP proposed that the ``Template`` and ``Interpolation`` -types be runtime checkable protocols rather than concrete types. +types be runtime checkable protocols rather than classes. + +In the end, we felt that using classes was more straightforward. + + +Overridden ``__eq__`` and ``__hash__`` for ``Template`` and ``Interpolation`` +----------------------------------------------------------------------------- + +Earlier versions of this PEP proposed that the ``Template`` and ``Interpolation`` +types should have their own implementations of ``__eq__`` and ``__hash__``. + +``Templates`` were considered equal if their ``strings`` and ``interpolations`` +were equal; ``Interpolations`` were considered equal if their ``value``, ``expr``, +``conv``, and ``format_spec`` were equal. Interpolation hashing was similar to +tuple hashing: an ``Interpolation`` was hashable if and only if its ``value`` +was hashable. + +This was rejected because ``Template.__hash__`` so defined was not useful as a +cache key in template processing code; we were concerned that it would be +confusing to developers. + +By dropping these implementations of ``__eq__`` and ``__hash__``, we lose the +ability to write asserts such as: + +.. code-block:: python -In the end, we felt that using concrete types was more straightforward. + name = "World" + assert t"Hello " + t"{name}" == t"Hello {name}" + +Because ``Template`` instances are intended to be quickly processed by further +code, we felt that the utility of these asserts was limited. An Additional ``Decoded`` Type @@ -1140,13 +1223,19 @@ We rejected this in favor of the simpler approach of using plain ``str`` and allowing combination of ``r`` and ``t`` prefixes. -Other Homes for ``Template`` and ``Interpolation`` --------------------------------------------------- +The Final Home for ``Template`` and ``Interpolation`` +----------------------------------------------------- -Previous versions of this PEP proposed that the ``Template`` and ``Interpolation`` -types be placed in the ``types`` module. This was rejected in favor of creating -a new top-level standard library module, ``templatelib``. This was done to avoid -polluting the ``types`` module with seemingly unrelated types. +Previous versions of this PEP proposed placing the ``Template`` and +``Interpolation`` types in: ``types``, ``collections``, ``collections.abc``, +and even in a new top-level module, ``templatelib``. As of this writing, no core +team consensus has emerged on the final location for these types. The current +PEP leaves this open for a final decision. + +One argument in favor of a new top-level ``templatelib`` module is that it would +allow for future addition of related methods (like ``convert()``) and for +potential future template processing code to be added to submodules +(``templatelib.shell``, etc.). Enable Full Reconstruction of Original Template Literal @@ -1173,7 +1262,7 @@ source text: value = 42 precision = 2 template = t"Value: {value:.{precision}f}" - assert template.args[1].format_spec == ".2f" + assert template.interpolations[0].format_spec == ".2f" We do not anticipate that these limitations will be a significant issue in practice. Developers who need to obtain the original template string literal can always @@ -1251,27 +1340,18 @@ This was rejected in favor of keeping t-string syntax as close to f-string synta as possible. -A Lazy Conversion Specifier ---------------------------- - -We considered adding a new conversion specifier, ``!()``, that would explicitly -wrap the interpolation expression in a lambda. - -This was rejected in favor of the simpler approach of using explicit lambdas -when lazy evaluation is desired. - - -Alternate Layouts for ``Template.args`` ---------------------------------------- +Alternate Layouts for ``Template`` +----------------------------------- During the development of this PEP, we considered several alternate layouts for -the ``args`` attribute of the ``Template`` type. This included: +the contents of ``Templates``. Many focused on a single ``args`` tuple that +contained both strings and interpolations. Variants included: - Instead of ``args``, ``Template`` contains a ``strings`` attribute of type - ``Sequence[str]`` and an ``interpolations`` attribute of type - ``Sequence[Interpolation]``. There are zero or more interpolations and + ``tuple[str, ...]`` and an ``interpolations`` attribute of type + ``tuple[Interpolation, ...]``. There are zero or more interpolations and there is always one more string than there are interpolations. Utility code - could build an interleaved sequence of strings and interpolations from these + could build an interleaved tuple of strings and interpolations from these separate attributes. This was rejected as being overly complex. - ``args`` is typed as a ``Sequence[tuple[str, Interpolation | None]]``. Each @@ -1279,7 +1359,7 @@ the ``args`` attribute of the ``Template`` type. This included: string part has no corresponding interpolation. This was rejected as being overly complex. -- ``args`` remains a ``Sequence[str | Interpolation]`` but does not support +- ``args`` remains a ``tuple[str | Interpolation, ...]`` but does not support interleaving. As a result, empty strings are not added to the sequence. It is no longer possible to obtain static strings with ``args[::2]``; instead, instance checks or structural pattern matching must be used to distinguish @@ -1302,12 +1382,21 @@ tag.) While exciting, this PEP does not propose any specific mechanism. It is our hope that, over time, the community will develop conventions for this purpose. +Binary Template Strings +----------------------- + +The combination of t-strings and bytes (``tb``) is considered out of scope for +this PEP. However, unlike f-strings, there is no fundamental reason why t-strings +and bytes cannot be combined. Support could be considered in a future PEP. + + Acknowledgements ================ Thanks to Ryan Morshead for contributions during development of the ideas leading to template strings. Special mention also to Dropbox's `pyxl `_ for tackling similar ideas years ago. +Andrea Giammarchi provided thoughtful feedback on the early drafts of this PEP. Finally, thanks to Joachim Viide for his pioneering work on the `tagged library `_. Tagged was not just the precursor to template strings, but the place where the whole effort started via a GitHub issue