# Generalize replace() function

**URL:** <https://discuss.python.org/t/generalize-replace-function/28511>\
**Category:** Ideas\
**Created:** [June 26, 2023, 9:42am UTC](https://discuss.python.org/t/generalize-replace-function/28511 "2023-06-26T09:42:34Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\
**Post date:** [June 26, 2023, 9:42am UTC](https://discuss.python.org/t/generalize-replace-function/28511/1 "2023-06-26T09:42:34Z")

</div>

It is common task to create a new object based on the existing object, but with some attributes changed. `dataclasses.replace()` provides this feature for dataclasses, named tuples have the `_raplace()` method, and some concrete classes (`date`, `time`, `datetime`, `inspect.Signature`, `inspect.Parameter`, code object) have the `replace()` method.

I propose to make `dataclasses.replace()` extensible and work with all these classes and user classes which support corresponding protocol. It should simple call the ` __replace__ ()` method. All classes mentioned above should provide this method (as an alias of existing `_replace()` or `replace()` method), and user classes can implement it as well. Good candidates for adding the ` __replace__ ()` method are `SimpleNamespace` and `AttrDict` (if we keep the latter).

Advantage of using `replace(obj)` over `obj.replace()` is that the method name does not conflict with attribute name (especially important for dataclasses, named tuples and `SimpleNamespace`).

Advantage of using `replace(obj)` over `obj._replace()` is that the latter looks like using non-public API.

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [June 26, 2023, 12:18pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/2 "2023-06-26T12:18:44Z")

</div>

> [@storchaka](#):
>
> Advantage of using `replace(obj)` over `obj.replace()` is that the method name does not conflict with attribute name (especially important for dataclasses, named tuples and `SimpleNamespace`).
> 
> Advantage of using `replace(obj)` over `obj._replace()` is that the latter looks like using non-public API.

Another advantage of the free function is the general [object oriented design principle](http://www.gotw.ca/gotw/084.htm) that free functions should be preferred over methods when possible.

---

<div class="post-metadata">

**Author:** ![effigies](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/effigies/32/35835_2.png) [@effigies](https://discuss.python.org/u/effigies)\
**Post date:** [June 26, 2023, 2:40pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/3 "2023-06-26T14:40:17Z")

</div>

I really like this idea. Is the proposal to keep this in `dataclasses`? I feel like it would be nice to have somewhere more generic, but I can’t think of anywhere more appropriate.

---

<div class="post-metadata">

**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\
**Post date:** [June 26, 2023, 4:38pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/4 "2023-06-26T16:38:06Z")

</div>

Yes, it is perhaps not the best place for such function, but what would be better? At least `dataclasses.replace()` already exists.

---

<div class="post-metadata">

**Author:** ![Jelle](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jelle/32/1049_2.png) [@Jelle](https://discuss.python.org/u/Jelle)\
**Post date:** [June 26, 2023, 4:59pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/5 "2023-06-26T16:59:12Z")

</div>

Maybe the `types` module?

---

<div class="post-metadata">

**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\
**Post date:** [June 26, 2023, 6:00pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/6 "2023-06-26T18:00:49Z")

</div>

The `types` module contains:

1. A number of built-in types which are not exposed in the `builtins` module.
2. Some functions related to building new types.

`replace()` is not related to types.

---

<div class="post-metadata">

**Author:** ![effigies](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/effigies/32/35835_2.png) [@effigies](https://discuss.python.org/u/effigies)\
**Post date:** [June 26, 2023, 6:14pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/7 "2023-06-26T18:14:54Z")

</div>

Fair enough. I suppose it mostly applies to dataclass-like classes, in any case, as your internal fields-to-replace have to pretty closely map to your ` __init__ ` arguments for such a generic function to be useful.

---

<div class="post-metadata">

**Author:** ![jsbueno](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jsbueno/32/1210_2.png) [@jsbueno](https://discuss.python.org/u/jsbueno)\
**Post date:** [June 27, 2023, 2:59pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/8 "2023-06-27T14:59:42Z")

</div>

If a ` __replace__ ` special method gets traction (and I very much like the idea), the `replace` call would (and could) naturally be a built-in function.

Although there are some methods-that-use dunders that aren’t - like `copy.copy` and `deepcopy`, and all the pickle protocol, so it is not a given.

Since it looks like the default behavior for `. __replace__ ` is to create a new shallow copy with the requested replacements, maybe the `copy` module itself could be a coherent place for it.

---

<div class="post-metadata">

**Author:** ![a-reich](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/a-reich/32/5885_2.png) [@a-reich](https://discuss.python.org/u/a-reich)\
**Post date:** [June 27, 2023, 4:50pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/9 "2023-06-27T16:50:26Z")

</div>

My two cents are that if this were added, `copy` seems a reasonable location since the method promises to return a new object, so it’s basically a copy-with-replacing. Relatedly, are the implementations/protocol contract supposed to return a shallow or deep copy? Or should the API have an option for either?

---

<div class="post-metadata">

**Author:** ![effigies](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/effigies/32/35835_2.png) [@effigies](https://discuss.python.org/u/effigies)\
**Post date:** [June 27, 2023, 5:07pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/10 "2023-06-27T17:07:39Z")

</div>

I would expect it to be the same as calling ` __init__ ()` with some identical values and some alternative values. If ` __init__ ` makes a (deep)copy, then ` __replace__ ` will make a (deep)copy. Any deviations from that I would want documented.

And if I wanted a deepcopy, regardless, I would `deepcopy(replace(obj, **kwargs))` or `replace(deepcopy(obj),**kwargs)`.

---

<div class="post-metadata">

**Author:** ![ntessore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ntessore/32/10786_2.png) [@ntessore](https://discuss.python.org/u/ntessore)\
**Post date:** [June 27, 2023, 5:21pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/11 "2023-06-27T17:21:16Z")

</div>

Absorbing the `replace()` functionality into `copy(obj, attr=newvalue)` and `deepcopy(obj, attr=newvalue)` is not the worst interface either.

---

<div class="post-metadata">

**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\
**Post date:** [June 27, 2023, 5:58pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/12 "2023-06-27T17:58:49Z")

</div>

Yes, at first glance the `copy` module looks the best candidate. I thought about this. And it would be nice to support wider class of objects in `replace()` by falling back to `copy()` or the pickle protocol. But there are differences between `copy()` and `replace()` which makes this difficult.

1. `copy()` supports immutable objects. Setting attributes will fail later, but with wrong exception. Some of these objects could be supported using the pickle protocol, but ` __copy__ ()` and global registry have priority.
2. `copy()` treats classes and functions as atomic objects and return the argument. Most of attributes of Python classes and functions are mutable. Changing them will affect the original object.
3. By default (when the pickle protocol is used) `copy()` bypasses ` __init__ ()`. For `replace()` we usually want to call ` __init__ ()` to set calculated attribute which depend on specified attributes.
4. By default `copy()` sets all attributes, including internal attributes which should not be specified by user and should not be shared between instances (in dataclasses they are defined as fields with init=False).

The behavior of `copy(obj)` and `replace(obj)` will be too different to merge them in one function, and perhaps too different to have them in the same module. It is possible to add support of more general objects in `replace()`, but it will either be limited to very narrow class of objects, or work incorrectly in many cases.

---

<div class="post-metadata">

**Author:** ![stoneleaf](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/stoneleaf/32/88_2.png) [@stoneleaf](https://discuss.python.org/u/stoneleaf)\
**Post date:** [June 27, 2023, 11:16pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/13 "2023-06-27T23:16:13Z")

</div>

> [@storchaka](#):
>
> The behavior of `copy(obj)` and `replace(obj)` will be too different to merge them in one function, and perhaps too different to have them in the same module.

I think `replace()` should be its own function and not merged into `copy()`, but the `copy` module seems like a fine place for it – you are, essentially, copying an obect, and then making a change to it.

---

<div class="post-metadata">

**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\
**Post date:** [June 27, 2023, 11:30pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/14 "2023-06-27T23:30:37Z")

</div>

I also think a `replace` that worked on immutable objects via ` __init__ ` could be useful when trying to write in somewhat functional style. Imagine I had a list of `namedtuple`s and I wanted to blank out a field. I could write `[replace(t, big_secret="") for t in my_list]` to map them.

edit: of course, I didn’t use the proposed form 😅

---

<div class="post-metadata">

**Author:** ![thejcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/thejcannon/32/23759_2.png) [@thejcannon](https://discuss.python.org/u/thejcannon)\
**Post date:** [June 28, 2023, 2:26am UTC](https://discuss.python.org/t/generalize-replace-function/28511/15 "2023-06-28T02:26:42Z")

</div>

Maybe a pro, maybe a con, but right now I don’t see `mypy` or `pyright` typechecking `dataclasses.replace` semantically. ([opened an issue for `pyright`](https://github.com/microsoft/pyright/issues/5397))

Perhaps ` __prepare__ ` would make that easier? Hopefully not harder 😅

+1 overall though, and +1 to `copy.replace` or similar.

---

<div class="post-metadata">

**Author:** ![Tinche](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/tinche/32/12507_2.png) [@Tinche](https://discuss.python.org/u/Tinche)\
**Post date:** [June 28, 2023, 8:43am UTC](https://discuss.python.org/t/generalize-replace-function/28511/16 "2023-06-28T08:43:54Z")

</div>

I can add this to attrs if we decide to do it.

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [June 28, 2023, 9:53am UTC](https://discuss.python.org/t/generalize-replace-function/28511/17 "2023-06-28T09:53:17Z")

</div>

> [@thejcannon](#):
>
> right now I don’t see `mypy` or `pyright` typechecking `dataclasses.replace` semantically.

It might be more convincing if there were a type checking operator to convert the dataclass fields to a `TypedDict`. Then, the annotation for `replace` would be:

```python
def replace[T: DataclassInstance](x: T,
                                  /,
                                  **kwargs: Unpack[DataclassFields[T]]) -> T:
    ...

```

I’m not sure about the more general `replace` requested here. Are all non-method attributes replaceable? If so, maybe replace `DataclassFields` with `NonMethodAttributes` in the above?

---

<div class="post-metadata">

**Author:** ![antonagestam](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/antonagestam/32/12473_2.png) [@antonagestam](https://discuss.python.org/u/antonagestam)\
**Post date:** [July 2, 2023, 6:45pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/18 "2023-07-02T18:45:58Z")

</div>

I guess that would need some special operator to make all fields optional?

---

<div class="post-metadata">

**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\
**Post date:** [September 1, 2023, 10:22am UTC](https://discuss.python.org/t/generalize-replace-function/28511/19 "2023-09-01T10:22:32Z")

</div>

Thank you all. By the results of the discussion the new function will be added in the `copy` module.

Issue and PR:

> <https://github.com/python/cpython/issues/108751>
>
> \# Feature or enhancement
> 
> \### Has this already been discussed elsewhere?
> 
> I have… already discussed this feature proposal on Discourse
> 
> \### Links to previous discussion of this feature:
> 
> https://discuss.python.org/t/generalize-replace-function/28511
> 
> \### Proposal:
> 
> Some classes have the \`replace()\` method, which creates a modified copy of the object (modified values are provided as keyword arguments). Named tuples have the \`\_replace()\` for this (to avoid conflict with attribute \`replace\`). Dataclasses provide a global function for this.
> 
> I proposed to generalize \`dataclasses.replace()\` to support all classes which need this feature. By the result of the discussion on discuss.python.org, the new function will be added in the \`copy\` module as \`copy.replace()\`. \`dataclasses.replace()\` will continue to support only dataclasses. Dataclasses, named tuples and all classes which currently have the \`replace()\` method with suitable semantic will get also the \`\_\_replace\_\_()\` method. Now you can add such feature in new classes without conflicting with the \`replace\` attribute, and use this feature in general code without conflicting with \`str.replace()\` and like.
> 
> For now, \`copy.replace()\` is more limited than \`copy.copy()\` and does not fall back to use the powerful pickle protocol.
> 
> 
> \### Linked PRs
> \* gh-108752

> <https://github.com/python/cpython/pull/108752>
>
> It creates a modified copy of an object by calling the object's \`\_\_replace\_\_()\` …method.
> 
> It is a generalization of dataclasses.replace(), named tuple's \_replace() method and replace() methods in various classes, and supports all these stdlib classes.
> 
> 
> 
> \* Issue: gh-108751
> 
> 
> 
> 
> \----
> :books: Documentation preview :books:: https://cpython-previews--108752.org.readthedocs.build/

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [September 1, 2023, 5:43pm UTC](https://discuss.python.org/t/generalize-replace-function/28511/20 "2023-09-01T17:43:13Z")

</div>

That’s awesome!!

Will the typeshed have an appropriate base class? Maybe something like:

```auto
class SupportsReplace[**P]:
    def __replace__ (self, **kwargs: P.kwargs) -> Self:
        ...

# copy.replace:
def replace[T: SupportsReplace[P], **P](obj: T, /,**changes: P.kwargs) -> T:
    ...

```

I’m not sure if the latter annotation is allowed?

[Next page](https://discuss.python.org/t/generalize-replace-function/28511.md?page=2)