Skip to content

py/runtime: Do generic lookup if type->attr fails. - #6253

Closed
laurensvalk wants to merge 2 commits into
micropython:masterfrom
pybricks:attr_load
Closed

laurensvalk wants to merge 2 commits into
micropython:masterfrom
pybricks:attr_load

Conversation

@laurensvalk

@laurensvalk laurensvalk commented Jul 17, 2020 •

Copy link
Copy Markdown
Contributor

If the custom type->attr fails, don't give up just yet but look in the
locals_dict. This allows type->attr to extend rather than completely
replace the lookup.

This is useful for custom builtin classes that have mostly regular
methods but just a few special attributes/properties. This way,
type->attr needs to deal with the special cases only and the default
lookup will be used for generic methods.

This is an alternative to #4969.

The original was perhaps slightly more convenient since no custom
attr handler was required, but this is a much smaller change overall.
This new PR is also a bit more generic, as this makes it quite easy to add
properties that do a bit more than just returning an existing attribute.

EDIT: Rebased onto latest master.

@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Jul 20, 2020
@dpgeorge

Copy link
Copy Markdown
Member

Thanks for this alternative, it's definitely much simpler.

One thing to check is if there are existing types (eg user defined classes and instances thereof) which do a full lookup in type->attr(...) and if it fails then doing an extra lookup in locals_dict would be incorrect/inefficient. Eg doing int.xyz will call type_attr(...) which will search locals_dict, fail, and then it'll be searched again by this PR's change.

For your use-case would you prefer #4969 or this here? Ultimately both of the patches could be useful for different things, they can both coexist.

@laurensvalk

Copy link
Copy Markdown
Contributor Author

Thanks. What about the following?

    // dest[0] = MP_OBJ_NULL means load
    //  return: for fail, do nothing
    //          for continue lookup in locals_dict, dest[0] = MP_OBJ_SENTINEL    <---- **NEW**
    //          for attr, dest[0] = value
    //          for method, dest[0] = method, dest[1] = self

This way, the attr function can explicitly decide what is done next and, if I understand correctly, existing code would not be affected. If this makes sense, I'll update the PR. The change to runtime.c is almost the same.

For your use-case would you prefer #4969 or this here? Ultimately both of the patches could be useful for different things, they can both coexist.

Probably still #4969 (or something like it) for most of our use cases, which is returning attributes without running code on them.

This new PR originated from a different use case where we did want to run some code on reading a property (return transpose on my_matrix.T). It was after experimenting with it that I realized it could theoretically be an alternative if #4969 is too big to merge, but I agree that having them coexist would be nice.

@dpgeorge

Copy link
Copy Markdown
Member

This way, the attr function can explicitly decide what is done next and, if I understand correctly, existing code would not be affected. If this makes sense, I'll update the PR.

That does sound like a good idea.

@dpgeorge

Copy link
Copy Markdown
Member

Another thing to keep in mind with this and #4969 is that if/when they get merged they become part of the public C API, which makes it harder to change/refactor things later on, eg to make lookups for native types more efficient, see #4360 .

@laurensvalk

Copy link
Copy Markdown
Contributor Author

This way, the attr function can explicitly decide what is done next and, if I understand correctly, existing code would not be affected. If this makes sense, I'll update the PR.

That does sound like a good idea.

This turns out to be not entirely trivial.

  • Setting dest[0] = MP_OBJ_SENTINEL alone as suggested above is not really possible because a lot of code just checks for dest[0] == MP_OBJ_NULL. Also, dest[0] = MP_OBJ_SENTINEL looks to be used already for a slightly unrelated use case.
  • A similar issue arises with using dest[1] == MP_OBJ_SENTINEL. When it is not NULL after the initial lookup, some code (e.g. mp_load_attr) then tries to treat a normal attribute as a bound method and crashes. Both do work if you immediately set it back to MP_OBJ_NULL after checking.

So again quite a few options (and variations of each) and I'm not sure which is the best.

In the update that just pushed, I've opted to use dest[1] == MP_OBJ_SENTINEL as the criterion as I think it was not used anywhere yet. Instead of resetting it to NULL right after the check, I've set dest[1] = MP_OBJ_NULL in mp_convert_member_lookup in a separate commit, which seems to be consistent with most lookup/loading functions.

Make it possible to proceed to a regular lookup in locals_dict if
the custom type->attr fails. This allows type->attr to extend
rather than completely replace the lookup.

This is useful for custom builtin classes that have mostly regular
methods but just a few special attributes/properties. This way,
type->attr needs to deal with the special cases only and the default
lookup will be used for generic methods.

Signed-off-by: Laurens Valk <[email protected]>
Clearing it is consistent with other lookups. It is required because
type->attr may set dest[1] to MP_OBJ_SENTINEL on failure.

Signed-off-by: Laurens Valk <[email protected]>
@dpgeorge

Copy link
Copy Markdown
Member

Implemented in e2ca8ab

tannewt pushed a commit to tannewt/circuitpython that referenced this pull request May 12, 2022
…n-main

Translations update from Hosted Weblate
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

py-core Relates to py/ directory in source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants