Repository navigation
uniffi_bindgen: make recursive type detection deterministic and follow custom types - #3014
Conversation
`find_recursive_enum_names` skipped any node that an earlier DFS had already finished, so a node that is on a cycle only through a finished node was never marked. The graph is a `HashMap`, so whether that happened depended on iteration order. Replace the DFS with Tarjan's strongly connected components algorithm and mark every node in a component with more than one member, plus any node with a self-edge. `type_names_in_type` also ignored `Type::Custom`, so a cycle through a custom type was never found. Follow the builtin type in both the pipeline copy and the `ComponentInterface` copy. Fixes #3005
bendk
left a comment
There was a problem hiding this comment.
The custom type fix looks good. The change to Tarjan's strongly connected components algorithm feels overengineered to me. Would switching from HashMap to IndexMap work instead?
| lowlink: HashMap<&'a T, usize>, | ||
| stack: Vec<&'a T>, | ||
| on_stack: HashSet<&'a T>, | ||
| recursive: HashSet<T>, |
There was a problem hiding this comment.
This feels like extreme overkill to me. Based on the bug description, the only reason that the search is non-deterministic is because we're iterating over a hash map. We could fix that by switching to an IndexMap and keeping everything else the same.
A type is recursive if it can reach itself. Check that with a small DFS per type and no shared visited set, so the result is order independent.
|
Fair point, Tarjan was more machinery than this needs. One catch with With So the field order would still decide whether I've pushed a simpler version instead: a type is recursive if it can reach itself, checked with a small DFS per type and no shared visited set. It is order independent, so no map change is needed, and the custom type fix and tests are unchanged. |
Which seems fine to me, since the |
| /// A node is in a cycle if any path of edges leads back to itself. | ||
| /// A node is recursive if some path of edges leads back to it. This is checked | ||
| /// separately for each node, so the result does not depend on the iteration | ||
| /// order of `deps`. |
There was a problem hiding this comment.
I think this comment change should be reverted now.
| } | ||
| if seen.insert(node) { | ||
| stack.extend(deps.get(node).into_iter().flatten()); | ||
| } |
There was a problem hiding this comment.
I like changing the DFS into a while loop with a stack, it seems simpler overall. However, I'm not sure I like how this is starting a new search for each type. IIUC, this changes it so if we have a cycle A -> B -> C -> A, then the new code will flag A, B, and C while the old code would only flag one of them. I think I prefer the old behavior for the since we only need to flag one to ensure the cycle is broken. I could easily be missing something here though.
| orig_name: name.to_string(), | ||
| builtin: Box::new(builtin), | ||
| } | ||
| } |
There was a problem hiding this comment.
I'd rather see the type written out than have this helper function. It's not many lines and I think it's easier to understand tests when the level of indirection is low.
| - Detection of recursive enums and records is now deterministic. Previously a type in two | ||
| overlapping cycles was sometimes not marked `recursive`, depending on hash map iteration order, | ||
| and cycles that pass through a custom type were not detected at all, so the Swift bindings were | ||
| missing `indirect` and failed to compile (via [#3005](https://github.com/mozilla/uniffi-rs/issues/3005)). |
There was a problem hiding this comment.
The way I'm understanding this is there were 2 issues:
- Custom types weren't detected, which caused compile errors
- Cycle detection was non-deterministic, which caused different fields to be marked indirect for different builds. However, it still compiled.
Is this correct? If so, please update the changelog.
Go back to the single DFS from main, with its shared visited set, and make it deterministic by building the dependency graph as an `IndexMap<T, IndexSet<T>>` instead of a `HashMap<T, HashSet<T>>`. Both the pipeline and `ComponentInterface` build the graph from ordered `Vec`s, so the same types are marked recursive on every build. Every cycle still gets at least one member marked, which is enough to break it. Also revert the doc comment on `find_recursive_enum_names`, write the custom type out in the tests instead of using a helper, and split the changelog entry into the two separate fixes.
|
I've pushed 67b668a, which keeps the original DFS and switches both dependency graphs to The Yes, your reading of the two issues is correct. For what it's worth, the old code already flagged every member of a simple cycle like The overlap test now checks that the same graph always gives I appreciate the careful review on this one. |
bendk
left a comment
There was a problem hiding this comment.
This new version looks great, thanks for working through this with me.
|
Thanks bendk. Your review notes were spot on, especially keeping the original DFS and treating one flagged member per overlapping cycle as enough. Using IndexMap for the dep graph so visit order stays stable taught me a lot about what "deterministic" actually has to mean here. Really appreciate the careful pass! |
The
recursiveflag on enums and records, andComponentInterface::is_recursivewhich Swift uses to emitindirect, had two problems described in #3005.Non-deterministic result
find_recursive_enum_namesran a DFS that skipped any node already finished by an earlier traversal, assuming every cycle through it had been found. That does not hold: withT -> {U, V},U -> T,V -> U, visitingUfirst fromTmarksTandUand finishesU, thenT -> V -> Ustops atUandVis never marked. The graph is aHashMap, so the outcome changed from run to run.This replaces the DFS with Tarjan's strongly connected components algorithm, written in place with no new dependency. Every node in a component with more than one member is marked, plus any node with a self-edge. Component membership does not depend on visit order. The function keeps its generic signature, so the
ComponentInterfacepath gets the fix too.Cycles through custom types
Both copies of
type_names_in_type(the pipeline one and the one ininterface/mod.rs) fell through to_ => vec![]forType::Custom, soNode -> Wrapped (custom over Inner) -> Inner -> Nodehad no edge out ofWrapped. Both now follow the builtin type.Tests
find_recursive_overlapping_cycles_is_order_independentbuilds the graph above 64 times (eachHashMapgets a new seed) and checks that all three nodes are marked. It failed on every one of 10 runs against the old code and passes with this change.dep_graph_custom_type_propagates_dep(pipeline) checks that a custom type over a record creates the edge.infer_recursive_through_custom_typerunsinfer_recursive_typeson theNode/Wrapped/Innerexample from the issue and checks thatNodeandInnerare marked.interface::test::dep_graph_custom_type_propagates_depcovers theComponentInterfacecopy through UDL ([Custom] typedef Inner Wrapped;), includingis_recursive.The Python
NameErrormentioned in the issue (theWrapped = Inneralias emitted beforeclass Inner) comes from the sort order inpipeline/general/sort.rsand is intentionally left out of this PR.Fixes #3005