Skip to content

Conversation

tantaman
Copy link
Contributor

@tantaman tantaman commented Aug 2, 2024

So I was trying to figure out the best way to go about updating the IVM code given the core types flowing through it are changing. Rather than update the code and have to track down a million type errors and try to change everything all at once it strikes me as being easier to create an ivm-2 directory and re-implement (potentially using copy-pasta from the ivm dir) the operators, sources and views.

Once ivm-2 is done we can swap it into the rest of the machinery (ast, query builder, pipeline builder) and delete the old ivm directory.

This PR lays out the scaffolding for IVM and re-introduces (while also fixing, streamlining and renaming):

  • Materialite
  • Source
  • DifferenceStream

The PR also implements the NestedIterable. NestedIterable has been renamed to IterableTree here. I think IterableTree is a more accurate of a description since sibling iterables can exist.

Now that we have the scaffolding and IterableTree, plugging in the operators should be relatively straightforward.

Next steps:

  • Further define pull
  • A view implementation
  • filter/join/topk
  • Wiring together with an AST that supports subqueries / implementing a PipelineBuilder that understands subqueries

So I was trying to figure out the best way to go about updating the IVM code given the core types flowing through it are changing.
Rather than update the code and have to track down a million type errors and try to change everything all at once it strikes me as being easier to create an `ivm-2` directory and re-implement (potentially using copy-pasta from the ivm dir) the operators, sources and views.

Once `ivm-2` is done we can swap it into the rest of the machinery (ast, query builder, query api) and delete the old `ivm` directory.

This PR lays out the scaffolding for IVM and re-introduces (while also fixing, streamlining and renaming):
- Materialite
- Source
- DifferenceStream

And implements the `NestedIterable`. `NestedIterable` has been renamed to `IterableTree` here. I think `IterableTree` is a bit more accurate of a description since the sibling iterables can exist.

Now that we have the scaffolding and `IterableTree`, plugging in the operators should be relatively straightforward.

Next steps:
- Further define `pull`
- A view implementation
- filter/join/topk
- Wiring together with an AST that supports subqueries / implementing a PipelineBuilder that understands subqueries
Copy link

vercel bot commented Aug 2, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
replicache-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 2, 2024 1:44pm
zeppliear ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 2, 2024 1:44pm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant