Skip to content

Apache Arrow for table cells #315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

Apache Arrow for table cells #315

wants to merge 1 commit into from

Conversation

mbostock
Copy link
Member

@mbostock mbostock commented Nov 5, 2022 •
edited
Loading

TODO

  • Implement table operations in __arrow

@mbostock mbostock changed the title support Apache Arrow for table cells Apache Arrow for table cells Nov 6, 2022
@mbostock mbostock mentioned this pull request Nov 7, 2022
Merged
mbostock
mbostock commented Nov 7, 2022
View reviewed changes
src/table.mjs
Comment on lines +409 to +414
// This function applies table cell operations to an in-memory Apache Arrow
// table; it should be equivalent to the corresponding SQL query.
function __arrow(source, operations) {
operations;
return source; // TODO
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the Arrow DataFrame class and predicates removed in Apache Arrow 7.0.0 and later (apache/arrow#10371), I’m not really sure there’s a practical way to implement basic data transformations like filter and sort directly on Arrow Tables. ☹️ At least, it looks hard. Perhaps we need to instantiate a DuckDBClient #313 or Arquero Table here?

@mbostock
Copy link
Member Author

mbostock commented Nov 8, 2022

Mostly folded into #313, leaving only the need to implement __table for Apache Arrow tables… which I’m not sure how to do, so closing for now.

@mbostock mbostock closed this Nov 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@mbostock