From 424be0a0f009c006dcc767d3ed9609a125e56130 Mon Sep 17 00:00:00 2001 From: Brock Date: Wed, 21 Feb 2024 12:28:57 -0800 Subject: [PATCH] Roadmap: remove ArrayManager --- web/pandas/about/roadmap.md | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/web/pandas/about/roadmap.md b/web/pandas/about/roadmap.md index 4c12e3e786a32..aba95ec2c03fc 100644 --- a/web/pandas/about/roadmap.md +++ b/web/pandas/about/roadmap.md @@ -90,33 +90,6 @@ data types within pandas. This will let us take advantage of its I/O capabilities and provide for better interoperability with other languages and libraries using Arrow. -### Block manager rewrite - -We'd like to replace pandas current internal data structures (a -collection of 1 or 2-D arrays) with a simpler collection of 1-D arrays. - -Pandas internal data model is quite complex. A DataFrame is made up of -one or more 2-dimensional "blocks", with one or more blocks per dtype. -This collection of 2-D arrays is managed by the BlockManager. - -The primary benefit of the BlockManager is improved performance on -certain operations (construction from a 2D array, binary operations, -reductions across the columns), especially for wide DataFrames. However, -the BlockManager substantially increases the complexity and maintenance -burden of pandas. - -By replacing the BlockManager we hope to achieve - -- Substantially simpler code -- Easier extensibility with new logical types -- Better user control over memory use and layout -- Improved micro-performance -- Option to provide a C / Cython API to pandas' internals - -See [these design -documents](https://dev.pandas.io/pandas2/internal-architecture.html#removal-of-blockmanager-new-dataframe-internals) -for more. - ### Decoupling of indexing and internals The code for getting and setting values in pandas' data structures