CLN: rebase to 0.12

BUG: groupby filter that return a series/ndarray truth testing

BUG: refixed GH3880, prop name index

BUG: not handling sparse block deletes in internals/_delete_from_block

BUG: refix generic/truncate

TST: refixed generic/replace (bug in core/internals/putmask) revealed as well

TST: fix spare_array to put up correct type exceptions rather than Exception

CLN: cleanups

BUG: fix stata dtype inference (error in core/internals/astype)

BUG: fix ujson handling of new series object

BUG: fixed scalar coercion (e.g. calling float(series)) to work

BUG: fixed astyping with and w/o copy

ENH: added _propogate_attributes method to generic.py to allow
     subclasses to automatically propogate things like name

DOC: added v0.13.0.txt feature descriptions

CLN: pep8ish cleanups

BUG: fix 32-bit,numpy 1.6.1 issue with datetimes in astype_nansafe

PERF: speedup for groupby by passing a SNDArray (Series like ndarray) object to evaluation functions
      if allowed, can avoid Series creation overhead

BUG: issue with older numpy (1.6.1) in SeriesGrouper, fallback to passing a Series
     rather than SNDArray

DOC: release notes & doc updates

DOC: fixup doc build failures

DOC: change pasing of direct ndarrays to cython doc functions (enhancedperformance.rst)

Author: jreback

doc/source/basics.rst

@@ -477,7 +477,7 @@ maximum value for each column occurred:
 
    tsdf = DataFrame(randn(1000, 3), columns=['A', 'B', 'C'],
                     index=date_range('1/1/2000', periods=1000))
-   tsdf.apply(lambda x: x.index[x.dropna().argmax()])
+   tsdf.apply(lambda x: x[x.idxmax()])
 
 You may also pass additional arguments and keyword arguments to the ``apply``
 method. For instance, consider the following function you would like to apply:

doc/source/dsintro.rst

@@ -44,10 +44,15 @@ When using pandas, we recommend the following import convention:
 Series
 ------
 
-:class:`Series` is a one-dimensional labeled array (technically a subclass of
-ndarray) capable of holding any data type (integers, strings, floating point
-numbers, Python objects, etc.). The axis labels are collectively referred to as
-the **index**. The basic method to create a Series is to call:
+.. warning::
+
+   In 0.13.0 ``Series`` has internaly been refactored to no longer sub-class ``ndarray``
+   but instead subclass ``NDFrame``, similarly to the rest of the pandas containers. This should be
+   a transparent change with only very limited API implications (See the :ref:`release notes <release.refactoring_0_13_0>`)
+
+:class:`Series` is a one-dimensional labeled array capable of holding any data
+type (integers, strings, floating point numbers, Python objects, etc.). The axis
+labels are collectively referred to as the **index**. The basic method to create a Series is to call:
 
 ::
 
@@ -109,9 +114,8 @@ provided. The value will be repeated to match the length of **index**
 Series is ndarray-like
 ~~~~~~~~~~~~~~~~~~~~~~
 
-As a subclass of ndarray, Series is a valid argument to most NumPy functions
-and behaves similarly to a NumPy array. However, things like slicing also slice
-the index.
+``Series`` acts very similary to a ``ndarray``, and is a valid argument to most NumPy functions.
+However, things like slicing also slice the index.
 
 .. ipython :: python
 
@@ -177,7 +181,7 @@ labels.
 
 The result of an operation between unaligned Series will have the **union** of
 the indexes involved. If a label is not found in one Series or the other, the
-result will be marked as missing (NaN). Being able to write code without doing
+result will be marked as missing ``NaN``. Being able to write code without doing
 any explicit data alignment grants immense freedom and flexibility in
 interactive data analysis and research. The integrated data alignment features
 of the pandas data structures set pandas apart from the majority of related
@@ -924,11 +928,11 @@ Here we slice to a Panel4D.
     from pandas.core import panelnd
     Panel5D = panelnd.create_nd_panel_factory(
         klass_name   = 'Panel5D',
-        axis_orders  = [ 'cool', 'labels','items','major_axis','minor_axis'],
-        axis_slices  = { 'labels' : 'labels', 'items' : 'items',
-	                 'major_axis' : 'major_axis', 'minor_axis' : 'minor_axis' },
-        slicer       = Panel4D,
-        axis_aliases = { 'major' : 'major_axis', 'minor' : 'minor_axis' },
+        orders  = [ 'cool', 'labels','items','major_axis','minor_axis'],
+        slices  = { 'labels' : 'labels', 'items' : 'items',
+	                'major_axis' : 'major_axis', 'minor_axis' : 'minor_axis' },
+        slicer  = Panel4D,
+        aliases = { 'major' : 'major_axis', 'minor' : 'minor_axis' },
         stat_axis    = 2)
 
     p5d = Panel5D(dict(C1 = p4d))

doc/source/enhancingperf.rst

@@ -26,7 +26,7 @@ Enhancing Performance
 Cython (Writing C extensions for pandas)
 ----------------------------------------
 
-For many use cases writing pandas in pure python and numpy is sufficient. In some 
+For many use cases writing pandas in pure python and numpy is sufficient. In some
 computationally heavy applications however, it can be possible to achieve sizeable
 speed-ups by offloading work to `cython <http://cython.org/>`_.
 
@@ -35,7 +35,7 @@ trying to remove for loops and making use of numpy vectorization, it's always wo
 optimising in python first.
 
 This tutorial walks through a "typical" process of cythonizing a slow computation.
-We use an `example from the cython documentation <http://docs.cython.org/src/quickstart/cythonize.html>`_ 
+We use an `example from the cython documentation <http://docs.cython.org/src/quickstart/cythonize.html>`_
 but in the context of pandas. Our final cythonized solution is around 100 times
 faster than the pure python.
 
@@ -68,7 +68,7 @@ Here's the function in pure python:
 We achieve our result by by using ``apply`` (row-wise):
 
 .. ipython:: python
-   
+
    %timeit df.apply(lambda x: integrate_f(x['a'], x['b'], x['N']), axis=1)
 
 But clearly this isn't fast enough for us. Let's take a look and see where the
@@ -83,7 +83,7 @@ By far the majority of time is spend inside either ``integrate_f`` or ``f``,
 hence we'll concentrate our efforts cythonizing these two functions.
 
 .. note::
- 
+
   In python 2 replacing the ``range`` with its generator counterpart (``xrange``)
   would mean the ``range`` line would vanish. In python 3 range is already a generator.
 
@@ -125,7 +125,7 @@ is here to distinguish between function versions):
 
    %timeit df.apply(lambda x: integrate_f_plain(x['a'], x['b'], x['N']), axis=1)
 
-Already this has shaved a third off, not too bad for a simple copy and paste. 
+Already this has shaved a third off, not too bad for a simple copy and paste.
 
 .. _enhancingperf.type:
 
@@ -175,7 +175,7 @@ in python, so maybe we could minimise these by cythonizing the apply part.
   We are now passing ndarrays into the cython function, fortunately cython plays
   very nicely with numpy.
 
-.. ipython:: 
+.. ipython::
 
    In [4]: %%cython
       ...: cimport numpy as np
@@ -205,20 +205,38 @@ The implementation is simple, it creates an array of zeros and loops over
 the rows, applying our ``integrate_f_typed``, and putting this in the zeros array.
 
 
+.. warning::
+
+   In 0.13.0 since ``Series`` has internaly been refactored to no longer sub-class ``ndarray``
+   but instead subclass ``NDFrame``, you can **not pass** a ``Series`` directly as a ``ndarray`` typed parameter
+   to a cython function. Instead pass the actual ``ndarray`` using the ``.values`` attribute of the Series.
+
+   Prior to 0.13.0
+
+   .. code-block:: python
+
+        apply_integrate_f(df['a'], df['b'], df['N'])
+
+   Use ``.values`` to get the underlying ``ndarray``
+
+   .. code-block:: python
+
+        apply_integrate_f(df['a'].values, df['b'].values, df['N'].values)
+
 .. note::
 
     Loop like this would be *extremely* slow in python, but in cython looping over
     numpy arrays is *fast*.
 
 .. ipython:: python
 
-   %timeit apply_integrate_f(df['a'], df['b'], df['N'])
+   %timeit apply_integrate_f(df['a'].values, df['b'].values, df['N'].values)
 
 We've gone another three times faster! Let's check again where the time is spent:
 
 .. ipython:: python
 
-   %prun -l 4 apply_integrate_f(df['a'], df['b'], df['N'])
+   %prun -l 4 apply_integrate_f(df['a'].values, df['b'].values, df['N'].values)
 
 As one might expect, the majority of the time is now spent in ``apply_integrate_f``,
 so if we wanted to make anymore efficiencies we must continue to concentrate our
@@ -261,7 +279,7 @@ advanced cython techniques:
 
 .. ipython:: python
 
-   %timeit apply_integrate_f_wrap(df['a'], df['b'], df['N'])
+   %timeit apply_integrate_f_wrap(df['a'].values, df['b'].values, df['N'].values)
 
 This shaves another third off!
 
@@ -270,4 +288,4 @@ Further topics
 
 - Loading C modules into cython.
 
-Read more in the `cython docs <http://docs.cython.org/>`_.
+Read more in the `cython docs <http://docs.cython.org/>`_.

doc/source/release.rst

@@ -42,6 +42,68 @@ pandas 0.13
 
 **API Changes**
 
+**Internal Refactoring**
+
+.. _release.refactoring_0_13_0:
+
+In 0.13.0 there is a major refactor primarily to subclass ``Series`` from ``NDFrame``,
+which is the base class currently for ``DataFrame`` and ``Panel``, to unify methods
+and behaviors. Series formerly subclassed directly from ``ndarray``.
+
+- Refactor of series.py/frame.py/panel.py to move common code to generic.py
+  - added _setup_axes to created generic NDFrame structures
+  - moved methods
+
+    - from_axes,_wrap_array,axes,ix,shape,empty,swapaxes,transpose,pop
+    - __iter__,keys,__contains__,__len__,__neg__,__invert__
+    - convert_objects,as_blocks,as_matrix,values
+    - __getstate__,__setstate__ (though compat remains in frame/panel)
+    - __getattr__,__setattr__
+    - _indexed_same,reindex_like,reindex,align,where,mask
+    - filter (also added axis argument to selectively filter on a different axis)
+    - reindex,reindex_axis (which was the biggest change to make generic)
+    - truncate (moved to become part of ``NDFrame``)
+
+- These are API changes which make ``Panel`` more consistent with ``DataFrame``
+  - swapaxes on a Panel with the same axes specified now return a copy
+  - support attribute access for setting
+  - filter supports same api as original DataFrame filter
+
+- Reindex called with no arguments will now return a copy of the input object
+
+- Series now inherits from ``NDFrame`` rather than directly from ``ndarray``.
+  There are several minor changes that affect the API.
+
+  - numpy functions that do not support the array interface will now
+    return ``ndarrays`` rather than series, e.g. ``np.diff`` and ``np.where``
+  - ``Series(0.5)`` would previously return the scalar ``0.5``, this is no
+    longer supported
+  - several methods from frame/series have moved to ``NDFrame``
+    (convert_objects,where,mask)
+  - ``TimeSeries`` is now an alias for ``Series``. the property ``is_time_series``
+    can be used to distinguish (if desired)
+
+- Refactor of Sparse objects to use BlockManager
+
+  - Created a new block type in internals, ``SparseBlock``, which can hold multi-dtypes
+    and is non-consolidatable. ``SparseSeries`` and ``SparseDataFrame`` now inherit
+    more methods from there hierarchy (Series/DataFrame), and no longer inherit
+    from ``SparseArray`` (which instead is the object of the ``SparseBlock``)
+  - Sparse suite now supports integration with non-sparse data. Non-float sparse
+    data is supportable (partially implemented)
+  - Operations on sparse structures within DataFrames should preserve sparseness,
+    merging type operations will convert to dense (and back to sparse), so might
+    be somewhat inefficient
+  - enable setitem on ``SparseSeries`` for boolean/integer/slices
+  - ``SparsePanels`` implementation is unchanged (e.g. not using BlockManager, needs work)
+
+- added ``ftypes`` method to Series/DataFame, similar to ``dtypes``, but indicates
+  if the underlying is sparse/dense (as well as the dtype)
+
+- All ``NDFrame`` objects now have a ``_prop_attributes``, which can be used to indcated various
+  values to propogate to a new object from an existing (e.g. name in ``Series`` will follow
+  more automatically now)
+
 **Experimental Features**
 
 **Bug Fixes**

doc/source/v0.13.0.txt

@@ -9,15 +9,76 @@ enhancements along with a large number of bug fixes.
 API changes
 ~~~~~~~~~~~
 
-  - ``read_excel`` now supports an integer in its ``sheetname`` argument giving
-    the index of the sheet to read in (:issue:`4301`).
-
 Enhancements
 ~~~~~~~~~~~~
 
   - ``read_html`` now raises a ``URLError`` instead of catching and raising a
     ``ValueError`` (:issue:`4303`, :issue:`4305`)
 
+  - ``read_excel`` now supports an integer in its ``sheetname`` argument giving
+    the index of the sheet to read in (:issue:`4301`).
+
+Internal Refactoring
+~~~~~~~~~~~~~~~~~~~~
+
+In 0.13.0 there is a major refactor primarily to subclass ``Series`` from ``NDFrame``,
+which is the base class currently for ``DataFrame`` and ``Panel``, to unify methods
+and behaviors. Series formerly subclassed directly from ``ndarray``. (:issue:`4080`,:issue:`3862`,:issue:`816`)
+
+- Refactor of series.py/frame.py/panel.py to move common code to generic.py
+  - added _setup_axes to created generic NDFrame structures
+  - moved methods
+
+    - from_axes,_wrap_array,axes,ix,shape,empty,swapaxes,transpose,pop
+    - __iter__,keys,__contains__,__len__,__neg__,__invert__
+    - convert_objects,as_blocks,as_matrix,values
+    - __getstate__,__setstate__ (though compat remains in frame/panel)
+    - __getattr__,__setattr__
+    - _indexed_same,reindex_like,reindex,align,where,mask
+    - filter (also added axis argument to selectively filter on a different axis)
+    - reindex,reindex_axis (which was the biggest change to make generic)
+    - truncate (moved to become part of ``NDFrame``)
+
+- These are API changes which make ``Panel`` more consistent with ``DataFrame``
+  - swapaxes on a Panel with the same axes specified now return a copy
+  - support attribute access for setting
+  - filter supports same api as original DataFrame filter
+
+- Reindex called with no arguments will now return a copy of the input object
+
+- Series now inherits from ``NDFrame`` rather than directly from ``ndarray``.
+  There are several minor changes that affect the API.
+
+  - numpy functions that do not support the array interface will now
+    return ``ndarrays`` rather than series, e.g. ``np.diff`` and ``np.where``
+  - ``Series(0.5)`` would previously return the scalar ``0.5``, this is no
+    longer supported
+  - several methods from frame/series have moved to ``NDFrame``
+    (convert_objects,where,mask)
+  - ``TimeSeries`` is now an alias for ``Series``. the property ``is_time_series``
+    can be used to distinguish (if desired)
+
+- Refactor of Sparse objects to use BlockManager
+
+  - Created a new block type in internals, ``SparseBlock``, which can hold multi-dtypes
+    and is non-consolidatable. ``SparseSeries`` and ``SparseDataFrame`` now inherit
+    more methods from there hierarchy (Series/DataFrame), and no longer inherit
+    from ``SparseArray`` (which instead is the object of the ``SparseBlock``)
+  - Sparse suite now supports integration with non-sparse data. Non-float sparse
+    data is supportable (partially implemented)
+  - Operations on sparse structures within DataFrames should preserve sparseness,
+    merging type operations will convert to dense (and back to sparse), so might
+    be somewhat inefficient
+  - enable setitem on ``SparseSeries`` for boolean/integer/slices
+  - ``SparsePanels`` implementation is unchanged (e.g. not using BlockManager, needs work)
+
+- added ``ftypes`` method to Series/DataFame, similar to ``dtypes``, but indicates
+  if the underlying is sparse/dense (as well as the dtype)
+
+- All ``NDFrame`` objects now have a ``_prop_attributes``, which can be used to indcated various
+  values to propogate to a new object from an existing (e.g. name in ``Series`` will follow
+  more automatically now)
+
 Bug Fixes
 ~~~~~~~~~
 

pandas/core/array.py

@@ -34,3 +34,19 @@
     globals()[_f] = getattr(np.random, _f)
 
 NA = np.nan
+
+#### a series-like ndarray ####
+
+class SNDArray(Array):
+
+    def __new__(cls, data, index=None, name=None):
+        data = data.view(SNDArray)
+        data.index = index
+        data.name = name
+
+        return data
+
+    @property
+    def values(self):
+        return self.view(Array)
+

pandas/core/base.py

@@ -8,20 +8,6 @@ class StringMixin(object):
     """implements string methods so long as object defines a `__unicode__` method.
     Handles Python2/3 compatibility transparently."""
     # side note - this could be made into a metaclass if more than one object nees
-    def __str__(self):
-
-class PandasObject(object):
-    """ The base class for pandas objects """
-
-    #----------------------------------------------------------------------
-    # Reconstruction
-
-    def save(self, path):
-        com.save(self, path)
-
-    @classmethod
-    def load(cls, path):
-        return com.load(path)
 
     #----------------------------------------------------------------------
     # Formatting