Skip to content

SparseDataFrame to_csv returns IndexError: too many indices for array #19384

Closed
@hannahlindsley

Description

@hannahlindsley

Calling to_csv on a SparseDataFrame returns IndexError: too many indices for array.

import pandas as pd
from scipy.sparse import rand

x = rand(100, 10001, density=0.2, format='csr')
df = pd.SparseDataFrame(x)
df.to_csv("tmp.csv")

Calling .to_dense() on the df prior to writing out, of course, works fine.

df.to_dense().to_csv("tmp.csv")

I'd like to be able to save sparse data to disk. If for some reason it shouldn't be written out this way, is there any way to add that in the doc or the error message?

Expected Output

Nontype, no exception, and a csv on the file system.

INSTALLED VERSIONS

commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Darwin
OS-release: 16.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.0
pytest: None
pip: 9.0.1
setuptools: 28.8.0
Cython: None
numpy: 1.13.3
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.1.0
psycopg2: 2.7.3.2 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions