Skip to content

Commit 95427d5

Browse files
bnaulTomAugspurger
authored andcommitted
[ENH] Add read support for Google Cloud Storage (#20729)
* Google Cloud Storage support using gcsfs
1 parent dbd102c commit 95427d5

18 files changed

+116
-19
lines changed

ci/appveyor-27.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ dependencies:
66
- beautifulsoup4
77
- bottleneck
88
- dateutil
9+
- gcsfs
910
- html5lib
1011
- jinja2=2.8
1112
- lxml

ci/check_imports.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55

66
blacklist = {
77
'bs4',
8+
'gcsfs',
89
'html5lib',
910
'ipython',
1011
'jinja2'

ci/circle-36-locale_slow.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ channels:
55
dependencies:
66
- beautifulsoup4
77
- cython
8+
- gcsfs
89
- html5lib
910
- ipython
1011
- jinja2

ci/requirements-optional-conda.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ blosc
33
bottleneck
44
fastparquet
55
feather-format
6+
gcsfs
67
html5lib
78
ipython>=5.6.0
89
ipykernel

ci/requirements-optional-pip.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ blosc
55
bottleneck
66
fastparquet
77
feather-format
8+
gcsfs
89
html5lib
910
ipython>=5.6.0
1011
ipykernel

ci/travis-27.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ dependencies:
99
- fastparquet
1010
- feather-format
1111
- flake8=3.4.1
12+
- gcsfs
1213
- html5lib
1314
- ipython
1415
- jemalloc=4.5.0.post

ci/travis-36.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ dependencies:
88
- dask
99
- fastparquet
1010
- feather-format
11+
- gcsfs
1112
- geopandas
1213
- html5lib
1314
- ipython

doc/source/install.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -276,6 +276,7 @@ Optional Dependencies
276276
* `Jinja2 <http://jinja.pocoo.org/>`__: Template engine for conditional HTML formatting.
277277
* `s3fs <http://s3fs.readthedocs.io/>`__: necessary for Amazon S3 access (s3fs >= 0.0.7).
278278
* `blosc <https://pypi.org/project/blosc>`__: for msgpack compression using ``blosc``
279+
* `gcsfs <http://gcsfs.readthedocs.io/>`__: necessary for Google Cloud Storage access (gcsfs >= 0.1.0).
279280
* One of
280281
`qtpy <https://github.com/spyder-ide/qtpy>`__ (requires PyQt or PySide),
281282
`PyQt5 <https://www.riverbankcomputing.com/software/pyqt/download5>`__,

doc/source/whatsnew/v0.24.0.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Other Enhancements
1818
- :func:`Series.mode` and :func:`DataFrame.mode` now support the ``dropna`` parameter which can be used to specify whether NaN/NaT values should be considered (:issue:`17534`)
1919
- :func:`to_csv` now supports ``compression`` keyword when a file handle is passed. (:issue:`21227`)
2020
- :meth:`Index.droplevel` is now implemented also for flat indexes, for compatibility with :class:`MultiIndex` (:issue:`21115`)
21-
21+
- Added support for reading from Google Cloud Storage via the ``gcsfs`` library (:issue:`19454`)
2222

2323
.. _whatsnew_0240.api_breaking:
2424

pandas/conftest.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import importlib
2+
13
import pytest
24

35
import numpy as np
@@ -249,3 +251,17 @@ def any_int_dtype(request):
249251
"""
250252

251253
return request.param
254+
255+
256+
@pytest.fixture
257+
def mock():
258+
"""
259+
Fixture providing the 'mock' module.
260+
261+
Uses 'unittest.mock' for Python 3. Attempts to import the 3rd party 'mock'
262+
package for Python 2, skipping if not present.
263+
"""
264+
if PY3:
265+
return importlib.import_module("unittest.mock")
266+
else:
267+
return pytest.importorskip("mock")

0 commit comments

Comments
 (0)