Skip to content

Commit 5f584bd

Browse files
authored
CI: unpin pyarrow, fix failing test (#51175)
* unpin pyarrow, fix failing test * cleanup * handle NaT/NaN
1 parent 9eec5bf commit 5f584bd

File tree

12 files changed

+36
-11
lines changed

12 files changed

+36
-11
lines changed

.github/actions/setup-conda/action.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ runs:
1818
- name: Set Arrow version in ${{ inputs.environment-file }} to ${{ inputs.pyarrow-version }}
1919
run: |
2020
grep -q ' - pyarrow' ${{ inputs.environment-file }}
21-
sed -i"" -e "s/ - pyarrow<11/ - pyarrow=${{ inputs.pyarrow-version }}/" ${{ inputs.environment-file }}
21+
sed -i"" -e "s/ - pyarrow/ - pyarrow=${{ inputs.pyarrow-version }}/" ${{ inputs.environment-file }}
2222
cat ${{ inputs.environment-file }}
2323
shell: bash
2424
if: ${{ inputs.pyarrow-version }}

ci/deps/actions-310.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ dependencies:
4242
- psycopg2
4343
- pymysql
4444
- pytables
45-
- pyarrow<11
45+
- pyarrow
4646
- pyreadstat
4747
- python-snappy
4848
- pyxlsb

ci/deps/actions-311.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ dependencies:
4242
- psycopg2
4343
- pymysql
4444
# - pytables>=3.8.0 # first version that supports 3.11
45-
- pyarrow<11
45+
- pyarrow
4646
- pyreadstat
4747
- python-snappy
4848
- pyxlsb

ci/deps/actions-38-downstream_compat.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ dependencies:
4040
- openpyxl
4141
- odfpy
4242
- psycopg2
43-
- pyarrow<11
43+
- pyarrow
4444
- pymysql
4545
- pyreadstat
4646
- pytables

ci/deps/actions-38.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ dependencies:
4040
- odfpy
4141
- pandas-gbq
4242
- psycopg2
43-
- pyarrow<11
43+
- pyarrow
4444
- pymysql
4545
- pyreadstat
4646
- pytables

ci/deps/actions-39.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ dependencies:
4141
- pandas-gbq
4242
- psycopg2
4343
- pymysql
44-
- pyarrow<11
44+
- pyarrow
4545
- pyreadstat
4646
- pytables
4747
- python-snappy

ci/deps/circle-38-arm64.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ dependencies:
4040
- odfpy
4141
- pandas-gbq
4242
- psycopg2
43-
- pyarrow<11
43+
- pyarrow
4444
- pymysql
4545
# Not provided on ARM
4646
#- pyreadstat

environment.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ dependencies:
4343
- odfpy
4444
- py
4545
- psycopg2
46-
- pyarrow<11
46+
- pyarrow
4747
- pymysql
4848
- pyreadstat
4949
- pytables

pandas/core/arrays/arrow/array.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,18 @@ def _from_sequence_of_strings(
275275
from pandas.core.tools.timedeltas import to_timedelta
276276

277277
scalars = to_timedelta(strings, errors="raise")
278+
if pa_type.unit != "ns":
279+
# GH51175: test_from_sequence_of_strings_pa_array
280+
# attempt to parse as int64 reflecting pyarrow's
281+
# duration to string casting behavior
282+
mask = isna(scalars)
283+
if not isinstance(strings, (pa.Array, pa.ChunkedArray)):
284+
strings = pa.array(strings, type=pa.string(), from_pandas=True)
285+
strings = pc.if_else(mask, None, strings)
286+
try:
287+
scalars = strings.cast(pa.int64())
288+
except pa.ArrowInvalid:
289+
pass
278290
elif pa.types.is_time(pa_type):
279291
from pandas.core.tools.times import to_time
280292

pandas/core/tools/timedeltas.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -240,7 +240,9 @@ def _convert_listlike(
240240
# returning arg (errors == "ignore"), and where the input is a
241241
# generator, we return a useful list-like instead of a
242242
# used-up generator
243-
arg = np.array(list(arg), dtype=object)
243+
if not hasattr(arg, "__array__"):
244+
arg = list(arg)
245+
arg = np.array(arg, dtype=object)
244246

245247
try:
246248
td64arr = sequence_to_td64ns(arg, unit=unit, errors=errors, copy=False)[0]

0 commit comments

Comments
 (0)