Skip to content

fix(demo/trino-taxi-data): Use new listener service #254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 20 additions & 17 deletions demos/trino-taxi-data/create-table-in-trino.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,29 +59,32 @@ data:

run_query(connection, "CREATE SCHEMA IF NOT EXISTS hive.demo WITH (location = 's3a://demo/')")
run_query(connection, """
CREATE TABLE IF NOT EXISTS hive.demo.ny_taxi_data_raw (
VendorID BIGINT,
tpep_pickup_datetime TIMESTAMP,
tpep_dropoff_datetime TIMESTAMP,
passenger_count DOUBLE,
trip_distance DOUBLE,
payment_type BIGINT,
Fare_amount DOUBLE,
Tip_amount DOUBLE,
Total_amount DOUBLE
) WITH (
external_location = 's3a://demo/ny-taxi-data/raw/',
format = 'parquet'
)
CREATE TABLE IF NOT EXISTS hive.demo.ny_taxi_data_raw (
VendorID BIGINT,
tpep_pickup_datetime TIMESTAMP,
tpep_dropoff_datetime TIMESTAMP,
passenger_count DOUBLE,
trip_distance DOUBLE,
payment_type BIGINT,
Fare_amount DOUBLE,
Tip_amount DOUBLE,
Total_amount DOUBLE
) WITH (
external_location = 's3a://demo/ny-taxi-data/raw/',
format = 'parquet'
)
""")

loaded_rows = run_query(connection, "SELECT COUNT(*) FROM hive.demo.ny_taxi_data_raw")[0][0]
print(f"Loaded {loaded_rows} rows")
assert loaded_rows > 0

print("Analyzing table ny_taxi_data_raw")
analyze_rows = run_query(connection, """ANALYZE hive.demo.ny_taxi_data_raw""")[0][0]
assert analyze_rows == loaded_rows
# Disabling ANALYZE for now, as it seems to fail when run twice (which can happen if Trino OOMs)
# See issue: https://github.com/trinodb/trino/issues/26214
# print("Analyzing table ny_taxi_data_raw")
# analyze_rows = run_query(connection, """ANALYZE hive.demo.ny_taxi_data_raw""")[0][0]
# assert analyze_rows == loaded_rows

stats = run_query(connection, """show stats for hive.demo.ny_taxi_data_raw""")
print("Produced the following stats:")
print(*stats, sep="\n")
Expand Down
2 changes: 1 addition & 1 deletion demos/trino-taxi-data/setup-superset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ data:
import logging
import requests

base_url = "http://superset-node-default:8088" # For local testing / developing replace it, afterwards change back to http://superset-node-default:8088
base_url = "http://superset-node:8088" # You can change this while running locally, but remember not to commit the change.
superset_username = open("/superset-credentials/adminUser.username").read()
superset_password = open("/superset-credentials/adminUser.password").read()
trino_username = "admin"
Expand Down
Binary file modified demos/trino-taxi-data/superset-assets.zip
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ This demo should not be run alongside other demos.
To run this demo, your system needs at least:

* 10 {k8s-cpu}[cpu units] (core/hyperthread)
* 32GiB memory
* 34GiB memory
* 35GiB disk storage

== Overview
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/demos/pages/trino-taxi-data.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ This demo should not be run alongside other demos.
To run this demo, your system needs at least:

* 7 {k8s-cpu}[cpu units] (core/hyperthread)
* 16GiB memory
* 18GiB memory
* 28GiB disk storage

== Overview
Expand Down
2 changes: 1 addition & 1 deletion stacks/trino-superset-s3/trino.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ spec:
min: "1"
max: "4"
memory:
limit: '4Gi'
limit: '6Gi'
roleGroups:
default:
replicas: 1
Expand Down