Skip to content

SQLiteDatabaseClient schema methods #283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 7, 2022
Merged

Conversation

mbostock
Copy link
Member

@mbostock mbostock commented Apr 26, 2022

This implements the describeTables and describeColumns methods for SQLite.

Fixes #288.

@mbostock mbostock requested a review from visnup April 27, 2022 22:26
@mbostock
Copy link
Member Author

Asking @visnup for another opinion here on how to pass through the raw database type.

@visnup
Copy link
Member

visnup commented Apr 27, 2022

I like what's here plus a single key called maybe databaseType or vendorType that includes the vendor-specific name for the type. And I'd want to re-work the schema object coming from data-connector to match this too, replacing the int32: true, float32: true entries with something like databaseType: "int32"

The only other thing worth considering is calling out bigint and date since the original JSON schema specification on ajv didn't provide for either. These were {type: "string", [bigint|date]: true} since over the wire these values were serialized as strings.

It seems since then there's JSON Type Definition which seems more promising?? And this more recent JSON Schema reference seems to document types for datetime.

@mbostock mbostock force-pushed the mbostock/sqlite-schema branch from 3357b35 to 5d56bed Compare May 11, 2022 21:17
@mbostock
Copy link
Member Author

I’ve populated the databaseType field. Here’s what else I would like to do, but does not appear possible with our SQLite implementation, sql.js:

  • Return a query result schema on db.query. The exec function doesn’t return any schema information other than the column names, so we don’t know what the types of the returned columns are. We’re populating the columns field but that’s it.
  • Coerce date strings to Date instances. SQLite has DATE and DATETIME types, but the returned query results for these types are strings. Since we lack type information on the query results (previous point), we can’t know which columns are dates and then coerce them. We could map the DATE and DATETIME types to strings on describe, but… I dunno if that’s better.

We could probably fork sql.js to fix these issues.

@annie
Copy link
Contributor

annie commented May 24, 2022

since this is a Table cell blocker, i think we should merge this once we've finalized the DatabaseClient schema API (based on Wiltse's PR) and updated this PR to match that specification (i've got a draft PR in the wings to make those updates).

it seems reasonable to me to address the query results schema and date string issues as fast follows, if that sounds good? i can create separate issues for those!

@mbostock mbostock force-pushed the mbostock/sqlite-schema branch from 1cf6d2e to a93d445 Compare June 7, 2022 01:40
@mbostock mbostock force-pushed the mbostock/sqlite-schema branch from a93d445 to c8599fb Compare June 7, 2022 01:41
@mbostock mbostock changed the title SQLiteDatabaseClient.schema SQLiteDatabaseClient schema methods Jun 7, 2022
@mbostock mbostock requested review from wiltsecarpenter and annie June 7, 2022 01:46
@mbostock mbostock merged commit 205f277 into main Jun 7, 2022
@mbostock mbostock deleted the mbostock/sqlite-schema branch June 7, 2022 02:41
@mbostock
Copy link
Member Author

mbostock commented Jun 7, 2022

FYI, the describeColumns query had a syntax error which I fixed in d381b56.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants