Add support for loading relations in findAll()

[jmdobry]

[techniq]

👍

I was about to submit an issue under the main js-data project when I came across this. It would be nice to grab a relation/property for all items in the result set, but in a single request. I guess something like this would work (but with n+1 queries, albeit asynchronously)

Post.findAll(params).then(posts => {
  return Promise.all(posts.map(post => Post.loadRelations(post, ['user']));
});

It would be better if Post.loadRelations could take in an array for the first parameter such as:

Post.findAll(params).then(posts => Post.loadRelations(posts, ['user']));

and then the adapter could smartly determine how the relation for each item should be retrieved. For example, js-data-sql would run the query:

select * from user where id in (1, 5, 9, ...)

while js-data-http could perform the request:

/api/v1/users?id=1&id=5&id=9&id=...

where 1,5,9,... would equal all the post.user_id values.

This might be best to discuss under js-data since it has farther reach affects than an individual adapter, but I thought I'd start here since this issue already existed.

Note, I don't have much experience with js-data thus far (I'm currently evaluating it for a new project, and looking to use it with a koa backend (instead of bookshelf.js) and an Angular frontend.

[jmdobry]

Yes, I've thought about a DS#loadRelationsForAll method that would make it more convenient to load relations for entire collections. Though, being able to do something like Post.findAll(params, { with: ['user'] }) would be pretty slick. This issue hasn't been resolved because I haven't had the time. But since you've shown interest, I can bump it higher up in my queue of things to do.

[techniq]

Both those methods sound reasonable as long as they wouldn't (or could be setup to not) run n+1 queries/requests. I know the js-data-http is all dependent on the server implementation, and from my understanding of the docs, would need to be handled in the queryTransform

[jmdobry]

An ideal:

• client: js-data + js-data-angular
• server: js-data + js-data-sql

In the browser:

// js-data-angular using an http adapter
// GET /post?published=true&with=user
Post.findAll({ published: true }, { params: { with: ['user'] } }).then(function (posts) {
  // posts[0].user;
  // posts[1].user;
  // etc.
});

On the server:

// using the js-data-sql adapter
app.get('/post', function (req, res) {
  var options = {};
  if (req.query.with) {
    options.with = req.query.with;
  }
  delete req.query.with;
  Post.findAll(req.query, options).then(function (posts) {
    // posts[0].user;
    // posts[1].user;
    // etc.
    res.status(200).json(posts).end();
  }).catch(function () {
    res.status(500).end();
  });
});

[techniq]

Looks exactly what I'm looking for :). The loadRelationsForAll could still be handy when you have a collection already and want to fill in those relations.

Also, how would/could this work for "grandchildren" relations, such as post.user.profile?

[techniq]

Awesome, thanks for implementing this so quickly. I just got around to taking a look at it (I wanted to look at the produced queries) and it looks good. Btw, I setup a mysql docker container to run the tests. My notes are here if you find them helpful - https://gist.github.com/techniq/12016da47ebb9de4aba1

[jmdobry]

Thanks. This should support (theoretically) infinitely nested relations, e.g.

Post.findAll(null, { with: ['user', 'comment', 'user.post', 'comment.user', 'comment.post', 'user.post.user', 'user.post.user.post', 'user.post.user.post.user' ] })

Though why you would do user.post.user.post.user I don't know.

[techniq]

I saw that. Thanks again.

I can log this as a separate issue, but I couldn't find anything in the docs regarding this: Can you setting a column mapping for your model for cases where you want the property name different from the database column name? For example, the database column might be active_ind but I want the property to be isActive?

Looking at bookshelf.js, it looks like this is mostly handled by the format and parse methods - http://bookshelfjs.org/#Model-format

[jmdobry]

Neither js-data nor js-data-sql have any field mapping capabilities.

[techniq]

Ok. Any chance for any hook points to have control of this?

[jmdobry]

There are several places in js-data where you can do that mapping yourself. beforeCreateInstance, afterCreateInstance and if you're injecting into the store, beforeInject

I see the value in js-data having a field mapping capability, though I'm not sure if it should be a js-data thing or an adapter thing. Different adapters may very well have different mapping needs.

[techniq]

Thanks. I don't have a direct need now, but would for another project that has an existing database. I'm still determining whether to use Bookshelf.js on the backend and js-data on the frontend, or using js-data for both.

[glebmachine]

+1