Transactions

[SystemParadox]

I think for many people a big barrier to adopting js-data on the server side is a lack of transactions support. This is actually one of the main reasons I couldn't use backbone or any other existing ORM, and ended up writing my own data layer... again (sound familiar?). I would really love to solve this once and for all with js-data.

The big problem with transactions on node is having to pass the transaction to everything explicitly because there are no thread-locals. Does anyone have any opinion on using continuation-local-storage for this?

Otherwise, I can't see how to implement this without either making JSData.DS transaction-aware or creating a new store instance (and all the resource objects) for every request.

Maybe creating multiple stores is actually necessary in order to avoid transactions sharing objects in memory that should be different. But then we've lost most of our caching. Immutable objects would probably help a lot here.

It would be good to know if anyone has any plans or thoughts on how this should work.

Thanks.

[jmdobry]

js-data-sql uses knex.js under the hood, which apparently has support for transactions. Would it be enough to modify the adapter methods in js-data-sql to use transactions as described here?

Nothing in the store is modified until adapter operations complete. I personally never use DS#save because I don't like the idea of dirtying the data in the store before the change has been committed to the database, so instead I used DS#update. If you were to always to use DS#update in conjunction with the knex.js transactions, it seems like you could avoid the problem you describe.

Disclaimer: I personally have never used transactions in Node

Also,

I think for many people a big barrier to adopting js-data on the server side is a lack of transactions support.

Only the js-data-sql adapter (except for js-data-redis, which already has transactions) works with persistence layers that support transactions, so to be more accurate:

barrier to adopting js-data + js-data-sql on the server side is a lack of transactions support

[SystemParadox]

Only the js-data-sql adapter (except for js-data-redis, which already has transactions) works with persistence layers that support transactions, so to be more accurate: barrier to adopting js-data + js-data-sql on the server side is a lack of transactions support

For me transactionless data stores just aren't an option ;)

I looked in js-data-redis to see how transactions were implemented there but I couldn't find anything! If you could point me in the right direction that would be great. It would seem sensible to keep these similar if possible.

In the very simple case, implementing transactions would look a bit like this:

var User = store.defineResource('user');

adapter.query.transaction(function (tx) {
    var user = {...};
    User.update(1, user, { transaction: tx });
});

(obviously we should come up with something nicer than adapter.query.transaction)

However, I really hate passing the transaction around everywhere like that. It's verbose, it's error prone, and my application logic should not know about transactions.

Adding an option that would cause an error to be thrown if anything tries to modify the database without a transaction would help, but it's still really horrible to work with.

knex also supports this kind of syntax, which is somewhat nicer:

knex.transaction(function(tx) {
    tx.insert(...);
});

The tx object provides the same interface as the usual database object, but does everything within the transaction. This is closer to how my current data-layer works. I'm not sure how this could be implemented in an ORM like js-data, where everything operates on global resource objects. Maybe the only options are either passing the transaction around everywhere or using continuation-local-storage?

[jmdobry]

Might be able to make this work by doing something like this:

var store = new JSData.DS();
var adapter = new DSSqlAdapter({
  client: 'mysql', // or "pg" or "sqlite3"
  connection: {
    host: '123.45.67.890',
    user: 'ubuntu',
    password: 'welcome1234'
  }
});

store.registerAdapter('sql', adapter, { default: true });

var User = store.defineResource('user');

adapter.query.transaction(function(trx) {
  // the adapter will use the transaction as the query builder
  return User.update(1, { foo: 'bar' }, { transaction: trx });
});

[SystemParadox]

Indeed, but passing around the transaction is just so horrible.

What I've ended up doing is creating a new DS object for every request so they are all isolated. Each request has its own unique "agent" object for accessing the database. Each request's agent is registered as the adapter for the request-specific store. When a transaction is started (using a similar transaction function as above), all calls to that agent are run within the transaction, without needing to pass the transaction to each call individually...

req.store = new JSData.DS();
req.agent = db.createAgent();
req.store.registerAdapter('sql', new AgentAdapter(req.agent), { default: true });

req.agent.transaction(function () {
    // Uses transaction automatically
    return req.store.update('user', 1, { foo: 'bar' });
});

The remaining issue for me is how to inject the updated data into the shared DS instance used for subscriptions. It would be nice to have some kind of deferred hook (like afterInject) on the transaction-store that would not be triggered until after the transaction is committed.

[techniq]

Fixed by #32