Amalgamated release builds?

[bnoordhuis]

Amalgamated = bundling all source files into a single source file, for easy distribution and building. Well-known example: sqlite

The split into quickjs.c, quickjs-libc.c, libunicode.c, etc., isn't really relevant or interesting to downstream users, just a minor nuisance when trying to integrate it into your build.

One way amalgamation could work is:

1. Bundle quickjs.c (and quickjs.h) etc. with qjs.c so you only have to cc -o qjs qjc.c

2. Ditto for qjs.c

3. We don't ship run-test262.c, unicode_gen.c, etc.

4. Amalgamated quickjs.c is a bundle of quickjs.c, quickjs-libc.c, etc. with a define to enable/disable quickjs-libc (default on or off?)

5. We must ship quickjs.h of course but do we also roll a separate-but-identical copy into the amalgamated quickjs.c?

[saghul]

I assume the upside here is simplified integration in other (than CMake, that is) build systems, right?

+1 to that!

Now, and I assume you might have imagined this one coming: why not split quickjs.c into some manageable units so edtiors can actually show useful info without choking on the huge file? :-P

[nickva]

Just my $.02 as an upstream user of QuickJS for the JS engine in Apache CouchDB.

From QuickJS we only use libquickjs.a (libquickjs.lto.a actually),quickjs.h and qjsc to compile bytecode. It's a benefit that we can just pick those out and not have to worry about qjs or quickjs-libc for instance. A single .c file compiling to libquickjs{.lto}.a plus the include file would work for us, I think.

In general, we like QuickJS (vs other engines) because of being able to exclude various features: promises, proxy, typed arrays, and a few others. I even wanted an option to force excluding atomics (linking in pthreads) bellard/quickjs#300, but that's still pending ...

Now, and I assume you might have imagined this one coming: why not split quickjs.c into some manageable units so edtiors can actually show useful info without choking on the huge file?

Not a bad idea; I had noticed some other QuickJS forks did that. But would that make it more difficult to merge patches back and forth?

Thanks for maintaining QuickJS-NG I am debating switching to it eventually as I see a lot of awesome features added here almost daily!

[TooTallNate]

I am also not using quickjs-libc. IIRC, last time I tried, there was some incompatible features being used that are not available in my environment.

[saghul]

Just my $.02 as an upstream user of QuickJS for the JS engine in Apache CouchDB.

Good to see you here! I noticed CouchDB recently added support for QuickJS, nice!

From QuickJS we only use libquickjs.a (libquickjs.lto.a actually),quickjs.h and qjsc to compile bytecode. It's a benefit that we can just pick those out and not have to worry about qjs or quickjs-libc for instance. A single .c file compiling to libquickjs{.lto}.a plus the include file would work for us, I think.

Yeah I think that would work.

In general, we like QuickJS (vs other engines) because of being able to exclude various features: promises, proxy, typed arrays, and a few others. I even wanted an option to force excluding atomics (linking in pthreads) bellard/quickjs#300, but that's still pending

Why, is it because of pthreads? Note that unless atomics are used the core QuickJS no longer needs pthreads, as we fixed how class IDs are allocated.

Not a bad idea; I had noticed some other QuickJS forks did that. But would that make it more difficult to merge patches back and forth?

Yes indeed. My comment was partially in jest :-) We did have that discussion in the past and that argument tilts it. FWIW, bellard/quickjs is stalled again, so at some point that argument will become moot I think.

Thanks for maintaining QuickJS-NG I am debating switching to it eventually as I see a lot of awesome features added here almost daily!

Sweet! We just hit the 1 year mark and still going strong. The project does go beyond simple code changes: https://github.com/quickjs-ng/quickjs/pull/685/files#diff-470544b48379eb103b88603d9588c367fc53ff973a3aa25fe7af7dfda8a55aee

[saghul]

I am also not using quickjs-libc. IIRC, last time I tried, there was some incompatible features being used that are not available in my environment.

Makes sense!

[bnoordhuis]

I assume the upside here is simplified integration in other (than CMake, that is) build systems, right?

Yep.

why not split quickjs.c into some manageable units so edtiors can actually show useful info without choking on the huge file?

The One True Editor doesn't have the slightest problem with quickjs.c so for me this is a non-issue :-)

there was some incompatible features being used that are not available in my environment

@TooTallNate do you remember what they were? For quickjs-libc, ifdef'ing out code that doesn't work in environment X is on the table, if it doesn't turn into super brittle spaghetti code.

[nickva]

Why, is it because of pthreads? Note that unless atomics are used the core QuickJS no longer needs pthreads, as we fixed how class IDs are allocated.

It's because of wanting to ensure no workers or any threads are available in that environment. Not linking to pthreads also has a benefit of a slightly smaller executables, and most of all, on Windows, since we still use msys2, it doesn't need distributing the winpthreads additional library along side our image.

The general idea is guess is in an embedded environment it's a nice feature to have the ability to reduce features easily (no workers, no async stuff, etc).

[saghul]

Workers are part of libc, so if you just use the interpreter then there would be no worker or any other std or os APIs. Or do you want those?

I think the pthreads part is a bit tangential to the issue of amalgamation, can you please open a new issue?

[nickva]

I think the pthreads part is a bit tangential to the issue of amalgamation, can you please open a new issue?

The pr for the upstream seem to do what we want: bellard/quickjs#300

Ah yes, it's workers for the libc and CONFIG_ATOMICS for quickjs.c. For a top level toggle name I went with CONFIG_NO_WORKER to avoid having two separate toggles.

Yeah it's tangential to the issue, I was just trying to show an example how we use various features in QuickJS and how we prioritize the ability to toggle off features. The amalgamated release could make it harder or easier to do that depending how it's done. SQLite has a bunch of defines to turn things on and off and like Ben mentioned that could work, provided it wouldn't look too spaghettified :-)

[TooTallNate]

do you remember what they were?

No dlfcn.h, so no dynamic library stuffs.

[bnoordhuis]

That should be easy to fix. We already have a stub module loader for WASI. Want to send a PR?

[saghul]

It's because of wanting to ensure no workers or any threads are available in that environment. Not linking to pthreads also has a benefit of a slightly smaller executables, and most of all, on Windows, since we still use msys2, it doesn't need distributing the winpthreads additional library along side our image.

FYI, we don't use winpthreads, we use native Windows APIs for threading.

[TooTallNate]

Also FWIW, I would not mind sharing some generic API implementations. I am interested in adding Worker, for example. In my case, a pthreads-compatible API is available, so the implementation in quickjs-libc would probably work.

I do wonder about where the line is drawn though. If threads are being used for Worker, then next comes a threadpool, and next after that comes libuv 😄 (which I would also love to have libuv, but I've had a hard time getting that working in my environment). At that point, it's already a mini-runtime.

It would make sense to me if quickjs was only the core JS engine, but there could exist other optional libraries offered alongside / in extension to the core quickjs. Typing that out, I guess that's already how -libc is separated. I guess my suggestion is to break apart -libc into more self-contained / defined libraries? Idk… food for thought.

[nickva]

FYI, we don't use winpthreads, we use native Windows APIs for threading.

Thanks, that's great to hear! And presumably it just as easy to stub out its usage with a user level config similar to how bellard/quickjs#300 works? Ideally, if we're not using atomics, or workers, it would be nice to keep everything related to it out of the build.