Improve workflow when not using Env vars

[apfelbox]

Hi 👋

we are currently migrating to Flex and have encountered a small DX issue. I currently don't have a nice proposal, but just wanted to point out that there might be an area to improve.

If you are not using env variables, the EnvConfiguration command currently just bails out:

flex/src/Configurator/EnvConfigurator.php

Lines 35 to 38 in 9f2d70e

	private function configureEnvDist(Recipe $recipe, $vars)
	{
	$distenv = getcwd().'/.env.dist';
	if (!is_file($distenv) || $this->isFileMarked($recipe, $distenv)) {

In this case though, the information about which configuration parameters (in form of env variables or env(VAR): value in your parameters.yaml) the bundle expects is just lost:

I am not sure about how to improve the situation except printing a message like:

Hey, this bundle needs some env variables. This is what we would have added to your .env.dist file, but since you don't have one, you have to add these parameters yourself: [... print manifest.json["env"] ...]

So currently you have to hunt down the recipe manually in Github (potentially search the correct version), look at the manifest.json and proceed from there. This might be improved by just printing the already available info to the CLI in the installation process?

What do you think?

[Pierstoval]

I think it's just about a few lines of code in the EnvConfigurator, so I'd say: submit a PR, it sounds nice 👍

[fabpot]

I think that not using a .env.dist is not supported. We make it so that it's possible, but that's definitely not a use case I want to support. If you don't have a .env.dist file, for whatever reasons, you are on your own. Keep in mind that Flex is flexible BUT opinionated.

[Pierstoval]

As it's not supported, I think it's a good reason to show a warning to the user telling him/her to either create the file, or be warned that env vars defined by recipes won't be usable if he/she does not set it.

[apfelbox]

@fabpot our issues with parameters as env vars is that we just have to many.
We need to duplicate them around multiple entry points in nginx, we need to copy them in the deploy script and every bin/console call is prepended with around 10 env vars. That is not very comfortable and quite error prone.

Is there a better way to do this? I am very thankful for every tip, as I really have no idea on how to improve this.

What is the performance impact of DotEnv in production? It will be slower than compiled-in parameters, but is it significant? One file access and some parsing?

[javiereguiluz]

@apfelbox I agree that repeating each env var in so many places (.env, .env.dist, phpunit.xml, console commands, your web server, etc.) is not nice. We are at least trying to remove the phpunit repetition (see #251).

[fabpot]

I don't have a definitive answer but some hints:

• First, not everything should be an env vars. I'm trying to reduce the number of env vars to the strict minimum. Most configs do not need to be env vars.

• With Symfony 3.4, you can store most things in a file that is exposed as 1 env var. Can be nice.

• Performance is not that significant, not using DotEnv in prod is more like a "good practice" than anything else. If you want to use it in prod, I don't think it will kill your server :)

[Pierstoval]

Once env vars are set in the virtual host of your nginx / apache instance, it's not needed to repeat them.

Same for the command-line: of course you will have to duplicate the same config, but you can put it in either /etc/environment or /etc/profile or in /etc/profile.d/your_file.sh.

[apfelbox]

@Pierstoval

Once env vars are set in the virtual host of your nginx / apache instance, it's not needed to repeat them.

Hmm, can't really confirm that:

#fastcgi_param TEST "test123";     # <- does not work
location /app.php {
    include php-fpm.conf;
    fastcgi_param TEST "test123";  # <- does work

    fastcgi_pass php7.2-testproject;
}

So I either have to repeat them for every entry point or have to create a new include file for every project.

[...]you can put it in either /etc/environment or /etc/profile or in /etc/profile.d/your_file.sh.

That is true, doesn't work however in shared hosting environments. We have quite a few projects on the same server (they are too small for an own VM) so I would need to manually source different files then. Also calling bin/console in a script becomes "difficult" too.

@fabpot thank you for your input.

Most configs do not need to be env vars.

How do you define the config, that doesn't need to be env vars, but differs between the servers? Do you still have a parameters.yaml around?

With Symfony 3.4, you can store most things in a file that is exposed as 1 env var. Can be nice.

Exposing it as 1 file is quite nice (by using the env() prefixes as described here) but this version leads to having one huge app.config blob with an arbitrary structure. By using different, specific app.config.db_url (etc...) I have a small sense of security that these parameters must be defined.

Is it possible to destructure env vars, something like (don't mind the concrete syntax please):

services.yaml

parameters:
    app.config.db: `%env(json:file:CONFIG_FILE).db%`
    app.config.secret: `%env(json:file:CONFIG_FILE).secret%`

?

That could be a nice integration to combine the best of both worlds.
(It would need to error early if the destructuring fails)

[Pierstoval]

How do you define the config, that doesn't need to be env vars, but differs between the servers? Do you still have a parameters.yaml around?

You can still use one if you need. It was previously handled by incenteev/composer-parameter-handler, and you can still use it even with Flex if you need fils.

Exposing it as 1 file is quite nice (by using the env() prefixes as described here) but this version leads to having one huge app.config blob with an arbitrary structure. By using different, specific app.config.db_url (etc...) I have a small sense of security that these parameters must be defined.

This is where you can take advantage from extensions or compiler passes: you can retrieve this array and pass it through a Configuration class, or make manual checks, to validate the config structure at compile time, or store this configuration in a service that will check it at runtime, possibly making a checksum of it before, to know if it's possible to retrieve it from cache and reduce the amount of env processing at runtime… Plenty solutions, actually.

At compile time, you can even split it and insert it in the container afterwards with something like foreach ($config as $key => $value) { $container->setParameter('config.'.$key, $value); }.

Hope this helps

[nicolas-grekas]

Closing as this improved a lot recently, with symfony/recipes#501 as the last step.