[FrameworkBundle] forced use of "fake" environment

[akomm]

Why are this the only options:

// Load cached env vars if the .env.local.php file exists
// Run "composer dump-env prod" to create it (requires symfony/flex >=1.2)
if (is_array($env = @include dirname(__DIR__).'/.env.local.php')) {
    $_SERVER += $env;
    $_ENV += $env;
} elseif (!class_exists(Dotenv::class)) {
    throw new RuntimeException('Please run "composer require symfony/dotenv" to load the ".env" files configuring the application.');
} else {
    // load all the .env files
    (new Dotenv())->loadEnv(dirname(__DIR__).'/.env');
}

Why not for example this?

if (!($_SERVER['APP_ENV'] ?? $_ENV['APP_ENV'] ?? null)) {
    if (is_array($env = @include dirname(__DIR__).'/.env.local.php')) {
        $_SERVER += $env;
        $_ENV += $env;
    } elseif (!class_exists(Dotenv::class)) {
        throw new RuntimeException('Please run "composer require symfony/dotenv" to load the ".env" files configuring the application.');
    } else {
        // load all the .env files
        (new Dotenv())->loadEnv(dirname(__DIR__).'/.env');
    }
}

Is there a reason not allowing "native" environment to be used I missed? I know, this is all not optimized for docker, because docker is not the only way of doing things, but:

Docker is not the problem! Docker only makes the design flaw visible. The same applies, when you run without docker on a "user per app" basis and the user has environment set to run the app and there might be some task executed that has to run with the same user and share the environment. The environment is here isolated to the user the app runs on. No collisions at all. Now if you do not have this setup, the above would still work with DotEnv or dumped .env.local.php.

My strategy using docker would be then:

• remove symfony/dotenv
• use .env.local as .env before the flex change
• use .env as .env.dist before the flex change
• instead of "overriding" only some vars from .env in .env.local, have them all defined, like before in .env

This would allow to load native environment als also have a single environment "object". But would also allow the other way of usage you intended for also good reasons.

[nicolas-grekas]

Why do you think native env is not supported? It's perfectly supported actually. Dotenv is used to load default env vars from .env* files sure, but that doesn't prevent using native env at all.
What you propose creates all sort of DX issues people are not comfortable dealing with.

If you want to completely ignore .env files, you can create a .env.local.php file that contains <?php return [];. But it should be opt-in so that things work as expected by default.

[akomm]

@nicolas-grekas

Dotenv does not prevent me using native env the same way google does not prevent me from doing so. This is an issue regarding the FrameworkBundle recipe. This recipe does prevent using native env. I posted code above which should explain why it does so. My fix gives you the ability to use native env. It does not force it on you.

Could you please explain me how this ability introduces DX issues? :)

[nicolas-grekas]

eg:

• Improve workflow when not using Env vars flex#287
• [DX] Flaws in the way environment variable are used ?  symfony#27962
• Reading .env in PHPUnit versus env variables in phpunit.xml.dist flex#251
• --env no longer equivalent to APP_ENV #497
• Bad error message if you set APP_EN=prod and pass --env=prod #500
• see also [WIP] Updating docs for a few more .env changes symfony-docs#10793

I still fail to understand what is not possible currently that you want to allow.

[akomm]

Okay, maybe we think about different things, when we talk about "not allowing". I mean "out of the box" it fails to run, when you provide native env, because it will either throw because you do not have DotEnv or throw because you do not have .env file.

Sure, I can start changing the bootstrap so that it works.

Also t he issues you post there are not becaue you have the ability to use native env.

[nicolas-grekas]

OK, so yes: the defaults are opinionated - that's what recipes are in the first place, opinions about what good defaults are. Here the defaults are on purpose: we want .env files to be loaded by default, so that Dotenv is in the default composer.json and missing .env files are reported to reduce the time to spot mistakes. Defaults don't fit 100% of the use cases. You have two choices here: you can create a .env.local.php file that contains <?php return []; - or you can change the bootstrap file as you noted - that's perfectly fine. I wouldn't change the default - we spent many months settling on them (see list above) and we reached a generic enough consensus to me.

[akomm]

It will still have this default behavior. It will tell you when you do not provide native env APP_ENV, that you have no env provided. Is this such a an issue?

[nicolas-grekas]

I'd suggest to add symfony/dotenv as a requirement, and to check the dump-env command described in symfony/symfony-docs#10793 (unreleased yet).
The if you suggest to add existed before, and removing it is precisely what unlocks some nasty DX issues (from above linked list).

[akomm]

This is the only change, you can talk of, which is similar to my:

3e471cb#diff-998ae036ea7e12dc983860bfbc56ca89

My proposal has a slightly different behavior, than what you'v removed in the diff above.
It should already solve most of the DX you have posted. The one that is left is the --env CLI option DX issue, which has it roots elsewhere.

The issue is the twofoldness of APP_ENV. It is used both as:

1. an environment variable.
2. an identifier for a set of environment variables.

The idea behind the config/bootstrap.php is to serve as a general and consistent environment setup to be used from different contexts (CLI, WEB, TEST).

The supposed DX issue:

• My proposal introduces an environment check on $_SERVER to recognize natively passed environment.
• The console front controller (bin) uses config/bootstrap.php. But it alters $_SERVER's APP_ENV beforehand, when the --env option has been passed.
• The altered config/bootstrap.php would now assume, that native environment has been passed, because APP_ENV is set in $_SERVER

Actual mistake(s):

• config/bootstrap.php is supposed to be used as a common environment loader, but console hacks into this concept, altering it beforehand
• The option --env has an implication: it does not set an arbitrary environment variable to an arbitrary value, but it sets the value of a specific environment variable, namely APP_ENV, to a given value. This implies its usage as identifier (see 2. above) for a set of variables. Therefore it implies some sort of source to load this set of variables from. This implies, that you do not use native environment, when you use --env. Therefore the console front controller does not handle this case correctly. It does handle the option by just setting the APP_ENV variable (see 1. above).

That causes the DX issue.
I understand, when you do not want to change this forever and want to come to a certain point. But maybe at least realize an issue to maybe do it better at least long-term? :)

[Pierstoval]

--env is still here for DX but nothing prevents you from doing APP_ENV=custom_env php bin/console (on *nix) either.

The kernel environment has been here since Symfony 2.0 and is used in tons of projects that do things based on env name if ($kernelEnv === 'dev') { /* do something */ }.

We can't consider the kernel env as an "arbitrary value". If you consider it as an identifier, well, I think it's a success in terms of DX, because this special var is an identifier: the application environment in which it is executed. Therefore, config loading and setup can be different depending on env vars, and IMO that's the best option.

I don't know about how the maintainers will see this, but I think the current behavior of having APP_ENV as an identifier is precisely the reason why we use it. All other config options may depend on it. Hence the config/packages/{env}/... directory structure.

[akomm]

--env is still here for DX but nothing prevents you from doing APP_ENV=custom_env php bin/console (on *nix) either.

You clearly did neither test this, nor properly read the discussion, as this is exactly what fails on bootstrap.

The kernel environment has been here since Symfony 2.0 and is used in tons of projects that do things based on env name if ($kernelEnv === 'dev') { /* do something */ }.

Many things have been there since Symfony 2.0, and many of them have been changed. We would not have flex or env at all, if we would speak about "since when something is there". Based on this arguing you should never have introduced flex, .env and many other things at all. Can you give some actual arguments?

We can't consider the kernel env as an "arbitrary value".

Nobody is considering kernel env an arbitrary value. Please read properly...

If you consider it as an identifier, well, I think it's a success in terms of DX, because this special var is an identifier: the application environment in which it is executed.

Nobody considered anything, its a fact of how twofold it is used. You just repeat that "it is what it is" and that "therefore it is good". I do not understand your arguing...

Therefore, config loading and setup can be different depending on env vars, and IMO that's the best option.

Env var or env varS? You just tripped over my argument, which you obviously did not understand.

I don't know about how the maintainers will see this, but I think the current behavior of having APP_ENV as an identifier is precisely the reason why we use it. All other config options may depend on it. Hence the config/packages/{env}/... directory structure.

Would be nice to have something you could reason about, but I could not find any additions in your post. :(

[nicolas-grekas]

I'm still on the side that the current state is the best compromise. --env is a first-class supported way to configure the Symfony env, and there need to be some precedence rules with APP_ENV.
See symfony/symfony#28984 for some background on this.

[akomm]

Consider this in context of my recent points.

Nothing speaks against --env it is just, that its intent is to set an identifier to load a set of variables. So it is used as identifier not a "set a single variable". But the CLI front controller currently sets only a single variable and that is in my opinion the core mistake, which prevents to do it "right" and also allow native env like I proposed, without producing the given DX issues.

My proposal would be BC to expected behavior, except when some "consumer" of the bootstrap starts to adjust env by itself (CLI front in this case). In my opinion, when config/bootstrap.php is "reused" from different context to bootstrap env, then there should be a contract, that env is not fiddled with before its "required". And that is what console does. And only that produces the DX issues currently.

[nicolas-grekas]

the CLI front controller currently sets only a single variable and that is in my opinion the core mistake

so, how would you "fix" that?

[akomm]

My preferred way would be to not alter env before require bootstrap and that would require tapping into the weak contract currently between config/bootstrap.php and its current consumers.

However, I think I found a solution, which is BC and does not require alteration of consumers and allows them to override APP_ENV & APP_DEBUG safely.

if (is_array($env = @include dirname(__DIR__).'/.env.local.php')) {
    $_SERVER += $env;
    $_ENV += $env;
} elseif (class_exists(Dotenv::class)) {
    (new Dotenv())->loadEnv(dirname(__DIR__).'/.env');
} elseif (!($_SERVER['APP_ENV'] ?? $_ENV['APP_ENV'] ?? null)) {
    throw new RuntimeException('Please run "composer require symfony/dotenv" to load the ".env" files configuring the application. Or provide a native APP_ENV env var');
}

The exception text slightly modified.

I compared behavior current and altered bootstrap and both produce same results, also including exceptions to catch errors. Did I miss something?