Hanami 2.0: zero configuration/settings policy

[jodosha]

Goal

Hanami 2.0 should be simple for newcomers (or people that upgrade).

That implies that reading code under config/ directory of a newly generated application shouldn’t be overwhelming.
The framework must define defaults for configuration/settings, and just work . Users that wants to override default choices can add configuration/settings for the relevant runtime environment (development, testing, etc…).

Example

Let’s take an example to explain the principle and its proposed implementation: logging defaults.

I would suggest set the defaults for the logger like this:

• level: debug
• stream: $stdout

---

In the current alpha2, we use settings (config/settings.rb) to define the value of those defaults.
Then we manually assign the values from settings into config/application.rb.

This setup introduces useless code to be generated us and digested by the final user.

Proposal

Step 1

I would assign default values to Hanami::Application::Settings, to be inherited by MyApp::Settings:

    class Hanami::Application::Settings
      def self.inherited(base)
        # ...
        base.class_eval do
          setting :logger do
            setting :level, default: :debug
            setting :stream, default: $stdout
          end
        end
      end
    end

When MyApp::Settings inherits from Hanami::Application::Settings it would get the defaults that we define at the framework level.

Step 2

When Hanami application loads the settings, then we can assign their values to the application configuration.

class Hanami::Application
  def self.init
    # ...
    load_settings
    assign_settings # NEW METHOD
  end

  private

  def self.assign_settings
    config.logger = settings.logger
  end
end

Step 3

With this change, we can remove from the application template, any code that configures the logger under config/ directory.

In case final users would override defaults, they could do it in a few ways:

• Add the relative ENV var in .env.* files. E.g. `HANAMI_LOGGER__LEVEL=“info”
• Add the relative lines of codes in config/application.rb. E.g. config.logger.stream = "#{ENV['DEPLOY_PATH']}/log/production.log"

Assumptions

• The way to override an existing setting value would be via ENV var (in .env.*) files.
• The way to add a new setting would be via config/settings.rb and optionally via ENV vars.
• The way to override an existing configuration that cannot be done via ENV, or the behavior differs between environments, would be via config/application.rb.

Required Changes

dry-configurable powers application settings.

• Make dry-configurable, be able to be assigned and return nested settings.
• Make dry-configurable, be able to work with ENV variables to assign nested setting values.

[timriley]

Fully on board with this idea, @jodosha, and I like your guiding principal of ensuring config/ is as simple as possible. Thanks also for writing it up so clearly!

I had thought about this particular logging setup a while ago too, and actually have a card for it at the top of our 2.0 Trello board. However, that card was concerned more about the logging set up in particular, rather than a generalisable approach like you’ve outlined above.

And it’s true we’ll want something to handle more than just logging, because we’d want e.g. the the database URL also to be be a zero-config/zero-boilerplate thing, if possible.

What I think would be useful to clarify here, however, is whether you think all Hanami framework configuration should be settable via this extra assign_settings hook, or whether it might just be a small subset of things? I wonder if thinking about the full range of settings we’d want to support here might be the best way to discover the most helpful approach?

If we did go down this path, I think a good way to start would be to have Hanami::Application::Settings take care of populating a configurable object from the ENV rather than building it straight into dry-configurable. This way it starts off internal to Hanami and gives us the greatest control in the early days. If it works well, however, I’d be good for us to consider pushing it down into part of dry-configurable.

All of this said, however, one thing I did have in mind when I built the application settings (i.e. config/settings.rb) is that they would belong to the user, rather than the framework, so this proposed change does change that equation a little, and I think we’d want to think about whether we’re happy with the flow-on effects of that — e.g. it’s reserving some names that the user could no longer use for their own domain-specific settings, etc.

Another, perhaps more primitive, option here could be to have Hanami::Configuration or a related component look directly into the ENV for some well-known vars and use those as framework default values, if they’re set.

Anyway, overall this is definitely a problem we need to solve. Definitely keen to work with you to find a good way

[jodosha]

@timriley Yes you’re right: this is more about a general approach, rather than just logging, which served as example.

Regarding the list of configurations that we may need, we don’t have a full list at this time. What I’m looking for is to implement a solution, which can be used by future features like database integration, assets setup, etc…

@timriley

All of this said, however, one thing I did have in mind when I built the application settings (i.e. config/settings.rb ) is that they would belong to the user , rather than the framework

Right, I’ve got confused by the logger settings in the application template.
This is good news: we can set default values at the application configuration level.
That also implies that we don’t need assign_settings anymore.

---

Recap

Please let me know if this makes sense for you:

Application settings

• For final user needs (custom settings)
• Located under config/settings.rb
• Code generated with zero settings
• No defaults defined by Hanami
• Works with application defined types (lib/my_app/types.rb)
• It’s able to handle (nested) settings (e.g. setting :foo { setting :bar })
• It’s able to load (nested) ENV variables and automatically assign them to the corresponding value (e.g. foo.bar via HANAMI_FOO__BAR)
• It supports multiple environments by loading the corresponding .env.* files.

Application configuration

• For framework configuration
• Located under config/application.rb
• Code generated with zero or few settings
• It has defaults defined by Hanami per environment. When MyApp::Application inherits from Hanami::Application, we assign Hanami defaults.
• It allows final users to override defaults (see examples below)
• It supports multiple environments via the config.environment API

Example: Override defaults via hardcoded value

# config/application.rb

module MyApp
  class Application < Hanami::Application
    config.logger.stream = "/dev/null"
  end
end

Example: Override defaults via ENV var

# config/application.rb

module MyApp
  class Application < Hanami::Application
    config.logger.stream = ENV["LOGGER_FILE"] # custom env var name
  end
end

Example: Override defaults via application setting

# .env.test
HANAMI_LOGGER__STREAM="/dev/null"

# config/settings.rb

module MyApp
  class Settings < Hanami::Application::Settings
    setting :logger do
      setting :stream, constructor: Types::String
    end
  end
end

# config/application.rb

module MyApp
  class Application < Hanami::Application
    config.logger.stream = settings.logger.stream
  end
end

Example: Work with different environments

# config/application.rb

module MyApp
  class Application < Hanami::Application
    environment(:test) do
      config.logger.stream = "/dev/null"
    end

    environment(:production) do
      config.logger.options = "daily" # log rotation
    end    
  end
end

[timriley]

Hey @jodosha!

Glad my thoughts here were helpful for you

Your recap is on the mark!

The only thing I’d note is:

@jodosha

It’s able to load (nested) ENV variables and automatically assign them to the corresponding value (e.g. foo.bar via HANAMI_FOO__BAR )

It is definitely loads from ENV currently, but support for loading nested variables is not yet in place. I think this would be a helpful feature and should be something we put in place before releasing 2.0.

For the sake of clarity, probably a slightly better example of a nested env would be FOO__BAR for a setting nested as foo.bar (This is just removing the HANAMI_ prefix from your example since, given this file is for defining user-owned settings, it’s not likely they’d prefix their setting names with hanami_).

In this same vein, your Example: Override defaults via application setting would actually require the env var to be LOGGER__STREAM to match the nested logger.stream setting defined inside settings.rb — there’s nothing about the settings loading code that would look for env vars prefixed with HANAMI_.

Aside from that, all of your example scenarios for application configuration make sense to me too!

Now in the spirit of your original message, around reducing boilerplate while still allowing users to make simple application configuration changes, I would like to propose an additional behaviour: the application configuration should look to well-known ENV vars to populate the defaults for settings that users will frequently wish to customise.

So back on that example of logging: if we consider that our users may want to change the logger configuration fairly often, and maybe change it in a way that is different for the various deployment targets for their app, we should have Hanami::Configuration (or a closely related component) look to the ENV for these values, for example (and this is where that HANAMI_ prefix comes back in handy) HANAMI_LOGGER_LEVEL and HANAMI_LOGGER_STREAM.

This way, config/application.rb can retain a high-level of signal to noise and users can have an easy way to tune certain baseline defaults.

This behaviour is akin to how e.g. the bundler or aws CLI tool can be configured by setting certain env vars as an alternative to passing explicit flags. Or even closer to home, it’s a lot like how you can set RAILS_ENV or HANAMI_ENV to set another aspect of an app’s configuration, it’s environment.

If any of these env vars are not available, the Hanami::Configuration settings would fall back to their statically defined defaults, of course.

Now you might ask whether this may be confusing, given that both application and the settings objects will both look to the ENV. I think we can help with this by documenting the clear distinctions around how they each operate:

• Application configuration looks to ENV only for a small number of frequently customised settings. Those used fixed var names, prefixed by HANAMI_, and cannot be changed by the user.
• Application settings loads from ENV based entirely on names the user defines, with no fixed prefix, and does not affect the Hanami configuration unless the user chooses to use one of those settings when explicitly assigning to the configuration.

I think in actual usage this distinction would feel pretty natural, and would be made clearer still by the fact that we would also have a single page in our guides outlining the small number of “special” env vars that the framework uses to adjust its configuration (here’s how bundler does it).

The upshot of this is that it gives us another way to do “zero config” and keep the code in our config/ directory as simple as possible in newly generated applications.

How does this sound to you?

[jodosha]

@timriley

but support for loading nested variables is not yet in place

Yup, that should be added IMO.

---

Everything else makes sense to me.

Except one thing to clarify, implement, and document: the match between env variables and settings.

1. Hanami init process loads the .env.* relevant to the current environment.
2. Settings would apply values from ENV only for matching keys. In this way we won’t try to automagically assign ENV vars outside of the app realm (e.g. UNIX PATH).

[timriley]

Awesome, sounds like we’re good to go with this, @jodosha!

@jodosha

Except one thing to clarify, implement, and document: the match between env variables and settings.

1. Hanami init process loads the .env.* relevant to the current environment.
2. Settings would apply values from ENV only for matching keys . In this way we won’t try to automagically assign ENV vars outside of the app realm (e.g. UNIX PATH ).

Yes, this is right in my eyes.

Of course, the ENV is still the ENV and if a user did create a :path setting inside their config/settings.rb, then its value would indeed be assigned by whatever is ENV["PATH"]. So in this light our docs should also remind people to be careful not to clash with the basic env vars that are expected from their standard process execution environment

On a related note, another thing that would be good to sort out is just what language we can use to help clearly distinguish between the framework-provided settings (those typically customised inside config/application.rb) and the user-defined settings (defined in config/settings.rb). I’ve tried to call the former configuration and the latter aplication settings, but the trouble is that when you want to talk about an individual thing inside the configuration, you end up calling it a “setting” anyway. So I’m definitely open to naming suggestions or conventions.

Lastly, all this talk about expanding our usage of the ENV makes me realise that we should the logic for conditionally loading dotenv out of the Hanami::Application::Settings::DotenvStore (turning it into jus ta simple EnvStore relying on a pre-loaded ENV) and instead do the dotenv loading as part of the standard Hanami application boot. That way our users can use .env* files for setting those special HANAMI_-prefixed env vars as well as their own specific ones. What do you think?

[jodosha]

@timriley Yes, ENV var boot should happen before loading settings and configuration. In any case, as part of the init process and outside of those settings/configuration responsibilities.

---

RE: Clash with existing variables.
I assume that there is a way to force a different value via dry-configurable. E.g. default: option?

[timriley]

@jodosha

RE: Clash with existing variables.
I assume that there is a way to force a different value via dry-configurable . E.g. default: option?

Certainly, providing defaults is part of the experience we look to provide for defining application settings, but the value from the ENV will always be preferred, if it is there; defaults are there as a fallback for when there is no matching var in the ENV.

So for names that are commonly part of a standard Unix env, this is very much a “buyer beware” kind of situation — just don’t use them In reality, I doubt this will be much of an issue.