What is the best way to initialize dependencies?

questions
1

Hello. :wave: I’m running into a situation where some of my application and provider configurations are getting cluttered with initialization code and am curious what the desired solution is for initializating dependencies in a Hanami application? Here’s how I’ve solved the problem for the moment:

Initializers

# demo/lib/demo/initializers/dry.rb
Dry::Schema.load_extensions :monads
Dry::Validation.load_extensions :monads

# demo/lib/demo/initializers/rack_attack.rb
require "rack/attack"

Rack::Attack.safelist("allow localhost") { |request| %w[127.0.0.1 ::1].include? request.ip }
Rack::Attack.throttle("requests by IP", limit: 100, period: 60, &:ip)

# demo/lib/demo/initializers/sequel.rb
require "sequel"

Sequel::Database.extension :constant_sql_override, :pg_enum
Sequel.database_timezone = :utc
Sequel.application_timezone = :local

Application

# demo/config/app.rb

require "hanami"
require "refinements/pathnames"

module Demo
  class App < Hanami::App
    using Refinements::Pathnames

    # This line loads all initializers.
    Pathname.require_tree root.join("lib/demo/initializers")
  end
end

:bulb: The Pathname firepower comes from the Refinements gem.

As you can see the above has some pros and cons:

  • Pros
    • I’m using a dedicated lib/demo/initializers folder in order to add a separate file for any dependency that I need to initialize and configure. This definitely cleans up code that is awkward to put in the application configuration or provider files.
    • Each initializer is loaded alphabetically so order of initialization is always the same and guaranteed.
  • Cons
    • Use of demo/lib/demo/initializers doesn’t feel right. It works but seems like config/initializers or app/initializers would be a better location?
    • These initializers are loaded outside of Dry System’s prepare, start, and shutdown lifecycle. Maybe that’s OK but would it be better to eager or lazy load initializers in a similar manner as the providers?

Anyway, some thoughts. Curious what others are doing to tackle this situation and if there are better ways to do this better?

2

Hanami ships with Provider API to deal with this type of code. Why did you decide to use “initializers” files instead?

3

Like @solnic mentioned, I’d definitely love to see how you could go using providers here.

One thing that might not be clear about providers (I’ve just checked our relevant guide and we could definitely do better explaining this) is that they can register zero or more components, emphasis on the zero.

So many of the initializers you’re sharing here could be providers that just do not register any components, and instead do the kinds of global gem configuration that your examples show.

A helpful lens to apply here is: what happens if I only want to partially boot my app, i.e. boot it without certain slices, or prepare the app and then load specific components only? In your initializers arrangement, even in these cases you’d be loading every single initializer, which in many cases would mean that your app is loading more code and doing more setup work than necessary.

An example here might be if you have part of your app that is not web-facing that you want to boot independently. In that case, it makes no sense to require and configure rack/attack, so you’d be better off just not loading it at all.

The related advantage of providers is that they can be loaded in different ways that mesh well with Hanami’s booting flexibility. For example:

  • For a fully booted Hanami app (when requiring "hanami/boot" or running Hanami.boot), every provider will be loaded, which is on par with your provided initializers arrangement
  • They can be started explicitly, e.g. Hanami.app.start :my_provider
  • They can also be started implicitly, when a component is resolved with a base key segment that matches a provider name, e.g. when a "foo.bar" component is resolved, Hanami will look for a :foo provider and start it if found
  • They can be put into separate slices as needed (e.g. slices/foo/config/providers/my_provider.rb); this could of course be manually replicated with a collection of initializers like you shared, but that’s still extra work

I suspect that at least some of your initializers could be ported to providers to take advantage of these benefits.

Another way to do certain kinds of configuration could be to use parent classes. This might suit your question in How should Dry Schema and Validations be prepared with monads? - #2 by bkuhlmann (which I’ve had on my list to answer since you posted, sorry for the delay), and the related dry.rb initializer example from your post here.

What it would look like is having a parent e.g. MyApp::Validation::Contract class that all your validation contracts inherit from. It’d look something like this (at app/validation/contract.rb):

# auto_register: false

require "dry/validation"

Dry::Schema.load_extensions :monads
Dry::Validation.load_extensions :monads

module MyApp
  module Validation
    class Contract < Dry::Validation::Contract
    end
  end
end

Then in any place where you create your own validation contracts, instead of inheriting from Dry::Validation::Contract directly, instead inherit from MyApp::Validation::Contract. This will ensure your base contract class is autoloaded, and with it, the global configuration for the validation gems that it uses.

The general question I ask when I try to organise Hanami configuration is this: given a prepared app only (Hanami.prepare), when I resolve a single component, can everything be configured sufficiently for that single component to work, and nothing more?

I then try and arrange things to create a “yes” answer to that question :slight_smile:

I’m really glad you’re asking these things now, @bkuhlmann, because I’m sure there are aspects of the framework that could be improved to better handle the needs from a wider range of users (and use cases) out in the real world!

I’d definitely be keen to hear your thoughts on everything above.

4

Peter

Thanks, yep, I admit I have gaps in my Dry System knowledge (I’ve been more heavily invested in Dry Container + Infusible) so sometimes ask strange questions but trying to course correct as fast as possible. :sweat_smile: The initial trigger for going down this path was when I posted this question. I never got a response so figured I was either not asking the right question, no one had an answer, or I was on my own entirely…but Tim has responded. :bow:

Tim

[Providers] can register zero or more components, emphasis on the zero.

Yes, I hadn’t realized the significance of zero until you clarified this. Thanks!

A helpful lens to apply here is: what happens if I only want to partially boot my app.

Yeah, I don’t want to give the false impression that I’m trying to break out of the provider lifecycle. I’m fully on board with leveraging the lifecycle as much as possible. This is a powerful feature. The primary source of my confusion is when I don’t see providers behave as I intuit from reading the documentation and/or source code. :sweat_smile:

Another way to do certain kinds of configuration could be to use parent classes

Ah, yes, this solidifies and nicely ties together the power of the auto_register: false pragma. Thanks!

given a prepared app only (Hanami.prepare), when I resolve a single component, can everything be configured sufficiently for that single component to work, and nothing more?

Yep, I’m on board and very much want to stay in this space…but there is often subtle nuance that gets tricky to grok when using Hanami.prepare or Hanami.boot that has been tripping me up.

Tim, to better respond to your comments above, I thought it might be beneficial to include a couple short videos that my illuminate my confusion (which might be of interest to others getting used to Dry System’s provider lifecycle). I’ll work my way through my original examples in the order listed:

Dry Schema/Validation

I only focus on Dry Schema (since the situation is the same for Dry Validation).

dry-schema-provider

I want to emphasize, Tim, that I’m not ignoring your superclass suggestion. I agree, that’s a strong and valid solution but I think it’d be neat to simply prepare Dry Schema/Validation with monad support.

Rack Attack

rack-attack-provider

If you were to attempt to launch the console (which essentially messages Hanami.prepare) the same exception would be thrown.

Sequel

sequel

I realize this demonstration is a bit unfair because ROM + Sequel persistence hasn’t been fully sorted out or supported yet. The problem illustrated here is of an order or operations issue and could be partially resolved by ensuring my persistence provider is called first.

I hope the above helps illustrate more of the confusion around the provider lifecyles and why I keep getting tripped up between prepare and boot states, especially when prepare doesn’t exactly behave the way I expect. :sweat_smile:

5

Your providers won’t start automatically, that’s why you get these exceptions. You’d have to start them in your app via start(:dry_schema) etc. There’s no way of doing this automatically in case of providers that don’t register anything in the container. We could make it automatic based on some conventions or configuration though. ie register_provider(:dry_schema, auto_start: true) or something like that.

6

There’s no way of doing this automatically in case of providers that don’t register anything in the container.

Yeah, so that brings me back to my rudimentary use of initializers and a need for initializing the application for non-container based work. It feels like two different life cycles paths are at play here?

  • Providers: Have the single responsibility (life cycle) of preparing, starting, and shutting down dependencies. Adding more responsibility by overloading providers to handle non-container based life cycles may not be wise? Granted, I started down that path because I was confused at first but now that you mention use of auto_start: true I’m worried that could be too much responsibility since boolean control coupling tends to lead to hard to maintain outcomes. :sweat_smile:
  • Initializers: I’m not sure what to call this, there’s probably a better name. …but this would be the lifecycle that deals with preparing, starting, and shutting down of the app of objects which are not related to containers but mostly configuration based.
7

I think we should stick to providers. They really are designed to handle such cases. So far nobody wanted to autostart them though. Loading things in isolation can only be managed through providers and if we add another concept to the framework, things will get more complicated. There’s no extra responsibility here either as providers are managed by a separate abstraction called “provider registrar” which is used by the container. Now that I think about it again, it should not be called auto_start though. We want to only call prepare since it’s only configuring things, not starting, so maybe auto_prepare would be a better name. One downside of this approach is that if you forget to set it, things won’t work and you may waste time debugging it, so an alternative would be to have a dedicated provider type or a new lifecycle step that is always called. Probably the latter is better.

8

Yeah, after chewing on this, having parallel life cycles wouldn’t be a great benefit as you mention. At one point, I was thinking a very lightweight/generic lifecycle framework (for lack of better name) for which to hang providers and other uses cases off of (i.e. a lifecycle not directly tied to providers) might be neat but not sure that makes sense either.

Riffing off of your feedback, I like your idea of having a new dedicated lifecycle step that is always called. You’ve been thinking about this much longer than I have so I defer to you but think this could be a powerful addition. Rolling with this train of thought, this could be neat:

Lifecycle Steps

  • Init: I keep using “init/initialize” terminology in these discussions only because I think that is the right way to label this lifecycle step. I’m also borrowing this terminology from the Dry System documentation but not entirely clear if that is what Dry System intended (it’s only briefly mentioned in the docs). Definitely defer to you and others for better terminology. Rolling with this, though, this seems like a great place to resolve all of my initialization issues documented above. A few questions:
    • You mention this step would always be called. I assume that means that if you don’t don’t provide this step, it’s a no-op right?
    • I would assume that use of register would not be available in this step either? That should be reserved for the prepare step? This would also encourage good separation of concerns between the init and prepare steps which seems nice.
    • Having this step would diminish the prepare step somewhat. I suppose the documentation would have to be very clear in stating that init is akin to eager loading and prepare is akin to lazy loading so you’d potentially want to use one or the other depending on your needs but not both?
  • Prepare: No change, works as designed but does require you to take special care if init is in play.
  • Start: No change, works as designed.
  • Shutdown: No change, works as designed.

Anyway, some thoughts in case it helps. :sweat_smile:

1 Like
10

Yes, I’d also like to explore options for using providers here. Will make sure to revisit this thread when I get a chance!

2 Likes
11

So, fun fact: prepare use to be called init :sweat_smile:

I actually just realized we may not even need a step though. What if you just put your code inside the block? Then it would always run whenever providers are established BUT it may be too early in some cases and it would depend on implementation details of the framework (which means it may change from version to version). I figured it’s worth mentioning nevertheless.

Yes

I wouldn’t go so far with the restrictions, it would only make things more complicated to implement. People can decide for themselves whether they want to register stuff or not.

Yeah I think the name should include a hint that this is something that always runs at boot time. Something like eager_prepare maybe?

12

fun fact: prepare use to be called init

Ah! :bow:

What if you just put your code inside the block?

I experimented with this and, interestingly, using code like this:

Hanami.app.register_provider(:demo) { puts "I have no lifecycle." }

Ends up behaving as if it was a start block and doesn’t help in this particular situation, I’m afraid. So I ended up with the following solution instead:

# config/providers/init_dry_schema.rb
Hanami.app.register_provider :init_dry_schema do
  prepare { Dry::Schema.load_extensions :monads }
end

# bin/console AND spec/hanami_helper.rb
# TODO: Remove once Hanami supports eager loading of non-registered components.

require "refinements/pathnames"

using Refinements::Pathnames

Hanami.app
      .container
      .providers
      .provider_files
      .select { |path| path.basename.fnmatch? "init_*" }
      .each { |path| Hanami.app.start path.name.to_s.to_sym }

It’s possible that I might stab myself on a sharp edge with this solution but seems to be working nicely for now which has the following benefits:

  • Eager loads for the development environment only.
  • Use of the init prefix and key allows me to dynamically start these providers which don’t register any components.
  • Safely loads when launching the server locally or deployed in production because Hanami.boot is called at that point which allows these special “init” providers to behave as normal.
  • Reduces several lines of code found in the config/app.rb or original providers. Now my non-init prefixed providers can focus on registering and using components/dependencies. :rocket:

:information_source: At one point I tried to eager load these “special” providers via the config/app.rb file within an environment block but anything you do in config/app.rb happens before Hanami.start is executed so that doesn’t work.

I wouldn’t go so far with the restrictions

OK, fair (especially by not introducing more complications to the implementation).

Something like eager_prepare maybe?

Naming is hard. :sweat_smile: I seem to be stuck on :init because it’s short, sweet, and feels like what you do as a very basic first step which doesn’t register a component. I suppose these names could work too: :allocate, :configure, :wake. Trying to stick to a single word for less typing but probably failing. :upside_down_face: