Hanami persistence proposal for 2.x

APP structure

app/

app/
  relations/
    coffees.rb       # TestApp::Relations::Coffees < TestApp::Relation
  repositories/
    coffee_repo.rb   # TestApp::Repositories::CoffeeRepo < TestApp::Repository
  structs/
    coffee.rb        # TestApp::Structs::Coffee < TestApp::Struct
  relation.rb        # TestApp::Relation < Hanami::Relation
  repository.rb      # TestApp::Repository < Hanami::Repository
  struct.rb          # TestApp::Struct < Hanami::Struct
db/
  migrations/
  schema.rb

slices/

slices/
  admin/
    relations/
      users.rb         # Admin::Relations::Coffees < Admin::Relation
    repositories/
      coffee_repo.rb   # Admin::Repositories::CoffeeRepo < Admin::Repository
    structs/
      coffee.rb        # Admin::Structs::Coffee < TestApp::Struct
    relation.rb        # Admin::Relation < TestApp::Relation
    repository.rb      # Admin::Repository < TestApp::Repository
    struct.rb          # Admin::Struct < TestApp::Struct
    db/
      migrations/
      schema.rb

ENV Vars & Settings

  • DATABASE_URL: App database
  • ADMIN_DATABASE_URL: Admin database
# config/settings.rb

module TestApp
  class Settings < Hanami::Settings
    setting :database do
      setting :url, constructor: Types::String
    end
  end
end

TestApp.settings.database.url

Note: My suggestion is to use database.url instead of database_url to capture the following use case.

setting :database do
  setting :host, constructor: Types::String
  setting :port, constructor: Types::String
  setting :username, constructor: Types::String
  setting :password, constructor: Types::String
end

TestApp.settings.database      # => { ... } A Hash
TestApp.settings.database.host # => "..." A String
# slices/admin/config/settings.rb

module Admin
  class Settings < TestApp::Settings
    setting :database do
      setting :url, constructor: Types::String
    end
  end
end

Admin.settings.database.url

CLI

Generators

Model

$ bundle exec hanami generate model book
      create  app/structs/book.rb
      create  app/repositories/book_repository.rb
      create  app/relations/books.rb
      create  db/migrations/20230213123250_create_books.rb
$ bundle exec hanami generate model book --slice=admin
      create  slices/admin/structs/book.rb
      create  slices/admin/repositories/book_repository.rb
      create  slices/admin/relations/books.rb
      create  slices/admin/db/migrations/20230213123250_create_books.rb

Migration

$ bundle exec hanami generate migration create_books
      create  db/migrations/20230213123250_create_books.rb
$ bundle exec hanami generate migration create_books --slice=admin
      create  slices/admin/db/migrations/20230213123250_create_books.rb

Database

This is a 2.1 proposal

$ bundle exec hanami generate database --slice=admin --url="postgresql://localhost:5432/admin"
      add     .env                            # ADMIN_DATABASE_URL="..."
      add     .env.development                # ADMIN_DATABASE_URL="..."
      add     .env.test                       # ADMIN_DATABASE_URL="..."
      add     slices/admin/config/settings.rb # Settings block to match ENV var
      create  slices/admin/db/schema.rb
      create  slices/admin/db/migrations/.keep

The option --slice is mandatory

If --url is provided, fill in all .env* files.
If --url is NOT provided, only add ADMIN_DATABASE_URL with an empty string

Database

See 1.x commands for reference

Create

$ bundle exec hanami db create # CREATES ALL DATABASES
$ bundle exec hanami db create --database=app # CREATES App (app/) DATABASE
$ bundle exec hanami db create --database=admin # CREATES Admin (slices/admin) DATABASE

Drop

$ bundle exec hanami db drop # DROPS ALL DATABASES
$ bundle exec hanami db drop --database=app # DROPS App (app/) DATABASE
$ bundle exec hanami db drop --database=admin # DROPS Admin (slices/admin) DATABASE

Migrate

$ bundle exec hanami db migrate # MIGRATES ALL DATABASES
$ bundle exec hanami db migrate --database=app # MIGRATES App (app/) DATABASE
$ bundle exec hanami db migrate --database=admin # MIGRATES Admin (slices/admin) DATABASE

Rollback

$ bundle exec hanami db rollback # ROLLS BACK App (app/) DATABASE
$ bundle exec hanami db rollback 3 # ROLLS BACK App (app/) DATABASE
$ bundle exec hanami db rollback --database=app # ROLLS BACK App (app/) DATABASE
$ bundle exec hanami db rollback 3 --database=app # ROLLS BACK App (app/) DATABASE
$ bundle exec hanami db rollback --database=admin # ROLLS BACK Admin (slices/admin) DATABASE
$ bundle exec hanami db rollback 2 --database=admin # ROLLS BACK Admin (slices/admin) DATABASE

Prepare

$ bundle exec hanami db prepare # PREPARES ALL DATABASES
$ bundle exec hanami db prepare --database=app # PREPARES App (app/) DATABASE
$ bundle exec hanami db prepare --database=admin # PREPARES Admin (slices/admin) DATABASE

Apply

$ bundle exec hanami db apply # RUNS FOR ALL DATABASES
$ bundle exec hanami db apply --database=app # RUNS FOR App (app/) DATABASE

It generates db/schema.sql file

$ bundle exec hanami db apply --database=admin # RUNS FOR Admin (slices/admin) DATABASE

It generates slices/admin/db/schema.sql file

Version

$ bundle exec hanami db version # PRINTS ALL DATABASES VERSIONS
$ bundle exec hanami db version --database=app # PRINTS App (app/) DATABASE VERSION
$ bundle exec hanami db version --database=admin # PRINTS Admin (slices/admin) DATABASE VERSION

Rake

For compatibility with Ruby hosting vendors, we need to expose a db:migrate Rake task

$ bundle exec rake db:migrate # MIGRATES ALL THE DATABASES
$ HANAMI_DATABASE=app bundle exec rake db:migrate # MIGRATES App (app/) DATABASE
$ HANAMI_DATABASE=admin bundle exec rake db:migrate # MIGRATES Admin (slices/admin) DATABASE

@jodosha What is your reasoning behind placing relations in the same place where you have app logic? - Relations are part of persistence layer, and repositories belong to the application layer. Relations are adapter-specific, and map to to the underlaying structure of database(s) - slices structure could be completely different than your db.

I am afraid that with this configuration we could fail into the trap of building applications based on the database schema which is one of the caveats with other frameworks. I feel like keeping db structure independent of slices is important for apps to grow without technical debt increasing too much.

@swilgosz Thanks for the feedback.

Let’s split your concerns.


Relations. Where do you believe they should go?
My thinking was to keep them in app/, alongside with repos and structs. The three of them go together (IMO) and so they have the same location.


Slices. In my proposal, they will share the same App database.

But they can have their own database, so when you build your Modular Monolith, you’re already partitioning the data on a domain basis (slice). In this way you avoid data coupling.


Can you please share with me a structure that you would like to see? Including the reasons. Thanks! :pray:

1 Like

How do you anticipate handling DATABASE_URL in test suites?

I greatly prefer this approach (as opposed to Rails’ database.yml) but it has lead to accidental test runs against the wrong database connection, because changing your env variables for a test run is easy to forget.

My solution to this problem has been twofold: using this helper method in my persistence provider:

def database_url
  uri = target[:settings].database_url

  if Hanami.env?(:test)
    uri = URI(uri)
    uri.path = "/app_test"
    uri = uri.to_s
  end

  uri
end

clearly that won’t do for a framework feature.

The second thing I ended up doing is implementing something similar to Rails’ database protection via a system_metadata table. (This was necessary following a production incident where I accidentally ran DatabaseCleaner on our production database :flushed:)

The database protection feature is good to have as a fail-safe, perhaps something I could upstream.

The same way we did in 1.x: via different .env.* files.

Read more at: :point_down:


Regarding the protection of production database, we already had this in 1.x.

Quoting from the guides:

In order to preserve production data, this command can’t be run in the production environment.

Read more at: :point_down:

Slices:

I’m not sure, unfortunately. I got a feeling that in Hanami 2, /app and /slices are the application layer, where business logic could be put. However, persistence (relations) should be split in case we have different databases - not per slice but allowing to pull data from multiple DBs.

AFAIK one of ROM’s powers is allowing such cross-DB, and cross-adapter even data fetching.

Relations

Having the above in mind, this is why I’m more into keeping persistence (Relations) in the /lib folder. I’m not too strict about it, just thinking loudly.

When you have relations (that correspond to your DB structures), then you can implement repositories, and app-level structs even in the /app or /slices.

To summarize, I’m fine with having repositories and App-specific structs in the app and slices, just relations seem to be sth different to me.

Proposal:

lib
  |-- persistence
    |-- DB1
      |-- relations
    |-- DB 2
      |-- relations 
    |-- File-based DB
    |-- CSV DB
    |-- HTTP based DB... 
  |-- ...
app
  |-- repositories # (common classes)
  |-- structs
  |-- ...
slices
  |-- admin
    |-- repositories # figures out which relations to use?
    |-- queries # objects with advanced queries, across multiple dbs?
    |-- structs

Please forgive me if in my reasoning there is a missing concept of ROM possibilities and assumptions, I’m still ActiveRecord-ish in thinking, don’t feel fluent in understanding all these concepts just yet, for very large or complex apps/domains.

So I’ll comment on this in full soon enough, but in the meantime, I did want to point out one thing.

Over in my example app I’ve just moved my relation classes into app/, as a way to start getting a feel for the structure @jodosha has outlined above: Move relations to app/ by timriley · Pull Request #17 · decafsucks/decafsucks · GitHub

This change required I change my rom auto-registration setup to the following:

    rom_config.auto_registration(
      target.root.join("app"),
      namespace: Hanami.app.namespace.to_s
    )

With target.root.join("app") being given as the auto_registration directory, it means that rom will also look there for mappers/ and commands/ subdirectories, as well as the expected relations/.

Now defining custom mappers and commands is fairly advanced rom usage, but either way, I don’t think we’d want to see these defined in top-level directories under app. Those directories feel much too prominent for what are otherwise fairly niche concerns.

In addition, there’s a fair chance from naming collisions here too: “mappers” and “commands” are fairly common terms and there’s a non-zero chance that a user will try and create directories like those as the namespace for unrelated objects.

I tried passing this option to rom’s auto_registration method:

component_dirs: {
  relations: :relations
}

As well as this:

component_dirs: {
  relations: relations,
  mappers: nil,
  commands: nil,
}

But both of these failed because rom’s auto-registration setup expects these to be defined at all times.

I think this points to the need for rom’s auto-registration setup to be a little more flexible. Ideally I think we’d want the following:

  1. Ability to auto-register relations only in with one given root dir (app/) and namespace (e.g. "AppNamespace::Relations")
  2. And then to separately auto-register mappers and commands with different given root dir and namespace

Implicit in the above is that any for any single rom auto_registration call, it should be possible to disable registration for particular component types, such as mappers and commends.

Then after this, as a separate matter, we’d want to think about how whether we want to support default auto-registration for rom mappers and commands within Hanami apps, and if so, what directories they should use.

I suppose an alternative for Hanami’s rom setup could be to avoid rom’s auto_register entirely, and instead find and manually register all the rom components. But it’d probably be nice if we could make rom’s auto_register more flexible anyway :slight_smile:

cc @solnic since I think these could be useful things to solve with the upcoming version of rom.

I need to revisit this because better auto-registration was one of the reasons of the setup overhaul in rom 6.0. I’ll get back to you once I confirm how this could be arranged in rom 6.0.