It’s called LogBench and it’s free and open source, feel free to dig into the repo and put a star ⭐️ on it if you like it.

Installation

Add it to your Gemfile, run log_bench, and watch your requests stream in.

group :development do
  gem 'log_bench'
end

bundle install
log_bench

That’s it. No config needed.

If you want to use it in production just don’t put it inside the :development group and force enable it with an initializer:

# in config/initializers/log_bench.rb
if defined?(LogBench)
  LogBench.setup do |config|
    config.enabled = true
  end
end

Preview

More screenshots from Omarchy themes below, and even more here

Features

Here’s the full list of things it does:

• Real-time log reading with auto-scroll

• Fast JSON log parsing

• Split-view for request list and individual request details

• Syntax highlighting and ANSI color support

• Nice summary of the queries performed in a request.

• Performance breakdown per request: duration, allocations, db time, view time

• Sort by timestamp, duration, or status

• Advanced filters: method, path, status, controller, action, request id

• Free-text filter on the right pane to find specific SQL or log lines

• Copy to clipboard:

  • Left pane: full request summary

  • Right pane: selected SQL with call site

• Text selection mode to use the mouse for copy

• Keyboard controls:

  • Navigation: ↑ ↓ or j k

  • Pane switch: ← → or h l

  • Filter dialog: f

  • Clear filter: c

  • Sort: s

  • Auto-scroll toggle: a

  • Copy: y

  • Text selection toggle: t

  • Clear requests in memory: Ctrl+L

  • Undo clear: Ctrl+R

  • Quit: q

• Background job context:

  • Colored job prefixes like [JobClass#job-id]

  • Works with ActiveJob when using the job logger

  • Works with plain Sidekiq via middleware

  • SQL queries inherit job context

With this simple class you can print tables like this:

records = Post.includes(:user).where(user_id: [1,2])
attributes = [:created_at, :title, {user: :name}]

TablePrinter.new(records, attributes).print_table

+-------------------------+-------------------+-----------------+
|  Created at             |  Title            |  User Name      |
+-------------------------+-------------------+-----------------+
| 2020-12-14 02:02:02 UTC | Un titulo 3       | Magdalena Silva |
| 2020-12-14 02:18:25 UTC | ¿Qué es escribir? | Benjamín Silva  |
| 2020-12-20 18:41:46 UTC | Primero para leo  | Magdalena Silva |
| 2021-02-06 14:18:54 UTC | A File            | Benjamín Silva  |
| 2020-12-20 04:09:37 UTC | aasdasd           | Benjamín Silva  |
| 2021-02-12 18:14:34 UTC |                   | Benjamín Silva  |
| 2020-12-14 01:50:20 UTC | Test super title  | Benjamín Silva  |
| 2021-02-06 03:32:02 UTC | lala5             | Benjamín Silva  |
+-------------------------+-------------------+-----------------+

This is an adaptation of this response, https://stackoverflow.com/a/28685559/1275069 to make it work with ActiveRecord right away.

It supports deeply nested attributes (example: post.user.some_model.some_other.the_attribute), but remember to include them in the query to avoid n+1.

# frozen_string_literal: true

class TablePrinter
  def initialize(records, attrs)
    @records = records
    @attrs = attrs
  end

  # rubocop:disable Rails/Output
  def print_table
    buffer = [divider, header, divider, content, divider].join("\n")
    puts buffer
  end
  # rubocop:enable Rails/Output

  private

  attr_reader :records, :attrs

  def content
    records.map { |row| line(row) }
  end

  def header
    "| #{columns.map { |_, g| g[:label].ljust(g[:width]) }.join(' | ')} |"
  end

  def divider
    "+-#{columns.map { |_, g| '-' * g[:width] }.join('-+-')}-+"
  end

  def line(row)
    str = attrs.map { |attr| row_value(row, attr).ljust(columns[attr][:width]) }.join(' | ')
    "| #{str} |"
  end

  def row_value(row, attr)
    if attr.is_a?(Hash)
      row_value(row.send(attr.keys.first), attr.values.first)
    else
      row ? row.send(attr).to_s : ''
    end
  end

  def columns
    @columns ||= col_labels.each_with_object({}) do |(attr, label), h|
      h[attr] = {
        label: label,
        width: [records.map { |row| row_value(row, attr).to_s.size }.max, label.size].max
      }
    end
  end

  def col_labels
    @col_labels ||= attrs.index_with { |attr| nested_name(attr) }
  end

  def nested_name(attr, memo = '')
    if attr.is_a?(Hash)
      nested_name(attr.values.first, "#{memo} #{attr.keys.first.to_s.humanize}")
    else
      "#{memo} #{attr.to_s.humanize}"
    end
  end
end

Feel free to use it in your projects! Happy Coding.

Performance

• Cacheable Content.
• Global Reach.
• Persistent Connections.
• Collapsed Forwarding.

Security

• Distributed Denial of Service (DDOS)
• Encryption in Transit
• Web Application Firewall (WAF).

Keep in mind there is an additional fee for using Cloudfront, but the traffic inside the aws network has no costs.

Then, of course we need to load balance our EC2 instances for high availability. Then the requests are handled by nginx and passed via proxy_pass to Puma (rails server).

That's a lot of jumps to serve a request!

I wanted to log rails requests with the real user IP address, but with all these jumps, it's not a trivial thing.

What you need to do is forward to the upstream the X-Forwarded-For header, which comes all the way up from Cloudfront, passing through ELB to finally be passed to Nginx

upstream your_app {
  server unix:/home/ubuntu/www/shared/tmp/sockets/puma.sock fail_timeout=0;
}
server {
    listen 80 default_server;
    server_name yourapp.com;
    gzip on;

     location / {
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # <- this header !
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass http://your_app;
    }
}

That header comes in the form of a comma separated string, so for every jump, an ip is appended. For this reason, the first ip "might be" the real user ip. (unless someone manually injects that header in the request)

Having this, we can now use it in Rails. I changed this in production.rb

-  config.log_tags = %i[request_id remote_ip]
+  config.log_tags = [:request_id, ->(request) { request.forwarded_for&.first || request.remote_ip}]

The interesting part is adding this lambda to the log_tags

->(request) { request.forwarded_for&.first || request.remote_ip }

Where forwarded_for is defined in the Rack request class:

def forwarded_for
  if value = get_header(HTTP_X_FORWARDED_FOR)
    split_header(value).map do |authority|
      split_authority(wrap_ipv6(authority))[1]
    end
  end
end

We try to find the first ip in the array, which as mentioned earlier, would be the first jump (real user ip). We use safe navigator in case the header is not present, and we fallback to remote_ip in case we could not find a forwarded ip.

I, [2020-12-22T19:46:31.273143 #39407]  INFO -- : [dfbe8700-908a-4405-bb0c-a23ac5359a5f] [190.8.117.22] Started GET "/" for 64.252.68.65 at 2020-12-22 19:46:31 +0000

the first ip in this log is the real ip of the user (190.8.117.22), while the last ip (64.252.68.65) is the IP of the Cloudfront edge location.

3 important files:

• Dockerfile
• docker-compose.yml
• dev.exs

Dockerfile

FROM bitwalker/alpine-elixir-phoenix:latest

# Cache elixir deps
ADD mix.exs mix.lock ./
RUN mix do deps.get, deps.compile

# Same with npm deps
ADD assets/package.json assets/
RUN cd assets && npm install

CMD ["mix", "phx.server"]

docker-compose.yml

version: '3.7'
services:
  web:
    build: '.'
    ports:
      - "4000:4000"
    volumes:
      - /opt/app/assets/node_modules
      - /opt/app/deps
      - .:/opt/app
    depends_on:
      - db
    environment:
      MIX_ENV: 'dev'
  db:
    image: postgres:alpine
    ports:
      - 5432:5432

here we mount 3 volumes, the first two hold the deps so we don't recompile elixir deps or install npm packages every time we build or run the service. The last volume is our app itself, this way, changes reflect immediately without rebuilding or restarting the service.

dev.exs

config :yourapp, Yourapp.Repo,
  username: "postgres",
  password: "postgres",
  database: "yourapp_dev",
  hostname: "db",
  show_sensitive_data_on_connection_error: true,
  pool_size: 10

Just make sure to override the hostname with db and use the default user and password for postgres.

after this you can do docker-compose build and the docker-compose up and head to http://localhost:4000 in your browser.

if you want to enter to the container that is running:

docker container ls -a

Find the one with the _web suffix, mine is myapp_web

A one liner to enter to it would be something like this:

docker exec -it $(docker ps | grep 'myapp_web' | awk '{print $1}') /bin/bash

Something I've been doing more extensively lately is Unit Testing, not necessarily doing TDD but Unit testing per se. When you start enjoying it it becomes like a challenge and also an addiction, getting you coverage reporting tool show a 100% coverage badge in your project is a big joy. Unit Testing also makes you produce better code and you feel safer, so it's something positive to add to your development process.

Hands on

First, you will have to register at travis-ci.org and codecov.io both have free plans for public repositories. (sign up using github)

Select the repositories you want to integrate in both services like shown above:

For the reporting tool we will be using the package ExCoveralls, add the following to your mix.exs file in your Phoenix project

defmodule Yourapp.MixProject do
  use Mix.Project

  def project do
    [
      app: :yourapp,
      version: "0.1.0",
      elixir: "~> 1.5",
      elixirc_paths: elixirc_paths(Mix.env()),
      compilers: [:phoenix, :gettext] ++ Mix.compilers(),
      start_permanent: Mix.env() == :prod,
      aliases: aliases(),
      deps: deps(),
      test_coverage: [tool: ExCoveralls] # ADD THIS LINE
    ]
  end

  ...

  defp deps do
    [
      {:phoenix, "~> 1.4.9"},
      {:phoenix_pubsub, "~> 1.1"},
      {:phoenix_ecto, "~> 4.0"},
      {:ecto_sql, "~> 3.1"},
      {:postgrex, ">= 0.0.0"},
      {:phoenix_html, "~> 2.11"},
      {:excoveralls, "~> 0.11.1"}, # ADD THIS LINE
      {:phoenix_live_reload, "~> 1.2", only: :dev},
      {:gettext, "~> 0.11"},
      {:jason, "~> 1.0"},
      {:plug_cowboy, "~> 2.0"}
    ]
  end

  ...

end

Also create a .travis.yml file in the root folder of your project, as follows:

language: elixir

elixir:
  - '1.9.1'
otp_release:
  - '22.0.7'

env:
  - MIX_ENV=test

addons:
  postgresql: '9.6'

services:
  - postgresql

before_script:
  - cp config/travis.exs config/test.exs
  - mix do ecto.create, ecto.migrate

script:
  - mix do compile --warnings-as-errors, coveralls.json

after_success:
  - bash <(curl -s https://codecov.io/bash)

cache:
  directories:
    - _build
    - deps

The before_script will replace a test.exs file for the travis.exs so it will run an environment with specific things for travis to work properly.

So we need to create this travis.exs under config folder

use Mix.Config

# Configure your database
config :yourapp, yourapp.Repo,
  username: "postgres",
  password: "",
  database: "yourapp_test",
  hostname: "localhost",
  pool: Ecto.Adapters.SQL.Sandbox

# We don't run a server during test. If one is required,
# you can enable the server option below.
config :yourapp, yourappWeb.Endpoint,
  http: [port: 4002],
  server: false

# Print only warnings and errors during test
config :logger, level: :warn

You will also want to configure the coverage reporting tool to exclude some files we are not going to test nor take in account for our coverage metrics

create a coveralls.json file in the root folder of the project

{
  "coverage_options": {
    "minimum_coverage": 100
  },
  "skip_files": [
    "test/",
    "lib/yourapp_web.ex",
    "lib/yourapp.ex",
    "lib/yourapp/application.ex",
    "lib/yourapp/repo.ex",
    "lib/yourapp_web/endpoint.ex",
    "lib/yourapp_web/gettext.ex",
    "lib/yourapp_web/views/error_helpers.ex"
  ]
}

This minimum_coverage config will mark as fail a build that doesn't have 100% coverage, which is hard to achieve in a real life project (but not impossible!).

to show badges of the build passing and coverage percentage like this:

add this, replacing yourgithub with your github username and yourrepo with the repository name

[![codecov](https://codecov.io/gh/yourgithub/yourrepo/branch/master/graph/badge.svg)](https://codecov.io/gh/yourgithub/yourrepo)
[![Build Status](https://travis-ci.org/yourgithub/yourrepo.svg?branch=master)](https://travis-ci.org/yourgithub/yourrepo)

With all this configuration, every time you create a pull request you will see something like this:

and the PR list will show checks or crosses depending on the check status

This same story happens with Phoenix Framework, but even adding a third language to the stage: Erlang, Elixir and Node.js

So how do we manage all involved languages versions with one single version manager?

meet asdf version manager

You can install it following the installation instructions

After installing, simply install an Erlang version by running

# to add the erlang plugin
$ asdf plugin-add erlang https://github.com/asdf-vm/asdf-erlang.git
# to list all available erlang versions
$ asdf list-all erlang

A long list may appear, you usually would want to install the latest (at the bottom)

$ asdf install erlang 22.0.7

After installing Erlang, you can proceed to install Elixir with the same approach

$ asdf plugin-add elixir https://github.com/asdf-vm/asdf-elixir.git
$ asdf list-all elixir
$ asdf install elixir 1.4.9

Then, in your project folder, you can tell asdf to select specific versions of software by adding a .tool-versions file with some content like this:

elixir 1.9.1
erlang 22.0.7
nodejs 10.16.0

If you call asdf current to see whats the current versions of each configured plugin inside the current working directory

$ asdf current

  elixir   1.9.1    (set by /Users/benja/dev/newslettex/.tool-versions)
  erlang   22.0.7   (set by /Users/benja/dev/newslettex/.tool-versions)
  version 10.16.0 is not installed for nodejs

You notice that it says node 10.16.0 is not installed, because it reads the .tool-versions and finds out we don't have the requested nodejs version. To install it, simply:

$ asdf plugin-add nodejs https://github.com/asdf-vm/asdf-nodejs.git
$ brew install coreutils
$ brew install gpg
$ bash ~/.asdf/plugins/nodejs/bin/import-release-team-keyring
$ asdf install nodejs 10.16.0

Now you have all your versions managed under asdf.