502 Bad Gatewayを解決しrailsアプリを画面に表示させたい

Question

解決したいこと

railsアプリを作成し、AWSのEC2で本番環境にデプロイをしました。
問題なく本番環境で動作していたのですが、突然エラーが起こりアクセスできなくなってしまいました。

nginxがエラー原因に絡んでいると考えたのでいろいろ試しましたがエラーが解決しません。
どうかご協力いただきたいです。

開発環境

• MacOS Catarina
• Ruby2.6.5
• Ruby on Rails 6.0
• AWS(EC2)
• Nginx
• Unicorn
• Capistrano

エラー画面

試したこと

1. Nigixのエラーログを確認

$ sudo cat /var/log/nginx/error.log

# 実行結果
[error] 10239#0: *1 connect() to unix:/var/www/entertainer/shared/tmp/sockets/unicorn.sock failed (111: Connection refused) while connecting to upstream

1の実行結果より
unix:/var/www/rails/xxxxxx/tmp/sockets/.unicorn.sock failedでググるとこちらの記事に
行き着きました
↓
Nginx + Unicorn + Rails ドハマリしたエラーを3つ解説

2. Nginxのステータスを確認
先程の記事とエラーの内容が同じだったのでNginxのステータスを確認しました。

$ systemctl status nginx.service

# 実行結果
systemd[1]: Failed to read PID from file /run/nginx.pid: Invalid argument

ここから先はコマンドの意味が理解できなかったのですが、とりあえず記事の通りに実行しました。

$ mkdir /etc/systemd/system/nginx.service.d

# 実行結果
mkdir: ディレクトリ `/etc/systemd/system/nginx.service.d' を作成できません: File exists

# rootに移行
$ sudo su - root
[root@ip-172-31-45-167 ~]# 

[root@ip-172-31-45-167 ~]# printf "[Service]\nExecStartPost=/bin/sleep 0.1\n" > /etc/systemd/system/nginx.service.d/override.conf
[root@ip-172-31-45-167 ~]# systemctl daemon-reload
[root@ip-172-31-45-167 ~]# systemctl restart nginx 

# 念の為
[root@ip-172-31-45-167 ~]# exit
[ec2-user@ip-172-31-45-167 app]$ sudo systemctl daemon-reload
[ec2-user@ip-172-31-45-167 app]$ sudo systemctl restart nginx

もう一度ステータスを確認すると2で確認したエラーは解消されていました。

$ systemctl status nginx.service

# 実行結果
2月 17 19:24:31 ip-172-31-45-167.ap-northeast-1.compute.internal systemd[1]: Starting The nginx HTTP and reverse proxy server...
 2月 17 19:24:31 ip-172-31-45-167.ap-northeast-1.compute.internal nginx[10224]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
 2月 17 19:24:31 ip-172-31-45-167.ap-northeast-1.compute.internal nginx[10224]: nginx: configuration file /etc/nginx/nginx.conf test is successful
 2月 17 19:24:31 ip-172-31-45-167.ap-northeast-1.compute.internal systemd[1]: Started The nginx HTTP and reverse proxy server.
```　　


しかし、確認のためIPアドレスをブラウザでうってもまだ502のエラーは解消されませんでした。
ここから手詰まりになってしまいました。。。

# /etc/nginx/conf.d/rails.conf
```php: /etc/nginx/conf.d/rails.conf
upstream app_server {
  server unix:/var/www/entertainer/shared/tmp/sockets/unicorn.sock;
}

server {
  listen 80;
  server_name   54.150.30.40;
  # currnetの中の中身を参照するよう変更
  root /var/www/entertainer/currnet/public;

  location ^~ /assets/ {
    gzip_static on;
    expires max;
    add_header Cache-Control public;
    # currentの中を参照するよう変更
    root  /var/www/entertainer/current/public;
  }

  try_files $uri/index.html $uri @unicorn;

  location @unicorn {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://app_server;
  }

  error_page 500 502 503 504 /500.html;
}

unicorn.rb

# ../が一つ増えている
app_path = File.expand_path('../../../', __FILE__)

worker_processes 1
# currentを指定
working_directory "#{app_path}/current"

# それぞれ、sharedの中を参照するよう変更
listen "#{app_path}/shared/tmp/sockets/unicorn.sock"
pid "#{app_path}/shared/tmp/pids/unicorn.pid"
stderr_path "#{app_path}/shared/log/unicorn.stderr.log"
stdout_path "#{app_path}/shared/log/unicorn.stdout.log"

#Railsアプリケーションの応答を待つ上限時間を設定
timeout 60

#以下は応用的な設定なので説明は割愛

preload_app true
GC.respond_to?(:copy_on_write_friendly=) && GC.copy_on_write_friendly = true

check_client_connection false

run_once = true

before_fork do |server, worker|
  defined?(ActiveRecord::Base) &&
    ActiveRecord::Base.connection.disconnect!

  if run_once
    run_once = false # prevent from firing again
  end

  old_pid = "#{server.config[:pid]}.oldbin"
  if File.exist?(old_pid) && server.pid != old_pid
    begin
      sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
      Process.kill(sig, File.read(old_pid).to_i)
    rescue Errno::ENOENT, Errno::ESRCH => e
      logger.error e
    end
  end
end

after_fork do |_server, _worker|
  defined?(ActiveRecord::Base) && ActiveRecord::Base.establish_connection
end

config/environments/production.rb

Rails.application.configure do
  # Settings specified here will take precedence over those in config/application.rb.

  # Code is not reloaded between requests.
  config.cache_classes = true

  # Eager load code on boot. This eager loads most of Rails and
  # your application in memory, allowing both threaded web servers
  # and those relying on copy on write to perform better.
  # Rake tasks automatically ignore this option for performance.
  config.eager_load = true

  # Full error reports are disabled and caching is turned on.
  config.consider_all_requests_local       = false
  config.action_controller.perform_caching = true

  # Ensures that a master key has been made available in either ENV["RAILS_MASTER_KEY"]
  # or in config/master.key. This key is used to decrypt credentials (and other encrypted files).
  # config.require_master_key = true

  # Disable serving static files from the `/public` folder by default since
  # Apache or NGINX already handles this.
  # config.public_file_server.enabled = ENV['RAILS_SERVE_STATIC_FILES'].present?
  config.public_file_server.enabled = true

  # Compress CSS using a preprocessor.
  # config.assets.css_compressor = :sass

  # Do not fallback to assets pipeline if a precompiled asset is missed.
  config.assets.compile = false

  # Enable serving of images, stylesheets, and JavaScripts from an asset server.
  # config.action_controller.asset_host = 'http://assets.example.com'

  # Specifies the header that your server uses for sending files.
  # config.action_dispatch.x_sendfile_header = 'X-Sendfile' # for Apache
  # config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for NGINX

  # Store uploaded files on the local file system (see config/storage.yml for options).
  config.active_storage.service = :local

  # Mount Action Cable outside main process or domain.
  # config.action_cable.mount_path = nil
  # config.action_cable.url = 'wss://example.com/cable'
  # config.action_cable.allowed_request_origins = [ 'http://example.com', /http://example.*/ ]

  # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
  # config.force_ssl = true

  # Use the lowest log level to ensure availability of diagnostic information
  # when problems arise.
  config.log_level = :debug

  # Prepend all log lines with the following tags.
  config.log_tags = [ :request_id ]

  # Use a different cache store in production.
  # config.cache_store = :mem_cache_store

  # Use a real queuing backend for Active Job (and separate queues per environment).
  # config.active_job.queue_adapter     = :resque
  # config.active_job.queue_name_prefix = "entertainer_production"

  config.action_mailer.perform_caching = false

  # Ignore bad email addresses and do not raise email delivery errors.
  # Set this to true and configure the email server for immediate delivery to raise delivery errors.
  # config.action_mailer.raise_delivery_errors = false

  # Enable locale fallbacks for I18n (makes lookups for any locale fall back to
  # the I18n.default_locale when a translation cannot be found).
  config.i18n.fallbacks = true

  # Send deprecation notices to registered listeners.
  config.active_support.deprecation = :notify

  # Use default logging formatter so that PID and timestamp are not suppressed.
  config.log_formatter = ::Logger::Formatter.new

  # Use a different logger for distributed setups.
  # require 'syslog/logger'
  # config.logger = ActiveSupport::TaggedLogging.new(Syslog::Logger.new 'app-name')

  if ENV["RAILS_LOG_TO_STDOUT"].present?
    logger           = ActiveSupport::Logger.new(STDOUT)
    logger.formatter = config.log_formatter
    config.logger    = ActiveSupport::TaggedLogging.new(logger)
  end

  # Do not dump schema after migrations.
  config.active_record.dump_schema_after_migration = false

  # Inserts middleware to perform automatic connection switching.
  # The `database_selector` hash is used to pass options to the DatabaseSelector
  # middleware. The `delay` is used to determine how long to wait after a write
  # to send a subsequent read to the primary.
  #
  # The `database_resolver` class is used by the middleware to determine which
  # database is appropriate to use based on the time delay.
  #
  # The `database_resolver_context` class is used by the middleware to set
  # timestamps for the last write to the primary. The resolver uses the context
  # class timestamps to determine how long to wait before reading from the
  # replica.
  #
  # By default Rails will store a last write timestamp in the session. The
  # DatabaseSelector middleware is designed as such you can define your own
  # strategy for connection switching and pass that into the middleware through
  # these configuration options.
  # config.active_record.database_selector = { delay: 2.seconds }
  # config.active_record.database_resolver = ActiveRecord::Middleware::DatabaseSelector::Resolver
  # config.active_record.database_resolver_context = ActiveRecord::Middleware::DatabaseSelector::Resolver::Session
end

どうかご教授願いたいです。。

Answer 1

nginxがフロントエンド、unicornがバックエンドの環境で
nginxが応答(502エラー)を返していることから、unicorn側でNGになっているものと予想されます。

単にunicornが落ちてるのではないでしょうか。その場合、unicornを起動してからブラウザでアクセスしてみてください。

Comments

[anago1030]

ご返信ありがとうございます！
ps -ef | grep unicorn | grep -v grepをターミナルで実行したところ、yukkyさんのおっしゃる通りunicornが停止していました。

RAILS_SERVE_STATIC_FILES=1 unicorn_rails -c config/unicorn.rb -E production -Dコマンドで
再起動し、ブラウザで確認してみると、502のエラーは解消したのですが、We're sorry, but something went wrong.のエラーが新しく発生しました。

less log/unicorn.stderr.logでunicornのログを確認したところ
Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) (Mysql2::Error::ConnectionError)というエラーを確認しました。

データベースが起動していないのではないかという仮説が立ったので
sudo systemctl status mariadbコマンドとsudo systemctl status nginx.serviceコマンドを実行し確認しましたが、mariadbとnginxはactive runningで起動していました。。

unicorn、nginx、mariadb全て起動していますが、ブラウザWe're sorry, but something went wrong.の状態です。。