Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
1495777
Avoid monkey patching rails/rack/logger
quixoten Mar 3, 2014
1c999bf
Avoid monkey patching instrumentation
quixoten Mar 4, 2014
446c7e7
Upgrade logstash-event dependency
quixoten Mar 4, 2014
d9c0194
Cleanup specs a bit
quixoten Mar 4, 2014
e6eda35
Prep for initial release of the md fork
quixoten Mar 5, 2014
2c8dc44
Create LICENSE.md
quixoten Mar 5, 2014
b41381d
Update README.md
quixoten Mar 5, 2014
58c9024
Update README.md
quixoten Mar 5, 2014
eda9127
Do not suppress app logs by default
quixoten Mar 5, 2014
1d532cb
Update README.md
quixoten Mar 17, 2014
e6ad03c
Refactor railtie
quixoten Mar 8, 2014
6b25c1f
Remove supurfluous class
quixoten Mar 19, 2014
8bdc4f2
Bump version 1.0.0.beta1
quixoten Mar 19, 2014
03c07e7
Add a syslog device
quixoten Mar 19, 2014
2e5ac0b
Bump version to 1.0.0.beta2
quixoten Mar 19, 2014
13e6000
Accept string options
quixoten Mar 19, 2014
20a3df3
Bump version to 1.0.0.beta3
quixoten Mar 19, 2014
a8e27d4
Put runtimes in their own namespace
quixoten Mar 19, 2014
f81c69a
Bump version to 1.0.0.beta4
quixoten Mar 19, 2014
6777e09
Fix params cleaning
quixoten Mar 20, 2014
39a17cc
Bump version to 1.0.0.beta5
quixoten Mar 20, 2014
03173d2
Re-factor log silencing strategy
quixoten Mar 20, 2014
ddf03fb
Bump version to 1.0.0.beta6
quixoten Mar 20, 2014
df1776a
Re-factor silent logger
quixoten Mar 20, 2014
5b1c81c
Bump version to 1.0.0.beta7
quixoten Mar 20, 2014
cc6bc8e
Limit silencing to LogSubscriber instances
quixoten Mar 20, 2014
f446a79
Bump version to 1.0.0.beta8
quixoten Mar 20, 2014
d7c7372
Update README.md
quixoten Mar 21, 2014
c090d9a
Update README.md
quixoten Mar 21, 2014
ebe8a91
Bump version to 1.0.0.rc1
quixoten Mar 21, 2014
59495a3
Fix memoization of ::LogStasher.enabled?
quixoten Mar 21, 2014
c6d1f7c
Bump version to 1.0.0.rc2
quixoten Mar 21, 2014
1bb1c76
Add a factory for logstasher devices
quixoten Mar 31, 2014
ba71325
Bump version to 1.0.0.rc3
quixoten Mar 31, 2014
4123629
Stringify config keys when creating devices
quixoten Mar 31, 2014
965d16c
Bump version to 1.0.0.rc4
quixoten Mar 31, 2014
975f8fd
Inherit from LogSubscriber rather than Subscriber
quixoten Apr 1, 2014
69b2540
Bump version to 1.0.0.rc5
quixoten Apr 1, 2014
a6de048
Bump version from 1.0.0.rc5 to 1.0.0
quixoten Aug 7, 2014
d13db1f
Add option to serialize parameters
quixoten Sep 9, 2014
cf26118
Merge pull request #1 from moneydesktop/unserialized_params
quixoten Sep 10, 2014
295bd1e
Bump version to 1.0.1
quixoten Sep 10, 2014
c6d8480
Fix Syslog.log to include string option
fergmastaflex Jun 10, 2015
25e9f87
Merge pull request #2 from moneydesktop/ferg/syslog_fix
fergmastaflex Jun 10, 2015
230f71b
bump version to 1.0.2
fergmastaflex Jun 10, 2015
681bb0f
Fix specs
quixoten Jun 10, 2015
c260f4c
Merge branch 'master' of github.com:moneydesktop/logstasher
quixoten Jun 10, 2015
e8854d1
Fix issue where we were adding facility instead of priority when writ…
fergmastaflex Jun 17, 2015
471118c
Update README
quixoten Jun 17, 2015
e99bc64
Fix specs
fergmastaflex Jun 17, 2015
abefb74
Merge pull request #3 from moneydesktop/ferg/facility_2_priority
quixoten Jun 17, 2015
921301c
Bump version to 1.0.3
quixoten Jun 17, 2015
2ab4f8e
Require logstash-event
fergmastaflex Aug 11, 2015
e298f93
Merge pull request #4 from moneydesktop/ferg/require_logstash_event
fergmastaflex Aug 11, 2015
5262538
Move require to logstash.rb
fergmastaflex Aug 11, 2015
2a37453
Merge pull request #5 from moneydesktop/ferg/move_require
fergmastaflex Aug 11, 2015
afaa536
Bump version to 1.0.4
quixoten Aug 11, 2015
be6238c
Push defaults config above initializers
quixoten Aug 12, 2015
a40e7ec
Merge pull request #6 from moneydesktop/quixoten-patch-1
quixoten Aug 12, 2015
f38069c
Bump version to 1.0.5
quixoten Aug 12, 2015
5345296
Remove mutex from syslog device
quixoten Jun 9, 2016
01f0ebb
Merge pull request #7 from mxenabled/nomoresync
quixoten Jun 9, 2016
2adb696
Bump version to 1.1.0
quixoten Jun 9, 2016
7e236fc
Add request id to payload extract
Mar 30, 2017
4b6c4e6
Merge pull request #1 from erikanderson/add_request_id
erikanderson Mar 30, 2017
581818f
Extract request id to tie to nginx requests
Apr 4, 2017
4926030
Set to action dispatch request id
erikanderson Apr 4, 2017
51c1982
Update log_subscriber.rb
erikanderson Apr 4, 2017
02284a9
Update log_subscriber.rb
erikanderson Apr 4, 2017
2c439ac
Action dispatch request id
erikanderson Apr 4, 2017
dd586df
Merge pull request #8 from erikanderson/request_env_request_id
quixoten Apr 4, 2017
c73851c
Bump version from 1.1.0 to 1.2.0
quixoten Apr 4, 2017
0500bc2
add simple udp logging device.
Dec 14, 2021
22846b4
Merge pull request #9 from jonphilpott/add_udp_logging
film42 Jan 5, 2022
a624a18
Bump version to 1.3.0
film42 Jan 5, 2022
6e6fc6d
send namespace name through config and add it to payload
Jan 25, 2022
6266fb7
added a hashtable to part of the payload
Jan 26, 2022
b0e3035
remove extra linebreaks
Jan 26, 2022
307eb4e
remove extra linebreaks
Jan 26, 2022
89a5bca
remove extra linebreaks
Jan 26, 2022
cd63627
remove extra ','
Jan 26, 2022
40c4bcb
remove extra space
Jan 26, 2022
ece43ca
remove extra space
Jan 26, 2022
856e25c
Fix the {} check logic
Jan 26, 2022
eb394d1
Merge pull request #10 from mxenabled/gk/add_namespace_to_payload
goutkannan Jan 26, 2022
bc67524
Bump version to 1.4.0
film42 Jan 26, 2022
8ef9d5b
Add support for stdout device
sshock Jan 26, 2022
af4c9c3
Merge pull request #11 from sshock/plh/support-stdout-device
film42 Jan 26, 2022
fc7c50d
Bump version to 1.4.1
film42 Jan 26, 2022
97ec020
Add a new #log_as_json method to make it easy to log json messages
film42 Jan 27, 2022
e42a97a
Allow all types that can receive #to_json; only hash gets metadata
film42 Jan 27, 2022
a496fdf
Merge pull request #12 from mxenabled/gt/log_as_json
film42 Jan 27, 2022
a0a24a5
Bump version to 1.5.0
film42 Jan 27, 2022
8915ff4
logstasher: add option to not touch log file.
Sep 9, 2022
f29c04c
update message, don't set value of variable to test default.
Sep 9, 2022
041406f
update rspec based on feedback.
Sep 9, 2022
434a120
Merge pull request #14 from jonphilpott/cant_touch_this
film42 Sep 9, 2022
1612aa2
Bump version to 1.6.0
film42 Sep 9, 2022
e3a8c26
Merge metadata into LogStash::Events as well
film42 Jan 7, 2023
514afb2
Add test aroudn LogStash::Event types
film42 Jan 7, 2023
73566ff
Merge pull request #16 from mxenabled/gt/more_meta
film42 Jan 7, 2023
97ddd8c
Bump version to 1.7.0
film42 Jan 7, 2023
469c927
added load_from_config to make configuration simpler
skunkworker Jan 18, 2024
686039b
Merge pull request #17 from skunkworker/add_load_from_config
skunkworker Jan 22, 2024
6c47ad0
bump to v1.8.0
skunkworker Jan 22, 2024
b9114ea
Add dry-validation
brettmortensen Sep 11, 2025
47d75e8
Update dry_validation_errors value format
brettmortensen Sep 12, 2025
f17487f
Bump version to 1.9.0
brettmortensen Sep 12, 2025
1de51ac
Merge pull request #18 from mxenabled/bm/dry_validation
skunkworker Sep 12, 2025
a96df1c
Format dry-validation payload
brettmortensen Sep 17, 2025
5482c60
Merge pull request #19 from mxenabled/bm/format_payload
skunkworker Sep 17, 2025
077d021
Add support for type coercion
brettmortensen Sep 18, 2025
8d8b3dc
Merge pull request #20 from mxenabled/bm/coercion
skunkworker Sep 19, 2025
a1f6250
fix spec kwargs for newer ruby versions
skunkworker Apr 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,9 @@ source "https://rubygems.org"
gemspec

group :test do
gem 'growl'
gem 'guard'
gem 'guard-rspec'
gem 'rails', "~> #{ENV["RAILS_VERSION"] || "3.2.0"}"
gem 'rb-fsevent', '~> 0.9'
gem 'rcov', :platforms => :mri_18
gem 'redis', :require => false
gem 'simplecov', :platforms => :mri_19, :require => false
gem 'pry-nav'
gem 'rails'
gem 'timecop'

gem 'syslog'
end
10 changes: 0 additions & 10 deletions Guardfile

This file was deleted.

21 changes: 21 additions & 0 deletions LICENSE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
The MIT License (MIT)

Copyright (c) 2014 Shadab Ahmed

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
88 changes: 42 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,22 @@
# Logstasher [![Gem Version](https://badge.fury.io/rb/logstasher.png)](http://badge.fury.io/rb/logstasher) [![Build Status](https://secure.travis-ci.org/shadabahmed/logstasher.png)](https://secure.travis-ci.org/shadabahmed/logstasher)
### Awesome Logging for Rails !!
# Logstasher

This gem is heavily inspired from [lograge](https://github.com/roidrage/lograge), but it's focused on one thing and one thing only. That's making your logs awesome like this:
**Awesome Logging for Rails**

[![Awesome Logs](http://i.imgur.com/zZXWQNp.png)](http://i.imgur.com/zZXWQNp.png)
## History

How it's done ?
This is a fork of
[shadabahmed/logstasher](https://github.com/shadabahmed/logstasher). It has
been updated to use the [latest event
schema](https://logstash.jira.com/browse/LOGSTASH-675) and customized to better
fit the needs of MoneyDesktop--though it is still generally useful. It is not
backward compatible with its progenitor.

By, using these awesome tools:
* [Logstash](http://logstash.net) - Store and index your logs
* [Kibana](http://kibana.org/) - for awesome visualization. This is optional though, and you can use any other visualizer
## Purpose

To know how to setup these tools - visit my [blog](http://shadabahmed.com/blog/2013/04/30/logstasher-for-awesome-rails-logging)
This gem makes it easy to generate logstash compatible logs for your rails app.

## About logstasher

This gem purely focuses on how to generate logstash compatible logs i.e. *logstash json event format*, without any overhead. Infact, logstasher logs to a separate log file named `logstash_<environment>.log`.
The reason for this separation:
* To have a pure json log file
* Prevent any logger messages(e.g. info) getting into our pure json logs

Before **logstasher** :

```
A request that looks like this in your `production.log`:
```text
Started GET "/login" for 10.109.10.135 at 2013-04-30 08:59:01 -0400
Processing by SessionsController#new as HTML
Rendered sessions/new.html.haml within layouts/application (4.3ms)
Expand All @@ -34,68 +28,70 @@ Processing by SessionsController#new as HTML
Completed 200 OK in 532ms (Views: 62.4ms | ActiveRecord: 0.0ms | ND API: 0.0ms)
```

After **logstasher**:

```
{"@source":"unknown","@tags":["request"],"@fields":{"method":"GET","path":"/","format":"html","controller":"file_servers"
Will look like this in your `logstash_production.log`:
```json
{"tags":["request"],"method":"GET","path":"/","format":"html","controller":"file_servers"
,"action":"index","status":200,"duration":28.34,"view":25.96,"db":0.88,"ip":"127.0.0.1","route":"file_servers#index",
"parameters":"","ndapi_time":null,"uuid":"e81ecd178ed3b591099f4d489760dfb6","user":"shadab_ahmed@abc.com",
"site":"internal"},"@timestamp":"2013-04-30T13:00:46.354500+00:00"}
"site":"internal","@timestamp":"2013-04-30T13:00:46.354500+00:00","@version":"1"}
```

By default, the older format rails request logs are disabled, though you can enable them.
From there, it's trivial to forward them to your logstash indexer. You can even
use the included redis log device to send the logs directly to a redis broker
instead.

## Installation

In your Gemfile:

gem 'logstasher'
gem 'md-logstasher'

### Configure your `<environment>.rb` e.g. `development.rb`

# Enable the logstasher logs for the current environment
config.logstasher.enabled = true

# This line is optional if you do not want to suppress app logs in your <environment>.log
config.logstasher.suppress_app_log = false
# Optionally silience the standard logging to <environment>.log
config.logstasher.silence_standard_logging = true

## Logging params

## Logging params hash
By default, Logstasher will add params as a JSON encoded string. To disable,
add the following to your `<environment>.rb`

Logstasher can be configured to log the contents of the params hash. When enabled, the contents of the params hash (minus the ActionController internal params)
will be added to the log as a deep hash. This can cause conflicts within the Elasticsearch mappings though, so should be enabled with care. Conflicts will occur
if different actions (or even different applications logging to the same Elasticsearch cluster) use the same params key, but with a different data type (e.g. a
string vs. a hash). This can lead to lost log entries. Enabling this can also significantly increase the size of the Elasticsearch indexes.
# Disable logging of request parameters
config.logstasher.include_parameters = false

To enable this, add the following to your `<environment>.rb`
If you want the parameters to be sent unserialized, so they are indexed and
searchable individually in elasticsearch, skip the serialization step.

# Log parameters individually
config.logstasher.serialize_parameters = false

# Enable logging of controller params
config.logstasher.log_controller_parameters = true

## Adding custom fields to the log

Since some fields are very specific to your application for e.g. *user_name*, so it is left upto you, to add them. Here's how to add those fields to the logs:
Since some fields are very specific to your application, e.g., *user_name*,
it is left upto you to add them. Here's how to do it:

# Create a file - config/initializers/logstasher.rb

if LogStasher.enabled
LogStasher.add_custom_fields do |fields|
# This block is run in application_controller context,
# so you have access to all controller methods
LogStasher.append_fields do |fields|
# This block is run within the context of the controller handling the
# request, so you have access to all its methods and instance variables
fields[:user] = current_user && current_user.mail
fields[:site] = request.path =~ /^\/api/ ? 'api' : 'user'

# If you are using custom instrumentation, just add it to logstasher custom fields
LogStasher.custom_fields << :myapi_runtime
end
end

## Versions
All versions require Rails 3.0.x and higher and Ruby 1.9.2+. Tested on Rails 4 and Ruby 2.0
All versions require Rails 3.0.x and higher and Ruby 1.9.2+. Tested on Rails 4
and Ruby 2.0

## Development
- Run tests - `rake`
- Generate test coverage report - `rake coverage`. Coverage report path - coverage/index.html

## Copyright

Copyright (c) 2013 Shadab Ahmed, released under the MIT license
Copyright (c) 2014 Shadab Ahmed, released under the MIT license
25 changes: 2 additions & 23 deletions Rakefile
Original file line number Diff line number Diff line change
@@ -1,42 +1,21 @@
require "bundler/gem_tasks"
require 'bundler/gem_tasks'
require 'rspec/core/rake_task'

# Add default task. When you type just rake command this would run. Travis CI runs this. Making this run spec
desc 'Default: run specs.'
task :default => :spec

# Defining spec task for running spec
desc "Run specs"
RSpec::Core::RakeTask.new('spec') do |spec|
# Pattern filr for spec files to run. This is default btw.
spec.pattern = "./spec/**/*_spec.rb"
end

# Run the rdoc task to generate rdocs for this gem
require 'rdoc/task'
RDoc::Task.new do |rdoc|
require "logstasher/version"
require 'logstasher/version'
version = LogStasher::VERSION

rdoc.rdoc_dir = 'rdoc'
rdoc.title = "logstasher #{version}"
rdoc.rdoc_files.include('README*')
rdoc.rdoc_files.include('lib/**/*.rb')
end

# Code coverage tasks. Different for Ruby 1.8 vs 1.9
if RUBY_VERSION =~ /^1\.8/
# Ruby 1.8 uses rcov for code coverage
RSpec::Core::RakeTask.new(:coverage) do |spec|
spec.pattern = 'spec/**/*_spec.rb'
spec.rcov = true
spec.rcov_opts = %w{--exclude pkg\/,spec\/,features\/}
end
else
# Ruby 1.9+ using simplecov. Note: Simplecov config defined in spec_helper
desc "Code coverage detail"
task :coverage do
ENV['COVERAGE'] = "true"
Rake::Task['spec'].execute
end
end
Loading