Shooting in the dark won't take you far. If you have no clue what’s happening in your application, all the automation and management in the world will only help you so much. Certain tools provide insight into the workings and status of your app. Some provide a dashboard for error tracking, some reach out to notify you when something needs your attention, and others test your app on a continuous basis. You'll find tools to monitor your application for the operations side with issues like uptime, and for the developer-side with issues like exceptions and errors.
Rackspace’s Airbrake comes with a web based dashboard to capture and track the errors and exceptions in your application. The dashboard will give you a real insight into resolved and unresolved errors, along with the information regarding each one of them and an overview graph. Each error includes the environment in which it happened, the moment it was first identified, error type and more.
It provides an easy integration with other applications like Github, Hipchat, Asana and many other services. If you have a custom app you want to integrate into Airbrake, use Airbrake’s public API or their web hooks functionality to your advantage.
Airbrake comes with a wonderful user interface, giving you easy access to features like application insights and intelligent error grouping, session details, and backtraces. All these allow you to squash those bugs at a much faster pace.
Errbit happens to be a self-hosted error catcher for Airbrake. By using Airbrake's error tracking tools to send error logs to your own ErrBit based web service, it lets you keep errors off Airbrake's servers. This is extremely useful for when errors contain sensitive client data.
You won't have to worry about errors affecting your page load times, by including Airbrake's client side error tracking JS into your code. It won't cause severe delays for your site's visitors since the library loads asynchronously.
Its setup can prove slightly difficult.
It comes with a less friendly interface than other tools.
Filtering for personally identifiable information can be impossible, so security can be a concern, requiring a manual effort on your part.
You might face issues tracking errors.
NewRelic is used to monitor the application environment and the hardware. Out of the box, you get information about performance, method usages, app downtimes, and lots of other useful data. This is a great tool for sites which have a lot of server side processing.
New Relic comes with a great interface which helps you debug the server side of performance issues. It gives a detailed report on issues by keeping track of the entire run time stack, while running on your servers. This tool will allow you to quickly pinpoint performance issues, like which query is called too many times, which queries take too long and which function is running for way too long. All these issues become visible for you to fix.
New Relic comes with a free trial. They charge you based on the number of hosts you install the service on, allowing you to get a lot more for your money. This way you don't have to worry about traffic scaling dictating your costs.
While the interface is better than some other performance sites, it can get a little tricky. There are a lot of navigation items, and almost everything on the page is clickable. It is easy to swing between overall trends and specific threads without realizing it.
The tool does not let you debug very well what on the web page was causing the page to render slow. It just notifies you about the web page rendering slowly. You get the network, DOM processing, web application time, and page rendering times, but no real way to see exactly what is causing all these issues.
New Relic works by setting acceptance thresholds and then sending you detailed reports on requests don't come under the acceptable range. Now if you want to look at detailed reports which fall under acceptable range, you can't. Meaning to continue improving your site, you have to lower your threshold constantly.
No single tool will fit all your needs. There's just too many specialized things to put into a single monitoring app. However, some specialized apps play well together to give you great monitoring coverage of your Rails apps and infrastructure. Keeping your stack running could prove to be a time-sink and monitoring software is complicated. The last thing you want is unreliable software monitoring your apps. The bigger your app gets, chances are it will need to integrate with a whole lot of external services, APIs, and providers.
If you want an all-in-one solution, go with NewRelic. It will probably be the most expensive, and its exception tracking capabilities are not as sophisticated as Airbrake. That said, it provides nice analytics for slow-performing endpoints, slow SQL queries, time spent in ruby GC, etc. Error tracking tools are an essential part of the application life cycle, and each one can introduce you to a different way of using your application.