Here at Kisko we make heavy use of Heroku; almost all of our clients’ and our own apps run there. As we have so many apps (a few of which are moderately busy), we generate quite a lot of logs. Our own WODconnect generates an average of 600-700 megabytes of logs per day.
We looked into all the Heroku compatible logging services and eventually came to the conclusion that none of them provided what we wanted at a price point that we were willing to pay.
In the course of my “research” I also looked at AWS Cloudwatch Logs which has very reasonable pay-as-you-go pricing. If we could use Cloudwatch to store our Heroku logs, it could save us several hundred euros per month compared to some of the other providers. The catch was that Heroku can’t “natively” stream logs to Cloudwatch.
The solution was to create a small web app that receives logs from Heroku (over HTTPS) and forwards them to Cloudwatch. I happened to mention this to LibPixel‘s Joao Cardoso and we came to an agreement that he’d write the web app for us as he already had some experience with the Cloudwatch APIs.
Since we found this piece of tech so handy, we’ve decided to open source it. So if you’d like to run your own instance of this, check out the heroku-cloudwatch-drain repository on GitHub.