carbon-cache supervisord FATAL start2015-07-26T11:02:54.755910-05:00 2015-07-26T11:02:54.755910-05:00 2015-08-23T09:59:22.776083-05:00
Getting carbon-cache working with supervisord is a common thing you need to do when setting up Graphite or even the lighter weight graphite-api projects. Lots of the tutorials out there explain how to get this setup and most use supervisor to handle running carbon-cache.py.
When installing this recently via SaltStack, I ran into a peculiar error:
root@logs:/opt/statsd-0.7.2# supervisorctl status carbon FATAL Exited too quickly (process log may have details)
The supervisor logs weren’t much help either:
carbon-cache (instance a) is already running with pid 17044 carbon-cache (instance a) is already running with pid 17044 carbon-cache (instance a) is already running with pid 17044 carbon-cache (instance a) is already running with pid 17044
So it turns out that is a normal situation with supervisor working with scripts that daemonize themselves. Don’t get me wrong, carbon-cache is started and running just perfectly when you see this fatal error in your supervisor status. It’s just supervisor can’t tell that it’s running correctly. To correct this situation you need to run carbon-cache.py with the —debug flag enabled like so:
[program:carbon] command=/opt/graphite/bin/carbon-cache.py --debug start
Hopefully this help some of you to more quickly diagnose this issue that I ran into today.