We wanted to have a service that would optionally tail logs from #Kubernetes for apps we deploy and report them over UDP syslog—in an existing JSON log format that we use from #Mesos.

• It should make the log scrape/relay decision based on Annotations on the Pods.
• It should rate limit by *pod* and not by host/node so that we don't overrun our log provider (e.g. when someone forgets to turn of debug logging) or starve other apps on the same node from being able to send their logs.
• It should report rate limiting to our metrics system so we can track which pods are getting limited.

There was nothing that we could find that was able to do all of that. So I spent the last two days writing it in #Golang and we're doing initial deployment of that as a DaemonSet. Seems to work nicely 🎉