Add upstart example

Update link to example.cachet-monitor.service under Init Script section

CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at management@castawaylabs.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

cli/main.go

@@ -23,7 +23,6 @@ const usage = `cachet-monitor
 Usage:
   cachet-monitor (-c PATH | --config PATH) [--log=LOGPATH] [--name=NAME] [--immediate]
   cachet-monitor -h | --help | --version
-  cachet-monitor print-config
 
 Arguments:
   PATH     path to config.json
@@ -39,7 +38,6 @@ Options:
   -h --help                      Show this screen.
   --version                      Show version
   --immediate                    Tick immediately (by default waits for first defined interval)
-  print-config                   Print example configuration
   
 Environment varaibles:
   CACHET_API      override API url from configuration
@@ -179,14 +177,6 @@ func getConfiguration(path string) (*cachet.CachetMonitor, error) {
 			var s cachet.DNSMonitor
 			err = mapstructure.Decode(rawMonitor, &s)
 			t = &s
-		case "icmp":
-			var s cachet.ICMPMonitor
-			err = mapstructure.Decode(rawMonitor, &s)
-			t = &s
-		case "tcp":
-			var s cachet.TCPMonitor
-			err = mapstructure.Decode(rawMonitor, &s)
-			t = &s
 		default:
 			logrus.Errorf("Invalid monitor type (index: %d) %v", index, monType)
 			continue

dns.go

@@ -1,5 +1,121 @@
 package cachet
 
+import (
+	"net"
+	"regexp"
+	"strings"
+
+	"github.com/Sirupsen/logrus"
+	"github.com/miekg/dns"
+)
+
+type DNSAnswer struct {
+	Regex  string
+	regexp *regexp.Regexp
+	Exact  string
+}
+
 type DNSMonitor struct {
 	AbstractMonitor `mapstructure:",squash"`
+
+	// IP:port format or blank to use system defined DNS
+	DNS string
+
+	// A(default), AAAA, MX, ...
+	Question string
+	question uint16
+
+	Answers []DNSAnswer
+}
+
+func (monitor *DNSMonitor) Validate() []string {
+	errs := monitor.AbstractMonitor.Validate()
+
+	if len(monitor.DNS) == 0 {
+		config, _ := dns.ClientConfigFromFile("/etc/resolv.conf")
+		if len(config.Servers) > 0 {
+			monitor.DNS = net.JoinHostPort(config.Servers[0], config.Port)
+		}
+	}
+
+	if len(monitor.DNS) == 0 {
+		monitor.DNS = "8.8.8.8:53"
+	}
+
+	if len(monitor.Question) == 0 {
+		monitor.Question = "A"
+	}
+	monitor.Question = strings.ToUpper(monitor.Question)
+
+	monitor.question = findDNSType(monitor.Question)
+	if monitor.question == 0 {
+		errs = append(errs, "Could not look up DNS question type")
+	}
+
+	for i, a := range monitor.Answers {
+		if len(a.Regex) > 0 {
+			monitor.Answers[i].regexp, _ = regexp.Compile(a.Regex)
+		}
+	}
+
+	return errs
+}
+
+func (monitor *DNSMonitor) test() bool {
+	m := new(dns.Msg)
+	m.SetQuestion(dns.Fqdn(monitor.Target), monitor.question)
+	m.RecursionDesired = true
+
+	c := new(dns.Client)
+	r, _, err := c.Exchange(m, monitor.DNS)
+	if err != nil {
+		logrus.Warnf("DNS error: %v", err)
+		return false
+	}
+
+	if r.Rcode != dns.RcodeSuccess {
+		return false
+	}
+
+	for _, check := range monitor.Answers {
+		found := false
+		for _, answer := range r.Answer {
+			found = matchAnswer(answer, check)
+			if found {
+				break
+			}
+		}
+
+		if !found {
+			logrus.Warnf("DNS check failed: %v. Not found in any of %v", check, r.Answer)
+			return false
+		}
+	}
+
+	return true
+}
+
+func findDNSType(t string) uint16 {
+	for rr, strType := range dns.TypeToString {
+		if t == strType {
+			return rr
+		}
+	}
+
+	return 0
+}
+
+func matchAnswer(answer dns.RR, check DNSAnswer) bool {
+	fields := []string{}
+	for i := 0; i < dns.NumField(answer); i++ {
+		fields = append(fields, dns.Field(answer, i+1))
+	}
+
+	str := strings.Join(fields, " ")
+
+	if check.regexp != nil {
+		return check.regexp.Match([]byte(str))
+	}
+
+	return str == check.Exact
 }

example.cachet-monitor.service

@@ -0,0 +1,20 @@
+[Unit]
+Description=Cachet Monitor
+After=syslog.target
+After=network.target
+#After=mysqld.service
+#After=postgresql.service
+#After=memcached.service
+#After=redis.service
+
+[Service]
+Type=simple
+User=root
+Group=root
+WorkingDirectory=/root
+ExecStart=/root/cachet-monitor -c /etc/cachet-monitor.yaml
+Restart=always
+Environment=USER=root HOME=/root
+
+[Install]
+WantedBy=multi-user.target

example.config.json

@@ -2,21 +2,58 @@
   "api": {
     "url": "https://demo.cachethq.io/api/v1",
     "token": "9yMHsdioQosnyVK4iCVR",
-    "insecure": true
+    "insecure": false
   },
+  "date_format": "02/01/2006 15:04:05 MST",
   "monitors": [
     {
       "name": "google",
-      "url": "https://google.com",
-      "threshold": 80,
+      "target": "https://google.com",
+      "strict": true,
+      "method": "POST",
       "component_id": 1,
-      "interval": 10,
-      "timeout": 5,
+      "metric_id": 4,
+      "template": {
+        "investigating": {
+          "subject": "{{ .Monitor.Name }} - {{ .SystemName }}",
+          "message": "{{ .Monitor.Name }} check **failed** (server time: {{ .now }})\n\n{{ .FailReason }}"
+        },
+        "fixed": {
+          "subject": "I HAVE BEEN FIXED"
+        }
+      },
+      "interval": 1,
+      "timeout": 1,
+      "threshold": 80,
       "headers": {
         "Authorization": "Basic <hash>"
       },
       "expected_status_code": 200,
-      "strict_tls": true
+      "expected_body": "P.*NG"
+    },
+    {
+      "name": "dns",
+      "target": "matej.me.",
+      "question": "mx",
+      "type": "dns",
+      "component_id": 2,
+      "interval": 1,
+      "timeout": 1,
+      "dns": "8.8.4.4:53",
+      "answers": [
+        {
+          "regex": "[1-9] alt[1-9].aspmx.l.google.com."
+        },
+        {
+          "exact": "10 aspmx2.googlemail.com."
+        },
+        {
+          "exact": "1 aspmx.l.google.com."
+        },
+        {
+          "exact": "10 aspmx3.googlemail.com."
+        }
+      ]
     }
   ]
 }

example.config.yml

@@ -1,14 +1,65 @@
 api:
+  # cachet url
   url: https://demo.cachethq.io/api/v1
+  # cachet api token
   token: 9yMHsdioQosnyVK4iCVR
+  insecure: false
+# https://golang.org/src/time/format.go#L57
+date_format: 02/01/2006 15:04:05 MST
 monitors:
+  # http monitor example
   - name: google
+    # test url
     target: https://google.com
-    threshold: 80
+    # strict certificate checking for https
+    strict: true
+    # HTTP method
+    method: POST
+    
+    # set to update component (either component_id or metric_id are required)
     component_id: 1
-    interval: 10
-    timeout: 5
+    # set to post lag to cachet metric (graph)
+    metric_id: 4
+
+    # custom templates (see readme for details)
+    template:
+      investigating:
+        subject: "{{ .Monitor.Name }} - {{ .SystemName }}"
+        message: "{{ .Monitor.Name }} check **failed** (server time: {{ .now }})\n\n{{ .FailReason }}"
+      fixed:
+        subject: "I HAVE BEEN FIXED"
+    
+    # seconds between checks
+    interval: 1
+    # seconds for timeout
+    timeout: 1
+    # If % of downtime is over this threshold, open an incident
+    threshold: 80
+
+    # custom HTTP headers
     headers:
       Authorization: Basic <hash>
+    # expected status code (either status code or body must be supplied)
     expected_status_code: 200
-    strict: true
+    # regex to match body
+    expected_body: "P.*NG"
+  # dns monitor example
+  - name: dns
+    # fqdn
+    target: matej.me.
+    # question type (A/AAAA/CNAME/...)
+    question: mx
+    type: dns
+    # set component_id/metric_id
+    component_id: 2
+    # poll every 1s
+    interval: 1
+    timeout: 1
+    # custom DNS server (defaults to system)
+    dns: 8.8.4.4:53
+    answers:
+      # exact/regex check
+      - regex: [1-9] alt[1-9].aspmx.l.google.com.
+      - exact: 10 aspmx2.googlemail.com.
+      - exact: 1 aspmx.l.google.com.
+      - exact: 10 aspmx3.googlemail.com.

example.upstart.conf

@@ -0,0 +1,14 @@
+description "Cachet Monitor"
+
+start on startup
+
+env USER=root
+env HOME=/root
+
+setuid root
+setgid root
+chdir /root
+
+script
+	exec cachet-monitor -c /cachet-monitor.json --immediate
+end script

icmp.go

@@ -1,5 +0,0 @@
-package cachet
-
-type ICMPMonitor struct {
-	AbstractMonitor `mapstructure:",squash"`
-}

monitor.go

@@ -28,7 +28,7 @@ type AbstractMonitor struct {
 	Name   string
 	Target string
 
-	// (default)http, tcp, dns, icmp
+	// (default)http / dns
 	Type   string
 	Strict bool
 
@@ -50,10 +50,10 @@ type AbstractMonitor struct {
 
 	// lag / average(lagHistory) * 100 = percentage above average lag
 	// PerformanceThreshold sets the % limit above which this monitor will trigger degraded-performance
-	PerformanceThreshold float32
+	// PerformanceThreshold float32
 
 	history []bool
-	lagHistory     []float32
+	// lagHistory     []float32
 	lastFailReason string
 	incident       *Incident
 	config         *CachetMonitor
@@ -141,7 +141,6 @@ func (mon *AbstractMonitor) ClockStop() {
 
 func (mon *AbstractMonitor) test() bool { return false }
 
-// TODO: test
 func (mon *AbstractMonitor) tick(iface MonitorInterface) {
 	reqStart := getMs()
 	up := iface.test()
@@ -153,7 +152,7 @@ func (mon *AbstractMonitor) tick(iface MonitorInterface) {
 	}
 
 	if len(mon.history) == histSize-1 {
-		logrus.Warnf("%v is now saturated\n", mon.Name)
+		logrus.Warnf("%v is now saturated", mon.Name)
 	}
 	if len(mon.history) >= histSize {
 		mon.history = mon.history[len(mon.history)-(histSize-1):]

monitor_test.go

@@ -0,0 +1,7 @@
+package cachet
+
+import (
+	"testing"
+)
+
+func TestAnalyseData(t *testing.T) {}

readme.md

@@ -1,103 +1,176 @@
 ![screenshot](https://castawaylabs.github.io/cachet-monitor/screenshot.png)
 
-Features
---------
+## Features
 
 - [x] Creates & Resolves Incidents
-- [x] Check URLs by response code and/or body contents
 - [x] Posts monitor lag to cachet graphs
+- [x] HTTP Checks (body/status code)
+- [x] DNS Checks
 - [x] Updates Component to Partial Outage
-- [x] Updates Component to Major Outage if already in Partial Outage (works well with distributed monitoring)
+- [x] Updates Component to Major Outage if already in Partial Outage (works with distributed monitors)
 - [x] Can be run on multiple servers and geo regions
 
-Configuration
--------------
+## Example Configuration
 
-```
-{
-  // URL for the API. Note: Must end with /api/v1
-  "api_url": "https://<cachet domain>/api/v1",
-  // Your API token for Cachet
-  "api_token": "<cachet api token>",
-  // optional, false default, set if your certificate is self-signed/untrusted
-  "insecure_api": false,
-  "monitors": [{
-    // required, friendly name for your monitor
-    "name": "Name of your monitor",
-    // required, url to probe
-    "url": "Ping URL",
-    // optional, http method (defaults GET)
-    "method": "get",
-    // optional, http Headers to add (default none)
-    "headers": [
-        // specify Name and Value of Http-Header, eg. Authorization
-        { "name": "Authorization", "value": "Basic <hash>" }
-    ],
-    // self-signed ssl certificate
-    "strict_tls": true,
-    // seconds between checks
-    "interval": 10,
-    // seconds for http timeout
-    "timeout": 5,
-    // post lag to cachet metric (graph)
-    // note either metric ID or component ID are required
-    "metric_id": <metric id>,
-    // post incidents to this component
-    "component_id": <component id>,
-    // If % of downtime is over this threshold, open an incident
-    "threshold": 80,
-    // optional, expected status code (either status code or body must be supplied)
-    "expected_status_code": 200,
-    // optional, regular expression to match body content
-    "expected_body": "P.*NG"
-  }],
-  // optional, system name to identify bot (uses hostname by default)
-  "system_name": "",
-  // optional, defaults to stdout
-  "log_path": ""
-}
+**Note:** configuration can be in json or yaml format. [`example.config.json`](https://github.com/CastawayLabs/cachet-monitor/blob/master/example.config.json), [`example.config.yaml`](https://github.com/CastawayLabs/cachet-monitor/blob/master/example.config.yml) files.
+
+```yaml
+api:
+  # cachet url
+  url: https://demo.cachethq.io/api/v1
+  # cachet api token
+  token: 9yMHsdioQosnyVK4iCVR
+  insecure: false
+# https://golang.org/src/time/format.go#L57
+date_format: 02/01/2006 15:04:05 MST
+monitors:
+  # http monitor example
+  - name: google
+    # test url
+    target: https://google.com
+    # strict certificate checking for https
+    strict: true
+    # HTTP method
+    method: POST
+    
+    # set to update component (either component_id or metric_id are required)
+    component_id: 1
+    # set to post lag to cachet metric (graph)
+    metric_id: 4
+
+    # custom templates (see readme for details)
+    # leave empty for defaults
+    template:
+      investigating:
+        subject: "{{ .Monitor.Name }} - {{ .SystemName }}"
+        message: "{{ .Monitor.Name }} check **failed** (server time: {{ .now }})\n\n{{ .FailReason }}"
+      fixed:
+        subject: "I HAVE BEEN FIXED"
+    
+    # seconds between checks
+    interval: 1
+    # seconds for timeout
+    timeout: 1
+    # If % of downtime is over this threshold, open an incident
+    threshold: 80
+
+    # custom HTTP headers
+    headers:
+      Authorization: Basic <hash>
+    # expected status code (either status code or body must be supplied)
+    expected_status_code: 200
+    # regex to match body
+    expected_body: "P.*NG"
+  # dns monitor example
+  - name: dns
+    # fqdn
+    target: matej.me.
+    # question type (A/AAAA/CNAME/...)
+    question: mx
+    type: dns
+    # set component_id/metric_id
+    component_id: 2
+    # poll every 1s
+    interval: 1
+    timeout: 1
+    # custom DNS server (defaults to system)
+    dns: 8.8.4.4:53
+    answers:
+      # exact/regex check
+      - regex: [1-9] alt[1-9].aspmx.l.google.com.
+      - exact: 10 aspmx2.googlemail.com.
+      - exact: 1 aspmx.l.google.com.
+      - exact: 10 aspmx3.googlemail.com.
 ```
 
-Installation
-------------
+## Installation
 
 1. Download binary from [release page](https://github.com/CastawayLabs/cachet-monitor/releases)
-2. Create your configuration ([example](https://raw.githubusercontent.com/CastawayLabs/cachet-monitor/master/example.config.json))
-3. `cachet-monitor -c /etc/cachet-monitor.config.json`
+2. Create a configuration
+3. `cachet-monitor -c /etc/cachet-monitor.yaml`
 
 pro tip: run in background using `nohup cachet-monitor 2>&1 > /var/log/cachet-monitor.log &`
 
 ```
-Usage of cachet-monitor:
-  -c="/etc/cachet-monitor.config.json": Config path
-  -log="": Log path
-  -name="": System Name
+Usage:
+  cachet-monitor (-c PATH | --config PATH) [--log=LOGPATH] [--name=NAME] [--immediate]
+  cachet-monitor -h | --help | --version
+
+Arguments:
+  PATH     path to config.json
+  LOGPATH  path to log output (defaults to STDOUT)
+  NAME     name of this logger
+
+Examples:
+  cachet-monitor -c /root/cachet-monitor.json
+  cachet-monitor -c /root/cachet-monitor.json --log=/var/log/cachet-monitor.log --name="development machine"
+
+Options:
+  -c PATH.json --config PATH     Path to configuration file
+  -h --help                      Show this screen.
+  --version                      Show version
+  --immediate                    Tick immediately (by default waits for first defined interval)
+  
+Environment varaibles:
+  CACHET_API      override API url from configuration
+  CACHET_TOKEN    override API token from configuration
+  CACHET_DEV      set to enable dev logging
 ```
 
-Environment variables
----------------------
+## Init script
 
-| Name         | Example Value                  | Description                 |
-| ------------ | ------------------------------ | --------------------------- |
-| CACHET_API   | http://demo.cachethq.io/api/v1 | URL endpoint for cachet api |
-| CACHET_TOKEN | APIToken123                    | API Authentication token    |
-| CACHET_DEV   | 1                              | Strips logging              |
+If your system is running systemd (like Debian, Ubuntu 16.04, Fedora or Archlinux) you can use the provided example file: [example.cachet-monitor.service](https://github.com/CastawayLabs/cachet-monitor/blob/master/example.cachet-monitor.service).
 
-Vision and goals
-----------------
+1. Simply put it in the right place with `cp example.cachet-monitor.service /etc/systemd/system/cachet-monitor.service`
+2. Then do a `systemctl daemon-reload` in your terminal to update Systemd configuration
+3. Finally you can start cachet-monitor on every startup with `systemctl enable cachet-monitor.service`! 👍
+
+## Templates
+
+This package makes use of [`text/template`](https://godoc.org/text/template). [Default HTTP template](https://github.com/CastawayLabs/cachet-monitor/blob/master/http.go#L14)
+
+The following variables are available:
+
+| Root objects  |
+| ------------- | -----------------
+| `.SystemName` | system name
+| `.API`        | `api` object from configuration
+| `.Monitor`    | `monitor` object from configuration
+| `.now`        | formatted date string
+
+| Monitor variables  |
+| ------------------ |
+| `.Name`            |
+| `.Target`          |
+| `.Type`            |
+| `.Strict`          |
+| `.MetricID`        |
+| ...                |
+
+All monitor variables are available from `monitor.go`
+
+## Vision and goals
 
 We made this tool because we felt the need to have our own monitoring software (leveraging on Cachet).
 The idea is a stateless program which collects data and pushes it to a central cachet instance.
 
 This gives us power to have an army of geographically distributed loggers and reveal issues in both latency & downtime on client websites.
 
-Package usage
--------------
+## Package usage
 
-When using `cachet-monitor` as a package in another program, you should follow what `cli/main.go` does. It is important to call `ValidateConfiguration` on `CachetMonitor` and all the monitors inside.
+When using `cachet-monitor` as a package in another program, you should follow what `cli/main.go` does. It is important to call `Validate` on `CachetMonitor` and all the monitors inside.
 
 [API Documentation](https://godoc.org/github.com/CastawayLabs/cachet-monitor)
 
+# Contributions welcome
+
+We'll happily accept contributions for the following (non exhaustive list).
+
+- Implement ICMP check
+- Implement TCP check
+- Any bug fixes / code improvements
+- Test cases
+
 ## License
 
 MIT License

tcp.go

@@ -1,15 +0,0 @@
-package cachet
-
-type TCPMonitor struct {
-	AbstractMonitor `mapstructure:",squash"`
-
-	// same as output from net.JoinHostPort
-	// defaults to parsed config from /etc/resolv.conf when empty
-	DNSServer string
-
-	// Will be converted to FQDN
-	Domain string
-	Type   string
-	// expected answers (regex)
-	Expect []string
-}