Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds groupCounters feature #134

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Adds groupCounters feature #134

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ metrics:
The Dropwizard InfluxDb reporter factory comes with some sensible defaults. Some
of the defaults are for reducing storage requirements in the database (1m
precision reporting every minute and some default exclusions for
example). Others are to get a better fit for the influx model (gauge grouping
example). Others are to get a better fit for the influx model (gauge/counter grouping
and measurement mapping).

### Measurement/Metric Mappings
Expand All @@ -115,9 +115,9 @@ configured by `tagsTransformer`. By default the `ClassBasedTransformer` is used
and it creates tha following tags: `metricName`, `package`, `className`, and
`method`.

### Gauge Grouping
### Gauge and Counter Grouping

Gauge grouping is enabled by default. This will turn a set of metrics into a
Gauge/Counter grouping is enabled by default (both configured separately). This will turn a set of metrics into a
measurement with separate fields like so:


Expand Down Expand Up @@ -189,6 +189,7 @@ fields:
timers: [p50, p75, p95, p99, p999, m1_rate]
meters: [m1_rate]
groupGauges: yes
groupCounters: yes
# exclude some pre-calculated metrics
excludes:
- ch.qos.logback.core.Appender.debug
Expand Down
2 changes: 2 additions & 0 deletions ...wizard-metrics-influxdb/src/main/java/com/izettle/metrics/dw/InfluxDbReporterFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,8 @@ public class InfluxDbReporterFactory extends BaseReporterFactory {

private boolean groupGauges = true;

private boolean groupCounters = false;

private ImmutableMap<String, String> measurementMappings = ImmutableMap.of();

private ImmutableMap<String, String> defaultMeasurementMappings = ImmutableMap.<String, String>builder()
Expand Down
71 changes: 51 additions & 20 deletions metrics-influxdb/src/main/java/com/izettle/metrics/influxdb/InfluxDbReporter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import com.codahale.metrics.Histogram;
import com.codahale.metrics.Meter;
import com.codahale.metrics.Metered;
import com.codahale.metrics.Metric;
import com.codahale.metrics.MetricFilter;
import com.codahale.metrics.MetricRegistry;
import com.codahale.metrics.ScheduledReporter;
Expand All @@ -20,6 +21,7 @@
import java.util.SortedMap;
import java.util.TreeMap;
import java.util.concurrent.TimeUnit;
import java.util.function.Function;
import java.util.regex.Pattern;
import java.util.regex.PatternSyntaxException;

Expand All @@ -37,6 +39,7 @@ public static class Builder {
private MetricFilter filter;
private boolean skipIdleMetrics;
private boolean groupGauges;
private boolean groupCounters;
private Set<String> includeTimerFields;
private Set<String> includeMeterFields;
private Map<String, Pattern> measurementMappings;
Expand Down Expand Up @@ -119,6 +122,20 @@ public Builder groupGauges(boolean groupGauges) {
return this;
}

/**
* Group counters by metric name with field names as everything after the last period
* <p>
* If there is no `.', field name will be `value'. If the metric name terminates in a `.' field name will be empty.
* </p>
*
* @param groupCounters true/false for whether to group counters or not
* @return {@code this}
*/
public Builder groupCounters(boolean groupCounters) {
this.groupCounters = groupCounters;
return this;
}

/**
* Only report timer fields in the set.
*
Expand Down Expand Up @@ -172,7 +189,7 @@ public Builder tagsTransformer(Transformer tagsTransformer) {
public InfluxDbReporter build(final InfluxDbSender influxDb) {
return new InfluxDbReporter(
registry, influxDb, tags, rateUnit, durationUnit, filter, skipIdleMetrics,
groupGauges, includeTimerFields, includeMeterFields, measurementMappings, tagsTransformer
groupGauges, groupCounters, includeTimerFields, includeMeterFields, measurementMappings, tagsTransformer
);
}
}
Expand All @@ -182,6 +199,7 @@ public InfluxDbReporter build(final InfluxDbSender influxDb) {
private final boolean skipIdleMetrics;
private final Map<String, Long> previousValues;
private final boolean groupGauges;
private final boolean groupCounters;
private final Set<String> includeTimerFields;
private final Set<String> includeMeterFields;
private final Map<String, Pattern> measurementMappings;
Expand All @@ -196,6 +214,7 @@ private InfluxDbReporter(
final MetricFilter filter,
final boolean skipIdleMetrics,
final boolean groupGauges,
final boolean groupCounters,
final Set<String> includeTimerFields,
final Set<String> includeMeterFields,
final Map<String, Pattern> measurementMappings,
Expand All @@ -206,6 +225,7 @@ private InfluxDbReporter(
this.influxDb = influxDb;
this.skipIdleMetrics = skipIdleMetrics;
this.groupGauges = groupGauges;
this.groupCounters = groupCounters;
this.includeTimerFields = includeTimerFields;
this.includeMeterFields = includeMeterFields;
this.previousValues = new TreeMap<String, Long>();
Expand Down Expand Up @@ -233,9 +253,7 @@ public void report(

reportGauges(gauges, now);

for (Map.Entry<String, Counter> entry : counters.entrySet()) {
reportCounter(entry.getKey(), entry.getValue(), now);
}
reportCounters(counters, now);

for (Map.Entry<String, Histogram> entry : histograms.entrySet()) {
reportHistogram(entry.getKey(), entry.getValue(), now);
Expand All @@ -259,11 +277,24 @@ public void report(
}
}

private void reportCounters(SortedMap<String, Counter> counters, long now) {
if (groupCounters) {
Map<String, Map<String, Counter>> groupedCounters = groupMetrics(counters);
for (Map.Entry<String, Map<String, Counter>> entry : groupedCounters.entrySet()) {
reportMetricGroup(entry.getKey(), entry.getValue(), Counter::getCount, now);
}
} else {
for (Map.Entry<String, Counter> entry : counters.entrySet()) {
reportCounter(entry.getKey(), entry.getValue(), now);
}
}
}

private void reportGauges(SortedMap<String, Gauge> gauges, long now) {
if (groupGauges) {
Map<String, Map<String, Gauge>> groupedGauges = groupGauges(gauges);
Map<String, Map<String, Gauge>> groupedGauges = groupMetrics(gauges);
for (Map.Entry<String, Map<String, Gauge>> entry : groupedGauges.entrySet()) {
reportGaugeGroup(entry.getKey(), entry.getValue(), now);
reportMetricGroup(entry.getKey(), entry.getValue(), Gauge::getValue, now);
}
} else {
for (Map.Entry<String, Gauge> entry : gauges.entrySet()) {
Expand All @@ -272,9 +303,9 @@ private void reportGauges(SortedMap<String, Gauge> gauges, long now) {
}
}

private Map<String, Map<String, Gauge>> groupGauges(SortedMap<String, Gauge> gauges) {
Map<String, Map<String, Gauge>> groupedGauges = new HashMap<String, Map<String, Gauge>>();
for (Map.Entry<String, Gauge> entry : gauges.entrySet()) {
private <T extends Metric> Map<String, Map<String, T>> groupMetrics(SortedMap<String, T> metrics) {
Map<String, Map<String, T>> groupedMetrics = new HashMap<String, Map<String, T>>();
for (Map.Entry<String, T> entry : metrics.entrySet()) {
final String metricName;
String fieldName;
int lastDotIndex = entry.getKey().lastIndexOf(".");
Expand All @@ -289,23 +320,23 @@ private Map<String, Map<String, Gauge>> groupGauges(SortedMap<String, Gauge> gau
metricName = entry.getKey();
fieldName = "value";
}
Map<String, Gauge> fields = groupedGauges.get(metricName);
Map<String, T> fields = groupedMetrics.get(metricName);

if (fields == null) {
fields = new HashMap<String, Gauge>();
fields = new HashMap<String, T>();
}
fields.put(fieldName, entry.getValue());
groupedGauges.put(metricName, fields);
groupedMetrics.put(metricName, fields);
}
return groupedGauges;
return groupedMetrics;
}

private void reportGaugeGroup(String name, Map<String, Gauge> gaugeGroup, long now) {
private <T extends Metric> void reportMetricGroup(String name, Map<String, T> gaugeGroup, Function<T, Object> valueObtainer, long now) {
Map<String, Object> fields = new HashMap<String, Object>();
for (Map.Entry<String, Gauge> entry : gaugeGroup.entrySet()) {
Object gaugeValue = sanitizeGauge(entry.getValue().getValue());
if (gaugeValue != null) {
fields.put(entry.getKey(), gaugeValue);
for (Map.Entry<String, T> entry : gaugeGroup.entrySet()) {
Object metricValue = sanitizeValue(valueObtainer.apply(entry.getValue()));
if (metricValue != null) {
fields.put(entry.getKey(), metricValue);
}
}

Expand All @@ -325,7 +356,7 @@ private void reportGaugeGroup(String name, Map<String, Gauge> gaugeGroup, long n
* @param value the value to sanitize
* @return value, or null if value is a number and is finite
*/
private Object sanitizeGauge(Object value) {
private Object sanitizeValue(Object value) {
final Object finalValue;
if (value instanceof Double && (Double.isInfinite((Double) value) || Double.isNaN((Double) value))) {
finalValue = null;
Expand Down Expand Up @@ -414,7 +445,7 @@ private void reportCounter(String name, Counter counter, long now) {

private void reportGauge(String name, Gauge<?> gauge, long now) {
Map<String, Object> fields = new HashMap<String, Object>();
Object sanitizeGauge = sanitizeGauge(gauge.getValue());
Object sanitizeGauge = sanitizeValue(gauge.getValue());
if (sanitizeGauge != null) {

fields.put("value", sanitizeGauge);
Expand Down
62 changes: 62 additions & 0 deletions metrics-influxdb/src/test/java/com/izettle/metrics/influxdb/InfluxDbReporterTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,68 @@ public void reportsCounters() throws Exception {

}

@Test
public void reportsGroupedCounters() throws Exception {
final Counter counter = mock(Counter.class);
Mockito.when(counter.getCount()).thenReturn(10L);

InfluxDbReporter groupReporter = InfluxDbReporter
.forRegistry(registry)
.convertRatesTo(TimeUnit.SECONDS)
.convertDurationsTo(TimeUnit.MILLISECONDS)
.filter(MetricFilter.ALL)
.groupCounters(true)
.build(influxDb);

groupReporter.report(this.<Gauge>map(), this.map("counter", counter), this.<Histogram>map(),
this.<Meter>map(), this.<Timer>map());
final ArgumentCaptor<InfluxDbPoint> influxDbPointCaptor = ArgumentCaptor.forClass(InfluxDbPoint.class);
verify(influxDb, atLeastOnce()).appendPoints(influxDbPointCaptor.capture());
verify(influxDb).setTags(globalTags);
InfluxDbPoint point = influxDbPointCaptor.getValue();
assertThat(point.getMeasurement()).isEqualTo("counter");
assertThat(point.getFields()).isNotEmpty();
assertThat(point.getFields()).hasSize(1);
assertThat(point.getFields()).contains(entry("value", 10L));
assertThat(point.getTags()).containsEntry("metricName", "counter");

groupReporter.report(this.<Gauge>map(), this.map("counter.1", counter), this.<Histogram>map(),
this.<Meter>map(), this.<Timer>map());
verify(influxDb, atLeastOnce()).appendPoints(influxDbPointCaptor.capture());
point = influxDbPointCaptor.getValue();
assertThat(point.getMeasurement()).isEqualTo("counter");
assertThat(point.getFields()).isNotEmpty();
assertThat(point.getFields()).hasSize(1);
assertThat(point.getFields()).contains(entry("1", 10L));
assertThat(point.getTags()).containsEntry("metricName", "counter");

// if metric name terminates in `.' field name should be empty
groupReporter.report(this.<Gauge>map(), this.map("counter.", counter), this.<Histogram>map(),
this.<Meter>map(), this.<Timer>map());
verify(influxDb, atLeastOnce()).appendPoints(influxDbPointCaptor.capture());
point = influxDbPointCaptor.getValue();
assertThat(point.getMeasurement()).isEqualTo("counter");
assertThat(point.getFields()).isNotEmpty();
assertThat(point.getFields()).hasSize(1);
assertThat(point.getFields()).contains(entry("", 10L));
assertThat(point.getTags()).containsEntry("metricName", "counter");

SortedMap<String, Counter> counters = this.map("counter.a", counter);
counters.put("counter.b", counter);
counters.put("counter.", counter);
counters.put("counter", counter);
groupReporter.report(this.<Gauge>map(), counters, this.<Histogram>map(),
this.<Meter>map(), this.<Timer>map());
verify(influxDb, atLeastOnce()).appendPoints(influxDbPointCaptor.capture());

point = influxDbPointCaptor.getValue();
assertThat(point.getMeasurement()).isEqualTo("counter");
assertThat(point.getFields()).isNotEmpty();
assertThat(point.getFields()).hasSize(4);
assertThat(point.getFields()).contains(entry("", 10L), entry("a", 10L), entry("b", 10L), entry("value", 10L));
assertThat(point.getTags()).containsEntry("metricName", "counter");
}

@Test
public void reportsGauges() throws Exception {
final Gauge gauge = mock(Gauge.class);
Expand Down