Spymemcached
- Introduction
- Installing it
- Create the client
- Traces
- Stats
- Tags
- End to end example
- Examining your traces
- Examining your metrics
- Observability signal names
- References
Introduction
Memcached is one of the most used server caching and scaling technologies.
It was created by Brad Fitzpatrick in 2003 as a solution to scale his social media product Live Journal
net.spy.memcached is a popular Java API client for Memcached. We’ve created an observability instrumented wrapper using OpenCensus. It provides traces and metrics that you can then extract from your applications.
Installing it
<!-- https://mvnrepository.com/artifact/io.orijtech.integrations/ocspymemcached -->
<dependency>
<groupId>io.orijtech.integrations</groupId>
<artifactId>ocspymemcached</artifactId>
<version>0.0.1</version>
</dependency>
// https://mvnrepository.com/artifact/io.orijtech.integrations/ocspymemcached
compile group: 'io.orijtech.integrations', name: 'ocspymemcached', version: '0.0.1'
<!-- https://mvnrepository.com/artifact/io.orijtech.integrations/ocspymemcached -->
<dependency org="io.orijtech.integrations" name="ocspymemcached" rev="0.0.1"/>
# https://mvnrepository.com/artifact/io.orijtech.integrations/ocspymemcached
'io.orijtech.integrations:ocspymemcached:jar:0.0.1'
Creating the client
To get started, one needs to a create an OcWrapClient
which wraps MemcachedClient
Its constructor matches that of MemcachedClient
import io.orijtech.integrations.ocspymemcached.OcWrapClient;
import java.net.InetSocketAddress;
public class MemcachedOpenCensusTutorial {
public static void main(String[] args) {
OcWrapClient mc;
try {
// Create the wrapped Memcached client.
mc = new OcWrapClient(new InetSocketAddress("localhost", 11211));
} catch (Exception e) {
System.err.println("Failed to create Memcached client: " + e.toString());
return;
}
}
}
Traces
Each method that performs a network request is traced as per Observability signal names
Both synchronous and asynchronous methods that make network requests have been instrumented. Please remember to enable an OpenCensus Java stats exporter
Stats
To enable extraction of stats, please make sure to:
1) Invoke Observability.registerAllViews
2) Enable an OpenCensus Java stats exporter
Enabling stats
import io.orijtech.integrations.ocspymemcached.Observability;
public class MemcachedOpenCensusTutorial {
public static void main(String[] args) {
// Enable exporting of all the Memcached specific metrics and views.
Observability.registerAllViews();
}
private static void setupOpenCensusExporters() {
// Enable the stats exporter in here.
}
}
Available stats
We’ve compounded stats into the following
View | Search name | Description | Unit | Tags | Aggregation |
---|---|---|---|---|---|
Latency | “net.spy.memcached/latency” | The latencies of the various methods in milliseconds | ms | “method”, “error”, “status” | Distribution |
Lengths | “net.spy.memcached/length” | The lengths of either keys or values | By | “method”, “error”, “status” | Distribution |
Calls | “net.spy.memcached/calls” | The number of the various method calls | 1 | “method”, “type” | Count |
Tags
Tag | Description | Enumeration of values |
---|---|---|
method | Any of the qualified names of the spymemcached.MemcachedClient methods | For example “net.spy.memcached.MemcachedClient.shutdown” as per Observability signal-names |
type | Disambiguates between the various lengths | “KEY”, “VALUE” |
status | Indicates either success or failure of an operation | “OK”, “ERROR” |
error | Only set if tag key “status” value is “ERROR” | The error string collected from the operation for example “io.IOException: closed connection” |
End to end example
This demo uses the following dependencies, please install them first
Resource | URL |
---|---|
Memcached | Memcached Installation wiki |
Prometheus | Prometheus setup guided codelab |
Zipkin | Zipkin setup guided codelab |
With Memcached now installed and running, we can now start the code sample.
For simplicity examining metrics, we’ll use Prometheus for examining our stats and Zipkin to examine our traces.
Please place the Java source code in the following file src/main/java/io/opencensus/tutorials/ocspymemcached/MemcachedOpenCensusTutorial.java
in your current working directory. You can do this for example by:
mkdir -p src/main/java/io/opencensus/tutorials/ocspymemcached
touch src/main/java/io/opencensus/tutorials/ocspymemcached/MemcachedOpenCensusTutorial.java
and then the pom.xml
file too
Source code
// Please place this code sample in your current working directory in this file:
// src/main/java/io/opencensus/tutorials/ocspymemcached/MemcachedOpenCensusTutorial.java
package io.opencensus.tutorials.ocspymemcached;
import io.opencensus.common.Scope;
import io.opencensus.exporter.stats.prometheus.PrometheusStatsCollector;
import io.opencensus.exporter.trace.zipkin.ZipkinTraceExporter;
import io.opencensus.trace.Tracer;
import io.opencensus.trace.Tracing;
import io.opencensus.trace.config.TraceConfig;
import io.opencensus.trace.samplers.Samplers;
import io.orijtech.integrations.ocspymemcached.Observability;
import io.orijtech.integrations.ocspymemcached.OcWrapClient;
import io.prometheus.client.exporter.HTTPServer;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.InetSocketAddress;
import net.spy.memcached.internal.GetFuture;
import net.spy.memcached.internal.OperationFuture;
public class MemcachedOpenCensusTutorial {
public static void main(String[] args) {
OcWrapClient mc;
try {
// Create the wrapped Memcached client.
mc = new OcWrapClient(new InetSocketAddress("localhost", 11211));
} catch (Exception e) {
System.err.println("Failed to create Memcached client: " + e.toString());
return;
}
// Enable exporting of all the Memcached specific metrics and views.
Observability.registerAllViews();
// Now enable OpenCensus exporters.
setupOpenCensusExporters();
// Create the tracer that we'll use to create custom spans.
Tracer tracer = Tracing.getTracer();
// Then prepare the source for queries i.e. standard input.
BufferedReader stdin = new BufferedReader(new InputStreamReader(System.in));
while (true) {
String query = "";
try {
// Print out the prompt.
System.out.print("> ");
System.out.flush();
// Read the input.
query = stdin.readLine();
} catch (Exception e) {
System.err.println("Exception " + e);
return;
}
if (query.length() == 0) query = "*";
// Create our custom span that will be the parent of all the child
// spans from the instrumentation inside the Memcached client wrapper.
// This span is optional but is useful to group and examine the flow
// of requests.
Scope ss = tracer.spanBuilder("MemcachedOpenCensusTutorial").startScopedSpan();
System.out.println(query);
try {
// Perform an asynchronous Get to get back a Future.
GetFuture<Object> getRes = mc.asyncGet(query);
// You can perform some work here on the main thread,
// since the asyncGet is run asynchronously/in-the-background.
String result = "";
// Now attempt to get the result from the Future.
try {
result = (String) getRes.get();
} catch (Exception e) {
System.err.println("Result.Get exception: " + e);
}
Boolean cacheHit = result != null && result != "";
if (cacheHit) {
System.out.println("Cache hit!");
} else {
// Cache miss, so process the data and then
// memoize it for later cache hits.
System.out.println("Cache miss");
// Performing some "expensive" processing here.
// This could be something more sophisticated
// like searching for media, emails, reservations etc.
result = query.toUpperCase();
// Process it in the background so that
// we that we don't block on our critical path.
mc.set(query, 3600, result);
}
System.out.println("< " + result + "\n");
if (cacheHit && System.nanoTime() % 2 == 1) {
OperationFuture<Boolean> deleteFuture = mc.delete(query);
// In this case we have to wait for the deletion to complete.
// We can otherwise take out the waiting code.
try {
Boolean success = deleteFuture.get();
if (success) System.out.println("Successfully performed delete!");
else System.err.println("Failed to perform delete!");
} catch (Exception e) {
System.err.println("Deletion failed with exception: " + e);
}
}
} finally {
// End our custom span.
ss.close();
}
}
}
private static void setupOpenCensusExporters() {
// Firstly, change the sampling rate to always sample so
// that our demo can always produce trace spans.
// This rate is very high, please lower it in production apps!
TraceConfig traceConfig = Tracing.getTraceConfig();
traceConfig.updateActiveTraceParams(
traceConfig.getActiveTraceParams().toBuilder().setSampler(Samplers.alwaysSample()).build());
try {
// Create the Zipkin Trace exporter.
ZipkinTraceExporter.createAndRegister(
"http://localhost:9411/api/v2/spans", "spymemcached-opencensus");
// Create the Prometheus stats scrape endpoint.
PrometheusStatsCollector.createAndRegister();
// Run the server as a daemon on address "localhost:8888"
HTTPServer server = new HTTPServer("localhost", 8888, true);
} catch (Exception e) {
System.err.println("Failed to setup OpenCensus Exporters " + e);
}
}
}
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.opencensus.tutorials</groupId>
<artifactId>ocspymemcached</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>ocspymemcached</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<opencensus.version>0.18.0</opencensus.version>
<ocspymemcached.version>0.0.2</ocspymemcached.version>
<prometheus.server.version>0.3.0</prometheus.server.version>
<maven.plugin.version>1.5.0.Final</maven.plugin.version>
<java.source.version>1.8</java.source.version>
<maven.compiler.version>3.7.0</maven.compiler.version>
<codehaus.version>1.10</codehaus.version>
</properties>
<dependencies>
<dependency>
<groupId>io.orijtech.integrations</groupId>
<artifactId>ocspymemcached</artifactId>
<version>${ocspymemcached.version}</version>
</dependency>
<dependency>
<groupId>io.opencensus</groupId>
<artifactId>opencensus-api</artifactId>
<version>${opencensus.version}</version>
</dependency>
<dependency>
<groupId>io.opencensus</groupId>
<artifactId>opencensus-impl</artifactId>
<version>${opencensus.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>io.opencensus</groupId>
<artifactId>opencensus-exporter-trace-zipkin</artifactId>
<version>${opencensus.version}</version>
</dependency>
<dependency>
<groupId>io.opencensus</groupId>
<artifactId>opencensus-exporter-stats-prometheus</artifactId>
<version>${opencensus.version}</version>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_httpserver</artifactId>
<version>${prometheus.server.version}</version>
</dependency>
</dependencies>
<build>
<extensions>
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>${maven.plugin.version}</version>
</extension>
</extensions>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.version}</version>
<configuration>
<source>${java.source.version}</source>
<target>${java.source.version}</target>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>appassembler-maven-plugin</artifactId>
<version>${codehaus.version}</version>
<configuration>
<programs>
<program>
<id>MemcachedOpenCensus</id>
<mainClass>io.opencensus.tutorials.ocspymemcached.MemcachedOpenCensusTutorial</mainClass>
</program>
</programs>
</configuration>
</plugin>
</plugins>
</build>
</project>
Running it
With Memcached running, your pom.xml
file and Java source code placed in src/main/java/io/opencensus/tutorials/ocspymemcached/MemcachedOpenCensusTutorial.java
relative to the same directory that pom.xml
exists
mvn install && mvn exec:java -Dexec.mainClass=io.opencensus.tutorials.ocspymemcached.MemcachedOpenCensusTutorial
and this should then produce a prompt which requires input. On typing and hitting “Enter”, it should look like this:
> searched.
Cache miss
< SEARCHED.
> Two
Cache miss
< TWO
> attempts
Cache hit!
< ATTEMPTS
Successfully performed delete!
> of
Cache hit!
< OF
> the
Cache hit!
< THE
Successfully performed delete!
> sort
Cache miss
< SORT
Examining your traces
Navigate to the Zipkin UI at http://localhost:9411/zipkin and you should see something like
All traces
Single trace with explanation
Cache miss
Cache hit
Examining your metrics
Firstly we need to start Prometheus. To do this we need a prom.yaml
file in the current working directory
scrape_configs:
- job_name: 'ocspymemcachedtutorial'
scrape_interval: 10s
static_configs:
- targets: ['localhost:8888']
With that file saved as prom.yaml
, let’s now start Prometheus
prometheus --config.file=prom.yaml
and then navigate to the Prometheus UI at http://localhost:9090/graph and you should see something like
- All metrics
Calls
Latency
p95th latency graph by
histogram_quantile(0.95,
sum(rate(net_spy_memcached_latency_bucket[5m])) by (method, status, error, le))
- Lengths
Observability signal names
Methods that make network calls have been traced and are the following.
Span name and stats method | Java method signature |
---|---|
“net.spy.memcached.MemcachedClient.add” | OperationFuture |
“net.spy.memcached.MemcachedClient.add” | |
“net.spy.memcached.MemcachedClient.addObserver” | boolean addObserver(ConnectionObserver) |
“net.spy.memcached.MemcachedClient.append” | OperationFuture |
“net.spy.memcached.MemcachedClient.append” | |
“net.spy.memcached.MemcachedClient.asyncCAS” | |
“net.spy.memcached.MemcachedClient.asyncCAS” | OperationFuture |
“net.spy.memcached.MemcachedClient.asyncCAS” | |
“net.spy.memcached.MemcachedClient.asyncDecr” | OperationFuture |
“net.spy.memcached.MemcachedClient.asyncGet” | GetFuture |
“net.spy.memcached.MemcachedClient.asyncGet” | |
“net.spy.memcached.MemcachedClient.asyncGetBulk” | BulkFuture |
“net.spy.memcached.MemcachedClient.asyncGetBulk” | |
“net.spy.memcached.MemcachedClient.asyncGetBulk” | BulkFuture |
“net.spy.memcached.MemcachedClient.asyncGetBulk” | |
“net.spy.memcached.MemcachedClient.asyncGets” | OperationFuture |
“net.spy.memcached.MemcachedClient.asyncIncr” | OperationFuture |
“net.spy.memcached.MemcachedClient.cas” | CASResponse cas(String, long, Object) |
“net.spy.memcached.MemcachedClient.cas” | CASResponse cas(String, long, int, Object) |
“net.spy.memcached.MemcachedClient.cas” | |
“net.spy.memcached.MemcachedClient.cas” | |
“net.spy.memcached.MemcachedClient.decr” | long decr(String, long) |
“net.spy.memcached.MemcachedClient.decr” | long decr(String, int) |
“net.spy.memcached.MemcachedClient.decr” | long decr(String, int, long, int) |
“net.spy.memcached.MemcachedClient.decr” | long decr(String, long, long, int) |
“net.spy.memcached.MemcachedClient.delete” | OperationFuture |
“net.spy.memcached.MemcachedClient.flush” | OperationFuture |
“net.spy.memcached.MemcachedClient.flush” | OperationFuture |
“net.spy.memcached.MemcachedClient.getAndTouch” | CASValue |
“net.spy.memcached.MemcachedClient.getBulk” | Map |
“net.spy.memcached.MemcachedClient.getBulk” | Map |
“net.spy.memcached.MemcachedClient.getBulk” | |
“net.spy.memcached.MemcachedClient.getBulk” | Map |
“net.spy.memcached.MemcachedClient.getStats” | Map |
“net.spy.memcached.MemcachedClient.getStats” | Map |
“net.spy.memcached.MemcachedClient.getVersions” | Map |
“net.spy.memcached.MemcachedClient.get” | |
“net.spy.memcached.MemcachedClient.get” | Object get(String) |
“net.spy.memcached.MemcachedClient.gets” | CASValue |
“net.spy.memcached.MemcachedClient.gets” | |
“net.spy.memcached.MemcachedClient.incr” | long incr(String, int) |
“net.spy.memcached.MemcachedClient.incr” | long incr(String, long) |
“net.spy.memcached.MemcachedClient.incr” | long incr(String, int, long) |
“net.spy.memcached.MemcachedClient.incr” | long incr(String, int, long, int) |
“net.spy.memcached.MemcachedClient.incr” | long incr(String, long, long, int) |
“net.spy.memcached.MemcachedClient.listSaslMechanisms” | Set |
“net.spy.memcached.MemcachedClient.prepend” | OperationFuture |
“net.spy.memcached.MemcachedClient.prepend” | |
“net.spy.memcached.MemcachedClient.replace” | OperationFuture |
“net.spy.memcached.MemcachedClient.replace” | |
“net.spy.memcached.MemcachedClient.set” | OperationFuture |
“net.spy.memcached.MemcachedClient.set” | |
“net.spy.memcached.MemcachedClient.shutdown” | void shutdown() |
“net.spy.memcached.MemcachedClient.shutdown” | boolean shutdown(long timeout, TimeUnit unit) |
References
Resource | URL |
---|---|
ocspymemcached on Maven Central | https://mvnrepository.com/artifact/io.orijtech.integrations/ocspymemcached |
ocspymemcached source code on Github | https://github.com/opencensus-integrations/ocspymemcached |
Memcached project | https://memcached.org |
SpyMemcached JavaDoc | http://dustin.sallings.org/java-memcached-client/apidocs/net/spy/memcached/internal/package-frame.html |
SpyMemcached project on Github | https://github.com/couchbase/spymemcached |
OpenCensus Java exporters | Java exporters |