Writing Custom Metrics (#2057)

This adds write_point() and write_time_series() as Client methods.

Author: waprin

docs/monitoring-usage.rst

@@ -30,7 +30,7 @@ of the API:
 - Querying of time series.
 - Querying of metric descriptors and monitored resource descriptors.
 - Creation and deletion of metric descriptors for custom metrics.
-- (Writing of custom metric data will be coming soon.)
+- Writing of custom metric data.
 
 .. _Stackdriver Monitoring API: https://cloud.google.com/monitoring/api/v3/
 
@@ -278,3 +278,88 @@ follows::
 
 .. _Time Series:
     https://cloud.google.com/monitoring/api/ref_v3/rest/v3/TimeSeries
+
+
+Writing Custom Metrics
+---------------------------
+
+The Stackdriver Monitoring API can be used to write data points to custom metrics. Please refer to
+the documentation on `Custom Metrics`_ for more information.
+
+To write a data point to a custom metric, you must provide an instance of
+:class:`~google.cloud.monitoring.metric.Metric` specifying the metric type as well as the values for
+the metric labels. You will need to have either created the metric descriptor earlier (see the
+`Metric Descriptors`_ section) or rely on metric type auto-creation (see `Auto-creation of
+custom metrics`_).
+
+You will also need to provide a :class:`~google.cloud.monitoring.resource.Resource` instance
+specifying a monitored resource type as well as values for all of the monitored resource labels,
+except for ``project_id``, which is ignored when it's included in writes to the API. A good
+choice is to use the underlying physical resource where your application code runs – e.g., a
+monitored resource type of ``gce_instance`` or ``aws_ec2_instance``. In some limited
+circumstances, such as when only a single process writes to the custom metric, you may choose to
+use the ``global`` monitored resource type.
+
+See `Monitored resource types`_ for more information about particular monitored resource types.
+
+>>> from google.cloud import monitoring
+>>> # Create a Resource object for the desired monitored resource type.
+>>> resource = client.resource('gce_instance', labels={
+...     'instance_id': '1234567890123456789',
+...     'zone': 'us-central1-f'
+... })
+>>> # Create a Metric object, specifying the metric type as well as values for any metric labels.
+>>> metric = client.metric(type='custom.googleapis.com/my_metric', labels={
+...      'status': 'successful'
+... })
+
+With a ``Metric`` and ``Resource`` in hand, the :class:`~google.cloud.monitoring.client.Client`
+can be used to write :class:`~google.cloud.monitoring.timeseries.Point` values.
+
+When writing points, the Python type of the value must match the *value type* of the metric
+descriptor associated with the metric. For example, a Python float will map to ``ValueType.DOUBLE``.
+
+Stackdriver Monitoring supports several *metric kinds*: ``GAUGE``, ``CUMULATIVE``, and ``DELTA``.
+However, ``DELTA`` is not supported for custom metrics.
+
+``GAUGE`` metrics represent only a single point in time, so only the ``end_time`` should be
+specified::
+
+    >>> client.write_point(metric=metric, resource=resource,
+    ...                    value=3.14, end_time=end_time) # API call
+
+By default, ``end_time`` defaults to :meth:`~datetime.datetime.utcnow()`, so metrics can be written
+to the current time as follows::
+
+   >>> client.write_point(metric, resource, 3.14) # API call
+
+``CUMULATIVE`` metrics enable the monitoring system to compute rates of increase on metrics that
+sometimes reset, such as after a process restart. Without cumulative metrics, this
+reset would otherwise show up as a huge negative spike. For cumulative metrics, the same start
+time should be re-used repeatedly as more points are written to the time series.
+
+In the examples below, the ``end_time`` again defaults to the current time::
+
+    >>> RESET = datetime.utcnow()
+    >>> client.write_point(metric, resource, 3, start_time=RESET) # API call
+    >>> client.write_point(metric, resource, 6, start_time=RESET) # API call
+
+To write multiple ``TimeSeries`` in a single batch, you can use
+:meth:`~google.cloud.monitoring.client.write_time_series`::
+
+    >>> ts1 = client.time_series(metric1, resource, 3.14, end_time=end_time)
+    >>> ts2 = client.time_series(metric2, resource, 42, end_time=end_time)
+    >>> client.write_time_series([ts1, ts2]) # API call
+
+While multiple time series can be written in a single batch, each ``TimeSeries`` object sent to
+the API must only include a single point.
+
+All timezone-naive Python ``datetime`` objects are assumed to be UTC.
+
+.. _TimeSeries: https://cloud.google.com/monitoring/api/ref_v3/rest/v3/TimeSeries
+.. _Custom Metrics: https://cloud.google.com/monitoring/custom-metrics/
+.. _Auto-creation of custom metrics:
+    https://cloud.google.com/monitoring/custom-metrics/creating-metrics#auto-creation
+.. _Metrics: https://cloud.google.com/monitoring/api/v3/metrics
+.. _Monitored resource types:
+    https://cloud.google.com/monitoring/api/resources

google/cloud/monitoring/client.py

@@ -30,6 +30,7 @@
 
 import datetime
 
+from google.cloud._helpers import _datetime_to_rfc3339
 from google.cloud.client import JSONClient
 from google.cloud.monitoring.connection import Connection
 from google.cloud.monitoring.group import Group
@@ -312,7 +313,7 @@ def time_series(metric, resource, value,
         :type start_time: :class:`~datetime.datetime`
         :param start_time:
             The start time for the point to be included in the time series.
-            Assumed to be UTC if no time zone information is present
+            Assumed to be UTC if no time zone information is present.
             Defaults to None. If the start time is unspecified,
             the API interprets the start time to be the same as the end time.
 
@@ -321,6 +322,11 @@ def time_series(metric, resource, value,
         """
         if end_time is None:
             end_time = _UTCNOW()
+
+        end_time = _datetime_to_rfc3339(end_time, ignore_zone=False)
+        if start_time:
+            start_time = _datetime_to_rfc3339(start_time, ignore_zone=False)
+
         point = Point(value=value, start_time=start_time, end_time=end_time)
         return TimeSeries(metric=metric, resource=resource, metric_kind=None,
                           value_type=None, points=[point])
@@ -495,3 +501,86 @@ def list_groups(self):
         :returns: A list of group instances.
         """
         return Group._list(self)
+
+    def write_time_series(self, timeseries_list):
+        """Write a list of time series objects to the API.
+
+        The recommended approach to creating time series objects is using
+        the :meth:`~google.cloud.monitoring.client.Client.time_series` factory
+        method.
+
+        Example::
+
+            >>> client.write_time_series([ts1, ts2])
+
+        If you only need to write a single time series object, consider using
+        the :meth:`~google.cloud.monitoring.client.Client.write_point` method
+        instead.
+
+        :type timeseries_list:
+            list of :class:`~google.cloud.monitoring.timeseries.TimeSeries`
+        :param timeseries_list:
+            A list of time series object to be written
+            to the API. Each time series must contain exactly one point.
+        """
+        path = '/projects/{project}/timeSeries/'.format(
+            project=self.project)
+        timeseries_dict = [timeseries._to_dict()
+                           for timeseries in timeseries_list]
+        self.connection.api_request(method='POST', path=path,
+                                    data={'timeSeries': timeseries_dict})
+
+    def write_point(self, metric, resource, value,
+                    end_time=None,
+                    start_time=None):
+        """Write a single point for a metric to the API.
+
+        This is a convenience method to write a single time series object to
+        the API. To write multiple time series objects to the API as a batch
+        operation, use the
+        :meth:`~google.cloud.monitoring.client.Client.time_series`
+        factory method to create time series objects and the
+        :meth:`~google.cloud.monitoring.client.Client.write_time_series`
+        method to write the objects.
+
+        Example::
+
+            >>> client.write_point(metric, resource, 3.14)
+
+        :type metric: :class:`~google.cloud.monitoring.metric.Metric`
+        :param metric: A :class:`~google.cloud.monitoring.metric.Metric`
+                       object.
+
+        :type resource: :class:`~google.cloud.monitoring.resource.Resource`
+        :param resource: A :class:`~google.cloud.monitoring.resource.Resource`
+                         object.
+
+        :type value: bool, int, string, or float
+        :param value:
+            The value of the data point to create for the
+            :class:`~google.cloud.monitoring.timeseries.TimeSeries`.
+
+            .. note::
+
+               The Python type of the value will determine the
+               :class:`~ValueType` sent to the API, which must match the value
+               type specified in the metric descriptor. For example, a Python
+               float will be sent to the API as a :data:`ValueType.DOUBLE`.
+
+        :type end_time: :class:`~datetime.datetime`
+        :param end_time:
+            The end time for the point to be included in the time series.
+            Assumed to be UTC if no time zone information is present.
+            Defaults to the current time, as obtained by calling
+            :meth:`datetime.datetime.utcnow`.
+
+        :type start_time: :class:`~datetime.datetime`
+        :param start_time:
+            The start time for the point to be included in the time series.
+            Assumed to be UTC if no time zone information is present.
+            Defaults to None. If the start time is unspecified,
+            the API interprets the start time to be the same as the end time.
+        """
+        timeseries = self.time_series(
+            metric, resource, value, end_time, start_time)
+        self.write_time_series([timeseries])

google/cloud/monitoring/metric.py

@@ -348,3 +348,15 @@ def _from_dict(cls, info):
             type=info['type'],
             labels=info.get('labels', {}),
         )
+
+    def _to_dict(self):
+        """Build a dictionary ready to be serialized to the JSON format.
+
+        :rtype: dict
+        :returns: A dict representation of the object that can be written to
+                  the API.
+        """
+        return {
+            'type': self.type,
+            'labels': self.labels,
+        }

google/cloud/monitoring/resource.py

@@ -187,3 +187,15 @@ def _from_dict(cls, info):
             type=info['type'],
             labels=info.get('labels', {}),
         )
+
+    def _to_dict(self):
+        """Build a dictionary ready to be serialized to the JSON format.
+
+        :rtype: dict
+        :returns: A dict representation of the object that can be written to
+                  the API.
+        """
+        return {
+            'type': self.type,
+            'labels': self.labels,
+        }

google/cloud/monitoring/timeseries.py

@@ -90,6 +90,23 @@ def header(self, points=None):
         points = list(points) if points else []
         return self._replace(points=points)
 
+    def _to_dict(self):
+        """Build a dictionary ready to be serialized to the JSON wire format.
+
+        Since this method is used when writing to the API, it excludes
+        output-only fields.
+
+        :rtype: dict
+        :returns: The dictionary representation of the time series object.
+        """
+        info = {
+            'metric': self.metric._to_dict(),
+            'resource': self.resource._to_dict(),
+            'points': [point._to_dict() for point in self.points],
+        }
+
+        return info
+
     @classmethod
     def _from_dict(cls, info):
         """Construct a time series from the parsed JSON representation.
@@ -124,6 +141,38 @@ def __repr__(self):
         )
 
 
+def _make_typed_value(value):
+    """Create a dict representing a TypedValue API object.
+
+    Typed values are objects with the value itself as the value, keyed by the
+    type of the value. They are used when writing points to time series. This
+    method returns the dict representation for the TypedValue.
+
+    This method uses the Python type of the object to infer the correct
+    type to send to the API. For example, a Python float will be sent to the
+    API with "doubleValue" as its key.
+
+    See: https://cloud.google.com/monitoring/api/ref_v3/rest/v3/TypedValue
+
+    :type value: bool, int, float, str, or dict
+    :param value: value to infer the typed value of.
+
+    :rtype: dict
+    :returns: A dict
+    """
+    typed_value_map = {
+        bool: "boolValue",
+        int: "int64Value",
+        float: "doubleValue",
+        str: "stringValue",
+        dict: "distributionValue",
+    }
+    type_ = typed_value_map[type(value)]
+    if type_ == "int64Value":
+        value = str(value)
+    return {type_: value}
+
+
 class Point(collections.namedtuple('Point', 'end_time start_time value')):
     """A single point in a time series.
 
@@ -156,3 +205,24 @@ def _from_dict(cls, info):
             value = int(value)  # Convert from string.
 
         return cls(end_time, start_time, value)
+
+    def _to_dict(self):
+        """Build a dictionary ready to be serialized to the JSON wire format.
+
+        This method serializes a point in JSON format to be written
+        to the API.
+
+        :rtype: dict
+        :returns: The dictionary representation of the point object.
+        """
+        info = {
+            'interval': {
+                'endTime': self.end_time
+            },
+            'value': _make_typed_value(self.value)
+        }
+
+        if self.start_time is not None:
+            info['interval']['startTime'] = self.start_time
+
+        return info