[Citrix ADC] Add Integration Package with System Data Stream

packages/citrix_adc/_dev/build/docs/README.md

@@ -4,13 +4,13 @@
 
 The Citrix ADC integration allows you to monitor your Citrix ADC instance. Citrix ADC is an application delivery controller that performs application-specific traffic analysis to intelligently distribute, optimize, and secure Layer 4 - Layer 7 (L4–L7) network traffic for web applications.
 
-Use the Citrix ADC integration to collect metrics related to the interface, lbvserver, and service. Then visualize that data in Kibana, create alerts to notify you if something goes wrong, and reference logs when troubleshooting an issue.
+Use the Citrix ADC integration to collect metrics related to the interface, lbvserver, service and system. Then visualize that data in Kibana, create alerts to notify you if something goes wrong and reference logs when troubleshooting an issue.
 
 ## Data streams
 
 The Citrix ADC integration collects metrics data.
 
-Metrics give you insight into the statistics of the Citrix ADC. Metrics data streams collected by the Citrix ADC integration include [interface](https://developer-docs.citrix.com/projects/netscaler-nitro-api/en/12.0/statistics/network/interface/), [lbvserver](https://developer-docs.citrix.com/projects/netscaler-nitro-api/en/12.0/statistics/lb/lbvserver/), and [service](https://developer-docs.citrix.com/projects/netscaler-nitro-api/en/12.0/statistics/basic/service/), so that the user could monitor and troubleshoot the performance of the Citrix ADC instances.
+Metrics give you insight into the statistics of the Citrix ADC. Metrics data streams collected by the Citrix ADC integration include [interface](https://developer-docs.citrix.com/projects/netscaler-nitro-api/en/12.0/statistics/network/interface/), [lbvserver](https://developer-docs.citrix.com/projects/netscaler-nitro-api/en/12.0/statistics/lb/lbvserver/), [service](https://developer-docs.citrix.com/projects/netscaler-nitro-api/en/12.0/statistics/basic/service/) and [system](https://developer-docs.citrix.com/projects/netscaler-nitro-api/en/12.0/statistics/system/system/), so that the user could monitor and troubleshoot the performance of the Citrix ADC instances.
 
 Note:
 - Users can monitor and see the metrics inside the ingested documents for Citrix ADC in the logs-* index pattern from `Discover`.
@@ -29,28 +29,40 @@ Host Configuration Format: `http[s]://host[:port]`
 
 Example Host Configuration: `http://localhost:9080`
 
+### Troubleshooting
+
+There could be a possibility that for some of the fields, Citrix ADC sets dummy values. For example, a field `cpuusagepcnt` is represented by `citrix_adc.system.cpu.utilization.pct`. `cpuusagepcnt` is set to `4294967295` for some [instances](https://github.com/citrix/citrix-adc-metrics-exporter/issues/44). If you also encounter it for some fields please reach out to the [Citrix ADC support team](https://support.citrix.com/plp/products/citrix_adc/tabs/popular-solutions).
+
 ## Metrics reference
 
 ### Interface
 
-This is the `interface` data stream. The Citrix ADC interfaces are numbered in slot/port notation. In addition to modifying the characteristics of individual interfaces, you can configure virtual LANs to restrict traffic to specific groups of hosts. `interface` data stream collects metrics related to id, state, inbound packets, outbound packets, and received packets.
+This is the `interface` data stream. The Citrix ADC interfaces are numbered in slot/port notation. In addition to modifying the characteristics of individual interfaces, you can configure virtual LANs to restrict traffic to specific groups of hosts. `interface` data stream collects metrics related to id, state, inbound packets, outbound packets and received packets.
 
 {{event "interface"}}
 
 {{fields "interface"}}
 
 ### Load Balancing Virtual Server
 
-This is the `lbvserver` data stream. The load balancing server is logically located between the client and the server farm, and manages traffic flow to the servers in the server farm. `lbvserver` data stream collects metrics related to name, state, client connections, requests, and responses.
+This is the `lbvserver` data stream. The load balancing server is logically located between the client and the server farm, and manages traffic flow to the servers in the server farm. `lbvserver` data stream collects metrics related to name, state, client connections, requests and responses.
 
 {{event "lbvserver"}}
 
 {{fields "lbvserver"}}
 
 ### Service
 
-This is the `service` data stream. With the help of the service endpoint, metrics like throughput, client-server connections, request bytes can be collected along with other statistics for Service resources. `service` data stream collects metrics related to name, IP address, port, throughput, and transactions.
+This is the `service` data stream. With the help of the service endpoint, metrics like throughput, client-server connections, request bytes can be collected along with other statistics for Service resources. `service` data stream collects metrics related to name, IP address, port, throughput and transactions.
 
 {{event "service"}}
 
 {{fields "service"}}
+
+### System
+
+This is the `system` data stream. With the help of the system endpoint, metrics like memory in use, total system memory, CPU count can be collected along with other statistics for system resources.
+
+{{event "system"}}
+
+{{fields "system"}}

packages/citrix_adc/_dev/deploy/docker/files/config.yml

@@ -4,7 +4,7 @@ rules:
     responses:
       - status_code: 200
         body: |-
-          {"errorcode":0,"message":"Done","severity":"NONE","Interface":[{"id":"0/1","curintfstate":"UP","curlinkuptime":"4.06:45:16","curlinkdowntime":"00:00:11","totrxbytes":"1103884064","rxbytesrate":4603,"tottxbytes":"776571619","txbytesrate":1924,"totrxpkts":"5396347","rxpktsrate":38,"tottxpkts":"2511171","txpktsrate":5,"jumbopktsreceived":"0","jumbopktsreceivedrate":0,"jumbopktstransmitted":"0","jumbopktstransmittedrate":0,"trunkpktsreceived":"0","trunkpktsreceivedrate":0,"trunkpktstransmitted":"0","trunkpktstransmittedrate":0,"nictotmulticastpkts":"278537","nicmulticastpktsrate":0,"totnetscalerpkts":"2493179","netscalerpktsrate":6,"rxlacpdu":"0","rxlacpdurate":0,"txlacpdu":"0","txlacpdurate":0,"errpktrx":"0","errpktrxrate":0,"errpkttx":"0","errpkttxrate":0,"errifindiscards":"0","errifindiscardsrate":0,"nicerrifoutdiscards":"0","nicerrifoutdiscardsrate":0,"errdroppedrxpkts":"2797172","errdroppedrxpktsrate":32,"errdroppedtxpkts":"0","errdroppedtxpktsrate":0,"errlinkhangs":"0","nicstsstalls":"0","nictxstalls":"0","nicrxstalls":"0","nicerrdisables":"0","linkreinits":"0","totmacmoved":"0","macmovedrate":0,"errnicmuted":"0","rxcrcerrors":"0","rxcrcerrorsrate":0,"interfacealias":"","curlinkstate":"DOWN"},{"id":"LO/1","curintfstate":"UP","curlinkuptime":"4.06:45:27","curlinkdowntime":"00:00:00","totrxbytes":"5134180570","rxbytesrate":14597,"tottxbytes":"6826375522","txbytesrate":18601,"totrxpkts":"36267172","rxpktsrate":100,"tottxpkts":"71618978","txpktsrate":195,"jumbopktsreceived":"0","jumbopktsreceivedrate":0,"jumbopktstransmitted":"0","jumbopktstransmittedrate":0,"trunkpktsreceived":"0","trunkpktsreceivedrate":0,"trunkpktstransmitted":"0","trunkpktstransmittedrate":0,"nictotmulticastpkts":"0","nicmulticastpktsrate":0,"totnetscalerpkts":"36267172","netscalerpktsrate":100,"rxlacpdu":"0","rxlacpdurate":0,"txlacpdu":"0","txlacpdurate":0,"errpktrx":"0","errpktrxrate":0,"errpkttx":"0","errpkttxrate":0,"errifindiscards":"0","errifindiscardsrate":0,"nicerrifoutdiscards":"0","nicerrifoutdiscardsrate":0,"errdroppedrxpkts":"0","errdroppedrxpktsrate":0,"errdroppedtxpkts":"0","errdroppedtxpktsrate":0,"errlinkhangs":"0","nicstsstalls":"0","nictxstalls":"0","nicrxstalls":"0","nicerrdisables":"0","linkreinits":"0","totmacmoved":"0","macmovedrate":0,"errnicmuted":"0","rxcrcerrors":"0","rxcrcerrorsrate":0,"interfacealias":"","curlinkstate":"DOWN"}]}
+          {"errorcode": 0, "message": "Done", "severity": "NONE", "Interface": [{"id": "0/1", "curintfstate": "UP", "curlinkuptime": "4.06:45:16", "curlinkdowntime": "00:00:11", "totrxbytes": "1103884064", "rxbytesrate": 4603, "tottxbytes": "776571619", "txbytesrate": 1924, "totrxpkts": "5396347", "rxpktsrate": 38, "tottxpkts": "2511171", "txpktsrate": 5, "jumbopktsreceived": "0", "jumbopktsreceivedrate": 0, "jumbopktstransmitted": "0", "jumbopktstransmittedrate": 0, "trunkpktsreceived": "0", "trunkpktsreceivedrate": 0, "trunkpktstransmitted": "0", "trunkpktstransmittedrate": 0, "nictotmulticastpkts": "278537", "nicmulticastpktsrate": 0, "totnetscalerpkts": "2493179", "netscalerpktsrate": 6, "rxlacpdu": "0", "rxlacpdurate": 0, "txlacpdu": "0", "txlacpdurate": 0, "errpktrx": "0", "errpktrxrate": 0, "errpkttx": "0", "errpkttxrate": 0, "errifindiscards": "0", "errifindiscardsrate": 0, "nicerrifoutdiscards": "0", "nicerrifoutdiscardsrate": 0, "errdroppedrxpkts": "2797172", "errdroppedrxpktsrate": 32, "errdroppedtxpkts": "0", "errdroppedtxpktsrate": 0, "errlinkhangs": "0", "nicstsstalls": "0", "nictxstalls": "0", "nicrxstalls": "0", "nicerrdisables": "0", "linkreinits": "0", "totmacmoved": "0", "macmovedrate": 0, "errnicmuted": "0", "rxcrcerrors": "0", "rxcrcerrorsrate": 0, "interfacealias": "", "curlinkstate": "DOWN"}, {"id": "LO/1", "curintfstate": "UP", "curlinkuptime": "4.06:45:27", "curlinkdowntime": "00:00:00", "totrxbytes": "5134180570", "rxbytesrate": 14597, "tottxbytes": "6826375522", "txbytesrate": 18601, "totrxpkts": "36267172", "rxpktsrate": 100, "tottxpkts": "71618978", "txpktsrate": 195, "jumbopktsreceived": "0", "jumbopktsreceivedrate": 0, "jumbopktstransmitted": "0", "jumbopktstransmittedrate": 0, "trunkpktsreceived": "0", "trunkpktsreceivedrate": 0, "trunkpktstransmitted": "0", "trunkpktstransmittedrate": 0, "nictotmulticastpkts": "0", "nicmulticastpktsrate": 0, "totnetscalerpkts": "36267172", "netscalerpktsrate": 100, "rxlacpdu": "0", "rxlacpdurate": 0, "txlacpdu": "0", "txlacpdurate": 0, "errpktrx": "0", "errpktrxrate": 0, "errpkttx": "0", "errpkttxrate": 0, "errifindiscards": "0", "errifindiscardsrate": 0, "nicerrifoutdiscards": "0", "nicerrifoutdiscardsrate": 0, "errdroppedrxpkts": "0", "errdroppedrxpktsrate": 0, "errdroppedtxpkts": "0", "errdroppedtxpktsrate": 0, "errlinkhangs": "0", "nicstsstalls": "0", "nictxstalls": "0", "nicrxstalls": "0", "nicerrdisables": "0", "linkreinits": "0", "totmacmoved": "0", "macmovedrate": 0, "errnicmuted": "0", "rxcrcerrors": "0", "rxcrcerrorsrate": 0, "interfacealias": "", "curlinkstate": "DOWN"}]}
   - path: /nitro/v1/stat/service
     methods: ['GET']
     responses:
@@ -17,3 +17,9 @@ rules:
       - status_code: 200
         body: |-
           {"errorcode":0,"message":"Done","severity":"NONE","lbvserver":[{"name":"elastic","avgcltttlb":"6","cltresponsetimeapdex":1,"vsvrsurgecount":"6","establishedconn":"6","inactsvcs":"6","vslbhealth":"67","primaryipaddress":"8.8.8.8","primaryport":80,"type":"HTTP","state":"DOWN","actsvcs":"10","cpuusagepm":"10","tothits":"10","hitsrate":5,"totalrequests":"5","requestsrate":5,"totalresponses":"5","responsesrate":5,"totalrequestbytes":"7","requestbytesrate":7,"totalresponsebytes":"7","responsebytesrate":7,"totalh2requests":"7","h2requestsrate":7,"totalh2responses":"7","h2responsesrate":7,"totalpktsrecvd":"7","pktsrecvdrate":8,"totalpktssent":"8","pktssentrate":8,"curclntconnections":"8","cursrvrconnections":"8","curpersistencesessions":"8","curbackuppersistencesessions":"8","surgecount":"8","svcsurgecount":"8","sothreshold":"8","totspillovers":"8","labelledconn":"8","pushlabel":"8","deferredreq":"13","deferredreqrate":13,"invalidrequestresponse":"13","invalidrequestresponsedropped":"13","totvserverdownbackuphits":"13","curmptcpsessions":"13","cursubflowconn":"13","totalconnreassemblyqueue75":"13","totalconnreassemblyqueueflush":"3","totalsvrbusyerr":"3","svrbusyerrrate":3,"reqretrycount":"3","reqretrycountexceeded":"3","httpmaxhdrszpkts":"3","httpmaxhdrfldlenpkts":"3","tcpmaxooopkts":"3","totcltttlbtransactions":"3","cltttlbtransactionsrate":3,"toleratingttlbtransactions":"3","toleratingttlbtransactionsrate":1,"frustratingttlbtransactions":"1","frustratingttlbtransactionsrate":1}]}
+  - path: /nitro/v1/stat/system
+    methods: ['GET']
+    responses:
+      - status_code: 200
+        body: |-
+          {"errorcode": 0, "message": "Done", "severity": "NONE", "system": { "voltagev12n": 0.000000, "voltagev5n": 0.000000, "cpuusage": "1", "rescpuusage": "4294967295", "slavecpuusage": "4294967295", "mastercpuusage": "4294967295", "auxvolt7": 0.000000, "auxvolt6": 0.000000, "auxvolt5": 0.000000, "auxvolt4": 0.000000, "auxvolt3": 0.000000, "auxvolt2": 0.000000, "auxvolt1": 0.000000, "auxvolt0": 0.000000, "voltagevsen2": 0.000000, "voltagev5sb": 0.000000, "voltagevtt": 0.000000, "voltagevbat": 0.000000, "voltagev12p": 0.000000, "voltagev5p": 0.000000, "voltagev33stby": 0.000000, "voltagev33main": 0.000000, "voltagevcc1": 0.000000, "voltagevcc0": 0.000000, "numcpus": "1", "memusagepcnt": 21.114572, "memuseinmb": "216", "addimgmtcpuusagepcnt": 0.000000, "mgmtcpu0usagepcnt": 0.800000, "mgmtcpuusagepcnt": 0.800000, "pktcpuusagepcnt": 1.100000, "cpuusagepcnt": 1.100000, "rescpuusagepcnt": 4294967295.000000, "starttimelocal": "Thu Sep 22 09:20:13 2022", "starttime": "Thu Sep 22 03:50:13 2022", "disk0perusage": 12, "disk1perusage": 12, "cpufan0speed": 0, "cpufan1speed": 0, "systemfanspeed": 0, "fan0speed": 0, "fanspeed": 0, "cpu0temp": 0, "cpu1temp": 0, "internaltemp": 0, "powersupply1status": "NOT SUPPORTED", "powersupply2status": "NOT SUPPORTED", "powersupply3status": "NOT SUPPORTED", "powersupply4status": "NOT SUPPORTED", "disk0size": 1585, "disk0used": 180, "disk0avail": 1278, "disk1size": 14179, "disk1used": 1603, "disk1avail": 11441, "fan2speed": 0, "fan3speed": 0, "fan4speed": 0, "fan5speed": 0, "auxtemp0": 0, "auxtemp1": 0, "auxtemp2": 0, "auxtemp3": 0, "timesincestart": "00:00:00", "memsizemb": "0"}}

packages/citrix_adc/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "0.4.0"
+  changes:
+    - description: Citrix ADC integration package with "system" data stream.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/4458
 - version: "0.3.0"
   changes:
     - description: Citrix ADC integration package with "lbvserver" data stream.

packages/citrix_adc/data_stream/system/_dev/test/pipeline/test-common-config.yml

@@ -0,0 +1,3 @@
+fields:
+  tags:
+    - preserve_original_event

packages/citrix_adc/data_stream/system/_dev/test/pipeline/test-system-metrics.log

@@ -0,0 +1,2 @@
+{"errorcode":0,"message":"Done","severity":"NONE","system":{"voltagev12n":0.000000,"voltagev5n":0.000000,"cpuusage":"1","rescpuusage":"4294967295","slavecpuusage":"4294967295","mastercpuusage":"4294967295","auxvolt7":0.000000,"auxvolt6":0.000000,"auxvolt5":0.000000,"auxvolt4":0.000000,"auxvolt3":0.000000,"auxvolt2":0.000000,"auxvolt1":0.000000,"auxvolt0":0.000000,"voltagevsen2":0.000000,"voltagev5sb":0.000000,"voltagevtt":0.000000,"voltagevbat":0.000000,"voltagev12p":0.000000,"voltagev5p":0.000000,"voltagev33stby":0.000000,"voltagev33main":0.000000,"voltagevcc1":0.000000,"voltagevcc0":0.000000,"numcpus":"1","memusagepcnt":21.105921,"memuseinmb":"216","addimgmtcpuusagepcnt":0.000000,"mgmtcpu0usagepcnt":0.900000,"mgmtcpuusagepcnt":0.900000,"pktcpuusagepcnt":0.900000,"cpuusagepcnt":0.900000,"rescpuusagepcnt":4294967295.000000,"starttimelocal":"Thu Sep 22 09:20:13 2022","starttime":"Tue Sep 13 08:40:04 2022","disk0perusage":12,"disk1perusage":12,"cpufan0speed":0,"cpufan1speed":0,"systemfanspeed":0,"fan0speed":0,"fanspeed":0,"cpu0temp":0,"cpu1temp":0,"internaltemp":0,"powersupply1status":"NOTSUPPORTED","powersupply2status":"NOTSUPPORTED","powersupply3status":"NOTSUPPORTED","powersupply4status":"NOTSUPPORTED","disk0size":1585,"disk0used":180,"disk0avail":1278,"disk1size":14179,"disk1used":1583,"disk1avail":11461,"fan2speed":0,"fan3speed":0,"fan4speed":0,"fan5speed":0,"auxtemp0":0,"auxtemp1":0,"auxtemp2":0,"auxtemp3":0,"timesincestart":"00:00:00","memsizemb":"0"}}
+{"errorcode":0,"message":"Done","severity":"NONE","system":{"voltagev12n":0,"voltagev5n":0,"cpuusage":"1","rescpuusage":"4294967295","slavecpuusage":"4294967295","mastercpuusage":"4294967295","auxvolt7":0,"auxvolt6":0,"auxvolt5":0,"auxvolt4":0,"auxvolt3":0,"auxvolt2":0,"auxvolt1":0,"auxvolt0":0,"voltagevsen2":0,"voltagev5sb":0,"voltagevtt":0,"voltagevbat":0,"voltagev12p":0,"voltagev5p":0,"voltagev33stby":0,"voltagev33main":0,"voltagevcc1":0,"voltagevcc0":0,"numcpus":"1","memusagepcnt":89.222422,"memuseinmb":"214","addimgmtcpuusagepcnt":0,"mgmtcpu0usagepcnt":0,"mgmtcpuusagepcnt":0,"pktcpuusagepcnt":0.6,"cpuusagepcnt":0.6,"rescpuusagepcnt":4294967295,"starttimelocal":"Fri Nov  4 09:56:30 2022","starttime":"Fri Nov  4 09:56:30 2022","disk0perusage":0,"disk1perusage":0,"cpufan0speed":0,"cpufan1speed":0,"systemfanspeed":0,"fan0speed":0,"fanspeed":0,"cpu0temp":0,"cpu1temp":0,"internaltemp":0,"powersupply1status":"NOT SUPPORTED","powersupply2status":"NOT SUPPORTED","powersupply3status":"NOT SUPPORTED","powersupply4status":"NOT SUPPORTED","disk0size":0,"disk0used":0,"disk0avail":0,"disk1size":0,"disk1used":0,"disk1avail":0,"fan2speed":0,"fan3speed":0,"fan4speed":0,"fan5speed":0,"auxtemp0":0,"auxtemp1":0,"auxtemp2":0,"auxtemp3":0,"timesincestart":"00:00:00","memsizemb":"0"}}