github.com/newrelic/go-agent@v3.26.0+incompatible/GUIDE.md (about)

     1  # New Relic Go Agent Guide
     2  
     3  * [Upgrading](#upgrading)
     4  * [Installation](#installation)
     5  * [Full list of `Config` options and `Application` settings](#full-list-of-config-options-and-application-settings)
     6  * [Logging](#logging)
     7  * [Transactions](#transactions)
     8  * [Segments](#segments)
     9    * [Datastore Segments](#datastore-segments)
    10    * [External Segments](#external-segments)
    11    * [Message Producer Segments](#message-producer-segments)
    12  * [Attributes](#attributes)
    13  * [Tracing](#tracing)
    14    * [Distributed Tracing](#distributed-tracing)
    15    * [Cross-Application Tracing](#cross-application-tracing)
    16    * [Tracing instrumentation](#tracing-instrumentation)
    17      * [Getting Tracing Instrumentation Out-of-the-Box](#getting-tracing-instrumentation-out-of-the-box)
    18      * [Manually Implementing Distributed Tracing](#manually-implementing-distributed-tracing)
    19  * [Distributed Tracing](#distributed-tracing)
    20  * [Custom Metrics](#custom-metrics)
    21  * [Custom Events](#custom-events)
    22  * [Request Queuing](#request-queuing)
    23  * [Error Reporting](#error-reporting)
    24    * [NoticeError](#noticeerror)
    25    * [Panics](#panics)
    26    * [Error Response Codes](#error-response-codes)
    27  * [Naming Transactions and Metrics](#naming-transactions-and-metrics)
    28  * [Browser](#browser)
    29  * [For More Help](#for-more-help)
    30  
    31  ## Upgrading
    32  
    33  This guide documents version 3.x of the agent which resides in package
    34  `"github.com/newrelic/go-agent/v3/newrelic"`.
    35  If you have already been using version 2.X of the agent and are upgrading to
    36  version 3.0, see our [Migration Guide](MIGRATION.md) for details.
    37  
    38  ## Installation
    39  
    40  (Also see [GETTING_STARTED](https://github.com/newrelic/go-agent/blob/master/GETTING_STARTED.md) if you are using the Go agent for the first time).
    41  
    42  In order to install the New Relic Go agent, you need a New Relic license key. 
    43  Then, installing the Go Agent is the same as installing any other Go library.  The
    44  simplest way is to run:
    45  
    46  ```
    47  go get github.com/newrelic/go-agent/v3/newrelic
    48  ```
    49  
    50  Then import the package in your application:
    51  ```go
    52  import "github.com/newrelic/go-agent/v3/newrelic"
    53  ```
    54  
    55  Initialize the New Relic Go agent by adding the following `Config` options and `Application` settings in the `main` function or in an `init` block:
    56  
    57  ```go
    58  app, err := newrelic.NewApplication(
    59      newrelic.ConfigAppName("Your Application Name"),
    60      newrelic.ConfigLicense("__YOUR_NEW_RELIC_LICENSE_KEY__"),
    61  )
    62  ```
    63  
    64  This will allow you to see Go runtime information.
    65  
    66  Now, add instrumentation to your Go application to get additional performance data:
    67  * Import any of our [integration packages](https://github.com/newrelic/go-agent#integrations) for out-of-the box support for many popular Go web 
    68  frameworks and libraries. 
    69  * [Instrument Transactions](#transactions)
    70  * [Use Distributed Tracing](#distributed-tracing) (Note that this is on by default.)
    71  * [(Optional) Instrument Segments](#segments) for an extra level of timing detail
    72    * External segments are needed for Distributed Tracing
    73  * Read through the rest of this GUIDE for more instrumentation
    74  
    75  Compile and deploy your application.
    76  
    77  Find your application in the New Relic UI.  Click on it to see application performance, 
    78  including the Go runtime page that shows information about goroutine counts, garbage 
    79  collection, memory, and CPU usage. Data should show up within 5 minutes.
    80  
    81  
    82  If you are working in a development environment or running unit tests, you may
    83  not want the Go Agent to spawn goroutines or report to New Relic.  You're in
    84  luck!  Use the `ConfigEnabled` function to disable the agent.  This makes the license key
    85  optional.
    86  
    87  ```go
    88  app, err := newrelic.NewApplication(
    89      newrelic.ConfigAppName("Your Application Name"),
    90      newrelic.ConfigEnabled(false),
    91  )
    92  ```
    93  
    94  
    95  
    96  ## Full list of `Config` options and `Application` settings
    97  
    98  * [Config godoc](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic#Config)
    99  * [Application godoc](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic#Application)
   100  
   101  
   102  
   103  ## Logging
   104  
   105  For information about logs in context, please see [the documentation here](https://docs.newrelic.com/docs/logs/logs-context/logs-in-context).
   106  As of Go Agent version 3.17.0, we support logs in context with the zerolog integration.
   107  
   108  The agent's logging system is designed to be easily extensible.  By default, no
   109  logging will occur.  To enable logging, use the following config functions
   110  with an [io.Writer](https://godoc.org/github.com/pkg/io/#Writer):
   111  [ConfigInfoLogger](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#ConfigInfoLogger),
   112  which logs at info level, and
   113  [ConfigDebugLogger](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#ConfigDebugLogger)
   114  which logs at debug level.
   115  
   116  To log at debug level to standard out, set:
   117  
   118  ```go
   119  app, err := newrelic.NewApplication(
   120      newrelic.ConfigAppName("Your Application Name"),
   121      newrelic.ConfigLicense("__YOUR_NEW_RELIC_LICENSE_KEY__"),
   122      // Add debug logging:
   123      newrelic.ConfigDebugLogger(os.Stdout),
   124  )
   125  ```
   126  
   127  To log at info level to a file, set:
   128  
   129  ```go
   130  w, err := os.OpenFile("my_log_file", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0644)
   131  if nil == err {
   132      app, _ := newrelic.NewApplication(
   133          newrelic.ConfigAppName("Your Application Name"),
   134          newrelic.ConfigLicense("__YOUR_NEW_RELIC_LICENSE_KEY__"),
   135          newrelic.ConfigInfoLogger(w),
   136      )
   137  }
   138  ```
   139  
   140  Popular logging libraries `logrus`, `logxi` and `zap` are supported by
   141  integration packages:
   142  
   143  * [v3/integrations/nrlogrus](https://godoc.org/github.com/newrelic/go-agent/v3/integrations/nrlogrus/)
   144  * [v3/integrations/nrlogxi](https://godoc.org/github.com/newrelic/go-agent/v3/integrations/nrlogxi/)
   145  * [v3/integrations/nrzap](https://godoc.org/github.com/newrelic/go-agent/v3/integrations/nrzap/)
   146  
   147  ## Transactions
   148  
   149  * [Transaction godoc](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic#Transaction)
   150  * [Naming Transactions](#naming-transactions-and-metrics)
   151  * [More info on Transactions](https://docs.newrelic.com/docs/apm/applications-menu/monitoring/transactions-page)
   152  
   153  Transactions time requests and background tasks.  The simplest way to create
   154  transactions is to use `Application.StartTransaction` and `Transaction.End`.
   155  
   156  ```go
   157  txn := app.StartTransaction("transactionName")
   158  defer txn.End()
   159  ```
   160  
   161  If you are instrumenting a background transaction, this is all that is needed. If, however,
   162  you are instrumenting a web transaction, you will want to use the
   163   `SetWebRequestHTTP` and `SetWebResponse` methods as well.
   164  
   165  [SetWebRequestHTTP](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#Transaction.SetWebRequestHTTP)
   166  marks the transaction as a web transaction. If the [http.Request](https://godoc.org/net/http#Request)
   167  is non-nil, `SetWebRequestHTTP` will additionally collect details on request
   168  attributes, url, and method. If headers are present, the agent will look for a
   169  distributed tracing header.
   170  
   171  If you want to mark a transaction as a web transaction, but don't have access
   172   to an `http.Request`, you can use the [SetWebRequest](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#Transaction.SetWebRequest)
   173  method, using a manually constructed [WebRequest](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#WebRequest)
   174  object.
   175  
   176  [SetWebResponse](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#Transaction.SetWebResponse)
   177  allows the Transaction to instrument response code and response headers. Pass in
   178  your [http.ResponseWriter](https://godoc.org/net/http#ResponseWriter) as a
   179  parameter, and then use the return value of this method in place of the input
   180  parameter in your instrumentation.
   181  
   182  Here is an example using both methods:
   183  
   184  ```go
   185  func (h *handler) ServeHTTP(writer http.ResponseWriter, req *http.Request) {
   186      txn := h.App.StartTransaction("transactionName")
   187      defer txn.End()
   188      // This marks the transaction as a web transactions and collects details on
   189      // the request attributes
   190      txn.SetWebRequestHTTP(req)
   191      // This collects details on response code and headers. Use the returned
   192      // Writer from here on.
   193      writer = txn.SetWebResponse(writer)
   194      // ... handler code continues here using the new writer
   195  }
   196  ```
   197  
   198  The transaction has helpful methods like `NoticeError` and `SetName`.
   199  See more in [godocs](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic#Transaction).
   200  
   201  If you are using [`http.ServeMux`](https://golang.org/pkg/net/http/#ServeMux),
   202  use `WrapHandle` and `WrapHandleFunc`.  These wrappers automatically start and
   203  end transactions with the request and response writer.
   204  
   205  ```go
   206  http.HandleFunc(newrelic.WrapHandleFunc(app, "/users", usersHandler))
   207  ```
   208  
   209  To access the transaction in your handler, we recommend getting it from the
   210   Request context:
   211  
   212  ```go
   213  func myHandler(w http.ResponseWriter, r *http.Request) {
   214      txn := newrelic.FromContext(r.Context())
   215      // ... handler code here
   216  }
   217  ```
   218  
   219  To monitor a transaction across multiple goroutines, use
   220  `Transaction.NewGoroutine()`. The `NewGoroutine` method returns a new reference
   221  to the `Transaction`, which is required by each segment-creating goroutine. It
   222  does not matter if you call `NewGoroutine` before or after the other goroutine
   223  starts.
   224  
   225  ```go
   226  go func(txn newrelic.Transaction) {
   227  	defer txn.StartSegment("async").End()
   228  	time.Sleep(100 * time.Millisecond)
   229  }(txn.NewGoroutine())
   230  ```
   231  
   232  ## Segments
   233  
   234  Find out where the time in your transactions is being spent!
   235  
   236  `Segment` is used to instrument functions, methods, and blocks of code. A
   237  segment begins when its `StartTime` field is populated, and finishes when its
   238  `End` method is called.
   239  
   240  ```go
   241  segment := newrelic.Segment{}
   242  segment.Name = "mySegmentName"
   243  segment.StartTime = txn.StartSegmentNow()
   244  // ... code you want to time here ...
   245  segment.End()
   246  ```
   247  
   248  `Transaction.StartSegment` is a convenient helper.  It creates a segment and
   249   starts it:
   250  
   251  ```go
   252  segment := txn.StartSegment("mySegmentName")
   253  // ... code you want to time here ...
   254  segment.End()
   255  ```
   256  
   257  Timing a function is easy using `StartSegment` and `defer`.  Just add the
   258  following line to the beginning of that function:
   259  
   260  ```go
   261  defer txn.StartSegment("mySegmentName").End()
   262  ```
   263  
   264  Segments may be nested.  The segment being ended must be the most recently
   265  started segment.
   266  
   267  ```go
   268  s1 := txn.StartSegment("outerSegment")
   269  s2 := txn.StartSegment("innerSegment")
   270  // s2 must be ended before s1
   271  s2.End()
   272  s1.End()
   273  ```
   274  
   275  A zero value segment may safely be ended.  Therefore, the following code
   276  is safe even if the conditional fails:
   277  
   278  ```go
   279  var s newrelic.Segment
   280  txn := newrelic.FromContext(ctx)
   281  if shouldDoSomething() {
   282      s.StartTime = txn.StartSegmentNow(),
   283  }
   284  // ... code you wish to time here ...
   285  s.End()
   286  ```
   287  
   288  ### Datastore Segments
   289  
   290  Datastore segments appear in the transaction "Breakdown table" and in the
   291  "Databases" page.
   292  
   293  * [More info on Databases page](https://docs.newrelic.com/docs/apm/applications-menu/monitoring/databases-slow-queries-page)
   294  
   295  Datastore segments are instrumented using
   296  [DatastoreSegment](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#DatastoreSegment).
   297  Just like basic segments, datastore segments begin when the `StartTime` field
   298  is populated and finish when the `End` method is called.  Here is an example:
   299  
   300  ```go
   301  s := newrelic.DatastoreSegment{
   302      // Product is the datastore type.  See the constants in
   303      // https://github.com/newrelic/go-agent/blob/master/v3/newrelic/datastore.go.  Product
   304      // is one of the fields primarily responsible for the grouping of Datastore
   305      // metrics.
   306      Product: newrelic.DatastoreMySQL,
   307      // Collection is the table or group being operated upon in the datastore,
   308      // e.g. "users_table".  This becomes the db.collection attribute on Span
   309      // events and Transaction Trace segments.  Collection is one of the fields
   310      // primarily responsible for the grouping of Datastore metrics.
   311      Collection: "users_table",
   312      // Operation is the relevant action, e.g. "SELECT" or "GET".  Operation is
   313      // one of the fields primarily responsible for the grouping of Datastore
   314      // metrics.
   315      Operation: "SELECT",
   316  }
   317  s.StartTime = txn.StartSegmentNow()
   318  // ... make the datastore call
   319  s.End()
   320  ```
   321  
   322  This may be combined into two lines when instrumenting a datastore call
   323  that spans an entire function call:
   324  
   325  ```go
   326  s := newrelic.DatastoreSegment{
   327      StartTime:  txn.StartSegmentNow(),
   328      Product:    newrelic.DatastoreMySQL,
   329      Collection: "my_table",
   330      Operation:  "SELECT",
   331  }
   332  defer s.End()
   333  ```
   334  
   335  If you are using the standard library's
   336  [database/sql](https://golang.org/pkg/database/sql/) package with
   337  [MySQL](https://github.com/go-sql-driver/mysql),
   338  [PostgreSQL](https://github.com/lib/pq), or
   339  [SQLite](https://github.com/mattn/go-sqlite3) then you can avoid creating
   340  DatastoreSegments by hand by using an integration package:
   341  
   342  * [v3/integrations/nrpq](https://godoc.org/github.com/newrelic/go-agent/v3/integrations/nrpq)
   343  * [v3/integrations/nrmysql](https://godoc.org/github.com/newrelic/go-agent/v3/integrations/nrmysql)
   344  * [v3/integrations/nrsqlite3](https://godoc.org/github.com/newrelic/go-agent/v3/integrations/nrsqlite3)
   345  * [v3/integrations/nrmongo](https://godoc.org/github.com/newrelic/go-agent/v3/integrations/nrmongo)
   346  
   347  ### External Segments
   348  
   349  External segments appear in the transaction "Breakdown table" and in the
   350  "External services" page. Version 1.11.0 of the Go Agent adds support for
   351  cross-application tracing (CAT), which will result in external segments also
   352  appearing in the "Service maps" page and being linked in transaction traces when
   353  both sides of the request have traces. Version 2.1.0 of the Go Agent adds
   354  support for distributed tracing, which lets you see the path a request takes as
   355  it travels through distributed APM apps.
   356  
   357  * [More info on External Services page](https://docs.newrelic.com/docs/apm/applications-menu/monitoring/external-services-page)
   358  * [More info on Cross-Application Tracing](https://docs.newrelic.com/docs/apm/transactions/cross-application-traces/introduction-cross-application-traces)
   359  * [More info on Distributed Tracing](https://docs.newrelic.com/docs/apm/distributed-tracing/getting-started/introduction-distributed-tracing)
   360  
   361  External segments are instrumented using `ExternalSegment`. There are three
   362  ways to use this functionality:
   363  
   364  1. Using `StartExternalSegment` to create an `ExternalSegment` before the
   365     request is sent, and then calling `ExternalSegment.End` when the external
   366     request is complete.
   367  
   368     For CAT support to operate, an `http.Request` must be provided to
   369     `StartExternalSegment`, and the `ExternalSegment.Response` field must be set
   370     before `ExternalSegment.End` is called or deferred.
   371  
   372     For example:
   373  
   374      ```go
   375      func external(txn *newrelic.Transaction, req *http.Request) (*http.Response, error) {
   376        s := newrelic.StartExternalSegment(txn, req)
   377        response, err := http.DefaultClient.Do(req)
   378        s.Response = response
   379        s.End()
   380        return response, err
   381      }
   382      ```
   383  
   384      If the transaction is `nil` then `StartExternalSegment` will look for a
   385      transaction in the request's context using
   386      [FromContext](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#FromContext).
   387  
   388  2. Using `NewRoundTripper` to get a
   389     [`http.RoundTripper`](https://golang.org/pkg/net/http/#RoundTripper) that
   390     will automatically instrument all requests made via
   391     [`http.Client`](https://golang.org/pkg/net/http/#Client) instances that use
   392     that round tripper as their `Transport`. This option results in CAT support,
   393     provided the Go Agent is version 1.11.0, and in distributed tracing support,
   394     provided the Go Agent is version 2.1.0.  `NewRoundTripper` will look for a
   395     transaction in the request's context using
   396     [FromContext](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#FromContext).
   397  
   398     For example:
   399  
   400      ```go
   401      client := &http.Client{}
   402      client.Transport = newrelic.NewRoundTripper(client.Transport)
   403      request, _ := http.NewRequest("GET", "http://example.com", nil)
   404      // Put transaction in the request's context:
   405      request = newrelic.RequestWithTransactionContext(request, txn)
   406      resp, err := client.Do(request)
   407      ```
   408  
   409  3. Directly creating an `ExternalSegment` via a struct literal with an explicit
   410     `URL` or `Request`, and then calling `ExternalSegment.End`. This option does
   411     not support CAT, and may be removed or changed in a future major version of
   412     the Go Agent. As a result, we suggest using one of the other options above
   413     wherever possible.
   414  
   415     For example:
   416  
   417      ```go
   418      func external(txn newrelic.Transaction, url string) (*http.Response, error) {
   419        es := newrelic.ExternalSegment{
   420          StartTime: txn.StartSegmentNow(),
   421          URL:   url,
   422        }
   423        defer es.End()
   424  
   425        return http.Get(url)
   426      }
   427      ```
   428  
   429  ### Message Producer Segments
   430  
   431  Message producer segments appear in the transaction "Breakdown table".
   432  
   433  Message producer segments are instrumented using
   434  [MessageProducerSegment](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic/#MessageProducerSegment).
   435  Just like basic segments, message producer segments begin when the `StartTime`
   436  field is populated and finish when the `End` method is called.  Here is an
   437  example:
   438  
   439  ```go
   440  s := newrelic.MessageProducerSegment{
   441      // Library is the name of the library instrumented.
   442      Library: "RabbitMQ",
   443      // DestinationType is the destination type.
   444      DestinationType: newrelic.MessageExchange,
   445      // DestinationName is the name of your queue or topic.
   446      DestinationName: "myExchange",
   447      // DestinationTemporary must be set to true if destination is temporary
   448      // to improve metric grouping.
   449      DestinationTemporary: false,
   450  }
   451  s.StartTime = txn.StartSegmentNow()
   452  // ... add message to queue here
   453  s.End()
   454  ```
   455  
   456  This may be combined into two lines when instrumenting a message producer
   457  call that spans an entire function call:
   458  
   459  ```go
   460  s := newrelic.MessageProducerSegment{
   461      StartTime:            txn.StartSegmentNow(),
   462      Library:              "RabbitMQ",
   463      DestinationType:      newrelic.MessageExchange,
   464      DestinationName:      "myExchange",
   465      DestinationTemporary: false,
   466  }
   467  defer s.End()
   468  ```
   469  
   470  ## Attributes
   471  
   472  Attributes add context to errors and allow you to filter performance data
   473  in Insights.
   474  
   475  You may add them using the `Transaction.AddAttribute` and `Segment.AddAttribute`
   476  methods.
   477  
   478  ```go
   479  txn.AddAttribute("key", "value")
   480  txn.AddAttribute("product", "widget")
   481  txn.AddAttribute("price", 19.99)
   482  txn.AddAttribute("importantCustomer", true)
   483  
   484  seg.AddAttribute("count", 14)
   485  ```
   486  
   487  * [More info on Custom Attributes](https://docs.newrelic.com/docs/insights/new-relic-insights/decorating-events/insights-custom-attributes)
   488  
   489  Some attributes are recorded automatically.  These are called agent attributes.
   490  They are listed here:
   491  
   492  * [newrelic package constants](https://godoc.org/github.com/newrelic/go-agent/v3/newrelic#pkg-constants)
   493  
   494  To disable one of these agents attributes, for example `AttributeHostDisplayName`,
   495  modify the config like this:
   496  
   497  ```go
   498  app, err := newrelic.NewApplication(
   499      newrelic.ConfigAppName("Your Application Name"),    
   500      newrelic.ConfigLicense("__YOUR_NEW_RELIC_LICENSE_KEY__"),
   501      func(cfg *newrelic.Config) {
   502          config.Attributes.Exclude = append(config.Attributes.Exclude, newrelic.AttributeHostDisplayName)
   503      }
   504  )
   505  ```
   506  
   507  * [More info on Agent Attributes](https://docs.newrelic.com/docs/agents/manage-apm-agents/agent-metrics/agent-attributes)
   508  
   509  ## Tracing
   510  
   511  New Relic's [distributed tracing](https://docs.newrelic.com/docs/apm/distributed-tracing/getting-started/introduction-distributed-tracing)
   512  is the next generation of the previous cross-application tracing feature.
   513  Compared to cross-application tracing, distributed tracing gives more detail
   514  about cross-service activity and provides more complete end-to-end
   515  visibility.  This section discusses distributed tracing and cross-application
   516  tracing in turn.
   517  
   518  ### Distributed Tracing
   519  
   520  New Relic's [distributed
   521  tracing](https://docs.newrelic.com/docs/apm/distributed-tracing/getting-started/introduction-distributed-tracing)
   522  feature lets you see the path that a request takes as it travels through distributed APM
   523  apps, which is vital for applications implementing a service-oriented or
   524  microservices architecture. Support for distributed tracing was added in
   525  version 2.1.0 of the Go Agent.
   526  
   527  The config's `DistributedTracer.Enabled` field has to be set. When true, the
   528  agent will add distributed tracing headers in outbound requests, and scan
   529  incoming requests for distributed tracing headers. Distributed tracing will
   530  override cross-application tracing.
   531  
   532  ```go
   533  app, err := newrelic.NewApplication(
   534      newrelic.ConfigAppName("Your Application Name"),
   535      newrelic.ConfigLicense("__YOUR_NEW_RELIC_LICENSE_KEY__"),
   536      newrelic.ConfigDistributedTracerEnabled(true),  
   537  )
   538  ```
   539  
   540  ### Cross-Application Tracing [Deprecated]
   541  
   542  New Relic's
   543  [cross-application tracing](https://docs.newrelic.com/docs/apm/transactions/cross-application-traces/introduction-cross-application-traces)
   544  feature, or CAT for short, links transactions between applications in APM to
   545  help identify performance problems within your service-oriented architecture.
   546  Support for CAT was added in version 1.11.0 of the Go Agent. We recommend using
   547  [Distributed Tracing](#distributed-tracing) as the most recent, complete feature.
   548  
   549  As CAT uses HTTP headers to track requests across applications, the Go Agent
   550  needs to be able to access and modify request and response headers both for
   551  incoming and outgoing requests.
   552  
   553  ### Tracing Instrumentation
   554  
   555  Both distributed tracing and cross-application tracing work by propagating
   556  [header information](https://docs.newrelic.com/docs/apm/distributed-tracing/getting-started/how-new-relic-distributed-tracing-works#headers)
   557  from service to service in a request path. In many scenarios, the Go Agent offers tracing instrumentation
   558  out-of-the-box, for both distributed tracing and cross-application tracing. For other scenarios customers may implement
   559  distributed tracing based on the examples provided in this guide.
   560  
   561  #### Getting Tracing Instrumentation Out-of-the-Box
   562  
   563  The Go Agent automatically creates and propagates tracing header information
   564  for each of the following scenarios:
   565  
   566  For server applications:
   567  
   568  1. Using `WrapHandle` or `WrapHandleFunc` to instrument a server that
   569     uses [`http.ServeMux`](https://golang.org/pkg/net/http/#ServeMux)
   570     ([Example](v3/examples/server/main.go)).
   571  
   572  2. Using any of the Go Agent's HTTP integrations, which are listed [here
   573  ](README.md#integrations).
   574  
   575  3. Using another framework or [`http.Server`](https://golang.org/pkg/net/http/#Server) while ensuring that:
   576  
   577        1. After calling `StartTransaction`, make sure to call `Transaction.SetWebRequest`
   578        and `Transaction.SetWebResponse` on the transaction, and
   579        2. the `http.ResponseWriter` that is returned from `Transaction.SetWebResponse`
   580        is used instead of calling `WriteHeader` directly on the original response
   581        writer, as described in the [transactions section of this guide](#transactions)
   582         ([Example](v3/examples/server-http/main.go)).
   583  
   584  For client applications:
   585  
   586  1. Using `NewRoundTripper`, as described in the
   587     [external segments section of this guide](#external-segments)
   588     ([Example](v3/examples/client-round-tripper/main.go)).
   589  
   590  2. Using the call `StartExternalSegment` and providing an `http.Request`, as
   591     described in the [external segments section of this guide](#external-segments)
   592     ([Example](v3/examples/client/main.go)).
   593  
   594  #### Manually Implementing Distributed Tracing
   595  
   596  Consider [manual instrumentation](https://docs.newrelic.com/docs/apm/distributed-tracing/enable-configure/enable-distributed-tracing#agent-apis)
   597  for services not instrumented automatically by the Go Agent. In such scenarios, the
   598  calling service has to insert the appropriate header(s) into the request headers:
   599  
   600  ```go
   601  var h http.Headers
   602  callingTxn.InsertDistributedTraceHeaders(h)
   603  ```
   604  
   605  These headers have to be added to the call to the destination service, which in
   606  turn invokes the call for accepting the headers:
   607  
   608  ```go
   609  var h http.Headers
   610  calledTxn.AcceptDistributedTraceHeaders(newrelic.TransportOther, h)
   611  ```
   612  
   613  A complete example can be found
   614  [here](v3/examples/custom-instrumentation/main.go).
   615  
   616  
   617  ## Custom Metrics
   618  
   619  * [More info on Custom Metrics](https://docs.newrelic.com/docs/agents/go-agent/instrumentation/create-custom-metrics-go)
   620  
   621  You may [create custom metrics](https://docs.newrelic.com/docs/agents/manage-apm-agents/agent-data/collect-custom-metrics)
   622  via the `RecordCustomMetric` method.
   623  
   624  ```go
   625  app.RecordCustomMetric(
   626      "CustomMetricName", // Name of your metric
   627      132,                // Value
   628  )
   629  ```
   630  
   631  **Note:** The Go Agent will automatically prepend the metric name you pass to
   632  `RecordCustomMetric` (`"CustomMetricName"` above) with the string `Custom/`.
   633  This means the above code would produce a metric named
   634  `Custom/CustomMetricName`.  You'll also want to read over the
   635  [Naming Transactions and Metrics](#naming-transactions-and-metrics) section
   636  below for advice on coming up with appropriate metric names.
   637  
   638  ## Custom Events
   639  
   640  You may track arbitrary events using custom Insights events.
   641  
   642  ```go
   643  app.RecordCustomEvent("MyEventType", map[string]interface{}{
   644      "myString": "hello",
   645      "myFloat":  0.603,
   646      "myInt":    123,
   647      "myBool":   true,
   648  })
   649  ```
   650  
   651  ## Request Queuing
   652  
   653  If you are running a load balancer or reverse web proxy then you may configure
   654  it to add a `X-Queue-Start` header with a Unix timestamp.  This will create a
   655  band on the application overview chart showing queue time.
   656  
   657  * [More info on Request Queuing](https://docs.newrelic.com/docs/apm/applications-menu/features/request-queuing-tracking-front-end-time)
   658  
   659  ## Error Reporting
   660  
   661  The Go Agent captures errors in three different ways:
   662  
   663  1. [the Transaction.NoticeError method](#noticeerror)
   664  2. [panics recovered in defer Transaction.End](#panics)
   665  3. [error response status codes recorded with Transaction.WriteHeader](#error-response-codes)
   666  
   667  ### NoticeError
   668  
   669  You may track errors using the `Transaction.NoticeError` method.  The easiest
   670  way to get started with `NoticeError` is to use errors based on
   671  [Go's standard error interface](https://blog.golang.org/error-handling-and-go).
   672  
   673  ```go
   674  txn.NoticeError(errors.New("my error message"))
   675  ```
   676  
   677  `NoticeError` will work with *any* sort of object that implements Go's standard
   678  error type interface -- not just `errorStrings` created via `errors.New`.  
   679  
   680  If you're interested in sending more than an error *message* to New Relic, the
   681  Go Agent also offers a `newrelic.Error` struct.
   682  
   683  ```go
   684  txn.NoticeError(newrelic.Error{
   685      Message: "my error message",
   686      Class:   "IdentifierForError",
   687      Attributes: map[string]interface{}{
   688          "important_number": 97232,
   689          "relevant_string":  "zap",
   690      },
   691  })
   692  ```
   693  
   694  Using the `newrelic.Error` struct requires you to manually marshal your error
   695  data into the `Message`, `Class`, and `Attributes` fields.  However, there's two
   696  **advantages** to using the `newrelic.Error` struct.
   697  
   698  First, by setting an error `Class`, New Relic will be able to aggregate errors
   699  in the *Error Analytics* section of APM.  Second, the `Attributes` field allows
   700  you to send through key/value pairs with additional error debugging information
   701  (also exposed in the *Error Analytics* section of APM).
   702  
   703  ### Panics
   704  
   705  When the Transaction is ended using `defer`, the Transaction will optionally recover any
   706  panic that occurs, record it as an error, and re-throw it. You can enable this feature by
   707  setting the configuration:
   708  
   709  ```go
   710  app, err := newrelic.NewApplication(
   711      newrelic.ConfigAppName("Your Application Name"),
   712      newrelic.ConfigLicense("__YOUR_NEW_RELIC_LICENSE_KEY__"),
   713      func(cfg *newrelic.Config) {
   714          cfg.ErrorCollector.RecordPanics = true
   715      }
   716  )
   717  ```
   718  
   719  As a result of this configuration, panics may appear to be originating from `Transaction.End`.
   720  
   721  ```go
   722  func unstableTask(app newrelic.Application) {
   723      txn := app.StartTransaction("unstableTask", nil, nil)
   724      defer txn.End()
   725  
   726      // This panic will be recorded as an error.
   727      panic("something went wrong")
   728  }
   729  ```
   730  
   731  ### Error Response Codes
   732  
   733  Setting the WebResponse on the transaction using `Transaction.SetWebResponse`
   734  returns an
   735  [http.ResponseWriter](https://golang.org/pkg/net/http/#ResponseWriter), and you
   736  can use that returned ResponseWriter to call `WriteHeader` to record the response
   737  status code.  The transaction will record an error if the status code is
   738  at or above 400 or strictly below 100 and not in the ignored status codes
   739  configuration list.  The ignored status codes list is configured by the
   740  `Config.ErrorCollector.IgnoreStatusCodes` field or within the New Relic UI
   741  if your application has server side configuration enabled.
   742  
   743  As a result, using `Transaction.NoticeError` in situations where your code is
   744  returning an erroneous status code may result in redundant errors.
   745  `NoticeError` is not affected by the ignored status codes configuration list.
   746  
   747  ## Naming Transactions and Metrics
   748  
   749  You'll want to think carefully about how you name your transactions and custom
   750  metrics.  If your program creates too many unique names, you may end up with a
   751  [Metric Grouping Issue (or MGI)](https://docs.newrelic.com/docs/agents/manage-apm-agents/troubleshooting/metric-grouping-issues).
   752  
   753  MGIs occur when the granularity of names is too fine, resulting in hundreds or
   754  thousands of uniquely identified metrics and transactions.  One common cause of
   755  MGIs is relying on the full URL name for metric naming in web transactions.  A
   756  few major code paths may generate many different full URL paths to unique
   757  documents, articles, page, etc. If the unique element of the URL path is
   758  included in the metric name, each of these common paths will have its own unique
   759  metric name.
   760  
   761  ## Browser
   762  
   763  To enable support for
   764  [New Relic Browser](https://docs.newrelic.com/docs/browser), your HTML pages
   765  must include a JavaScript snippet that will load the Browser agent and
   766  configure it with the correct application name. This snippet is available via
   767  the `Transaction.BrowserTimingHeader` method.  Include the byte slice returned
   768  by `Transaction.BrowserTimingHeader().WithTags()` as early as possible in the
   769  `<head>` section of your HTML after any `<meta charset>` tags.
   770  
   771  ```go
   772  func indexHandler(w http.ResponseWriter, req *http.Request) {
   773      io.WriteString(w, "<html><head>")
   774      // The New Relic browser javascript should be placed as high in the
   775      // HTML as possible.  We suggest including it immediately after the
   776      // opening <head> tag and any <meta charset> tags.
   777      txn := newrelic.FromContext(req.Context())
   778      hdr, err := txn.BrowserTimingHeader()
   779      if nil != err {
   780          log.Printf("unable to create browser timing header: %v", err)
   781      }
   782      // BrowserTimingHeader() will always return a header whose methods can
   783      // be safely called.
   784      if js := hdr.WithTags(); js != nil {
   785          w.Write(js)
   786      }
   787      io.WriteString(w, "</head><body>browser header page</body></html>")
   788  }
   789  ```
   790  
   791  
   792  ## For More Help
   793  
   794  There's a variety of places online to learn more about the Go Agent.
   795  
   796  [The New Relic docs site](https://docs.newrelic.com/docs/agents/go-agent/get-started/introduction-new-relic-go)
   797  contains a number of useful code samples and more context about how to use the Go Agent.
   798  
   799  [New Relic's discussion forums](https://discuss.newrelic.com) have a dedicated
   800  public forum [for the Go Agent](https://discuss.newrelic.com/c/support-products-agents/go-agent).
   801  
   802  When in doubt, [the New Relic support site](https://support.newrelic.com/) is
   803  the best place to get started troubleshooting an agent issue.