github.com/hashicorp/packer@v1.14.3/website/content/docs/plugins/creation/index.mdx (about)

     1  ---
     2  description: |
     3    Learn about extending Packer by creating custom plugins for builders, provisioners, and
     4    post-processors. 
     5  page_title: Create custom plugins to extend Packer overview
     6  ---
     7  
     8  # Create Custom Plugins to Extend Packer
     9  
    10  Packer is extensible and supports plugins that let you
    11  create and use custom builders, provisioners, post-processors, and data sources. This page explains how to develop Packer plugins. Before you begin, we recommend reviewing the Packer documentation and the instructions for [installing external plugins](/packer/docs/plugins/install).
    12  
    13  ~> **Warning** This is an advanced topic. You should have strong knowledge of Packer before you start writing plugins.
    14  
    15  ## Language Requirements
    16  
    17  You must write Packer plugins in [Go](https://go.dev/).
    18  
    19  
    20  ## Plugin System Architecture
    21  
    22  A Packer plugin is just a Go binary. Instead of loading plugins directly into a
    23  running application, Packer runs each plugin as a _separate application_.
    24  The multiple separate Packer plugin processes communicate with the Core using
    25  an RPC defined in the packer-plugin SDK. The Packer core itself is responsible
    26  launching and cleaning up the plugin processes.
    27  
    28  ## Plugin Development Basics
    29  
    30  The components that can be created and used in a Packer plugin are builders,
    31  provisioners, post-processors, and data sources.
    32  
    33  Each of these components has a corresponding [interface](https://go.dev/doc/effective_go.html#interfaces_and_types).
    34  
    35  All you need to do to create a plugin is:
    36  
    37  1. create an implementation of the desired interface, and
    38  2. serve it using the server provided in the [packer-plugin-sdk](https://github.com/hashicorp/packer-plugin-sdk).
    39  
    40  The core and the SDK handle all of the communication details inside the server.
    41  
    42  Your plugin must use two packages from the SDK to implement the server and
    43  interfaces. You're encouraged to use whatever other packages you want in your
    44  plugin implementation. Because plugins are their own processes, there is no
    45  danger of colliding dependencies.
    46  
    47  - [`github.com/hashicorp/packer-plugin-sdk/packer`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer) - Contains all the interfaces that you have to implement for any given plugin.
    48  
    49  - [`github.com/hashicorp/packer-plugin-sdk/plugin`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/plugin) - Contains the code to serve the plugin. This handles all the inter-process communication.
    50  
    51  Basic examples of serving your component are shown below. 
    52  
    53  ```go
    54  // main.go
    55  
    56  import (
    57    "github.com/hashicorp/packer-plugin-sdk/plugin"
    58  )
    59  
    60  // Assume this implements the packer.Builder interface
    61  type ExampleBuilder struct{}
    62  
    63  // Assume this implements the packer.PostProcessor interface
    64  type FooPostProcessor struct{}
    65  
    66  // Assume this implements the packer.Provisioner interface
    67  type BarProvisioner struct{}
    68  
    69  func main() {
    70  	pps := plugin.NewSet()
    71  	pps.RegisterBuilder("example", new(ExampleBuilder))
    72  	pps.RegisterBuilder(plugin.DEFAULT_NAME, new(AnotherBuilder))
    73  	pps.RegisterPostProcessor("foo", new(FooPostProcessor))
    74  	pps.RegisterProvisioner("bar", new(BarProvisioner))
    75  	err := pps.Run()
    76  	if err != nil {
    77  		fmt.Fprintln(os.Stderr, err.Error())
    78  		os.Exit(1)
    79  	}
    80  }
    81  ```
    82  
    83  This `plugin.NewSet` invocation handles all the details of communicating with
    84  Packer core and serving your component over RPC. As long as your struct being
    85  registered implements one of the component interfaces, Packer will now be able
    86  to launch your plugin and use it.
    87  
    88  If you register a component with its own name, the component name will be
    89  appended to the plugin name to create a unique name. If you register a component
    90  using the special string constant `plugin.DEFAULT_NAME`, then the component will
    91  be referenced by using only the plugin name. For example:
    92  
    93  If your plugin is named `packer-plugin-my`, the above set definition would make
    94  the following components available:
    95  
    96  - the `my-example` builder
    97  - the `my` builder
    98  - the `my-foo` post-processor
    99  - the `my-bar` provisioner
   100  
   101  Next, build your plugin as you would any other Go application. The resulting
   102  binary is the plugin that can be installed using
   103  [standard installation procedures](https://developer.hashicorp.com/packer/docs/plugins#installing-plugins).
   104  
   105  This documentation explains how to implement each type of plugin interface: builders, data sources, provisioners, and post-processors.
   106  
   107  ~> **Lock your dependencies!** Using `go mod` is highly recommended since
   108  the Packer codebase will continue to improve, potentially breaking APIs along
   109  the way until there is a stable release. By locking your dependencies, your
   110  plugins will continue to work with the version of Packer you lock to.
   111  
   112  ## Logging and Debugging
   113  
   114  Plugins can use the standard Go `log` package to log. Anything logged using
   115  this will be available in the Packer log files automatically. The Packer log is
   116  visible on stderr when the `PACKER_LOG` environment var is set.
   117  
   118  Packer will prefix any logs from plugins with the path to that plugin to make
   119  it identifiable where the logs come from. Some example logs are shown below:
   120  
   121  ```text
   122  2013/06/10 21:44:43 Loading builder: custom
   123  2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin minimum port: 10000
   124  2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin maximum port: 25000
   125  2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin address: :10000
   126  ```
   127  
   128  As you can see, the log messages from the custom builder plugin are prefixed
   129  with "packer-builder-custom". Log output is _extremely_ helpful in debugging
   130  issues and you're encouraged to be as verbose as you need to be in order for
   131  the logs to be helpful.
   132  
   133  ### Creating a GitHub Release
   134  
   135  `packer init` does not work using a centralized registry. Instead, it requires
   136  you to publish your plugin in a GitHub repo with the name
   137  `packer-plugin-*` where \* represents the name of your plugin. You also need to
   138  create a GitHub release of your plugin with specific assets for the
   139  `packer init` download to work. We provide a pre-defined release workflow
   140  configuration using
   141  [GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions). We
   142  strongly encourage maintainers to use this configuration to make sure the
   143  release contains the right assets with the right names for Packer to leverage
   144  `packer init` installation.
   145  
   146  Here's what you need to create releases using GitHub Actions:
   147  
   148  1. Generate a GPG key to be used when signing releases (See [GitHub's detailed instructions](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/generating-a-new-gpg-key)
   149     for help with this step)
   150  2. Copy the [GoReleaser configuration from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.goreleaser.yml) to the root of your repository.
   151     ```sh
   152     curl -L -o .goreleaser.yml \
   153     https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.goreleaser.yml
   154     ```
   155  3. Copy the [GitHub Actions workflow from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.github/workflows/release.yml) to `.github/workflows/release.yml` in your repository.
   156     ```sh
   157     mkdir -p .github/workflows &&
   158     curl -L -o .github/workflows/release.yml \
   159     https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.github/workflows/release.yml
   160     ```
   161  4. Go to your repository page on GitHub and navigate to Settings > Secrets. Add
   162     the following secrets:
   163     - `GPG_PRIVATE_KEY` - Your ASCII-armored GPG private key. You can export this with `gpg --armor --export-secret-keys [key ID or email]`.
   164     - `GPG_PASSPHRASE` - The passphrase for your GPG private key.
   165  5. Push a new valid version tag (e.g. `v1.2.3`) to test that the GitHub Actions
   166     releaser is working. The tag must be a valid
   167     [Semantic Version](https://semver.org/) preceded with a `v`. Once the tag is pushed, the github actions you just configured will automatically build release binaries that Packer can download using `packer init`. For more details on how
   168     to install a plugin using `packer init`, see the
   169     [init docs](https://developer.hashicorp.com/packer/docs/commands/init).
   170  
   171  ## Registering Plugins
   172  
   173  ~> Note: Registering a plugin as an integration requires the documentation to match the [Scaffolding example layout](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/.web-docs).
   174  
   175  To help with the discovery of Packer plugins, plugins maintainers can choose to register their plugin as a [Packer Integration](https://developer.hashicorp.com/packer/integrations).
   176  
   177  The registration process requires [metadata configuration](https://github.com/hashicorp/integration-template#metadata-configuration) to be added to your plugin repository for configuring the Packer integration pipeline and 
   178  a specific directory structure for plugin documentation to be rendered on the [Packer Integrations](https://developer.hashicorp.com/packer/integrations) portal.
   179  
   180  You can execute the following steps to register your plugin as an integration:
   181  
   182  1.  Update your plugin documentation structure to match the [Scaffolding example layout](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/.web-docs).
   183  New plugins generated from this template have the necessary structure in place. If so you can jump to step 3.
   184  1. For the integrations library, only one top-level README per integration is supported. Any top-level index.mdx files that exist
   185  within a plugin's existing documentation will need to migrate to a top-level README. 
   186  1. Update your top-level integration README to include a description, plugin installation steps, available components section, and, any, additional sections
   187  needed to inform users on how to work with your integration. Refer to [Packer scaffolding plugin](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/docs/README.md) for an example. 
   188  1. Update the top-level README for each of the components within your integration to follow the structure defined in the scaffolding template.
   189  1. Add the integration configuration file [metadata.hcl](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.web-docs/metadata.hcl) to the plugins `.web-docs` directory.
   190  1. Open a request for integration issue with the Packer team - [Open Request](https://github.com/hashicorp/packer/issues/new?labels=new-plugin-contribution&template=plugin_integration.md).
   191  Provide all the requested information to help expedite the integration request. 
   192  
   193  #### [Example] Add integration files to existing plugin repository
   194  
   195  ```shell
   196  ## Update Plugin repository with integration config, workflows, and scripts
   197  cd packer-plugin-name
   198  mkdir -p .web-docs/scripts
   199  
   200  # Download packer-plugin-scaffolding repo copy files
   201  wget https://github.com/hashicorp/packer-plugin-scaffolding/archive/refs/heads/main.zip
   202  unzip main.zip
   203  cp packer-plugin-scaffolding-main/.web-docs/metadata.hcl .web-docs/
   204  cp -r packer-plugin-scaffolding-main/.web-docs/scripts/ .web-docs/scripts/
   205  cp packer-plugin-scaffolding-main/.github/workflows/notify-integration-release-via-* .github/workflows/
   206  
   207  # Remove downloaded scaffolding project
   208  rm main.zip
   209  rm -rf packer-plugin-scaffolding-main
   210  
   211  # Add the following commands to your plugin GNUmakefile
   212  generate: install-packer-sdc
   213      @go generate ./...
   214      @rm -rf .docs
   215      @packer-sdc renderdocs -src docs -partials docs-partials/ -dst .docs/
   216      @./.web-docs/scripts/compile-to-webdocs.sh "." ".docs" ".web-docs" "<orgname>"
   217      @rm -r ".docs"
   218  ```
   219  
   220  By opening an integration request, you are asking a member of the to Packer team to review your plugin integration configuration, plugin documentation, 
   221  and, finally, to open an internal pull-request to finalize the integration setup. 
   222  
   223  Plugin integrations will be listed as a [Packer Integration](https://developer.hashicorp.com/packer/integrations), with details on how to install and use your the plugin. 
   224  
   225  Plugin integrations, once deployed, can be updated manually, or automatically upon a new release, by the plugin authors. Changes to the defined documentation structure 
   226  or parent repository should be communicated to the Packer team to ensure a working integration pipeline. 
   227  
   228  ## Plugin Development Tips and FAQs
   229  
   230  ### Working Examples
   231  
   232  Here's a non-exhaustive list of Packer plugins that you can check out:
   233  
   234  - [github.com/hashicorp/packer-plugin-docker](https://github.com/hashicorp/packer-plugin-docker)
   235  - [github.com/exoscale/packer-plugin-exoscale](https://github.com/exoscale/packer-plugin-exoscale)
   236  - [github.com/sylviamoss/packer-plugin-comment](https://github.com/sylviamoss/packer-plugin-comment)
   237  
   238  Looking at their code will give you good examples.
   239  
   240  ### Naming Conventions
   241  
   242  It is standard practice to name the resulting plugin application in the format
   243  of `packer-plugin-NAME`. For example, if you're building a new builder for
   244  CustomCloud, it would be standard practice to name the resulting plugin
   245  `packer-plugin-customcloud`. This naming convention helps users identify the
   246  scope of a plugin.
   247  
   248  ### Testing Plugins
   249  
   250  To install plugins from local source, you can use the `packer plugins install` command with the `--path` flag:
   251  
   252  ```shell-session
   253  $ packer plugins install --path <path-to-binary> <hostname>/<namespace>/<plugin-name>
   254  ```
   255  
   256  For example let's install happycloud plugin from a locally sourced binary
   257  
   258  ```shell-session
   259  $ packer plugins install --path packer-plugin-happycloud github.com/hashicorp/happycloud
   260  ```
   261  
   262  This will install the happycloud plugin from the `packer-plugin-happycloud` binary so Packer can discover it.
   263  If you want to use this in an HCL2 template, you can optionally add it to the `required_plugins` section like so:
   264  
   265  ```hcl
   266    required_plugins {
   267      happycloud = {
   268        source = "github.com/hashicorp/happycloud"
   269        version = ">=0.0.1"
   270    }
   271  }
   272  ```
   273  
   274  For further information on how Packer discovers and loads plugins, you may refer to our [documentation](/packer/docs/plugins/creation/plugin-load-spec) on the subject.
   275  
   276  ### Distributing Plugins
   277  
   278  We recommend that you use a tool like the GoReleaser in order to cross-compile
   279  your plugin for every platform that Packer supports, since Go applications are
   280  platform-specific. If you have created your plugin from the
   281  [packer-plugin-scaffolding](https://github.com/hashicorp/packer-plugin-scaffolding)
   282  repo, simply tagging a commit and pushing the tag to GitHub will correctly build
   283  and release the binaries using GoReleaser.