# Go Plugin System over RPC
`go-plugin` is a Go (golang) plugin system over RPC. It is the plugin system
-that has been in use by HashiCorp tooling for over 3 years. While initially
-created for [Packer](https://www.packer.io), it has since been used by
-[Terraform](https://www.terraform.io) and [Otto](https://www.ottoproject.io),
-with plans to also use it for [Nomad](https://www.nomadproject.io) and
+that has been in use by HashiCorp tooling for over 4 years. While initially
+created for [Packer](https://www.packer.io), it is additionally in use by
+[Terraform](https://www.terraform.io), [Nomad](https://www.nomadproject.io), and
[Vault](https://www.vaultproject.io).
While the plugin system is over RPC, it is currently only designed to work
you just use and call functions on an interface as if it were in the same
process. This plugin system handles the communication in between.
+**Cross-language support.** Plugins can be written (and consumed) by
+almost every major language. This library supports serving plugins via
+[gRPC](http://www.grpc.io). gRPC-based plugins enable plugins to be written
+in any language.
+
**Complex arguments and return values are supported.** This library
provides APIs for handling complex arguments and return values such
as interfaces, `io.Reader/Writer`, etc. We do this by giving you a library
**Built-in Logging.** Any plugins that use the `log` standard library
will have log data automatically sent to the host process. The host
process will mirror this output prefixed with the path to the plugin
-binary. This makes debugging with plugins simple.
+binary. This makes debugging with plugins simple. If the host system
+uses [hclog](https://github.com/hashicorp/go-hclog) then the log data
+will be structured. If the plugin also uses hclog, logs from the plugin
+will be sent to the host hclog and be structured.
**Protocol Versioning.** A very basic "protocol version" is supported that
can be incremented to invalidate any previous plugins. This is useful when
properly. `NewClient` takes a `ReattachConfig` to determine if and how to
reattach.
+**Cryptographically Secure Plugins.** Plugins can be verified with an expected
+checksum and RPC communications can be configured to use TLS. The host process
+must be properly secured to protect this configuration.
+
## Architecture
The HashiCorp plugin system works by launching subprocesses and communicating
-over RPC (using standard `net/rpc`). A single connection is made between
-any plugin and the host process, and we use a
-[connection multiplexing](https://github.com/hashicorp/yamux)
-library to multiplex any other connections on top.
+over RPC (using standard `net/rpc` or [gRPC](http://www.grpc.io)). A single
+connection is made between any plugin and the host process. For net/rpc-based
+plugins, we use a [connection multiplexing](https://github.com/hashicorp/yamux)
+library to multiplex any other connections on top. For gRPC-based plugins,
+the HTTP2 protocol handles multiplexing.
This architecture has a number of benefits:
panic the plugin user.
* Plugins are very easy to write: just write a Go application and `go build`.
- Theoretically you could also use another language as long as it can
- communicate the Go `net/rpc` protocol but this hasn't yet been tried.
+ Or use any other language to write a gRPC server with a tiny amount of
+ boilerplate to support go-plugin.
* Plugins are very easy to install: just put the binary in a location where
the host will find it (depends on the host but this library also provides
* Plugins can be relatively secure: The plugin only has access to the
interfaces and args given to it, not to the entire memory space of the
- process. More security features are planned (see the coming soon section
- below).
+ process. Additionally, go-plugin can communicate with the plugin over
+ TLS.
## Usage
1. Choose the interface(s) you want to expose for plugins.
2. For each interface, implement an implementation of that interface
- that communicates over an `*rpc.Client` (from the standard `net/rpc`
- package) for every function call. Likewise, implement the RPC server
- struct this communicates to which is then communicating to a real,
- concrete implementation.
+ that communicates over a `net/rpc` connection or over a
+ [gRPC](http://www.grpc.io) connection or both. You'll have to implement
+ both a client and server implementation.
3. Create a `Plugin` implementation that knows how to create the RPC
client/server for a given plugin type.
At this point in time, the roadmap for the plugin system is:
-**Cryptographically Secure Plugins.** We'll implement signing plugins
-and loading signed plugins in order to allow Vault to make use of multi-process
-in a secure way.
-
**Semantic Versioning.** Plugins will be able to implement a semantic version.
This plugin system will give host processes a system for constraining
versions. This is in addition to the protocol versioning already present
When we started using plugins (late 2012, early 2013), plugins over RPC
were the only option since Go didn't support dynamic library loading. Today,
-Go still doesn't support dynamic library loading, but they do intend to.
-Since 2012, our plugin system has stabilized from millions of users using it,
-and has many benefits we've come to value greatly.
-
-For example, we intend to use this plugin system in
-[Vault](https://www.vaultproject.io), and dynamic library loading will
-simply never be acceptable in Vault for security reasons. That is an extreme
+Go supports the [plugin](https://golang.org/pkg/plugin/) standard library with
+a number of limitations. Since 2012, our plugin system has stabilized
+from tens of millions of users using it, and has many benefits we've come to
+value greatly.
+
+For example, we use this plugin system in
+[Vault](https://www.vaultproject.io) where dynamic library loading is
+not acceptable for security reasons. That is an extreme
example, but we believe our library system has more upsides than downsides
over dynamic library loading and since we've had it built and tested for years,
-we'll likely continue to use it.
+we'll continue to use it.
Shared libraries have one major advantage over our system which is much
higher performance. In real world scenarios across our various tools,
we've never required any more performance out of our plugin system and it
has seen very high throughput, so this isn't a concern for us at the moment.
-
projects / github / fretlink / terraform-provider-statuscake.git / blobdiff