Plugin protocol reference

How Nu runs plugins

Nu plugins must be an executable file with a filename starting with nu_plugin_. Plugins can run in one of two modes:

  1. Stdio mode, which must be supported. The plugin is passed --stdio as a command line argument. All interaction with the plugin is handled over standard input (stdin) and output (stdout). Standard error (stderr) is not redirected, and can be used by the plugin to print messages directly.

  2. Local socket mode, which may be supported (advertised via the LocalSocket feature). The plugin is passed --local-socket as the first command line argument, and then the path of the Unix domain socket or name of the Windows named pipe to use for communication. None of the standard input or output streams are redirected, and they may all be used to interact with the user's terminal. See the documentation specific to the feature for more details.

Other command line arguments are reserved for options that might be added in the future, including other communication methods. Plugins that support the protocol as described in this document should reject other arguments and print an informational message to stderr.

Immediately after spawning a plugin, Nu expects the plugin to send its encoding type. Currently two encoding types are supported: json and msgpack. The desired encoding type should be sent first with the length of the string as a single byte integer and then the encoding type string. That is, with C-like escape syntax, "\x04json" or "\x07msgpack". In this document, the JSON format will be used for readability, but the MessagePack format is largely equivalent. See the Encoding section for specific intricacies of the formats.

Nu will then send messages in the desired encoding. The first message is always Hello. The plugin must send a Hello message indicating the expected Nu version that it is compatible with, and any supported protocol features. The engine will also send a Hello message with its version, and any supported protocol features. The plugin may verify that it is compatible with the Nu version provided by the engine, but the engine will end communication with a plugin if it is determined to be unsupported. The plugin must not use protocol features it supports if they are not also confirmed to be supported by the engine in its Hello message. It is not permitted to send any other messages before sending Hello.

The plugin should then receive and respond to messages until its input stream is closed.

Typical plugin interaction after the initial handshake looks like this:

  1. The engine sends a Call. The call contains an ID used to identify the response.
  2. If the input of the call specified a stream, the engine will send stream messages. These do not need to be consumed before the plugin sends its response.
  3. The plugin sends a CallResponse, with the same ID from step 1.
  4. If the plugin specified stream data as output in the response, it should now send stream messages with the corresponding stream ID(s).

The plugin should respond to further plugin calls. The engine may send additional plugin calls before responses have been received, and it is up to the plugin to decide whether to handle each call immediately as it is received, or to process only one at a time and hold on to them for later. In any case, sending another plugin call before a response has been received should not cause an error.

The plugin may send engine calls during the execution of a call to request operations from the engine. Engine calls are only valid within the context of a call and may not be sent otherwise.

The engine may send a Goodbye message to the plugin indicating that it will no longer send any more plugin calls. Upon receiving this message, the plugin may choose not to accept any more plugin calls, and should exit after any in-progress plugin calls have finished.

Hello

After the encoding type has been decided, both the engine and plugin must send a Hello message containing relevant version and protocol support information.

FieldTypeUsage
protocolstringMust be "nu-plugin".
versionstringThe engine's version, or the target version of Nu that the plugin supports.
featuresarrayProtocol features supported by the plugin. Unrecognized elements must be ignored.

To be accepted, the version specified must be semveropen in new window compatible with the engine's version. "0.x.y" and "x.y.z" for differing values of "x" are considered to be incompatible.

Plugins may decide to refuse engine versions with more strict criteria than specified here.

Example:

{
  "Hello": {
    "protocol": "nu-plugin",
    "version": "0.94.0",
    "features": []
  }
}

Features

All features are maps that must contain at least a name key, and may contain other keys. Features that are not recognized by name must be ignored, and not cause an error. Plugins must only advertise support for features they implement, and should not determine the features they will advertise depending on the engine's Hello message.

LocalSocket feature

This feature advertises support for local socket communication, instead of stdio.

Example:

{
  "name": "LocalSocket"
}

When local socket communication is advertised to an engine supporting the feature, the engine will cease stdio communication and launch the plugin again with the --local-socket command line argument. The second argument is either a path to a Unix domain socket on Linux, Android, macOS, and other Unix-like operating systems, or the name of a named pipe (without the \\.\pipe\ prefix) on Windows.

In either case, during startup, the plugin is expected to establish two separate connections to the socket, in this order:

  1. The input stream connection, used to send messages from the engine to the plugin
  2. The output stream connection, used to send messages from the plugin to the engine

The connections are separate in order to facilitate ownership of the streams by separate threads. After these connections are both established, the engine will remove the socket, and will not accept further connections.

If local socket communication fails to initialize, the engine will abort, stop the plugin, and start it again with the stdio mode, even if the plugin supports local sockets. Whether local socket mode initialized successfully, and therefore the plugin is allowed to use stdio, can be observed when EngineInterface::is_using_stdio()open in new window returns false for Rust plugins.

Input messages

These are messages sent from the engine to the plugin. Hello and Stream messages are also included.

Call

The body of this message is a 2-tuple (array): (id, call). The engine sends unique IDs for each plugin call it makes. The ID is needed to send the CallResponse.

Signature plugin call

Ask the plugin to send its command signatures. Takes no arguments. Returns Signature or Error

Example:

{
  "Call": [0, "Signature"]
}

Run plugin call

Tell the plugin to run a command. The argument is the following map:

FieldTypeUsage
namestringThe name of the command to run
callEvaluatedCallInformation about the invocation, including arguments
inputPipelineDataHeaderPipeline input to the command

EvaluatedCall is a map:

FieldTypeUsage
headSpanThe position of the beginning of the command execution.
positionalValue arrayPositional arguments.
named2-tuple (string, Value or null) arrayNamed arguments, such as switches.

Named arguments are always sent by their long name, never their short name.

Returns PipelineData or Error.

Example:

{
  "Call": [
    0,
    {
      "Run": {
        "name": "inc",
        "call": {
          "head": {
            "start": 40400,
            "end": 40403
          },
          "positional": [
            {
              "String": {
                "val": "0.1.2",
                "span": {
                  "start": 40407,
                  "end": 40415
                }
              }
            }
          ],
          "named": [
            [
              "major",
              {
                "Bool": {
                  "val": true,
                  "span": {
                    "start": 40404,
                    "end": 40406
                  }
                }
              }
            ]
          ]
        }
      }
    }
  ]
}

CustomValueOp plugin call

Perform an operation on a custom value received from the plugin. The argument is a 2-tuple (array): (custom_value, op).

The custom value is specified in spanned format, as a PluginCustomValue without the type field, and not as a Value - see the examples.

ToBaseValue

Returns a plain value that is representative of the custom value, or an error if this is not possible. Sending a custom value back for this operation is not allowed. The response type is PipelineData or Error. If the operation produces a stream, it will be consumed to a value.

Example:

{
  "Call": [
    0,
    {
      "CustomValueOp": [
        {
          "item": {
            "name": "version",
            "data": [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0]
          },
          "span": {
            "start": 90,
            "end": 96
          }
        },
        "ToBaseValue"
      ]
    }
  ]
}
FollowPathInt

Returns the result of following a numeric cell path (e.g. $custom_value.0) on the custom value. This is most commonly used with custom types that act like lists or tables. The argument is a spanned unsigned integer. The response type is PipelineData or Error. The result may be another custom value. If the operation produces a stream, it will be consumed to a value.

Example:

$version.0
{
  "Call": [
    0,
    {
      "CustomValueOp": [
        {
          "item": {
            "name": "version",
            "data": [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0]
          },
          "span": {
            "start": 90,
            "end": 96
          }
        },
        {
          "FollowPathInt": {
            "item": 0,
            "span": {
              "start": 320,
              "end": 321
            }
          }
        }
      ]
    }
  ]
}
FollowPathString

Returns the result of following a string cell path (e.g. $custom_value.field) on the custom value. This is most commonly used with custom types that act like lists or tables. The argument is a spanned string. The response type is PipelineData or Error. The result may be another custom value. If the operation produces a stream, it will be consumed to a value.

Example:

$version.field
{
  "Call": [
    0,
    {
      "CustomValueOp": [
        {
          "item": {
            "name": "version",
            "data": [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0]
          },
          "span": {
            "start": 90,
            "end": 96
          }
        },
        {
          "FollowPathString": {
            "item": "field",
            "span": {
              "start": 320,
              "end": 326
            }
          }
        }
      ]
    }
  ]
}
PartialCmp

Compares the custom value to another value and returns the Ordering that should be used, if any. The argument type is a Value, which may be any value - not just the same custom value type. The response type is Ordering. Error may also be returned, but at present the error is unlikely to be presented to the user - the engine will act as if you had sent {"Ordering": null}.

Example (comparing two version custom values):

{
  "Call": [
    0,
    {
      "CustomValueOp": [
        {
          "item": {
            "name": "version",
            "data": [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0]
          },
          "span": {
            "start": 90,
            "end": 96
          }
        },
        {
          "PartialCmp": {
            "Custom": {
              "val": {
                "type": "PluginCustomValue",
                "name": "version",
                "data": [0, 0, 0, 0, 1, 0, 0, 0, 3, 0, 0, 0]
              },
              "span": {
                "start": 560,
                "end": 566
              }
            }
          }
        }
      ]
    }
  ]
}
Operation

Returns the result of evaluating an Operator on this custom value with another value. The argument is a 2-tuple: (operator, value), where operator is a spanned Operator and value is a Value, which may be any value - not just the same custom value type. The response type is PipelineData or Error. The result may be another custom value. If the operation produces a stream, it will be consumed to a value.

Example:

$version + 7
{
  "Call": [
    0,
    {
      "CustomValueOp": [
        {
          "item": {
            "name": "version",
            "data": [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0]
          },
          "span": {
            "start": 90,
            "end": 96
          }
        },
        {
          "Operation": [
            {
              "item": {
                "Math": "Plus"
              },
              "span": {
                "start": 180,
                "end": 181
              }
            },
            {
              "Int": {
                "val": 7,
                "span": {
                  "start": 183,
                  "end": 184
                }
              }
            }
          ]
        }
      ]
    }
  ]
}
Dropped

This op is used to notify the plugin that a PluginCustomValue that had notify_on_drop set to true was dropped in the engine - i.e., all copies of it have gone out of scope. For more information on exactly under what circumstances this is sent, see the drop notification section of the plugin reference. The response type is Empty pipeline data or Error.

Example:

{
  "Call": [
    0,
    {
      "CustomValueOp": [
        {
          "item": {
            "name": "handle",
            "data": [78, 60],
            "notify_on_drop": true
          },
          "span": {
            "start": 1820,
            "end": 1835
          }
        },
        "Dropped"
      ]
    }
  ]
}

EngineCallResponse

A response to an engine call made by the plugin. The argument is a 2-tuple (array): (engine_call_id, engine_call)

The engine_call_id refers to the same number that the engine call being responded to originally contained. The plugin must send unique IDs for each engine call it makes. Like CallResponse, there are multiple types of responses:

Error engine call response

A failure result. Contains a LabeledError.

Example:

{
  "EngineCallResponse": [
    0,
    {
      "Error": {
        "LabeledError": {
          "msg": "The connection closed.",
          "labels": [],
          "code": null,
          "url": null,
          "help": null,
          "inner": []
        }
      }
    }
  ]
}

PipelineData engine call response

A successful result with a Nu Value or stream. The body is a PipelineDataHeader.

Example:

{
  "EngineCallResponse": [
    0,
    {
      "ListStream": {
        "id": 23,
        "span": {
          "start": 8081,
          "end": 8087
        }
      }
    }
  ]
}

Config engine call response

A successful result of a Config engine call. The body is a Config.

Example:

{
  "EngineCallResponse": [
    0,
    {
      "Config": {
        "external_completer": null,
        "filesize_metric": true,
        "table_mode": "Rounded",
        "table_move_header": false,
        ...
      }
    }
  ]
}

This example is abbreviated, as the Config object is large and ever-changing.

ValueMap engine call response

A successful result for engine calls that produce plain maps, such as the GetEnvVars engine call. The body is a map from strings to Values.

Example:

{
  "EngineCallResponse": [
    0,
    {
      "ValueMap": {
        "FOO": {
          "String": {
            "val": "bar",
            "span": {
              "start": 2020,
              "end": 2024
            }
          }
        }
      }
    }
  ]
}

Goodbye

Indicate that no further plugin calls are expected, and that the plugin should exit as soon as it is finished processing any in-progress plugin calls.

This message is not a map, it is just a bare string, as it takes no arguments.

Example:

"Goodbye"

Output messages

These are messages sent from the plugin to the engine. Hello and Stream messages are also included.

CallResponse

Error plugin call response

An error occurred while attempting to fulfill the request. The body is a LabeledError.

It is strongly preferred to provide labeled messages whenever possible to let the user know where the problem might be in their script. If there is no more suitable span from a value that can be used, head from EvaluatedCall is a good fallback.

Example:

{
  "CallResponse": [
    0,
    {
      "Error": {
        "msg": "A really bad error occurred",
        "labels": [
          {
            "text": "I don't know, but it's over nine thousand!",
            "span": {
              "start": 9001,
              "end": 9007
            }
          }
        ],
        "code": "my_plugin::bad::really_bad",
        "url": "https://example.org/my_plugin/error/bad/really_bad.html",
        "help": "you can solve this by not doing the bad thing",
        "inner": [
          {
            "msg": "The bad thing"
          }
        ]
      }
    }
  ]
}

Signature plugin call response

A successful response to a Signature plugin call. The body is an array of signaturesopen in new window.

Example:

{
  "CallResponse": [
    0,
    {
      "Signature": [
        {
          "sig": {
            "name": "len",
            "usage": "calculates the length of its input",
            "extra_usage": "",
            "search_terms": [],
            "required_positional": [],
            "optional_positional": [],
            "rest_positional": null,
            "vectorizes_over_list": false,
            "named": [
              {
                "long": "help",
                "short": "h",
                "arg": null,
                "required": false,
                "desc": "Display the help message for this command",
                "var_id": null,
                "default_value": null
              }
            ],
            "input_type": "String",
            "output_type": "Int",
            "input_output_types": [],
            "allow_variants_without_examples": false,
            "is_filter": false,
            "creates_scope": false,
            "allows_unknown_args": false,
            "category": "Default"
          },
          "examples": []
        }
      ]
    }
  ]
}

Ordering plugin call response

A successful response to the PartialCmp custom value op. The body is either Ordering if the comparison is possible, or null if the values can't be compared.

Example:

{
  "CallResponse": [
    0,
    {
      "Ordering": "Less"
    }
  ]
}

Example with incomparable values:

{
  "CallResponse": [
    0,
    {
      "Ordering": null
    }
  ]
}

PipelineData plugin call response

A successful result with a Nu Value or stream. The body is a PipelineDataHeader.

Example:

{
  "CallResponse": [
    0,
    {
      "Value": {
        "Int": {
          "val": 42,
          "span": {
            "start": 12,
            "end": 14
          }
        }
      }
    }
  ]
}

EngineCall

Plugins can make engine calls during execution of a call. The body is a map with the following keys:

FieldTypeUsage
contextintegerThe ID of the call that this engine call relates to.
idintegerA unique ID for this engine call, in order to send the response.
callEngineCallOne of the options described below.

The context must be an ID of a call that was received that is currently in one of two states:

  1. The response has not been sent yet.
  2. The response contained stream data (i.e. ListStream or ByteStream), and at least one of the streams started by the response is still sending data (i.e. End has not been sent).

After a response has been fully sent, and streams have ended, the context from that call can no longer be used.

The engine call ID must be unique for the lifetime of the plugin, and it is suggested that this be a sequentially increasing number across all engine calls made by the plugin. It is not separated by context; the response only contains the id.

GetConfig engine call

Get the Nushell engine configuration. Returns a Config response if successful.

Example:

{
  "EngineCall": {
    "context": 0,
    "id": 0,
    "call": "GetConfig"
  }
}

GetPluginConfig engine call

Get the configuration for the plugin, from its section in $env.config.plugins.NAME if present. Returns a PipelineData response if successful, which will contain either a Value or be Empty if there is no configuration for the plugin set.

If the plugin configuration was specified as a closure, the engine will evaluate that closure and return the result, which may cause an error response.

Example:

{
  "EngineCall": {
    "context": 3,
    "id": 8,
    "call": "GetPluginConfig"
  }
}

GetEnvVar engine call

Get an environment variable from the caller's scope. Returns a PipelineData response if successful, which will contain either a Value or be Empty if the environment variable is not present.

Example:

{
  "EngineCall": {
    "context": 7,
    "id": 41,
    "call": {
      "GetEnvVar": "PATH"
    }
  }
}

GetEnvVars engine call

Get all environment variables from the caller's scope. Returns a ValueMap response if successful, with all of the environment variables in the scope.

Example:

{
  "EngineCall": {
    "context": 9,
    "id": 72,
    "call": "GetEnvVars"
  }
}

GetCurrentDir engine call

Get the current directory path in the caller's scope. This always returns an absolute path as a string Value pipeline data response if successful. The span contained within the value response is unlikely to be useful, and may be zero.

Example:

{
  "EngineCall": {
    "context": 7,
    "id": 40,
    "call": "GetCurrentDir"
  }
}

AddEnvVar engine call

Set an environment variable in the caller's scope. The environment variable can only be propagated to the caller's scope if called before the plugin call response is sent. Either way, it is propagated to other engine calls made within the same context. The argument is a 2-tuple: (name, value). The response type is Empty pipeline data when successful.

Example:

{
  "EngineCall": {
    "context": 7,
    "id": 42,
    "call": {
      "AddEnvVar": [
        "FOO",
        {
          "String": {
            "val": "bar",
            "span": {
              "start": 2020,
              "end": 2024
            }
          }
        }
      ]
    }
  }
}

GetHelp engine call

Get fully formatted help text for the current command. This can help with implementing top-level commands that just list their subcommands, rather than implementing any specific functionality. The response on success is Value pipeline data that always contains a string.

Example:

{
  "EngineCall": {
    "context": 1,
    "id": 2,
    "call": "GetHelp"
  }
}

EnterForeground engine call

Moves the plugin to the foreground group for direct terminal access, in an operating system-defined manner. This should be called when the plugin is going to drive the terminal in raw mode, for example to implement a terminal UI. It will likely be necessary for the plugin to also be running in local socket mode in that case.

This call responds with Empty pipeline data on success when no action is required by the plugin. On Unix-like operating systems, if the response is Value pipeline data, it contains an Int which is the process group ID the plugin must join using setpgid() in order to be in the foreground.

This call will fail with an error if the plugin is already in the foreground.

The plugin should call LeaveForeground when it no longer needs to be in the foreground. Note that the plugin will also automatically be removed from the foreground when the plugin call response is received, even if the plugin call returns a stream.

Example:

{
  "EngineCall": {
    "context": 0,
    "id": 0,
    "call": "EnterForeground"
  }
}

LeaveForeground engine call

Resets the state set by EnterForeground.

If the plugin had been requested to change process groups by the response of EnterForeground, it should also reset that state by calling setpgid(0), since plugins are normally in their own process group.

This call responds with Empty pipeline data on success.

Example:

{
  "EngineCall": {
    "context": 0,
    "id": 0,
    "call": "LeaveForeground"
  }
}

GetSpanContents engine call

Get the contents of a Span from the engine. This can be used for viewing the source code that generated a value. The argument is a Span. The response on success is Value pipeline data containing a Binary value, as the result is not guaranteed to be valid UTF-8.

Example:

{
  "EngineCall": {
    "id": 72,
    "call": {
      "GetSpanContents": {
        "start": 38881,
        "end": 39007
      }
    }
  }
}

EvalClosure engine call

Pass a Closure and arguments to the engine to be evaluated. Returns a PipelineData response if successful with the output of the closure, which may be a stream.

FieldTypeUsage
closurespanned ClosureThe closure to call, generally from a Value.
positionalValue arrayPositional arguments for the closure.
inputPipelineDataHeaderInput for the closure.
redirect_stdoutbooleanWhether to redirect stdout if the closure ends in an external command.
redirect_stderrbooleanWhether to redirect stderr if the closure ends in an external command.

The Closure is not wrapped as a Value - i.e., it doesn't have {"Closure": ...} around it.

Example:

{
  "EngineCall": {
    "context": 7,
    "id": 40,
    "call": {
      "EvalClosure": {
        "closure": {
          "item": {
            "block_id": 72,
            "captures": []
          },
          "span": {
            "start": 780,
            "end": 812
          }
        },
        "positional": [
          {
            "Int": {
              "val": 7,
              "span": {
                "start": 3080,
                "end": 3081
              }
            }
          }
        ],
        "input": "Empty",
        "redirect_stdout": true,
        "redirect_stderr": false
      }
    }
  }
}

Option

Sets options that affect how the engine treats the plugin. No response is expected for this message.

GcDisabled option

Set to true to stop the plugin from being automatically garbage collected, or false to enable it again.

Example:

{
  "Option": {
    "GcDisabled": true
  }
}

Stream messages

Streams can be sent by both the plugin and the engine. The agent that is sending the stream is known as the producer, and the agent that receives the stream is known as the consumer.

All stream messages reference a stream ID. This identifier is an integer starting at zero and is specified by the producer in the message that described what the stream would be used for: for example, Call or CallResponse. A producer should not reuse identifiers it has used once before. The most obvious implementation is sequential, where each new stream gets an incremented number. It is not necessary for stream IDs to be totally unique across both the plugin and the engine: stream 0 from the plugin and stream 0 from the engine are different streams.

Data

This message is sent from producer to consumer. The body is a 2-tuple (array) of (id, data).

The data is either a List map for a list stream, in which case the body is the Value to be sent, or Raw for a raw stream, in which case the body is either an Ok map with a byte buffer, or an Err map with a LabeledError.

Examples:

{
  "Data": [
    0,
    {
      "List": {
        "String": {
          "val": "Hello, world!",
          "span": {
            "start": 40000,
            "end": 40015
          }
        }
      }
    }
  ]
}
{
  "Data": [
    0,
    {
      "Raw": {
        "Ok": [72, 101, 108, 108, 111, 44, 32, 119, 111, 114, 108, 100, 33]
      }
    }
  ]
}
{
  "Data": [
    0,
    {
      "Raw": {
        "Err": {
          "IOError": {
            "msg": "disconnected"
          }
        }
      }
    }
  ]
}

End

This message is sent from producer to consumer. The body is a single value, the id.

Must be sent at the end of a stream by the producer. The producer must not send any more Data messages after the end of the stream.

The consumer must send Drop in reply unless the stream ended because the consumer chose to drop the stream.

Example:

{
  "End": 0
}

Ack

This message is sent from consumer to producer. The body is a single value, the id.

Sent by the consumer in reply to each Data message, indicating that the consumer has finished processing that message. Ack is used for flow control. If a consumer does not need to process a stream immediately, or is having trouble keeping up, it should not send Ack messages until it is ready to process more Data.

Example:

{
  "Ack": 0
}

Drop

This message is sent from consumer to producer. The body is a single value, the id.

Sent by the consumer to indicate disinterest in further messages from a stream. The producer may send additional Data messages after Drop has been received, but should make an effort to stop sending messages and End the stream as soon as possible.

The consumer should not consider Data messages sent after Drop to be an error, unless End has already been received.

The producer must send End in reply unless the stream ended because the producer ended the stream.

Example:

{
  "Drop": 0
}

Encoding

JSON

The JSON encoding defines messages as JSON objects. No separator or padding is required. Whitespace within the message object as well as between messages is permitted, including newlines.

The engine is more strict about the format it emits: every message ends with a newline, and unnecessary whitespace and newlines will not be emitted within a message. It is explicitly supported for a plugin to choose to parse the input from the engine by parsing each line received as a separate message, as this is most commonly supported across all languages.

Byte arrays are encoded as plain JSON arrays of numbers representing each byte. While this is inefficient, it is maximally portable.

MessagePack should be preferred where possible if performance is desired, especially if byte streams are expected to be a common input or output of the plugin.

MessagePack

MessagePackopen in new window is a machine-first binary encoding format with a data model very similar to JSON. Messages are encoded as maps. There is no separator between messages, and no padding character is accepted.

Most messages are encoded in the same way as their JSON analogue. For example, the following Hello message in JSON:

{
  "Hello": {
    "protocol": "nu-plugin",
    "version": "0.94.0",
    "features": []
  }
}

is encoded in the MessagePack format as:

81                   // map, one element
  a5 "Hello"         // 5-character string
  83                 // map, three elements
    a8 "protocol"    // 8-character string
    a9 "nu-plugin"   // 9-character string
    a7 "version"     // 7-character string
    a6 "0.94.0"      // 6-character string
    a8 "features"    // 8-character string
    90               // array, zero elements

(verbatim byte strings quoted for readability, non-printable bytes in hexadecimal)

Byte arrays are encoded with MessagePack's native byte arrays, which impose zero constraints on the formatting of the bytes within. In general, the MessagePack encoding is much more efficient than JSON and should be the first choice for plugins where performance is important and MessagePack is available.

Value types

Rust documentationopen in new window

The Value enum describes all structured data used in Nu.

Example:

{
  "Int": {
    "val": 5,
    "span": {
      "start": 90960,
      "end": 90963
    }
  }
}

Bool

A boolean.

FieldType
valboolean
spanSpan

Example:

true
{
  "Bool": {
    "val": true,
    "span": {
      "start": 4040,
      "end": 4044
    }
  }
}

Int

A 64-bit signed integer.

FieldType
valinteger
spanSpan

Example:

-2
{
  "Int": {
    "val": -2,
    "span": {
      "start": 4040,
      "end": 4042
    }
  }
}

Float

A 64-bit (double precision) floating point number.

FieldType
valdouble
spanSpan

Example:

36.4
{
  "Float": {
    "val": 36.4,
    "span": {
      "start": 8040,
      "end": 8044
    }
  }
}

Filesize

A quantity of bytes, internally a 64-bit signed integer representing the number of bytes. This is pretty-printed to the user with a more human scale, e.g. 32.4 MiB.

FieldType
valinteger
spanSpan

Example:

32.4MiB
{
  "Filesize": {
    "val": 33973248,
    "span": {
      "start": 7740,
      "end": 7747
    }
  }
}

Duration

A duration of time, internally a 64-bit signed integer representing the number of nanoseconds. This is pretty-printed to the user with a more human scale, e.g. 8sec 375ms 604µs 528ns.

FieldType
valinteger
spanSpan

Example:

8375604528ns
{
  "Duration": {
    "val": 8375604528,
    "span": {
      "start": 181462,
      "end": 181465
    }
  }
}

Date

A date/time value, including the time zone, represented in RFC 3339open in new window format. This is printed to the user according to their locale.

FieldType
valstring
spanSpan

Example:

1996-12-19T16:39:57-08:00
{
  "Date": {
    "val": "1996-12-19T16:39:57-08:00",
    "span": {
      "start": 181525,
      "end": 181528
    }
  }
}

Range

A range of values.

FieldType
valRange
spanSpan

Range has two variants, IntRange and FloatRange:

IntRange

FieldType
startinteger
stepinteger
endBound integer

Examples:

0..
{
  "Range": {
    "val": {
      "IntRange": {
        "start": 0,
        "step": 1,
        "end": "Unbounded"
      }
    },
    "span": {
      "start": 1380,
      "end": 1383
    }
  }
}
7..10
{
  "Range": {
    "val": {
      "IntRange": {
        "start": 7,
        "step": 1,
        "end": { "Included": 10 }
      }
    },
    "span": {
      "start": 1380,
      "end": 1385
    }
  }
}
7..<10
{
  "Range": {
    "val": {
      "IntRange": {
        "start": 7,
        "step": 1,
        "end": { "Excluded": 10 }
      }
    },
    "span": {
      "start": 1380,
      "end": 1386
    }
  }
}
0..64..128
{
  "Range": {
    "val": {
      "IntRange": {
        "start": 0,
        "step": 64,
        "end": { "Included": 128 }
      }
    },
    "span": {
      "start": 1380,
      "end": 1390
    }
  }
}

FloatRange

Identical to IntRange but for floats instead.

FieldType
startdouble
stepdouble
endBound double

Example:

7.5..10.5
{
  "Range": {
    "val": {
      "FloatRange": {
        "start": 7.5,
        "step": 1,
        "end": { "Included": 10.5 }
      }
    },
    "span": {
      "start": 1380,
      "end": 1389
    }
  }
}

String

A UTF-8 string.

FieldType
valstring
spanSpan

Example:

"Hello, nu!"
{
  "String": {
    "val": "Hello, nu!",
    "span": {
      "start": 8990,
      "end": 9002
    }
  }
}

Glob

A filesystem glob, selecting multiple files or directories depending on the expansion of wildcards.

If no_expand is true, the expansion of wildcards is disabled and this just acts as a literal path.

FieldType
valstring
no_expandboolean
spanSpan

Example:

"src/**/*.rs" | into glob
{
  "Glob": {
    "val": "src/**/*.rs",
    "no_expand": false,
    "span": {
      "start": 9400,
      "end": 9413
    }
  }
}

Record

An associative key-value map. If records are contained in a list, this renders as a table. The keys are always strings, but the values may be any type.

FieldType
valmap: string ⇒ Value
spanSpan

Example:

{foo: 5, bar: "hello nushell"}
{
  "Record": {
    "val": {
      "foo": {
        "Int": {
          "val": 42,
          "span": {
            "start": 659813,
            "end": 659814
          }
        }
      },
      "bar": {
        "String": {
          "val": "hello nushell",
          "span": {
            "start": 659821,
            "end": 659836
          }
        }
      }
    },
    "span": {
      "start": 659807,
      "end": 659837
    }
  }
}

List

A list of values of any type.

FieldType
valsValue array
spanSpan

Example:

[1, 2, foo, bar]
{
  "List": {
    "vals": [
      {
        "Int": {
          "val": 1,
          "span": {
            "start": 659951,
            "end": 659952
          }
        }
      },
      {
        "Int": {
          "val": 2,
          "span": {
            "start": 659954,
            "end": 659955
          }
        }
      },
      {
        "String": {
          "val": "foo",
          "span": {
            "start": 659957,
            "end": 659960
          }
        }
      },
      {
        "String": {
          "val": "bar",
          "span": {
            "start": 659962,
            "end": 659965
          }
        }
      }
    ],
    "span": {
      "start": 659950,
      "end": 659966
    }
  }
}

Block

A reference to a parsed block of Nushell code, without any captured variables.

FieldType
valunsigned integer (block id)
spanSpan

Example:

{
  "Block": {
    "val": 44500,
    "span": {
      "start": 59400,
      "end": 59480
    }
  }
}

Closure

A reference to a parsed block of Nushell code, with variables captured from scope.

FieldType
valClosure
spanSpan

Closure is defined as:

FieldType
block_idunsigned integer
capturesarray of pairs (unsigned integer var_id, Value)

The plugin should not try to inspect the contents of the closure. It is recommended that this is only used as an argument to the EvalClosure engine call. The exact representation of a closure is likely to change in the future to avoid serializing all of the captures.

Example:

let foo = "bar"
{ || $foo }
{
  "Closure": {
    "val": {
      "block_id": 1965,
      "captures": [
        [
          862,
          {
            "String": {
              "val": "bar",
              "span": {
                "start": 660030,
                "end": 660041
              }
            }
          }
        ]
      ]
    },
    "span": {
      "start": 660030,
      "end": 660041
    }
  }
}

Nothing

The absence of a value, represented by null within Nushell.

FieldType
spanSpan

Example:

null
{
  "Nothing": {
    "span": {
      "start": 64550,
      "end": 64554
    }
  }
}

Error

An error contained within a value. Trying to operate on the value will most likely cause the error to be forwarded. When writing plugins, error values should typically be handled by returning the error from the command when encountered.

FieldType
valLabeledError
spanSpan

Example:

error make {
  msg: "foo"
  label: {
    text: "bar"
    span: {
      start: 0
      end: 0
    }
  }
}
{
  "Error": {
    "val": {
      "msg": "foo",
      "labels": [
        {
          "text": "bar",
          "span": {
            "start": 0,
            "end": 0
          }
        }
      ],
      "code": null,
      "url": null,
      "help": null,
      "inner": []
    }
  }
}

Binary

An array of raw bytes. This is sometimes returned from operations that detect data that isn't valid as UTF-8, but can also be created with into binary or binary literals.

FieldType
valbyte array
spanSpan

Note that the encoding of byte arrays in JSON and MessagePack is different - the former uses an array of numbers, but the latter uses the native byte array support.

Example:

0x[aa bb cc dd]
{
  "Binary": {
    "val": [170, 187, 204, 221],
    "span": {
      "start": 659637,
      "end": 659652
    }
  }
}

CellPath

Represents a path into subfields of lists, records, and tables.

FieldType
valCellPath
spanSpan

CellPath is defined as:

FieldType
membersPathMember

PathMember has two variants, String or Int, and both contain the following fields:

FieldType
valstring / unsigned integer
spanSpan
optionalboolean

Optional path members will not cause errors if they can't be accessed - the path access will just return Nothing instead.

Example:

foo.0?.bar
# [foo {value: 0, optional: true} bar] | into cell-path
{
  "CellPath": {
    "val": {
      "members": [
        {
          "String": {
            "val": "foo",
            "span": {
              "start": 659835,
              "end": 659838
            },
            "optional": false
          }
        },
        {
          "Int": {
            "val": 0,
            "span": {
              "start": 659847,
              "end": 659848
            },
            "optional": true
          }
        },
        {
          "String": {
            "val": "bar",
            "span": {
              "start": 659866,
              "end": 659869
            },
            "optional": false
          }
        }
      ]
    },
    "span": {
      "start": 659873,
      "end": 659887
    }
  }
}

Custom

Represents data types that extend the base nushell types with custom functionality. Plugins can use custom values to implement native-like data types that can be indexed by cell paths, operated on by operators, and compared in plugin-defined ways.

Custom values for plugins may only contain the following content map:

FieldTypeUsage
typestringMust be "PluginCustomValue".
namestringThe human-readable name of the custom value emitted by the plugin.
databyte arrayPlugin-defined representation of the custom value.
notify_on_dropbooleanEnable drop notification. Default false if not present.

Plugins will only be sent custom values that they have previously emitted. Custom values from other plugins or custom values used within the Nu engine itself are not permitted to be sent to or from the plugin.

notify_on_drop is an optional field that should be omitted if false, to save bytes. If this is not convenient for your implementation, "notify_on_drop": false is still valid, but it's preferred to not include it.

Example:

{
  "Custom": {
    "val": {
      "type": "PluginCustomValue",
      "name": "database",
      "data": [36, 190, 127, 40, 12, 3, 46, 83],
      "notify_on_drop": true
    },
    "span": {
      "start": 320,
      "end": 340
    }
  }
}

Embedded Nu types

Several types used within the protocol come from elsewhere in Nu's source code, especially the nu-protocolopen in new window crate.

Rust enums are usually encoded in serdeopen in new window's default format:

"Variant"             // Variant
{ "Variant": value }  // Variant(value)
{ "Variant": [a, b] } // Variant(a, b)
{
  "Variant": {
    "one": 1,
    "two": 2
  }
}                     // Variant { one: 1, two: 2 }

Structs are encoded as maps of their fields, without the name of the struct.

Span

Documentationopen in new window

Describes a region of code in the engine's memory, used mostly for providing diagnostic error messages to the user with context about where a value that caused an error came from.

FieldTypeUsage
startintegerThe index of the first character referenced.
endintegerThe index after the last character referenced.

PipelineDataHeader

Describes either a single value, or the beginning of a stream.

VariantUsage
EmptyNo values produced; an empty stream.
ValueA single value
ListStreamSpecify a list stream that will be sent.
ByteStreamSpecify a byte stream that will be sent.

Empty header variant

An empty stream. Nothing will be sent. There is no identifier, and this is equivalent to a Nothing value.

The representation is the following string:

"Empty"

Value header variant

A single value. Does not start a stream, so there is no identifier. Contains a Value.

Example:

{
  "Value": {
    "Int": {
      "val": 2,
      "span": {
        "start": 9090,
        "end": 9093
      }
    }
  }
}

ListStream header variant

Starts a list stream. Expect Data messages of the List variant with the referenced ID.

Contains ListStreamInfo, a map:

FieldTypeUsage
idintegerThe stream identifier
spanSpanThe source code reference that caused the stream.

Example:

{
  "ListStream": {
    "id": 2,
    "span": {
      "start": 33911,
      "end": 33942
    }
  }
}

ByteStream header variant

Starts a byte stream. Expect Data messages of the Raw variant with the referenced ID.

FieldTypeUsage
idintegerThe stream identifier
spanSpanThe source code reference that caused the stream.
typeByteStreamTypeThe expected type of the stream.

Byte streams carry a type field with one of the three following strings:

typeMeaning
"Binary"The stream contains binary data of unknown encoding, and should be treated as a binary value.
"String"The stream contains text data that is valid UTF-8, and should be treated as a string value.
"Unknown"The type of the byte stream is unknown and should be inferred depending on whether its contents can be decoded as valid UTF-8 or not.

The Unknown type is used by Nu to represent the output of external commands if they are not passed through into string or into binary to explicitly set their type. A command that declares an output type of exclusively either string or binary must explicitly type its output byte streams appropriately, to ensure they coerce to the correct type, rather than using Unknown.

Example:

{
  "ByteStream": {
    "id": 7,
    "span": {
      "start": 49011,
      "end": 49027
    },
    "type": "String"
  }
}

LabeledError

Documentationopen in new window

A flexible, generic error type, with any number of labeled spans.

FieldTypeUsage
msgstringThe main error message to show at the top of the error.
labelsErrorLabel array?Spans and messages to label the error in the source code.
codestring?A unique machine- and search-friendly code that can be matched against, e.g. nu::shell::missing_config_value
urlstring?A URL that links to additional information about the error.
helpstring?Additional help for the error, usually a hint about what the user might try.
innerLabeledError array?Additional errors referenced by the error, possibly the cause(s) of this error.

Most of the fields are not required - only msg must be present. ErrorLabel (in the labels array) is as follows:

FieldTypeUsage
textstringThe message for the label.
spanSpanThe span in the source code that the label should point to.

TIP

When reading the Rust source code for the nu-plugin crates, many places where LabeledError is specified here are actually represented as ShellError in that implementation. However, ShellError always serializes as if it were LabeledError, so the difference between the two can be ignored within the protocol.

Example:

{
  "msg": "A really bad error occurred",
  "labels": [
    {
      "text": "I don't know, but it's over nine thousand!",
      "span": {
        "start": 9001,
        "end": 9007
      }
    }
  ],
  "code": "my_plugin::bad::really_bad",
  "url": "https://example.org/my_plugin/error/bad/really_bad.html",
  "help": "you can solve this by not doing the bad thing",
  "inner": [
    {
      "msg": "The bad thing"
    }
  ]
}

Config

Documentationopen in new window

This struct describes the configuration of Nushell. It is quite large and frequently changing, so please refer to the Rust documentation if there is anything you need from it.

Ordering

Documentationopen in new window

We serialize the Rust Ordering type as literal strings, for example:

'Less'; // left hand side is less than right hand side
'Equal'; // both values are equal
'Greater'; // left hand side is greater than right hand side

Operator

Documentationopen in new window

Serialized with serde's default enum representation. Examples:

{ "Math": "Append" }           // ++   Math(Append)
{ "Bits": "BitOr" }            // |    Bits(BitOr)
{ "Comparison": "RegexMatch" } // =~   Comparison(RegexMatch)