Nushell 0.93.0

Nushell, or Nu for short, is a new shell that takes a modern, structured approach to your command line. It works seamlessly with the data from your filesystem, operating system, and a growing number of file formats to make it easy to build powerful command line pipelines.

Today, we're releasing version 0.93.0 of Nu. This release reworks plugin management, allows plugins to interact with the terminal, adds a new Polars plugin, and makes some changes around setting environment variables. Please note that these changes come paired with several deprecations that are explained further below.

Where to get it

Nu 0.93.0 is available as pre-built binaries or from crates.io. If you have Rust installed you can install it using cargo install nu.

Note

The optional dataframe functionality is available by cargo install nu --features=dataframe.

Note for Windows users

During the release we had to change our Windows installer to install to the user profile rather than the system program files directory. When upgrading from 0.92.2 or earlier, we advise that you uninstall the package and then reinstall it. We do not recommend using winget upgrade for this release.

If you end up with two versions installed, you must remove it from Apps » Installed Apps in the Windows settings. WinGet will not be able to remove the duplicate version.

As part of this release, we also publish a set of optional plugins you can install and use with Nu. To install, use cargo install nu_plugin_<plugin name>.

Table of content

Themes of this release / New features [toc]

Redesigned plugin management commands [toc]

Deprecation warning

register is now deprecated and will be removed in the next major release. Read this section for alternatives.

We've changed the way plugins are managed in Nushell in this release. In 0.92 and earlier, plugins were added with register, e.g.:

> register ~/.cargo/bin/nu_plugin_gstat

This ran the plugin to get the signatures of the commands it supports, and then did two things: it added them to your $nu.plugin-file (previously called plugin.nu), so that they would be ready next time nu is run without having to ask the plugin again, and it also added them to your scope so you could use them immediately. While this was convenient, it had the drawback of always running the plugin at parse time (see issue #11923), which was less than ideal.

We now have two commands to replace register that break this functionality into two steps:

  1. plugin add, a normal command, which runs the plugin to get command signatures and adds them to your $nu.plugin-file, replacing any existing plugin definition with the same name
  2. plugin use, a parser command, which loads the plugin command signatures from your $nu.plugin-file and makes them available in scope.

All plugins are still loaded automatically from the plugin registry file at startup, so it is not necessary to add plugin use to your config file.

The previous example would now be equivalent to this, at the REPL:

# Add the plugin's commands to your plugin registry file:
> plugin add ~/.cargo/bin/nu_plugin_gstat

# Load it into scope:
> plugin use gstat

Note that this will not work in a script in this order, as plugin use is a parser command and would get evaluated first. If you would like to make sure a certain plugin is available in a script, there are a few options:

  1. Prepare a plugin registry file in advance, and use --plugin-config with plugin use:

    plugin use --plugin-config /path/to/custom.msgpackz your_required_plugin

  2. The same thing, but pass the plugin config to nu:

    nu --plugin-config /path/to/custom.msgpackz your-script.nu

  3. Pass a NUON list to --plugins (new in 0.93) to nu, to use plugins without a registry file:

    nu --plugins '[/path/to/nu_plugin_your_required_plugin]' your-script.nu

# or you can use `to nuon` from inside nu:
let plugins = [
    '/path/to/nu_plugin_foo'
    '/path/to/nu_plugin_bar'
]
nu --plugins ($plugins | to nuon) your-script.nu

The last option will run the plugin to get signatures at startup, without modifying the user's registry file, and can even be used when --no-config-file is specified.

Finally, we have also added plugin rm, which removes a plugin from the registry file. This was something that previously needed to be done manually in a text editor, but is made possible by the new format.

New plugin registry format [toc]

Breaking change

See a full overview of the breaking changes

The plugin registry format has been changed from the previous Nushell script format (containing a sequence of register commands) to a custom, declarative format based on MessagePack compressed with brotli. This new format is called msgpackz, and the default plugin file is plugin.msgpackz instead of plugin.nu.

Your existing plugin.nu file will be automatically upgraded to plugin.msgpackz the first time version 0.93 runs.

The declarative format brings a number of benefits:

  1. It can be edited more structurally, allowing us to replace the entire definition of one plugin at once, which is more logically consistent if a plugin removes commands from one version to the next, and also allowing us to support new commands like plugin rm.
  2. It supports adding metadata about plugins as a whole rather than just the command signatures, if we need to.
  3. It contains the Nushell version that generated it, allowing us to potentially fix issues between versions.
  4. It fails gracefully - if something about a particular plugin can't be parsed because of a change to the serialization format, other plugins that don't have that issue can still load.

We chose to use compressed MessagePack because it is very fast to load, and the file is loaded every time nu starts, so with a lot of plugins or plugins that expose a lot of commands (e.g. the new polars plugin), this was a big problem for startup times in general.

The most significantly breaking part of this breaking change is that the following is no longer possible:

source $nu.plugin-file

You can use the plugin use command instead to (re-)load a specific plugin from the plugin registry file:

> plugin use gstat

Improved terminal interaction for plugins [toc]

Plugins are now able to run over a UNIX domain socket or Windows named pipe instead of stdin/stdout if they support it (#12448). All Rust plugins support this mode by default, but it is optional; plugins written in other languages are not forced to implement this, and the stdio mode is still always attempted. The feature is advertised as LocalSocket and the upgrade is automatic when supported by the plugin and confirmed to be functional.

This frees up stdio for interaction with the user's terminal, which makes plugins that rely on that, including terminal UI plugins like @amtoine's nu_plugin_explore simpler and more reliable to implement.

We also added engine calls for moving the plugin into the terminal controlling (foreground) process group, mainly for Unix platforms. This was also necessary to fully support terminal UI plugins, after the last release's new persistence functionality caused plugins to run in a new, background process group by default.

Introduction of the Polars Plugin [toc]

With the enhancements to the plugins framework, we have migrated the dataframes crate to the nu_plugin_polars plugin. In this release, the functionality of the Polars plugin closely mirrors that of the dataframes plugin, ensuring seamless migration for existing scripts. After installing the plugin, simply replace dfr with polars to adapt your scripts.

Changes from the Dataframes Crate

  • The command name has been updated from dfr to polars, setting the stage for future support of additional dataframe libraries.
  • The dtypes command has been removed, as its functionality is largely duplicated by the newer polars schema command.
  • The equivalent of dfr ls is now polars store-ls, offering insight into the currently cached dataframes However, there isn't a direct way to display the variables to which they have been assigned.
  • Introducing polars store-get and polars store-rm to respectively retrieve and remove values from the plugin's internal object cache.

Installation

Installing the Polars plugin follows the same process as other plugins:

# Install the polars plugin
> cargo install nu_plugin_polars

# Add the plugin's commands to your plugin registry file:
> plugin add ~/.cargo/bin/nu_plugin_polars

# Load it into scope:
> plugin use polars

Dataframes deprecation [toc]

Deprecation warning

With the introduction of Polars plugin, the dataframe functionality is now deprecated and slated for removal in version 0.94.0.

As such, the dataframe cargo feature will also be removed in 0.94.0. This means that the "full" versions of the pre-compiled nushell binaries will no longer be provided on Github, since these were the same as the standard binaries but with the dataframe feature enabled.

Lazy record deprecation [toc]

Deprecation warning

This version deprecates the lazy make command and lazy records in general. In the next version, 0.94.0, lazy records will be removed entirely.

This motivation behind this is to simplify the language, as lazy records already had flaky support across the various nushell commands. Issue #12622 explains the reasoning in more detail. In the future, the sys and debug info commands will return regular records instead of lazy records. The plan is to introduce new APIs for these commands as a replacement for the existing lazy evaluation.

Stricter rules around setting environment variables [toc]

Breaking change

See a full overview of the breaking changes

The setting of environment variables via the following ways:

  • $env.VAR_NAME = ... assignment
  • scopes with the with-env command
  • the short hand syntax FOO=bar command-to-run
  • the load-env command

will now consistently prohibit you from changing a set of special environment variables controlled by Nushell. Currently this is the following set (already valid for $env. assignment):

  • $env.PWD
  • $env.FILE_PWD
  • $env.CURRENT_FILE This set may be expanded by future breaking changes to ensure consistent semantics of those special variables

Furthermore the FOO=bar BAZ=bla command shorthand syntax to set an environment variable for the scope of one command is now stricter. It will disallow you from repeating the same environment variable name twice and instead return an error (#12523).

# Now raises an error
FOO=bar FOO=bla command

Additionally, we standardize on using records to pass a map of environment variables to the with-env command, and thus deprecate the other previously allowed forms with this release.

Improved parsing of input/output types [toc]

Thanks to @texastoland in #12339, the input/output types for custom commands can now extend across multiple lines.

def "into bool" []: [
  int -> bool
  string -> bool
] {
  match $in {
    0 => false
    1 => true
    "false" => false
    "true" => true
  }
}

New CLI flag: --no-newline [toc]

This release adds a new CLI flag to the nu binary: --no-newline. Providing this flag will disable the trailing new line from being printed by nushell (#12410).

Hall of fame [toc]

Bug fixes [toc]

Thanks to all the contributors below for helping us solve issues and bugs 🙏

authordescriptionurl
@fdncredrestore query web --as-table to working order#12693
@IanManskeMake exit code available in catch block#12648
@merelymyselfmake grid throw an error when not enough columns#12672
@IanManskeFix an into bits example#12668
@IanManskeeach signature fix#12666
@IanManskeMake the same file error more likely to appear#12601
@WindSoilderset the type of default NU_LIB_DIRS and NU_PLUGIN_DIRS to list<string>#12573
@devynReplace subtraction of Instants and Durations with saturating subtractions#12549
@amtoinefix std log#12470
@singh-priyankFix negative value file size for "into filesize" (issue #12396)#12443
@IanManskeexplain refactor#12432
@ysthakurDon't check if stderr empty in test_xdg_config_symlink#12435
@IanManskePrevent panic on date overflow#12427
@IanManskeFix merging child stack into parent#12426
@ysthakurFix #12416 by canonicalizing XDG_CONFIG_HOME before comparing to config_dir()#12420
@IanManskeRange refactor#12405
@merelymyselfprevent select (negative number) from hanging shell#12393
@merelymyselfMake auto-cd check for permissions#12342
@devynAdd BufWriter to ChildStdin on the plugin interface#12419
@devynFix deadlock on PluginCustomValue drop#12418
@eopbFix stop suggesting --trash when already enabled (issue #12361)#12362
@devynFix #12391: mkdir uses process startup directory instead of current script directory#12394
@merelymyselfprevent parser from parsing variables as units#12378
@IanManskeFix hooks on 0.92.0#12383
@sholderbachMake default config conservative about clipboard#12385
@WindSoilderAvoid panic when pipe a variable to a custom command which have recursive call#12491
@TheLostLambdafix(shell_integration): set window title on startup#12569
@texastolandEnsure currently_parsed_cwd is set for config files#12338
@YizhePKUFix circular source causing Nushell to crash#12262
@friaesFix circular source causing Nushell to crash#12411

Enhancing the documentation [toc]

Thanks to all the contributors below for helping us making the documentation of Nushell commands better 🙏

authordescriptionurl
@deepanchalfix simple typo in commandline_.rs#12387
@SylvanBrocardUpdate list of supported formats in dfr open error message.#12408
@IanManskeMention print in the echo help text#12436
@oornaqueFix typo in help stor import#12442
@KAAtheWiseGitFix example wording in seq date#12665
@IanManskeFix an into bits example#12668
@fdncredimprove nu --lsp command tooltips#12589
@NotTheDr01dsFixes #12482 by pointing help links for ndjson to a non-spam source (take 2)#12509
@woosaaahhChange cal --week-start examples + error message#12597
@maxim-uvarovadd search_term "str extract" to parse command#12600
@IanManskeImprove the "input and output are the same file" error text#12619

Our set of commands is evolving [toc]

New commands [toc]

plugin add [toc]

Adds a plugin to the plugin registry file (default $nu.plugin-path).

plugin add ~/.cargo/bin/nu_plugin_gstat

A different registry file can be used if desired:

plugin add --plugin-config ~/polars.msgpackz ~/.cargo/bin/nu_plugin_polars

Replaces part of register, but can be used as a normal command, so it can be used within a script on a generated path. For example the following is now possible:

glob ~/.cargo/bin/nu_plugin_* | each { |file| plugin add $file }

plugin rm [toc]

Removes a plugin from the plugin registry file (default $nu.plugin-path).

plugin rm gstat
plugin rm --plugin-config ~/polars.msgpackz polars

The commands will still remain in scope, but will not be present the next time nu is started.

plugin use [toc]

Loads a plugin from the plugin registry file (default $nu.plugin-path) at parse time.

This replaces the other part of register - actually adding the commands to scope. It does not run the plugin executable, though, it just loads the previously added commands from the registry file.

It is not necessary to add this to your config file to use plugins that are in the registry file that nu was started with. Those are all automatically loaded at startup, just as they were before.

from msgpack [toc]

Reads MessagePack format data into Nu values.

Example:

> 0x[a7 4e 75 73 68 65 6c 6c] | from msgpack
Nushell

# Open a MessagePack formatted file:
> open nushell.msgpack
╭─────────┬───────╮
│ nushell │ rocks │
╰─────────┴───────╯

You can read a stream of multiple MessagePack objects in the same format as the plugin protocol by using from msgpack --objects:

> open --raw ~/nu_plugin_example.in.raw | from msgpack --objects
╭───┬─────────────────────────────────────────────────╮
│ 0 │ ╭───────┬────────────────────────────────────╮  │
│   │ │       │ ╭──────────┬─────────────────────╮ │  │
│   │ │ Hello │ │ protocol │ nu-plugin           │ │  │
│   │ │       │ │ version  │ 0.93.0              │ │  │
│   │ │       │ │          │ ╭───┬─────────────╮ │ │  │
│   │ │       │ │ features │ │ # │    name     │ │ │  │
│   │ │       │ │          │ ├───┼─────────────┤ │ │  │
│   │ │       │ │          │ │ 0 │ LocalSocket │ │ │  │
│   │ │       │ │          │ ╰───┴─────────────╯ │ │  │
│   │ │       │ ╰──────────┴─────────────────────╯ │  │
│   │ ╰───────┴────────────────────────────────────╯  │
│   │                                                 │
│ 1 │ ...                                             │
│ 2 │ ...                                             │
│ 3 │ ...                                             │
│ 4 │ ...                                             │
│   │                                                 │
│ 5 │ Goodbye                                         │
╰───┴─────────────────────────────────────────────────╯

Example captured with nu_plugin_tracer.

to msgpack [toc]

Converts Nu values into MessagePack.

Example:

> 'Nushell' | to msgpack
Length: 8 (0x8) bytes | printable whitespace ascii_other non_ascii
00000000:   a7 4e 75 73  68 65 6c 6c                             ×Nushell

# Save in MessagePack format to nushell.msgpack:
> {nushell: rocks} | save nushell.msgpack

from msgpackz [toc]

Just like from msgpack, but decompresses with brotli first.

The msgpackz format is used by the new plugin file format:

> open ~/.config/nushell/plugin.msgpackz
╭─────────────────┬─────────────────────────────────────────────────────────────────────────────────╮
│ nushell_version │ 0.92.3                                                                          │
│                 │ ╭───┬───────────────┬────────────────────────────────────┬───────┬────────────╮ │
│ plugins         │ │ # │     name      │              filename              │ shell │  commands  │ │
│                 │ ├───┼───────────────┼────────────────────────────────────┼───────┼────────────┤ │
│                 │ │ 0 │ custom_values │ .cargo/bin/nu_plugin_custom_values │       │ [table 8   │ │
│                 │ │   │               │                                    │       │ rows]      │ │
│                 │ │ 1 │ dbus          │ .cargo/bin/nu_plugin_dbus          │       │ [table 7   │ │
│                 │ │   │               │                                    │       │ rows]      │ │
│                 │ │ 2 │ emoji         │ .cargo/bin/nu_plugin_emoji         │       │ [table 1   │ │
│                 │ │   │               │                                    │       │ row]       │ │
│                 │ │ 3 │ example       │ .cargo/bin/nu_plugin_example       │       │ [table 13  │ │
│                 │ │   │               │                                    │       │ rows]      │ │
│                 │ │ 4 │ explore       │ .cargo/bin/nu_plugin_explore       │       │ [table 1   │ │
│                 │ │   │               │                                    │       │ row]       │ │
│                 │ │ 5 │ formats       │ .cargo/bin/nu_plugin_formats       │       │ [table 4   │ │
│                 │ │   │               │                                    │       │ rows]      │ │
│                 │ │ 6 │ gstat         │ .cargo/bin/nu_plugin_gstat         │       │ [table 1   │ │
│                 │ │   │               │                                    │       │ row]       │ │
│                 │ │ 7 │ inc           │ .cargo/bin/nu_plugin_inc           │       │ [table 1   │ │
│                 │ │   │               │                                    │       │ row]       │ │
│                 │ ╰───┴───────────────┴────────────────────────────────────┴───────┴────────────╯ │
╰─────────────────┴─────────────────────────────────────────────────────────────────────────────────╯

to msgpackz [toc]

Just like to msgpack, but compresses with brotli after encoding.

For example, to use it to manually edit the plugin file:

open ~/.config/nushell/plugin.msgpackz |
    update plugins {
        # Remove plugins with missing files:
        where { get filename | path exists }
    } |
    save -f ~/.config/nushell/plugin.msgpackz

As with other format commands, open and save will automatically use the msgpackz conversion if the file extension matches.

metadata set [toc]

The new metadata set command added in #12564 allows one to modify the metadata of a pipeline/value.

For example, you can set the data source to be ls, which will activate coloring using LS_COLORS:

[[name]; [Cargo.lock] [Cargo.toml] [README.md]] | metadata set --datasource-ls

Changes to existing commands [toc]

with-env [toc]

Breaking change

See a full overview of the breaking changes

Passing environment variables to with-env via the list-like or single-row table form is now deprecated after #12523:

# deprecated
with-env [X Y W Z] { $env.X }
# also deprecated
with-env [[X Y]; [W Z]] { $env.X }

Instead, you should use the record form:

with-env { X: 'Y', W: 'Z' } { $env.X }

This also explicitly stops you from repeating the name of a environment variable, which was a potential source of bugs.

# silently: bar
with-env [X foo X bar] { $env.X }
# error
with-env { X: 'foo', X: 'bar' } { $env.X }

Additionally, setting the PWD variable through with-env is now disallowed.

load-env [toc]

Breaking change

See a full overview of the breaking changes

With #12522, setting the PWD variable through load-env is now disallowed, both when the record is given as a pipeline input or command argument.

last [toc]

Breaking change

See a full overview of the breaking changes

To be consistent with first and other commands, last now errors if the input is empty (#12478).

To suppress the error and return null if an input is empty, wrap last in a try block:

[] | try { last }

drop [toc]

Breaking change

See a full overview of the breaking changes

With #12479, drop will now error if the number of rows/columns is negative.

skip [toc]

Breaking change

See a full overview of the breaking changes

To be consistent with take and other commands, skip no longer accepts external streams as input (#12559).

group-by [toc]

Breaking change

See a full overview of the breaking changes

If a closure was used as the grouper for group-by, then errors inside the closure would previously be ignored. Instead, group-by would put the row/item under the error group. With #12508, errors are no longer ignored and will immediately be bubbled up.

timeit [toc]

Breaking change

See a full overview of the breaking changes

After #12465, the timeit command will no longer capture external command output, instead redirecting output to the terminal/stdio.

kill [toc]

Breaking change

See a full overview of the breaking changes

The kill command used to return a stream of values in certain cases. In #12480 this was changed so that kill now always returns a single value (null or a string).

each [toc]

The signature of each previously showed that the closure provided to each had a second Int parameter. This parameter was removed some time ago, and the signature has been updated in #12666.

ls [toc]

Instead of a single optional path, the ls command now takes a rest argument of paths (#12327). I.e., you can spread a list into ls:

ls ...[directory1 directory2]

or simply ls multiple directories:

ls directory1 directory2

du [toc]

In the ls PR (#12327), the same changes were also made to du. So now, du takes a rest argument of paths just like ls.

try [toc]

With #12648, the exit code of the failed external command can be retrieved in a catch block via the usual $env.LAST_EXIT_CODE.

version [toc]

Thanks to @poliorcetics, the version command now has additional columns for the major, minor, patch, and pre portions of the current nushell version (#12593).

into filesize [toc]

Thanks to @singh-priyank in #12443, into filesize can now parse negative filesizes from strings.

view source [toc]

Thanks to @merelymyself, the view source command now displays more information like type signatures and default argument values (#12359).

grid [toc]

Instead of returning an error string, grid now triggers an error if it is unable to fit the grid in the available or specified number of columns (#12654).

explain [toc]

With #12432, the type column returned from explain should no longer contain string for every row.

Deprecated commands [toc]

register [toc]

This command is deprecated in favor of the new plugin add and plugin use commands. See the relevant section for the reasoning behind this change.

lazy make [toc]

The lazy make command has been deprecated in #12656 following the deprecation of lazy records (see lazy record deprecation).

describe --collect-lazyrecords [toc]

Since lazy records are now deprecated (see lazy record deprecation), the --collect-lazyrecords flag for describe was also deprecated in #12667.

Removed commands and flags [toc]

run-external flags [toc]

The --redirect-stdout, --redirect-stderr, --redirect-combine, and --trim-end-newline flags have been removed in #12659 following their deprecation in 0.92.0. See the previous release notes for migration info.

commandline flags [toc]

The --cursor, --cursor-end, --append, --insert, and --replace flags on commandline were removed with #12658. These flags were deprecated back in 0.91.0 following the introduction of a new command API for commandline. See the previous release notes for more information.

For plugin developers [toc]

New protocol features [toc]

New engine calls [toc]

See the Rust docs for EngineInterface for specific documentation for Rust plugins.

Breaking changes [toc]

  • #12659 Remove deprecated flags on run-external
  • #12658 Remove deprecated flags on commandline
  • #12610 Shrink the size of Expr
  • #12601 Make the same file error more likely to appear
  • #12579 Overhaul the plugin cache file with a new msgpack+brotli format
  • #12582 Remove the Value::Block case
  • #12559 Don't allow skip on external stream
  • #12577 Removed the polars dtypes command
  • #12566 Update crate feature flags
  • #12508 Make group-by return errors in closure
  • #12523 Improve with-env robustness
  • #12522 Disallow setting the PWD via load-env input
  • #12478 Refactor first and last
  • #12480 Return value instead of stream from kill
  • #12479 drop refactor
  • #12465 Force timeit to not capture stdout
  • #12365 Implement De-/Serialize for Record manually

Full changelog [toc]