Nushell 0.93.0

Nushell, or Nu for short, is a new shell that takes a modern, structured approach to your command line. It works seamlessly with the data from your filesystem, operating system, and a growing number of file formats to make it easy to build powerful command line pipelines.

Today, we're releasing version 0.93.0 of Nu. This release reworks plugin management, allows plugins to interact with the terminal, adds a new Polars plugin, and makes some changes around setting environment variables. Please note that these changes come paired with several deprecations that are explained further below.

Where to get it

Nu 0.93.0 is available as pre-built binariesopen in new window or from crates.ioopen in new window. If you have Rust installed you can install it using cargo install nu.

Note

The optional dataframe functionality is available by cargo install nu --features=dataframe.

Note for Windows users

During the release we had to change our Windows installer to install to the user profile rather than the system program files directory. When upgrading from 0.92.2 or earlier, we advise that you uninstall the package and then reinstall it. We do not recommend using winget upgrade for this release.

If you end up with two versions installed, you must remove it from Apps » Installed Apps in the Windows settings. WinGet will not be able to remove the duplicate version.

As part of this release, we also publish a set of optional plugins you can install and use with Nu. To install, use cargo install nu_plugin_<plugin name>.

Table of content

Themes of this release / New features [toc]

Redesigned plugin management commands [toc]

Deprecation warning

register is now deprecated and will be removed in the next major release. Read this section for alternatives.

We've changed the way plugins are managed in Nushell in this release. In 0.92 and earlier, plugins were added with register, e.g.:

> register ~/.cargo/bin/nu_plugin_gstat

This ran the plugin to get the signatures of the commands it supports, and then did two things: it added them to your $nu.plugin-file (previously called plugin.nu), so that they would be ready next time nu is run without having to ask the plugin again, and it also added them to your scope so you could use them immediately. While this was convenient, it had the drawback of always running the plugin at parse time (see issue #11923open in new window), which was less than ideal.

We now have two commands to replace register that break this functionality into two steps:

  1. plugin add, a normal command, which runs the plugin to get command signatures and adds them to your $nu.plugin-file, replacing any existing plugin definition with the same name
  2. plugin use, a parser command, which loads the plugin command signatures from your $nu.plugin-file and makes them available in scope.

All plugins are still loaded automatically from the plugin registry file at startup, so it is not necessary to add plugin use to your config file.

The previous example would now be equivalent to this, at the REPL:

# Add the plugin's commands to your plugin registry file:
> plugin add ~/.cargo/bin/nu_plugin_gstat

# Load it into scope:
> plugin use gstat

Note that this will not work in a script in this order, as plugin use is a parser command and would get evaluated first. If you would like to make sure a certain plugin is available in a script, there are a few options:

  1. Prepare a plugin registry file in advance, and use --plugin-config with plugin use:

    plugin use --plugin-config /path/to/custom.msgpackz your_required_plugin
    
  2. The same thing, but pass the plugin config to nu:

    nu --plugin-config /path/to/custom.msgpackz your-script.nu
    
  3. Pass a NUON list to --plugins (new in 0.93) to nu, to use plugins without a registry file:

    nu --plugins '[/path/to/nu_plugin_your_required_plugin]' your-script.nu
    
    # or you can use `to nuon` from inside nu:
    let plugins = [
        '/path/to/nu_plugin_foo'
        '/path/to/nu_plugin_bar'
    ]
    nu --plugins ($plugins | to nuon) your-script.nu
    

The last option will run the plugin to get signatures at startup, without modifying the user's registry file, and can even be used when --no-config-file is specified.

Finally, we have also added plugin rm, which removes a plugin from the registry file. This was something that previously needed to be done manually in a text editor, but is made possible by the new format.

New plugin registry format [toc]

Breaking change

See a full overview of the breaking changes

The plugin registry format has been changed from the previous Nushell script format (containing a sequence of register commands) to a custom, declarative format based on MessagePackopen in new window compressed with brotliopen in new window. This new format is called msgpackz, and the default plugin file is plugin.msgpackz instead of plugin.nu.

Your existing plugin.nu file will be automatically upgraded to plugin.msgpackz the first time version 0.93 runs.

The declarative format brings a number of benefits:

  1. It can be edited more structurally, allowing us to replace the entire definition of one plugin at once, which is more logically consistent if a plugin removes commands from one version to the next, and also allowing us to support new commands like plugin rm.
  2. It supports adding metadata about plugins as a whole rather than just the command signatures, if we need to.
  3. It contains the Nushell version that generated it, allowing us to potentially fix issues between versions.
  4. It fails gracefully - if something about a particular plugin can't be parsed because of a change to the serialization format, other plugins that don't have that issue can still load.

We chose to use compressed MessagePack because it is very fast to load, and the file is loaded every time nu starts, so with a lot of plugins or plugins that expose a lot of commands (e.g. the new polars plugin), this was a big problem for startup times in general.

The most significantly breaking part of this breaking change is that the following is no longer possible:

source $nu.plugin-file

You can use the plugin use command instead to (re-)load a specific plugin from the plugin registry file:

> plugin use gstat

Improved terminal interaction for plugins [toc]

Plugins are now able to run over a UNIX domain socket or Windows named pipe instead of stdin/stdout if they support it (#12448open in new window). All Rust plugins support this mode by default, but it is optional; plugins written in other languages are not forced to implement this, and the stdio mode is still always attempted. The feature is advertised as LocalSocket and the upgrade is automatic when supported by the plugin and confirmed to be functional.

This frees up stdio for interaction with the user's terminal, which makes plugins that rely on that, including terminal UI plugins like @amtoine's nu_plugin_exploreopen in new window simpler and more reliable to implement.

We also added engine calls for moving the plugin into the terminal controlling (foreground) process group, mainly for Unix platforms. This was also necessary to fully support terminal UI plugins, after the last release's new persistence functionality caused plugins to run in a new, background process group by default.

Introduction of the Polars Plugin [toc]

With the enhancements to the plugins framework, we have migrated the dataframes crate to the nu_plugin_polars plugin. In this release, the functionality of the Polars plugin closely mirrors that of the dataframes plugin, ensuring seamless migration for existing scripts. After installing the plugin, simply replace dfr with polars to adapt your scripts.

Changes from the Dataframes Crate

  • The command name has been updated from dfr to polars, setting the stage for future support of additional dataframe libraries.
  • The dtypes command has been removed, as its functionality is largely duplicated by the newer polars schema command.
  • The equivalent of dfr ls is now polars store-ls, offering insight into the currently cached dataframes However, there isn't a direct way to display the variables to which they have been assigned.
  • Introducing polars store-get and polars store-rm to respectively retrieve and remove values from the plugin's internal object cache.

Installation

Installing the Polars plugin follows the same process as other plugins:

# Install the polars plugin
> cargo install nu_plugin_polars

# Add the plugin's commands to your plugin registry file:
> plugin add ~/.cargo/bin/nu_plugin_polars

# Load it into scope:
> plugin use polars

Dataframes deprecation [toc]

Deprecation warning

With the introduction of Polars plugin, the dataframe functionality is now deprecated and slated for removal in version 0.94.0.

As such, the dataframe cargo feature will also be removed in 0.94.0. This means that the "full" versions of the pre-compiled nushell binaries will no longer be provided on Github, since these were the same as the standard binaries but with the dataframe feature enabled.

Lazy record deprecation [toc]

Deprecation warning

This version deprecates the lazy make command and lazy records in general. In the next version, 0.94.0, lazy records will be removed entirely.

This motivation behind this is to simplify the language, as lazy records already had flaky support across the various nushell commands. Issue #12622open in new window explains the reasoning in more detail. In the future, the sys and debug info commands will return regular records instead of lazy records. The plan is to introduce new APIs for these commands as a replacement for the existing lazy evaluation.

Stricter rules around setting environment variables [toc]

Breaking change

See a full overview of the breaking changes

The setting of environment variables via the following ways:

  • $env.VAR_NAME = ... assignment
  • scopes with the with-env command
  • the short hand syntax FOO=bar command-to-run
  • the load-env command

will now consistently prohibit you from changing a set of special environment variables controlled by Nushell. Currently this is the following set (already valid for $env. assignment):

  • $env.PWD
  • $env.FILE_PWD
  • $env.CURRENT_FILE This set may be expanded by future breaking changes to ensure consistent semantics of those special variables

Furthermore the FOO=bar BAZ=bla command shorthand syntax to set an environment variable for the scope of one command is now stricter. It will disallow you from repeating the same environment variable name twice and instead return an error (#12523open in new window).

# Now raises an error
FOO=bar FOO=bla command

Additionally, we standardize on using records to pass a map of environment variables to the with-env command, and thus deprecate the other previously allowed forms with this release.

Improved parsing of input/output types [toc]

Thanks to @texastolandopen in new window in #12339open in new window, the input/output types for custom commands can now extend across multiple lines.

def "into bool" []: [
  int -> bool
  string -> bool
] {
  match $in {
    0 => false
    1 => true
    "false" => false
    "true" => true
  }
}

New CLI flag: --no-newline [toc]

This release adds a new CLI flag to the nu binary: --no-newline. Providing this flag will disable the trailing new line from being printed by nushell (#12410open in new window).

Hall of fame [toc]

Bug fixes [toc]

Thanks to all the contributors below for helping us solve issues and bugs 🙏

authordescriptionurl
@fdncredopen in new windowrestore query web --as-table to working order#12693open in new window
@IanManskeopen in new windowMake exit code available in catch block#12648open in new window
@merelymyselfopen in new windowmake grid throw an error when not enough columns#12672open in new window
@IanManskeopen in new windowFix an into bits example#12668open in new window
@IanManskeopen in new windoweach signature fix#12666open in new window
@IanManskeopen in new windowMake the same file error more likely to appear#12601open in new window
@WindSoilderopen in new windowset the type of default NU_LIB_DIRS and NU_PLUGIN_DIRS to list<string>#12573open in new window
@devynopen in new windowReplace subtraction of Instants and Durations with saturating subtractions#12549open in new window
@amtoineopen in new windowfix std log#12470open in new window
@singh-priyankopen in new windowFix negative value file size for "into filesize" (issue #12396)#12443open in new window
@IanManskeopen in new windowexplain refactor#12432open in new window
@ysthakuropen in new windowDon't check if stderr empty in test_xdg_config_symlink#12435open in new window
@IanManskeopen in new windowPrevent panic on date overflow#12427open in new window
@IanManskeopen in new windowFix merging child stack into parent#12426open in new window
@ysthakuropen in new windowFix #12416 by canonicalizing XDG_CONFIG_HOME before comparing to config_dir()#12420open in new window
@IanManskeopen in new windowRange refactor#12405open in new window
@merelymyselfopen in new windowprevent select (negative number) from hanging shell#12393open in new window
@merelymyselfopen in new windowMake auto-cd check for permissions#12342open in new window
@devynopen in new windowAdd BufWriter to ChildStdin on the plugin interface#12419open in new window
@devynopen in new windowFix deadlock on PluginCustomValue drop#12418open in new window
@eopbopen in new windowFix stop suggesting --trash when already enabled (issue #12361)#12362open in new window
@devynopen in new windowFix #12391: mkdir uses process startup directory instead of current script directory#12394open in new window
@merelymyselfopen in new windowprevent parser from parsing variables as units#12378open in new window
@IanManskeopen in new windowFix hooks on 0.92.0#12383open in new window
@sholderbachopen in new windowMake default config conservative about clipboard#12385open in new window
@WindSoilderopen in new windowAvoid panic when pipe a variable to a custom command which have recursive call#12491open in new window
@TheLostLambdaopen in new windowfix(shell_integration): set window title on startup#12569open in new window
@texastolandopen in new windowEnsure currently_parsed_cwd is set for config files#12338open in new window
@YizhePKUopen in new windowFix circular source causing Nushell to crash#12262open in new window
@friaesopen in new windowFix circular source causing Nushell to crash#12411open in new window

Enhancing the documentation [toc]

Thanks to all the contributors below for helping us making the documentation of Nushell commands better 🙏

authordescriptionurl
@deepanchalopen in new windowfix simple typo in commandline_.rs#12387open in new window
@SylvanBrocardopen in new windowUpdate list of supported formats in dfr open error message.#12408open in new window
@IanManskeopen in new windowMention print in the echo help text#12436open in new window
@oornaqueopen in new windowFix typo in help stor import#12442open in new window
@KAAtheWiseGitopen in new windowFix example wording in seq date#12665open in new window
@IanManskeopen in new windowFix an into bits example#12668open in new window
@fdncredopen in new windowimprove nu --lsp command tooltips#12589open in new window
@NotTheDr01dsopen in new windowFixes #12482 by pointing help links for ndjson to a non-spam source (take 2)#12509open in new window
@woosaaahhopen in new windowChange cal --week-start examples + error message#12597open in new window
@maxim-uvarovopen in new windowadd search_term "str extract" to parse command#12600open in new window
@IanManskeopen in new windowImprove the "input and output are the same file" error text#12619open in new window

Our set of commands is evolving [toc]

New commands [toc]

plugin add [toc]

Adds a plugin to the plugin registry file (default $nu.plugin-path).

plugin add ~/.cargo/bin/nu_plugin_gstat

A different registry file can be used if desired:

plugin add --plugin-config ~/polars.msgpackz ~/.cargo/bin/nu_plugin_polars

Replaces part of register, but can be used as a normal command, so it can be used within a script on a generated path. For example the following is now possible:

glob ~/.cargo/bin/nu_plugin_* | each { |file| plugin add $file }

plugin rm [toc]

Removes a plugin from the plugin registry file (default $nu.plugin-path).

plugin rm gstat
plugin rm --plugin-config ~/polars.msgpackz polars

The commands will still remain in scope, but will not be present the next time nu is started.

plugin use [toc]

Loads a plugin from the plugin registry file (default $nu.plugin-path) at parse time.

This replaces the other part of register - actually adding the commands to scope. It does not run the plugin executable, though, it just loads the previously added commands from the registry file.

It is not necessary to add this to your config file to use plugins that are in the registry file that nu was started with. Those are all automatically loaded at startup, just as they were before.

from msgpack [toc]

Reads MessagePackopen in new window format data into Nu values.

Example:

> 0x[a7 4e 75 73 68 65 6c 6c] | from msgpack
Nushell

# Open a MessagePack formatted file:
> open nushell.msgpack
╭─────────┬───────╮
 nushell rocks
╰─────────┴───────╯

You can read a stream of multiple MessagePack objects in the same format as the plugin protocol by using from msgpack --objects:

> open --raw ~/nu_plugin_example.in.raw | from msgpack --objects
╭───┬─────────────────────────────────────────────────╮
 0 ╭───────┬────────────────────────────────────╮
 ╭──────────┬─────────────────────╮
 Hello protocol nu-plugin
 version 0.93.0
 ╭───┬─────────────╮
 features # │    name     │ │ │  │
 ├───┼─────────────┤
 0 LocalSocket
 ╰───┴─────────────╯
 ╰──────────┴─────────────────────╯
 ╰───────┴────────────────────────────────────╯

 1 ...
 2 ...
 3 ...
 4 ...

 5 Goodbye
╰───┴─────────────────────────────────────────────────╯

Example captured with nu_plugin_traceropen in new window.

to msgpack [toc]

Converts Nu values into MessagePackopen in new window.

Example:

> 'Nushell' | to msgpack
Length: 8 (0x8) bytes | printable whitespace ascii_other non_ascii
00000000:   a7 4e 75 73  68 65 6c 6c                             ×Nushell

# Save in MessagePack format to nushell.msgpack:
> {nushell: rocks} | save nushell.msgpack

from msgpackz [toc]

Just like from msgpack, but decompresses with brotliopen in new window first.

The msgpackz format is used by the new plugin file format:

> open ~/.config/nushell/plugin.msgpackz
╭─────────────────┬─────────────────────────────────────────────────────────────────────────────────╮
 nushell_version 0.92.3
 ╭───┬───────────────┬────────────────────────────────────┬───────┬────────────╮
 plugins # │     name      │              filename              │ shell │  commands  │ │
 ├───┼───────────────┼────────────────────────────────────┼───────┼────────────┤
 0 custom_values .cargo/bin/nu_plugin_custom_values [table 8
 rows]      
 1 dbus .cargo/bin/nu_plugin_dbus [table 7
 rows]      
 2 emoji .cargo/bin/nu_plugin_emoji [table 1
 row]       
 3 example .cargo/bin/nu_plugin_example [table 13
 rows]      
 4 explore .cargo/bin/nu_plugin_explore [table 1
 row]       
 5 formats .cargo/bin/nu_plugin_formats [table 4
 rows]      
 6 gstat .cargo/bin/nu_plugin_gstat [table 1
 row]       
 7 inc .cargo/bin/nu_plugin_inc [table 1
 row]       
 ╰───┴───────────────┴────────────────────────────────────┴───────┴────────────╯
╰─────────────────┴─────────────────────────────────────────────────────────────────────────────────╯

to msgpackz [toc]

Just like to msgpack, but compresses with brotliopen in new window after encoding.

For example, to use it to manually edit the plugin file:

open ~/.config/nushell/plugin.msgpackz |
    update plugins {
        # Remove plugins with missing files:
        where { get filename | path exists }
    } |
    save -f ~/.config/nushell/plugin.msgpackz

As with other format commands, open and save will automatically use the msgpackz conversion if the file extension matches.

metadata set [toc]

The new metadata set command added in #12564open in new window allows one to modify the metadata of a pipeline/value.

For example, you can set the data source to be ls, which will activate coloring using LS_COLORS:

[[name]; [Cargo.lock] [Cargo.toml] [README.md]] | metadata set --datasource-ls

Changes to existing commands [toc]

with-env [toc]

Breaking change

See a full overview of the breaking changes

Passing environment variables to with-env via the list-like or single-row table form is now deprecated after #12523open in new window:

# deprecated
with-env [X Y W Z] { $env.X }
# also deprecated
with-env [[X Y]; [W Z]] { $env.X }

Instead, you should use the record form:

with-env { X: 'Y', W: 'Z' } { $env.X }

This also explicitly stops you from repeating the name of a environment variable, which was a potential source of bugs.

# silently: bar
with-env [X foo X bar] { $env.X }
# error
with-env { X: 'foo', X: 'bar' } { $env.X }

Additionally, setting the PWD variable through with-env is now disallowed.

load-env [toc]

Breaking change

See a full overview of the breaking changes

With #12522open in new window, setting the PWD variable through load-env is now disallowed, both when the record is given as a pipeline input or command argument.

last [toc]

Breaking change

See a full overview of the breaking changes

To be consistent with first and other commands, last now errors if the input is empty (#12478open in new window).

To suppress the error and return null if an input is empty, wrap last in a try block:

[] | try { last }

drop [toc]

Breaking change

See a full overview of the breaking changes

With #12479open in new window, drop will now error if the number of rows/columns is negative.

skip [toc]

Breaking change

See a full overview of the breaking changes

To be consistent with take and other commands, skip no longer accepts external streams as input (#12559open in new window).

group-by [toc]

Breaking change

See a full overview of the breaking changes

If a closure was used as the grouper for group-by, then errors inside the closure would previously be ignored. Instead, group-by would put the row/item under the error group. With #12508open in new window, errors are no longer ignored and will immediately be bubbled up.

timeit [toc]

Breaking change

See a full overview of the breaking changes

After #12465open in new window, the timeit command will no longer capture external command output, instead redirecting output to the terminal/stdio.

kill [toc]

Breaking change

See a full overview of the breaking changes

The kill command used to return a stream of values in certain cases. In #12480open in new window this was changed so that kill now always returns a single value (null or a string).

each [toc]

The signature of each previously showed that the closure provided to each had a second Int parameter. This parameter was removed some time ago, and the signature has been updated in #12666open in new window.

ls [toc]

Instead of a single optional path, the ls command now takes a rest argument of paths (#12327open in new window). I.e., you can spread a list into ls:

ls ...[directory1 directory2]

or simply ls multiple directories:

ls directory1 directory2

du [toc]

In the ls PR (#12327open in new window), the same changes were also made to du. So now, du takes a rest argument of paths just like ls.

try [toc]

With #12648open in new window, the exit code of the failed external command can be retrieved in a catch block via the usual $env.LAST_EXIT_CODE.

version [toc]

Thanks to @poliorceticsopen in new window, the version command now has additional columns for the major, minor, patch, and pre portions of the current nushell version (#12593open in new window).

into filesize [toc]

Thanks to @singh-priyankopen in new window in #12443open in new window, into filesize can now parse negative filesizes from strings.

view source [toc]

Thanks to @merelymyselfopen in new window, the view source command now displays more information like type signatures and default argument values (#12359open in new window).

grid [toc]

Instead of returning an error string, grid now triggers an error if it is unable to fit the grid in the available or specified number of columns (#12654open in new window).

explain [toc]

With #12432open in new window, the type column returned from explain should no longer contain string for every row.

Deprecated commands [toc]

register [toc]

This command is deprecated in favor of the new plugin add and plugin use commands. See the relevant section for the reasoning behind this change.

lazy make [toc]

The lazy make command has been deprecated in #12656open in new window following the deprecation of lazy records (see lazy record deprecation).

describe --collect-lazyrecords [toc]

Since lazy records are now deprecated (see lazy record deprecation), the --collect-lazyrecords flag for describe was also deprecated in #12667open in new window.

Removed commands and flags [toc]

run-external flags [toc]

The --redirect-stdout, --redirect-stderr, --redirect-combine, and --trim-end-newline flags have been removed in #12659open in new window following their deprecation in 0.92.0. See the previous release notesopen in new window for migration info.

commandline flags [toc]

The --cursor, --cursor-end, --append, --insert, and --replace flags on commandline were removed with #12658open in new window. These flags were deprecated back in 0.91.0 following the introduction of a new command API for commandline. See the previous release notesopen in new window for more information.

For plugin developers [toc]

New protocol features [toc]

New engine calls [toc]

See the Rust docs for EngineInterfaceopen in new window for specific documentation for Rust plugins.

Breaking changes [toc]

Full changelog [toc]