Nushell 0.106.0

Today, we're releasing version 0.106.0 of Nu. This release adds a system to test out new behavior with experimental options at run-time. This means, we don't need to break the behavior for regular users before we commit to a change. Furthermore this release ships many quality-of-life improvements to the help system, completions and many other areas.

Where to get it

Nu 0.106.0 is available as pre-built binaries or from crates.io. If you have Rust installed you can install it using cargo install nu.

As part of this release, we also publish a set of optional plugins you can install and use with Nushell.

Table of contents

• Highlights and themes of this release
  • NEW: Experimental options
    • Experimental options introduced by this release
  • Improved fuzzy completion
    • Arguments over possible commands
    • Prefer prefix matches
  • Improvements to errors and warnings
• Changes
  • Breaking changes
    • Regression: bare word interpolation on both sides does not work
    • path and directory command arguments are no longer expanded
    • Stricter compound assignment operator type checking
    • parse: unmatched optional capture groups are now null
  • Additions
    • Experimental options
    • clip graduates from std-rfc
    • Improved documentation, errors, and warnings
    • Commands
  • Deprecations
    • --ignore-errors (-i) renamed to --optional (-o)
  • Bug fixes and other changes
    • table
    • Interrupting endless loops
    • Dynamic config with hooks and overlays
    • Preserve type of float values in conversions to various formats
    • Keybind action: KillLine
    • Polars
    • Additional bug fixes
• Hall of fame
• Full changelog

Highlights and themes of this release [toc]

NEW: Experimental options [toc]

This release adds a new "experimental options" system that will help us test out changes to Nushell. This will help us find any possible breakages and gather feedback on ideas before we roll out new changes to everyone.

See the full section below for more information!

Experimental options introduced by this release [toc]

• reorder-cell-paths

Improved fuzzy completion [toc]

Arguments over possible commands [toc]

Thanks to @blindFS, with #16112 fuzzy completion now prioritizes argument completions over possible command names in, which would push argument completions too far down to be useful.

Prefer prefix matches [toc]

By enabling nucleo's prefer_prefix option in #16183, @Bahex made suggestions with matches closer to the start rank higher than suggestions that have matches further from the start.

Improvements to errors and warnings [toc]

This release adds some polish to errors and warnings. It also comes along with some internal changes that will help us add more warnings and minimize breaking changes in the future.

To see some of the new additions (along with some screenshots), check out the full section.

Changes [toc]

Breaking changes [toc]

Regression: bare word interpolation on both sides does not work [toc]

Unfortunately, we caught a regression shortly after release. Bare word interpolations (introduced in 0.105.0) which have expressions before and after a string component cause an error in 0.106.0. This was caused by the fix introduced in #16204. Here's an example which worked in 0.105.0, but not 0.106.0:

let x = 123
($x)/foo/($x)

Note that these forms do still work in 0.106.0:

let x = 123
($x)/foo
foo/($x)

We plan to address this with a 0.106.1 patch release in the coming days. We are hoping to resolve the regression, while still keeping the bug fix from #16204.

path and directory command arguments are no longer expanded [toc]

As of #15878, unquote file/directory paths as command arguments will no longer be expanded into absolute paths. This may break scripts which rely on the old absolute expansion. These paths should now be manually expanded.

Paths with path components such as .. and ~ are still expanded, but paths will not automatically be expanded into their absolute form.

Here's an example with a simple custom command with a path parameter. Let's say we are in the directory /home/user, and we call it with the argument test/../bar:

def foo [my_path: path] { $my_path }

foo test/../bar

The .. component is still expanded, but the path remains relative.

Stricter compound assignment operator type checking [toc]

Compound assignment operators (+=, /=, etc.) now verify that the operation is compatible between the types. Anything that was disallowed with the "plain" operator (+, /, etc.) should now also be disallowed with the compound assignment operators.

This mostly affects edge cases, but the mostly likely one to cause an issue is /=:

mut x = 1

# This was already an error
$x = $x / 2

# This is now an error
$x /= 2

parse: unmatched optional capture groups are now null [toc]

As of #16094, parse returns null for unmatched optional capture groups, rather than an empty string. This makes it possible distinguish unmatched groups from groups matching empty strings.

let example = r#'
user=0 action="bar"
user=0 action=""
user=0
'#

$example
| parse --regex '(?m)^user=(?<user>[0-9]+)(?: action="(?<action>[^"]*)")?$'
| to nuon
# => [[user, action]; ["0", bar], ["0", ""], ["0", ""]]

user=0 action="" and user=0 are different, yet indistinguishable in parse's output before this change. Now it's clear:

[[user, action]; ["0", bar], ["0", ""], ["0", null]]

Additions [toc]

Experimental options [toc]

In #16028, #16093, #16095, #16096, #16121 and #15682, @cptpiepmatz set up the infrastructure for "experimental options".

This allows us try out, as the name suggests, experimental changes from drastic refactors to unstable features without breaking daily usage. If you want to opt in and try them out yourself, you can enable them in the following ways:

• By setting the NU_EXPERIMENTAL_OPTIONS variable to a comma separated list of option names before running nu.

  NU_EXPERIMENTAL_OPTIONS="reorder-cell-paths,.." nu

  This environment variable is not respected when running scripts to avoid breaking independent executable scripts.

• By starting nu with the --experimental-options flag, passing in a list of option names.

  nu --experimental-options=[reorder-cell-paths,..]

if $nu.is-login and $env.NU_EXPERIMENTAL_OPTIONS? != "all" {
    $env.NU_EXPERIMENTAL_OPTIONS = "all"
    exec $nu.current-exe
}

Our first experimental option: reorder-cell-paths

In #15682, @Bahex changed the way cell-path access (when you access rows/columns of a value: $table.foo.2) works.

Previously (or rather normally until this option becomes the default) cell-path access happened in the exact order the cell-path itself was specified.

let val = [[bar]; [{foo: [A, B]}], [{foo: [C, D]}]]

$val.bar
# => [[foo]; [[A, B]], [[C, D]]]

$val.bar.foo
# => [[A, B], [C, D]]

$val.bar.foo.0
# => [A, B]

$val.bar.foo.0.0
# => A

Even when you specified the exact cell (as with the last example), each step would happen in order. So there were a lot of intermediate values that would be thrown away.

While this is not much of a problem with small and relatively flat values, accessing a cell in large table had a noticeable performance impact in addition to using more memory.

This could be worked around by reordering the cell-path to prioritize row accesses over column accesses:

$val.0.bar.foo.0
# => A

With this change, this reordering is done automatically whenever possible. This also has the secondary effect of errors one may encounter:

let val = [{foo: A}, {baz: B}]
$val
# => ╭───┬─────┬─────╮
# => │ # │ foo │ baz │
# => ├───┼─────┼─────┤
# => │ 0 │ A   │ ❎  │
# => │ 1 │ ❎  │ B   │
# => ╰───┴─────┴─────╯

$val.foo
# => Error: nu::shell::column_not_found
# =>
# =>   × Cannot find column 'foo'
# =>    ╭─[entry #2:1:22]
# =>  1 │ let val = [{foo: A}, {baz: B}]
# =>    ·                      ────┬───
# =>    ·                          ╰── value originates here
# =>    ╰────
# =>    ╭─[entry #4:1:6]
# =>  1 │ $val.foo
# =>    ·      ─┬─
# =>    ·       ╰── cannot find column 'foo'
# =>    ╰────

$val.foo.0
# => A

clip graduates from std-rfc [toc]

After spending some time in std-rfc, with #15877 clip module graduates to std, the first module from std-rfc to do so!

Using std-rfc/clip will give a deprecation warning, instructing you to switch to the std version.

Improved documentation, errors, and warnings [toc]

• With #15892, text written in backticks inside help text will now be highlighted as code. If it's valid Nushell code, it will be highlighted as such. Otherwise, it will use a "dimmed" style. You can make use of this yourself in custom command documentation as well! Here's what this looks like with some examples on the select command documentation (notice name and first 4 being highlighted):

  

• Warnings should now be a bit more distinct: they will now show up with a orange caution symbol, rather than the same red X used for errors (#16146)

  

• As of #16166, "generic" errors and warnings will now show up with a default error code (ex. nu::shell::error) indicating which part of Nushell it originates from. This should make errors appear more consistent. (This also applies to errors created with error make!)

  Before	After

• The history isolation feature only works when using the SQLite history. However, previously there was no indication of this. #16151 adds a warning if you attempt to use history isolation in combination with the text format history.

  

Commands [toc]

• This release adds only to std-rfc, inside the iter module (#16015). This behaves similar to first in that it errors if the input is empty, but it also ensures that the input has no additional elements. This can be useful when chained with where for example, when you expect only one element should match a condition:

  ls | where name == "foo.txt" | only modified

  This example is similar to get 0.modified, while also ensuring there's only one matching row.
• Thanks to @JohnSwiftC, the parse command now allows you to specify a backtrack limit for the regex engine in #16000. This can help out for complicated regexes, where the default backtricking limit isn't large enough.
• Thanks to @kumarUjjawal, the drop nth command now supports spreading its arguments. This allows you to spread a list of indices directly into drop nth (#15897). Here's a (contrived) example:

  ls | drop nth ...[1 2 3]

• There are some improvements to the find command, thanks to @new-years-eve:
  • Text in nested structures will now be properly highlighted. (#15850)
  • The help command now re-uses the internals of the find command, rather than reimplementing it (#15982)
• The http command can now be used without manually specifying get or post, thanks to @noahfraiture. If just http is used, the request type will automatically be chosen to be GET if there is no payload. If there is a payload, the request type will be POST (#15862).
• Thanks to @weirdan, query xml allows using namespace prefixes in the XPath query by specifying the namespace with the --namespaces flag (#16008).
• It's now possible to get the span of a pipeline using the metadata and metadata access commands with #16014. This makes it possible to have nicer errors which point at the source of a value. Here's an example, using the span from metadata access to point at ls:

  ls | metadata access {|meta|
    error make {msg: "error", label: {text: "here", span: $meta.span}}
  }
  # => Error: nu::shell::error
  # =>
  # =>   × error
  # =>    ╭─[entry #1:1:1]
  # =>  1 │ ls | metadata access {|meta|
  # =>    · ─┬
  # =>    ·  ╰── here
  # =>  2 │   error make {msg: "error", label: {text: "here", span: $meta.span}}
  # =>    ╰────
• Thanks to @echasnovski, the gstat plugin now has a state entry indicating the current state of the git repo. For example, it can indicate whether the repo state is clean, or in the middle of a merge or rebase. (#15965)
• There is a new table theme called double, thanks to @echasnovski in #16013. This is similar to the single theme, but, well, double! Here's an example:

  ╔═══╦═══╦════╗
  ║ # ║ a ║ b  ║
  ╠═══╬═══╬════╣
  ║ 0 ║ 1 ║ 11 ║
  ║ 1 ║ 2 ║ 12 ║
  ╚═══╩═══╩════╝

• In #16099, @yertto style reset codes to ansi

Deprecations [toc]

--ignore-errors (-i) renamed to --optional (-o) [toc]

Affected commands:

• get --ignore-errors
• select --ignore-errors
• reject --ignore-errors

--ignore-errors (-i) flag is renamed to --optional (-o) to better reflect it's behavior and use terminology consistent with cell-paths.

This is also intended to free the -i short flag to stand for --ignore-case (not yet implemented) which will treat all parts of the cell-path arguments as case insensitive.

There will of course be long deprecation period to avoid breaking existing code.

Implemented by @Bahex in #16007.

alias get-builtin = get
def get [--optional (-o), --ignore-errors (-i), --sensitive (-s), cell_path: cell-path, ...rest: cell-path] {
  get-builtin --optional=($optional or $ignore_errors) $cell_path ...$rest

alias get-builtin = get

# Extract data using a cell path.
#
# This is equivalent to using the cell path access syntax: `$env.OS` is the same as `$env | get OS`.
#
# If multiple cell paths are given, this will produce a list of values.
@example "Get an item from a list" { [0 1 2] | get 1 } --result 1
@example "Get a column from a table" { [{A: A0}] | get A } --result [A0]
@example "Get a cell from a table" { [{A: A0}] | get 0.A } --result A0
@example "Extract the name of the 3rd record in a list (same as `ls | $in.name.2`)" { ls | get name.2 }
@example "Extract the name of the 3rd record in a list" { ls | get 2.name }
@example "Getting Path/PATH in a case insensitive way" { $env | get paTH! }
@example "Getting Path in a case sensitive way, won't work for `PATH`" { $env | get Path }
def get [
  --optional (-o) # make all cell path members optional (returns null for missing values),
  --ignore-errors (-i) # ignore missing data (make all cell path members optional) (deprecated),
  --sensitive (-s) # get path in a case sensitive manner (deprecated),
  cell_path: cell-path # The cell path to the data.
  ...rest: cell-path # Additional cell paths.
]: [
  list<any> -> any,
  table -> any,
  record -> any,
  nothing -> nothing
] {
  get-builtin --optional=($optional or $ignore_errors) --sensitive=$sensitive $cell_path ...$rest
}

Bug fixes and other changes [toc]

table [toc]

Previously, table --expand could panic under some circumstances while attempting to wrap content in cells. Thanks to @zhiburt, this is no longer the case and table -e works as expected. @zhiburt also optimized the table command in #15900, #15901, #15902.

Interrupting endless loops [toc]

By making the IR evaluator check for interrupts before certain instructions, @132ikl made it possible to break out of endless loops in #16134.

Dynamic config with hooks and overlays [toc]

Thanks to @Bahex, with #16021 hooks ($env.config.hooks) can update $env.config and have the config change take effect as expected, where previously the change wouldn't take effect until $env.config was manually updated in some way.

With #16154, changes to $env.config caused by overlay use and overlay hide take effect as expected.

Preserve type of float values in conversions to various formats [toc]

Previously, nushell omitted the decimal part of round float numbers. This affected:

• conversions to delimited formats: csv, tsv
• textual formats: html, md, text
• pretty printed json (--raw was unaffected)
• float values are displayed in the REPL

With #16016, float numbers are always represented with at least on decimal place which means:

• round trips through csv, tsv, and json preserve the type of round floats
• It's always clear whether a number is an integer or a float in the REPL

  4 / 2
  # => 2.0

Keybind action: KillLine [toc]

In #16221, @sholderbach Added {edit: KillLine}, which mimics emacs' Ctrl-K behavior.

Polars [toc]

@ayax79 made several PRs improving the polars plugin:

• #15963: Make polars last consistent with polars first
• #15964: Allow polars schema --datatype-list to be used without pipeline input
• #16132: Added flag limit for polars arg-sort
• #15953: Add groupby support for polars last

Additional bug fixes [toc]

• In #15980, @Bahex fixed the std/log module to work when the logging related environment variables aren't set.
• In #16155, @Bahex made the get command work similar to normal cell-path accesses on null values.
• In #16184, @Bahex fixed an issue where errors inside of export-env blocks would not be properly reported when using overlay use.
• In #16192, @Bahex fixed an (annoying) issue where duplicated text would appear on the command line when typing a where command.
• In #16204, @Bahex fixed an issue where repeated parenthesis caused memory leak.
• In #16219, @Bahex fixed group-by --to-table to return an empty list on empty input.
• In #15919, @JoaquinTrinanes, changed argument parsing to use the latest specified flag value when repeated. This makes having flags in aliases more convenient.
• In #16018, @Mrfiregem allowed update cells to work on single records.
• In #15955, @Mrfiregem made it possible to use default with a closure that produces a stream.
• In #15904, @WindSoilder fixed an issue where hide-env would incorrectly hide environment variables outside of an overlay.
• In #16171, @WindSoilder fixed an issue where piping to an external command into a value would cause the command to continue running. It now sends a SIGPIPE to the process, similarly to how piping a long running command into a command which immediately exits will send a SIGPIPE to the long running command.
• In #16033, @adithyaov update the behavior of start to expand paths before passing them to external programs, making it work reliably.
• In #15780, @blindFS refactored how columns are checked for in records, increasing performance.
• In #15998, @blindFS fixed an issue where tab completion for a path argument would sometimes incorrectly expand paths with prefixes such as tildes.
• In #16001, @kumarUjjawal fixed a panic for the random dice command when using --sides 0.
• In #16049, @kumarUjjawal added an error when both --datasource-filepath and -datasource-ls are used together, as they are incompatibe with each other.
• In #15950, @liquid-dragons improved the precision of parsing filesize values. Previously, there were some rounding errors for large filesize units.
• In #16194, @marienz fixed type checking for optional arguments to a closure. Before, only positional arguments would be type checked.
• In #16152, @ryanxcharles fixed typo on the pipeline redirection operatros documentation (help pipe-and-redirect).
• In #16176, @sgvictorino fixed spans pointing to rest parameters. Using metadata on a ...rest argument should now give the span for all of the arguments, not just the first one.
• In #15985, @sholderbach restricted $env.config.show_banner to valid options. Assigning an invalid option now throws an error.
• In #15838, @fdncred fixed stor insert/delete collision
• In #16133, @dead10ck fixed polars' datetime type conversion

Hall of fame [toc]

Thanks to all the contributors below for helping us solve issues, improve documentation, refactor code, and more! 🙏

Full changelog [toc]