Nushell 0.102.0

Today, we're releasing version 0.102.0 of Nu. This release adds runtime pipeline input type checking, several new commands and operators, and various other miscellaneous improvements.

Where to get it

Nu 0.102.0 is available as pre-built binaries or from crates.io. If you have Rust installed you can install it using cargo install nu.

As part of this release, we also publish a set of optional plugins you can install and use with Nushell.

Table of contents

• Highlights and themes of this release
  • Run-time pipeline input typechecking
• Changes
  • Additions
    • New command: version check
    • generate with input
    • New operators: has and not-has
    • User autoload directories
    • Custom completers fall back to file completions
    • get is now const
    • format number (renamed from fmt)
    • chunks supports binary input
    • New command: bytes split
    • New command: help pipe-and-redirect
    • The start command now works with non-HTTP URIs
    • Duration value output from debug profile
    • Several into commands can now operate on their own type
    • touch now supports globbing
    • source null/use null
    • run-external and exec now support list spreading
    • New config option: use_ansi_coloring: 'auto'
    • New config option: banner: 'short'
    • move --first/--last flags
    • find --no-highlight switch
    • seq date enhancements
    • content_type metadata is now assigned to known filetypes that don't have a from convertor
  • Breaking changes
    • $env.config.filesize changes
    • format filesize
    • ENV_CONVERSIONS take effect immediately
    • External completers fall back to file completions only on null
    • Custom completions inherit case_sensitive option from config
    • Command pipeline input/output type changes
    • N-dots tweak
  • Deprecations
    • into bits
    • range
    • fmt
  • Removals
    • utouch
    • split-by
    • date to-record
    • date to-table
  • Bug fixes and other changes
    • Immediate error return
    • config reset fix
    • Fix empty list display when using print
    • Internal changes to PipelineData spans
    • Range related bug fixes
    • std enhancements
    • Nuon conversions of range values now keep step size
    • explore command panic when viewing small binary files
    • Convert Path to list in main and preserve case
    • Fix improperly escaped strings in stor update
    • Fix: Auto cd should not canonicalize symbolic path
    • Fix: Pressing Esc or Q was not closing explore
    • Custom values are now expanded in tables
    • Change how and and or operations are compiled to IR to support custom values
    • Fix stor reset when there are foreign keys
    • New, improved grid icons
    • SHLVL decrement on exec
    • cp reflink support on FreeBSD
    • Fix root directory traversal issue
    • Fix table panic
    • Parser fixes
    • Streaming improvements
• Notes for plugin developers
• Hall of fame
• Full changelog

Highlights and themes of this release [toc]

Run-time pipeline input typechecking [toc]

Previously, the type of a command's pipeline input was only checked against the input/output types of that command at parse-time. With #14741, this check also now happens at run-time. While these kinds of errors can often be caught at parse-time, adding run-time checks ensures commands can't receive unexpected input values.

Here's an example command, which should only accept integer pipeline input:

def cool-int-print []: int -> nothing {
  print $"my cool int is ($in)"
}

1 | cool-int-print
# => my cool int is 1

"evil string" | cool-int-print
# => Error: nu::parser::input_type_mismatch
# =>
# =>   × Command does not support string input.
# =>    ╭─[entry #12:1:17]
# =>  1 │ "evil string" | cool-int-print
# =>    ·                 ───────┬──────
# =>    ·                        ╰── command doesn't support string input
# =>    ╰────

The parser sees a string, and rejects the invalid input. Before this release however, if the parser couldn't determine an input was invalid, it would happily pass along the invalid value. In this example, the output type of from json is any, so the parser doesn't know we're trying to pass a string to cool-int-print:

"evil string" | from json | cool-int-print
# => my cool int is evil string

After this release, Nushell will also enforce the allowed input types at run-time (notice the error starts with nu::shell rather than nu::parse):

"evil string" | from json | cool-int-print
# => Error: nu::shell::only_supports_this_input_type
# =>
# =>   × Input type not supported.
# =>    ╭─[entry #18:1:17]
# =>  1 │ "evil string" | from json | cool-int-print
# =>    ·                 ────┬────   ───────┬──────
# =>    ·                     │              ╰── only int input data is supported
# =>    ·                     ╰── input type: string
# =>    ╰────

Because of this change, you may notice some new errors in your scripts like the one above. We've already adjusted the input/output types of many commands to better match their actual behavior (see Command pipeline input/output type changes), but if you run into an unexpected input type checking error, don't hesitate to file an issue!

Changes [toc]

Additions [toc]

New command: version check [toc]

Thanks to @fdncred for the new version check command (#14880) which can check your current Nushell version against the latest available release.

generate with input [toc]

In #14804, @Bahex added input support to generate. Without pipeline input, generate works same as before. But with pipeline input, generate's closure argument is supplied two arguments:

• an item from the input
• the next value from the previous iteration

This allows generate to act as a stateful each and more, filling the niche of a stateful and streaming filter command.

std/iter scan

Thanks to the new capabilities of generate, iter scan was updated to be streaming. It is also considerably more performant with large inputs.

New operators: has and not-has [toc]

In #14841, @Bahex added two new operators: has and not-has. They are analogous to in and not-in, with the order of operands switched. This makes certain operations more ergonomic.

[[name, children]; [foo, [a, b, c]], [bar [d, e, f]]] | where ("e" in $it.children)
# vs
[[name, children]; [foo, [a, b, c]], [bar [d, e, f]]] | where children has "e"

User autoload directories [toc]

Previous Nushell releases added the vendor/autoload directories which were automatically sourced during startup. In #14669, @NotTheDr01ds extended this to include user directories.

During startup, any .nu files in ($nu.default-config-dir)/autoload will be automatically sourced during startup. This occurs after any vendor files have been processed, allowing user override of vendor settings if needed.

Custom completers fall back to file completions [toc]

With #14781, custom completers can return null to fall back to file completions.

get is now const [toc]

With #14751, the get command can now be used during constant evaluation. For example:

const foo = [1 2 3] | get 2
$foo
# => 3

format number (renamed from fmt) [toc]

The fmt command, which is used for converting numbers to a variety of different formats, has been renamed to format number in #14875.

chunks supports binary input [toc]

Thanks to @Bahex in #14649, chunks command now works on binary values and binary streams, making it easier to inspect and work on binary values and allowing partial reads from streams without collecting.

For example, while working with a binary file or protocol, you might need to convert an IP address stored as raw bytes to the more commonly used string format.

0x[7F000001] | chunks 1 | each { into int } | str join "."
# => 127.0.0.1

New command: bytes split [toc]

In #14652, @Bahex added bytes split. It is similar to split row and lines:

• Like split row, it splits its input over an arbitrary delimiter.
• Like lines, it is streaming.

Some programs can output text delimited by characters or sequences other than newlines ("\n"), commonly the NUL byte ("\0"), as the returned text may contain newlines. If it is desirable to not collect the input or to act on a result as soon as possible bytes split can be used.

For example, reading events from a socket or pipe:

open --raw ./events.sock | bytes split (char nul) | each { decode }

New command: help pipe-and-redirect [toc]

Thanks @WindSoilder for the new help pipe-and-redirect command which lists the pipe and redirect operators (#14821).

The start command now works with non-HTTP URIs [toc]

Thanks to @anomius for adding support for additional URI types to the start command in (#14370). This will call a system-specific opener under the hood (open for macOS, start for Windows, and xdg-open or some other program for Linux).

This means that, for example, if you have Spotify installed and configured to open spotify: links, then the following will now start a Rickroll:

start spotify:track:4PTG3Z6ehGkBFwjybzWkR8?si=f9b4cdfc1aa14831

Of course, there are many more productive uses and URIs.

Duration value output from debug profile [toc]

The debug profile command now has a --duration-values flag, which allows outputting the run time of an instruction with proper duration values rather than floats representing milliseconds. This allows for more fine-grained measurement, and lets you use commands which work on duration values #14749.

Several into commands can now operate on their own type [toc]

Thanks to @Tyarel8 in #14845, #14881, and #14882, the into datetime, into cell-path, and into glob commands can now accept (respectively), a datimetime, cell-path, or glob. The commands will now return the values unaltered in these cases.

touch now supports globbing [toc]

Thanks to @hjetmundsen, the utouch (now touch) command supports globbing (#14674).

source null/use null [toc]

Thanks to @Bahex in #14773, source and use can be used with null, in which case the command is a noop. This allows conditionally sourcing files, if the condition is decidable at parse time.

const file = path self local.nu
const file = if ($file | path exists) { $file } else { null }

source $file

run-external and exec now support list spreading [toc]

With #14765, lists can now be spread directly into run-external and exec:

let command = ["cat" "hello.txt"]
run-external ...$command
# => hello world!

New config option: use_ansi_coloring: 'auto' [toc]

@cptpiepmatz config.use_ansi_coloring: 'auto' option in #14647. When set, Nushell will use ANSI coloring only when outputting to a terminal. This allows easier use of Nushell in embedded environments. When embedding Nushell in an environment that supports ANSI formatting, you can change the setting to true.

New command: config use-colors

The new config use-colors command will return the evaluated result of the use_ansi_coloring option. This allows scripts and custom commands to determine whether to add or strip ANSI formatting from their output (#14683).

New config option: banner: 'short' [toc]

With #14638, users can now configuration Nushell to start with a "short" welcome banner. The "short" banner will only show the startup time, allowing the user to populate the rest of the banner information as desired.

move --first/--last flags [toc]

@Coca162 added --first/--last flags to the move command in #14961, providing a convenient shorthand for moving a column to the first or last position in a table.

find --no-highlight switch [toc]

With #14970, @Mudada added --no-highlight to the find command. This provides an easy way to remove ANSI formatting and highlighting from non-regex finds and still keep the result as a table.

seq date enhancements [toc]

The seq date command can now use any duration for the --increment thanks to @pyz4's #14903.

content_type metadata is now assigned to known filetypes that don't have a from convertor [toc]

In (#14670), @NotTheDr01ds changed the open command to assign content_type metadata when a from convertor isn't available for a known filetype. This primarily allows open file.nu to provide application/x-nuscript as the content-type even when not using open --raw.

content_type set for config nu output

Similarly in #14666, @cptpiepmatz updated the config nu command to add content_type metadata. These two changes allow users to create a disable hook which will automatically nu-highlight the results.

Breaking changes [toc]

$env.config.filesize changes [toc]

This release saw an overhaul to file size formatting and related config settings (#14397). Previously, two separate config settings, $env.config.filesize.format and $env.config.filesize.metric, determined how file sizes were formatted. format determined the unit to use and metric determined whether to use metric or binary units. However, the two settings were allowed to conflict. E.g., format could be set to a binary unit, but metric could be set to true. To avoid this confusion, the file size unit is now controlled through a single option: $env.config.filesize.unit. It can be set to one of the following values:

• A metric unit: kB, MB, GB, TB, PB, or EB (case sensitive!).
• A binary unit: KiB, MiB, GiB, TiB, PiB, or EiB (case sensitive!).
• metric: automatically choose a metric unit of an appropriate scale. In the old config, this would be equivalent to { format: auto, metric: true }.
• binary: automatically choose a binary unit of an appropriate scale. In the old config, this would be equivalent to { format: auto, metric: false }.

# unit = kB
1kB  # => 1 kB
1KiB # => 1.024 kB

# unit = KiB
1kB  # => 0.9765625 KiB
1KiB # => 1 KiB

# unit = metric
1000B     # => 1 kB
1024B     # => 1.024 kB
10_000MB  # => 10 GB
10_240MiB # => 10.73741824 GB

# unit = binary
1000B     # => 1000 B
1024B     # => 1 KiB
10_000MB  # => 9.313225746154785 GiB
10_240MiB # => 10 GiB

In addition, you can now control how many decimal places file sizes should be formatted with using $env.config.filesize.precision. When set to null, as many decimal places as necessary will be printed. Otherwise, when set to a integer, that number of decimal places will be shown.

# precision = 1
1001B # => 1.0 kB

# precision = 0
1001B # => 1 kB

# precision = null
1001B # => 1.001 kB
1000B # => 1 kB

The default $env.config.filesize is { unit: metric, precision: 1 }.

format filesize [toc]

Related to changes above, format filesize is now case-sensitive for the file size unit to avoid ambiguity for KB (#14397).

ENV_CONVERSIONS take effect immediately [toc]

Thanks to @Bahex in #14591, when $env.ENV_CONVERSIONS is updated, changes take effect immediately. Previously, to make use of ENV_CONVERSIONS in your config.nu, you would need to set it up in env.nu, as it would only take effect after the file it was modified in was read. Now, you can set it in config.nu and immediately make use of its results. This also makes it useful for use in scripts and other modules.

External completers fall back to file completions only on null [toc]

With #14781, the external completer will fall back to file completions only if null is returned. The external completer used to do this for any invalid value, but completions will now be suppressed if a non-null invalid value is returned.

Custom completions inherit case_sensitive option from config [toc]

With #14738, when matching suggestions from a custom completions, the case sensitivity setting is always inherited from $env.config.completions.case_sensitive. Previously, if a custom completer returned a record containing an options field but didn't specify the case_sensitive option, it would default to true.

This change doesn't apply to custom completers that return either lists or records without an options field. For such completers, case_sensitive was always inherited from $env.config.completions.case_sensitive.

Command pipeline input/output type changes [toc]

As part of the introduction of run-time pipeline input type-checking (#14741, #14922), we discovered some inaccuracies in the listed input types for each command. The following commands had their input/output types adjusted to be more accurate:

• drop nth
• from csv
• from tsv
• get
• headers
• history import
• into string
• reject
• rotate

Additionally, the following commands have temporarily been given an input type of any to prevent undesirable errors. Unfortunately, this prevents input type checking at both parse-time and run-time for these commands, despite being possible previously. This doesn't affect many commands, and is intended as a workaround until a more robust solution can be implemented. For more details, see #14922.

• load-env
• nu-check
• open
• polars when
• stor insert
• stor update
• format date
• into datetime

N-dots tweak [toc]

With #14755, n-dots such as ... are no longer expanded when prefixed with ./ (ex. ... is expanded, ./... is no longer expanded). This should make the feature less surprising, while also providing better compatibility with external programs using ... as part of their CLI.

Deprecations [toc]

The following commands have been deprecated in 0.102.0. They remain available, but will be removed in a future release. We recommend updating existing code to use the replacement command.

into bits [toc]

into bits is deprecated and replaced with the new format bits command with (#14634.

range [toc]

range is deprecated and replaced with the new slice command with #14825.

fmt [toc]

fmt is deprecated and replaced with the new format number command with #14875.

Removals [toc]

utouch [toc]

utouch is now the touch command in #14721.

split-by [toc]

The split-by command was previously deprecated in 0.101 and have been removed in this release (#14726).

date to-record [toc]

The date to-record command was previously deprecated in 0.101 and have been removed in this release (#14726).

date to-table [toc]

The date to-table command was previously deprecated in 0.101 and have been removed in this release (#14726).

Bug fixes and other changes [toc]

Immediate error return [toc]

With #14874, if Nushell detects that an error value passed to a command as pipeline input or as an argument, it will immediately return that error rather than continuing execution. Nested errors (errors in lists, records, etc.) are unaffected.

config reset fix [toc]

With the config changes in the previous release, some of the internal files were moved around, causing config reset to use the "default" config (e.g., config nu --default) rather than the "scaffold" or empty config which gets created when you start Nushell for the first time. With #14756, config reset now works as intended again.

Fix empty list display when using print [toc]

Previously, the print command would print the "empty list" placeholder text without a newline, causing messy looking output. This is fixed by #14758, #14766.

Internal changes to PipelineData spans [toc]

With #14757, spans will now be maintained in some places where they were previously overridden by command calls. Some errors should now point to the actual source of a value, rather than pointing to the command which used it.

Range related bug fixes [toc]

In #14863, @Bahex fixed range related bugs in

• str substring
• str index-of
• slice
• bytes at

std enhancements [toc]

In #14763, @Bahex made small, backwards compatible enhancements to std

• iter find and iter find-index work better with streams, only consuming the input up to the first matching item.
• Added log set-level, a small convenience command for changing the current logging level.

Nuon conversions of range values now keep step size [toc]

Previously, the step size of range values were discarded when converting to nuon. With #14687, the to nuon conversion of range values now include the step size. As a result, to nuon/from nuon round trips now work for these values. Thanks @Bahex!

explore command panic when viewing small binary files [toc]

#14592 fixes a panic in explorer when viewing small binary files thanks to @ChetanXpro.

Convert Path to list in main and preserve case [toc]

With #14764, the system PATH is now converted to a list before the configuration files are processed. An explicit ENV_CONVERSIONS is no longer required for the PATH.

It should be safe to remove any existing ENV_CONVERSIONS entry for PATH.

The case of the PATH is also now preserved during this conversion.

Fix improperly escaped strings in stor update [toc]

@NotTheDr01ds fixed an issue in stor update which prevented single quotes from being stored properly (#14921).

Fix: Auto cd should not canonicalize symbolic path [toc]

With #14708, @WindSoilder fixes an issue where using .. to go up one directory from a symlink would result in the wrong PWD.

Fix: Pressing Esc or Q was not closing explore [toc]

Thanks to @dam4rus, Esc and Q now properly close explore once again (#14941).

Custom values are now expanded in tables [toc]

Previously, custom values were being stringified in tables. #14760 expands these to display as Nushell values thanks to @dead10ck.

Change how and and or operations are compiled to IR to support custom values [toc]

#14653 allows the and and or operators to support custom values once again thanks to @devyn.

Fix stor reset when there are foreign keys [toc]

Thanks to @fdncred for fixing an issue where stor reset wasn't properly dropping tables (#14772).

New, improved grid icons [toc]

grid --icons now uses the icons from the devicons crate after #14827.

SHLVL decrement on exec [toc]

Thanks @rikukiix for fixing exec to properly decrement the SHLVL environment variable (#14707).

cp reflink support on FreeBSD [toc]

Thanks to @sgvictorino, cp no longer errors with "--reflink is only supported on Linux and macOS" (#14677).

Fix root directory traversal issue [toc]

@userwiths fixed an issue where cd .. at the root of a drive was returning the incorrect directory (#14747). Thank you!

Fix table panic [toc]

#14710 fixes an issue where nested data was creating a panic when unexpanded in a display hook. Thanks @zhiburt!

Parser fixes [toc]

• Thanks @WindSoilder for fixing an issue where a duration suffix in a variable name in a range confused the parser (#14848).
• Thanks @blindFS for multiple parser fixes (and even more LSP fixes!):
  • Fixes $in/$it having an unknown span (#14789).
  • Fixed an issue where the span of $it/$in was set to the first character of its scope (#14817).
  • Fixed a missing span of the entire block of a module file (#14889).
  • Fixed an issue where the wrong span was highlighted when an else block had a parse error (#14912).
  • Fixed the span of a keyword expression not including its subsidiary expression (#14928).
• @RobbingDaHood fixed multiple issues with parsed comments not being prefixed with a space or tab, or being the beginning of a token (#14616). Thank you!

Streaming improvements [toc]

• du now streams thanks to @WindSoilder! (#14665)
• get and reject now support streaming thanks to @cosineblast! (#14622)
• bytes at now streams thanks to @simon-curtis in #13552. Additionally, skip and take should now stream binary output when provided streaming binary input.

Notes for plugin developers [toc]

@cptpiepmatz refactored I/O errors in #14927, simplifying them and encouraging including spans in error messages. A new ShellError::Io(IoError) variant has been added to ShellError, replacing FileNotFound, IOError, and other variants. The Io variant holds an instance of the newly added IoError struct, which contains information common to I/O errors, such as error kind, span, and optional extra information. See the linked PR for more information, including how to construct IoError values.

Hall of fame [toc]

Thanks to all the contributors below for helping us solve issues, improve documentation, refactor code, and more! 🙏

Full changelog [toc]