Nushell 0.101.0

Today, we're releasing version 0.101.0 of Nu. This release adds a simplified startup configuration, several new commands, and additional tools to introspect the Nushell engine.

Where to get it

Nushell 0.101.0 is available as pre-built binaries or from crates.io. If you have Rust installed, you can also install it using cargo install nu.

As part of this release, we also publish a set of optional plugins you can install and use with Nushell.

Table of contents

• Highlights and themes of this release
  • Simplified Startup Configuration
    • Related Startup Config Changes
    • Breaking Startup Config Changes
• Changes
  • Additions
    • path self
    • chunk-by
    • term query
    • merge deep
    • utouch
    • WASM support
    • sys net columns
    • Raw string pattern matching
    • Duration/Date Arithmetic
    • explore keybinds
    • Version in Startup banner
    • input --default
    • PowerShell script invocation on Windows
    • New introspection tools
  • Breaking changes
    • ++ operator
    • Stricter command signature parsing
    • group-by
    • timeit
    • sys cpu
    • from csv and from tsv
    • std/iter scan
    • Completion sorting
    • Import module naming
    • table formatting and the display_output hook
    • du flag changes
    • Code specific environment variables updated during source
  • Deprecations
    • split-by
    • date to-record and date to-table
    • do --ignore-shell-errors and --ignore-program-errors
  • Removals
    • NU_DISABLE_IR
  • Bug fixes and other changes
    • ls
    • SHLVL
    • from commands
    • Completions on custom commands
    • seq char
    • http multipart
    • scope variables
    • help system
• Notes for plugin developers
• Hall of fame
• Full changelog

Highlights and themes of this release [toc]

Simplified Startup Configuration [toc]

With #14249, Nushell now always loads its internal default_env.nu before the user env.nu is loaded, then loads the internal default_config.nu before the user's config.nu is loaded. This allows for a simpler user-configuration experience. Details are in this blog entry along with an updated Configuration Chapter which should go live sometime today.

Related Startup Config Changes [toc]

• #14345: Hooks fields are non-optional
• #14341: Hooks now default to an empty value of the proper type (e.g., [] or {}) when not otherwise specified. This means that you can always safely append or merge a new hook without first needing to check if it was a valid list/record.
• #14435: An $env.config is always created at startup, populated with default values, even when no configuration files are loaded.
• #14549: The const version of NU_LIB_DIRS is now populated by default instead of $env.NU_LIB_DIRS.
• #14553: The const version of NU_PLUGIN_DIRS is now populated by default instead of $env.NU_PLUGIN_DIRS.
• #14566: ENV_CONVERSIONS is now an empty record by default. The PATH/Path conversions are handled internally.
• #14579: More defaults are set in Rust. Also sets a default, empty TRANSIENT_PROMPT_COMMAND_RIGHT and TRANSIENT_PROMPT_MULTILINE_INDICATOR.

Breaking Startup Config Changes [toc]

There may be (hopefully) minor breaking changes due to the startup configuration handling. Known possible issues include:

• Cannot merge changes to default menus since their definitions are no longer in config.nu by default. Instead, the entire menu should be redefined.
• Querying the $env.NU_LIB_DIRS will return an empty result by default. Query the constant $NU_LIB_DIRS instead.

Changes [toc]

Additions [toc]

path self [toc]

Thanks to @Bahex in #14303, this release adds the path self command. path self is a parse-time only command for getting the absolute path of the source file containing it, or any file relative to the source file.

const this_file = path self
const this_directory = path self .

chunk-by [toc]

This release adds a new chunk-by command which will split a list into chunks based on a closure. The closure is applied to each element of the input list, and adjacent elements that share the same closure result value will be chunked together. This command was added in #14410 thanks to @cosineblast.

# chunk by a predicate
[1 3 -2 -2 0 1 2] | chunk-by {|x| $x >= 0 }
# [[1 3] [-2 -2] [0 1 2]]

# chunk duplicate, adjacent elements together
[a b b c a a] | chunk-by { $in }
# [[a] [b b] [c] [a a]]

term query [toc]

Thanks to @Bahex in #14427, this release adds the term query command. term query allows sending a query to your terminal emulator and reading the reply.

# Get cursor position
term query (ansi cursor_position) --prefix (ansi csi) --terminator 'R'

# Get terminal background color.
term query $'(ansi osc)10;?(ansi st)' --prefix $'(ansi osc)10;' --terminator (ansi st)

# Read clipboard content on terminals supporting OSC-52.
term query $'(ansi osc)52;c;?(ansi st)' --prefix $'(ansi osc)52;c;' --terminator (ansi st)

merge deep [toc]

To merge nested record structures we now have merge deep as a subcommand of merge which only operates on the flat structure of tables.

Here you can see how it is used to update a record with fields from a given record, by either adding the fields or replacing those leaf nodes that share a common cellpath.

{a: {foo: 123 bar: "overwrite me"}, b: [1, 2, 3]} | merge deep {a: {bar: 456, baz: 789}, b: [4, 5, 6]}
# => ╭───┬───────────────╮
# => │   │ ╭─────┬─────╮ │
# => │ a │ │ foo │ 123 │ │
# => │   │ │ bar │ 456 │ │
# => │   │ │ baz │ 789 │ │
# => │   │ ╰─────┴─────╯ │
# => │   │ ╭───┬───╮     │
# => │ b │ │ 0 │ 4 │     │
# => │   │ │ 1 │ 5 │     │
# => │   │ │ 2 │ 6 │     │
# => │   │ ╰───┴───╯     │
# => ╰───┴───────────────╯

While the record entries get updated based on the path, by default lists at a given path are updated by replacement.

merge deep also has different strategies for merging inner lists and tables. For example, you can use the append strategy to merge the inner b list instead of overwriting it.

{a: {foo: 123 bar: "overwrite me"}, b: [1, 2, 3]} | merge deep --strategy=append {a: {bar: 456, baz: 789}, b: [4, 5, 6]}
# => ╭───┬───────────────╮
# => │   │ ╭─────┬─────╮ │
# => │ a │ │ foo │ 123 │ │
# => │   │ │ bar │ 456 │ │
# => │   │ │ baz │ 789 │ │
# => │   │ ╰─────┴─────╯ │
# => │   │ ╭───┬───╮     │
# => │ b │ │ 0 │ 1 │     │
# => │   │ │ 1 │ 2 │     │
# => │   │ │ 2 │ 3 │     │
# => │   │ │ 3 │ 4 │     │
# => │   │ │ 4 │ 5 │     │
# => │   │ │ 5 │ 6 │     │
# => │   │ ╰───┴───╯     │
# => ╰───┴───────────────╯

This command is the work of a productive collaboration by @132ikl and @Bahex.

utouch [toc]

This release adds the new utouch command, utilizing uutils/coreutils! In addition to all the flags of touch, utouch also has a --timestamp and --date flag to specify the timestamp to use. Eventually, the utouch command will replace the touch command.

WASM support [toc]

Nushell used to have WASM support a while back, but at some point became incompatible with WASM. Thanks to the amazing work of @cptpiepmatz in #14418, Nushell can now be compiled to WASM again! In order to do so, certain (cargo) features have to be disabled meaning certain commands (like filesystem commands) will not be available.

sys net columns [toc]

Thanks to @rfaulhaber in #14389, the sys net command now has two additional columns. One is the mac column which lists mac address. The other is the ip column which lists the address(es) for each network interface.

Raw string pattern matching [toc]

With this release, raw strings can now be used as match patterns thanks to @sgvictorino in #14573.

match 'foo' {
    r#'foo'# => true
    _ => false
}

Duration/Date Arithmetic [toc]

With #14295, dates can now be added to durations. Previously only durations could be added to dates.

explore keybinds [toc]

In #14468 thanks to @paulie4, more default keybindings were added to explore to better match less.

Version in Startup banner [toc]

With #14625, the Nushell version now displays at startup in the banner.

input --default [toc]

Thanks to @WindSoilder in #14374, the input command now has a --default parameter to set a default value when none is provided in the input.

PowerShell script invocation on Windows [toc]

With #14379 by @fdncred we now launch PowerShell scripts with a .ps1 extension directly through powershell.exe. This behavior follows the same logic we offer for scripts launched by cmd.exe.

New introspection tools [toc]

@fdncred added a series of new tools permitting you to inspect internals of Nushell's engine for better understanding or debugging. We hope you can put them to good use to drill down into bugs or to learn more how the Nushell engine understands a piece of code or represents your current state. As they refer to the engine internals we still want to actively evolve their output or behavior may change as we continue development. So don't depend on them in scripts but feel free to use them to diagnose issues or learn about the behavior of the current version.

view source now supports internal IDs

To access the content of closures or other code view source can now accept an integer ID which represents the internal BlockId to access the code stored there. This is useful whenever our output provided you with <Closure {BlockID}>. For closures stored in a variable you can still access it directly via view source.

view blocks

This new commands lists the raw code blocks as stored in the engine after parsing including block_id, source code, and spans.

ast --flatten

The ast command can now also output the flattened and simplified representation of the AST used by syntax highlighting and some of the completion logic. This is accessible through the --flatten flag.

config flatten

To see the whole configuration in a simplified representation, especially to spot differences between different user configurations, config flatten provides a simplified view that is not nested like $env.config. Note that this output is not directly compatible with the config record and may not have the correct types.

Breaking changes [toc]

++ operator [toc]

The ++ operator previously performed both appending and concatenation.

# Appending
[1 2 3] ++ 4 == [1 2 3 4]

# Concatenation
[1 2 3] ++ [4 5 6] == [1 2 3 4 5 6]

This created ambiguity when operating on nested lists:

[[1 2] [3 4]] ++ [5 6]
# Should this be [[1 2] [3 4] [5 6]] or [[1 2] [3 4] 5 6] ?

Additionally, the ++= operator was able to change the type of a mutable a due to its dual role:

mut str: string = 'hello '
($str | describe) == string

$str ++= ['world']
($str | describe) == list<string>

After #14344, the ++ operator now only performs concatenation (between lists, strings, or binary values). To append a value to a list, either wrap the value in a list or use the append command.

mut list = [1 2]
$list ++= [3]
$list = $list | append 4

Stricter command signature parsing [toc]

Input/output types

Previous versions of Nushell would fail to parse input/output types on custom commands in some cases. However, a parse error would not be shown, making these silent parse errors.

# These input/output types are not valid, but no error would be shown:
def some_cmd [] -> string { '' }
def some_cmd [] : string { '' }

# Since the input/output types failed to parse, then this code would fail only at runtime:
(some_cmd) + 1

Thanks to @ratherforky, this has been fixed in #14510. Invalid input/output types will now cause an error to be shown at parse-time.

# The custom commands above will now cause a parse error and should instead be:
def some_cmd []: any -> string { '' }
def some_cmd [] : any -> string { '' }

# This will now fail at parse-time due to type checking, before any user code is run:
(some_cmd) + 1

Custom command arguments

This release also makes the parsing for custom command arguments more strict thanks to @sgvictorino in #14309. For example, a colon (:) without a corresponding type is now an error. Below are some more examples of code snippets which are now parse errors.

# expected parameter or flag
def foo [ bar: int: ] {}

# expected type
def foo [ bar: =  ] {}
def foo [ bar: ] {}

# expected default value
def foo [ bar = ] {}

group-by [toc]

Thanks to @Bahex in #14337, the group-by command now supports grouping by multiple criteria (henceforth referred to as a grouper). When using multiple groupers, the output is in the form of nested records.

let data = [
    [category color name];
    [fruit red apple]
    [fruit red strawberry]
    [vegetable red tomato]
    [fruit orange orange]
    [vegetable orange carrot]
    [vegetable orange pumpkin]
]

$data | group-by color category

╭────────┬───────────────────────────────────────────────────────╮
│        │ ╭───────────┬───────────────────────────────────────╮ │
│ red    │ │           │ ╭───┬──────────┬───────┬────────────╮ │ │
│        │ │ fruit     │ │ # │ category │ color │    name    │ │ │
│        │ │           │ ├───┼──────────┼───────┼────────────┤ │ │
│        │ │           │ │ 0 │ fruit    │ red   │ apple      │ │ │
│        │ │           │ │ 1 │ fruit    │ red   │ strawberry │ │ │
│        │ │           │ ╰───┴──────────┴───────┴────────────╯ │ │
│        │ │           │ ╭───┬───────────┬───────┬────────╮    │ │
│        │ │ vegetable │ │ # │ category  │ color │  name  │    │ │
│        │ │           │ ├───┼───────────┼───────┼────────┤    │ │
│        │ │           │ │ 0 │ vegetable │ red   │ tomato │    │ │
│        │ │           │ ╰───┴───────────┴───────┴────────╯    │ │
│        │ ╰───────────┴───────────────────────────────────────╯ │
│        │ ╭───────────┬──────────────────────────────────────╮  │
│ orange │ │           │ ╭───┬──────────┬────────┬────────╮   │  │
│        │ │ fruit     │ │ # │ category │ color  │  name  │   │  │
│        │ │           │ ├───┼──────────┼────────┼────────┤   │  │
│        │ │           │ │ 0 │ fruit    │ orange │ orange │   │  │
│        │ │           │ ╰───┴──────────┴────────┴────────╯   │  │
│        │ │           │ ╭───┬───────────┬────────┬─────────╮ │  │
│        │ │ vegetable │ │ # │ category  │ color  │  name   │ │  │
│        │ │           │ ├───┼───────────┼────────┼─────────┤ │  │
│        │ │           │ │ 0 │ vegetable │ orange │ carrot  │ │  │
│        │ │           │ │ 1 │ vegetable │ orange │ pumpkin │ │  │
│        │ │           │ ╰───┴───────────┴────────┴─────────╯ │  │
│        │ ╰───────────┴──────────────────────────────────────╯  │
╰────────┴───────────────────────────────────────────────────────╯

With the --to-table flag, instead of nested records, the output is in the form of a table with columns corresponding to each grouper and the items column. Each column corresponding to a grouper is named after it. For closure groupers, the columns are named with the scheme closure_{i}.

$data | group-by color category --to-table

╭───┬────────┬───────────┬───────────────────────────────────────╮
│ # │ color  │ category  │                 items                 │
├───┼────────┼───────────┼───────────────────────────────────────┤
│ 0 │ red    │ fruit     │ ╭───┬──────────┬───────┬────────────╮ │
│   │        │           │ │ # │ category │ color │    name    │ │
│   │        │           │ ├───┼──────────┼───────┼────────────┤ │
│   │        │           │ │ 0 │ fruit    │ red   │ apple      │ │
│   │        │           │ │ 1 │ fruit    │ red   │ strawberry │ │
│   │        │           │ ╰───┴──────────┴───────┴────────────╯ │
│ 1 │ red    │ vegetable │ ╭───┬───────────┬───────┬────────╮    │
│   │        │           │ │ # │ category  │ color │  name  │    │
│   │        │           │ ├───┼───────────┼───────┼────────┤    │
│   │        │           │ │ 0 │ vegetable │ red   │ tomato │    │
│   │        │           │ ╰───┴───────────┴───────┴────────╯    │
│ 2 │ orange │ fruit     │ ╭───┬──────────┬────────┬────────╮    │
│   │        │           │ │ # │ category │ color  │  name  │    │
│   │        │           │ ├───┼──────────┼────────┼────────┤    │
│   │        │           │ │ 0 │ fruit    │ orange │ orange │    │
│   │        │           │ ╰───┴──────────┴────────┴────────╯    │
│ 3 │ orange │ vegetable │ ╭───┬───────────┬────────┬─────────╮  │
│   │        │           │ │ # │ category  │ color  │  name   │  │
│   │        │           │ ├───┼───────────┼────────┼─────────┤  │
│   │        │           │ │ 0 │ vegetable │ orange │ carrot  │  │
│   │        │           │ │ 1 │ vegetable │ orange │ pumpkin │  │
│   │        │           │ ╰───┴───────────┴────────┴─────────╯  │
╰───┴────────┴───────────┴───────────────────────────────────────╯

timeit [toc]

The timeit command previously had a special behavior where expressions passed as arguments would have their evaluation deferred in order to later be timed. This lead to an interesting bug (14401) where the expression would be evaluated twice, since the new IR evaluator eagerly evaluates arguments passed to commands.

To make the deferred evaluation more explicit, the timeit command can now only take a closure as an argument instead of any expression or value (#14483). Additionally, blocks are no longer supported by timeit, so any changes to the environment will be isolated to inside the closure.

sys cpu [toc]

The cpu_usage column outputted by sys cpu works by sampling the CPU over a 400ms period. This wait long time is unhelpful if you are only interested in other information about the CPU like the number of cores (i.e., sys cpu | length). With #14485, the cpu_usage column is now gated behind the --long flag. This way, sys cpu will take around 0-2ms instead of 400ms by default.

from csv and from tsv [toc]

Thanks to @Bahex in #14399, parsing csv and tsv content with the --flexible flag is more flexible than before. Previously, the first row of csv or tsv content would determine the number of columns, and rows containing more values than the determined columns would be truncated, losing those extra values.

With this release, that is no longer the case. Now, --flexible flag means the number of columns aren't limited by the first row and can have not just less but also more values than the first row.

value
1,aaa
2,bbb
3
4,ddd
5,eee,extra

.. | from csv --flexible --noheaders

╭─#─┬─column0─┬─column1─┬─column2─╮
│ 0 │ value   │   ❎    │   ❎    │
│ 1 │       1 │ aaa     │   ❎    │
│ 2 │       2 │ bbb     │   ❎    │
│ 3 │       3 │   ❎    │   ❎    │
│ 4 │       4 │ ddd     │   ❎    │
│ 5 │       5 │ eee     │ extra   │
╰─#─┴─column0─┴─column1─┴─column2─╯

std/iter scan [toc]

Thanks to @Bahex in #14596, the order of scan's closure's parameters are flipped to be consistent with reduce. The closure now also receives the accumulator value as pipeline input as well.

> [a b c d] | iter scan "" {|x, y| [$x, $y] | str join} -n
# To keep its behavior same, this command should be changed to either of the following
> [a b c d] | iter scan "" {|it, acc| [$acc, $it] | str join} -n
> [a b c d] | iter scan "" {|it| append $it | str join} -n

Completion sorting [toc]

In #14424, some changes were made to completion sorting. If you have a custom completer that returns a record with an options field, then this may affect you.

• If options contains sort: true, then completions will be sorted according to $env.config.completions.sort. Previously, they would have been sorted in alphabetical order.
• If options contains sort: false, completions will not be sorted.
• If options does not have a sort column, then that will be treated as sort: true. Previously, this would have been treated as sort: false.

Import module naming [toc]

Thanks to @sgvictorino in #14353, modules with special characters in their name will be normalized by converting these special characters to underscores (_). Previously, it was not possible to use these modules after importing them.

table formatting and the display_output hook [toc]

With #14361 by @132ikl our machinery to format structured now follows a simpler logic. If a $env.config.hooks.display_output hook is set, it is fully responsible for formatting the structured data, e.g. by invoking table with custom settings. Only if it is not set by null will table without arguments be run by default. Previously the output logic would invoke table regardless on top of the formatting by your display_output hook. This avoids some spurious formatting attempts.

du flag changes [toc]

Thanks to @WindSoilder in #14407, the du command shows a more condensed output by default. The files and (recursive) directories are not shown by default. Use du --long (-l) to show these columns. The --all (-a) switch has been removed -- Files and directories will both be shown with --long (-l).

Code specific environment variables updated during source [toc]

The variables $env.CURRENT_FILE and $env.FILE_PWD had already been set to the respective file paths whenever a script was executed or module loaded. Similarly $env.PROCESS_PATH is set for the execution of a script or module but unset when a module is simply used.

With #14486 by @fdncred $env.CURRENT_FILE and $env.FILE_PWD will now also be updated when a particular file is included via source. Similarly $env.PROCESS_PATH will be unset.

# test_source.nu
print $"$env.CURRENT_FILE = ($env.CURRENT_FILE?)"
print $"$env.FILE_PWD = ($env.FILE_PWD?)"
print $"$env.PROCESS_PATH = ($env.PROCESS_PATH?)"

source test_source.nu
# => $env.CURRENT_FILE = /Users/fdncred/src/nushell/test_source.nu
# => $env.FILE_PWD = /Users/fdncred/src/nushell
# => $env.PROCESS_PATH =

Deprecations [toc]

split-by [toc]

In #14019, the split-by command was deprecated. Instead, please use group-by with multiple groupers as shown above.

date to-record and date to-table [toc]

In #14319:

• date to-record has been deprecated and will be removed in a future release. Please use into record in its place.
• date to-table has been deprecated and will be removed in a future release. Please use into record | transpose | transpose -r in its place.

do --ignore-shell-errors and --ignore-program-errors [toc]

Thanks to @WindSoilder in #14385, --ignore-shell-errors and --ignore-program-errors for the do command have been deprecated. Use --ignore-errors (-i) instead.

Removals [toc]

NU_DISABLE_IR [toc]

With #14293, the NU_DISABLE_IR environment variable is no longer used. Nushell will now always use the new IR evaluator instead of previous AST based evaluator.

Bug fixes and other changes [toc]

ls [toc]

The ls command used to have deterministic output order, but this was broken in 0.94.0. Thanks to @userwiths in #13875, ls output will now be sorted by the name column. Future changes to the order of ls will be a breaking change.

Additionally, thanks to @sgvictorino in #14310, ls will now error if there are insufficient permissions to list the current working directory. Previously, ls would return empty output.

SHLVL [toc]

When starting Nushell as an interactive REPL, Nushell will now increment the SHLVL environment variable. This change was made in #14404 thanks to @rikukiix. (Note that it is a known issue that SHLVL is still incremented on exec.)

from commands [toc]

After #14602, the from commands now remove the content_type pipeline metadata thanks to @Bahex.

Completions on custom commands [toc]

Thanks to @RobbingDaHood in #14481, file completions are now triggered once again on custom commands after the first parameter. This was a regression due to the 0.99.x release.

seq char [toc]

Thanks to @anomius in #14261, seq char now works on any ASCII characters instead of only alphabetical ASCII characters.

http multipart [toc]

The http commands now use CRLF when joining headers to match the HTTP specification. This change was made in #14417 thanks to @Beinsezii.

scope variables [toc]

scope variables now lists constant variables in scope (when using the IR evaluator) thanks to [@sgvictorino](https://github.com/sgvictorino in #14577.

help system [toc]

@132ikl fixed a long standing issue in the help system, where commands with the same name but different module paths clashed. Now the help <name> correctly refers to the name of a command like it is visible in the scope through the definitions or use imports thanks to #14490.

Notes for plugin developers [toc]

As part of cleaning up we removed several enum variants actually unused in the core codebase:

• Type::ListStream as the engine represents PipelineData::ListStream more consistently as Type::List
• FlatShape::And/FlatShape::Or as they are never populated.

Furthermore Value::Filesize now stores its value in its own Filesize type instead of a bare i64 to avoid confusion about the particular unit.

Hall of fame [toc]

Thanks to all the contributors below for helping us solve issues, improve documentation, refactor code, and more! 🙏

Full changelog [toc]