Nushell 0.74

Nushell, or Nu for short, is a new shell that takes a modern, structured approach to your command line. It works seamlessly with the data from your filesystem, operating system, and a growing number of file formats to make it easy to build powerful command line pipelines.

Today, we're releasing version 0.74 of Nu. This release includes improvements on handling signatures of exec and known externals, improved help, initial support for parse-time constants, new commands, and many improvements to our existing commands.

Where to get it

Nu 0.74 is available as pre-built binaries or from crates.io. If you have Rust installed you can install it using cargo install nu.

NOTE: The optional dataframe functionality is available by cargo install nu --features=dataframe.

As part of this release, we also publish a set of optional plugins you can install and use with Nu. To install, use cargo install nu_plugin_<plugin name>.

Themes of this release / New features

merelymyself, WindSoilder, kubouch)

A common pitfall in Nushell when defining custom signatures using extern used to be that unrecognized arguments passed to the command would throw an error. Now, arguments are still checked against the extern signature but those that are not recognized are simply ignored.

> extern `git checkout` []
> git checkout -b foo  # does not throw an error

exec uses similar style which fixes errors with tools like ssh and gdb that internally invoke exec.

> nu -c 'exec ls -l'  # does not throw an error

kubouch)

For a long time, Nushell had the option to provide custom help messages for commands via comments:

# Some Command
#
# More description goes here
def some-command [] { 'foo' }

help some-command  # displays the comments in the help message

In this release, we allow user-defined help messages with aliases and modules. This goes hand-in-hand with a handful of new help subcommands to explore these help messages.

The help messages now also treat the first line followed by an empty line as a "brief" description displayed in summary tables generated by help commands, $nu.scope.aliases, etc. The full description is available when querying a particular command/alias/module (e.g., help spam). This brief vs. full separation was already present for built-in commands, like path, but now it is applied also to all user-defined help messages.

An example of using custom module and alias help messages:

The current batch of improvements can still be taken further. For example, custom help messages could possibly be defined also for variables and environment variables (via comments adjacent to let and let-env). We could also further improve the presentation of existing help xxx commands.

kubouch)

This is a proof-of-concept that we plan to expand in the future.

Tip: We also added a new book section with an in-depth explanation of Nushell's parsing and evaluation, hopefully clearing up some confusion about things like "Why can't I source a dynamic path?". It also touches on the concept of parse-time constants.

A new const keyword is added to Nushell to define "parse-time" constants. Constants defined with const behave the same as variables defined with let, but in addition, they are usable in some contexts that require values known at parse-time. Currently, this applies to files or names passed to use, overlay use, source, and source-env. For example, the following now works:

# Source a file from a constant
> 'print hello!' | save --raw say_hello.nu
> const fname = 'say_hello.nu'
> source $fname
hello!

# Load an overlay from a constant name:
> module spam {
	export def foo [] { 'foo' }
}
> const name = 'spam'
> const new_name = 'eggs'
> overlay use $name as $new_name
> overlay list | last
eggs
> foo
foo

Only a limited subset of values is allowed to be a constant. In general, "simple" values, like strings or integers, and their collections (lists, records) are allowed but values requiring some evaluation (string interpolation, subexpressions, environment variables) are not allowed. The current selection is not set in stone, however, and might change in the future.

Some future direction we can take this:

• Move parts of $nu to be constant to allow things like source $nu.config-path
• Allow modules to have constants (module spam { const CONTENTS = [ 'eggs', 'bacon', 'sausage', 'spam' ] })
• Some limited form of parse-time evaluation to allow static control flow, for example

const fname = const if $nu.os-info.name == 'windows' {
    'C:\Users\viking\spam.nu'
} else {
    '/home/viking/spam.nu'
}
overlay use $fname

In general, we want to be very conservative with parse-time constants and evaluation because it can be easy to introduce awkward side-effects and performance pitfalls. We plan to extend this only where it brings some tangible benefit to Nushell's user experience.

MehulG)

To encode text that is used as a path component in a URL we now provide url encode.

By default it preserves the structure of a URL and only replaces invalid characters. With --all the whole string gets encoded.

> 'https://example.com/foo bar' | url encode
https://example.com/foo%20bar
> 'https://example.com/foo bar' | url encode --all
https%3A%2F%2Fexample%2Ecom%2Ffoo%20bar

webbedspace)

This is a complement to columns, designed to allow the values of a record to be easily filtered and iterated over using the standard list tools like each and where. The name values is derived from similar functions in Ruby, Python and JavaScript.

>  {a: "Happy", b: "new", c: "year"} | values
╭───┬───────╮
│ 0 │ Happy │
│ 1 │ new   │
│ 2 │ year  │
╰───┴───────╯

It can also operate on tables to convert them to lists-of-lists:

>  [[a b]; [4 7] [5 8] [6 9]] | values
╭───┬───────────╮
│ 0 │ ╭───┬───╮ │
│   │ │ 0 │ 4 │ │
│   │ │ 1 │ 5 │ │
│   │ │ 2 │ 6 │ │
│   │ ╰───┴───╯ │
│ 1 │ ╭───┬───╮ │
│   │ │ 0 │ 7 │ │
│   │ │ 1 │ 8 │ │
│   │ │ 2 │ 9 │ │
│   │ ╰───┴───╯ │
╰───┴───────────╯

kubouch, webbedspace)

Consider the following operations performed on a table:

• [{foo: 'bar'}, {}] | select foo
• [{foo: 'bar'}, {}] | get foo
• [{foo: 'bar'}, {}].foo

Formerly, this would produce ['bar', null] - converting the table hole into a null. Now, however, they will produce an error. The original null-conversion behaviour can, as usual, be opted into using the -i flag for get and select: [{foo: 'bar'}, {}] | get -i foo produces ['bar', null]. (There are also plans for a future version of Nushell to expand the cell path syntax to allow certain cell names to be "nullable" - converted to null if they don't exist.)

Behavior of -i/--ignore-errors flag for get and select when the entire column is absent has changed

Formerly, if select -i or get -i couldn't find any value for the given column, it would produce a single null:

〉[{a:1} {b:2} {a:3}] | select -i foo | to nuon
null

This has been changed so that it now produces a table (or, in the case of get, a list) of all nulls:

〉[{a:1} {b:2} {a:3}] | select -i foo | to nuon
[[foo]; [null], [null], [null]]

This change was made to make this flag work more consistently with default and compact:

〉[{a:1} {b:2} {a:3}] | select -i a | default 0 a
[[a]; [1], [0], [3]]
〉[{a:1} {b:2} {a:3}] | select -i foo | default 0 foo
[[foo]; [0], [0], [0]]

As you can see, default in the above example can reliably populate an entire table column when some or all of the values don't exist.

webbedspace, webbedspace)

The following misuses of punctuation in def definitions now produce errors:

• Placing a comma between a long flag and its short alternative (e.g. def a [--foo, (-f)] {})
• Consecutive commas, like def a [foo,,bar] {}
• Consecutive colons, like def a [foo::int] {}
• Using ^ in command names, like def ^a [] {}

$in now works in catch closures

$in in catch closures now behaves identically to how it does in other closures, like those given to each (it's equivalent to what would be the first named argument).

try { 'x' | math abs } catch { print $in } behaves the same as try { 'x' | math abs } catch {|e| print $e }.

fdncred)

To find out what application your operating system associates with a particular file, you can now use the --mime-type or -m flag on our ls command. This simplifies filtering for particular files and can help you dispatch files to particular programs. When opening files with Nushell directly, open will still follow the same heuristics using file endings and the built-in from ... command parsers.

rgwood)

The primary motivation for this is to make regex and =~ operator uses in hooks and color_config closures more performant.

sholderbach)

A few releases back commands internally got the capability to declare not only the types of parameters but also pairs for the input and output on the pipeline. With this release we finally declare those input and output types for all core nushell commands. This can help you as user to see what a command expects from the pipeline and might return. We are exploring how nushell can leverage that for more useful diagnostics and completions. In the future we may introduce syntax for user-defined commands to declare their input and output types explicitly.

Breaking changes

• Nushell 0.74
• Where to get it
• Themes of this release / New features
  • Known externals commands and exec now have "fall-through" signatures (merelymyself, WindSoilder, kubouch)
  • help is now more helpful (kubouch)
  • Initial support for parse-time constants (kubouch)
  • New url encode command to percent-encode URLs (MehulG)
  • values command to programmatically interact with records (webbedspace)
  • get, select, cell path access on tables will now error when encountering a hole (kubouch, webbedspace)
  • Behavior of -i/--ignore-errors flag for get and select when the entire column is absent has changed
  • Certain misused punctuation in def definitions are now errors (webbedspace, webbedspace)
  • $in now works in catch closures
  • MIME-types are supported in ls with an additional flag. (fdncred)
  • Regular expression queries are cached for performance (rgwood)
  • All built-in commands now declare their pipeline input and output types (sholderbach)
• Breaking changes
• Full changelog
  • Nushell
  • Documentation
  • Nu_Scripts
  • reedline

Full changelog

Nushell

• sholderbach created Commit the lockfile for 0.74, and Bump version to 0.74.0, and [Chore] cleanup in where implementation, and Try to use the latest tagged virtualenv, and Fix quoting of empty string in to nuon, and Update Cargo.lock to powierza-coefficient 1.0.2, and Fix the syntax highlighting in help metadata, and Fix usage of deprecated C-style logical and, and Include clippy check for dataframe in CI, and Further cleanup of Span::test_data usage + span fixes, and Fix && quotation in to nuon after proptest fail, and Bump to new development version 0.73.1, and Fix line-end trimming in subexpression, and Add more input/output type annotations, and Remove math eval command
• WindSoilder created Auto-Completion: put ` tildes around filenames with parentheses, and revert changes on prepend and append, and last, skip, drop, take until, take while, skip until, skip while, where, reverse, shuffle, append, prepend and sort-by raise error when given non-lists , and fix python plugin example, and avoid panic when using from nuon
• rgwood created Revert "Primitives now use color closures...", and Add benchmarks for evaluating default env+config, and Add Criterion benchmarks for parser, and Upgrade all remaining crates to Rust 2021, and Reorder flags in nu --help, and Clarify url base command, and Delete unused files, and Tweak new input type error message, and Make stream info visible to users in describe, and Fix error message when interrupting table with ctrl+c, and Add LRU regex cache, and Re-enable test_bits, and Make to text stream ListStreams, and Add some cell path tests, and Add cross-rs config, and Clarify --stdin flag, and table: Check stream timeout on every item
• stormasm created clean up some extra logging code in the cli, and move BufferedReader out of nu-command
• webbedspace created Improve wording of str replace help messages, and Primitives now use color closures when printed on the command line, and Make get hole errors and cell path hole errors identical (improvement on #7002), and Slight edits to ls and zip's help text, and def: make various punctuation misuses into errors, and Disallow ^ in def command names, and Add extra_usage messages for subcommand-only commands, and columns now errors when given a non-record non-table, and Change instances of Value::string("foo", Span::test_data()) to Value::test_string("foo"), and Add values command (see #7166), and Fix signatures of commands which accept records also, and Change other instances of $nothing to null, and Fix #7551 record support in color_config, and Remove preview.rs, and Make $in work in catch closures, and Make config.filesize_format/config.filesize_metric conflict resolution consistent, and Standardise the use of ShellError::UnsupportedInput and ShellError::TypeMismatch and add spans to every instance of the former
• dependabot[bot] created Bump tokio from 1.21.2 to 1.24.1
• fdncred created align durations to the right, and fix register-plugins script, and add --mime-type(-m) to ls in the type column, and add metadata to wrap, and fix the wix file to overwrite with save -f
• kubouch created Remove environment variable hiding from hide, and Expand Nushell's help system, and Remove shape-directed import pattern parsing, and Small parser refactors, and Add "fall-through" signatures, and Initial support for parse-time constants
• Mehrbod2002 created Continue and Break on Try/Catch , and Recursion def, and Fix and Allow Number and Boolean type to be key in from yaml, and Fix #6888 and rename fill-na to fill-nan
• hikilaka created Make user parameter optional in fetch
• xxnuo created Fix build-all-windows.cmd
• aniou created Extend config support from F1-F12 to F1-F20, #7666
• MehulG created Url encode to escape special characters
• TornaxO7 created adding link to list of nu-plugins
• Decodetalkers created fix: ci problem, and fix: empty cell in select, and return Error if get meet nothing and without "i"
• Yethal created Simplify register-plugins.nu
• jaudiger created doc: correct some really tiny typos.
• micouy created Update powierza-coefficient to 1.0.2
• Kangaxx-0 created early return for parsing closure and block with interchanged shape
• hustcer created Fix const examples, and Fix an example of env command
• NotLebedev created Fix save error handling, and To toml fix
• zhiburt created Fix table expand wrap in case no header is there, and Fix some issues with table wrapping of lists
• kianmeng created Fix typos by codespell
• merelymyself created let start open anything and everything, and let case_insensitive option work for variable completion as well, and tighter restrictions on alias and def names
• swarnimarun created chore: make the config setup messages consistent
• raccmonteiro created uniq and uniq-by optimization (#7477)

Documentation

• kubouch created Link to the new parse-eval page, and Start Design Notes chapter; Add parse-eval notes, and Fix wrong PATH append example, and Add image; Fmt
• WindSoilder created update breaking changes, and Try to rename from blog text to changeLog
• sholderbach created Fix #705: sidebar links German., and Fix styling on 0.73 release notes
• hustcer created fix i18n.nu script and update i18n-meta conf, and Refresh docs for v0.73, and Fix book/make_docs.nu for nu v0.73+, and Update book/make_docs.nu for nu v0.72.1+
• webbedspace created Add my changes, and Fixed various issues in make_docs.nu
• TornaxO7 created Update plugins.md, and Update custom_completions.md, and Update aliases.md, and Fix syntax error for menu
• Tengs-Penkwe created Add section "Output result to external commands"
• sschneider-ihre-pvs created Update help.md
• merelymyself created clarify columns and rows
• rgwood created Mention breaking save change

Nu_Scripts

• kubouch created Fix broken conda prompt
• fdncred created update coloring scripts with newer syntax, and update 20k script with save changes, and update sparkline script
• skelly37 created Fix dict and cdpath for 0.73
• maxim-uvarov created Function to format nicely big numbers: number-format.nu

reedline

• sholderbach created Fix the Github actions badge
• CozyPenguin created Make reedline handling cursor shapes more configurable