Nushell
Get Nu!
Getting Started
  • The Nushell Book
  • Command Reference
  • Cookbook
  • Language Reference Guide
  • Contributing Guide
Blog
  • English
  • 中文
  • Deutsch
  • Français
  • Español
  • 日本語
  • Português do Brasil
  • Русский язык
  • 한국어
GitHub
Get Nu!
Getting Started
  • The Nushell Book
  • Command Reference
  • Cookbook
  • Language Reference Guide
  • Contributing Guide
Blog
  • English
  • 中文
  • Deutsch
  • Français
  • Español
  • 日本語
  • Português do Brasil
  • Русский язык
  • 한국어
GitHub
  • Introduction
  • Installation
    • Default Shell
  • Getting Started
    • Quick Tour
    • Moving Around the System
    • Thinking in Nu
    • Nushell Cheat Sheet
  • Nu Fundamentals
    • Types of Data
    • Loading Data
    • Data Formats
    • Working with SQLite
    • Pipelines
    • Working with Strings
    • Regular Expressions
    • Working with Binary Data
    • Working with Lists
    • Working with Records
    • Working with Numbers
    • Working with Tables
    • Working with Paths
    • Navigating and Accessing Structured Data
    • Special Variables
  • Programming in Nu
    • Custom Commands
    • Aliases
    • Operators
    • Variables
    • Control Flow
    • Scripts
    • Modules
      • Using Modules
      • Creating Modules
    • Overlays
    • Sorting
    • Testing your Nushell Code
    • Debugging Scripts
    • Language Server
    • Best Practices
  • Nu as a Shell
    • Configuration
    • Configuration Reference
    • Environment
    • Stdout, Stderr, and Exit Codes
    • Working with the Terminal
    • Running System (External) Commands
    • How to Configure 3rd Party Prompts
    • Directory Stack
    • Reedline, Nu's Line Editor
    • Custom Completions
    • Externs
    • Coloring and Theming in Nu
    • Hooks
    • Background Jobs
  • Coming to Nu
    • Coming from Bash
    • Coming from CMD.EXE
    • Coming from PowerShell
    • Nu map from other shells and domain specific languages
    • Nu Map from Imperative Languages
    • Nu Map from Functional Languages
    • Nushell operator map
  • Design Notes
    • How Nushell Code Gets Run
  • (Not So) Advanced
    • Standard Library
      • std/assert
      • std/bench
      • std/clip
      • std/config
      • std/dt
      • std/formats
      • std/help
      • std/input
      • std/iter
      • std/log
      • std/math
      • std/random
      • std/util
      • std/xml
      • std-rfc
    • Dataframes
      • Dataframe Operations
      • Column Selectors
      • Math Expressions
      • String Expressions
      • Dates and Times in Dataframes
      • Masks and Conditionals
      • Aggregations and Window Functions
    • Metadata
    • Creating Your Own Errors
    • Parallelism
    • Plugins
    • explore
    • Building TUIs with tui
    • MCP Server

String Expressions

The Working with strings section of the Dataframes chapter covers polars str-lengths. This page covers the other Polars commands for string columns. They change case, search and replace with regular expressions, cut strings apart, join them together, and parse numbers out of them.

  • Expressions and one-column dataframes
  • Changing case
  • Searching with regular expressions
  • Replacing text
  • Slicing
  • Splitting
  • Stripping characters
  • Joining strings
  • Parsing numbers

Expressions and one-column dataframes

Each command on this page builds an expression. Most of them take a column expression as input, as in polars col name | polars uppercase. You use the expression in polars select, polars with-column, polars filter, or polars agg, as the chapter's Lazy Dataframes section explains. Some of the commands also work on a dataframe with a single string column and return a dataframe:

CommandAlso takes a one-column dataframe
polars uppercase, polars lowercaseyes
polars containsyes
polars str-replace, polars str-replace-allyes
polars str-sliceyes
polars str-joinyes, joined with another dataframe
polars str-split, polars str-strip-charsno
polars concat-strno, it ignores its input
polars integer, polars decimalno

On a dataframe with more than one column, these commands fail with Error using as series, so take one column first with polars get. They also take a lazy dataframe, but they collect it and return an eager dataframe. The result keeps the column's name.

polars str-split and polars str-strip-chars also take a selector as input. With the other commands, pick several string columns with polars col --type str.

A string command used on a column of another type fails when you collect the result, with an error such as invalid series dtype: expected `String`, got `i64`. On a one-column dataframe it fails straight away. Null values stay null.

The examples use this dataframe:

let people = [[name city]; ["Ada Lovelace" London] ["Grace Hopper" "New York"] ["Émile Zola" Paris]] | polars into-df

Changing case

polars uppercase and polars lowercase follow Unicode's case rules, so É and é convert into each other and ß becomes SS:

$people | polars select (polars col name | polars uppercase) (polars col city | polars lowercase) | polars collect
# => ╭───┬──────────────┬──────────╮
# => │ # │     name     │   city   │
# => ├───┼──────────────┼──────────┤
# => │ 0 │ ADA LOVELACE │ london   │
# => │ 1 │ GRACE HOPPER │ new york │
# => │ 2 │ ÉMILE ZOLA   │ paris    │
# => ╰───┴──────────────┴──────────╯

Searching with regular expressions

polars contains returns true for each string that a regular expression matches somewhere. The pattern isn't anchored, so add ^ or $ to match at the start or end, and (?i) makes it ignore case. Use it with polars filter to keep the matching rows:

$people | polars filter (polars col city | polars contains '(?i)york|paris') | polars collect
# => ╭───┬──────────────┬──────────╮
# => │ # │     name     │   city   │
# => ├───┼──────────────┼──────────┤
# => │ 0 │ Grace Hopper │ New York │
# => │ 1 │ Émile Zola   │ Paris    │
# => ╰───┴──────────────┴──────────╯

On a one-column dataframe it returns a boolean column, which works as a mask for polars filter-with:

$people | polars get name | polars contains '^[A-G]'
# => ╭───┬───────╮
# => │ # │ name  │
# => ├───┼───────┤
# => │ 0 │ true  │
# => │ 1 │ true  │
# => │ 2 │ false │
# => ╰───┴───────╯

A pattern that isn't a valid regular expression doesn't cause an error. Every result is null instead, so polars contains '(' matches nothing. To match a character that has a meaning in regular expressions, escape it with a backslash, as in '\(', or put it in brackets, as in '[(]'. Single quotes keep Nushell from reading the backslash itself.

Replacing text

polars str-replace replaces the first match of a regular expression in each string, and polars str-replace-all replaces every match. Both need --pattern (-p) and --replace (-r). Leaving either one out fails with Missing required flag:

let phones = [[phone]; ["555-123-4567"] ["555.987.6543"] ["(555) 000 1111"]] | polars into-df
$phones | polars select [
    (polars col phone)
    (polars col phone | polars str-replace -p '\D' -r '' | polars as first)
    (polars col phone | polars str-replace-all -p '\D' -r '' | polars as all)
] | polars collect
# => ╭───┬────────────────┬───────────────┬────────────╮
# => │ # │     phone      │     first     │    all     │
# => ├───┼────────────────┼───────────────┼────────────┤
# => │ 0 │ 555-123-4567   │ 555123-4567   │ 5551234567 │
# => │ 1 │ 555.987.6543   │ 555987.6543   │ 5559876543 │
# => │ 2 │ (555) 000 1111 │ 555) 000 1111 │ 5550001111 │
# => ╰───┴────────────────┴───────────────┴────────────╯

The replacement can use the pattern's groups. $1 stands for the text the first group matched, and $$ for a literal $. Write ${1} when letters, digits, or an underscore come right after the group number. A string in double quotes doesn't treat $ specially in Nushell, so "$1" reaches Polars unchanged:

$phones | polars select (polars col phone | polars str-replace -p '(\d{3})-(\d{3})' -r '$2-$1') | polars collect
# => ╭───┬────────────────╮
# => │ # │     phone      │
# => ├───┼────────────────┤
# => │ 0 │ 123-555-4567   │
# => │ 1 │ 555.987.6543   │
# => │ 2 │ (555) 000 1111 │
# => ╰───┴────────────────╯

Unlike polars contains, both commands fail on an invalid pattern, with regex parse error.

Slicing

polars str-slice takes a start position and an optional --length (-l), and counts both in characters, not bytes. A negative start counts back from the end of each string. Without --length the slice runs to the end of the string:

let codes = [[code]; ["GB-001"] ["US-042"] ["Zürich-7"]] | polars into-df
$codes | polars select [
    (polars col code)
    (polars col code | polars str-slice 0 --length 2 | polars as first2)
    (polars col code | polars str-slice -3 | polars as last3)
] | polars collect
# => ╭───┬──────────┬────────┬───────╮
# => │ # │   code   │ first2 │ last3 │
# => ├───┼──────────┼────────┼───────┤
# => │ 0 │ GB-001   │ GB     │ 001   │
# => │ 1 │ US-042   │ US     │ 042   │
# => │ 2 │ Zürich-7 │ Zü     │ h-7   │
# => ╰───┴──────────┴────────┴───────╯

A start past the end of a string gives an empty string. A slice that begins before the start of the string keeps only the part inside it, so polars str-slice -8 --length 3 gives G for GB-001. In an expression, --length can't be negative. A negative length fails on collect with conversion from `i32` to `u64` failed.

Splitting

polars str-split splits each string at a separator and returns a column of type list<str>. The separator is literal text, not a regular expression, so polars str-split "." splits at dots. An empty separator splits a string into its characters. To get one row per piece, follow it with polars explode, which Exploding list columns describes:

let posts = [[id tags]; [1 "nu,shell"] [2 "polars"] [3 "nu,data,polars"]] | polars into-df
$posts | polars with-column (polars col tags | polars str-split ",") | polars explode tags | polars collect
# => ╭───┬────┬────────╮
# => │ # │ id │  tags  │
# => ├───┼────┼────────┤
# => │ 0 │  1 │ nu     │
# => │ 1 │  1 │ shell  │
# => │ 2 │  2 │ polars │
# => │ 3 │  3 │ nu     │
# => │ 4 │  3 │ data   │
# => │ 5 │  3 │ polars │
# => ╰───┴────┴────────╯

The separator can also be an expression, which gives each row its own separator:

[[text sep]; ["a,b" ","] ["x|y|z" "|"]] | polars into-df | polars select (polars col text | polars str-split (polars col sep)) | polars explode text | polars collect
# => ╭───┬──────╮
# => │ # │ text │
# => ├───┼──────┤
# => │ 0 │ a    │
# => │ 1 │ b    │
# => │ 2 │ x    │
# => │ 3 │ y    │
# => │ 4 │ z    │
# => ╰───┴──────╯

Stripping characters

polars str-strip-chars removes characters from both ends of each string. Its argument is a set of characters, not a word. Every character in the set is stripped, in any order, until a character outside the set comes up. --start (-s) strips only the start and --end (-e) only the end. If you pass both, it strips only the start. There is no default set, so to strip whitespace, list the whitespace characters, as in polars str-strip-chars " \t\n". A string made only of characters in the set becomes empty:

[[raw]; ["--ab-c-ba--"] ["b-a-bab"]] | polars into-df | polars select [
    (polars col raw)
    (polars col raw | polars str-strip-chars "-ab" | polars as both)
    (polars col raw | polars str-strip-chars --start "-ab" | polars as start)
    (polars col raw | polars str-strip-chars --end "-ab" | polars as end)
] | polars collect
# => ╭───┬─────────────┬──────┬────────┬────────╮
# => │ # │     raw     │ both │ start  │  end   │
# => ├───┼─────────────┼──────┼────────┼────────┤
# => │ 0 │ --ab-c-ba-- │ c    │ c-ba-- │ --ab-c │
# => │ 1 │ b-a-bab     │      │        │        │
# => ╰───┴─────────────┴──────┴────────┴────────╯

The argument can also be an expression, such as a column holding a different set for each row. A number fails on collect, because it's an integer and not a string, so write "1" rather than 1.

Joining strings

polars concat-str joins values across columns, one result per row. It takes a separator and a list of columns. A string in the list is a column name, so to add fixed text, wrap it in polars lit. Numbers in the list have to be wrapped the same way, because a plain 3 fails with can't convert int to expression. Columns of other types are converted to text. The command ignores its input, so it only makes sense inside polars select or polars with-column:

let born = [[first last year]; [Ada Lovelace 1815] [Grace Hopper 1906] [Linus null 1969]] | polars into-df
$born | polars select (polars concat-str "" [(polars col last | polars uppercase) (polars lit ", b. ") year] | polars as label) | polars collect
# => ╭───┬───────────────────╮
# => │ # │       label       │
# => ├───┼───────────────────┤
# => │ 0 │ LOVELACE, b. 1815 │
# => │ 1 │ HOPPER, b. 1906   │
# => │ 2 │                   │
# => ╰───┴───────────────────╯

If any value in a row is null, the result for that row is null, as the third row shows. The result is named after the first column in the list, so name it with polars as.

polars str-join works down a column. As an expression, it joins all the strings in the column into one, with --delimiter (-d) between them. The delimiter defaults to nothing. A null value makes the whole result null, unless you pass --ignore-nulls (-n), which skips nulls. In polars agg it joins each group:

let teams = [[team name]; [a Ada] [b Grace] [a Linus] [b null]] | polars into-df
$teams | polars group-by team | polars agg (polars col name | polars str-join -d ", " --ignore-nulls) | polars sort-by team | polars collect
# => ╭───┬──────┬────────────╮
# => │ # │ team │    name    │
# => ├───┼──────┼────────────┤
# => │ 0 │ a    │ Ada, Linus │
# => │ 1 │ b    │ Grace      │
# => ╰───┴──────┴────────────╯

Given a one-column dataframe, polars str-join does something else. It takes another one-column string dataframe as its argument and joins the two row by row, with nothing in between:

let first = [Ada Grace Linus] | polars into-df | polars rename "0" first
let last = [" Lovelace" " Hopper" " Torvalds"] | polars into-df
$first | polars str-join $last
# => ╭───┬────────────────╮
# => │ # │     first      │
# => ├───┼────────────────┤
# => │ 0 │ Ada Lovelace   │
# => │ 1 │ Grace Hopper   │
# => │ 2 │ Linus Torvalds │
# => ╰───┴────────────────╯

In this form --delimiter and --ignore-nulls don't apply, and a null on either side gives null. The two dataframes need the same number of rows, unless the argument has one row, which is then joined to every row. Without the argument the command fails with Missing parameter: other, and a column that isn't a string fails with an error saying that polars str-join works only with string columns.

Parsing numbers

polars integer parses strings as integers. A string it can't parse becomes null. That includes decimals, strings with spaces, and prefixes such as 0x. --strict (-s) makes collecting fail instead, with a message that lists the bad values, such as strict integer parsing failed for 1 value(s): ["x"].

The first optional argument is the base, from 2 to 36, which defaults to 10. It can be an expression, which gives each row its own base. The second optional argument is the result type, such as u8 or i32, and it defaults to i64. A type goes after the base, so polars integer u8 reads u8 as the base and fails, while polars integer 10 u8 works. A value that doesn't fit the type becomes null, or fails with --strict:

[[s base]; ["42" 10] ["ff" 16] ["101" 2] ["300" 10] ["x" 10]] | polars into-df | polars select [
    (polars col s)
    (polars col s | polars integer | polars as base10)
    (polars col s | polars integer (polars col base) | polars as per_row)
    (polars col s | polars integer 10 u8 | polars as as_u8)
] | polars collect
# => ╭───┬─────┬────────┬─────────┬───────╮
# => │ # │  s  │ base10 │ per_row │ as_u8 │
# => ├───┼─────┼────────┼─────────┼───────┤
# => │ 0 │ 42  │     42 │      42 │    42 │
# => │ 1 │ ff  │        │     255 │       │
# => │ 2 │ 101 │    101 │       5 │   101 │
# => │ 3 │ 300 │    300 │     300 │       │
# => │ 4 │ x   │        │         │       │
# => ╰───┴─────┴────────┴─────────┴───────╯

polars decimal parses strings as decimal numbers. Its argument sets how many digits to keep after the decimal point. Extra digits are rounded to the nearest value, and a value exactly halfway rounds to an even last digit. The result is then converted to f64, so the column's type is f64, not a decimal type. Strings it can't parse, including ones with spaces or a comma as the decimal mark, become null. Scientific notation such as 1e3 works:

[[s]; ["2.456"] ["2.45"] ["2.55"] ["1e3"] ["1,5"]] | polars into-df | polars select [
    (polars col s)
    (polars col s | polars decimal 1 | polars as one_digit)
] | polars collect
# => ╭───┬───────┬───────────╮
# => │ # │   s   │ one_digit │
# => ├───┼───────┼───────────┤
# => │ 0 │ 2.456 │      2.50 │
# => │ 1 │ 2.45  │      2.40 │
# => │ 2 │ 2.55  │      2.60 │
# => │ 3 │ 1e3   │   1000.00 │
# => │ 4 │ 1,5   │           │
# => ╰───┴───────┴───────────╯

A negative argument fails with Negative value passed when positive one is required.

Edit this page on GitHub
Contributors: fdncred
Prev
Math Expressions
Next
Dates and Times in Dataframes