DataFrame

Value: used at row boundaries only (from_rows / to_rows / with_column). Int/Float columns are backed by NativeArray (flat, contiguous storage); String/Bool columns by TypedArray. Sum/Mean aggregation reads the NativeArray directly (native_int_arr_sum/native_float_arr_sum — the same primitives auto-vectorize in --compile builds; see stdlib/native_array.march). Most other per-column operations (column-column arithmetic, ColExpr Add/Sub/Mul/Div, fill_null) still round-trip through List(Value)/List(Float) rather than staying in NativeArray form throughout — correct, just not yet on the fast path; see specs/optimizations.md P10 for the current state.

Column: typed columnar storage. Each variant holds (name, data [, null_bitmap]). Data is stored as a contiguous TypedArray for O(1) random access and cache efficiency. Null bitmap: parallel TypedArray(Bool) where true = that row is null.

Row: a snapshot of one row, as an ordered assoc list. Slow path only.

DataFrame: an ordered list of equal-length typed columns.

Internal builder: accumulates values during row-by-row construction.

ColExpr: a typed expression evaluated against a DataFrame column-by-column. Used for filter predicates. Evaluation is vectorized - no Row materialization.

Aggregation expression for use with GroupedFrame.agg.

Structural group key: a list of typed Values, one per group column.

WindowExpr: window function expressions applied over an entire DataFrame (no partitioning in Phase 9; partitioned windows are a future extension).

Appends a column at the end. Errors if a column with that name already exists, or if the frame has rows and the column's length differs. A frame with zero rows (including one with no columns) accepts a column of any length.

Aggregates a GroupedFrame: one output row per distinct key, holding the group columns followed by one column per expression. Group (row) order is unspecified (it follows the internal hash map). Err if a group column is missing; an empty frame gives the empty frame, and an empty group_cols list puts every row in one group.

• Count gives an Int; CountDistinct(c) an Int (null is one value);

`First(c)`/`Last(c)` keep the column's values.

• Sum, Mean, Min, Max, Median, Std, Variance give Floats

(also for Int columns). `Std` and `Variance` are sample statistics
(n - 1); `Variance` of one row is 0.0 and `Std` of one row is null.

• An aggregate over a missing, non-numeric or nullable column is null.

Output columns are rebuilt with widening inference, so such nulls read
back as the column type's default (e.g. 0.0) unless every group is
null.

Output names: the source column ("count" for Count), or the alias of AggAs(expr, name). When that name is already taken by a group column or an earlier output, a suffix is added: _sum, _mean, _min, _max, _count, _n_distinct, _std, _variance, _first, _last, _median (or _ plus the alias for AggAs).

Examples

DataFrame.group_by(sales, ["region"])
  |> DataFrame.agg([DataFrame.Count,
                    DataFrame.Sum("amount"),
                    DataFrame.AggAs(DataFrame.Mean("amount"), "avg")])
-- Ok(columns: region, count, amount, avg; one row per region)

The unsuffixed output name of an aggregate: its source column, count for Count, or the alias of AggAs.

The output column name for expr: agg_base_name, plus _ and agg_type_suffix when the base name is already in existing_names.

The disambiguating suffix of an aggregate (sum, mean, n_distinct, ...; the alias for AggAs).

Keep the rows of df where expr evaluates to true; Err if it fails or is not a Bool column.

Group df by group_cols and evaluate agg_exprs per group (see agg).

Inner join of two materialized frames (see inner_join): hash the right keys, probe with each left row, rebuild from rows.

Dispatch a join of two materialized frames on on_cols to the implementation for kind (see inner_join and friends).

Left join of two materialized frames (see left_join): hash the right keys, probe with each left row.

Full outer join of two materialized frames (see outer_join). Finding the unmatched right rows is O(right rows x matches).

Right join of two materialized frames (see right_join): hash the left keys, probe with each right row.

Sort all rows by keys (see sort_by for the ordering and its caveats). Always Ok.

Compute f(row) for every row, type the results with values_to_column, drop any existing column name and append the new one.

The column name a ColumnBuilder is accumulating.

Push val into builder b without type conversion (used by from_rows). Returns Err when val's type differs from the builder's, when a NullVal arrives in a typed builder, or when a non-null value arrives in a builder that has only seen nulls (nullable inference is not supported in strict mode).

Push val into builder b, widening the builder's type when needed (used by the CSV/JSON readers, group_by, joins, with_column and friends). Widening order is Bool < Int < Float < String: e.g. a Float arriving in an Int builder converts the earlier Ints to Float, and a String arriving in a numeric builder stringifies the earlier values. A Bool after Int or Float widens to String; Int/Float after Bool converts the Bools to 1/0.

Nulls are not preserved once a type is known: a NullVal pushed into a typed builder stores the type's default (0, 0.0, false; a String builder stores ""), and nulls seen before the first typed value are backfilled with that default.

Finalize a builder into a non-nullable column of its type, in push order. A builder that only ever saw nulls becomes an all-null NullableStrCol.

• Int + Int -> IntCol (named after col1)
• Float + Float, Int + Float, Float + Int -> FloatCol (named after col1)

Errors if the columns have different lengths or are non-numeric (nullable numeric columns count as non-numeric).

Examples

let a   = DataFrame.IntCol("a", native_int_arr_from_list([1, 2, 3]))
let b   = DataFrame.IntCol("b", native_int_arr_from_list([10, 20, 30]))
let sum = DataFrame.col_add_col(a, b)  -- Ok(IntCol("a", [11, 22, 33]))

Returns a FloatCol with the same name. Errors on other column types (including nullable numeric columns).

Examples

let c  = DataFrame.FloatCol("price", native_float_arr_from_list([1.0, 2.0, 3.0]))
let c2 = DataFrame.col_add_float(c, 10.0)  -- Ok(FloatCol("price", [11.0, 12.0, 13.0]))

Returns the number of columns.

Returns one ColStats(name, type, count, mean, std, min, p25, median, p75, max) per column, in column order. type is "Int", "Float", "String", "Bool" or "Nullable" (any nullable column) and count is the row count. The statistics are Some only for a non-empty IntCol/FloatCol (std is the sample standard deviation, 0.0 for a single row); every other column, and a zero-row numeric one, gets None for all of them.

The ColStats of one column (see col_describe).

Whether row i of col is null (always false for a non-nullable column). i must be in range.

Returns true if the column is a nullable variant.

Returns the length (number of rows) of a column.

Returns a FloatCol with the same name. Errors on other column types (including nullable numeric columns).

Examples

let c  = DataFrame.IntCol("qty", native_int_arr_from_list([1, 2, 3]))
let c2 = DataFrame.col_mul_float(c, 2.5)  -- Ok(FloatCol("qty", [2.5, 5.0, 7.5]))

Returns the name of a column.

Min and max of an IntCol/FloatCol, computed directly over the column's NativeArray via native_{float,int}_arr_min/max (tight vectorizable C loops, see runtime/march_runtime.c) instead of materializing a List(Float) and folding, as float_list-based callers do. An earlier version of this helper used a March-level recursive index loop over native__arr_get, which looked allocation-free but wasn't: get()'s Float/Int return value is boxed at each call site under the generic call convention, so it paid the same O(n) boxing cost as float_list — the true fix has to stay inside a single C-level reduction, exactly like col_native_sum's native__arr_sum. Err on an empty column (matches Stats.min_safe/max_safe on Nil).

(count, ssd) of an IntCol/FloatCol, where ssd is the sum of squared deviations from the mean; (0, 0.0) for an empty column, Err for a missing or non-numeric (including nullable) column. Computed via two native C-level reduction passes (native__arr_sum for the mean, then native__arr_sumsq_dev(arr, mean) for the deviations) instead of materializing a List(Float) and folding; see col_native_min_max's doc for why a March-level index loop can't do this allocation-free. This is the standard two-pass variance algorithm rather than Stats' single-pass Welford, so results may differ in the last bit or two but are equally stable (no catastrophic cancellation) for any realistic column. Sample variance is ssd / (count - 1); sample std dev is its square root.

Sum and element count of an IntCol/FloatCol, computed directly over the column's NativeArray (native_int_arr_sum/native_float_arr_sum — already vectorized in compiled builds) instead of materializing a List(Float) and folding, as float_list-based callers do. Sum(0.0) on an empty column matches Stats.sum's fold_left-over-Nil behavior.

Min-max normalize an IntCol or FloatCol to [0, 1]: (x - min) / (max - min). Returns a FloatCol of the same name; a column whose values are all equal gives all 0.0. Err for any other column type, including nullable numeric columns.

A zero-row column returns a zero-row FloatCol, for the same reason as col_z_score above: it is the n = 0 instance of the zero-range case below.

Examples

DataFrame.col_normalize(DataFrame.IntCol("x", native_int_arr_from_list([10, 15, 20])))
-- Ok(FloatCol("x", [0.0, 0.5, 1.0]))

Returns the number of null entries in a nullable column (0 for non-nullable).

Wrap a non-nullable column in a nullable variant (all-false bitmap). No-op on already-nullable columns.

All values of col, in row order, as List(Value) (nulls as NullVal). O(n).

The value at row i of col as a Value (NullVal for a null slot of a nullable column). O(1). i must be in 0 .. col_len(col) - 1; out of range indices are not checked here and fail in the underlying array get.

The value at row i of the named column, or NullVal if there is no such column.

Z-score normalize an IntCol or FloatCol: (x - mean) / std with the sample standard deviation. Returns a FloatCol of the same name; a column whose values are all equal gives all 0.0. Err for any other column type, including nullable numeric columns.

A zero-row column returns a zero-row FloatCol rather than an error: it is the n = 0 instance of the zero-variance case just below, which already returns a same-length all-zeros column, and head(df, 0) or a filter that matches nothing produces one in ordinary use. The guard is on the list actually passed to Stats.*, which declare {List(Float) | len(_) > 0} and panic otherwise.

Examples

DataFrame.col_z_score(DataFrame.IntCol("x", native_int_arr_from_list([1, 2, 3])))
-- Ok(FloatCol("x", [-1.0, 0.0, 1.0]))

Runs the lazy plan (see eval_plan) and returns the resulting DataFrame, or the first error any node produced.

Compare rows i and j of df key by key with compare_values, negating for Desc; the first non-zero result wins, 0 if all keys tie. A missing key column reads as NullVal for both rows (a tie).

Three-way comparison used for sorting and ColExpr comparisons: -1, 0 or

1. NullVal sorts before every other value. Int and Float compare

numerically with each other; strings compare lexicographically; false sorts before true. Any other mixed-type pair (e.g. Int vs String) returns 1 in both directions, so it is not a consistent order.

Wrap cell in quote (doubling any inner quote) when it contains delim, quote or a newline; otherwise return it unchanged.

Join cells with delim, quoting each as needed (csv_quote_cell).

One CSV line (without header) per row of df, cells rendered with value_to_string and quoted by csv_quote_cell.

Default CSV write options: comma delimiter, double-quote quoting, with a header line.

Drops columns by name; names that are not present are ignored.

Removes every column with the given name. No-op if not found.

Remove rows that have a null in any column. Only nullable columns can hold nulls; the columns keep their (nullable) types.

Examples

DataFrame.drop_nulls(joined)   -- e.g. the matched rows of a left_join

Remove rows that have a null in any of the named columns; names that are not present are ignored.

Drops the first n rows (all rows if n exceeds the row count, none if n <= 0). Used by the Offset plan node.

Returns an empty DataFrame with no rows and no columns.

Evaluate one aggregate expression on a group's sub-frame (see agg for each aggregate's result). The output-name argument is unused.

Evaluate every aggregate on one group's sub-frame, returning (output name, value) pairs in expression order (names from agg_output_name).

Evaluate a ColExpr against a DataFrame, producing a typed Column with one entry per row. Literals are broadcast to the frame's row count.

• Comparisons (Eq, Lt, ...) use compare_values and give a BoolCol

(so Int and Float compare numerically, and nulls are smallest).

• And/Or/Not need (non-nullable) Bool operands, else Err.
• Add/Sub/Mul give an IntCol, or a FloatCol if either side has a

Float; `Div` always gives a FloatCol (Int division is not truncated).
Null or non-numeric operands count as 0 (as 1 for a `Div` divisor).

• StrContains/StrStartsWith/StrEndsWith need a (non-nullable)

String column, else `Err`.

• IsNull/IsNotNull read the null bitmap; a non-nullable column has

no nulls.

Err when a referenced column is missing.

Interpret a Plan tree bottom-up, returning the first error any node produces. Limit/Offset use head/drop_rows, Select uses select_columns, Rename uses rename_column, DropCols ignores missing names, and Join evaluates both sides then calls apply_join.

Replace the nulls of a nullable column with fill_val, returning the NON-nullable column of the same type and name. The value must match the column type (an IntVal is also accepted for a Float column). Errors on a type mismatch, and on a non-nullable column (even though it has nothing to fill).

Examples

DataFrame.fill_null(score_col, DataFrame.FloatVal(0.0))
-- Ok(FloatCol("score", ...)) with every null replaced by 0.0

Backward-fill nulls in a nullable column: each null takes the next non-null value below it. Returns the non-nullable column of the same type; trailing nulls (no value below) become 0, 0.0, "" or false. Non-nullable columns are returned unchanged.

Apply fill_null to the named column of a DataFrame, replacing it in place (same position). Errors if the column is missing or fill_null fails.

Forward-fill nulls in a nullable column: each null takes the last non-null value above it. Returns the non-nullable column of the same type; leading nulls (no value above) become 0, 0.0, "" or false. Non-nullable columns are returned unchanged.

Appends a Filter node: keep the rows where the vectorized ColExpr predicate is true (see eval_col_expr). At collect, Err if the expression references a missing column or does not produce a Bool column. Nulls compare as smaller than every value, so Lt(Col(c), x) keeps null rows of c; use IsNotNull to exclude them.

A new column (same name and variant) holding the rows at indices, in the order given; indices may repeat. O(len(indices)). Every index must be in range.

Keep the given row indices (in that order) in every column of df.

A new column keeping the rows where mask is true (the null bitmap of a nullable column is filtered alongside). mask must have the column's length.

Returns a column as List(Float) for use with Stats functions. IntCol values are promoted; any other column type (including nullable numeric columns) is an Err.

Constructs a DataFrame from a list of pre-built columns, in order. All columns must have equal length; returns Err naming the first column whose length differs from the first column's. Column names are not checked for duplicates. An empty list gives Ok(empty()).

Parse a CSV string into a DataFrame. The first non-blank line is the header; blank lines are skipped. Cells are split on commas and trimmed (no quote handling, see split_csv_line), then typed per cell by parse_cell and per column with widening (from_rows_widen): Int, then Float, then String; true/false (any of three casings) are Bools. Empty cells are not kept as nulls: they read back as 0, 0.0, false or "" unless the whole column is empty.

A row with more cells than the header is truncated to the header's width; a row with fewer cells is an Err. An input with no non-blank line gives Ok(empty()).

Examples

DataFrame.from_csv_string("id,name,score\n1,ann,9.5\n2,bob,7")
-- Ok: IntCol "id" [1, 2], StrCol "name" ["ann", "bob"],
--     FloatCol "score" [9.5, 7.0]

Parse a JSON array of objects into a DataFrame, one row per object. Column names and order come from the first object; later objects are matched by POSITION (key order), not by key, and must have the same number of keys. Integral numbers become Int and others Float (widened per column), strings String, booleans Bool; nested arrays/objects are stored as their JSON text. JSON nulls are not preserved (see from_rows_widen). Err on invalid JSON, a non-array top level or a non-object element; [] gives the empty frame.

Examples

DataFrame.from_json_string("[{\"a\": 1, \"b\": \"x\"}, {\"a\": 2.5, \"b\": \"y\"}]")
-- Ok: FloatCol "a" [1.0, 2.5], StrCol "b" ["x", "y"]

Constructs a DataFrame from a list of Row values. Column names and types come from the first row; later rows are matched to columns by POSITION, not by name (their field names are ignored), and must have the same number of fields (else Err).

Type inference is strict: if a column starts as Int, every later value must also be IntVal (no Int-to-Float promotion). NullVal is rejected in a typed column, and a column whose first value is NullVal may only hold nulls (it becomes an all-null NullableStrCol). An empty row list gives the empty frame. For widening inference (Int, Float, String) use from_rows_widen.

Examples

DataFrame.from_rows([
  DataFrame.Row([("a", DataFrame.IntVal(1)), ("b", DataFrame.StrVal("x"))]),
  DataFrame.Row([("a", DataFrame.IntVal(2)), ("b", DataFrame.StrVal("y"))])
])
-- Ok(frame with IntCol "a" = [1, 2] and StrCol "b" = ["x", "y"])
DataFrame.from_rows([
  DataFrame.Row([("a", DataFrame.IntVal(1))]),
  DataFrame.Row([("a", DataFrame.FloatVal(1.0))])
])
-- Err("from_rows: column 'a' is Int but got Float")

Like from_rows (names from the first row, later rows matched by position, Err on a field-count mismatch) but with widening type inference via builder_push_widen: mixed Int/Float columns become Float, anything mixed with a String becomes String. Nulls are NOT preserved: a null in a typed column is stored as 0, 0.0, false or "", and only a column that is null in every row stays null. Used by the CSV/JSON readers, group_by, joins, melt, pivot and summarize.

Returns the data of a named BoolCol as a list, or Err if the column is missing or is not a (non-nullable) BoolCol.

Returns the first column with the given name, or an Err naming the missing column.

Returns the data of a named FloatCol as a list, or Err if the column is missing or is not a (non-nullable) FloatCol. Int columns are not promoted (see float_list).

Returns the data of a named IntCol as a list, or Err if the column is missing or is not a (non-nullable) IntCol; a NullableIntCol is rejected too.

Materializes row i as a Row of (column name, value) pairs in column order. O(col_count). Slow path. i must be in 0 .. row_count(df) - 1.

Returns the data of a named StrCol as a list, or Err if the column is missing or is not a (non-nullable) StrCol.

Groups a DataFrame by the specified columns. Nothing is computed until agg; missing columns are reported there. Rows whose key values are equal by values_equal share a group (nulls group together).

Group keys are equal when their value lists are (values_list_eq).

FNV-style combination of the value_hash of each key value, masked to 31 bits.

Returns the first n rows (all rows if n exceeds the row count; none if n <= 0).

One output row per matching (left row, right row) pair: all left columns followed by the right frame's non-key columns (right key columns are not duplicated; a non-key column name present on both sides appears twice). Keys match with values_equal, so an Int key never matches a Float key. Output rows follow left-row order; several matches for one left row come in reverse right-row order. The result is rebuilt row by row with widening inference, so nulls in nullable columns are not preserved, and a join with no matching rows yields a frame with no columns. Err at collect if a key column is missing on either side.

Examples

let lf     = DataFrame.lazy(orders_df)
             |> DataFrame.inner_join(products_df, ["product_id"])
let result = DataFrame.collect(lf)
-- Only orders that reference a known product_id are included

Convert parsed JSON array elements to a frame (see from_json_string); Err if any element is not an object.

Convert one JSON value to a cell Value: null, Bool, integral number as IntVal (else FloatVal), string; arrays and objects become StrVal of their JSON text.

Wraps a materialized DataFrame in a lazy frame. Nothing runs until collect; the plan nodes then execute in the order they were added.

Examples

DataFrame.lazy(df)
  |> DataFrame.filter(DataFrame.Gt(DataFrame.Col("age"), DataFrame.LitInt(30)))
  |> DataFrame.sort_by([("name", DataFrame.Asc)])
  |> DataFrame.select(["name", "age"])
  |> DataFrame.limit(10)
  |> DataFrame.collect
-- Ok(at most 10 rows of name and age, for age > 30, sorted by name)

Rows in the left frame that have no match in the right frame are still included once, with all right-only columns set to Null; those columns always come back as NullableXxxCol. Left columns keep their types. Output follows left-row order (several matches for one left row in reverse right-row order). Right key columns are not duplicated. Err at collect if a key column is missing on either side.

Examples

let lf   = DataFrame.lazy(orders_df)
           |> DataFrame.left_join(customers_df, ["customer_id"])
let result = DataFrame.collect(lf)
-- Every order row is present; customer_name is Null for unknown customers

Limits the result to the first n rows (see head).

An empty ColumnBuilder named name whose type is that of val (a NullVal gives an all-null builder).

Construct a DataFrame directly from a list of columns, in order. An empty list gives the empty frame. Panics if any column has a different length from the first: ragged frames silently corrupt every downstream aggregation, so we fail fast instead.

Use from_columns for the Result-returning version when columns may come from user input and the caller wants to handle the error.

Examples

let df = DataFrame.make_df([
  DataFrame.IntCol("id", native_int_arr_from_list([1, 2, 3])),
  DataFrame.StrCol("name", typed_array_from_list(["ann", "bob", "cy"]))
])
DataFrame.schema(df)     -- ["id", "name"]
DataFrame.row_count(df)  -- 3

The GroupKey of row row_idx: the values of group_cols in that row (NullVal for a missing column).

id_vars - columns kept as-is in every output row (in the frame's column order; names that are not present are skipped). value_vars - columns whose names become values of var_col and whose values become values of val_col (a missing one contributes nulls, which widening turns into the type's default). One output row is produced per (input row x value_var), row-major. The value column is typed with widening inference across all value_vars.

Examples

melt(df, ["id"], ["jan","feb","mar"], "month", "value")
-- turns:  id | jan | feb | mar
-- into:   id | month | value   (3 output rows per original row)

Skips the first n rows (see drop_rows).

The union of left_join and right_join: the left_join rows (in left-row order) followed by the right rows that matched nothing (in right-row order). Every column comes back nullable: all left columns, then the right non-key columns. The key columns come from the left side only, so for a right-only row the KEY columns are Null too (its key values are lost). Err at collect if a key column is missing on either side.

Examples

let lf   = DataFrame.lazy(employees_df)
           |> DataFrame.outer_join(departments_df, ["dept_id"])
let result = DataFrame.collect(lf)
-- All employees and all departments appear; Nulls where there is no match

Type one CSV cell: "" is NullVal, then Int, then Float (as accepted by string_to_int / string_to_float), then true/True/TRUE and false/False/FALSE as Bool, and anything else as a String.

index_col - column whose distinct values become output rows. cols_col - column whose distinct values become new output column names (via value_to_string). vals_col - column whose values fill the output cells. Rows and new columns appear in first-appearance order. When several input rows share an (index, col) pair, the first one wins. Missing (index, col) combinations are null before typing, but the output is rebuilt with widening inference, so they read back as the column type's default (0, 0.0, "", false) unless the whole column is missing. Errors if any of the three columns is missing. O(index values x new columns x rows).

Examples

pivot(df, "region", "product", "revenue")
-- turns:  region | product | revenue
-- into:   region | A | B | ...   (one column per distinct product)

Print a DataFrame to stdout as a simple text table: the column names joined with " | ", a dashed rule as wide as that header, then one line per row (cells via value_to_string). Columns are not padded or aligned.

Renames a column in the pipeline. Err at collect if old_name is missing or new_name already exists.

The same column (data and null bitmap shared) under a new name.

Renames a column. Errors if old_name is not found or new_name already exists (including new_name == old_name).

Reorder col by an index permutation; same as filter_col_by_indices.

The mirror image of left_join. Every row in right appears in the output (in right-row order); rows with no match in the left frame get Null for every left-only column. The output columns are the left frame's non-key columns (as NullableXxxCol) followed by ALL right columns, including the keys. Err at collect if a key column is missing on either side.

Examples

let lf   = DataFrame.lazy(transactions_df)
           |> DataFrame.right_join(reference_df, ["code"])
let result = DataFrame.collect(lf)
-- Every reference row is included; transaction cols are Null for unmatched codes

Returns the number of rows (the first column's length; 0 for a frame with no columns). O(1).

Look up a field in a Row by name (the first field with that name). Returns None if not found; a null field is Some(NullVal).

Look up a Bool field in a Row. None if the field is missing, null or not a BoolVal.

Look up a Float field in a Row. Int values are promoted to Float. None if the field is missing, null or not numeric.

Look up an Int field in a Row. None if the field is missing, null or not an IntVal.

Look up a String field in a Row. None if the field is missing, null or not a StrVal.

Selects rows 0, step, 2*step, ... with step = row_count / n (integer division), so the sample is spread across the frame but biased towards its start. Returns the full DataFrame unchanged when n >= row_count(df), and zero rows (same columns) when n <= 0.

Examples

let df  = DataFrame.make_df([DataFrame.IntCol("v", native_int_arr_from_list([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]))])
let s   = DataFrame.sample(df, 3)
-- picks rows 0, 3, 6  ->  IntCol("v", [0, 3, 6])

Returns the ordered list of column names ([] for the empty frame).

Appends a projection to the named columns, in the given order (select_columns). Err at collect on a missing name.

Returns a DataFrame with exactly the named columns, in the order given (a name listed twice gives the column twice). Errors on the first missing name.

Returns rows [start, start + len), clamped to the available rows: a negative start counts as 0, and a len reaching past the end stops at the last row. A non-positive len gives zero rows.

A new column of the len rows starting at start (no clamping: the range must be in bounds). Zero rows when len <= 0.

Appends a sort node. keys is a list of (column_name, SortDir) pairs; later keys break ties of earlier ones. Values order as in compare_values (nulls first under Asc, last under Desc). The sort is not stable: rows that tie on every key come out in reverse input order. A key naming a missing column is not an error; it compares every row as equal.

Sort (Int, Int) pairs by their first element, ascending (used to restore row order after ranking).

Split a CSV line on delim and trim whitespace around each cell. Quotes are NOT interpreted: a quoted field containing the delimiter is split, and the quote characters are kept in the cell.

Columns in the result: "column", "type", "count", "mean", "std", "min", "p25", "median", "p75", "max" (see col_describe). The table is rebuilt with widening inference, so the missing statistics of a non-numeric or nullable column read back as 0.0 (not null) whenever the frame also has a non-empty numeric column; they are null only when no column has statistics.

Examples

let df = DataFrame.make_df([DataFrame.IntCol("x", native_int_arr_from_list([1, 2, 3, 4, 5]))])
let d  = DataFrame.summarize(df)
-- d has 1 row: column="x", type="Int", count=5, mean=3.0, ...

Returns the last n rows (all rows if n exceeds the row count; none if n <= 0).

Serialize a DataFrame to a CSV string with default_csv_opts(): a header line, then one line per row, joined with \n (no trailing newline). Values are written with value_to_string (nulls as empty cells) and a cell containing a comma, a double quote or a newline is quoted with inner quotes doubled. Note that from_csv_string does not undo that quoting.

Examples

DataFrame.to_csv_string(df)   -- "id,name\n1,ann\n2,bob"

Serialize a DataFrame to CSV with custom options CsvWriteOpts(delimiter, quote_char, write_header); see to_csv_string for the format. Cells containing the delimiter, the quote string or a newline are quoted.

Examples

DataFrame.to_csv_string_opts(df, DataFrame.CsvWriteOpts(";", "'", false))
-- "1;ann\n2;bob"

Render a DataFrame as an HTML table string: a <caption> with the dimensions ("N rows x M cols"), a header row of column names and one <tr> per row (cells via value_to_string). Note: header and cell text is only partially escaped; just the first occurrence of each of &, <, > and " is replaced, so do not feed it untrusted text. The output is detected automatically by the March notebook and displayed as a styled table rather than raw text.

Serialize a DataFrame to a JSON string: an array with one object per row, keys in column order. Nulls become JSON null; Ints are written as JSON numbers (so they read back as Int).

Examples

DataFrame.to_json_string(df)   -- "[{\"id\":1,\"name\":\"ann\"},{\"id\":2,\"name\":\"bob\"}]"

Returns all rows, in order, as a list of Row values. O(rows x cols). Slow path.

ratio is the fraction of rows placed in the training set (0.0 < ratio < 1.0). The first float_to_int(row_count * ratio) rows (truncated) become the training set; the remainder become the test set. Row order is preserved. A ratio >= 1.0 puts every row in train, one <= 0.0 every row in test.

Examples

let (train, test) = DataFrame.train_test_split(df, 0.8)
-- 80% of rows -> train, 20% -> test

Count the frequency of each distinct value of a column. Returns a DataFrame with a String column named col (each value's value_to_string form, so a null and "" count together, as do 1 and "1") and an Int column "count", sorted by count descending. Err if the column is missing; an empty column gives the empty frame.

Examples

DataFrame.value_counts(df, "city")
-- Ok: city ["paris", "rome"], count [3, 1]

A hash of a value used for group-by / join keys: the Int itself, the Float scaled by 1e6 and truncated, djb2 over a string's characters,

1. Equal values hash equally; not a

cryptographic hash.

Convert a cell value to JSON: numbers (Int via Float), string, bool, NullVal as null.

The display form of a value, as used by CSV output, to_html, print_table and value_counts: ints and floats via int_to_string / float_to_string, strings unchanged, "true" / "false", and "" for NullVal (so a null and an empty string print the same).

The type name of a value (Int, Float, String, Bool or Null), for error messages.

Structural equality of two values. Values of different variants are never equal (so IntVal(1) is not equal to FloatVal(1.0)); NullVal equals NullVal.

Element-wise values_equal of two value lists; lists of different lengths are unequal.

Type a list of values as a column named name using widening inference (builder_push_widen, so nulls become defaults unless every value is null). An empty list gives an empty StrCol. Always Ok.

a new column named out_col. No partitioning (operates over all rows). Errors if out_col already exists.

• RowNum: 0-based row number (IntCol).
• Rank(c, dir) / DenseRank(c, dir): 1-based rank of each row when

sorted by `c` (ties share a rank; `Rank` then skips, 1,2,2,4, while
`DenseRank` does not, 1,2,2,3). A missing column ranks every row 1.

• RunningSum(c): cumulative sum (IntCol for Int input, FloatCol for

Float). `RunningMean(c)`: cumulative mean (FloatCol). Both error on a
missing or non-numeric (including nullable) column.

• Lag(c, n, fill) / Lead(c, n, fill): the value n rows before /

after, or `fill` past the edge; typed with widening inference, so a
`fill` of another type widens the column. O(rows^2). Panics if `n` is
negative and the shifted row falls outside the frame.

Examples

DataFrame.window(df, DataFrame.Rank("score", DataFrame.Desc), "rank")
DataFrame.window(df, DataFrame.Lag("price", 1, DataFrame.FloatVal(0.0)), "prev_price")

Appends a node that computes a column row by row with a March closure over each Row and adds it at the END of the frame, replacing (and moving) any existing column of that name. The results are typed with widening inference (builder_push_widen), so NullVal results do not produce a nullable column. Slow path: materializes every row.

Examples

DataFrame.lazy(df)
  |> DataFrame.with_column("double", fn row ->
       DataFrame.IntVal(Option.unwrap_or(DataFrame.row_get_int(row, "id"), 0) * 2))
  |> DataFrame.collect