DataFrame

Adds a column to a DataFrame. Errors if name already exists or length mismatches.

Aggregates a GroupedFrame using the given expressions. Returns a new DataFrame.

• Int + Int → IntCol (named after col1)
• Float + Float, Int + Float, Float + Int → FloatCol (named after col1)

Errors if the columns have different lengths or are non-numeric.

Example:
  let a   = IntCol("a", [1, 2, 3])
  let b   = IntCol("b", [10, 20, 30])
  let sum = DataFrame.col_add_col(a, b)  -- Ok(IntCol("a", [11, 22, 33]))

Returns a FloatCol. Errors on other column types.

Example:
  let c  = FloatCol("price", [1.0, 2.0, 3.0])
  let c2 = DataFrame.col_add_float(c, 10.0)  -- Ok(FloatCol("price", [11.0, 12.0, 13.0]))

Returns the number of columns.

Returns summary statistics for each column.

Returns true if the column is a nullable variant.

Returns the length (number of rows) of a column.

Returns a FloatCol. Errors on other column types.

Example:
  let c  = IntCol("qty", [1, 2, 3])
  let c2 = DataFrame.col_mul_float(c, 2.5)  -- Ok(FloatCol("qty", [2.5, 5.0, 7.5]))

Returns the name of a column.

Min and max of an IntCol/FloatCol, computed directly over the column's NativeArray via native_{float,int}_arr_min/max (tight vectorizable C loops, see runtime/march_runtime.c) instead of materializing a List(Float) and folding, as float_list-based callers do. An earlier version of this helper used a March-level recursive index loop over native__arr_get, which looked allocation-free but wasn't: get()'s Float/Int return value is boxed at each call site under the generic call convention, so it paid the same O(n) boxing cost as float_list — the true fix has to stay inside a single C-level reduction, exactly like col_native_sum's native__arr_sum. Err on an empty column (matches Stats.min_safe/max_safe on Nil).

(count, mean, sum-of-squared-deviations) of an IntCol/FloatCol, computed via two native C-level reduction passes (native__arr_sum for the mean, then native__arr_sumsq_dev(arr, mean) for the deviations) instead of materializing a List(Float) and folding — see col_native_min_max's doc for why a March-level index loop can't do this allocation-free. This is the standard two-pass variance algorithm rather than Stats' single-pass Welford, so results may differ in the last bit or two but are equally stable (no catastrophic cancellation) for any realistic column. Sample variance is ssd / (count - 1); sample std dev is its square root.

Sum and element count of an IntCol/FloatCol, computed directly over the column's NativeArray (native_int_arr_sum/native_float_arr_sum — already vectorized in compiled builds) instead of materializing a List(Float) and folding, as float_list-based callers do. Sum(0.0) on an empty column matches Stats.sum's fold_left-over-Nil behavior.

Min-max normalize a column to [0, 1]. Returns FloatCol.

Returns the number of null entries in a nullable column (0 for non-nullable).

Wrap a non-nullable column in a nullable variant (all-false bitmap). No-op on already-nullable columns.

Z-score normalize an IntCol or FloatCol. Returns FloatCol.

Materializes the lazy plan into a DataFrame.

Default CSV write options: comma delimiter, double-quote, with header.

Drops columns by name.

Removes a column by name. No-op if not found.

Remove rows that have a null in any column.

Remove rows that have a null in any of the specified columns.

Returns an empty DataFrame with no rows and no columns.

Replace null values in a nullable column with fill_val. Errors on type mismatch.

Backward-fill nulls in a nullable column: propagate the next non-null value upward.

Apply fill_null to a named column in a DataFrame, replacing it in-place.

Forward-fill nulls in a nullable column: propagate the last non-null value downward.

Appends a Filter node that uses a vectorized ColExpr predicate.

Returns a column as List(Float) for use with Stats functions. Int columns are promoted.

Constructs a DataFrame from a list of pre-built columns. All columns must have equal length. Returns Err if they don't.

Parse a CSV string into a DataFrame. Infers column types with widening.

Parse a JSON string (array of objects) into a DataFrame.

Constructs a DataFrame from a list of Row values. Type inference is strict: if a column starts as Int, every subsequent value must also be IntVal. Errors on type mismatch or NullVal in Phase 1. For widening inference (Int→Float→String), use from_rows_widen.

Returns the Bool data list for a named column, or Err.

Returns the column with the given name, or Err if not found.

Returns the Float data list for a named column, or Err.

Returns the Int data list for a named column, or Err.

Materializes row i as a Row snapshot. O(col_count × i). Slow path.

Returns the String data list for a named column, or Err.

Groups a DataFrame by the specified columns.

Returns the first n rows.

Only rows where every key column matches in both left and right are included in the output. Right key columns are not duplicated.

Example:
  let lf     = DataFrame.lazy(orders_df)
               |> DataFrame.inner_join(products_df, ["product_id"])
  let result = DataFrame.collect(lf)
  -- Only orders that reference a known product_id are included

Wraps a materialized DataFrame in a lazy frame.

Rows in the left frame that have no match in the right frame are still included, with all right-only columns set to Null (NullableXxxCol). Right columns that are join keys are not duplicated.

Example:
  let lf   = DataFrame.lazy(orders_df)
             |> DataFrame.left_join(customers_df, ["customer_id"])
  let result = DataFrame.collect(lf)
  -- Every order row is present; customer_name is Null for unknown customers

Limits the result to the first n rows.

Construct a DataFrame directly from a list of columns. Panics if any column has a different length from the first — ragged frames silently corrupt every downstream aggregation, so we fail fast instead.

Use `from_columns` for the `Result`-returning version when columns may
come from user input and the caller wants to handle the error.

id_vars — columns kept as-is in every output row. value_vars — columns whose names become values of var_col and whose values become values of val_col. One output row is produced per (input row × value_var).

Example:
  melt(df, ["id"], ["jan","feb","mar"], "month", "value")
  turns:  id | jan | feb | mar
  into:   id | month | value   (3 output rows per original row)

Skips the first n rows.

The union of left_join and right_join: every row from both the left and right frames appears in the output. Rows with no match on the other side get Null for all columns from that side.

Example:
  let lf   = DataFrame.lazy(employees_df)
             |> DataFrame.outer_join(departments_df, ["dept_id"])
  let result = DataFrame.collect(lf)
  -- All employees and all departments appear; Nulls where there is no match

index_col — column whose distinct values become output rows. cols_col — column whose distinct values become new output column names. vals_col — column whose values fill the output cells. Missing (index, col) combinations get NullVal.

Example:
  pivot(df, "region", "product", "revenue")
  turns:  region | product | revenue
  into:   region | A | B | ...   (one column per distinct product)

Print a DataFrame as a simple text table.

Renames a column in the pipeline.

Renames a column. Errors if old_name not found or new_name already exists.

The mirror image of left_join. Every row in right appears in the output; rows with no match in the left frame get Null for every left-only column.

Example:
  let lf   = DataFrame.lazy(transactions_df)
             |> DataFrame.right_join(reference_df, ["code"])
  let result = DataFrame.collect(lf)
  -- Every reference row is included; transaction cols are Null for unmatched codes

Returns the number of rows. O(1) for single-column frames; O(cols) otherwise.

Look up a field in a Row by name. Returns None if not found.

Look up a Bool field in a Row.

Look up a Float field in a Row. Int values are promoted to Float.

Look up an Int field in a Row.

Look up a String field in a Row.

Selects n rows at evenly-spaced positions across the DataFrame. Returns the full DataFrame unchanged when n >= row_count(df).

Example:
  let df  = DataFrame.make_df([IntCol("v", [0,1,2,3,4,5,6,7,8,9])])
  let s   = DataFrame.sample(df, 3)
  -- picks rows 0, 3, 6  ->  IntCol("v", [0, 3, 6])

Returns the ordered list of column names.

Projects a subset of columns.

Returns a DataFrame with columns reordered/selected by name list. Errors on missing names.

Returns rows [start, start+len). Clamps to available rows.

Appends a sort node. Keys is a list of (column_name, SortDir) pairs.

Columns in the result: "column", "type", "count", "mean", "std", "min", "p25", "median", "p75", "max". Non-numeric columns get NullVal for every numeric stat.

Example:
  let df = DataFrame.make_df([IntCol("x", [1,2,3,4,5])])
  let d  = DataFrame.summarize(df)
  -- d has 1 row: column="x", type:"Int", count:5, mean:3.0, ...

Returns the last n rows.

Serialize a DataFrame to a CSV string. Uses default options.

Serialize a DataFrame to a CSV string with custom options.

Render a DataFrame as an HTML table string. The output is detected automatically by the March notebook and displayed as a styled table rather than raw text.

Serialize a DataFrame to a JSON string (array of objects).

Returns all rows as a list of Row values. O(rows × cols). Slow path.

ratio is the fraction of rows placed in the training set (0.0 < ratio < 1.0). The first floor(row_count * ratio) rows become the training set; the remainder become the test set. Row order is preserved.

Example:
  let (train, test) = DataFrame.train_test_split(df, 0.8)
  -- 80% of rows → train, 20% → test

Count frequency of each distinct value. Returns DataFrame with columns [col_name, 'count'] sorted by count desc.

Converts a Value to its string representation.

a new column named out_col. No partitioning (operates over all rows).

Adds or replaces a column computed row-by-row using a March closure.