Acquis 25 - Data Processing

Extensions to the table and string standard libraries for common data processing operations: aggregation, transformation, matrix operations, and string manipulation.

Table aggregation

table.sum

Computes the sum of numeric values in a table. Non-numeric values are skipped. Returns 0 for an empty table.

local total = table.sum({1, 2, 3, 4, 5})
assert(total == 15)

assert(table.sum({1, "skip", 2, 3}) == 6)
assert(table.sum({}) == 0)

table.mean

Computes the arithmetic mean of numeric values. Returns 0/0 (NaN) if no numeric values exist.

assert(table.mean({1, 2, 3, 4, 5}) == 3)
assert(table.mean({10, 20}) == 15)

local nan = table.mean({})
assert(nan ~= nan) -- NaN

table.median

Computes the median of numeric values. Returns the middle value for odd-length arrays, or the average of the two middle values for even-length arrays. Does not modify the original table.

assert(table.median({5, 1, 3, 2, 4}) == 3)
assert(table.median({1, 2, 3, 4}) == 2.5)

table.stdev

Computes the standard deviation of numeric values. When sample is true, divides by n-1 (sample standard deviation); otherwise divides by n (population standard deviation).

local values = {2, 4, 4, 4, 5, 5, 7, 9}
local pop = table.stdev(values)        -- population
local sam = table.stdev(values, true)  -- sample

Table transformation

table.map

Applies a function to each element, returning a new table of results. The function receives (element, index).

local doubled = table.map({1, 2, 3}, function(x) return x * 2 end)
assert(doubled[1] == 2 and doubled[2] == 4 and doubled[3] == 6)

local names = table.map(
    {{name = "Alice"}, {name = "Bob"}},
    function(p) return p.name end
)
assert(names[1] == "Alice")

table.filter

Returns a new table containing only elements for which the predicate returns a truthy value.

local evens = table.filter({1, 2, 3, 4, 5}, function(x) return x % 2 == 0 end)
assert(#evens == 2)
assert(evens[1] == 2 and evens[2] == 4)

table.reduce

Reduces a table to a single value by iteratively applying a function. The function receives (accumulator, element, index). If no initial value is provided, the first element is used.

local sum = table.reduce({1, 2, 3, 4, 5}, function(acc, x) return acc + x end, 0)
assert(sum == 15)

local product = table.reduce({1, 2, 3, 4}, function(acc, x) return acc * x end)
assert(product == 24)

table.groupby

Groups elements by the result of a key function.

local people = {
    {name = "Alice", dept = "eng"},
    {name = "Bob", dept = "sales"},
    {name = "Charlie", dept = "eng"},
}

local by_dept = table.groupby(people, function(p) return p.dept end)
assert(#by_dept.eng == 2)
assert(#by_dept.sales == 1)

table.sortby

Sorts a table in-place by a key function. When asc is false, sorts in descending order. Default is ascending.

local people = {
    {name = "Charlie", age = 35},
    {name = "Alice", age = 30},
    {name = "Bob", age = 25},
}

table.sortby(people, function(p) return p.age end)
assert(people[1].name == "Bob")

table.sortby(people, function(p) return p.age end, false)
assert(people[1].name == "Charlie")

Table combining

table.zip

Combines multiple tables element-wise into a table of tuples. The result length equals the shortest input.

local names = {"Alice", "Bob", "Charlie"}
local ages = {30, 25, 35}

local zipped = table.zip(names, ages)
assert(zipped[1][1] == "Alice" and zipped[1][2] == 30)
assert(zipped[2][1] == "Bob" and zipped[2][2] == 25)

table.unzip

Splits a table of tuples into separate tables (inverse of zip).

local zipped = {{"Alice", 30}, {"Bob", 25}}
local names, ages = table.unzip(zipped)
assert(names[1] == "Alice" and ages[2] == 25)

Matrix operations

table.transpose

Transposes a 2D table (matrix) so that result[i][j] == matrix[j][i].

local m = {{1, 2, 3}, {4, 5, 6}}
local t = table.transpose(m)
assert(t[1][1] == 1 and t[1][2] == 4)
assert(t[2][1] == 2 and t[2][2] == 5)
assert(t[3][1] == 3 and t[3][2] == 6)

table.reshape

Reshapes a 1D array into a 2D matrix with the specified dimensions, filled in row-major order. The array length must equal rows * cols.

local arr = {1, 2, 3, 4, 5, 6}
local matrix = table.reshape(arr, 2, 3)
assert(matrix[1][1] == 1 and matrix[1][3] == 3)
assert(matrix[2][1] == 4 and matrix[2][3] == 6)

String operations

string.split

Splits a string on a literal delimiter. An empty delimiter splits into individual characters.

local parts = string.split("apple,banana,cherry", ",")
assert(parts[1] == "apple" and parts[3] == "cherry")

local chars = string.split("hello", "")
assert(#chars == 5 and chars[1] == "h")

-- Consecutive delimiters produce empty strings
local dirs = string.split("a//b", "/")
assert(dirs[1] == "a" and dirs[2] == "" and dirs[3] == "b")

string.join

Joins table elements into a single string with a delimiter. Non-string values are converted via tostring.

assert(string.join({"hello", "world"}, " ") == "hello world")
assert(string.join({1, 2, 3}, ",") == "1,2,3")
assert(string.join({}, ",") == "")

string.trim, string.ltrim, string.rtrim

Remove leading and/or trailing whitespace (or specified characters) from a string.

assert(string.trim("  hello  ") == "hello")
assert(string.ltrim("  hello  ") == "hello  ")
assert(string.rtrim("  hello  ") == "  hello")

-- Custom characters
assert(string.trim("xxxhelloxxx", "x") == "hello")
assert(string.ltrim("...test", ".") == "test")
assert(string.rtrim("test...", ".") == "test")

Motivation

Data processing without dependencies

Lus is designed to be productive out of the box. Operations like filtering, mapping, grouping, and statistical aggregation are foundational to data processing. Providing them in the standard library eliminates the need for external dependencies in common workflows.

Consistency with existing patterns

The table and string modules already provide core operations. These extensions follow the same conventions: functions on existing modules, consistent parameter ordering, and predictable error behavior. The table.sort function already exists; table.sortby extends it for the common case of sorting by a derived key.

String gaps

Lua’s string library lacks basic operations like splitting, joining, and trimming that are standard in other languages. These are among the most frequently reimplemented utilities in Lua codebases.