| Title: | Read, Write, Validate, Stream, and Convert TOON Data |
|---|---|
| Description: | A minimal-dependency, performance-first R package for reading, writing, validating, streaming, and converting TOON (Token-Oriented Object Notation) data. Optimized for very large tabular files with robust diagnostics. Supports lossless JSON conversion and tabular CSV/Parquet/Feather conversion. |
| Authors: | Alejandro Jiménez Rico [aut, cre] |
| Maintainer: | Alejandro Jiménez Rico <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.1.0 |
| Built: | 2026-05-20 09:00:21 UTC |
| Source: | https://github.com/aljrico/toonlite |
Convert array-of-objects to tabular representation
as_tabular_toon(x, strict = TRUE, warn = TRUE)as_tabular_toon(x, strict = TRUE, warn = TRUE)
x |
List of lists/named lists (array of objects). |
strict |
Logical. If TRUE (default), validate input structure. |
warn |
Logical. If TRUE (default), emit warnings for type promotions. |
In permissive mode:
Union schema across all records
Missing fields filled with NA
Type promotion to accommodate all values
A data.frame suitable for write_toon_df().
# Convert array of objects to data.frame records <- list( list(name = "Alice", age = 30), list(name = "Bob", age = 25) ) df <- as_tabular_toon(records)# Convert array of objects to data.frame records <- list( list(name = "Alice", age = 30), list(name = "Bob", age = 25) ) df <- as_tabular_toon(records)
Assert TOON validity
assert_toon( x, is_file = FALSE, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE )assert_toon( x, is_file = FALSE, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE )
x |
Character scalar containing TOON text, or file path if is_file=TRUE. |
is_file |
Logical. If TRUE, x is treated as a file path. |
strict |
Logical. If TRUE (default), enforce strict TOON syntax. |
allow_comments |
Logical. If TRUE (default), allow # and // comments. |
allow_duplicate_keys |
Logical. If TRUE (default), allow duplicate keys. |
Throws toonlite_parse_error condition if invalid, with attached
location and snippet information.
Invisibly returns TRUE if valid.
# Valid TOON (returns invisibly) assert_toon('name: "Alice"') # Invalid TOON (throws error) ## Not run: assert_toon('invalid: [') ## End(Not run)# Valid TOON (returns invisibly) assert_toon('name: "Alice"') # Invalid TOON (throws error) ## Not run: assert_toon('invalid: [') ## End(Not run)
Functions for creating and signaling toonlite-specific conditions.
Convert CSV to TOON
csv_to_toon( path_csv, path_toon, tabular = TRUE, strict = TRUE, col_types = NULL )csv_to_toon( path_csv, path_toon, tabular = TRUE, strict = TRUE, col_types = NULL )
path_csv |
Character scalar. Path to input CSV file. |
path_toon |
Character scalar. Path to output TOON file. |
tabular |
Logical. If TRUE (default), write as tabular TOON array. |
strict |
Logical. If TRUE (default), enforce strict syntax on output. |
col_types |
Named character vector specifying column types. |
Invisibly returns NULL.
## Not run: csv_to_toon("data.csv", "data.toon") ## End(Not run)## Not run: csv_to_toon("data.csv", "data.toon") ## End(Not run)
Convert Feather to TOON
feather_to_toon(path_feather, path_toon, tabular = TRUE, strict = TRUE)feather_to_toon(path_feather, path_toon, tabular = TRUE, strict = TRUE)
path_feather |
Character scalar. Path to input Feather file. |
path_toon |
Character scalar. Path to output TOON file. |
tabular |
Logical. If TRUE (default), write as tabular TOON array. |
strict |
Logical. If TRUE (default), enforce strict syntax on output. |
Requires the arrow package. If not installed, an error is thrown.
Invisibly returns NULL.
## Not run: feather_to_toon("data.feather", "data.toon") ## End(Not run)## Not run: feather_to_toon("data.feather", "data.toon") ## End(Not run)
Format/pretty-print TOON
format_toon( x, is_file = FALSE, indent = 2L, canonical = FALSE, allow_comments = TRUE )format_toon( x, is_file = FALSE, indent = 2L, canonical = FALSE, allow_comments = TRUE )
x |
Character scalar containing TOON text, or file path if is_file=TRUE. |
is_file |
Logical. If TRUE, x is treated as a file path. |
indent |
Integer. Number of spaces for indentation (default 2). |
canonical |
Logical. If TRUE, use stable representation with lexicographic key ordering. Default FALSE (preserve original order). |
allow_comments |
Logical. If TRUE (default), allow comments in input. |
Character scalar with formatted TOON. If is_file=TRUE, returns formatted text (does not rewrite file).
# Format with default indent format_toon('name:"Alice"') # Format with canonical key ordering format_toon('b: 1\na: 2', canonical = TRUE)# Format with default indent format_toon('name:"Alice"') # Format with canonical key ordering format_toon('b: 1\na: 2', canonical = TRUE)
Parse TOON from string
from_toon( text, strict = TRUE, simplify = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE )from_toon( text, strict = TRUE, simplify = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE )
text |
Character scalar or raw vector containing TOON data. If raw, treated as UTF-8 bytes. |
strict |
Logical. If TRUE (default), enforce strict TOON syntax. |
simplify |
Logical. If TRUE (default), simplify homogeneous arrays to atomic vectors. |
allow_comments |
Logical. If TRUE (default), allow # and // comments. |
allow_duplicate_keys |
Logical. If TRUE (default), allow duplicate keys in objects (last-one-wins semantics). |
R object representing the parsed TOON data:
object -> named list
array -> list or atomic vector (if simplified)
primitives -> logical/integer/double/character/NULL
# Parse simple object from_toon('name: "Alice"\nage: 30') # Parse array from_toon('[3]:\n - 1\n - 2\n - 3') # Parse with comments from_toon('# comment\nkey: "value"', allow_comments = TRUE)# Parse simple object from_toon('name: "Alice"\nage: 30') # Parse array from_toon('[3]:\n - 1\n - 2\n - 3') # Parse with comments from_toon('# comment\nkey: "value"', allow_comments = TRUE)
Convert JSON to TOON
json_to_toon(json, pretty = TRUE, strict = TRUE, allow_comments = TRUE)json_to_toon(json, pretty = TRUE, strict = TRUE, allow_comments = TRUE)
json |
Character scalar containing JSON. |
pretty |
Logical. If TRUE (default), use multi-line formatting. |
strict |
Logical. If TRUE (default), enforce strict syntax. |
allow_comments |
Logical. If TRUE (default), allow comments in output. |
Requires the jsonlite package. If not installed, an error is thrown.
Character scalar with class "toon".
## Not run: toon <- json_to_toon('{"name": "Alice", "age": 30}') ## End(Not run)## Not run: toon <- json_to_toon('{"name": "Alice", "age": 30}') ## End(Not run)
Convert Parquet to TOON
parquet_to_toon(path_parquet, path_toon, tabular = TRUE, strict = TRUE)parquet_to_toon(path_parquet, path_toon, tabular = TRUE, strict = TRUE)
path_parquet |
Character scalar. Path to input Parquet file. |
path_toon |
Character scalar. Path to output TOON file. |
tabular |
Logical. If TRUE (default), write as tabular TOON array. |
strict |
Logical. If TRUE (default), enforce strict syntax on output. |
Requires the arrow package. If not installed, an error is thrown.
Invisibly returns NULL.
## Not run: parquet_to_toon("data.parquet", "data.toon") ## End(Not run)## Not run: parquet_to_toon("data.parquet", "data.toon") ## End(Not run)
Print method for toon class
## S3 method for class 'toon' print(x, ...)## S3 method for class 'toon' print(x, ...)
x |
TOON string object |
... |
Additional arguments (ignored) |
The input x, invisibly.
x <- to_toon(list(a = 1, b = 2)) print(x)x <- to_toon(list(a = 1, b = 2)) print(x)
Read TOON from file
read_toon( file, strict = TRUE, simplify = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE, encoding = "UTF-8" )read_toon( file, strict = TRUE, simplify = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE, encoding = "UTF-8" )
file |
Character scalar. Path to TOON file. |
strict |
Logical. If TRUE (default), enforce strict TOON syntax. |
simplify |
Logical. If TRUE (default), simplify homogeneous arrays to atomic vectors. |
allow_comments |
Logical. If TRUE (default), allow # and // comments. |
allow_duplicate_keys |
Logical. If TRUE (default), allow duplicate keys in objects. |
encoding |
Character. File encoding (default "UTF-8"). |
R object representing the parsed TOON data.
# Read from file ## Not run: data <- read_toon("config.toon") ## End(Not run)# Read from file ## Not run: data <- read_toon("config.toon") ## End(Not run)
Read tabular TOON to data.frame
read_toon_df( file, key = NULL, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE, warn = TRUE, col_types = NULL, ragged_rows = c("expand_warn", "error"), n_mismatch = c("warn", "error"), max_extra_cols = Inf )read_toon_df( file, key = NULL, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE, warn = TRUE, col_types = NULL, ragged_rows = c("expand_warn", "error"), n_mismatch = c("warn", "error"), max_extra_cols = Inf )
file |
Character scalar. Path to TOON file. |
key |
Character scalar or NULL. If non-NULL, extract tabular array at root[key] (root must be object). |
strict |
Logical. If TRUE (default), enforce strict TOON syntax. |
allow_comments |
Logical. If TRUE (default), allow # and // comments. |
allow_duplicate_keys |
Logical. If TRUE (default), allow duplicate keys. |
warn |
Logical. If TRUE (default), emit warnings for schema changes. |
col_types |
Named character vector specifying column types: "logical", "integer", "double", or "character". |
ragged_rows |
Character. How to handle rows with different field counts:
|
n_mismatch |
Character. How to handle declared row count mismatch:
|
max_extra_cols |
Numeric. Maximum new columns allowed via schema expansion. Default Inf (no limit). |
A base data.frame.
## Not run: # Read tabular TOON file df <- read_toon_df("data.toon") # Read nested tabular array df <- read_toon_df("config.toon", key = "records") ## End(Not run)## Not run: # Read tabular TOON file df <- read_toon_df("data.toon") # Read nested tabular array df <- read_toon_df("config.toon", key = "records") ## End(Not run)
Functions for reading and writing tabular TOON data as data.frames.
Serialize R object to TOON
to_toon(x, pretty = TRUE, indent = 2L, strict = TRUE, allow_comments = FALSE)to_toon(x, pretty = TRUE, indent = 2L, strict = TRUE, allow_comments = FALSE)
x |
R object to serialize. |
pretty |
Logical. If TRUE (default), use multi-line formatting. |
indent |
Integer. Number of spaces for indentation (default 2). |
strict |
Logical. If TRUE (default), reject NaN/Inf values. |
allow_comments |
Logical. For future use (writers generally should not emit comments). |
Type conversions:
factor -> character
Date -> ISO 8601 string (YYYY-MM-DD)
POSIXct -> ISO 8601 string (YYYY-MM-DDTHH:MM:SSZ)
NA values -> null
Character scalar with class "toon" containing the TOON representation.
# Serialize simple list to_toon(list(name = "Alice", age = 30)) # Serialize vector to_toon(c(1, 2, 3)) # Compact format to_toon(list(x = 1, y = 2), pretty = FALSE)# Serialize simple list to_toon(list(name = "Alice", age = 30)) # Serialize vector to_toon(c(1, 2, 3)) # Compact format to_toon(list(x = 1, y = 2), pretty = FALSE)
Get TOON file info
toon_info(file, allow_comments = TRUE)toon_info(file, allow_comments = TRUE)
file |
Character scalar. Path to TOON file. |
allow_comments |
Logical. If TRUE (default), allow comments. |
A list with components:
array_count: Integer. Number of arrays in file.
object_count: Integer. Number of objects in file.
has_tabular: Logical. Whether file contains tabular arrays.
declared_rows: Integer or NA. Declared row count if tabular.
## Not run: info <- toon_info("data.toon") cat("Arrays:", info$array_count, "\n") cat("Has tabular:", info$has_tabular, "\n") ## End(Not run)## Not run: info <- toon_info("data.toon") cat("Arrays:", info$array_count, "\n") cat("Has tabular:", info$has_tabular, "\n") ## End(Not run)
Peek at TOON file structure
toon_peek(file, n = 50L, allow_comments = TRUE)toon_peek(file, n = 50L, allow_comments = TRUE)
file |
Character scalar. Path to TOON file. |
n |
Integer. Number of lines to preview (default 50). |
allow_comments |
Logical. If TRUE (default), allow comments. |
A list with components:
type: Character. Top-level type ("object", "array", "tabular_array", "unknown").
first_keys: Character vector. First few keys if top-level is object.
preview: Character vector. First n lines.
## Not run: info <- toon_peek("data.toon") cat("Type:", info$type, "\n") cat("Preview:\n", info$preview, sep = "\n") ## End(Not run)## Not run: info <- toon_peek("data.toon") cat("Type:", info$type, "\n") cat("Preview:\n", info$preview, sep = "\n") ## End(Not run)
Stream non-tabular array items
toon_stream_items( file, key = NULL, callback, batch_size = 1000L, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE, warn = TRUE, simplify = TRUE )toon_stream_items( file, key = NULL, callback, batch_size = 1000L, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE, warn = TRUE, simplify = TRUE )
file |
Character scalar. Path to TOON file. |
key |
Character scalar or NULL. If non-NULL, extract array at root[key]. |
callback |
Function. Called with each batch as a list. |
batch_size |
Integer. Number of items per batch (default 1000). |
strict |
Logical. If TRUE (default), enforce strict TOON syntax. |
allow_comments |
Logical. If TRUE (default), allow # and // comments. |
allow_duplicate_keys |
Logical. If TRUE (default), allow duplicate keys. |
warn |
Logical. If TRUE (default), emit warnings. |
simplify |
Logical. If TRUE (default), simplify homogeneous batches. |
Invisibly returns NULL.
## Not run: # Stream array items toon_stream_items("items.toon", callback = function(batch) { cat("Got", length(batch), "items\n") } ) ## End(Not run)## Not run: # Stream array items toon_stream_items("items.toon", callback = function(batch) { cat("Got", length(batch), "items\n") } ) ## End(Not run)
Stream tabular TOON rows
toon_stream_rows( file, key = NULL, callback, batch_size = 10000L, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE, warn = TRUE, col_types = NULL, ragged_rows = c("expand_warn", "error"), n_mismatch = c("warn", "error"), max_extra_cols = Inf )toon_stream_rows( file, key = NULL, callback, batch_size = 10000L, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE, warn = TRUE, col_types = NULL, ragged_rows = c("expand_warn", "error"), n_mismatch = c("warn", "error"), max_extra_cols = Inf )
file |
Character scalar. Path to TOON file. |
key |
Character scalar or NULL. If non-NULL, extract tabular array at root[key]. |
callback |
Function. Called with each batch as a data.frame. Return value is ignored; exceptions propagate and abort parsing. |
batch_size |
Integer. Number of rows per batch (default 10000). |
strict |
Logical. If TRUE (default), enforce strict TOON syntax. |
allow_comments |
Logical. If TRUE (default), allow # and // comments. |
allow_duplicate_keys |
Logical. If TRUE (default), allow duplicate keys. |
warn |
Logical. If TRUE (default), emit warnings. |
col_types |
Named character vector specifying column types. |
ragged_rows |
Character. How to handle rows with different field counts. |
n_mismatch |
Character. How to handle declared row count mismatch. |
max_extra_cols |
Numeric. Maximum new columns allowed. |
Guarantees:
Batches are independent and may be garbage collected
Stable column order within stream
Invisibly returns NULL.
## Not run: # Stream large file in batches toon_stream_rows("large.toon", callback = function(batch) { cat("Processing", nrow(batch), "rows\n") }, batch_size = 10000 ) ## End(Not run)## Not run: # Stream large file in batches toon_stream_rows("large.toon", callback = function(batch) { cat("Processing", nrow(batch), "rows\n") }, batch_size = 10000 ) ## End(Not run)
Stream write tabular rows
toon_stream_write_rows( file, schema, row_source, batch_size = 10000L, indent = 2L )toon_stream_write_rows( file, schema, row_source, batch_size = 10000L, indent = 2L )
file |
Character scalar. Path to output file. |
schema |
Character vector of column names. |
row_source |
Function that returns next batch as data.frame, or NULL to end. |
batch_size |
Integer. Hint for batch size (passed to row_source). |
indent |
Integer. Number of spaces for indentation (default 2). |
Writes a tabular TOON array without holding all rows in memory.
Invisibly returns the number of rows written.
## Not run: # Stream write rows i <- 0 row_source <- function() { i <<- i + 1 if (i > 3) return(NULL) data.frame(x = i, y = i * 2) } toon_stream_write_rows("output.toon", schema = c("x", "y"), row_source = row_source ) ## End(Not run)## Not run: # Stream write rows i <- 0 row_source <- function() { i <<- i + 1 if (i > 3) return(NULL) data.frame(x = i, y = i * 2) } toon_stream_write_rows("output.toon", schema = c("x", "y"), row_source = row_source ) ## End(Not run)
Convert TOON to CSV
toon_to_csv( path_toon, path_csv, key = NULL, strict = TRUE, allow_comments = TRUE, warn = TRUE )toon_to_csv( path_toon, path_csv, key = NULL, strict = TRUE, allow_comments = TRUE, warn = TRUE )
path_toon |
Character scalar. Path to input TOON file. |
path_csv |
Character scalar. Path to output CSV file. |
key |
Character scalar or NULL. If non-NULL, extract tabular array at root[key]. |
strict |
Logical. If TRUE (default), enforce strict syntax. |
allow_comments |
Logical. If TRUE (default), allow comments. |
warn |
Logical. If TRUE (default), emit warnings. |
Invisibly returns NULL.
## Not run: toon_to_csv("data.toon", "data.csv") ## End(Not run)## Not run: toon_to_csv("data.toon", "data.csv") ## End(Not run)
Convert TOON to Feather
toon_to_feather( path_toon, path_feather, key = NULL, strict = TRUE, allow_comments = TRUE, warn = TRUE )toon_to_feather( path_toon, path_feather, key = NULL, strict = TRUE, allow_comments = TRUE, warn = TRUE )
path_toon |
Character scalar. Path to input TOON file. |
path_feather |
Character scalar. Path to output Feather file. |
key |
Character scalar or NULL. If non-NULL, extract tabular array. |
strict |
Logical. If TRUE (default), enforce strict syntax. |
allow_comments |
Logical. If TRUE (default), allow comments. |
warn |
Logical. If TRUE (default), emit warnings. |
Requires the arrow package. If not installed, an error is thrown.
Invisibly returns NULL.
## Not run: toon_to_feather("data.toon", "data.feather") ## End(Not run)## Not run: toon_to_feather("data.toon", "data.feather") ## End(Not run)
Convert TOON to JSON
toon_to_json(toon, pretty = FALSE, strict = TRUE, allow_comments = TRUE)toon_to_json(toon, pretty = FALSE, strict = TRUE, allow_comments = TRUE)
toon |
Character scalar containing TOON. |
pretty |
Logical. If TRUE, use multi-line JSON formatting (default FALSE). |
strict |
Logical. If TRUE (default), enforce strict syntax. |
allow_comments |
Logical. If TRUE (default), allow comments in input. |
Requires the jsonlite package. If not installed, an error is thrown.
Character scalar containing JSON.
## Not run: json <- toon_to_json('name: "Alice"\nage: 30') ## End(Not run)## Not run: json <- toon_to_json('name: "Alice"\nage: 30') ## End(Not run)
Convert TOON to Parquet
toon_to_parquet( path_toon, path_parquet, key = NULL, strict = TRUE, allow_comments = TRUE, warn = TRUE )toon_to_parquet( path_toon, path_parquet, key = NULL, strict = TRUE, allow_comments = TRUE, warn = TRUE )
path_toon |
Character scalar. Path to input TOON file. |
path_parquet |
Character scalar. Path to output Parquet file. |
key |
Character scalar or NULL. If non-NULL, extract tabular array. |
strict |
Logical. If TRUE (default), enforce strict syntax. |
allow_comments |
Logical. If TRUE (default), allow comments. |
warn |
Logical. If TRUE (default), emit warnings. |
Requires the arrow package. If not installed, an error is thrown.
Invisibly returns NULL.
## Not run: toon_to_parquet("data.toon", "data.parquet") ## End(Not run)## Not run: toon_to_parquet("data.toon", "data.parquet") ## End(Not run)
Validate TOON
validate_toon( x, is_file = FALSE, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE )validate_toon( x, is_file = FALSE, strict = TRUE, allow_comments = TRUE, allow_duplicate_keys = TRUE )
x |
Character scalar containing TOON text, or file path if is_file=TRUE. |
is_file |
Logical. If TRUE, x is treated as a file path. |
strict |
Logical. If TRUE (default), enforce strict TOON syntax. |
allow_comments |
Logical. If TRUE (default), allow # and // comments. |
allow_duplicate_keys |
Logical. If TRUE (default), allow duplicate keys. |
Never throws unless there's an internal error (e.g., file unreadable). May warn for permissive recoveries.
Logical scalar.
TRUE if valid.
FALSE if invalid, with attribute attr(result, "error")
containing a structured error object with components: type, message,
line, column, snippet, file.
# Valid TOON validate_toon('key: "value"') # Invalid TOON (returns FALSE with error attribute) result <- validate_toon('key: {invalid') if (!result) print(attr(result, "error")$message)# Valid TOON validate_toon('key: "value"') # Invalid TOON (returns FALSE with error attribute) result <- validate_toon('key: {invalid') if (!result) print(attr(result, "error")$message)
Write R object to TOON file
write_toon(x, file, pretty = TRUE, indent = 2L, strict = TRUE)write_toon(x, file, pretty = TRUE, indent = 2L, strict = TRUE)
x |
R object to serialize. |
file |
Character scalar. Path to output file. |
pretty |
Logical. If TRUE (default), use multi-line formatting. |
indent |
Integer. Number of spaces for indentation (default 2). |
strict |
Logical. If TRUE (default), reject NaN/Inf values. |
Invisibly returns NULL.
## Not run: write_toon(list(x = 1, y = 2), "output.toon") ## End(Not run)## Not run: write_toon(list(x = 1, y = 2), "output.toon") ## End(Not run)
Write data.frame to tabular TOON
write_toon_df( df, file, tabular = TRUE, pretty = TRUE, indent = 2L, strict = TRUE )write_toon_df( df, file, tabular = TRUE, pretty = TRUE, indent = 2L, strict = TRUE )
df |
A data.frame to write. |
file |
Character scalar. Path to output file. |
tabular |
Logical. If TRUE (default), write as tabular TOON array. |
pretty |
Logical. If TRUE (default), use multi-line formatting. |
indent |
Integer. Number of spaces for indentation (default 2). |
strict |
Logical. If TRUE (default), reject NaN/Inf values. |
Invisibly returns NULL.
## Not run: # Write data.frame as tabular TOON write_toon_df(mtcars[1:3, 1:4], "cars.toon") ## End(Not run)## Not run: # Write data.frame as tabular TOON write_toon_df(mtcars[1:3, 1:4], "cars.toon") ## End(Not run)