Skip to content

Add Delimited Text Serialization #168

Description

@tiksn

Add Delimited Text Serialization

Goal

Add ROFSDB support for delimited text files so collections can read .csv and .tsv files as strongly typed documents.

Repository Context

ROFSDB treats directories as collections and files as documents. File formats are implemented with TIKSN.ROFSDB.Serialization.ISerialization, registered as keyed ISerialization services in ServiceCollectionExtensions, and consumed by DatabaseEngine.GetDocumentsAsync<T>(). Tests use xUnit and Shouldly with an in-memory Zio filesystem fixture.

Implementation Guidance

  • Add a serializer class for delimited text under src/ROFSDB/Serialization.
  • Register the serializer in ServiceCollectionExtensions.
  • Support .csv and .tsv extensions.
  • Map one data row to one T.
  • Use the first row as headers.
  • Match headers to public writable properties case-insensitively.
  • Support quoted CSV fields, escaped quotes, delimiters inside quoted fields, empty fields, and CRLF/LF line endings.
  • Use a real CSV parser package if the implementation would otherwise grow custom parsing logic.
  • Preserve the existing ISerialization contract and where T : class, new() constraint.
  • Skip no rows other than a single optional blank trailing row; malformed rows should fail with a clear FormatException.

Test Requirements

  • Add CSV and TSV entries to the shared city/country fixture matrix.
  • Cover multiple rows in one file.
  • Cover quoted values and embedded delimiters.
  • Cover case-insensitive header-to-property matching.
  • Cover missing optional/reference fields and missing required/value-type fields.
  • Cover malformed row length or malformed quoting.
  • Cover cancellation token behavior.

Acceptance Criteria

  • .csv and .tsv files are discovered by extension and read through GetDocumentsAsync<T>().
  • Existing JSON, YAML, TOML, KDL, HCL, PSD1, and Parquet behavior remains unchanged.
  • dotnet test passes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions