Decouple IngestionDocumentReader from IngestionPipeline constructor

[Copilot]

IngestionPipeline<T> required an IngestionDocumentReader at construction time, making it impossible to use with in-memory or programmatically created documents.

Changes

• IngestionPipeline<T> constructor — removed reader parameter; pipeline now only requires chunker and writer
• New overload — ProcessAsync(IAsyncEnumerable<IngestionDocument>, CancellationToken) processes documents directly without any file-system dependency
• File-system overloads — ProcessAsync(IngestionDocumentReader, DirectoryInfo, ...) and ProcessAsync(IngestionDocumentReader, IEnumerable<FileInfo>, ...) now take reader as a mandatory first argument
• DiagnosticsConstants — added ProcessDocuments / ProcessDocument activity names for the new overload's tracing
• Tests — updated existing tests for the new signatures; added CanProcessDocumentsWithoutReader demonstrating direct document ingestion
• Template + snapshots — updated DataIngestor.cs and all 5 integration-test snapshots
• READMEs / CHANGELOG — updated Microsoft.Extensions.DataIngestion, MarkItDown, and Markdig docs

Usage without a reader

using IngestionPipeline<string> pipeline = new(CreateChunker(), CreateWriter());

IngestionDocument document = new("my-doc-id");
document.Sections.Add(new IngestionDocumentSection());
document.Sections[0].Elements.Add(new IngestionDocumentParagraph("In-memory content."));

await foreach (IngestionResult result in pipeline.ProcessAsync(new[] { document }.ToAsyncEnumerable()))
{
    Console.WriteLine($"{result.DocumentId}: {result.Succeeded}");
}

Usage with a reader (file system)

IngestionDocumentReader reader = new MarkdownReader();
using IngestionPipeline<string> pipeline = new(CreateChunker(), CreateWriter());

await foreach (IngestionResult result in pipeline.ProcessAsync(reader, new DirectoryInfo("docs"), "*.md"))
{
    Console.WriteLine($"{result.DocumentId}: {result.Succeeded}");
}

Microsoft Reviewers: Open in CodeFlow