TokenShrink (Document to Markdown Converter)

TokenShrink is a privacy-first, local web application designed to convert PDFs and other documents into clean, structured Markdown.

When feeding large documents to AI models (like ChatGPT, Claude, or local LLMs), raw PDFs often waste context window limits and rack up high token costs. This tool extracts the content while preserving headings, lists, and structure—creating highly optimized, token-efficient plain text.

Key Features

• AI-Optimized Output: Drastically reduces token usage while maintaining document context and structure.
• 100% Local & Private: No cloud uploads. All processing happens locally on your machine via a lightweight Python backend.
• Sleek UI: A modern, dark-themed interface built with Vanilla HTML/CSS/JS for maximum speed and simplicity.
• Built on MarkItDown: Leverages Microsoft's robust markitdown library for accurate document parsing.

Prerequisites

• Python 3.8+ installed on your machine.

Installation

1. Clone or Download this repository.
2. Navigate to the backend directory:

   cd backend

3. Create and Activate a Virtual Environment:

   # Windows
   python -m venv venv
   .\venv\Scripts\activate
   
   # macOS/Linux
   python3 -m venv venv
   source venv/bin/activate

4. Install Dependencies:

   pip install -r requirements.txt

Running the App

1. Make sure your virtual environment is activated.
2. Start the FastAPI backend server:

   uvicorn main:app --port 8000

3. Open the frontend/index.html file directly in your favorite web browser (no web server needed).
4. Drag and drop your file to convert!

License

This project is licensed under the MIT License - see the LICENSE file for details.