Skip to content

[Backend] Add content-based file validation (magic number checks) #667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
SAI-HARISH2007 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[Backend] Add content-based file validation (magic number checks) #667

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4ed5441
feat: add content-based file validation (magic number checks)
SAI-HARISH2007 Apr 15, 2026
ab46ce3
fix: preserve upload API contract and improve DOCX validation
SAI-HARISH2007 Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 45 additions & 9 deletions backend/Generator/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import os
import fitz
import mammoth
from werkzeug.utils import secure_filename

class MCQGenerator:

Expand Down Expand Up @@ -368,19 +369,54 @@ def extract_text_from_docx(self, file_path):
return result.value

def process_file(self, file):
file_path = os.path.join(self.upload_folder, file.filename)
if not file.filename:
raise ValueError("Empty filename")

filename = secure_filename(file.filename)
if not filename:
raise ValueError("Invalid filename")

target_dir = os.path.abspath(self.upload_folder)
file_path = os.path.abspath(os.path.join(target_dir, filename))

if os.path.commonpath([target_dir, file_path]) != target_dir:
raise ValueError("Path traversal detected")

file.save(file_path)
content = ""

if file.filename.endswith('.txt'):
with open(file_path, 'r') as f:
content = f.read()
elif file.filename.endswith('.pdf'):
content = self.extract_text_from_pdf(file_path)
elif file.filename.endswith('.docx'):
content = self.extract_text_from_docx(file_path)
try:
if '.' not in filename:
raise ValueError("Unsupported file extension")
ext = filename.rsplit('.', 1)[1].lower()

with open(file_path, 'rb') as f:
header = f.read(512)

if ext == 'pdf':
if not header.startswith(b"%PDF"):
raise ValueError("Invalid file content: PDF signature not found")
content = self.extract_text_from_pdf(file_path)

elif ext == 'docx':
if not header.startswith(b"PK\x03\x04"):
raise ValueError("Invalid file content: DOCX signature not found")
content = self.extract_text_from_docx(file_path)
Comment thread
coderabbitai[bot] marked this conversation as resolved.

elif ext == 'txt':
try:
header.decode('utf-8')
except UnicodeDecodeError:
raise ValueError("Invalid file content: TXT must be valid UTF-8 text")
with open(file_path, 'r', encoding='utf-8') as f:
content = f.read()
Comment on lines +416 to +422

@coderabbitai coderabbitai Bot Apr 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Validate the full text file, not just the first 512 bytes.

Lines 407-410 decode only the header. A file with valid UTF-8 in the first 512 bytes and invalid bytes later will pass this check and then fail during f.read(), bypassing the intended ValueError path. Decode the full payload once and raise ValueError on any UnicodeDecodeError.

Suggested fix 
             elif ext == 'txt':
-                try:
-                    header.decode('utf-8')
-                except UnicodeDecodeError:
-                    raise ValueError("Invalid file content: TXT must be valid UTF-8 text")
-                with open(file_path, 'r', encoding='utf-8') as f:
-                    content = f.read()
+                with open(file_path, 'rb') as f:
+                    raw = f.read()
+                try:
+                    content = raw.decode('utf-8')
+                except UnicodeDecodeError as exc:
+                    raise ValueError("Invalid file content: TXT must be valid UTF-8 text") from exc
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
elif ext == 'txt':
try:
header.decode('utf-8')
except UnicodeDecodeError:
raise ValueError("Invalid file content: TXT must be valid UTF-8 text")
with open(file_path, 'r', encoding='utf-8') as f:
content = f.read()
elif ext == 'txt':
with open(file_path, 'rb') as f:
raw = f.read()
try:
content = raw.decode('utf-8')
except UnicodeDecodeError as exc:
raise ValueError("Invalid file content: TXT must be valid UTF-8 text") from exc
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@backend/Generator/main.py` around lines 406 - 412, The current TXT handling
only decodes the 512-byte header (header) which misses invalid UTF-8 later in
the file; change the logic for the ext == 'txt' branch to read the full file
bytes from file_path (open in 'rb'), attempt to decode the entire payload with
utf-8 and if a UnicodeDecodeError occurs raise ValueError("Invalid file content:
TXT must be valid UTF-8 text"); on successful decode assign the decoded string
to content (or use it when opening as text) so the full file is validated before
any further processing.

else:
raise ValueError("Unsupported file extension")

finally:
if os.path.exists(file_path):
os.remove(file_path)
Comment thread
coderabbitai[bot] marked this conversation as resolved.

os.remove(file_path)
return content

class QuestionGenerator:
Expand Down