Secure File and Input Processing
Secure file processing begins by minimizing what the application accepts and what the processing worker can do. Define the necessary formats, sizes, names, schema and output behavior before selecting a parser or extraction method.
Treat an upload as untrusted throughout its lifecycle. A plausible extension, declared content type or successful parse does not establish that its contents are appropriate, that processing stays within resource limits or that another user may retrieve the result.
TL;DR
- Accept a minimal format/schema contract and verify actual content rather than trusting labels.
- Prefer generated storage identities and avoid using supplied paths as filesystem authority.
- Bound compressed input, expanded content, nesting, member counts and processing resources.
- Avoid executable object deserialization and unsafe parser features for untrusted input.
- Keep quarantine, processing, promotion and retrieval authorization as separate states.
Quick Example
This complete Python 3.12 example accepts a deliberately narrow ZIP contract: exactly one regular, unencrypted member named manifest.json, bounded input/content, and a JSON object containing one integer count. It reads bytes in memory and extracts no files. The limits are teaching values, not recommended production defaults.
The valid sample returns count 3. Extend the tests to reject unsafe or unexpected names, duplicate members or JSON keys, symlink metadata, invalid schemas, types, ranges, or UTF-8, and size-bound violations. This is not a general archive extractor, upload endpoint, complete malformed-format test or process sandbox. orig_filename is an undocumented zipfile attribute, so check compatibility with your Python version before relying on it.
Core Concepts
Labels are claims, not authority
Uploaded names and Content-Type values are supplied by a caller. OWASP recommends format allowlists, generated filenames, size limits and layered content checks; signatures or MIME checks alone are not sufficient. Associate the accepted type with the parser and output behavior actually required. Upload controls
Parsing is not trusted execution
Avoid unpickling untrusted data or invoking loaders that instantiate arbitrary objects. OWASP specifically warns that Python pickle loading can execute code and distinguishes safe YAML loading from unsafe constructors. A data-only format still needs schema and resource controls. Deserialization guidance
Archives introduce another naming system
Archive members carry their own names, metadata and expansion behavior. Python warns against unreviewed extraction and notes that its archive Path interface does not sanitize member paths. The example avoids extraction entirely and admits one exact logical name. zipfile cautions
Design the lifecycle before accepting bytes
Identify who can upload, what purpose the upload serves, which worker interprets it and how an accepted result becomes visible. Keep the uploaded object quarantined until the required checks complete, and record which version of the processing policy produced the result.
Store supplied filenames only as display metadata when needed, with appropriate encoding. Do not concatenate them into authoritative paths or command strings. Keep storage outside an executable web path or on a segregated service according to the architecture. A generated name reduces naming risk but does not replace retrieval authorization.
Choose what happens if processing fails midway. Partial outputs should not appear as validated results. Define retry identity, cleanup, quarantine retention and owner notification without exposing raw sensitive content in an error message.
Bound archives and parsers at the right stage
Check incoming byte limits before buffering large bodies. For archives, bound member count, total expanded bytes, individual outputs, nesting and supported compression/encryption features. Declared metadata is useful for early rejection; also enforce actual read/output limits while processing.
If extraction is genuinely required, define a destination policy for the actual filesystem. Account for absolute paths, traversal, alternate separators, drive/UNC forms, duplicate names, case/Unicode aliases, symlinks, hard links and pre-existing destinations as applicable. Check-then-open path validation can race with a changed filesystem; use platform-appropriate safe creation and isolation rather than claiming a string check solves every case.
Avoid recursively opening nested archives without a shared resource budget. A compressed-size limit alone cannot describe expansion cost. A parser can also consume excessive CPU, memory or recursion depth even when the final decoded object looks small. Process isolation and deadlines address a different boundary from filename validation.
Turn off parser features the format does not require, such as external entity or network/resource resolution where applicable. Keep parsers updated through the project’s supported dependency process. Test representative malformed, truncated and oversized input in an isolated environment; do not treat the small negative fixture as exhaustive fuzzing.
For JSON, define required/allowed keys, types, ranges and duplicate-key policy. The example explicitly rejects boolean counts and repeated keys rather than relying on permissive type coercion or last-value selection. Business meaning still requires the consumer’s own rules.
Verify outputs and retrieval behavior
A scanner result and a schema result answer different questions. Malware scanning may be useful for the application’s risk model, but it does not prove that document rendering, image decoding or downstream conversion is safe. Keep processing privileges and resource limits even when a scan reports no finding.
Review where scanners and converters send data. An external scanning service can create another copy and disclosure boundary. Use only authorized destinations and the required data-handling controls.
For accepted output, set response handling according to intended use. Downloads, inline display and embedded content have different browser consequences. Preserve the correct content type and authorization, and test that one account cannot retrieve another account’s object by changing an identifier.
Define an audit record that is useful without retaining every rejected byte forever. Keep object/policy identity, outcome, relevant limits and controlled diagnostic references. Test cleanup and recovery under partial failure so rejected or timed-out work cannot quietly become a published result.
Best Practices
Minimize the accepted contract
Prefer a small data schema over a general object loader or arbitrary extraction. Admit additional formats only when a real use case and review justify them.
Separate metadata and actual resource checks
Use early rejection plus limits during reading and transformation. Include worker privileges, time, memory and output scope.
Verify the full lifecycle
Test quarantine, rejection, partial failure, promotion and authorized retrieval. An accepted parser result is not the end of the security review.
Common Mistakes
Bad: Trusting an uploaded filename or MIME label.
Correct: Use generated storage identities and validate the actual format under a minimal contract.
Bad: Checking compressed size but not expansion or worker cost.
Correct: Bound actual output and processing resources, including nested operations.
Bad: Treating a resolved path prefix check as universally race-safe.
Correct: Use filesystem-aware safe creation and isolated processing suited to the deployment.
FAQ
Does the example extract files safely?
It extracts none. Avoiding filesystem extraction is part of its narrow teaching contract, not proof of a reusable extraction implementation.
Does JSON remove every processing risk?
No. It avoids arbitrary object construction under an ordinary data parser, but still needs schema, depth/resource limits and correct consumer behavior.
Does a clean scan mean an upload is safe to publish?
No. Required content checks, processing isolation, output handling and retrieval authorization remain separate decisions.
Related Topics
- Security — Place file handling within the application threat model.
- Insecure Deserialization — Understand executable object-loading risks.
- Hash Collisions and Adversarial Inputs — Bound parser and data-structure work before accepting large objects.