Web2pdf

# 🌐 Web2PDF – Automated Web Page Archiver

Web2PDF is a Python-based tool that automatically captures multiple web pages, converts them into high-fidelity PDFs (preserving images, styles, and hyperlinks), and merges them into a single, well-formatted document.

It uses Playwright (headless Chromium) for rendering and pikepdf for PDF merging.

---

🚀 Features

• 🧭 Automatically fetches and renders web pages
• 🖨️ Exports each page as a properly formatted PDF (with images, CSS, and links)
• 📚 Merges all individual PDFs into one combined file
• ⚡ Asynchronous processing – fast and safe parallel rendering
• 🔄 Automatic retry mechanism for failed pages
• 🪶 Simple configuration — just a text file of URLs
• 🧹 Automatic cleanup of temporary files

---

📸 Demo

Here's an example of converting the DevsinC website to PDF:

Input (urls.txt):

https://www.devsinc.com/

Command:

python app_async.py urls.txt demo.pdf

Output:

📄 The generated demo.pdf contains a high-fidelity capture of the website with all images, styling, and layout preserved.

---

🧩 Requirements

1. Install Python 3.9 or higher

Check your version:

python --version

2. Clone or download this repository

git clone https://github.com/RasikhAli/Web2PDF.git
cd Web2PDF

3. Create and activate a virtual environment

python -m venv venv

On Windows:

venv\Scripts\activate

On macOS/Linux:

source venv/bin/activate

4. Install dependencies

pip install -r requirements.txt

requirements.txt contents:

playwright
pikepdf
tqdm

Then install Playwright's browser binaries:

python -m playwright install 
playwright install chromium

---

📄 Usage

1. Prepare your urls.txt

Create a text file with one URL per line:

https://www.devsinc.com/
https://www.wikipedia.org
https://www.python.org
https://github.com/RasikhAli

Blank lines and lines starting with # are ignored.

2. Run the script

python app_async.py urls.txt output.pdf

• The script will create temporary per-page PDFs.
• When all are rendered, it merges them into the single file output.pdf.
• Temporary files are automatically cleaned up after merging.

Example output:

Temporary directory: C:\Users\you\AppData\Local\Temp\web2pdf_abc123
Saved https://www.devsinc.com/ -> 001_www.devsinc.com.pdf
Merging the following PDFs:
  - C:\Users\you\AppData\Local\Temp\web2pdf_abc123\001_www.devsinc.com.pdf
Merged PDF saved to output.pdf

---

⚙️ Configuration

You can customize rendering settings inside app_async.py:

| Setting | Description | Default | | ---------------------------- | ----------------------------------------------------- | -------- | | PDF_SAVE_OPTIONS["format"] | PDF page size (A4, A3, Letter, etc.) | "A3" | | WAIT_AFTER_LOAD | Wait time (seconds) after page load (for lazy images) | 1 | | CONCURRENCY | Max number of pages rendered in parallel | 4 | | RETRY_COUNT | Retries per URL on failure | 2 | | NAVIGATION_TIMEOUT_MS | Max time (ms) to wait for a page to load | 30000 |

---

🧠 Notes & Tips

• Each webpage is opened in a fresh headless Chromium browser context — ideal for clean captures.
• The tool preserves images, CSS styling, and hyperlinks (where possible).
• Temporary PDF files are stored in your system's temp folder and automatically cleaned up.
• If a page fails to render, it will retry up to RETRY_COUNT times before skipping.
• The async version (app_async.py) is recommended for better performance and reliability.

---

🧰 Quick Start Example

# 1️⃣ Create venv & install dependencies
python -m venv venv
venv\Scripts\activate  # On Windows
# source venv/bin/activate  # On macOS/Linux

pip install -r requirements.txt
playwright install chromium

# 2️⃣ Prepare list of URLs
echo https://www.devsinc.com/ > urls.txt

# 3️⃣ Run the script
python app_async.py urls.txt output.pdf

---

📦 Project Structure

Web2PDF/
├── app_async.py          # Main async script (recommended)
├── urls.txt              # List of URLs to convert
├── requirements.txt      # Python dependencies
├── demo.pdf             # Example output
├── README.md            # This file
└── venv/                # Virtual environment (created by you)

---

🔧 Troubleshooting

Issue: "playwright not found"

Solution: Install Playwright browsers:

python -m playwright install 
playwright install chromium

Issue: "Page timeout"

Solution: Increase NAVIGATION_TIMEOUT_MS in app_async.py:

NAVIGATION_TIMEOUT_MS = 60000  # 60 seconds

Issue: "Memory issues with many URLs"

Solution: Reduce CONCURRENCY in app_async.py:

CONCURRENCY = 2  # Process 2 pages at a time

---

🧑‍💻 Author

Developed by Rasikh Ali
Freelance Python Developer
📧 rasikhali1234@gmail.com
🌐 LinkedIn Profile

---

📄 License

MIT License — feel free to use, modify, and distribute.

---

🤝 Contributing

Contributions, issues, and feature requests are welcome!
Feel free to check the issues page or submit a pull request.

---

⭐ Show Your Support

If this project helped you, please give it a ⭐ on GitHub!