Property Information Extractor

A Python-based web scraper that extracts property information from popular UAE real estate platforms: Bayut, Dubizzle, and PropertyFinder.

Features

• 🏢 Multi-Platform Support: Extract property details from Bayut, Dubizzle, and PropertyFinder
• 🔍 Comprehensive Data Extraction: Captures unit numbers, building names, locations, property types, room counts, and owner information
• 🤖 Smart Scraping: Uses both static (requests) and dynamic (Selenium) scraping methods
• ⚡ Persistent Browser Session: Chrome browser stays open throughout the session for faster extraction
• 💾 JSON Export: Save extracted property information to JSON files
• 🎯 Interactive CLI: User-friendly command-line interface for easy operation
• 🛡️ Anti-Detection: Built-in measures to avoid bot detection

Extracted Information

The tool extracts the following property details:

• Unit Number: Actual unit identifier (e.g., "Unit 1234", "A-505")
• Building Name: Name of the building/complex
• Property Location: Hierarchical location (e.g., "Dubai > Downtown > Burj Khalifa")
• Property Subtype: Type of property (Apartment, Villa, Townhouse, Penthouse, Studio)
• Rooms: Number of bedrooms or "Studio"
• Property Size: Area in square feet
• Reference Number: Property reference/permit number
• Listing ID: Platform-specific listing identifier
• Owner Information: Agent/owner name, phone number, and additional details

Installation

Prerequisites

• Python 3.8 or higher
• Google Chrome browser (for Selenium-based scraping)
• pip (Python package manager)

Setup

1. Create a virtual environment (recommended):

   python -m venv venv
   

2. Activate the virtual environment:

   • Windows:

     venv\Scripts\activate
     

   • macOS/Linux:

     source venv/bin/activate
     

3. Install required dependencies:

   pip install -r requirements.txt
   

   The tool will automatically download and configure the appropriate ChromeDriver version.

Usage

Interactive Mode

Run the application in interactive mode:

python app.py

You'll be prompted to enter property URLs. The tool will:

1. Detect the platform automatically
2. Extract property information
3. Display the results
4. Offer to save the data to a JSON file

Example Session

============================================================
Property Information Extractor
Supports: Bayut, Dubizzle, PropertyFinder
============================================================
Type 'exit' or 'quit' to stop the application
============================================================

Initializing Chrome browser...
ChromeDriver and Chrome versions are compatible.
✓ Chrome browser initialized and ready

Enter property URL (or 'exit' to quit): https://www.propertyfinder.ae/...

Detected platform: PROPERTYFINDER
Extracting property information from https://www.propertyfinder.ae/...

============================================================
EXTRACTION SUCCESSFUL
============================================================
Property Information:
--------------------
Unit Number: Unit 1234
Building Name: Burj Khalifa
Property Description: Apartment, Burj Khalifa, Unit 1234
Property Location: Dubai > Downtown Dubai
Property Subtype: Apartment
Rooms: 2
Property Size: 1200 sqft
Reference Number: 71234567890
Listing ID: 12345678

Save to JSON file? (y/n):

Programmatic Usage

You can also use the extractor in your own Python scripts:

from app import extract_property_from_url
from extractors.base_extractor import BaseExtractor

# Initialize browser once for multiple extractions
driver = BaseExtractor.initialize_shared_driver()

try:
    # Extract from multiple URLs using the same browser
    urls = [
        "https://www.bayut.com/property/...",
        "https://www.dubizzle.com/property/...",
        "https://www.propertyfinder.ae/..."
    ]

    for url in urls:
        property_info = extract_property_from_url(url, driver=driver)

        if property_info:
            print(property_info)
            # Access individual fields
            print(f"Building: {property_info.building_name}")
            print(f"Unit: {property_info.unit_number}")

            # Convert to dictionary
            data = property_info.to_dict()
finally:
    # Close browser when done
    BaseExtractor.close_shared_driver()

Project Structure

.
├── app.py                          # Main application entry point
├── src/                            # Source package
│   ├── __init__.py                # Package initialization
│   ├── models.py                  # Data models (PropertyInfo, PropertyOwner)
│   ├── utils.py                   # Utility functions
│   └── config.py                  # Configuration settings
├── extractors/                     # Platform-specific extractors
│   ├── __init__.py
│   ├── base_extractor.py          # Base extractor class
│   ├── bayut_extractor.py         # Bayut.com extractor
│   ├── dubizzle_extractor.py      # Dubizzle.com extractor
│   └── propertyfinder_extractor.py # PropertyFinder.ae extractor
├── requirements.txt                # Python dependencies
├── Chromedriver/                   # Auto-downloaded ChromeDriver
└── README.md                       # This file

Configuration

You can modify settings in src/config.py:

• USE_SELENIUM: Enable/disable Selenium (default: True)
• HEADLESS_MODE: Run browser in headless mode (default: True)
• REQUEST_TIMEOUT: HTTP request timeout in seconds (default: 30)
• USER_AGENT: Custom user agent string

Supported Platforms

| Platform | URL Pattern | Status | |----------|-------------|--------| | Bayut | bayut.com | ✅ Supported | | Dubizzle | dubizzle.com | ✅ Supported | | PropertyFinder | propertyfinder.ae | ✅ Supported |

Dependencies

• requests: HTTP library for static content
• beautifulsoup4: HTML parsing
• lxml: XML/HTML parser
• selenium: Browser automation for dynamic content
• setup-chromedriver: Automatic ChromeDriver management

Troubleshooting

ChromeDriver Issues

• The tool automatically downloads the correct ChromeDriver version
• If you encounter issues, delete the Chromedriver/ folder and restart the application

Extraction Failures

• Some websites may block automated access
• Try running with HEADLESS_MODE = False in src/config.py for debugging
• Check your internet connection
• Verify the URL is correct and accessible

Import Errors

• Ensure you've activated the virtual environment
• Reinstall dependencies: pip install -r requirements.txt

License

This project is for educational and personal use only. Please respect the terms of service of the websites you scrape.

Disclaimer

This tool is provided as-is for educational purposes. Users are responsible for ensuring their use complies with the terms of service of the target websites and applicable laws.