2026-09-30 15:48:03 +05:30

6.7 KiB

Raw Blame History

Browser Monitoring & Notification Agent

A Python application that continuously monitors websites using a real browser installed on your machine. It visits configured sites in round-robin order, captures snapshots (screenshots + metadata), and is designed for future OpenAI-based page understanding and alert notifications.

Features

Browser-first monitoring — Uses Playwright with your installed Chrome, Chromium, Edge, or Firefox
Persistent browser profile — Reuses cookies, sessions, and login state across runs
Round-robin scheduling — Visits one website at a time, sequentially
Snapshots — Screenshots, page title, visible text, HTML, and metadata
Snapshot retention — Configurable cleanup to prevent unbounded disk growth
OpenAI stub — Ready for future page understanding integration
Notification framework — SMS and email abstractions (logging by default)
Error resilience — One site failing does not stop the monitor
Browser recovery — Automatic restart on browser crashes
Graceful shutdown — Clean exit on Ctrl+C

Requirements

Python 3.10+
A supported browser installed locally (Chrome recommended)

Installation

cd browser_automation
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Playwright is used with your installed browser, not a separately downloaded Playwright browser bundle. If auto-detection fails, set browser.executable_path in config.yaml.

Copy environment variable template:

cp .env.example .env

Never commit .env — it is listed in .gitignore.

Browser Configuration

Edit config.yaml:

browser:
  type: chrome          # chrome | chromium | edge | firefox
  executable_path: null # set if auto-detection fails
  headless: false       # false = visible browser window
  profile_directory: "./profiles/default"

Set profile_directory to a dedicated folder (e.g. ./profiles/default).
Run the login launcher to open that profile and sign in manually.
Close the browser or press Ctrl+C when finished.
Start the monitor — sessions and cookies are preserved.

Do not store passwords in config.yaml. Use the browser profile for authentication.

Use login.py to open the monitor profile without starting the monitoring loop:

# Open profile with a blank tab
python login.py

# Open BookMyShow (or any URL)
python login.py --url "https://in.bookmyshow.com"

# Open all enabled websites from config.yaml as tabs
python login.py --websites

# Open a specific configured site by id
python login.py --website-id website_1

The launcher always opens a visible browser window, even if headless: true in config.

Auto-Detection Paths (Linux)

Browser	Typical paths
Chrome	`/usr/bin/google-chrome`, `/usr/bin/google-chrome-stable`
Chromium	`/usr/bin/chromium`, `/usr/bin/chromium-browser`
Edge	`/usr/bin/microsoft-edge`
Firefox	`/usr/bin/firefox`

Configuration

config.yaml controls monitoring behaviour:

monitor:
  round_robin_delay_seconds: 5    # pause between website visits
  page_load_timeout_seconds: 30
  render_wait_seconds: 3          # extra wait for JavaScript rendering
  snapshot_directory: "./snapshots"
  snapshot_retention_count: 50    # max screenshots per website

websites:
  - id: website_1
    name: Website One
    url: "https://example.com"
    enabled: true

notifications:
  sms:
    enabled: false
  email:
    enabled: false

Add as many websites as needed. Disabled sites are skipped.

Usage

Start monitoring

python main.py

Custom config file

python main.py --config config.yaml

Debug logging

python main.py --debug

Test browser setup

python main.py --test-browser
python main.py --test-browser --test-url "https://example.com"

Graceful shutdown

Press Ctrl+C. The monitor stops scheduling new checks, closes the browser, and exits cleanly.

Architecture

Browser → Snapshot → OpenAI Processor (stub) → Condition Engine → Notifications

Layer	Responsibility
OpenAI (future)	Page understanding, value extraction
Python	Scheduling, condition evaluation, notifications
Playwright	Real browser automation

Project Structure

browser_automation/
├── main.py
├── config.yaml
├── monitor/
│   ├── browser.py      # Browser discovery, launch, navigation
│   ├── scheduler.py    # Round-robin loop
│   ├── snapshot.py     # Page capture
│   ├── processor.py    # OpenAI stub + condition evaluation
│   ├── models.py       # PageSnapshot, ProcessingResult, etc.
│   └── config.py       # YAML loading
├── notifications/
│   ├── base.py         # NotificationService abstraction
│   ├── sms.py          # SMS provider stub
│   └── email.py        # SMTP / logging email
├── storage/
│   └── snapshots.py    # Screenshot storage + retention
├── profiles/           # Persistent browser profiles
└── snapshots/          # Captured screenshots

Snapshots

Screenshots are saved per website:

snapshots/
  website_1/
    2026-09-30T13-20-00.png
  website_2/
    2026-09-30T13-20-05.png

Older snapshots are removed automatically when snapshot_retention_count is exceeded.

Notifications

By default, notifications are logged to the console:

SMS WOULD BE SENT:
...

EMAIL WOULD BE SENT:
Subject: ...

Email (SMTP)

Set environment variables in .env:

SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_USER=your-user
SMTP_PASSWORD=your-password
EMAIL_FROM=monitor@example.com
EMAIL_TO=you@example.com

Enable in config.yaml:

notifications:
  email:
    enabled: true

SMS

Configure provider credentials via environment variables (SMS_PROVIDER, SMS_API_KEY, etc.). Enable in config when a provider is implemented.

Error Handling

Navigation timeouts, DNS failures, and page errors are logged per website
The monitor continues to the next site
Browser crashes trigger automatic restart with backoff
Bot/automation blocks are logged and skipped — no CAPTCHA or bypass logic

Security

No credentials in source code or config.yaml
Use .env for secrets (not committed)
No CAPTCHA bypass, Cloudflare bypass, or fingerprint spoofing

Future Extensions

The architecture supports adding:

OpenAI page understanding and condition triggers
Per-site instructions and check intervals
Multiple browser profiles
Alert deduplication and cooldown
Urmi, Twilio, WhatsApp integrations
Dashboard and database storage

6.7 KiB Raw Blame History