{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Reproduce the Mexela proxy route benchmark offline\n", "\n", "Download **mexela-proxy-route-observations-20261011.csv** beside this notebook and use that folder as the working directory. Opening the notebook needs a notebook environment such as Jupyter; the analysis itself uses Python's standard library only, with no network calls, credentials or package installs. Run the code cells in order. This notebook contains no new measurements.\n", "\n", "Observed UTC window: **2026-10-11T19:21:17+00:00** to **2026-10-11T19:33:07+00:00**. Original client: PHP 7.0.33, libcurl 7.47.0, Linux. Five opaque routes and a direct control, 20 rounds, 30-second pauses, fixed sequential order, fresh connections, 10-second total timeout and 5-second connect timeout; redirects off and TLS validation on.\n", "\n", "CSV has one row per attempt. The literal **null** means not available/not applicable; **false** and **0** remain distinct. Timing fields use milliseconds; **tls_complete_ms** is the recorded cumulative TLS-completion timing, not a separately calculated TLS phase. There are 120 attempts: 20 qualified direct controls, 98 qualified proxy checks, and 2 proxy failures. The CSV SHA-256 pins the exact observations; the original sanitized JSON SHA-256 is **b5293725fb508d53b9e1827326ea5fde5da92ab06d9518685f0f9e95d480d01d**.\n", "\n", "Source and attribution: [Mexela benchmark](https://mexela.com/blog/proxy-performance-case-study/) and [sanitized JSON](https://mexela.com/blog/wp-content/uploads/2026/10/mexela-proxy-route-observations-20261011.json). Original copyright and attribution terms remain in effect; this download does not grant a broader license.\n" ], "id": "mexela-analysis-01" }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "import csv\n", "import json\n", "import hashlib\n", "import statistics\n", "import math\n", "from datetime import datetime, timedelta\n", "from collections import Counter\n", "from pathlib import Path\n", "\n", "CSV_PATH = Path('mexela-proxy-route-observations-20261011.csv')\n", "EXPECTED_CSV_SHA256 = 'd8c5c79d5d1a203cad4bd8a3c533b564202973fd5e2553780e91ba9c13f72475'\n", "STARTED_UTC = '2026-10-11T19:21:17+00:00'\n", "FINISHED_UTC = '2026-10-11T19:33:07+00:00'\n", "FIELDS = [\"route\",\"round\",\"started_utc\",\"finished_utc\",\"http_status\",\"curl_errno\",\"tls_verify_result\",\"qualified\",\"checker_timestamp_fresh\",\"proxy_peer_matches_configured\",\"observed_exit_matches_configured\",\"observed_exit_differs_direct\",\"total_ms\",\"ttfb_ms\",\"connect_ms\",\"tls_complete_ms\"]\n", "ROUTES = ['direct', 'route-1', 'route-2', 'route-3', 'route-4', 'route-5']\n", "INTEGER_FIELDS = {'round', 'http_status', 'curl_errno', 'tls_verify_result'}\n", "BOOLEAN_FIELDS = {'qualified', 'checker_timestamp_fresh', 'proxy_peer_matches_configured', 'observed_exit_matches_configured', 'observed_exit_differs_direct'}\n", "TIMING_FIELDS = {'total_ms', 'ttfb_ms', 'connect_ms', 'tls_complete_ms'}\n", "NULLABLE_FIELDS = {'proxy_peer_matches_configured', 'observed_exit_matches_configured', 'observed_exit_differs_direct', 'tls_complete_ms'}\n" ], "id": "mexela-analysis-02" }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "# Read only the local CSV. A changed measurement must not silently pass.\n", "if hashlib.sha256(CSV_PATH.read_bytes()).hexdigest() != EXPECTED_CSV_SHA256:\n", " raise ValueError('CSV SHA-256 mismatch: expected the original immutable download')\n", "with CSV_PATH.open(encoding='utf-8', newline='') as handle:\n", " reader = csv.DictReader(handle)\n", " if reader.fieldnames != FIELDS:\n", " raise ValueError('Unexpected CSV fields')\n", " raw_rows = list(reader)\n", "if len(raw_rows) != 120:\n", " raise ValueError('Expected 120 attempts, including failed attempts')\n", "\n", "rows = []\n", "seen = set()\n", "window_start, window_finish = datetime.fromisoformat(STARTED_UTC), datetime.fromisoformat(FINISHED_UTC)\n", "for raw in raw_rows:\n", " if set(raw) != set(FIELDS) or any(value is None or value == '' for value in raw.values()):\n", " raise ValueError('Missing or extra CSV value; null must be explicit')\n", " row = {}\n", " for key, value in raw.items():\n", " if value == 'null':\n", " if key not in NULLABLE_FIELDS:\n", " raise ValueError('Unexpected null in ' + key)\n", " row[key] = None\n", " elif key in BOOLEAN_FIELDS:\n", " if value not in ('true', 'false'):\n", " raise ValueError('Invalid boolean in ' + key)\n", " row[key] = value == 'true'\n", " elif key in INTEGER_FIELDS:\n", " row[key] = int(value)\n", " if row[key] < 0:\n", " raise ValueError('Negative integer in ' + key)\n", " elif key in TIMING_FIELDS:\n", " row[key] = float(value)\n", " if not math.isfinite(row[key]) or row[key] < 0:\n", " raise ValueError('Invalid timing in ' + key)\n", " else:\n", " row[key] = value\n", " if row['route'] not in ROUTES or row['round'] not in range(1, 21):\n", " raise ValueError('Invalid route or round')\n", " identity = (row['route'], row['round'])\n", " if identity in seen:\n", " raise ValueError('Duplicate route/round')\n", " seen.add(identity)\n", " start, finish = datetime.fromisoformat(row['started_utc']), datetime.fromisoformat(row['finished_utc'])\n", " if start.utcoffset() != timedelta(0) or finish.utcoffset() != timedelta(0):\n", " raise ValueError('Timestamps must retain UTC offsets')\n", " if not window_start <= start <= finish <= window_finish:\n", " raise ValueError('Timestamp outside original UTC window')\n", " proxy_flags = [row[key] for key in ('proxy_peer_matches_configured', 'observed_exit_matches_configured', 'observed_exit_differs_direct')]\n", " if row['route'] == 'direct' and any(flag is not None for flag in proxy_flags):\n", " raise ValueError('Direct control has proxy-specific flags')\n", " if row['route'] != 'direct' and any(flag is None for flag in proxy_flags):\n", " raise ValueError('Missing proxy qualification flag')\n", " if row['qualified'] and (row['http_status'] != 200 or row['curl_errno'] != 0 or row['tls_verify_result'] != 0 or not row['checker_timestamp_fresh'] or (row['route'] != 'direct' and not all(proxy_flags))):\n", " raise ValueError('Inconsistent qualified-success flags')\n", " rows.append(row)\n", "if seen != {(route, round_number) for route in ROUTES for round_number in range(1, 21)}:\n", " raise ValueError('Incomplete route/round coverage')\n", "if sum(row['qualified'] for row in rows if row['route'] == 'direct') != 20 or sum(row['qualified'] for row in rows if row['route'] != 'direct') != 98:\n", " raise ValueError('Original qualified/failure counts changed')\n", "print('Verified SHA-256 and 120 complete attempt rows; nulls, UTC scope and failures retained.')\n" ], "id": "mexela-analysis-03" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Qualified timing summaries with all attempts retained\n", "\n", "Median uses the actual qualified successful total times. p95 uses nearest rank **ceil(0.95 × n)**. At only 18–20 successful samples per route, p95 is a coarse order statistic, not a precise population-tail estimate. Failed durations remain in the CSV and in the error rows; they do not enter successful timing quantiles. Success fractions retain the full 20-attempt denominator per route.\n" ], "id": "mexela-analysis-04" }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "summary = {}\n", "for route in ROUTES:\n", " attempts = [row for row in rows if row['route'] == route]\n", " times = sorted(row['total_ms'] for row in attempts if row['qualified'])\n", " summary[route] = {\n", " 'attempts': len(attempts),\n", " 'qualified_successes': len(times),\n", " 'failures': len(attempts) - len(times),\n", " 'success_fraction': len(times) / len(attempts),\n", " 'http_status_counts': dict(sorted(Counter(str(row['http_status']) for row in attempts).items())),\n", " 'curl_error_counts': dict(sorted(Counter(str(row['curl_errno']) for row in attempts).items())),\n", " 'successful_total_ms': {\n", " 'sample_count': len(times),\n", " # Original timings have 0.001 ms precision; an even median adds one decimal.\n", " 'median': round(statistics.median(times), 4) if times else None,\n", " 'p95': times[math.ceil(0.95 * len(times)) - 1] if times else None\n", " }\n", " }\n", "print(json.dumps(summary, indent=2))\n" ], "id": "mexela-analysis-05" }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "# Actual unqualified rows, including failed durations and recorded status/error codes.\n", "error_rows = [row for row in rows if not row['qualified']]\n", "print(json.dumps(error_rows, indent=2))\n" ], "id": "mexela-analysis-06" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Limits and interpretation\n", "\n", "- Dedicated human-authorized test assignments only, never customer assignments\n", "- One client host, one owned HTTPS destination, fresh connections and fixed route order\n", "- Client and exit geography, throughput, DNS leaks and anonymity were not measured\n", "- Spaced successful checks do not establish continuous uptime or future availability\n", "- No credentials, endpoint identities, observed source addresses or response/error bodies retained\n", "\n", "The direct control is a baseline for this client/destination window. Fixed sequential order is not a simultaneous client comparison. Opaque route labels convey no country, provider ranking or customer assignment. Original timing observations and limitations remain unchanged.\n", "\n", "## Optional standalone SVG\n", "\n", "Leave **WRITE_SVG = False** to create no files. Set it to **True** only when you want a local chart at the explicit filename below. The plot shows qualified total-time median and coarse p95, with failures retained in labels; it does not establish uptime or geography.\n" ], "id": "mexela-analysis-07" }, { "cell_type": "code", "metadata": {}, "execution_count": null, "outputs": [], "source": [ "WRITE_SVG = False\n", "SVG_PATH = Path('mexela-proxy-benchmark-summary-20261011.svg')\n", "if WRITE_SVG:\n", " if SVG_PATH.name.startswith('_'):\n", " raise ValueError('Use a filename that does not start with an underscore')\n", " maximum = max(item['successful_total_ms']['p95'] for item in summary.values())\n", " parts = ['',\n", " 'Mexela qualified total-time summaries',\n", " 'Observed 11 October 2026 UTC. Median and coarse nearest-rank p95 in milliseconds; failures excluded from timings and retained in labels.',\n", " '',\n", " '',\n", " 'Qualified total time (ms): blue median, orange coarse p95']\n", " for index, route in enumerate(ROUTES):\n", " item = summary[route]\n", " y = 65 + index * 52\n", " median, p95 = item['successful_total_ms']['median'], item['successful_total_ms']['p95']\n", " parts.append(f'{route}: {item[\"qualified_successes\"]}/{item[\"attempts\"]}, {item[\"failures\"]} failed')\n", " parts.append(f'')\n", " parts.append(f'')\n", " parts.append(f'{median:.4f} / {p95:.3f} ms')\n", " parts.extend(['One client and destination; 20 rounds; failures retained in denominators.', ''])\n", " SVG_PATH.write_text(''.join(parts), encoding='utf-8')\n", " print('Wrote', SVG_PATH)\n" ], "id": "mexela-analysis-08" } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.12" } }, "nbformat": 4, "nbformat_minor": 5 }