Google SERP Monitoring With Node.js, Playwright, and Proxies

A small Google SERP monitor can use Node.js and Playwright to launch Chromium through one authenticated proxy, create an isolated desktop context, request one signed-out query, classify the returned page, extract at most ten organic destination URLs, and emit structured JSON. Treat consent, unusual-traffic, HTTP, no-result, and parser outcomes as data rather than retry instructions. Google markup changes, so a supported API or search-data provider is usually more stable for recurring production monitoring.

The example below is deliberately one query and one navigation. It does not click results, accept consent, rotate addresses, or retry a warning page. That boundary makes the output understandable: each run is either a usable observation or a named failure state. Review the applicable Google terms and obtain approval for any scheduled collection before increasing frequency.

Install one browser and supply four secrets or inputs

Use a maintained Node.js release in an empty project. Run npm init -y, then npm install playwright, followed by npx playwright install chromium. These commands follow the official Playwright library installation, which distinguishes installing the package from installing browser binaries.

Set all four required variables in the shell that will launch the script. In PowerShell, complete example assignments are $Env:PROXY_SERVER='http://proxy.example:3128', $Env:PROXY_USERNAME='account-user', $Env:PROXY_PASSWORD='replace-with-secret', and $Env:QUERY='marlin desk lamp warranty'. Save the program as serp-check.mjs, then run node .\serp-check.mjs. Replace the fictional endpoint and credentials; do not commit them.

Playwright supports HTTP and SOCKS proxy servers at browser launch. Its username and password fields are documented for HTTP proxy authentication; do not assume those credentials work with a SOCKS endpoint. Structured fields avoid parsing HTTP credentials out of a URL. The current option is documented in the BrowserType proxy reference. For route and authentication diagnosis outside this script, use the Playwright proxy setup guide and proxy authentication guide.

Run a complete one-navigation monitor

The program validates configuration before launching. It explicitly creates browser, context, and page, fixes the locale and desktop viewport, and calls page.goto() exactly once. After navigation, it reads the HTTP status, final URL, and body, extracts a bounded set of candidate organic URLs, and then classifies the page from that evidence and the extracted count. Within classification, recognized interruption and no-result states take precedence over generic HTTP or parser states.

import { writeFile } from 'node:fs/promises';
import { chromium } from 'playwright';
const requiredNames = [
  'PROXY_SERVER',
  'PROXY_USERNAME',
  'PROXY_PASSWORD',
  'QUERY',
];
const missingNames = requiredNames.filter((name) => !process.env[name]?.trim());
if (missingNames.length > 0) {
  throw new Error(`Missing required environment variables: ${missingNames.join(', ')}`);
}
const config = {
  proxyServer: process.env.PROXY_SERVER.trim(),
  proxyUsername: process.env.PROXY_USERNAME.trim(),
  proxyPassword: process.env.PROXY_PASSWORD,
  query: process.env.QUERY.trim(),
  outputFile: process.env.OUTPUT_FILE?.trim() || null,
  maxOrganicUrls: 10,
};
let browser;
let context;
let page;
let output;
const startedAt = new Date();
function classifyPage({ status, finalUrl, bodyText, organicCount }) {
  const text = bodyText.toLowerCase();
  if (text.includes('unusual traffic from your computer network')) {
    return { state: 'unusual_traffic', detail: 'Google returned a traffic warning.' };
  }
  if (
    finalUrl.includes('consent.google.') ||
    text.includes('before you continue to google')
  ) {
    return { state: 'consent', detail: 'A consent page interrupted the result.' };
  }
  if (
    text.includes('did not match any documents') ||
    text.includes('no results found for')
  ) {
    return { state: 'no_results', detail: 'Google displayed an explicit empty result.' };
  }
  if (status >= 400) {
    return { state: 'http_error', detail: `HTTP ${status}` };
  }
  if (organicCount === 0) {
    return { state: 'parser_error', detail: 'No supported organic anchors were found.' };
  }
  return { state: 'ok', detail: null };
}
async function extractOrganicUrls(currentPage, limit) {
  const anchors = currentPage.locator('#search a:has(h3)');
  const count = await anchors.count();
  const urls = [];
  for (let index = 0; index < Math.min(count, limit * 3); index += 1) {
    const href = await anchors.nth(index).getAttribute('href');
    if (!href) continue;
    try {
      const parsed = new URL(href, currentPage.url());
      const isGoogleHost = parsed.hostname === 'google.com' || parsed.hostname.endsWith('.google.com');
      if (
        (parsed.protocol === 'http:' || parsed.protocol === 'https:') &&
        !isGoogleHost &&
        !urls.includes(parsed.href)
      ) {
        urls.push(parsed.href);
      }
    } catch {
      // Ignore malformed destinations and preserve the bounded result.
    }
    if (urls.length === limit) break;
  }
  return urls;
}
try {
  browser = await chromium.launch({
    headless: true,
    proxy: {
      server: config.proxyServer,
      username: config.proxyUsername,
      password: config.proxyPassword,
    },
  });
  context = await browser.newContext({
    locale: 'en-US',
    viewport: { width: 1365, height: 900 },
    geolocation: undefined,
  });
  page = await context.newPage();
  const searchUrl = new URL('https://www.google.com/search');
  searchUrl.searchParams.set('q', config.query);
  searchUrl.searchParams.set('hl', 'en');
  const response = await page.goto(searchUrl.href, {
    waitUntil: 'domcontentloaded',
    timeout: 30_000,
  });
  const status = response?.status() ?? 0;
  const finalUrl = page.url();
  const bodyText = await page.locator('body').innerText().catch(() => '');
  const organicUrls = await extractOrganicUrls(page, config.maxOrganicUrls);
  const classification = classifyPage({
    status,
    finalUrl,
    bodyText,
    organicCount: organicUrls.length,
  });
  output = {
    query: config.query,
    requestedAt: startedAt.toISOString(),
    completedAt: new Date().toISOString(),
    finalUrl,
    httpStatus: status,
    state: classification.state,
    detail: classification.detail,
    organicUrls: classification.state === 'ok' ? organicUrls : [],
    organicUrlLimit: config.maxOrganicUrls,
  };
} catch (error) {
  output = {
    query: config.query,
    requestedAt: startedAt.toISOString(),
    completedAt: new Date().toISOString(),
    finalUrl: page?.url() ?? null,
    httpStatus: null,
    state: 'navigation_error',
    detail: error instanceof Error ? error.message : String(error),
    organicUrls: [],
    organicUrlLimit: config.maxOrganicUrls,
  };
  process.exitCode = 1;
} finally {
  await browser?.close();
}
const serialized = JSON.stringify(output, null, 2);
if (config.outputFile) {
  await writeFile(config.outputFile, `${serialized}\n`, 'utf8');
}
console.log(serialized);

The optional OUTPUT_FILE variable writes the same JSON printed to standard output. It is optional and therefore not part of the four-value validation gate. Closing the browser in finally covers successful extraction, navigation failure, and parser failure. Playwright’s Page documentation describes the page, context, navigation, and locator objects used here.

Interpret the state before reading URLs

An ok state means the supported selector found at least one non-Google HTTP destination. It does not certify that every organic result was captured. The list is deduplicated and capped at ten, preventing a layout bug from producing an unbounded file. Keep the cap and selector version beside historical measurements if the script evolves.

consent means the run stopped on a consent surface. unusual_traffic means Google displayed its traffic warning. Neither state should trigger a loop that accepts, rotates, or retries automatically. These semantic states and an explicit no_results message take precedence over a generic HTTP classification, so a 429 traffic warning remains unusual_traffic; the separate httpStatus field still preserves 429. http_error covers a response of 400 or higher only when no recognized semantic page is present. parser_error means the expected anchors were absent. That last distinction prevents a markup change from being recorded as zero rankings.

navigation_error covers timeouts, proxy authentication failure, TLS failure, and other thrown errors before classification completes. Diagnose the route on a permitted neutral endpoint outside the measurement run. Never log the password or embed it in the result. If sessions require a stable country exit for the entire navigation, Mexela’s proxy plans can be compared on country coverage and session persistence.

Expected JSON is intentionally small

A successful run has the following shape. URLs and times are fictional; the strict JSON block exists to document the contract, not to claim a live Google result.

{
  "query": "marlin desk lamp warranty",
  "requestedAt": "2026-07-17T09:00:00.000Z",
  "completedAt": "2026-07-17T09:00:02.410Z",
  "finalUrl": "https://www.google.com/search?q=marlin+desk+lamp+warranty&hl=en",
  "httpStatus": 200,
  "state": "ok",
  "detail": null,
  "organicUrls": [
    "https://example.org/marlin-lamp/warranty",
    "https://docs.example.net/lighting/marlin"
  ],
  "organicUrlLimit": 10
}

Store the entire run object, not just ranks. State, query, final URL, status, and timestamps are necessary to distinguish a real result change from a consent redirect or broken parser. Credentials, cookies, and full page HTML are not part of this schema.

Know when browser monitoring is the wrong dependency

Rendered Google markup and module composition change. A CSS selector that works today can fail without warning, and browser automation adds consent, proxy, browser-binary, and policy dependencies. Recurring production reporting usually benefits from a supported search API or data provider with a documented schema and service terms. Use this browser example for a bounded, approved QA need where rendered evidence is essential.

Keep production frequency conservative, cache equivalent work, and stop on warnings. Google’s current Terms of Service remain authoritative for use of its services; a proxy is routing configuration, not permission to send automated queries.

Questions about the monitor