Understanding TargetCloseError in Puppeteer Cluster with Firefox BiDi

When running automated browser workflows at scale using puppeteer-cluster and Puppeteer's Firefox WebDriver BiDi support, you might occasionally run into mysterious crashes such as:

TargetCloseError: script.callFunction: Target closed

If your worker task includes an explicit cleanup step like await page.close() inside a finally block, you are likely triggering a race condition between your task code and the internal resource management of Puppeteer Cluster. Let’s dive into why this happens and what the proper page lifecycle pattern looks like.

The Core Problem: Lifecycle Ownership

In puppeteer-cluster, the library is strictly designed to govern the full lifecycle of targets (browsers, contexts, and pages) based on your selected concurrency model.

When using Cluster.CONCURRENCY_CONTEXT:

  • Puppeteer Cluster creates an isolated incognito browser context for each worker.
  • It opens a new page within that context and passes it into your task callback via { page }.
  • Once your task promise resolves or rejects, the cluster executes its internal post-task teardown, which cleans up the context and page resources.

If you manually call await page.close() inside the task, you introduce an ownership conflict. When your callback returns, the cluster attempts to perform its standard cleanup and status checks on an already-destroyed browsing context.

Why Firefox WebDriver BiDi is Extra Sensitive

While Chrome DevTools Protocol (CDP) sometimes gracefully swallows operations performed on closed targets, Firefox’s WebDriver BiDi implementation is strict and adheres closely to W3C specifications.

When a browsing context closes in Firefox BiDi:

  • Any pending script evaluations or internal checks (such as script.callFunction or execution context checks invoked by Puppeteer's internal wrappers) fail immediately.
  • Because Puppeteer or the cluster may still be listening for lifecycle hooks or attempting to detach frames, destroying the target out from under the driver throws an unhandled TargetCloseError.

The Solution: Let the Cluster Manage the Page

The solution is simple: never manually close the page or context provided by puppeteer-cluster. Always allow the library to handle opening and closing resources.

Refactored, Safe Code Pattern

Remove the manual page.close() invocation from your task handler:

import { Cluster } from 'puppeteer-cluster';
import puppeteer from 'puppeteer-core';

const cluster = await Cluster.launch({
  puppeteer,
  concurrency: Cluster.CONCURRENCY_CONTEXT,
  maxConcurrency: 4,
  retryLimit: 0,
  puppeteerOptions: {
    browser: 'firefox',
    protocol: 'webDriverBiDi',
    headless: true,
    executablePath: firefoxPath
  }
});

// Task handler: Do NOT manually close the page
await Promise.all(jobs.map(job =>
  cluster.execute(job, async ({ page, data }) => {
    // Process your job safely
    await processJob(page, data);
    // No "finally { await page.close(); }" needed!
  })
));

await cluster.idle();
await cluster.close();

Additional Tips for Firefox BiDi Stability in Puppeteer

If you have removed manual cleanup and still encounter intermittent TargetCloseError exceptions under high concurrency, consider the following stability improvements:

  • Watch Concurrency Limits: Firefox instances running via WebDriver BiDi consume substantial memory and CPU per context. If system resources spike, Firefox may crash or drop the WebSocket connection, mimicking target closure. Lower your maxConcurrency if your host machine struggles with load.
  • Avoid Premature Task Resolution: Ensure that all asynchronous operations on page (like navigations, clicks, and evaluations) are properly awaited before the task function returns. Leaving floating promises running after the task resolves will trigger errors when the cluster closes the page.
  • Prefer domcontentloaded: When working with Firefox BiDi, waiting for networkidle0 can sometimes hang or time out. Using waitUntil: 'domcontentloaded' often yields higher reliability during rapid parallel scraping.

Conclusion

Explicitly calling await page.close() inside a puppeteer-cluster worker is an anti-pattern. Under CONCURRENCY_CONTEXT with Firefox WebDriver BiDi, doing so directly leads to TargetCloseError: script.callFunction: Target closed. Letting the cluster manage the page lifecycle completely eliminates this race condition.