Scale

Overview

This guide covers how to run Kernel in production at scale — which architecture to build around browser creation, and when to reach for a browser pool. It assumes you’re comfortable creating and controlling browsers; for the mechanics of standing up a pool and acquiring from it, see Browser Pools.

Why a browser pool

A browser pool keeps a set of identically-configured browsers ready for immediate use. Compared to creating browsers on demand, it gives you:

Low-latency acquisition — the browser is already booted with your configuration applied (including settings like custom viewports, extensions, and kiosk-mode live view that otherwise restart Chromium on a fresh browser), so acquire hands you one that’s ready to drive.
Reserved, pre-configured capacity — a fixed set of browsers on your exact configuration, ready before traffic arrives.
Higher creation throughput — acquiring from a pool isn’t subject to the rate limit on browsers.create() that high-volume workloads hit.

The tradeoff: a browser pool counts against your concurrency limit whether or not its browsers are currently acquired — a pool sized to 40 holds 40 of your limit. Idle pooled browsers aren’t billed, but they hold the slot.

When to use a pool vs on-demand

Reach for a browser pool when:

you’re running the same workload repeatedly, in production
acquisition latency matters — a cold start is unacceptable (for example, a synchronous, user-facing action)
traffic is steady or high-frequency enough to keep the browser pool utilized
you’re hitting the browsers.create() rate limit at volume

Stick with on-demand browsers.create() when:

volume is low, bursty, one-off, or you’re still developing
each session needs a different configuration (a pool is one fixed config)
you need a GPU browser (not available in pools)

Concurrency and request patterns are how you size a pool once you’ve decided to use one — not a threshold that gates whether pools are worth it. Even a small pool pays off when acquisition latency matters and demand is steady.

Sizing

Watch available_count and target 10–20% available under normal load, resizing before traffic peaks rather than during them. See Sizing a browser pool for the full guidance.

Architecture patterns

Direct browser creation (POC)

For proof-of-concept work and early production systems with modest concurrency needs, creating browsers on-demand is the simplest approach. When to use:

Low or unpredictable volume
Infrequent or one-off workloads
Early development and testing

Example

import Kernel from '@onkernel/sdk';
import { chromium } from 'playwright';

const kernel = new Kernel();

async function processTask(taskData: any) {
  // Create browser with extended timeout for long tasks
  const session = await kernel.browsers.create({
    stealth: true,
    timeout_seconds: 3600, // Destroy browser after 1 hour of inactivity
  });

  try {
    const browser = await chromium.connectOverCDP(session.cdp_ws_url);
    const context = browser.contexts()[0];
    const page = context.pages()[0];

    // Your automation logic here
    await page.goto(taskData.url);
    // ... perform work ...

    return { success: true, data: /* results */ };
  } catch (error) {
    console.error('Task failed:', error);
    return { success: false, error: error.message };
  } finally {
    // Always clean up
    await kernel.browsers.deleteByID(session.session_id);
  }
}

Single browser pool (scaling)

For production systems with consistent, high-frequency workloads, a browser pool allows you to access higher concurrency plus predictable performance. When to use:

Consistent, high-frequency workloads on a fixed configuration
Steady request patterns, or latency-sensitive acquisition

Example

import Kernel from '@onkernel/sdk';
import { chromium } from 'playwright';

const kernel = new Kernel();
const POOL_NAME = 'production-pool';

// Initialize pool once (typically in deployment/startup)
async function initializePool() {
  await kernel.browserPools.create({
    name: POOL_NAME,
    size: 25, // Balance cost and availability
    timeout_seconds: 300, // Destroy browsers after 5 minutes of inactivity
    stealth: true,
    headless: false, // headless: true for cost savings if no live view needed
  });
}

async function processTask(taskData: any) {
  let session;
  
  try {
    // Acquire browser (returns immediately if available)
    session = await kernel.browserPools.acquire(POOL_NAME, {
      acquire_timeout_seconds: 30, // Wait up to 30s for availability
    });

    const browser = await chromium.connectOverCDP(session.cdp_ws_url);
    const context = browser.contexts()[0];
    const page = context.pages()[0];

    // Perform work
    await page.goto(taskData.url);
    // ... automation logic ...

    return { success: true, data: /* results */ };
  } catch (error) {
    console.error('Task failed:', error);
    return { success: false, error: error.message };
  } finally {
    // Critical: Always release back to pool
    if (session) {
      await kernel.browserPools.release(POOL_NAME, {
        session_id: session.session_id,
        reuse: true, // Reuse for efficiency
      });
    }
  }
}

Key considerations:

Pool size should match your typical concurrency
Always release browsers in a finally block to prevent browser pool exhaustion
Set acquire_timeout_seconds based on your SLA requirements

Queue-based processing (high scale)

For systems exceeding browser pool capacity or with unpredictable bursts, implement a task queue to manage workloads gracefully. When to use:

Request volume exceeds a single browser pool’s capacity
Highly variable traffic patterns
Need to prioritize certain tasks
Want to decouple request ingestion from processing

Example

import { Queue } from 'bullmq'; // or any queue system
import Kernel from '@onkernel/sdk';

const kernel = new Kernel();
const POOL_NAME = 'production-pool';
const POOL_SIZE = 50;

// Task queue configuration
const taskQueue = new Queue('browser-tasks', {
  connection: { /* Redis config */ }
});

// Worker that processes tasks
async function startWorker() {
  const worker = new Worker('browser-tasks', async (job) => {
    let session;
    
    try {
      // Acquire browser with reasonable timeout
      session = await kernel.browserPools.acquire(POOL_NAME, {
        acquire_timeout_seconds: 120,
      });

      const browser = await chromium.connectOverCDP(session.cdp_ws_url);
      const context = browser.contexts()[0];
      const page = context.pages()[0];

      // Process job data
      await page.goto(job.data.url);
      const result = await page.evaluate(() => /* extract data */);

      return { success: true, data: result };
    } catch (error) {
      // Handle errors with retry logic
      if (error.message.includes('timeout')) {
        throw new Error('RETRY'); // BullMQ will retry
      }
      throw error;
    } finally {
      if (session) {
        await kernel.browserPools.release(POOL_NAME, {
          session_id: session.session_id,
          reuse: true,
        });
      }
    }
  }, {
    connection: { /* Redis config */ },
    concurrency: POOL_SIZE, // Match pool size
  });

  return worker;
}

// Add tasks to queue
async function submitTask(taskData: any, priority?: number) {
  await taskQueue.add('process', taskData, {
    priority: priority || 5,
    attempts: 3,
    backoff: {
      type: 'exponential',
      delay: 2000,
    },
  });
}

Queue-specific considerations:

Set worker concurrency to match or slightly exceed browser pool size
Implement proper retry logic for transient failures
Monitor queue depth to scale browser pools dynamically
Use priority queues for different SLAs

Overview

Working with your browser

deploying your agent

Agent Skills

Integrations

Migrations

Community

Info

Overview

Why a browser pool

When to use a pool vs on-demand

Sizing

Architecture patterns

Direct browser creation (POC)

Single browser pool (scaling)

Queue-based processing (high scale)

​Overview

​Why a browser pool

​When to use a pool vs on-demand

​Sizing

​Architecture patterns

​Direct browser creation (POC)

​Single browser pool (scaling)

​Queue-based processing (high scale)

Overview

Why a browser pool

When to use a pool vs on-demand

Sizing

Architecture patterns

Direct browser creation (POC)

Single browser pool (scaling)

Queue-based processing (high scale)