Scope Cross-Browser Testing Realistically

"Test the latest and previous versions of Chrome, Safari, Firefox and Edge, plus real iOS and Android devices, on every screen." Few teams can sustain that on every release. This article covers how to set a realistic scope for cross-browser and real-device checks in the testing phase, and how to catch problems from environments you chose not to test.

Start by giving up on every combination

Browsers, operating systems, devices and screen sizes multiply into hundreds of combinations. Try to cover them all and every check ends up shallow. Split the work first: what you verify, and what you catch through user reports.

Build the target matrix from share and risk

Steps

1. Pull browser-by-OS usage shares from your analytics for the last three months
2. Mark everything up to roughly 90% cumulative share as "required"
3. Add environments used by business-critical users (key customers, internal staff) as "secondary"
4. Write "out of scope" explicitly for the rest

Example matrix

Weight by risk

Even within one environment, weight screens differently. Payment, login and file upload vary a lot between browsers and hurt most when broken, so check them on every required environment. A static explainer page may only need Chrome.

Suspect the usual places first

• Native browser widgets such as date inputs and file pickers
• Newer CSS features (:has(), dvh and so on)
• Layout around the iOS Safari bottom bar and on-screen keyboard
• Permission-bound APIs: media autoplay, clipboard, notifications

Before using a feature, check its support on a site like caniuse.com and decide at implementation time how it should behave where it is unsupported. That is cheaper than retrofitting.

Cover out-of-scope environments with a reporting path

Users exist in the environments you skipped. Set up three things so you notice when they hit trouble.

1. A report flow that always carries environment info: if browser, OS and screen size attach automatically, nobody has to ask "what are you using?"
2. A rule for reports from out-of-scope environments: one line deciding whether you fix, suggest a workaround, or say it is unsupported
3. A monthly look at report counts by environment: if an out-of-scope environment keeps growing, it is time to revisit the matrix

A failure example

A team had marked Safari out of scope. Then several reports arrived saying the submit button was hidden off-screen on certain iPhones. Only then did they look at analytics and find that iOS Safari was just over 10% of their users. Make revisiting the matrix against analytics part of your routine, not a one-time decision.

Putting it into practice with Bugoon

The Bugoon widget goes onto your site with a single tag, so QA staff and non-engineers can report from the browser. Each report carries a screenshot with annotations and the steps taken, along with environment details such as browser and screen size, attached automatically. That is the "catch it by report" path described above, ready to use. Reports can flow into GitHub Issues, and the kanban board tracks their status.

If the environment details in reports could be aggregated into a ranking of what users actually run, it would become an even more useful basis for choosing which environments to target.