sdet 3

 1. Your UI automation suite has 3,000 tests. It takes 4 hours in CI, while developers expect feedback within 15 minutes. What would you do?

Scenario

Your team has 3,000 UI regression tests. The application is growing rapidly.

Current situation:

Local execution: ~3 hours

CI execution: ~4 hours

20% of tests are occasionally flaky

Developers wait for the complete suite before merging

Management asks you to bring feedback below 15 minutes

How would you approach the problem?

Detailed Answer

I would not immediately add more CI machines or simply increase parallelism.

First, I would understand where the four hours are being spent.

Step 1 — Measure the suite

I would collect:

Test execution time

Setup/teardown time

Browser startup time

Authentication time

API/database setup time

Slowest tests

Slowest test suites

Retry frequency

Failure rate

Flake rate

Resource contention

For example:

Test execution       150 min

Environment setup     20 min

Browser startup       15 min

Authentication        25 min

Retries               30 min

Database/data setup   20 min

Infrastructure wait   20 min

Without this measurement, parallelization could simply move the bottleneck somewhere else.

Step 2 — Revisit the test pyramid

I would identify tests that are unnecessarily implemented through the UI.

For example:

UI:

    "Create customer"

API:

    Create customer

    Update customer

    Delete customer

Database/service:

    Validation/business-rule tests

If 100 tests verify the same business rule through the UI, many of them should probably move to API/component/integration levels.

The UI suite should concentrate on critical user journeys and cross-system behavior rather than testing every business rule through a browser.

Step 3 — Introduce test layers

For example:

                 Small number

                    UI/E2E

                      ▲

                 API/Contract

                      ▲

             Integration tests

                      ▲

                 Unit tests

                 Large number

Step 4 — Parallelize safely

After removing unnecessary UI coverage, I would parallelize the remaining tests.

But parallel execution requires:

Independent test data

Independent users/accounts where necessary

No shared mutable state

Unique resource names/IDs

Isolated browser contexts

Environment capacity planning

Otherwise:

Test A ---> modifies customer 123

Test B ---> expects customer 123 unchanged

can create race conditions.

Step 5 — Create CI test tiers

I would split execution into stages.

PR:

    Unit

    API/contract

    Critical UI smoke

    ~10-15 min

Post-merge:

    Broader regression

Nightly:

    Full regression

    Cross-browser

    Extended integration

This gives developers fast feedback without abandoning comprehensive regression coverage.

Step 6 — Optimize the test framework

For UI automation I would investigate:

Reusing authenticated state where safe

Avoiding unnecessary login flows

API-based test-data setup

Better fixtures

Parallel workers

Eliminating hard waits

Reducing unnecessary browser navigation

Using efficient locators

Avoiding unnecessary UI setup

Playwright, for example, recommends isolated tests and user-facing/explicit locators, and its locators provide auto-waiting and retryability. 

P

Playwright

+1

Senior-level point

The answer is not “increase parallel threads from 10 to 50.”

A senior SDET should first ask:

"Why are we using the browser to verify things that don't require a browser?"

The goal is to reduce feedback time, not simply increase infrastructure.

2. A test passes 100% locally but fails randomly in CI. How would you investigate it?

Scenario

A checkout test:

Local: 100/100 passed

CI: 92/100 passed

The failure occurs randomly.

The developer says:

"It's just a flaky test. Add a retry."

What do you do?

Detailed Answer

I would reject the assumption that it is automatically a flaky test.

A failure that appears nondeterministically may be caused by:

Timing

Race conditions

Shared data

Environment differences

Network instability

Resource exhaustion

Browser differences

Dependency failures

Test-order dependency

Application defects

First, I would classify the failure.

Step 1 — Capture evidence

I would collect:

CI logs

Screenshot

Video/trace

Browser console

Network logs

Application logs

API responses

Database state

Test data

Environment information

Commit/build information

Failure timestamp

Step 2 — Re-run repeatedly

I might execute:

test x 100

locally and in CI.

If:

Local: 100/100

CI: 94/100

then I investigate environmental differences.

If:

Local: 96/100

CI: 93/100

then the test itself is probably nondeterministic.

Step 3 — Check timing assumptions

Bad:

await page.click("#submit");

await sleep(3000);

expect(message).toBeVisible();

The problem isn't solved by choosing 5 seconds instead of 3 seconds.

I would wait for a specific condition.

For Playwright, web-first assertions automatically wait and retry until the expected condition is satisfied or the timeout is reached. 

P

Playwright

+1

Step 4 — Check test-data collision

Suppose parallel workers use:

customer@test.com

Every test may update the same customer.

Instead:

customer-worker1-<unique-id>

customer-worker2-<unique-id>

or generate isolated data through APIs.

Step 5 — Check external dependencies

For example:

UI

 ↓

Order Service

 ↓

Payment Service

 ↓

External Payment Gateway

If the external gateway is unstable, the UI test should not necessarily be blamed.

I would determine whether the test is supposed to verify:

Our checkout UI

Our payment integration

The external provider

These may require different test layers.

Step 6 — Only then consider retry

Retry is useful for transient infrastructure failures, but it should not hide genuine product defects.

A retry policy should therefore be observable:

Original failure

       ↓

Retry

       ↓

Pass

       ↓

Classify as possible transient failure

       ↓

Track it

I would not allow:

Failure → Retry → Pass → Ignore forever

Senior-level point

A senior SDET doesn't treat retry as a fix.

The objective is:

Identify why the same test produces different results under apparently identical conditions.

Research on flaky tests also identifies timing/concurrency, infrastructure, environment, and external factors among the important sources of nondeterminism. 

A

arXiv

+1

3. Your company has 25 microservices. There are almost no automated tests. You are asked to build the automation strategy from scratch. What is your first 30-day plan?

Detailed Answer

I would not start by creating a Selenium/Playwright framework.

The first step is understanding the system.

Week 1 — System discovery

I would identify:

Service

 ├── API

 ├── Database

 ├── Events

 ├── Dependencies

 ├── External systems

 └── Critical business flows

I would map critical workflows such as:

User

 ↓

Authentication

 ↓

Order

 ↓

Inventory

 ↓

Payment

 ↓

Notification

Then classify risk.

Week 2 — Define test layers

For each service:

Unit

Integration

Contract

API

Event/message

End-to-end

I would avoid making everything an end-to-end test.

For example:

Business rule → unit

Service API → API/integration

Service-to-service compatibility → contract

Critical customer journey → E2E

Week 3 — Build the foundation

I would establish:

Framework conventions

Test-data strategy

Environment strategy

Authentication strategy

Logging

Reporting

CI integration

Parallel execution

Failure artifacts

Test tagging

Ownership

Example:

@smoke

@critical

@api

@contract

@e2e

@nightly

Week 4 — Automate highest-risk flows

I would select perhaps:

Top 10 critical business flows

Top 20 high-risk APIs

Top service contracts

Top production failure scenarios

Then measure:

Before:

Manual regression = 2 days

After:

Critical automated regression = 20 minutes

Senior-level point

A senior SDET should create a quality strategy, not merely create a test framework.

The framework is an implementation detail of the broader quality architecture.

4. Two tests pass individually but fail when executed in parallel. How do you diagnose it?

Scenario

Test A → PASS

Test B → PASS

A + B in parallel → intermittent failures

Detailed Answer

My first suspicion would be shared mutable state.

I would investigate:

1. Shared database records

Test A updates user 100

Test B deletes user 100

2. Shared accounts

user@test.com

being logged in simultaneously by multiple tests.

3. Shared files

/download/report.csv

Both tests read/write the same file.

4. Shared environment configuration

For example:

Test A changes feature flag = ON

Test B expects feature flag = OFF

5. Static/global variables

Example:

static String customerId;

Parallel tests can overwrite the value.

6. Shared browser context/session

Tests should not unintentionally share:

Cookies

Local storage

Session storage

Authentication state

Solution

I would introduce isolation.

For example:

Worker 1

  customer-101

Worker 2

  customer-102

Worker 3

  customer-103

Test data should preferably be created specifically for the test and cleaned up safely afterward.

Playwright's current best-practice guidance explicitly emphasizes test isolation, including independent storage/session state, because isolation improves reproducibility and prevents cascading failures. 

P

Playwright

Senior-level point

Parallelization is not simply:

workers = 20

It is:

Parallelism

+

Data isolation

+

State isolation

+

Resource capacity

+

Deterministic cleanup