Automated vs Manual Testing: Where Each Fits in QA

What automation does well (and what it doesn’t)

Strengths

• Rapid, repeatable checks for missing labels, ARIA misuse, contrast flags, and title/landmark gaps.

• Executes with every commit/PR to prevent issues.

• Efficient across templates and high-traffic pages.

Limitations

• Fails to fully assess keyboard flows, focus order/visibility, modal traps, meaning of alt text, or screen reader experience.

• Often overlooks content-level issues and dynamic states (e.g., live validation, announcements).

Bottom line: Automation is your early warning system, not the final decision.

Manual testing essentials (short, high-value checks)

• Keyboard-only: Use Tab/Shift+Tab, Enter/Space/Arrows; open/close menus, dialogs, carousels; ensure there are no focus traps.

• Focus visibility: Clear, high-contrast outline everywhere; make sure focus is not obscured under sticky headers.

• Screen reader smoke test (NVDA/VoiceOver): Check title, headings, landmarks, links list, form labels, error announcements.

• Forms: Provide user-friendly validation; specific errors linked to fields; focus should automatically go to first error.

• Mobile: Check target sizes, reflow at 320px, and allow zooming; ensure motion respects prefers-reduced-motion.

Goal: Confirm task completion and user comfort on real devices—not just meeting criteria.

Cost, coverage, and risk (how to decide)

ApproachCost per runCoverageCatchesRisk if skippedAutomation (CI)LowBroad (templates)Code-level violations, common regressionsHigh churn of minor bugsKeyboard passLowHigh (critical flows)Focus order/visibility, usabilityTraps, blocked usersScreen reader skimLowMediumSemantics, labels, announcementsMisleading UX for AT usersExpert auditHigherTargetedComplex patterns, strategy gapsStrategic debt, legal exposure

A practical hybrid plan (what to run, when)

On every PR

• Automated ruleset on key templates (home, nav, product/service, blog, forms).

• Thresholds: 0 critical, ≤ N moderate (with ticket auto-created).

Before each release (30–45 minutes)

• Keyboard pass on top flows.

• Screen reader skim on changed areas.

• Contrast spot checks on changed components/states.

• Mobile check for targets/reflow/motion.

Quarterly (or after major UI changes)

• Short expert review of components (menus, dialogs, forms, carousels) and governance.

Example CI guardrails (pseudo-config)

name: a11y-checks
on: [pull_request]

Tip: Begin with a small, stable ruleset; expand only when the noise is minimized.

Ownership: who does what

• Engineering: Setup for CI, thresholds, fixing code-level issues, component patterns.

• Design/Content: Manage contrast tokens, states, headings, link text, alt text quality.

• QA/PM: Schedule manual passes, criteria for acceptance/rejection, risk sign-off.

• Leadership: Allocate time for audits and improvements.

Common pitfalls (and quick fixes)

• Green CI, red reality: Passing automation doesn’t ensure usable flows → add a 10–15 min keyboard/screen reader checkpoint.

• False positives → rule fatigue: Prioritize and fine-tune rules; suppress at a component level with comments and tickets.

• No owners: Assign components (Nav, Dialog, Forms) to dedicated maintainers.

• One-time audit syndrome: Plan for recurring checks; accessibility may drift without proper governance.

Copy-paste checklists

Automation scope (Markdown)

Manual release gate (Markdown)

Quick QA cadence (sample)

• Weekly: CI exclusively (on PRs).

• Fortnightly: CI plus a 30-minute manual gate across main journeys.

• Quarterly: CI plus manual gate and a small expert review of complex components.

How Adjustable helps (after structural fixes)

• Text to Speech assists users in consuming complex content hands-free.

• Language Translation extends reach to global audiences.

• Accessibility Profiles provide quick modes (e.g., “Readable” or “Low Vision”).

• Text Options, Reading Ruler, Cursor Adjustment, and Screen Mask enhance comprehension and focus.

• Page Options (page scaling, hide images/animations, highlight links) minimize distractions on dense UIs.

• Light/Dark/Color modes and configurable toolbar/widget placement match user preferences.

Try Adjustable

FAQs

Can automation ensure compliance?
No. It’s vital for spotting regressions, but manual checks are necessary for flows, meaning, and ease.

What pages should automation address first?
Templates that encourage traffic and revenue: home, nav, listings, product/service, blog, checkout/contact.

How tight should thresholds be?
Start strictly on critical (0 allowed). For moderate issues, allow a limited budget generating tickets until achieving 0.

Which manual test is most effective?
A keyboard-only check through your main journey—fast, enlightening, and highly linked with conversion obstacles.

Next steps

• Add a minimal a11y CI task with tight thresholds on key templates.

• Organize a 15–30 min manual gate before releases.

• Assign component ownership and a quarterly mini audit.

• Install Adjustable to enhance readability and task completion in real-world scenarios as updates are developed.