Solution
OTT QA and multi-device testing
Testing across the device matrix your viewers actually use, not the one that fits on a shelf.
- Tested on
- Smart TVs
- Streaming devices
- Consoles
- Mobile
- Web
- What we check
- Function
- Certification
- Memory
- Focus
- Playback
- Resume
- Engagement
- Pre-launch pass
- Pre-certification pass
- Ongoing
One build, every device at once
A run in progress
Every device in the matrix takes the same build at the same time, each on its own operating system version. A failure arrives named — the device it happened on, the version it was running, and the difference between what should have happened and what did. That is the gap between a bug report and a finding somebody can act on this afternoon.
What you get
- A device matrix built from your install base rather than from a catalogue
- Functional passes on real hardware, not emulators
- Platform passes against each store's own certification rules
- Failures reported with reproduction steps and a view on which will get you rejected
- Re-testing as platform operating systems update, if you want it ongoing
Typical timeline
Ongoing, or 3 to 6 weeks for a one-off pass
Who this is for
Services that already have a development team, and a device problem that team cannot reasonably solve on its own.
This is the one page here that is not about us building your app. You keep your engineers and your roadmap. What you get is the hardware, and the specific knowledge of how each platform fails — which is expensive to keep in-house for a single product and unavoidable if you ship to televisions.
Smart TV app testing
Starts with an app that works on the newest set in the office, and fails on the five-year-old model that is a third of the install base.
OTT device testing
Starts with a matrix nobody can justify, because it was chosen from a catalogue rather than from your own analytics.
Connected TV app testing
Starts with a certification rejection nobody saw coming, usually for a rule specific to one reviewer on one platform.
OTT test automation
Starts with a suite that passes every release, and misses the things that only happen when someone is holding a remote.
What we deliver
OTT application testing on real hardware, against the device matrix your viewers actually use rather than the one that fits on a shelf. Real OTT device testing means physical sets and sticks, not an emulator in a television-shaped window.
Smart TV app testing is where this matters most. A television is not a slow phone: it has a fraction of the memory, an input model built around five keys, and an operating system the manufacturer updates when it suits them. Connected TV app testing on the current flagship model tells you almost nothing about the set that is five years old and still a third of your install base.
We built VibeView because shipping to every platform means testing on every platform. It is live and you can use it. That is also the honest reason this page exists at all — the tooling came out of the delivery work, not the other way round.
How the testing runs
We start from your install base rather than our device list. Which devices your viewers are on decides the matrix, and the matrix is most of the cost.
Then the work splits in two. Functional passes cover what your team already knows to check. The platform passes cover what they usually do not: each store's own certification rules, the rejection reasons that are specific to that reviewer, and the behaviours — memory, focus handling, resume after standby — that only appear on the real thing.
Build the matrix from your analytics
Which devices your viewers are on decides the list. Ten chosen from your data beat thirty chosen from a catalogue.
Run the functional pass
What your own team already knows to check, run on hardware instead of on an emulator.
Run the platform pass
Each store's certification rules, and the rejection reasons specific to that reviewer.
Report so it can be acted on
Reproduction steps, the device it happened on, and which findings will actually block approval.
Re-test when the platforms move
Operating systems update on their own schedule. An app that passed in March can fail in September.
What actually varies
We do not publish a price. Here is the more useful thing: what moves it.
| Driver | What it changes |
|---|---|
| Size of the device matrix | The single biggest factor, and the one worth arguing about. Ten devices chosen from your analytics beat thirty chosen from a catalogue. |
| One pass or ongoing | A pre-launch pass is a fixed piece of work. A live service needs re-testing as platform operating systems move, which they do without asking you. |
| How much is automated | OTT test automation pays back on the checks you run every release. It does not replace someone holding the remote, and anyone saying otherwise has not shipped to a television. |
| Live or on-demand | Live is harder to test and harder to reproduce. Failures that only happen during a real event need a plan rather than a test case. |
One limit worth saying plainly, because it should change what you ask us for.
Testing tells you what is wrong, not why your architecture made it inevitable. If an app is slow on older televisions because of how it was built, a bigger device matrix produces a longer list of the same finding. When that is what we see, we will say so — and the answer is usually a piece of engineering work, not more testing.
What we need from you
- Your install base by device and operating system version. This is the whole basis of the matrix and you almost certainly have it.
- A build we can install, and a way to get new ones without a two-week wait.
- Test accounts, including ones with the entitlements your real subscribers have.
- Whoever decides what gets fixed, because a report nobody owns changes nothing.
You will be talking to the people doing the work. On this page that matters more than usual, because the useful output is not a list of failures — it is knowing which three of them will get you rejected.
Common questions
Can you test an app you did not build?
Yes, and it is most of this work. You keep your development team and your roadmap. We bring the device matrix and the platform-specific failures, which are the parts that are expensive to own in-house for one product.
Why not just use emulators?
Emulators are useful for layout and useless for the things that actually fail. Remote input timing, memory limits on older televisions, real network behaviour and each platform's own certification rules only show up on hardware.
Which devices should we test on?
The ones your install base is on, which is usually not the ones your team owns. Most teams test on the newest television in the office, and most failures arrive from a model that is five years old and still a large share of their viewers.
Is this a one-off or ongoing?
Either. A one-off pass before a launch or a certification attempt takes three to six weeks. Ongoing is the more honest answer for a live service, because platform operating systems update on their own schedule and an app that passed in March can fail in September without you changing anything.
Related
Last updated