coreelec:draft_alphatest

Public Alpha Testing

Welcome to CoreELEC Public Alpha Testing

Here we will introduce:

  • What Public Alpha Testing is
  • How you can contribute to the community
  • How to run a test and provide feedback

What is Public Alpha Testing?

Public Alpha Testing is community supported testing procedure that helps to evaluate the stability and quality of the product by gathering feedback from end-users, and discover show stopping defects, major usability problems, critical feature gaps, and in-the-wild interoperability problems. The product should be more than halfway complete, with testers focused on finding as many bugs and major issues as possible. Public Alpha Tests follow the same three requirements: real people, real environments, real products, but answers only one very specific question: does it work?

Why Should You Invest Time in Alpha Testing?

In a Agile development environment, an effective Public Alpha Test helps ensuring the product is ready for a Public Beta. A common mistake is skipping the Alpha stage and moving right into Beta. This often leads to critical blocking issues making it impossible to continue using the product, ending up having to return it to the development team and causing delays for the entire community.

What makes a Public Alpha Test unique is that by testing with end-users, our Team gains insight on how multiple devices work. Your contribution helps the Team find gaps and gain a better understanding of multiple device, it allows us to see how different devices perform in different ecosystems, and this in turn helps resolving blocking issues before starting a Beta. As such, the entire community benefits from your contribution.

When Does Public Alpha Testing Happen?

Public Alpha Testing predominantly occurs when the product is 60%-80% complete. While internal test teams focus on professional white box testing in controlled environments with automated acceptance tests and manufacturer sponsored hardware, Public Alpha Tests rely on real user environments to gather feedback on the functionality of the product. This leads to a more stable end product, with greater device compatibility.

How Do You Run Public Alpha Tests?

As with any test, Public Alpha Tests follow a specific plan, schedule, and message to meet certain objectives. Mapping topics, recruiting testers, structuring bug hunts, and analyzing feedback are only part of the equation.

Public Alpha Testing Routine

  • Download the latest CoreELEC Public Alpha
  • Prepare a removable medium ((μ)SD Card or USB Flash Drive) with the Public Alpha
  • Ensure the correct remote is setup
  • Boot CoreELEC Public Alpha from the removable medium
  • Test if all features work:
    • Bluetooth
    • Network (10/100/1gbit)
    • Sound
    • Different resolutions
    • SSH
    • SMB
  • Report the results together with a log in the End User Forum
  • Shutdown CoreELEC Public Alpha
  • Remove the (μ)SD Card or USB Flash Drive

If a developper requires more information, would like you to do some additional testing or try a custom build, they will reach out to you.