Conformance Testing

Many companies build MHP receivers, and in order to be successful in the marketplace it's important for receiver from different suppliers to be interoperable. All receivers should behave in a similar (but not necessarily identical) way, and application developers, consumers and network operators must e confident that an MHP application will run on all receivers that are available to customers. Customers be confident that the receiver they buy will be able to run all of the applications that are broadcast, or they will be less likely to buy a receiver and thus an MHP deployment will be more likely to fail.

Since MHP receivers are built on top of existing DVB standards, this gives an important advantage: the underlying standards for encoding and modulation, service information, and data broadcasting are mature and well-understood. This means that the basic functionality for receiving and decoding digital TV signals is likely to work without problems. Most problems that we see will probably be with the MHP implementation itself.

MHP is a complex specification, and there is a lot of room for parts of the specification to be interpreted differently by different implementors. Since MHP is an open standard, there is no one set of 'official' source code that can be ported to a new platform, as is the case with OpenTV or other proprietary middleware solutions. There is not even an official reference implementation. (IRT call their implementation the 'reference' implementation, but this has no official weight, and so is not very meaningful). What MHP does have, however, is a fairly comprehensive test suite.

The current test suite consists of approximately 20,000 separate tests, covering around 80% of the functionality in the MHP specification. This still leaves some areas that are not covered by the test suite, but any experienced engineer knows that 100% test coverage is very difficult to achieve.

DVB uses a self-certification process for MHP conformance testing. Companies can get a copy of the test suite by paying a 1,000 euro administration fee to ETSI, and they can then use this to test their MHP implementation as many times as they like. When they have successfully passed all of the tests, they send a copy of the test log to DVB, who will then certify the test results. Upon payment of a license fee (10,000 euros for the first year, and 5,000 euros for every subsequent year), the company can then use the MHP logo on its products to show that it is fully MHP-compliant. Use of the MHP logo is subject to strict rules, just like any other trademark, and so only MHP-compliant products can use this logo.

The full process for MHP conformance testing is as follows:

  • The manufacturer (who may be an STB manufacturer or a middleware implementor who is porting their middleware to a new platform) requests the MHP test suite from the custodian (ETSI) . This is subject to an administration fee of 1,000 euros.
  • After the manufacturer has signed the appropriate non-disclsure agreement, ETSI sends them the test suite.
  • The manufacturer uses the test suite to test their device. If their device passes the tests, they send a test certificate to ETSI.
  • Once ETSI receives a test certificate from the manufacturer, they will inform DVB that the manufacturer has passed the test suite.
  • The manufacturer pays DVB a fee of 10,000 euros for rights to use the MHP logo. For following years, the fee is 5,000 euros.
  • DVB registers the compliant product in their database and gives the STB manufacturer or middleware implementor a copy of the MHP logo and the rules for using it. The MHP logo can then be used on the conforming product and any related advertising material.

The test suite itself is still evolving, and DVB gave the MHP Experts Group (MEG) the responsibility of maintaining the test suite. This includes the job of updating the test suite to support newer versions of the MHP specification - the current test suite (version 1.0.2b) only supports MHP 1.0.2. Companies can also report bad tests to the MHP Experts Group for them to re-evaluate and possibly fix.

Interoperability testing

Conformance testing is part of the picture, but not all of it. There are several different places where conforming MHP implementations can behave differently, and so it's important to make sure that receivers are actually interoperable. This process has been going on since the early days of MHP, when Philips, Sony, Nokia, and Panasonic were co-operating to make sure that their MHP implementations could all run the same applications for the IFA and IBC exhibitions in 1999. As MHP has become more widespread, semi-formal 'plug-fests' have become the accepted way of ensuring interoperability. These events take place every 3-4 months (usually at IRT's offices in Munich, Germany) and give developers a chance to test their products against MHP products from other companies in an informal environment. This is not limited to MHP implementations - application developers, head-end equipment manufacturers and authoring tool developers are all welcome. This approach allows developers to identify areas that need to be fixed
in order to be compatible with other products in the marketplace,and for different companies to agree on an appropriate interpretation of some of the less clear areas of the specification.

This conformance and interoperability testing is not designed to eliminate all of the differences between MHP implementations, however. One of the main reasons for leaving so many areas of the specification open to interpretation is to allow developers room to differentiate their products. Both good and bad MHP implementations can pass all of the conformance tests and be interoperable with other MHP products.

One important thing to realise about conformance testing is that not all implementations will face the same test regime. In particular, the choice of JVM will affect the test suites that an implementation must pass. To ensure that all Java virtulsa machines behave basically the same, ETSI have published a test suite for JVMs that are used in an MHP implementation. While this is sufficien for any clean-room JVMs, it's not good enough for the official JVM fromSun Microsystems. Sun require that any products using their JVM pass the full Sun TCK (Technology Compatibility Kit) tests. This is a superset of the ETSI test suite, and so implementations using Sun's JVM must pass a more stringent set of tests than those using a clean-room JVM. There are also additional testing costs associated with using Sun's JVM, as well as more requirtements in terms of which Java APIs are supported.

Licensing issues

MHP may be an open standard, but that does not mean that it's free from intellectual property issues. As always, 'open' is not the same as 'free' - companies are free to sell their MHP implementation (or elements of an implementation) at any prices that they wish to set. Several companies will license their middleware stacks to STB manufacturers,and this seems to be the most common model for producing MHP receivers at the moment. As well as the MHP implementation itself, many manufturers will license a Java Virtual machine from a third party - either Sun Microsystems, or one of the companies who have developed a clean-room JVM. These license fees are a fairly common part of the MHP landscape, and so any manufacturer thinking about building an MHP implementation should take these into account.

Another issue that must be considered is royalty payments for patents that are used by MHP. At the moment, anybody is free to build an MHP implementation and broadcast MHP applications without paying any license fees (except for software they have bought from a third party). This will not always be the case, however. All of the patents used in MHP have been gathered together in to a 'patent pool' to ensure reasonable and non-discriminatory license terms for all user, and to ensure that the patent licensing process is as straightforward as possible. This is similar to the MPEG-LA patent pool that covers products using the MPEG standards, and so this is a fairly well-known approach in the DTV world.

The MHP and OCAP patent pools are administered by Via Licensing, a subsidiary of Dolby Labs who specialise in patent licensing. The initial license terms were announced in June 2005, and have not proved popular with the MHP community. Under the current licensing terms, MHP device manufacturers must pay a license fee of $2 per unit, while companies offering access to MHP pay-TV services must pay $0.25 per subscriber per year. Companies offering free-to-air services (either public broadcasters or those supported by advertising) currently do not have to pay any license fees, but these terms may change after 2009. More information about the license terms is available from the Via Licensing web site.

As we have already mentioned, these license terms are not very popular in the MHP community. Some companies think that the fees are too high,and are campaigning to have them reduced. Others are concerned about the uncertainty with respect to free-to-air services, which make up most MHP deployments to date. It is not clear how the number of subscribers for these services would be be measured, or whether any license fees will be introduced at a later date. Some people are concerned that this will slow the deployment of MHP services as companies wait and see whether license fees will be introduced, and that this will have an impact on every company involved in MHP.

Open-source MHP?

There are a couple of open-source MHP implementations available at present, and we are often asked if these are suitable for use in commercial products. Every time, the answer is the same: right now, the open-source MHP implementations require too much work for most companies to think about using them in a product. The problems are very simple:

  • The license terms are either GPL or LGPL, which means that companies must release any changes they make
  • The implementations are incomplete and miss a lot of the hard parts of an MHBP implementation, such as the MPEG-related functionality
  • The code still faces the intellectual property issues that require the payment of the MHP/OCAP patent pool royalties

Taken together, these issues mean that it's really not worth trying to use one of the open-source MHP implementations in a product at the moment.