Onvif: Testing for Conformance

Onvif began in 2008 as a small group of manufacturers collaborating to accelerate the acceptance of systems based on network surveillance cameras. The mission hasn’t changed much, but its application and influence has: now it is an industry alliance for the physical security industry, as a nearly 500 member strong organization with more than 13,000 Onvif conformant products. With members on six continents, our specifications for video and access control have also been adopted by the International Electrical Commission, one of the world’s most influential standards organizations.
Onvif is a member-driven organization and operates on the basis of consensus. The next Onvif profile will be developed based on feedback from Onvif members and the physical security industry at large.
As a member, companies can directly contribute to the development of new profiles, as well as claim Onvif conformance. Specific profiles dictate the features that products must have to ensure interoperability with other conformant products. Conformance is achieved when the technical specifications of the device or client meet the technical requirements stated in the specific Profile Specification document(s). To increase the confidence in the conformance statement, Onvif also provides Test Specification documents, a Device Test Tool and a Client Test Tool. This Conformance Process Specification defines how a member shall use these components to claim conformance to one or multiple profiles.

Testing for Conformance
Only members can claim conformance. Conformance is achieved through a self-declaration process where manufacturers test their products using the Client Test Tool or Device Test Tool. The test tools automatically generate three important documents: The Declaration of Conformance, Feature List Document and Onvif Test report. While the process itself is self-driven, manufacturers must prove that they have met all the requirements to claim conformance in order to be listed on the Onvif-conformant products page which is the only official source for a complete list.
Onvif improves its test tool as new profiles are developed, in order to include support for additional, more robust features. When Profile S for video streaming was developed, there were certain things the device had to do to claim conformance, such as having support for video streaming and metadata configuration, so the test tool was based upon those specific features. Today, as new profiles are released, updates are continuously being made to the test tool. Profile T for advanced video streaming has many features that Profile S does not, such as support for H.265 video compression and bidirectional audio streaming. In turn, Onvif needed to develop a test tool that would confirm conformance on Profile-T-specific features.
Regardless of the specific profile a manufacturer is testing, there are several hundred tests within the individual test tools that devices must go through to meet Onvif conformance. The original test tool was designed for basic communication between devices and clients. Once that communication basis was developed, Onvif started adding other, profile-specific features that needed to be evaluated, such as granting and revoking credentials with Profile A or edge storage retrieval with Profile G. The test tool is constantly being updated, with this past July marking the 20th version of the Device Test Tool and the 10th version of the client test tool. Recognizing that the needs of the industry are ever-changing, the test tool process gives Onvif the ability to, overtime, enhance the products and specifications that the manufacturers developed their devices to conform to.
As a new profile is developed, all manufacturers who are active within Onvif help to identify which features and specifications need to be a part of the profile. When a device or client is put under for conformance testing, the software validates to what features it conforms, which is different for each profile. The test tool runs the device through a series of tests, and the device then receives a pass or fail. Manufacturers are given the ability to make changes to their product and retest before it is released to the market as a conformant device or client. There is no limit on how many times a device is tested, and members are encouraged to retest every time the firmware is updated. The specific firmware version of each product is also included on the Onvif website as part of the Declaration of Conformance.

Developers’ Plugfests
Developers’ Plugfests have been a semi-annual norm since the founding of Onvif. The three-day event enables developers to test their implementations of Onvif profiles with other Onvif-conformant products, and to give feedback on possible future directions of the organization.
In early June, some of the brightest developers and engineers in the industry gathered in Tokyo to attend the 20th Onvif Developers’ Plugfest. In total, 47 developers and engineers from 22 Onvif member companies to test their products independently for Onvif conformance and were given the opportunity to test their devices with those from other manufacturers. Attendees logged nearly 250 hours of testing. There has been cumulative participation from more than 1400 technical experts over the years. In total, plugfests have logged approximately 5,000 hours of testing, demonstrating our member companies’ steadfast commitment to providing flexible, best of breed solutions.
All of the plugfest participants across the years have directly contributed to the ongoing development of standards. By choosing to work collaboratively, member companies have accelerated the use of open source solutions as an industry norm, and single manufacturer lock-ins are becoming a thing of the past. It is through the plugfests that Onvif receives valuable feedback to further drive the acceptance of interoperability.

False Conformance Claims
As the usage of Onvif has become increasingly more widespread, so to have false claims of conformance. In many cases, claims are based on a company’s misuse of the Onvif trademark or a member company’s misunderstanding of the requirements of membership. For example, companies that offer rebranded OEM products must retest and submit new documentation to show valid conformance for each product, even though the original product is Onvif certified, as conformance is specific to a product offered by a member company.
Onvif approaches false conformance claims by educating members and the industry at large about what the Onvif name means through attending trade shows and conferences, speaking at events and holding ongoing online training sessions. Onvif also has developed an online false conformance-reporting tool to encourage the industry to report suspected false claims. The organization maintains a current list of its members and a dynamic conformant products database on its website so that the industry can easily determine whether a manufacturer is an Onvif member and is marketing conformant products. Similar to other standards bodies, if a brand has value, counterfeiters will no doubt see the value in misusing the brand.

The Future
With the continued development and release of new profiles, Onvif is making a long-term commitment to open systems. Profile development can take years and requires the work of many Onvif member company representatives from all over the world. Conformance testing plays a huge role in this process by ensuring that products abide by the standards that Onvif has produced for continued interoperability. By providing a concrete conformance process, Onvif and its member companies can further contribute to the development of standards that can take the physical security market into the next decade. Conformance means flexibility to the end user, and flexibility means best of breed solutions.