Going 'official' means complying with official certification processes, too
Using a recent bug on the Oppo N1 as an example, CyanogenMod gave an interesting look into the process of making and pushing out an OTA to devices now that it is a Google-certified software vendor. Although it has a good history of managing updates — including complete nightly software builds — for multiple devices, things change a little bit when you go "official."
Those who have installed CyanogenMod through the official installer or have a CM edition Oppo N1 can benefit from what are called "incremental OTAs," or small patches that can fix just a specific part of the system without touching anything else. In the case of a recent bug involving the O-Click accessory for the N1, the fix was small and could really benefit from the use of an incremental update — but added layers of testing required to push an OTA to devices slows down the process.
In a nice bit of transparency, CM explains that once the engineers believe they have solved a software issue, the update is packaged up and sent off to CM's QA (quality assurance) team. QA is responsible for making sure the new software fixes the problem as intended but also doesn't mess up anything else in the process. The QA team finishes up their testing, and then it's onward to CTS — this is the test suite Google uses to certify that the new software is still compatible with Android devices, features and apps. Each run of CTS takes about eight hours to complete, and must be 100 passed in order to be certified — one single hiccup, and you're back to the start.
The end result for this small fix? CM claims it took about three weeks after the fix was completed in code before it was greenlit and ready to be rolled out to devices. The extra levels of testing within their own QA team and now required by Google simply add more time onto new releases than we're used to with typical custom ROMs.
Fortunately, CM says that the incremental update system is "the backbone for our future update and release plans," and that testing it with CM installer and CM edition N1 devices is just the beginning. Fixing small issues as they arise and starting the process sooner with a smaller change in the software should speed up the process, and in the end give you a smaller download and shorter install time. Just know that you may not always be getting the "nightly" update experience that you're used to from before.
Source: CyanogenMod
