Aadhaar: The Greatest Testimony To FOSS' Success In India!

It is heartening to see how FOSS is powering India's one of the most prestigious and mammoth eGovernance initiatives--Unique Identification Authority of India's Aadhaar project.

For those of you who need an introduction to the Aadhaar project, it is a 12-digit individual identification number issued by the Unique Identification Authority of India (UIDAI) on behalf of the Government of India. This number will serve as a proof of identity and address, anywhere in India. Each Aadhaar number will be unique to an individual and will remain valid for life. By providing a clear proof of identity, Aadhaar is targeted at empowering the poor and underprivileged residents of the country in accessing services such as the formal banking system and will give them the opportunity to easily avail various other services provided by the Government and the private sector.

More than 6 crore UIDs have been issued till date in an 18 month period. The systems were recently scaled to process up to 1 million UIDs per day. Enrolments are actively happening in many Indian states. (Refer to http://portal.uidai.gov.in for every day updates.)

The project is historic for it is one of the largest eGovernance initiatives in India till date and the target scale is unprecedented. What makes it all the more interesting is the fact that the project is driven extensively by the FOSS stack.

Where FOSS is the first and foremost choice
UIDAI technology team had indicated preference for open-standards and open source, says Regunath Balasubramanian, principal architect, UIDAI project. "We were prudent in the very early stages of the project. That has continued to be the philosophy. Initially, only the enrolment client application had a dependency on device drivers of a certain operating system, that too for a short while," he reveals.

Blasubramaniun explains why open source software (OSS) became the first choice for the project: "The primary technical requirements of the project were of scale and vendor neutrality at all levels. FOSS helped us achieve vendor-neutrality in many of our application components, which is very important for an initiative of national importance. The use of open standards has encouraged multi-vendor participation. This has driven costs down and improved the quality.

"The per CPU license costs that are normally associated with proprietary software has been largely eliminated. The benefits become more pronounced as the deployment footprint increases over thousands of CPU cores across multiple data centres.

"Access to the source code of the FOSS solutions has also helped us use them more effectively as we got a pretty good understanding of how it works, internally. Use of FOSS has also helped gain adoption by registrars and enrolment agencies, some of whom have strict guidelines around use of proprietary software."

A strategy of adopting OSS solutions that are mainstream, have a strong community backing, and are actively developed/enhanced, works well most of the time, affirms Balasubramanian.

But have there been any 'cons' or pitfalls in FOSS adoption? To this he says: "The challenges to using OSS are that of perception, developer willingness to explore and availability of support of all solutions available out there."

FOSS tools that are powering the UIDAI project
For the Aadhaar project, UIDAI is using FOSS, mostly. Below is a list of FOSS solutions that are being used, along with reasons why the team is using them.

RabbitMQ: This is an AMQP (advanced message queuing protocol) standards-based messaging for many of our asynchronous processing needs, integration and high scalability via the SEDA (staged event-driven architecture) implemented in Mule.

Mule: Light weight object broker and SEDA runtime instance used also in orchestrating various stages in batch processing.

The Hadoop stack: HDFS (Hadoop distributed file system) is used to provide high data read/write throughput in the order of many tera-bytes per day. Distributed architecture enables scale out as needed, Hive is used for building our data warehouse, HBase is used for indexed lookup of records across millions of rows, Zookeeper is used as a distributed coordination service for our server instances, Pig is used as an ETL (extract, transform and load) for loading data into Hive.

Pentaho: is used for report generation and OLAP (on-line analytical processing) analysis. It has good alignment with the rest of our stack.

Apache Tomcat: is used as Web container wherever Http access of dynamic data is needed.

MongoDB: The distributed document oriented database is giving us schema flexibility, low latency data reads, auto-sharding of content.

Apache Solr: is used for fast search index of full text data.

MySQL: is the RDBMS store for all relational data that requires ACID (atomicity, consistency, isolation, durability) properties, which is heavily sharded for scaling.

Apart from this the project makes use of Spring framework, an application container. Custom built runtime that provides a server-like runtime for configuration, administration, call interception, monitoring and metrics collection. "We also use many of its sub-projects like Spring security, Spring batch, Spring MVC (model-view-controller), etc," reveals Balasubramanian.

The biometric devices/scanners used for enrolment are procured from the market by enrolment agencies. Initially, most vendors supported device drivers only for Windows. Now, drivers are available for other operating systems like Linux. The enrolment client software works flawlessly with these drivers on non-windows operating systems as well, he affirms.

Since the data gathered for the UIDAI project is sensitive and is an access point for many different benefits, it also requires backups in case of crashing or physical corruption of data. So how does the UIDAI team ensure data security and disaster management? Balasubramanian divulges more details: â€œThe UIDAI has a comprehensive security policy to ensure the safety and integrity of its data. Data in transit is encrypted to protect against tampering. The Aadhaar datastores are secured both physically and electronically and access is given only to a few select individuals. Data is secured with the best encryption, and in a highly secure data vault. All access details is logged. There are strict data retention guidelines that are adhered to including redundant copies and multi-data centre deployment. We use FOSS distributed file systems (with data replication) for realising high throughput data read-writes within the data centre."

The FOSS advantages
From his own experience with the open source software stack, Balasubramanian enumerates the key benefits that the use and adoption of OSS can render to developers, software product development teams and organisations, respectively.

"For developers: OSS encourages getting your hands dirty and promotes the geek in you. OSS users are therefore better off than their proprietary software counterparts in their computer science skills. Access to source of well-written frameworks enhances one's own design and programming skills. The can-do attitude of OSS users and contributors make them better suited to solve complex design and programming issues.

"For software product teams: Many OSS frameworks have liberal licensing and distribution terms. This allows product teams to embed them into their proprietary solutions. Product teams that open source their solutions can still benefit from business models of support, multiple distributions, training etc.

"Organisations can also save on license costs by using low-risk, well-written tools like source code editors and such. Organisations that have a strong engineering culture and development teams to support their IT solutions are better suited to adopt OSS. Such organisations typically have OSS solutions in their roadmap and influence business units to build solutions on it. In general, overall costs can be low when adopting OSS that is stable and has an active user community. Finding skilled resources for mainstream OSS solutions is not difficult or expensive.

A few 'open' practices
Balasubramanian has been with the Aadhaar project from the time the work to build the core application was awarded to MindTree. He has since then lead the architecture, design and development efforts on the project. His team has built the enrolment client, enrolment and authentication servers, portals and the business intelligence platform for the project. He shares a few best practices that have helped him and his team in his journey with the project so far: "Some of the best practices that we follow have been driven in part by the chief architect of the project Dr Pramod Varma. For example, we follow agile development so that feature releases can happen more often and feedback can be incorporated. There is a strong design focus where no part of the system is stereotyped to a certain product or approach. Fresh thinking is encouraged and we try to keep our design principles simple and follow patterns whenever prudent."

An 'open' platform for future eGovernance projects
The Aadhaar project is consuming a lot of FOSS. The general interest in the project and nation building has invited contributions and support from volunteers, user communities and technology teams in general. "This collective wisdom has helped in feature design, technology selection and making course correction, especially when solutions don't work as required," reveals Balasubramanian.

But is the project team also contributing back to the community as well? Balasubramanian affirms: "As far as giving back to the community goes, the UIDAI team interacts with user and developer communities of certain solutions in that are being used in the project, significant being RabbitMQ, Hadoop and MongoDB. "Developer community of RabbitMQ has been particularly helpful in our early days of adoption and more recently it has been the MongoDB community."

Apart from this, the eGovernance platform that the team has built is intended for use by other e-governance initiatives in India, informs Balasubramanian. "I would recommend use of FOSS and in fact reuse of the eGovernance platform that we have built for the UIDAI project. We have written a few technology components as part of the platform that can integrate with existing frameworks. These may be open sourced when the time is appropriate and sustenance is taken care of," he adds.

Looking at this success story and considering how FOSS has worked well so far for the Aadhaar project, we are sure many other projects and initiatives will be inspired to leverage FOSS and enjoy many of its benefits.

Aadhaar at Open Source India 2011
Regunath Balasubramanian, principal architect, UIDAI project, one of the eminent speakers and also the guest of honour at the recently concluded open source India 2011 convention, shared with the audiences the complexities of the project as huge as Aadhaar, and explained in details the advantages that OSS stack has rendered to the project. Glimpses from this session, and many others will shortly be uploaded on osidays.com and Linuxforu.com.

--Vandana Sharma

Daily

Weekly

Overall