The Internet of Things or IoT redefines many aspects of our day such as how we control and automate the environment we live in, how we shop, how we get healthcare and many more. Wearables bring about a new digitized and connected economy, where people and organizations collaborate in different and deeper manners, it brings life to devices and customers choose what they want and need without any sort of physical constraints.
Intelligent orchestration, configuration, analytics, and management tools along with continuous delivery and deployment keep organizations not only sane but more productive than ever. With the ubiquitousness of IoT devices , there’s almost nothing that we can’t access virtually over the internet..
The pervasiveness of these devices leads to a huge volume of data being generated, and gleaning insights from this raw data is akin to finding needles from a haystack. But once there’s a focussed and well engineered process for generating these insights, the value we can gain from these is priceless.
IoT and wearables have emerged as an element of intense business and social change, as well as a result of global forces that are driving this change. It has reshaped modern society in many good ways and brought technology advancement to every corner. Wearables are changing business and management processes, operations and models, while creating new value, improving efficiency and leading to new ways of making money and delivering greater value to customers.
The client is a wearable health devices service provider, which has a platform that handles data for multiple wearable devices from several end customers.
Due to the nature and volume of the data involved, they required a scalable solution that can keep up as the enrolled number of devices and consumers increases. They also had to set up a proper data pipeline, so that all the collected data can be used for research purposes with consent for providing real-time insights and reports.
The client required us to set up the entire infrastructure, keeping in mind that we shouldn’t be tied into one cloud vendor, so that it is easy to switch in the future. The data that needed to be ingested included all the data collected by the wearable devices and pushed to a highly available endpoint set up by us. We also had to ingest health status data that was uploaded by the users to the cloud platform.
Along with this, usage analytics data of the devices, which needed to be kept on a separated data-store needed to be tracked.
Once the data ingestion pipeline was fully operational, we needed to implement optimized analytics pipelines on top of this data, so that insights and reports can be provided to the user on the platform over a set of scalable APIs.
This analytics was to be performed on the user generated data through the smart wearables, as well as on the software generated analytics data. Both had to be set up as separate pipelines — one for user insights, the other for the product team to have insights on their product usage.
We had to work with the constraints that the primary data generation source is a low power smart wearable device that can generate a high volume of time bound data. So we set up a high performance low latency HTTP/2 endpoint to which these devices can constantly push the data. Availability was of great concern as all these should be persisted without having to drop any. Once we had a highly available distributed system for data ingestion set up, we set up the pipelines for routing the data to their respective handlers.
Since the volume of data were very high and could increase as we keep on adding more devices and users, we chose a very highly performant queue based system to handle all of this data. Based on a metadata layer that was being added upon receipt to the initial endpoint, we routed the data through to separate data handling pipelines through the queues, and individual micro-services could consume this data and persist this in their corresponding data store.
After the segregated and cleaned data was being persisted to appropriate data-stores, we had multiple analytics pipelines attached to these so that reports as well as real-time insights can be
generated and exposed over another set of highly performant APIs so that the dashboard or platform can consume this.
Along the whole data pipeline, we also made sure that there’s a security layer which enforced role based access control and extremely secure authentication as the data being handled was very sensitive in nature.
We also made sure that the whole system can be moved across to another cloud if required, by making sure that we chose cloud agnostic, yet cloud native technologies and toolsets were used.
Kubernetes, Ambassador, Traefik, RabbitMQ, NATS, Timescale, OpenTSDB, Casandra, PostgreSQL, Redis, Apache Storm, Apache Flink, Apache Spark, Go ,Python.
When it comes to the world of healthcare, wearable technologies open up a whole new portal of possibilities. The ordinary medical wearable devices when connected to the internet, can collect essential data which could be life-saving. They also serve in providing far more insight into symptoms and trends of any particular physiological or even psychological disorder.
IoT devices, along with these insights we can generate from the data collected from these devices, opens up a world of possibilities in healthcare and personal fitness verticals.